Video processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video processing method and device, electronic equipment and a storage medium, relates to the technical field of video processing, and aims to see a leading portrait more easily and clearly. The video processing method comprises the steps that in a first time period, a first video picture is displayed, in the first time period, the distance between a first user and a camera is smaller than the distance between a second user and the camera, the distance between the first user and the camera is smaller than the distance between a third user and the camera, and the first video picture comprises a portrait of the first user; after the first user moves away from the camera, a second video picture is displayed, and the second video picture comprises portraits of a second user and a third user; after the fourth user moves into the capturing range of the camera, a third video picture is displayed, and the third video picture comprises portraits of the first user, the second user, the third user and the fourth user.

Description

[0001] This application is a divisional application of the Chinese patent application No. 202310786977.X, filed on June 29, 2023, entitled "Video processing method, device, electronic device and storage medium", the whole content of which is incorporated into the parent application. TECHNICAL FIELD

[0002] The present application relates to the technical field of video processing, in particular to a video processing method, device, electronic device and storage medium. BACKGROUND

[0003] With the development of technology, video shooting applications are used in more and more scenarios. In some scenarios, a single video picture cannot meet the requirements. For example, when making a video call through a television, the wide angle of the camera makes the picture larger, and the person in the picture smaller, so that the other party cannot clearly see the person in the video call. SUMMARY

[0004] The technical solution of the present application provides a video processing method, device, electronic device and storage medium, which can make it easier to clearly see the main character.

[0005] In a first aspect, a video processing method is provided, including: displaying a first video picture in a first period, the distance between a first user and a camera being less than the distance between a second user and the camera in the first period, and the distance between the first user and the camera being less than the distance between a third user and the camera in the first period, the first video picture including a portrait of the first user; displaying a second video picture after the first user moves away from the camera, the second video picture including portraits of the second user and the third user; displaying a third video picture after a fourth user moves into a capture range of the camera, the distance between the fourth user and the first user being less than the distance between the second user and the first user, and the distance between the fourth user and the second user being greater than the distance between the second user and the third user, the third video picture including portraits of the first user, the second user, the third user and the fourth user.

[0006] In a possible implementation, in the first period, the first user is located in a third region, and the second user and the third user are located in a fourth region; displaying the first video picture in the first period includes: based on the distance between the first user and the camera being less than the distance between the second user and the camera, and the distance between the first user and the camera being less than the distance between the third user and the camera in the first period, displaying the first video picture.

[0007] In a possible implementation, after the first user moves away from the camera, displaying the second video screen comprises: after the first user moves away from the camera, displaying the second video screen in a second time period, based on the fact that, in the second time period, the distance between the first user and the camera is equal to the distance between the second user and the camera, the distance between the first user and the camera is equal to the distance between the third user and the camera, and the number of users in the fourth area is greater than the number of users in the third area.

[0008] In a possible implementation, during the process from the first time period to the second time period, the video display interface gradually changes from displaying the first video screen to displaying the second video screen.

[0009] In a possible implementation, during the first time period, displaying the first video screen comprises: obtaining a second target area including at least one person image in the original video screen captured by the camera in the first time period; obtaining a third target area of the first time period based on the second target area of the first time period and a third target area of a time period adjacent to the first time period, the coordinates of the third target area of the first time period being closer to the coordinates of the second target area of the first time period relative to the coordinates of the third target area of the time period adjacent to the first time period, or the coordinates of the third target area of the first time period being equal to the coordinates of the second target area of the first time period; and cropping the original video screen captured by the camera in the first time period based on the third target area of the first time period to obtain a cropped first video screen, and displaying the first video screen.

[0010] In a possible implementation, the video processing method further comprises: determining a target person image in the person image in the original video screen captured by the camera in the first time period, and obtaining a first target area of the first time period, the first target area of the first time period being a minimum rectangular frame including all target person images; and obtaining a second target area including at least one person image in the original video screen in the first time period comprises: if a non-calculation condition is met, taking the second target area of a time period adjacent to the first time period as the second target area of the first time period; or if the non-calculation condition is not met, obtaining the second target area including the first target area of the first time period in the original video screen in the first time period, the size of the second target area of the first time period being greater than the size of the first target area of the first time period; the non-calculation condition comprises at least one or any combination of the following, when the non-calculation condition comprises multiple items, any one of the multiple items not being met means that the non-calculation condition is not met, and all of the multiple items being met means that the non-calculation condition is met: the preset resolution level to which the first target area of the first time period belongs is the same as the preset resolution level to which the first target area of the time period adjacent to the first time period belongs; the first target area of the first time period is covered by the second target area of the time period adjacent to the first time period; and the number of target person images in the first time period is equal to the number of target person images in the time period adjacent to the first time period.

[0011] In one possible implementation, during the first time period, the first user is located in the third area, and the second and third users are located in the fourth area. The image of the user closest to the camera in the original video frame is designated as the first image, and the other images are designated as second images. The first image and all second images whose distance difference from the camera to the first image is ≤ a preset distance are designated as images to be determined. The original video frame includes the first area corresponding to the third area and the second area corresponding to the fourth area. Determining the target image from the images in the original video frame captured by the camera during the first time period includes: if the number of images to be determined in the first and second areas of the original video frame captured by the camera during the first time period is not equal, then all images to be determined in the area with the larger number of images to be determined in the first and second areas of the original video frame captured by the camera during the first time period are designated as target images; if the number of images to be determined in the first and second areas of the original video frame captured by the camera during the first time period is equal, then all images to be determined in the original video frame captured by the camera during the first time period are designated as target images.

[0012] In one possible implementation, the process of obtaining the third target region of the first time period based on the second target region of the first time period and the third target region of the previous time period adjacent to the first time period includes: if the second target region of the first time period is smaller than the third target region of the previous time period adjacent to the first time period, then the third target region of the previous time period adjacent to the first time period is smaller to obtain the third target region of the first time period; if the second target region of the first time period has the same size as the third target region of the previous time period adjacent to the first time period, and the center coordinate position of the second target region of the first time period is different from that of the third target region of the previous time period adjacent to the first time period, then the third target region of the previous time period adjacent to the first time period is moved towards the second target region of the first time period to obtain the third target region of the first time period; if the second target region of the first time period is larger than the third target region of the previous time period adjacent to the first time period, and the center coordinate position of the second target region of the first time period is the same as that of the third target region of the previous time period adjacent to the first time period, then the third target region of the previous time period adjacent to the first time period is enlarged to obtain the third target region of the first time period.

[0013] In one possible implementation, the process of shrinking the third target region of the previous time period adjacent to the first time period to obtain the third target region of the first time period includes: shrinking the coordinate values ​​of the third target region of the previous time period adjacent to the first time period by 10 times the step size using an integer (int) to obtain 10 times the coordinate values ​​of the third target region of the first time period; the process of moving the third target region of the previous time period adjacent to the first time period towards the second target region of the first time period to obtain the third target region of the first time period includes: moving the coordinate values ​​of the third target region of the previous time period adjacent to the first time period by 10 times the step size using an integer (int) in a direction towards the second target region of the first time period to obtain the first time period. The process of magnifying the third target region of the previous time period adjacent to the first time period to obtain the third target region of the first time period includes: magnifying the coordinates of the third target region of the previous time period adjacent to the first time period by 10 times the step size using an integer int method to obtain the coordinates of the third target region of the first time period; the process of cropping the original video frame of the first time period based on the third target region of the first time period to obtain the cropped frame includes: dividing 10 times the coordinates of the third target region of the first time period by 10 to obtain the coordinates of the third target region of the first time period, and cropping the original video frame of the first time period based on the coordinates of the third target region of the first time period using a floating-point method to obtain the cropped frame.

[0014] In one possible implementation, the video processing method further includes: obtaining the distance between the user and the camera corresponding to each portrait in the original video frame captured by the camera in the current time period; obtaining the number of portraits in the first area and the number of portraits in the second area in the original video frame in the current time period; displaying any one of the first video frame, displaying the second video frame and displaying the third video frame includes: displaying the video frame based on the distance between each user and the camera, the number of portraits in the first area and the number of portraits in the second area.

[0015] In one possible implementation, the video processing method further includes: acquiring a second target region including at least one human image in the original video frame of the current time period; acquiring a third target region of the current time period based on the second target region of the current time period and a third target region of the previous time period, wherein the coordinates of the third target region of the current time period are closer to the coordinates of the second target region of the current time period than the coordinates of the third target region of the previous time period, or the coordinates of the third target region of the current time period are equal to the coordinates of the second target region of the current time period; cropping the original video frame of the current time period based on the third target region of the current time period to obtain a cropped frame, wherein any one of the first video frame, the second video frame, and the third video frame is the cropped frame; and in multiple consecutive time periods, if the coordinates of the second target region remain unchanged, the coordinates of the third target region gradually approach the second target region.

[0016] In one possible implementation, the video processing method further includes: determining target portraits among portraits in the original video frame of the current time period, and obtaining a first target region for the current time period, wherein the first target region for the current time period is the smallest rectangle including all target portraits; obtaining a second target region including at least one portrait in the original video frame of the current time period includes: if a non-calculation condition is met, then the second target region of the previous time period is used as the second target region for the current time period; if the non-calculation condition is not met, then a second target region including the first target region for the current time period is obtained in the original video frame of the current time period, wherein the size of the second target region for the current time period is larger than the size of the first target region for the current time period; the non-calculation condition includes at least one or any combination of the following, wherein if any one of the multiple non-calculation conditions is not met, the non-calculation condition is not met, and if all of the multiple conditions are met, the non-calculation condition is met: the preset resolution level to which the first target region of the previous time period belongs is the same as the preset resolution level to which the first target region of the current time period belongs; the first target region of the current time period is covered by the second target region of the previous time period; the number of target portraits in the current time period is equal to the number of target portraits in the previous time period.

[0017] In one possible implementation, the image of the user closest to the camera in the original video frame is designated as the first image, and the other images are designated as second images. The first image and all second images whose distance difference from the camera to the first image is ≤ a preset distance are designated as images to be determined. Determining the target image from the images in the original video frame of the current time period includes: if the number of images to be determined in the first region and the second region of the original video frame of the current time period is not equal, then all images to be determined in the region with the larger number of images to be determined in the first region and the second region of the original video frame of the current time period are designated as target images; if the number of images to be determined in the first region and the second region of the original video frame of the current time period is equal, then all images to be determined in the original video frame of the current time period are designated as target images.

[0018] In one possible implementation, the process of obtaining the third target region for the current time period based on the second target region for the current time period and the third target region for the previous time period includes: if the second target region for the current time period is smaller than the third target region for the previous time period, then the third target region for the previous time period is shrunk to obtain the third target region for the current time period; if the second target region for the current time period has the same size as the third target region for the previous time period, and the center coordinates of the second target region for the current time period are different from those of the third target region for the previous time period, then the third target region for the previous time period is moved towards the second target region for the current time period to obtain the third target region for the current time period; if the second target region for the current time period is larger than the third target region for the previous time period, and the center coordinates of the second target region for the current time period are the same as those of the third target region for the previous time period, then the third target region for the previous time period is enlarged to obtain the third target region for the current time period.

[0019] In one possible implementation, the video processing method further includes: the process of shrinking the third target region of the previous time period to obtain the third target region of the current time period includes: shrinking the coordinate values ​​of the third target region of the previous time period by 10 times based on a step size value using an integer int, to obtain 10 times the coordinate values ​​of the third target region of the current time period; the process of moving the third target region of the previous time period towards the second target region of the current time period to obtain the third target region of the current time period includes: moving the coordinate values ​​of the third target region of the previous time period by 10 times based on a step size value towards the second target region of the current time period using an integer int, to obtain the third target region of the current time period. The process of magnifying the third target area from the previous time period to obtain the third target area of ​​the current time period includes: magnifying the coordinates of the third target area from the previous time period by 10 times the step size using an integer (int) to obtain the coordinates of the third target area of ​​the current time period; the process of cropping the original video frame of the current time period based on the third target area of ​​the current time period to obtain the cropped frame includes: dividing the coordinates of the third target area of ​​the current time period by 10 to obtain the coordinates of the third target area of ​​the current time period, and cropping the original video frame of the current time period based on the coordinates of the third target area of ​​the current time period using a floating-point method to obtain the cropped frame.

[0020] In one possible implementation, the aspect ratio of the third target region is the same as that of the original video frame.

[0021] In one possible implementation, displaying the first video frame includes displaying the first video frame in at least one or any combination of a video call notification interface, a video call dialing interface, and a video call interface.

[0022] In a second aspect, a video processing apparatus is provided, comprising: a first display module for displaying a first video frame during a first time period, wherein the distance between a first user and a camera is less than the distance between a second user and a camera, and the distance between the first user and the camera is less than the distance between a third user and a camera, and the first video frame includes an image of the first user; a second display module for displaying a second video frame after the first user moves away from the camera, the second video frame including images of the second user and the third user; and a third display module for displaying a third video frame after a fourth user moves into the camera's capture range, wherein the distance between the fourth user and the first user is less than the distance between the second user and the first user, and the distance between the fourth user and the second user is greater than the distance between the second user and the third user, and the third video frame including images of the first user, the second user, the third user, and the fourth user.

[0023] Thirdly, an electronic device is provided, comprising: a processor and a memory, wherein the memory is used to store at least one instruction, which, when loaded and executed by the processor, causes the electronic device to perform the video processing method described above.

[0024] Fourthly, a computer-readable storage medium is provided, including a program or instructions, wherein the above-described method is executed when the program or instructions are run on a computer.

[0025] The video processing method, apparatus, electronic device, and storage medium in this application embodiment redetermine the main character based on the number of people corresponding to the portrait in the video frame and the change in camera distance. The newly determined main character is tracked and magnified through the cropped image, thereby highlighting the portrait of the main character in the video frame, achieving focus on the main character, and making it easier to see the portrait of the main character. Attached Figure Description

[0026] Figure 1 This is a structural block diagram of an electronic device according to an embodiment of this application; Figure 2 This is a software structure block diagram in an embodiment of this application; Figure 3 This is a schematic diagram of an interaction process between different layers in an embodiment of this application; Figure 4 This is a schematic diagram of an original video frame in an embodiment of this application; Figure 5 for Figure 3 A detailed flowchart of some of the steps in the middle section; Figure 6 for Figure 3 A flowchart illustrating the specific steps of another part of the process; Figure 7 for Figure 6 A detailed flowchart of some of the steps in the middle section; Figure 8 for Figure 6 A flowchart illustrating the specific steps of another part of the process; Figure 9 for Figure 8 A detailed flowchart of some of the steps in the middle section; Figure 10 This is a schematic diagram illustrating the relationship between a first target area, a second target area, and an original video frame in an embodiment of this application. Figure 11 for Figure 6 A flowchart illustrating the specific steps of another part of the process; Figure 12 This is a schematic diagram illustrating the gradual change of a third target region across multiple consecutive time periods in an embodiment of this application. Figure 13This is a schematic diagram illustrating a change in the user interface in an embodiment of this application; Figure 14 This is a schematic diagram of an original video frame in an embodiment of this application; Figure 15 This is a schematic diagram of another original video frame in an embodiment of this application; Figure 16 This is a schematic diagram of yet another original video frame in an embodiment of this application; Figure 17 This is a schematic diagram of another original video frame in an embodiment of this application. Detailed Implementation

[0027] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0028] Figure 1 A schematic diagram of the structure of the electronic device 100 is shown.

[0029] Electronic device 100 may include processor 110, internal memory 121, audio module 170, speaker 170A, microphone 170C, camera 193, display screen 194, etc.

[0030] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0031] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0032] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0033] The communication function of electronic device 100 can be achieved through wired or wireless means.

[0034] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0035] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0036] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0037] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0038] Internal memory 121 can be used to store computer executable program code, which includes instructions.

[0039] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0040] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Electronic device 100 can listen to music or make calls through the speaker 170A.

[0041] The microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call, the user can input their voice signal into the microphone 170C.

[0042] The following description uses a smart TV as an example to illustrate the process of applying the embodiments of this application in a video call scenario on a smart TV.

[0043] First, the embodiments of this application will be described in conjunction with the software architecture, and the software structure of the electronic device 100 will be illustrated by way of example. Figure 2 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0044] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the software system, from top to bottom, includes an Application layer, an Application Framework layer, and a Hardware Abstraction Layer (HAL). The Application layer may include applications such as camera and video calling applications. The Application Framework layer may include a camera service module such as CameraService. The Hardware Abstraction Layer processes the video stream captured by the camera and may include an image acquisition module, an algorithm detection module, and an image cropping module; these modules can be processes or threads.

[0045] like Figure 2 and Figure 3As shown, this application embodiment provides a video processing method that can be applied to the aforementioned electronic device. The method may include the following steps: Step 101: In response to a video call request or a video call notification request, the video call application sends a stream allocation request to the camera service module. The video call request is generated by an event where the user initiates a call, and the video call notification request is generated by an event where a call is received from the other party. The stream allocation request is used to request the camera to be turned on and to configure video stream parameters such as the camera's frame rate and resolution. In some embodiments, in response to a video call notification request, a stream allocation request is sent to the camera service module to trigger subsequent acquisition of the video stream captured by the camera to achieve a video preview function. In other embodiments, the video call notification request in Step 101 can be replaced by a video call connection request. That is, before the user connects the video call, it is not necessary to acquire the video stream captured by the camera; instead, it is acquired only when the call is connected. In Step 101, the video call application may specifically call the `configstream` function to send a stream allocation request to the camera service module. Step 102: In response to the configuration request, the camera service module sends stream allocation information to the image acquisition module in the hardware abstraction layer. Step 103: The image acquisition module performs stream allocation in response to the stream allocation information. The stream allocation process includes configuring the camera's frame rate and resolution. After step 103, step 104 is executed: The video call application sends a video stream acquisition request to the camera service module. Specifically, the video call application can call the RepeatRequest function to send the video stream acquisition request to the camera service module. Step 105: The camera service module sends a video stream acquisition request to the image acquisition module in response to the video stream acquisition request. Step 106: The image acquisition module sends a video stream acquisition request to the camera in response to the video stream acquisition request. Step 107: The image acquisition module acquires the original video frame, specifically by periodically acquiring the original video frame captured by the camera in time intervals. For example, every 30ms is a time interval, and the image acquisition module sends a video stream acquisition request to the camera every 30ms and acquires the original video frame of the current time interval from the camera, thus performing one image acquisition. The acquired image will be decoded, and the decoded original video frame will be provided to the algorithm detection module, i.e., step 108 is executed: The algorithm detection module acquires the original video frame, for example... Figure 4 As shown, the original video footage of the current time period obtained from the camera is 1920. A 1024-bit H.264 encoded image is decoded to 1920 bits. A 1024 pixel YUYV format image. Step 109: The algorithm detection module performs face detection based on the original video frame of the current time period, obtains face detection information, and sends the face detection information to the image cropping module. Step 110: The image cropping module crops the original video frame based on the face detection information. In each time period, the image cropping module obtains the third target region C of the current time period based on the face detection information of the current time period. The third target region C refers to the actual cropping frame of the current frame. Obtaining the third target region C means obtaining the coordinates of the left border, right border, top border, and bottom border corresponding to the third target region C. Then, the original video frame of the current time period is cropped based on the third target region C. Step 110: The camera service module obtains the cropped image. Step 111: The video call application obtains the cropped image. The final image displayed by the video call application is the cropped image of the original video frame based on the third target region of the current time period. The size of the cropped image is usually smaller than the size of the original video frame, so its size is adjusted to 1080P, and the cropped image is transmitted to the video call application for display. In each time period, the image cropping module crops the currently acquired original video frame based on the third target region of the current time period and transmits the cropped image to the video call application for display. The above steps 107-111 can be executed periodically, that is, each time period will perform human face detection based on the original video frame of the current time period and perform cropping based on the human face detection information of the current time period.

[0046] The following provides a detailed explanation of steps 101-106 above. For example... Figure 5 As shown, the electronic device may include the application package of the first application 401, the camera manager (e.g., denoted as CameraManager) 402, the camera device instance (e.g., denoted as CameraDeviceImpl) 403, the camera device client (e.g., denoted as CameraDeviceClient) 404, the camera device (e.g., denoted as Camera3Device) 405, and the camera device session (e.g., denoted as CameraDeviceSession) 406.

[0047] In one embodiment, the first application can be a video call application corresponding to a video call scenario. In another embodiment, the first application can be a camera application corresponding to a camera preview scenario or a camera recording scenario.

[0048] In one embodiment, the application package 401 may be the APK (Android application package) of the first application.

[0049] In one embodiment, the Camera 2 API (Application Programming Interface) may include a camera manager 402 and a camera device instance 403.

[0050] In one embodiment, the CameraService module in the application framework layer of the electronic device may include a camera device client 404 and a camera device 405.

[0051] In one embodiment, the CameraHal (camera hardware abstraction layer) of the electronic device may include a camera device session 406.

[0052] In one embodiment of this application, the camera device session 406 may include an image acquisition module, an algorithm detection module, and an image cropping module. For details on the implementation of these three modules, please refer to [reference needed]. Figure 3 The relevant technical descriptions in the illustrated embodiments are provided below.

[0053] like Figure 7 As shown, the process of steps 101 to 106 above may include the following: Step 4.1, in response to the first instruction, application package 401 sends a request to obtain the camera identifier (Identitydocument, id) to camera manager 402, so that camera manager 402 returns a list of camera identifiers.

[0054] The camera manager 402 can provide a list of camera identifiers to the application package 401 based on the received camera identifier acquisition request, so that the application package 401 can obtain the camera identifiers accordingly.

[0055] In one embodiment, the relevant implementation code for sending a request to obtain the camera identifier to the camera manager 402 may include: CameraManager.getCameraIdList().

[0056] In one embodiment of this application, when the first application is a video call application, the first instruction can be an instruction to start a video call. In one feasible implementation, the user can trigger a video initiation control within the video call application to issue an instruction to start a video call.

[0057] Step 4.2: Application package 401 obtains the camera identifier based on the list of camera identifiers sent by camera manager 402. In one embodiment, the relevant implementation code for obtaining the camera ID may include: CameraManager.getCameraCharacteristics(CameraId).

[0058] In one feasible implementation, the obtained camera identification number can be the identification number of the camera used for image acquisition, and the image acquisition module in the camera device session 406 can acquire the image captured by the camera.

[0059] Step 4.3: Application package 401 determines whether the key of the corresponding camera exists based on the obtained camera identifier. If it exists, it proceeds to step 4.4.

[0060] In one feasible implementation, if it is determined that the key for the corresponding camera does not exist, step 4.4 can be skipped.

[0061] In one embodiment, the relevant implementation code for determining whether a key exists may include: mCharacteristics.getKeys().contains(HIHONOR_FACEAUTOTRACK_SUPPORTED) KEY NAME: com.hihonor.device.capabilities.faceAutoTrackModeSupportedByte type.

[0062] In step 4.4, application package 401 determines whether the corresponding application supports human face tracking based on the obtained camera identifier. If it does, it triggers camera manager 402 to open the corresponding camera according to the camera's key. The camera in the open state can sequentially capture each frame of image within its image capture range.

[0063] In another embodiment of this application, step 4.4 can be executed first and then step 4.3, that is, step 4.3 is executed when step 4.4 passes the judgment, and step 4.5 is executed when step 4.3 passes the judgment.

[0064] In one embodiment, the relevant implementation code for determining whether the corresponding application supports facial tracking may include: mCharacteristics.get(HIHONOR_FACEAUTOTRACK_SUPPORTED) 0 does not support 1 supports byte type.

[0065] Since subsequent image capture by the camera can be used for human face detection to achieve user face tracking, and then image cropping can be performed based on the human face detection information to display the cropped image, if the judgment result is "not supported", the camera manager 402 will not be triggered to open the corresponding camera, and if the judgment result is "supported", the camera manager 402 will be triggered to open the corresponding camera.

[0066] In one embodiment, the relevant implementation code that triggers the camera manager 402 to open the corresponding camera may include: CameraManager.openCamera().

[0067] In step 4.5, application package 401 sends an instruction to create a capture request to camera device instance 403. In one embodiment, the relevant implementation code for step 4.5 may include: CameraDevice.createCaptureRequest().

[0068] Step 4.6: Camera device instance 403 sends a command to create a system configuration request to camera device client 404 according to the command to create a capture request. In one embodiment, the relevant implementation code for step 4.6 may include: createDefaultRequest(templateType).

[0069] Step 4.7, the camera device client 404 transmits the received instruction to create system settings request to the camera device 405.

[0070] Step 4.8: Camera device 405 sends a system settings request instruction to camera device session 406 according to the instruction to create a system settings request. In one embodiment, the relevant implementation code for step 4.8 may include: constructDefaultRequestSettings.

[0071] Step 4.9: Camera device session 406 obtains camera metadata and current application package name according to the instructions of the setting system setting request, and returns the obtained camera metadata and current application package name to camera device 405.

[0072] In one embodiment, the relevant implementation code for step 4.9 may include: com.hihonor.capture.metadata.packageName.

[0073] In one embodiment, the camera metadata obtained by the camera device session 406 may be the metadata of an already opened camera.

[0074] In one embodiment, the current application package name can be the package name of the application package 401 of the first application mentioned above.

[0075] Step 4.10: Camera device 405 adds the current application package name to the camera metadata and sends the camera metadata with the added application package name to camera device client 404. By adding the current application package name to the camera metadata, scene information for the current time period is incorporated.

[0076] Step 4.11, the camera device client 404 passes the camera metadata with the current application package name to the application package 401.

[0077] By passing camera metadata with the current application package name to application package 401, the CaptureRequest.Builder of application package 401 can include the package name and the camera's device capability.

[0078] After the application package 401 obtains the camera metadata with scene information, it can start the distribution process (see steps 4.12 to 4.16).

[0079] Step 4.12: Application package 401 generates a command to create a capture session based on camera metadata with the current application package name added, and sends the command to create a capture session to camera device instance 403.

[0080] In one embodiment, the relevant implementation code for step 4.12 may include: CameraDevice.CreateCaptureSession(SessionConfiguration config).

[0081] Step 4.13: According to the instruction to create a capture session, camera device instance 403 sends interface flow information and session key configuration information to camera device client 404.

[0082] In one embodiment, the streaming information may include frame rate and resolution. The streaming module of the HAL layer can configure the stream according to the streaming information, configuring the frame rate and resolution for the camera.

[0083] Step 4.14: The camera device client 404 processes the received interface flow information and configuration session key information, and sends the processed session key and the corresponding camera hardware abstraction layer (camera HAL layer) flow information to the camera device 405.

[0084] Step 4.15, the camera device 405 sends the session key and the corresponding camera hardware abstraction layer distribution information to the camera device session 406.

[0085] Step 4.16: The camera device session 406 performs the streaming process based on the session key and the streaming information of the corresponding camera hardware abstraction layer, and sends a streaming completion notification to the application package 401 after the streaming process is completed.

[0086] In one embodiment, the distribution process may include mode selection and pipeline configuration.

[0087] In one embodiment, a message "I implementation" can be sent to application package 401 after the distribution process is completed, so that application package 401 is aware that the distribution process has been completed.

[0088] In one embodiment, the technical implementation of the flow distribution process can be referred to Figure 3 The relevant technical content in the illustrated embodiments will not be repeated here.

[0089] After the camera device session 406 completes the streaming process, the application package 401 can send a repeat request to trigger the camera device session 406 to perform image processing (such as algorithm detection and cropping) on ​​the image captured by the camera through the camera service module.

[0090] In step 4.17, application package 401 sends a command to set a repeating request to camera device instance 403. In one embodiment, the relevant implementation code for step 4.17 may include: captureSession.setRepeatingRequest.

[0091] Step 4.18, the camera device instance 403 sends the instruction to submit request list (e.g., denoted as submitRequestsList) to the camera device client 404 according to the instruction to set repeated requests.

[0092] Step 4.19: The camera device client 404 sends the instruction to set the streaming request list (e.g., setStreamingRequestList) to the camera device 405 according to the instruction to submit the request list.

[0093] Step 4.20: Camera device 405 sends an image acquisition request to camera device session 406 according to the instruction to set the streaming request list.

[0094] Step 4.21: The camera device session 406 acquires a frame of image captured by the camera according to the image acquisition request, so as to process the acquired image to obtain a processed image, and sends the processed image to the application package 401 for display.

[0095] In one embodiment of this application, the camera device session 406 may include an image acquisition module, an algorithm detection module, and an image cropping module. The image acquisition module acquires images from the activated camera and places the acquired images into an image buffer queue. The algorithm detection module and the image cropping module asynchronously retrieve images from the image buffer queue.

[0096] The algorithm detection module performs human face detection on the acquired image and stores the generated human face detection information in a human face detection information buffer queue. The image cropping module, after acquiring the image, attempts to retrieve human face detection information from the buffer queue. If human face detection information is obtained, the image is cropped accordingly, and the cropped image is used as the capture result (e.g., denoted as CaptureResult). If no human face detection information is obtained, no cropping is performed, and the acquired image is directly used as the capture result; alternatively, the image is cropped based on the human face detection information used in the previous cropping, and the cropped image is used as the capture result.

[0097] The image cropping module then sends the captured result back to application package 401 for display. Thus, camera device session 406 completes one processing of an image acquisition request (e.g., denoted as processcapturerequest). The specific implementation of image processing will be described in detail later.

[0098] The specific processes of steps 109 and 110 are explained below. Before executing step 109, the original video footage for the current time period has already been acquired, such as... Figure 4 As shown, a coordinate system is defined based on the boundaries of the original video frame. The coordinates of any position in this system are (x, y), where x represents the horizontal coordinate and y represents the vertical coordinate. The origin (0, 0) is located at the top left corner of the frame, the top right corner is (1920, 0), the bottom left corner is (0, 1080), and the bottom right corner is (1920, 1080). The coordinate unit can be pixels, meaning the original video frame size is 1920 pixels. 1080 pixels, or the original video resolution is 1920. 1080.

[0099] Step 109 specifically involves performing face detection based on the original video footage of the current time period, obtaining at least one face detection piece of information. Each face detection piece of information includes a face ID, face frame coordinates, and distance. Within each time period, face detection, or facial detection, is first performed based on the currently acquired original video footage to identify faces in the original video footage of the current time period. A face facing the camera or a certain range towards the display screen, whether frontal or sideways, can be identified as a face. For example... Figure 5The original video footage shown can be processed using face detection to obtain face detection information for each individual. Each person corresponds to one set of face detection information, which can be used as an array. For example, the person in the middle is person A, and their corresponding face detection information includes A's face ID, the coordinates of the left, right, top, and bottom borders of A's face frame, and A's distance. Similarly, the person on the left is person B, and the person on the right is person C. The ID is used as an identifier for the face. The face frame refers to a rectangular area reflecting the position and size of the face, and the face frame coordinates can include the four coordinates corresponding to the four sides (top, bottom, left, and right). The distance, or camera distance, refers to the distance between the person and the camera in the actual physical space corresponding to the face. In other words, in step 109, the distance between the user and the camera corresponding to each face in the original video footage captured by the camera in the current time period is obtained. Figure 5 The person in the middle, A, is the closest, followed by B on the right, and C on the left is the farthest. This shows that the farther away the person is, the smaller the size of the portrait frame becomes.

[0100] like Figure 6 As shown, step 110 above includes steps 200 to 304.

[0101] Step 200: Remove the image detection information in at least one image detection information whose image frame coordinates exceed the boundary of the original video frame. For example, if the left border coordinate of the image frame of a certain image detection information obtained in step 109 is <0, it means that the image frame exceeds the edge of the original video frame. Therefore, remove the image detection information to avoid the problem of the subsequent cropped image exceeding the edge of the original frame.

[0102] Step 201: Sort at least one human detection information from the original video footage of the current time period in ascending order of distance to obtain sorted human detection information. Sort the human detection information in the array using sorting algorithms such as bubble sort; this embodiment will not elaborate on the specific sorting algorithm. Based on Figure 4 The sorted human image detection information is arranged in the order of A, B, and C.

[0103] Step 202: In the sorted human detection information, remove human detection information whose distance is 0.5m greater than the distance of the first human detection information, to obtain the removed human detection information. The human image corresponding to the user closest to the camera in the original video frame is the first human image, and the other human images are the second human images. The first human image and all second human images whose distance difference from the first human image is ≤0.5m within a preset distance are the human images to be determined. The human images corresponding to the removed human detection information in Step 202 are the human images to be confirmed. For example, if A and B are close, while C is 0.7m greater than A, then the human detection information corresponding to C is removed from the array, and the resulting removed human detection information only includes the human detection information of A and B.

[0104] Step 203: Determine whether the number of people in the removed human image detection information is ≤2. If yes, proceed to step 204; otherwise, proceed to step 205.

[0105] Step 204: Use the removed human detection information as valid human detection information. For example, in the previous example, if the removed human detection information only contains human detection information corresponding to individuals A and B, then the human detection information corresponding to individuals A and B will be directly used as valid human detection information for subsequent use. That is, if the number of undetermined human images in the first and second regions of the original video frame in the current time period is equal, then all undetermined human images in the original video frame in the current time period will be determined as target human images.

[0106] Step 205: Obtain the number of people whose center point of the portrait frame is located on the left side of the original video frame and the number of people whose center point of the portrait frame is located on the right side of the original video frame. The original video frame has 1920 pixels horizontally. If the center point of the portrait frame is ≤960, it is located on the left side of the frame, and the number of people on the left side is increased by 1. If the center point of the portrait frame is >960, it is located on the right side of the frame, and the number of people on the right side is increased by 1. The area on the left side of the original video frame is the first area, and the area on the right side is the second area. Step 205 obtains the number of unconfirmed portraits in the first area and the number of unconfirmed portraits in the second area of ​​the original video frame for the current time period. In subsequent processes, the video frame will be displayed based on the distance between each user and the camera, the number of unconfirmed portraits in the first area, and the number of unconfirmed portraits in the second area.

[0107] Step 206: Determine if the number of people on the left is equal to the number of people on the right. If yes, proceed to step 204; otherwise, proceed to step 207.

[0108] Step 207: The person detection information corresponding to the larger number of people on the left and right sides is taken as the valid person detection information. For example, if there are 3 people on the left side of the screen and 2 people on the right side, the person detection information corresponding to the 3 people on the left side is taken as the valid person detection information for subsequent use. In other words, if the number of undetermined persons in the first and second regions of the original video frame for the current time period is not equal, all undetermined persons in the region with the larger number of undetermined persons in the first and second regions of the original video frame for the current time period are identified as the target persons.

[0109] The faces in the valid face detection information determined in step 204 or step 207 above are the target faces. Steps 200-204 or 200-407 determine the target faces from the faces in the original video frame of the current time period. One valid face detection information corresponds to one face; therefore, the number of valid face detection information is the number of target faces. After determining the valid face detection information in step 204 or step 207 above, step 301 is executed to obtain the first target region A of the current time period, step 302 to obtain the second target region B of the current time period, and step 303 to obtain the third target region C of the current time period. For example... Figure 4 As shown, the first target area A refers to the smallest rectangular frame encompassing all valid human detection information corresponding to the human frames in the current time period. The second target area B in the current time period is the frame after expanding the first target area A by a certain range; that is, the second target area B in the current time period is the enlarged and centered frame of the human frames corresponding to the valid human detection information in the current time period, and the second target area B refers to the final target area of ​​the current image cropping. The third target area C is the actual cropping frame that gradually approaches the second target area B. The above process of obtaining target areas is actually the process of obtaining the coordinates of each target area, including the coordinates of the four borders (left, right, top, and bottom). In some cases, if the frame after expanding the first target area A by a certain range in the current time period exceeds the edge of the original video frame, it is necessary to ensure that the second target area does not exceed the edge of the original video frame. In this case, the second target area B will cover the first target area A, but it may not necessarily have a significant centering effect.

[0110] like Figure 7As shown, step 301 specifically includes steps 3011 to 3014. Step 3011: Determine if the number of valid human image detection information is 0. If yes, proceed to step 3012; otherwise, proceed to step 3013. Step 3012: Use the boundary coordinates of the original video frame in the current time period as the coordinates of the first target area in the current time period. When the number of valid human image detection information is determined to be 0, it means that no human image was detected in the frame in step 109, or that after removing the human image detection information exceeding the boundary in step 200, there is no valid human image detection information in the array. In this case, the boundary coordinates of the original video frame are directly used as the coordinates of the first target area. Step 3013: Use the coordinates of the human image frame of one of the valid human image detection information as the coordinates of the first target area in the current time period. Step 3014: Traverse other valid human image detection information. If the coordinates of any side of the human image frame of other valid human image detection information are outside the first target area in the current time period, then assign the coordinates of that side to the first target area in the current time period. For example, if there are three portraits in the original video frame, and these three portraits are all corresponding to the portrait frames in the valid portrait detection information (circular or elliptical), in step 3013, the coordinates of the portrait frame corresponding to any one portrait are first used as the coordinates of the first target area A. Then, in step 3014, the valid portrait detection information corresponding to the second portrait is traversed, and it is determined that the left and top border coordinates of the portrait frame corresponding to this valid portrait detection information are outside the first target area of ​​the current time period. Therefore, the coordinates of the upper portrait frame and the top border coordinates are assigned to the first target area of ​​the current time period. Then, the valid portrait detection information corresponding to the third portrait is traversed, and it is determined that the left border coordinate of the portrait frame corresponding to this valid portrait detection information is outside the first target area of ​​the current time period. Therefore, the left border coordinate of the left portrait frame is assigned to the first target area of ​​the current time period. In this way, by traversing, the first target area covers the portrait frames corresponding to each valid portrait detection information, and finally makes the first target area of ​​the current time period the smallest rectangle covering the portrait frames of all valid portrait detection information, thus realizing the acquisition of the first target area.

[0111] like Figure 8 As shown, step 302 above includes steps 3021 to 3029.

[0112] Step 3021: Among multiple preset resolution levels, determine the preset resolution level to which the first target area in the current time period belongs. For example, multiple resolution levels are preset based on the resolution of the original video frame, where the original frame resolution is 1920. 1080, the first gear is 1920. 1080, the second gear is 1760. 990, the third gear is 1600. 900, the fourth gear is 1440. 810, the fifth gear is 1280. 720. To ensure the final cropped image matches the original video aspect ratio and avoid image stretching during cropping, the aspect ratio of each preset resolution level can be set to equal the original video aspect ratio, which is 16:9. If the resolution of the first target area in the current time period is 1880... 1000, located in 1920 1080 and 1760 Between 990 and 1920, it belongs to the first gear. 1080; if the resolution of the first target region in the current time period is 1880. 880, located in 1920 1080 and 1440 Between 810 and 1920, it also belongs to the first gear. 1080; if the resolution of the first target region in the current time period is 640. 480, which is located at 1280 Below 720, it belongs to the fifth level, 1280. 720.

[0113] Step 3022: Determine whether the preset resolution level of the first target area in the previous time period is the same as the preset resolution level of the first target area in the current time period. If yes, proceed to step 3023; otherwise, proceed to step 3024. If the current time period is the first cycle, the boundary coordinates of the original video frame can be used as the coordinates of the first target area in the previous time period.

[0114] Step 3023: Determine whether the first target area in the current time period is covered by the second target area in the previous time period. If yes, proceed to step 3025; otherwise, proceed to step 3024. If the current time period is the first cycle, the border of the original video frame can be used as the second target area in the previous time period.

[0115] Step 3025: Determine whether the number of valid human face detection information in the current time period is the same as the number of valid human face detection information in the previous time period. If yes, proceed to step 3026; otherwise, proceed to step 3024.

[0116] Step 3026: Use the second target area from the previous time period as the second target area for the current time period. In other words, if the number of people with valid human detection information in the image has not changed compared to the previous time, and the movement of the people has not exceeded the range of the previous level and the second target area, then there is no need to calculate a new second target area, but to continue using the second target area from the previous time period. This is to avoid frequent changes in the second target area due to small changes in the human frame, thereby improving the image jitter problem.

[0117] Step 3024: Determine whether the coordinates of the first target area in the current time period are the same as the boundary coordinates of the original video frame in the current time period. If not, proceed to step 3027; if yes, proceed to step 3029. Here, whether the coordinates are the same means whether the coordinates of all four sides are the same. If they are all the same, proceed to step 3028; if the coordinates of any one of the four sides are different, proceed to step 3027.

[0118] Step 3027: Calculate the coordinates of the second target region in the current time period, that is, calculate the coordinates of the four sides of the second target region in the current time period. The specific calculation process will be described in detail later. In other words, if, compared with the previous time, there is a change in the number of effective human image detection information, a change in the preset resolution level, or the first target region exceeds the range of the second target region in the previous time period, and the first target region in the current time period is different from the boundary of the original image, then it is necessary to calculate the coordinates of the second target region in the current time period.

[0119] Step 3028: Determine whether the first target area in the current time period is the same as the second target area in the previous time period. If not, proceed to step 3027; if yes, proceed to step 3029.

[0120] Step 3029: Use the original video frame of the current time period as the second target area of ​​the current time period. That is, if the boundary of the first target area is the boundary of the original frame and is the same as the second target area of ​​the previous time period, then it is not necessary to calculate the coordinates of the second target area of ​​the current time period. It is only necessary to assign the boundary coordinates of the original frame to the second target area of ​​the current time period.

[0121] like Figure 9 As shown, step 3027 above includes steps 401 to 414.

[0122] Step 401: Determine whether the left border coordinates of the first target area A in the current time period are ≤16. If yes, proceed to step 402; otherwise, proceed to step 403.

[0123] Step 402: Set the left border coordinate of the second target region B in the current time period to 0, and the right border coordinate to W, where W is the width of the preset resolution level to which the first target region A in the current time period belongs. Assume the resolution of the first target region A in the current time period is 640. 480, the default resolution setting is 1280. 720, i.e. W=1280. If the left border coordinate of the first target area A is ≤16, it is considered that the first target area A is close to or has exceeded the left edge of the original image. Therefore, the width of the second target area B in the current time period is directly set to the left side of the image based on the preset resolution level.

[0124] Step 403: Determine whether the right frame coordinates of the first target area A in the current time period are ≥MW-16. If yes, proceed to step 404; otherwise, proceed to step 405. MW is the width of the original video frame, i.e., MW=1920.

[0125] Step 404: Set the left border coordinates of the second target region B in the current time period to MW-W=1920-1280=840, and the right border coordinates to MW, i.e., 1920. If the resolution is 640... If the right frame coordinate of the first target area A in the current time period is ≥1920-16, then the first target area A is considered to be close to or has exceeded the right edge of the original video frame. Therefore, the width of the second target area B in the current time period is directly set to the right side of the frame based on the preset resolution level.

[0126] Step 405: Determine if mleft - (W - mW) / 2 ≤ 0. If yes, proceed to step 402; otherwise, proceed to step 406. Figure 10 As shown, `mletf` represents the coordinates of the left border of the first target region A in the current time period, `mright` represents the coordinates of the right border of the first target region A in the current time period, `W` = 1280, and `mW` represents the width of the first target region A in the current time period, i.e., `mW` = `mright` - `mleft` = 640. `(W - mW) / 2` = `(1280 - 640) / 2` = 320. If `mleft - 320` ≤ 0, it means that if the first target region A is based on 1280... Using the left border coordinates of the 720 resolution level as the left border coordinates of the second target area would exceed the left edge of the original image. Therefore, step 402 is executed to directly set the width of the second target area B in the current time period to the left side of the image based on its preset resolution level.

[0127] Step 406: Set the left border coordinates of the second target region B in the current time period to mleft - (W - mW) / 2 = mleft - 320, and the right border coordinates to mleft - (W - mW) / 2 + W = mleft - 320 + 1280 = mleft + 960. In other words, if the first target region A in the current time period is far from the left and right edges of the original image, set the left and right border coordinates of the second target region B in the current time period based on the position of the first target region A in the current time period and the width of its corresponding preset resolution level.

[0128] After step 406, step 407 is executed to determine whether the right border coordinates of the second target area B in the current time period are ≥MW. If yes, step 404 is executed; otherwise, step 408 is executed. Since step 406 only ensures that the left border coordinates of the second target area B do not exceed the original image boundary, step 407 judges the right border coordinates of the second target area B. If it is ≥1920, it may exceed the right edge of the original image. Therefore, the process returns to step 404, and the width of the second target area B in the current time period is directly set to the right side of the image based on the preset resolution level. If the right border coordinates of the second target area B are less than 1920, it means that it will not exceed the right edge of the original image. Therefore, the subsequent process of determining the coordinates of the top and bottom borders can continue.

[0129] After confirming no in steps 402, 404, and 407, the coordinates of the left and right borders of the second target area B for the current time period are set. Next, step 408 is executed to continue setting the coordinates of the top and bottom borders.

[0130] Step 408: Determine whether the upper border coordinates of the first target area A in the current time period are ≤9. If yes, proceed to step 409; otherwise, proceed to step 410.

[0131] Step 409: Set the top border coordinates of the second target area B in the current time period to 0 and the bottom border coordinates to H, where H is the height of the preset resolution level to which the first target area A belongs in the current time period, i.e., H=720. If the top border coordinates of the first target area A in the current time period are ≤9, it is considered that the first target area A is close to or has exceeded the top edge of the original image. Therefore, the second target area B in the current time period is directly set to the top of the image based on the height of its preset resolution level.

[0132] Step 410: Determine whether the lower border coordinates of the first target area A in the current time period are ≥ MH-9. If yes, proceed to step 411; otherwise, proceed to step 412. MH is the height of the original video frame, i.e., MH=1080.

[0133] Step 411: Set the upper border coordinates of the second target area B in the current time period to MH-H=1080-720, and the lower border coordinates to MH=1080. If the lower edge coordinates of the first target area A in the current time period are greater than or equal to 1080-9, it is considered that the first target area A is close to or has exceeded the lower edge of the original image. Therefore, the height of the second target area B in the current time period is directly set at the bottom of the image based on the preset resolution level.

[0134] Step 412: Determine if mtop - (H - mH) / 2 ≤ 0. If yes, proceed to step 409; otherwise, proceed to step 413. mtop is the coordinate of the top border of the first target region A in the current time period, mbottom is the coordinate of the bottom border of the first target region A in the current time period, H = 720, and mH is the height of the first target region A in the current time period, i.e., mH = mbottom - mtop = 480. (H - mH) / 2 = (720 - 480) / 2 = 120. If mtop - 120 ≤ 0, it means that if the first target region A is based on 1280... The coordinates of the top border after the 720 resolution level is directly moved up will be used as the coordinates of the top border of the second target area, which will exceed the top edge of the original screen. Therefore, step 409 is executed to directly set the height of the second target area B in the current time period to the top of the screen based on its preset resolution level.

[0135] Step 413: Set the upper border coordinates of the second target area B in the current time period to mtop - (H - mH) / 2 = mtop - 120, and the lower border coordinates to mtop - (H - mH) / 2 + H = mtop - 120 + 720 = mtop + 600. In other words, if the first target area A in the current time period is far from the upper and lower edges of the original image, set the upper and lower border coordinates of the second target area B in the current time period based on the position of the first target area A in the current time period and the height of its corresponding preset resolution level.

[0136] After step 413, step 414 is executed to determine whether the lower border coordinates of the second target area B in the current time period are ≥MH. If yes, step 409 is executed; otherwise, step 303 is executed. Since step 413 only ensures that the left border coordinates of the second target area B do not exceed the edge of the original image, step 414 judges the lower border coordinates of the second target area B. If it is ≥1080, it may exceed the lower edge of the original image. Therefore, the process returns to step 409, and the height of the second target area B in the current time period is directly set to the bottom of the image based on the preset resolution level. If the lower border coordinates of the second target area B are less than 1080, it means that it will not exceed the lower edge of the original image. Therefore, the process of obtaining the third target area in the current time period can continue, i.e., step 303 is executed.

[0137] After confirming the above steps 409, 411 and 414 as no, the coordinates of the left border, right border, top border and bottom border of the second target area B in the current time period are set, which completes the process of obtaining the second target area B in the current time period. Next, step 303 is executed to continue to obtain the third target area in the current time period.

[0138] like Figure 11 As shown, step 303 above includes steps 501 to 508.

[0139] Step 501: Obtain the left border gap lg, right border gap rg, top border gap tg, and bottom border gap bg. Here, the gaps refer to the distance between the third target region C' of the previous time period and the second target region B of the current time period. The left border gap lg is the distance between their left borders, the right border gap rg is the distance between their right borders, the top border gap tg is the distance between their top borders, and the bottom border gap is the distance between their bottom borders. If the current time period is the first cycle, the border of the original video frame can be used as the third target region C' of the previous time period.

[0140] Step 502: Determine if the sum of the left and right border gaps (lg + rg) is less than the width speed (VW), and the sum of the top and bottom border gaps (rg + bg) is less than the height speed (VH). If yes, proceed to step 503; otherwise, proceed to step 504. The width speed (VW) and height speed (VH) can be preset values, for example, width speed (VW) of 48 pixels and height speed (VH) of 27 pixels. To ensure the final cropped image has the same aspect ratio as the original video image and avoid image stretching during cropping, the ratio of height speed (VH) to height speed (VH) can be set to equal the aspect ratio of the original video image, both being 16:9.

[0141] Step 503: Use the second target region B in the current time period as the third target region in the current time period. That is, if the difference between the border coordinates of the second target region B in the current time period and the border coordinates of the third target region C' in the previous time period is within a preset range, it means they are close enough, and therefore the coordinates of the second target region B in the current time period can be directly used as the coordinates of the third target region C in the current time period. If the difference between the coordinates of the second target region B in the current time period and the coordinates of the third target region C' in the previous time period is outside the preset range, it means they are not close enough, and a gradual transition process is still needed. In subsequent steps 505-508, the coordinates of the third target region in the current time period are made closer to the coordinates of the second target region in the current time period relative to the coordinates of the third target region in the previous time period. In this way, the tracking of the target portrait can be achieved through a gradual animation effect.

[0142] Step 504: Determine whether the second target area in the current time period is enlarged or reduced relative to the third target area in the previous time period. If it is reduced, proceed to step 505; if it is enlarged, proceed to step 506; if the size is the same, that is, neither enlarged nor reduced, proceed to step 506.

[0143] Step 505: Shrink the third target region C' from the previous time period to obtain the third target region C of the current time period. This shrinking can be done using width speed VW and height speed VH as step sizes. That is, if step 505 is executed in multiple consecutive cycles, the width of the third target region C' from the previous time period will be reduced by 48 pixels and the height by 27 pixels in each cycle. Understandably, the shrinking step size can be different from the VW and VH mentioned above, but if the 16:9 aspect ratio is maintained, the cropping ratio can be guaranteed to be the same as the original image. Furthermore, the shrinking step size can be a fixed value or a dynamically changing value.

[0144] Step 506: Determine whether the position of the second target area B in the current time period is the same as that of the third target area C' in the previous time period. If not, proceed to step 507; if yes, proceed to step 508.

[0145] Step 507: Move the third target region C' from the previous time period towards the second target region B in the current time period to obtain the third target region C in the current time period. The step size can be a fixed value or a dynamically changing value.

[0146] Step 508: Zoom in on the third target region C' from the previous time period to obtain the third target region C for the current time period. The logic for zooming in is similar to that for zooming out. Width speed VW and height speed VH can be used as the zoom step. The zoom step can also be different from VW and VH mentioned above. Understandably, the zoom step can differ from VW and VH, but if the 16:9 aspect ratio is maintained during zooming out, the cropping ratio can be guaranteed to be the same as the original image. Furthermore, the zoom step can be a fixed value or a dynamically changing value.

[0147] After steps 505, 507, or 508, the third target region C for the current time period has been obtained, meaning the coordinates of its four sides are known. Therefore, step 304 can be executed to crop the original video frame for the current time period based on the third target region C. For example, if the original video frame is 1080P and the third target region C for the current time period is 1900p... If the resolution is 900, the cropping process involves adjusting the third target area C to a 1080P resolution. The final display of the third target area C will then be in 1080P resolution.

[0148] In the process of obtaining the third target region C, the aspect ratio of the cropped area is kept the same as that of the original video frame to avoid stretching and distortion of the final cropped image.

[0149] The processes of shrinking and enlarging are explained below.

[0150] like Figure 12As shown, in multiple consecutive periods t11~t18, if the second target region B remains unchanged, the size and position of the third target region C gradually approach the second target region B. The third target region is the actual cropping frame of the image, and it is also a frame that may gradually change. The second target region B is the final cropping frame, and it is also the centered magnified frame of the portrait in the effective portrait detection information. The third target region C will gradually track the portrait in the effective portrait detection information in multiple consecutive periods, that is, the third target region C will gradually approach the second target region B. If the coordinates of the third target region C' in the previous period are close enough to the coordinates of the second target region B in the current period, the coordinates of the second target region B are directly assigned to the third target region C in the current period to obtain the third target region C. In cycle t11, in step 504, it is determined that the second target area B of the current time period is shrunk relative to the third target area C' of the previous time period. Therefore, step 505 is executed, the center point of the third target area C' of the previous time period remains unchanged, and the border coordinates after the size reduction are used as the border coordinates of the third target area C of the current time period. Then, the image is cropped and displayed using this third target area C. Then, in cycle t12, the third target area C' of the previous time period is shrunk and used as the third target area C of the current time period. And so on, in cycles t11 to t15, the third target area C' of the previous time period is shrunk and used as the third target area C of the current time period. In cycle t16, in step 504, it is determined that the size of the second target area B of the current time period is the same as that of the third target area C' of the previous time period, that is, it has been shrunk to the correct size. Therefore, step 506 is executed, determining that the size of the second target area B of the current time period is the same as that of the third target area C' of the previous time period. The position of the third target area C' in the previous time period is different, meaning it has not been moved to the correct position. The position can be determined based on the center coordinates of the two. Therefore, step 507 is executed to move the third target area C' of the previous time period towards the direction closer to the second target area B of the current time period to obtain the third target area C of the current time period. Similarly, in cycle t17, the third target area C' of the previous time period is also moved towards the direction closer to the second target area B of the current time period to obtain the third target area C of the current time period. In cycle t18, in step 502, it is determined that the sum of the gap between the left and right borders lg+rg < the width speed VW, and the sum of the gap between the top and bottom borders rg+bg < the height speed VH. That is, it is determined that the size and position of the third target area C' of the previous time period and the second target area B of the current time period have been adjusted to the preset range. Therefore, the second target area B of the current time period is directly used as the third target area C of the current time period. During the process from t11 to t18, the process of the third target area C gradually changing to the second target area B in multiple cycles is realized, that is, the tracking of the human figure of the effective target is realized through animation.In addition, during the process from t11 to t18, the previous third target area needs to be shrunk and moved. In the overall process, this is achieved by shrinking first and then moving. This gradual approach can avoid the problem that the third target area C may exceed the display edge if it is moved before shrinking.

[0151] Furthermore, in several consecutive cycles t21~t26, the third target region C exhibits a gradual pattern of first moving and then enlarging. In cycle t21, in step 504, it is determined that the second target region B in the current period is enlarged relative to the third target region C' in the previous period. Therefore, step 506 is executed to determine that the position of the second target region B in the current period is different from that of the third target region in the previous period. This can be determined based on the coordinates of their center points. Therefore, step 507 is executed to move the third target region C' from the previous period to the right to get closer to the second target region B in the current period, resulting in the third target region C in the current period. Then, in cycle t22, similar logic continues, moving the third target region C' from the previous period to the right to obtain the third target region C in the current period. Then, in cycle t23, similar logic continues, moving the third target region C' from the previous period to the right to obtain the third target region C in the current period. Then, in cycle t24, in step 506, it is determined that the second target region B in the current period is enlarged relative to the third target region C' in the previous period. Since the position of the third target area in the previous time period is the same, i.e., it has been moved into place, step 508 is executed to enlarge the third target area C' of the previous time period to obtain the third target area C of the current time period. Then, in the t25 cycle, in step 506, it is determined that the position of the second target area B of the current time period is the same as that of the third target area in the previous time period. Step 508 is executed to enlarge the third target area C' of the previous time period to obtain the third target area C of the current time period. In the t26 cycle, in step 502, it is determined that the sum of the gaps between the left and right borders lg+rg < the width speed VW, and the sum of the gaps between the top and bottom borders rg+bg < the height speed VH. That is, it is determined that the size and position of the third target area C' of the previous time period and the second target area B of the current time period have been adjusted to the preset range. Therefore, the second target area B of the current time period is directly used as the third target area C of the current time period. In the process of t21~t26, it is achieved by moving first and then enlarging. This gradual method can avoid the problem that the third target area C may exceed the display edge if it is enlarged before moving.

[0152] In one possible implementation, during the process of obtaining the third target region C of the current time period in steps 503, 505, 507, and 508, the relevant values ​​are multiplied by 10 and then calculated using integer (int) to obtain the third target region C of the current time period with coordinates magnified 10 times. The relevant values ​​include the coordinates of the four sides and the step size. In step 304, the coordinates of the third target region C of the current time period (magnified 10 times) are divided by 10. Based on the recovered third target region C of the current time period, the original video frame of the current time period is cropped using floating-point calculation. If the calculation of the coordinates of the third target region C is directly based on floating-point calculation, the gradual change of the third target region C will accumulate errors, potentially leading to a large final cropping error. In this embodiment, for example, the upper left corner coordinates of the third target area C in the previous time period are (100, 100), the lower right corner coordinates are (420, 280), and the movement step size of the third target area C is (4.8, 2.7). These are all relevant values. First, these relevant values ​​are multiplied by 10, i.e., multiplied by 10. The upper left corner coordinates become (1000, 1000), the lower right corner coordinates become (4200, 2800), and the movement step size of the third target area C becomes (4.8, 2.7). (48, 27) It can be seen that after magnification by 10 times, the calculation method of integers (int) can be used to calculate the third target region C after the movement, that is, the third target region C in the current time period is (1048, 1027) and (4248, 2827). Then, dividing this coordinate by 10 can restore the correct coordinate values ​​to continue the subsequent floating-point operations, that is, the coordinates of the third target region C in the current time period are restored to (104.8, 102.7) and (424.8, 282.7). In this way, the error in the calculation process can be reduced and the accuracy can be improved.

[0153] It should be noted that the above description only uses the application of the video processing method in a video call as an example. However, this application embodiment does not limit the application scenarios of the video processing method. For example, in other possible implementations, the video processing method can also be applied to other scenarios requiring video recording or video preview, such as video shooting scenarios in camera applications. Furthermore, the use of a smart TV as an electronic device is merely an example; the electronic device can also be any electronic device such as a mobile phone, tablet computer, laptop computer, or in-vehicle device. The camera can be the built-in camera of the electronic device or an external camera.

[0154] The following examples illustrate video processing methods.

[0155] In one possible implementation, such as Figure 13As shown, the above-mentioned display of the cropped image includes: displaying the cropped image in any one or any combination of the video call notification interface, the video call dialing interface, and the video call interface. The cropped image is the image after adjusting the image of the third target area C to a preset size.

[0156] In one possible implementation, such as Figure 13 As shown, when a video call is received, the above video processing method is executed, including acquiring the original video frame captured by the camera and displaying a video call notification interface. The video call notification interface displays the frame cropped in step 304 above, and the cropped frame is used as a preview of the video call; and / or, when making a video call, acquiring the original video frame captured by the camera and displaying a video call interface. The video call interface displays the frame cropped in step 304 above, and the cropped frame is used as a preview of the video call; and / or, during a video call, the above video processing method is executed, including periodically acquiring the original video frame captured by the camera and displaying a video call interface. The video call interface displays the frame cropped in step 304 above, and the cropped frame is used as the video call frame.

[0157] It should be noted that the above description only uses video passing scenario as an example. The video processing method of this application embodiment can also be applied to other camera shooting scenarios such as camera preview and camera recording.

[0158] In one possible implementation, such as Figure 14 As shown, in the first time period, the first video frame is displayed. During this first time period, the distance between the first user P1 and the camera is less than the distance between the second user P2 and the camera, and the distance between the first user P1 and the camera is less than the distance between the third user P3 and the camera. The first video frame includes the image of the first user P1. Steps 101-111 are executed in the first time period and each time period preceding it. The following describes the process of executing the above steps in the first time period. The original video frame is... Figure 14 The larger rectangular frame corresponds to the first video frame. Figure 14 The smaller rectangle in the image corresponds to the second target area B. In fact, the first video frame should refer to the frame corresponding to the third target area. In this embodiment, the third target area is the same as the second target area B. For Figure 14The original video frame shown is used for human detection in step 108, resulting in three human detection information sets. Circles or ellipses in the image represent human faces, and the size of a human face is positively correlated with its corresponding distance. In step 201, the human face corresponding to the first user on the right is the closest to the camera, and is therefore the first human face. Its corresponding human detection information is the first sorted human face detection information. However, the camera distances for the second user P2 and the third user P3 are relatively large. The difference between the camera distances for the second user P2 and the first user P1 is greater than 0.5m, and the difference between the camera distances for the third user P3 and the first user P1 is also greater than 0.5m. Therefore, in step 202, the human face detection information corresponding to the second user P2 and the third user P3 is removed. The removed human face detection information only corresponds to the first user P1, and the human face corresponding to the first user P1 is designated as the human face to be determined. Then, in step 204, the image corresponding to the first user P1 is determined as the target image. Following this, in steps 301 and 302, the first target region and the second target region B for the current time period are obtained based on the image corresponding to the first user P1. Figure 14 The image only shows the second target area B, which covers the image of the first user P1. Then, in step 303, the third target area for the current time period is obtained based on the second target area B of the current time period and the third target area of ​​the previous time period. Assuming the third target area of ​​the previous time period is the second target area B of the current time period, then in the current time period, the cropped image includes the image of the first user P1, but not the images of the second user P2 and the third user P3. If the third target area of ​​the previous time period is not the second target area B of the current time period, then the cropped image will gradually become one that only includes the image of the first user P1. The third area refers to the area in the actual physical space corresponding to the first area in the original video image, and the fourth area refers to the area in the actual physical space corresponding to the second area in the original video image. In the first time period, the first user P1 is located in the third area, and the second user P2 and the third user P3 are located in the fourth area. During the above steps 201 to 304, based on the fact that during the first time period, the distance between the first user P1 and the camera is less than the distance between the second user P2 and the camera, and the difference between the two is greater than 0.5m, and the distance between the first user P1 and the camera is less than the distance between the third user P3 and the camera, and the difference between the two is greater than 0.5m, the first video image is displayed, that is, the image with the first user P1 as the main character is displayed.

[0159] After the first user, P1, moved away from the camera, the original video footage changed from... Figure 14 The result is as shown. Figure 15 As shown, in the second time period, the second video frame is displayed; the second video frame is... Figure 15The smaller rectangle in the image shows the second video frame, which includes the images of the second user P2 and the third user P3. After the first user P1 moves away from the camera, steps 101-111 are executed in the second time period. In step 201, the image corresponding to the first user P1 is still the image closest to the camera, which is the first image, and the corresponding image detection information is the first image detection information after sorting. After the first user P1 moves away from the camera, the difference between the camera distance of the second user P2 and the camera distance of the first user P1 is <0.5m, and the difference between the camera distance of the third user P3 and the camera distance of the first user P1 is <0.5m. Therefore, in step 202, the image detection information corresponding to the second user P2 and the third user P3 is not removed, and the images corresponding to the first user P1, the second user P2, and the third user P3 are images to be determined. Then, in step 207, the portraits corresponding to the second user P2 and the third user P3 in the first region on the left are identified as target portraits. Subsequently, in steps 301 and 302, the first target region and the second target region B for the current time period are obtained based on the portraits corresponding to the second user P2 and the third user P3. Figure 15 The image only shows the second target area B, which covers the images of the second user P2 and the third user P3. Then, in step 303, the third target area for the current time period is obtained based on the second target area B of the current time period and the third target area of ​​the previous time period. That is, in the second time period after the first user P1 moves away from the camera, the cropped image includes the images of the second user P2 and the third user P3, but excludes the image of the first user P1. In the second time period, the positions of the images of the three users in the original video frame remain unchanged, but the distance between the three users and the camera is equal. Since the number of users in the fourth area is greater than the number of users in the third area, the image focuses on the second user P2 and the third user P3 in the area with the larger number of users.

[0160] During the transition from the first time period to the second time period, the video display interface gradually changes from displaying the first video image to displaying the second video image. In each time period between the first and second time periods, steps 101-111 are performed. It is assumed that the distance between the second user P2 and the camera, and the distance between the third user P3 and the camera, are equal and constant. As the first user P1 moves away from the camera, the distance between the first user P1 and the camera gradually decreases. When the distance difference between the second user P2 and the camera is ≤0.5m, in step 202, the human image detection information corresponding to the second user P2 and the third user P3 is no longer removed; that is, the second target area B is changed from... Figure 14 It became Figure 15 However, the mutation in the second target region B does not cause a sudden change in the image displayed on the video display interface; the image displayed on the video display interface gradually changes from...Figure 14 The second target region B in the text changes to Figure 15 The second target region B in the [theory / information].

[0161] like Figure 16 As shown, after the fourth user P4 moves into the camera's capture range, a third video frame is displayed in the third time period. During this third time period, the distance between the fourth user P4 and the first user P1 is less than the distance between the second user P2 and the first user P1 (i.e., the fourth user P4 is closer to the first user P1, while the second user P2 is farther from the first user P1). Furthermore, the distance between the fourth user P4 and the second user P2 is greater than the distance between the second user P2 and the third user P3 (i.e., the fourth user P4 is farther from the second user P2, while the second user P2 is closer to the third user P3). The third video frame includes the images of the first user P1, the second user P2, the third user P3, and the fourth user P4. The distances mentioned in this paragraph do not refer to the camera distance, but rather to the distances between the users. When the camera capture range is expanded to include a fourth user, P4, steps 101-111 are executed in the third time period, and in each time period between the second and third time periods. Taking the third time period as an example, in step 201, the image corresponding to the first user, P1, is still the image closest to the camera, and is the first image. The corresponding image detection information is the first image detection information after sorting. When the camera capture range is expanded to include a fourth user, P4, the difference in camera distance between other users and the first user, P1, is less than 0.5m. Therefore, in step 202, the image detection information corresponding to the second user, P2, the third user, P3, and the fourth user, P4, is not removed, and the images corresponding to the first user, P1, the second user, P2, the third user, P3, and the fourth user, P4, are images to be determined. Then, in step 204, the images corresponding to the first to fourth users are determined as target images. Then, in steps 301 and 302, the first target area and the second target area B for the current time period are obtained based on the images corresponding to these four users. Figure 16 The image only shows the second target area B, which covers the portraits of the four users. Then, in step 303, the third target area for the current time period is obtained based on the second target area B for the current time period and the third target area for the previous time period. That is, in multiple time periods after the camera's capture range is expanded to include the portraits of the fourth user P4, the cropped image gradually changes from including the portraits of the second user P2 and the third user P3 to including the portraits of all four users.

[0162] When the second user P2, the third user P3, and the fourth user P4 leave the camera's capture range, and the first user P1 moves, the original video feed changes from... Figure 16 The result is as shown. Figure 17As shown, in the fourth time period, a fourth video frame is displayed, which includes the image of the first user P1. When the second user P2, the third user P3, and the fourth user P4 leave the camera's capture range, steps 101-111 are executed in the fourth time period, and steps 101-111 are executed in each time period between the third and fourth time periods. Taking the fourth time period as an example, in step 201, the image of the first user P1 is identified as the target image in step 204. Then, in steps 301 and 302, the first target area and the second target area B for the current time period are obtained based on the image of the first user P1. Figure 17 The image only shows the second target area B, which covers the image of the first user P1. Then, in step 303, the third target area of ​​the current time period is obtained based on the second target area B of the current time period and the third target area of ​​the previous time period. That is, in the fourth time period after the second user P2, the third user P3, and the fourth user P4 leave the camera's capture range, the cropped image becomes one that includes the image of the first user P1.

[0163] As can be seen, over multiple consecutive time periods, the size and position of the second target area B change as the camera distance to the figures in different regions of the original video frame changes, as does the target figure itself. Meanwhile, the third target area continuously approaches the second target area B. Therefore, the cropped image displayed gradually tracks and transforms to cover the target figure. By redefining the main character based on the changes in the number of people corresponding to the figures in the video frame and the camera distance, and then tracking and magnifying the newly determined main character through the cropped image, the main character's image in the video frame is highlighted, achieving focus on the main character and making it easier to see.

[0164] This application embodiment also provides a video processing apparatus, including: a first display module, configured to display a first video frame during a first time period, wherein the distance between a first user and a camera is less than the distance between a second user and a camera, and the distance between the first user and the camera is less than the distance between a third user and a camera, and the first video frame includes the image of the first user; a second display module, configured to display a second video frame after the first user moves away from the camera, the second video frame including the images of the second user and the third user; and a third display module, configured to display a third video frame after a fourth user moves into the camera's capture range, wherein the distance between the fourth user and the first user is less than the distance between the second user and the first user, and the distance between the fourth user and the second user is greater than the distance between the second user and the third user, and the third video frame including the images of the first user, the second user, the third user, and the fourth user.

[0165] The video processing device can apply the video processing methods in any of the above embodiments. The specific process and principle are the same as those in the above embodiments, and will not be repeated here.

[0166] It should be understood that the above division of the video processing device is merely a logical functional division. In actual implementation, all or part of these modules can be integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely through software calls from processing elements; they can be implemented entirely in hardware; or some modules can be implemented through software calls from processing elements, while others are implemented in hardware. For example, any one of the display modules can be a separate processing element, or it can be integrated into the video processing device, such as being integrated into a chip within the video processing device. Alternatively, it can be stored as a program in the memory of the video processing device, and its functions can be called and executed by a processing element within the video processing device. The implementation of other modules is similar. Moreover, these modules can be integrated together, or they can be implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0167] For example, each display module can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs). As another example, when a module is implemented via a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0168] like Figure 1 As shown, this application embodiment also provides an electronic device 100, wherein the internal memory 121 is used to store at least one instruction, which, when loaded and executed by the processor 110, causes the electronic device 100 to perform the video processing method in any of the above embodiments.

[0169] The electronic devices involved in this application may be any product such as smart TVs, mobile phones, tablets, personal computers (PCs), personal digital assistants (PDAs), smartwatches, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, in-vehicle devices, drone devices, smart cars, smart speakers, robots, smart glasses, etc.

[0170] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the video processing method described in any of the above embodiments.

[0171] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive).

[0172] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or the second signal line may represent the cases where A exists alone, A and the second signal line exist simultaneously, or the second signal line exists alone. Here, A and the second signal line may be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, the second signal line, and the third signal line may represent: a, the second signal line, the third signal line, a - the second signal line, a - the third signal line, the second signal line - the third signal line, or a - the second signal line - the third signal line, where a, the second signal line, and the third signal line may be single or multiple.

[0173] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of video processing, the method comprising: Comprising: In the first period, the distance between the first user and the camera is less than the distance between the second user and the camera, and the distance between the first user and the camera is less than the distance between the third user and the camera; In the first period, a first video picture is displayed, the first video picture comprising a portrait of the first user, the first video picture not comprising a portrait corresponding to the second user and a portrait corresponding to the third user.

2. The method of claim 1, wherein, In the second period, the first user is located in a third area, and the second user is located in a fourth area; In the second period, the difference between the distance between the second user and the camera and the distance between the first user and the camera is less than a preset distance, and the difference between the distance between the third user and the camera and the distance between the first user and the camera is less than the preset distance; In the second period, a second video picture is displayed, the second video picture comprising a portrait corresponding to the second user and a portrait corresponding to the third user, the second video picture not comprising a portrait corresponding to the first user.

3. The method of claim 1, wherein, In the third period, the first user and the fourth user are located in a third area, and the second user and the third user are located in a fourth area; In the third period, the difference between the distance between the second user and the camera and the distance between the first user and the camera is less than a preset distance, the difference between the distance between the third user and the camera and the distance between the first user and the camera is less than the preset distance, and the difference between the distance between the fourth user and the camera and the distance between the first user and the camera is less than the preset distance; In the third period, a third video picture is displayed, the third video picture comprising a portrait corresponding to the first user, a portrait corresponding to the second user, a portrait corresponding to the third user, and a portrait corresponding to the fourth user.

4. The method of claim 1, wherein, In the first period, displaying the first video picture comprises: cropping an original video picture captured by the camera in the first period to obtain a cropped first video picture, and displaying the first video picture.

5. The method of claim 1, wherein, Further comprising: After the first user moves away from the camera, a second video picture is displayed, the second video picture comprising portraits of the second user and the third user; After a fourth user moves into the camera capture range, a third video picture is displayed, the distance between the fourth user and the first user being less than the distance between the second user and the first user, and the distance between the fourth user and the second user being greater than the distance between the second user and the third user, the third video picture comprising portraits of the first user, the second user, the third user, and the fourth user.

6. The video processing method of claim 5, wherein, In the first period, the first user is located in a third area, and the second user and the third user are located in a fourth area; In the first period, displaying the first video picture comprises: display the first video picture based on that, in the first time period, the distance between the first user and the camera is less than the distance between the second user and the camera, and the distance between the first user and the camera is less than the distance between the third user and the camera.

7. The video processing method of claim 6, wherein, after the first user moves away from the camera, displaying the second video picture comprises: after the first user moves away from the camera, in a second time period, displaying the second video picture based on that, in the second time period, the distance between the first user and the camera is equal to the distance between the second user and the camera, the distance between the first user and the camera is equal to the distance between the third user and the camera, and the number of users in the fourth region is greater than the number of users in the third region.

8. The video processing method of claim 7, wherein, in a process from the first time period to the second time period, the video display interface gradually changes from displaying the first video picture to displaying the second video picture.

9. The video processing method of claim 5, wherein, in the first time period, displaying the first video picture comprises: acquiring a second target region including at least one person in the original video picture captured by the camera in the first time period; based on the second target region in the first time period and a third target region in a time period adjacent to the first time period, acquiring a third target region in the first time period, the coordinates of the third target region in the first time period being closer to the coordinates of the second target region in the first time period relative to the coordinates of the third target region in the time period adjacent to the first time period, or the coordinates of the third target region in the first time period being equal to the coordinates of the second target region in the first time period; based on the third target region in the first time period, cropping the original video picture captured by the camera in the first time period to obtain a cropped first video picture, and displaying the first video picture.

10. The video processing method of claim 9, wherein, further comprising: determining a target person in the person in the original video picture captured by the camera in the first time period, and acquiring a first target region in the first time period, the first target region in the first time period being a minimum rectangular frame including all the target persons; the acquiring of the second target region including at least one person in the original video picture captured by the camera in the first time period comprises: if a non-calculation condition is met, taking the second target region in a time period adjacent to the first time period as the second target region in the first time period; if the non-calculation condition is not met, acquiring the second target region including the first target region in the first time period in the original video picture in the first time period, the size of the second target region in the first time period being greater than the size of the first target region in the first time period; the non-calculation condition comprises at least one or any combination of the following, when the non-calculation condition comprises multiple items, any one of the multiple items not being met means that the non-calculation condition is not met, and all of the multiple items being met means that the non-calculation condition is met: The preset resolution level to which the first target region of the last time period adjacent to the first time period belongs is the same as the preset resolution level to which the first target region of the first time period belongs; The first target region of the first time period is covered by a second target region of a time period adjacent to the first time period; The target portrait quantity of the first time period is equal to a target portrait quantity of a time period adjacent to the first time period.

11. The video processing method of claim 10, wherein, in the first time period, the first user is located in a third region, and the second user and the third user are located in a fourth region; a first portrait in the original video picture corresponds to a user closest to the camera, and other portraits are second portraits, the first portrait and all the second portraits having a distance difference from the first portrait less than a preset distance are to-be-determined portraits, and the original video picture includes a first region corresponding to the third region and a second region corresponding to the fourth region; the determining of the target portraits from the portraits in the original video picture collected by the camera in the first time period includes: if the number of the to-be-determined portraits in the first region and the second region of the original video picture collected by the camera in the first time period is not equal, all the to-be-determined portraits in the region with more to-be-determined portraits are determined as target portraits; if the number of the to-be-determined portraits in the first region and the second region of the original video picture collected by the camera in the first time period is equal, all the to-be-determined portraits in the original video picture collected by the camera in the first time period are determined as target portraits.

12. The video processing method of claim 9, wherein, the process of obtaining the third target region of the first time period based on the second target region of the first time period and a third target region of a time period adjacent to the first time period includes: if the second target region of the first time period is reduced relative to the third target region of the time period adjacent to the first time period, the third target region of the time period adjacent to the first time period is reduced to obtain the third target region of the first time period; if the second target region of the first time period has the same size as the third target region of the time period adjacent to the first time period, and the center coordinate position of the second target region of the first time period is different from that of the third target region of the time period adjacent to the first time period, the third target region of the time period adjacent to the first time period is moved towards the second target region of the first time period to obtain the third target region of the first time period; and If the second target region of the first time period is enlarged in size relative to the third target region of a previous time period adjacent to the first time period, and the second target region of the first time period has the same coordinate position as the center of the third target region of the previous time period adjacent to the first time period, the third target region of the previous time period adjacent to the first time period is enlarged to obtain the third target region of the first time period.

13. The video processing method of claim 12, wherein, the process of reducing the third target region of the previous time period adjacent to the first time period to obtain the third target region of the first time period includes reducing the 10 times of the coordinate values of the third target region of the previous time period adjacent to the first time period by 10 times of the step value by means of an integer int to obtain the 10 times of the coordinate values of the third target region of the first time period; the process of moving the third target region of the previous time period adjacent to the first time period to obtain the third target region of the first time period includes moving the 10 times of the coordinate values of the third target region of the previous time period adjacent to the first time period by 10 times of the step value to the direction of the 10 times of the coordinate values of the second target region of the first time period by means of an integer int to obtain the 10 times of the coordinate values of the third target region of the first time period; the process of enlarging the third target region of the previous time period adjacent to the first time period to obtain the third target region of the first time period includes enlarging the 10 times of the coordinate values of the third target region of the previous time period adjacent to the first time period by 10 times of the step value by means of an integer int to obtain the 10 times of the coordinate values of the third target region of the first time period; the process of cropping the original video picture of the first time period based on the third target region of the first time period to obtain the cropped picture includes: dividing the 10 times of the coordinate values of the third target region of the first time period by 10 to obtain the coordinate values of the third target region of the first time period, and cropping the original video picture of the first time period based on the coordinate values of the third target region of the first time period by means of a floating point to obtain the cropped picture.

14. The video processing method of claim 9, wherein, the aspect ratio of the third target region is the same as the aspect ratio of the original video picture.

15. The video processing method of claim 1, wherein, the displaying the first video picture includes displaying the first video picture in at least one or any combination of a video call notification interface, a video call dialing interface, and a video call interface.

16. An electronic device, comprising: including: a processor and a memory, the memory being used to store at least one instruction, the instruction being loaded and executed by the processor, so that the electronic device performs the video processing method as claimed in any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that, including a program or instruction, when the program or instruction is run on a computer, the method as claimed in any one of claims 1 to 15 is executed.

Images