Video image processing method, device, equipment and readable storage medium

Abstract

The present disclosure discloses a method, device, equipment and readable storage medium for processing video images, and relates to the field of computer technology, and in particular to the field of audio and video technology. The specific implementation scheme is: in the process of video shooting, the shooting image is obtained in real time to identify the target shooting object; when it is determined that the focusing processing conditions are met based on the recognition results of at least two consecutive shooting images, a reference focusing object is determined in the current shooting image according to the target shooting object and the shooting state of the shooting device; the target focusing point is determined according to the image position of the reference focusing object in the current shooting image, and the focus of the shooting device is adjusted according to the target focusing point, and the automatic focusing can be performed according to the state of the target shooting object, while minimizing the manual focusing operation, improving the quality of video shooting, and reducing the video shooting delay caused by manual focusing.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to the field of audio and video technology, and in particular to a method, device, equipment and readable storage medium for processing video images. Background Art

[0002] With the rapid popularization of smart terminals and the rapid development of mobile Internet, Internet search technology has become an important way for users to find information.

[0003] At present, search capabilities are mainly based on the World Wide Web (WEB) ecosystem, which is an open ecosystem that relies on standard languages ​​such as Hypertext Markup Language (HTML) for information transmission and web browser-based display. In order to improve the richness and quality of search content, and thus improve the user's full-process browsing experience, it has become a trend to support self-built private content in search. With the development of short video technology, users can shoot videos to form high-quality video content for publication, so that other users can obtain the corresponding content through search. At this time, the quality and effect of the content will be indirectly fed back to the search.

[0004] In the process of implementing the present invention, the inventors found that when the prior art is performing video shooting, the focus position of the shooting device does not match the focus position actually required by the user, especially in a recording scene involving multiple characters; therefore, the user is required to manually adjust the focus, resulting in poor stability of the shooting device, longer video shooting time, and interrupted continuity of the video content, affecting the quality and effect of the shot video, which in turn causes other users to search for content of poor quality, affecting the user's search experience. Summary of the invention

[0005] The present disclosure provides a method, device, equipment and readable storage medium for processing video images.

[0006] According to one aspect of the present disclosure, a method for processing a video picture is provided, comprising:

[0007] During the video shooting process, the shooting picture is obtained in real time to identify the target shooting object;

[0008] When it is determined that the focusing processing condition is met according to the recognition results of at least two consecutive shooting pictures, a reference focusing object is determined in the current shooting picture according to the target shooting object and the shooting state of the shooting device;

[0009] A target focus point is determined according to the image position of the reference focus object in the current shooting image, and the focus of the shooting device is adjusted according to the target focus point.

[0010] According to another aspect of the present disclosure, a video picture processing device is provided, comprising:

[0011] The target object recognition module is used to obtain the shooting picture in real time to recognize the target object during the video shooting process;

[0012] A reference focus object determination module, configured to determine a reference focus object in a current shooting picture according to the target shooting object and the shooting state of the shooting device when it is determined that the focus processing condition is met according to the recognition results of at least two consecutive shooting pictures;

[0013] The focus adjustment module is used to determine a target focus point according to the image position of the reference focus object in the current shooting image, and adjust the focus of the shooting device according to the target focus point.

[0014] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video picture processing method provided by the present disclosure.

[0018] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the video picture processing method provided by the present disclosure.

[0019] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the video picture processing method provided by the present disclosure is implemented.

[0020] The technology disclosed in the present invention solves the problem of optimizing self-built content through video recording in search, improves the quality of video recording, and further optimizes the quality of search content.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a flowchart of a method for processing a video image according to an embodiment of the present disclosure;

[0024] Figure 2a is a flowchart of another method for processing a video image according to an embodiment of the present disclosure;

[0025] Figure 2b is a schematic diagram of determining a reference focus object according to an embodiment of the present disclosure;

[0026] Figure 2c is a schematic diagram of another method of determining a reference focus object according to an embodiment of the present disclosure;

[0027] Figure 3a is a flowchart of another method for processing a video image according to an embodiment of the present disclosure;

[0028] Figure 3b is a schematic diagram of determining a reference focus object according to another embodiment of the present disclosure;

[0029] Figure 3c is a schematic diagram of another method for determining a reference focus object according to an embodiment of the present disclosure;

[0030] Figure 3d is a schematic diagram of another method of determining a reference focus object according to an embodiment of the present disclosure;

[0031] Figure 3e is a schematic diagram of another method of determining a reference focus object according to an embodiment of the present disclosure;

[0032] Figure 4 is a structural schematic diagram of a video picture processing device according to an embodiment of the present disclosure;

[0033] Figure 5 The block diagram is a block diagram of an electronic device used to implement the video picture processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0035] Figure 1This is a flowchart of a method for processing a video image according to an embodiment of the present disclosure. This embodiment is applicable to the case where content is created by recording a video during a search. The method can be executed by a video image processing device, which can be implemented by software and / or hardware and integrated into an electronic device such as a computer. Specifically, refer to Figure 1 , the method specifically comprises the following steps:

[0036] Step 110: During the video shooting process, the shooting picture is obtained in real time to identify the target shooting object.

[0037] When the video is shot, the user can publish the shot video to an application that supports information search. In order to improve the viewing quality of the search content obtained by other users when they search for information, the video screen can be processed during video shooting to improve the video quality.

[0038] In the present disclosure, there may be one or more target shooting objects in the shooting picture. The target shooting object may be a person, an animal, or an object, etc. The target shooting object may have different states. For example, the target shooting object may be in a stationary state or a moving state. Specifically, when a person is used as the target shooting object, the person may speak or make some physical movements such as running, jumping, or stretching arms, etc. When an animal is used as the target shooting object, the animal may have some behavior, such as wagging its tail or eating food, etc. In the present disclosure, automatic focusing can be performed according to the state change of the target shooting object, so as to automatically adjust the focus to the object that really needs to be focused, without the user manually adjusting the focal length. More specifically, the target shooting object may be a shooting object that meets the focusing processing conditions in the shooting picture. For example, the target shooting object may be a shooting object in a moving state; or, the target shooting object may be a shooting object in a stationary state, etc.

[0039] In the present disclosure, the recognition of the target photographing object can be achieved through one or more technologies. For example, the target photographing object can be recognized by image recognition technology. More specifically, in order to realize the recognition of the state of the target photographing object, multiple frames of shooting pictures can be extracted for feature detection. For example, when the target photographing object is a person, in order to identify whether the person is speaking, facial features such as facial features, mouth shape, etc. can be extracted based on image recognition technology, and the mouth shape state and change trend can be determined by detecting the features, thereby determining the person who is currently speaking or the person who is about to speak.

[0040] Step 120: When it is determined that the focusing processing condition is met based on the recognition results of at least two consecutive shooting pictures, a reference focusing object is determined in the current shooting picture according to the target shooting object and the shooting state of the shooting device.

[0041] Among them, in the present disclosure, the focus can be automatically focused on the target shooting object that really needs to be focused according to the specific shooting needs. In the present disclosure, the focus processing condition can be determined according to the number and / or state of the target shooting object. For example, when there are target shooting objects whose states have changed in the shooting picture, it can be determined that the focus processing condition is met, and the shooting picture needs to be focused. For another example, when the number of target shooting objects in the shooting picture changes, such as when the number increases or decreases, it can be determined that the focus processing condition is met, and the shooting picture needs to be focused. For another example, when the number of target shooting objects in the shooting picture is not unique, it is impossible to determine which specific object the shooting device needs to focus on, it can be determined that the focus processing condition is met, and the shooting picture needs to be focused.

[0042] In the present disclosure, the change in the number or state of the target shooting objects can be determined by the previous frames of the current shooting picture. Specifically, the features of the previous and next frames can be compared to determine the change.

[0043] In the present disclosure, the shooting device can have different shooting states. For example, the shooting device can be in a stationary state or a moving state. Specifically, the shooting device can maintain a stationary shooting state or the shooting device can move in the up and down direction and / or in the left and right direction to shoot. The state of the shooting device will also have a certain impact on the state of the target shooting object in the current shooting picture. Therefore, when focusing, the number and / or state of the target shooting object and the shooting state of the shooting device can be considered to improve the accuracy of focal length adjustment. Among them, the shooting state of the shooting device can be determined by a variety of technologies. For example, the state change of the shooting device can be calculated based on the gyroscope.

[0044] In the present disclosure, the reference focus object can be the basis for focusing. For example, the reference focus object can be used as the target focus to adjust the focal length. Specifically, the target shooting object with a state change and / or the target shooting object with a certain consistency with the moving direction of the shooting device can be determined as the reference focus object in the current shooting picture to adjust the focal length.

[0045] Step 130: determine a target focus point according to the image position of the reference focus object in the current shooting image, and adjust the focus of the shooting device according to the target focus point.

[0046] Among them, the reference focus object can be one or more. For each reference focus object, its specific screen position in the current shooting screen can be determined. For example, the screen position of the reference focus object can be marked by an identification frame. Specifically, the identification frame can be a rectangular frame. The specific position and size of the identification frame can be determined according to the maximum boundary point of the reference focus object. The center point of the identification frame can be used as the target focus point. When shooting the current shooting screen, the focus of the shooting device can be adjusted to the target focus by automatically adjusting the focal length. The focusing scheme disclosed in the present invention achieves the effect of automatically focusing the focus of the shooting device on the object that really needs to be focused without manual focusing; it avoids the phenomenon of shooting interruption, thereby ensuring the continuity of the picture and the dynamic effective focusing of the picture, thereby improving the quality of the shooting picture and reducing the probability of video re-recording. The video recorded by focusing through the disclosed scheme can be published to an application product with a search function. When other users obtain the video through search, the user can watch high-quality search content, realize the enrichment and quality of the search content, and thus improve the user's search and browsing experience.

[0047] The technical solution of the disclosed embodiment is to obtain the shooting picture in real time during the video shooting process to identify the target shooting object; when it is determined that the focusing processing conditions are met based on the recognition results of at least two consecutive shooting pictures, a reference focusing object is determined in the current shooting picture according to the target shooting object and the shooting status of the shooting device; the target focusing point is determined according to the picture position of the reference focusing object in the current shooting picture, and the focus of the shooting device is adjusted according to the target focusing point, which solves the problem of optimizing self-built content through video recording in content search, and can automatically focus according to the status of the target shooting object, while minimizing the manual focusing operation, improving the quality of video shooting, and reducing the video shooting delay caused by manual focusing, thereby improving the quality of search content and the user's search and browsing experience.

[0048] Figure 2a It is a flow chart of another method for processing video images according to an embodiment of the present disclosure. This embodiment is a further refinement of the above technical solution. The technical solution in this embodiment can be combined with one or more of the above implementation methods.

[0049] Specifically, in an optional implementation of the present disclosure, determining that the focusing processing condition is met according to the recognition results of at least two consecutive captured images includes:

[0050] When it is determined, based on the recognition results of at least two consecutive shooting pictures, that a first target shooting object in motion exists in the video segment formed by the at least two consecutive shooting pictures, it is determined that the focusing processing condition is met.

[0051] In an optional implementation of the present disclosure, determining a reference focus object in a current shooting picture according to a target shooting object and a shooting state of a shooting device includes:

[0052] If the photographing device is in a moving state, and there are multiple first target photographing objects in the current photographing picture, a reference focusing object is determined according to the moving direction of the photographing device and the layout of the multiple first target photographing objects in the current photographing picture;

[0053] If the photographing device is in a moving state, and there is only one first target photographing object in the current photographing picture, determining the single first target photographing object as the reference focusing object;

[0054] If the photographing device is in a stationary state, all first target photographing objects in the current photographing picture are determined as reference focus objects.

[0055] Specifically, refer to Figure 2a The video processing method provided by the present disclosure includes the following steps:

[0056] Step 210: During the video shooting process, the shooting picture is obtained in real time to identify the target shooting object.

[0057] Step 220: When it is determined based on the recognition results of at least two consecutive shooting pictures that there is a first target shooting object in motion in the video segment formed by the at least two consecutive shooting pictures, it is determined that the focusing processing condition is met.

[0058] Exemplarily, the motion state of the first target object may be a person speaking, or a person having body movement changes such as moving a microphone; or, the motion state of the first target object may be an animal moving; or, the motion state of the first target object may be a plant moving such as a flower blooming, etc.

[0059] In the present disclosure, when the first target shooting object is in motion, it can be preliminarily determined that the first target object is the object that needs to be focused in the current video screen, so that it can be determined that the focusing processing conditions are met and the focus adjustment needs to be performed. Before adjusting the focus, a reference focus object can be determined first, and the focus adjustment is performed based on the reference focus object, which can improve the accuracy of the focus adjustment. The determination of the reference focus object can be discussed on a case-by-case basis. Specifically, the reference focus object can be determined in different cases through step 230, step 240 or step 250.

[0060] Step 230: If the photographing device is in a moving state and there are multiple first target photographing objects in the current photographing picture, determine the reference focus object according to the moving direction of the photographing device and the layout of the multiple first target photographing objects in the current photographing picture.

[0061] Among them, when there is a first target object in motion in the current shooting picture, the shooting device is in a moving state, and there are multiple first target objects in the current shooting picture, the determination of the reference focus object needs to be based on the moving direction of the shooting device and the layout of the first target objects in the current shooting picture.

[0062] Specifically, the following principles may be followed when determining the reference focus object: the first target shooting object in motion has a higher focus priority than the target shooting object in a stationary state; the first target shooting object with high consistency with the moving direction of the shooting device has a higher focus priority than the first target shooting object with low consistency with the moving direction of the shooting device. In the present disclosure, the first target shooting object with high consistency with the direction of the shooting device and / or in motion may be determined as the reference focus object according to the moving direction of the shooting device and the layout of multiple first target shooting objects in the current shooting picture.

[0063] Furthermore, in an optional embodiment of the present disclosure, a reference focusing object is determined according to a moving direction of a shooting device and a layout of a plurality of first target shooting objects in a current shooting picture, including: constructing a direction vector between the first target shooting objects according to the layout of the plurality of first target shooting objects in the current shooting picture; matching each direction vector with a moving direction of the shooting device, and determining a target direction vector that is relatively consistent with the moving direction; and determining the end point target shooting object pointed to by the target direction vector as the reference focusing object.

[0064] The direction vector may be constructed by connecting the first target shooting objects in the current shooting picture in pairs to form a vector with directionality. Figure 2b FIG. 1 is a schematic diagram of determining a reference focus object according to an embodiment of the present disclosure. Figure 2b As shown, first target photographing object 1, first target photographing object 2, first target photographing object 3 and first target photographing object 4 have direction vectors between each other. Figure 2b Indicated by arrows.

[0065] In the present disclosure, the target direction vector may be a direction vector that is consistent with the moving direction of the shooting device. Specifically, the target direction vector may not be completely consistent with the moving direction of the shooting device, but may have a certain common trend. For example, Figure 2bIn the example, when the moving direction of the photographing device is rightward, the direction vector from the first target photographing object 1 to the first target photographing object 2 and the direction vector from the first target photographing object 3 to the first target photographing object 4 can both be determined as the target direction vector. Figure 2b In the figure, when the moving direction of the photographing device is to the lower right, the direction vector from the first target photographing object 1 to the first target photographing object 4 can be determined as the target direction vector.

[0066] In this step, the first target photographing object is an object in motion, and the end target photographing object pointed to by the target direction vector can be directly used as the reference focusing object. Figure 2b When the first target photographing objects are in motion and the moving direction of the photographing device is rightward, the first photographing object 2 and the first target photographing object 4 can be determined as reference focusing objects. Figure 2b When the first target photographing object is in motion and the moving direction of the photographing device is moving to the lower right, the first target photographing object 4 can be determined as the reference focusing object. That is, the focus priority of the target photographing object in the moving direction of the photographing device is higher than the focus priority of the target photographing object in the non-moving direction of the photographing device.

[0067] Step 240: If the photographing device is in a moving state and there is only one first target photographing object in the current photographing picture, the single first target photographing object is determined as a reference focusing object.

[0068] Among them, when there is a single first target shooting object in motion in the current shooting picture and the shooting device is in a moving state, the single first target shooting object can be directly determined as the reference focusing object for focusing.

[0069] Step 250: If the photographing device is in a stationary state, all first target photographing objects in the current photographing picture are determined as reference focus objects.

[0070] in, Figure 2c FIG. 4 is a schematic diagram of another method for determining a reference focus object according to an embodiment of the present disclosure. Figure 2c As shown, the camera device is in a stationary state. Figure 2cThere are target photographing object 5, target photographing object 6 and target photographing object 7. If only target photographing object 5 and target photographing object 6 are in a moving state, then target photographing object 5 and target photographing object 6 can be used as the first target photographing object and determined as the reference focusing object. If only target photographing object 5 or target photographing object 6 is in a moving state, then target photographing object 5 or target photographing object 6 in a moving state can be used as the first target photographing object and determined as the reference focusing object.

[0071] Step 260: determine a target focus point according to the image position of the reference focus object in the current shooting image, and adjust the focus of the shooting device according to the target focus point.

[0072] The technical solution of the disclosed embodiment is to obtain the shooting picture in real time during the video shooting process to identify the target shooting object; when it is determined according to the recognition results of at least two consecutive shooting pictures that there is a first target shooting object in motion in the video clip formed by the at least two consecutive shooting pictures, it is determined that the focusing processing condition is met; if the shooting device is in a moving state and there are multiple first target shooting objects in the current shooting picture, a reference focusing object is determined according to the moving direction of the shooting device and the layout of the multiple first target shooting objects in the current shooting picture; if the shooting device is in a moving state and there are multiple first target shooting objects in the current shooting picture, a reference focusing object is determined according to the moving direction of the shooting device and the layout of the multiple first target shooting objects in the current shooting picture; if the shooting device is in a moving state and there are multiple first target shooting objects in the current shooting picture, a reference focusing object is determined according to the moving direction of the shooting device and the layout of the multiple first target shooting objects in the current shooting picture. If the number of photographed objects is single, the single first target photographed object is determined as the reference focus object; if the photographing device is in a stationary state, all the first target photographed objects in the current shooting picture are determined as reference focus objects; according to the image position of the reference focus object in the current shooting picture, the target focus point is determined, and the focus of the photographing device is adjusted according to the target focus point, which solves the problem of optimizing self-built content through video recording in search, and can automatically focus according to the state of the target photographed object, while minimizing the manual focusing operation, improving the quality of video shooting, reducing the video shooting delay caused by manual focusing, and thereby improving the quality of search content.

[0073] Figure 3a It is a flow chart of another method for processing video images according to an embodiment of the present disclosure. This embodiment is a further refinement of the above technical solution. The technical solution in this embodiment can be combined with one or more of the above implementation methods.

[0074] Specifically, in an optional implementation of the present disclosure, determining that the focusing processing condition is met according to the recognition results of at least two consecutive captured images includes:

[0075] When it is determined based on the recognition results of at least two consecutive shooting pictures that all the target shooting objects in the video segment formed by the at least two consecutive shooting pictures are the second target shooting objects in a stationary state, it is determined that the focusing processing condition is met.

[0076] In an optional implementation of the present disclosure, determining a reference focus object in a current shooting picture according to a target shooting object and a shooting state of a shooting device includes:

[0077] If the photographing device is in a moving state, the reference focusing object is determined according to the moving direction of the photographing device and the positional relationship between each second target photographing object in the current photographing picture and the center point of the picture;

[0078] If the shooting device is in a stationary state, the second target shooting object that is closest to the center point of the current shooting picture is determined as the reference focus object.

[0079] refer to Figure 3a , the method specifically comprises the following steps:

[0080] Step 310: During the video shooting process, the shooting picture is obtained in real time to identify the target shooting object.

[0081] Step 320: When it is determined based on the recognition results of at least two consecutive shooting pictures that the target shooting objects in the video clip formed by the at least two consecutive shooting pictures are all second target shooting objects in a stationary state, it is determined that the focusing processing condition is met.

[0082] If there is a target photographing object in motion in the current shooting picture, the focusing process can be performed in the manner of the above embodiment. If the multiple target photographing objects in the current shooting picture are all in a stationary state, that is, there is no target photographing object in motion in the current shooting picture, it can be determined that the focusing process condition is met, and the focusing process is performed according to the following method steps, specifically, step 330 or step 340 can be performed to determine the reference focusing photographing object, and then step 350 can be performed to adjust the focus.

[0083] Specifically, if there is only a single target photographing object in the current shooting picture, which may be in a stationary state or a moving state, the focus of the shooting device is usually directly focused on the single target photographing object, and the focus adjustment may not be performed; however, in practice, the presence of only a single target photographing object in the current shooting picture may be caused by the departure of other target photographing objects. In order to make the picture focusing effect more accurate, the focus adjustment may also be performed for the case where there is only a single target photographing object in the current shooting picture. When adjusting the focus for the case where there is only a single target photographing object in the current shooting picture, the above method may be used for specific focus adjustment, or the following method and steps may be used for focus adjustment.

[0084] Step 330: If the photographing device is in a moving state, a reference focusing object is determined according to the moving direction of the photographing device and the positional relationship between each second target photographing object in the current photographing picture and the center point of the picture.

[0085] Among them, for the second target photographing object in a stationary state, when determining the reference focus object, the following principle can be followed: give priority to focusing on the photographing object near the center point of the picture and the photographing object that is consistent with the moving direction of the shooting device. Specifically, when determining whether the second target photographing object is near the center point of the picture, the second target photographing object closest to the center point of the picture can be selected. When determining whether the second target photographing object is consistent with the moving direction of the shooting device, the second target photographing object corresponding to the direction formed by the second target photographing object and the center point of the picture that is closest to the moving direction of the shooting device can be selected.

[0086] More specifically, in an optional embodiment of the present disclosure, a reference focus object is determined according to a moving direction of a shooting device and a positional relationship between each second target shooting object in a current shooting picture and a center point of the picture, including: respectively determining a distance value between each second target shooting object in the current shooting picture and the center point of the picture, and screening out candidate reference focus objects whose distance value is less than or equal to a preset distance threshold; if there are multiple candidate reference focus objects, determining a source moving area and a target moving area in the current shooting picture according to the moving direction of the shooting device; wherein the moving direction is from the source moving area to the target moving area; determining the candidate reference focus object located in the target moving area as the reference focus object; if there is a single candidate reference focus object, determining the single candidate reference focus object as the reference focus object.

[0087] Among them, the distance value between the second target photographic object and the center point of the picture can be determined by the distance value in the picture. A distance measurement module can be installed in the shooting device to measure the picture distance. The preset distance threshold can be a smaller value, such as a value close to 0; or the preset distance threshold can be determined based on the distance between multiple second target photographic objects, for example, the preset distance threshold can be the minimum distance between multiple second target photographic objects or half of the minimum distance, etc. In specific applications, the preset distance threshold can be specifically set according to different usage scenarios.

[0088] In the present disclosure, the second target photographing object that meets the preset distance threshold condition can be determined as an alternative reference focusing object, and the reference focusing object can be determined according to the moving direction of the photographing device. Specifically, if the number of alternative reference focusing objects is single, the single alternative reference focusing object can be directly determined as the reference focusing object. If the number of alternative reference focusing objects is multiple, the alternative reference focusing object that meets the consistency of the moving direction of the photographing device can be determined as the reference focusing object.

[0089] More specifically, a source moving area and a target moving area may be constructed in the current shooting picture according to the moving direction of the shooting device, and the candidate reference focus object in the target moving area may be determined as the reference focus object. The source moving area may be the area corresponding to before the shooting device moves, the target moving area may be the area corresponding to after the shooting device moves, and the moving direction of the shooting device may be from the source moving area to the target moving area.

[0090] For example, Figure 3b is a schematic diagram of yet another method of determining a reference focus object according to an embodiment of the present disclosure. Figure 3b There are second target shooting object 8, second target shooting object 9 and second target shooting object 10 in the image, and only second target shooting object 9 (at the center point of the image) meets the preset distance threshold condition, so second target shooting object 9 can be used as an alternative reference focusing object, and the number of alternative reference focusing objects is single, so second target shooting object 9 can be determined as the reference focusing object. Figure 3b The arrow in the middle indicates the direction of movement of the camera.

[0091] Another example, Figure 3c is a schematic diagram of another method for determining a reference focus object according to an embodiment of the present disclosure. Figure 3c There are second target photographing objects 9 and second target photographing objects 10 in the image, and the second target photographing objects 9 and second target photographing objects 10 meet the preset distance threshold condition. The second target photographing objects 9 and second target photographing objects 10 can be used as alternative reference focusing objects, and the number of alternative reference focusing objects is multiple (2). If the moving direction of the shooting device is rightward (such as Figure 3c The moving direction indicated by the arrow in the middle), that is, the photographing device moves from the source moving area where the second target photographing object 9 is located to the target moving area where the second target photographing object 10 is located, and the second target photographing object 10 in the candidate reference focusing objects can be determined as the reference focusing object. Figure 3c In the figure, the picture is divided into a source moving area and a target moving area by a dotted line.

[0092] Another example, Figure 3d is a schematic diagram of another method of determining a reference focus object according to an embodiment of the present disclosure. Figure 3d There are second target photographing objects 8, second target photographing objects 9, second target photographing objects 10 and second target photographing objects 11 in the image processing, and the second target photographing objects 8, second target photographing objects 9, second target photographing objects 10 and second target photographing objects 11 all meet the preset distance threshold condition. The second target photographing objects 8, second target photographing objects 9, second target photographing objects 10 and second target photographing objects 11 can be used as alternative reference focusing objects, and the number of alternative reference focusing objects is multiple (4). If the moving direction of the shooting device is moving to the lower right (such as Figure 3d The moving direction indicated by the arrow in the middle), that is, the photographing device moves from the source moving area where the second target photographing object 8 is located to the target moving area where the second target photographing object 11 is located, and the second target photographing object 11 among the candidate reference focusing objects can be determined as the reference focusing object. Figure 3d In the figure, the picture is divided into a source moving area and a target moving area by a dotted line.

[0093] Step 340: If the photographing device is in a stationary state, the second target photographing object that is closest to the center point of the current photographing picture is determined as the reference focusing object.

[0094] If the shooting device and the second target shooting object are both in a stationary state, the shooting object in the middle of the picture can be considered as the reference focus object. The second target shooting object closest to the center point of the picture can be determined by calculating the distance value between each second target shooting object and the center point of the picture. If there are multiple second target shooting objects closest to the center point of the picture, they can all be determined as reference focus objects.

[0095] For example, Figure 3e is a schematic diagram of yet another method of determining a reference focus object according to an embodiment of the present disclosure. Figure 3e There are second target photographing objects 8, second target photographing objects 9 and second target photographing objects 10, the photographing device is in a stationary state, and the second target photographing object 9 is closest to the center point of the picture. The second target photographing object 9 can be determined as the reference focusing object.

[0096] Step 350: determine a target focus point according to the image position of the reference focus object in the current shooting image, and adjust the focus of the shooting device according to the target focus point.

[0097] The technical solution of the disclosed embodiment is to obtain the shooting picture in real time during the video shooting process to identify the target shooting object; when it is determined according to the identification results of at least two consecutive shooting pictures that the target shooting objects in the video clip formed by at least two consecutive shooting pictures are all second target shooting objects in a static state, it is determined that the focusing processing condition is met; if the shooting device is in a moving state, a reference focusing object is determined according to the moving direction of the shooting device and the positional relationship between each second target shooting object and the center point of the picture in the current shooting picture; if the shooting device is in a static state, the second target shooting object closest to the center point of the picture in the current shooting picture is determined as the reference focusing object; according to the picture position of the reference focusing object in the current shooting picture, the target focusing point is determined, and the focus of the shooting device is adjusted according to the target focusing point, which solves the problem of optimizing self-built content through video recording in search, can automatically focus according to the state of the target shooting object, minimize the manual focusing operation, improve the quality of video shooting, reduce the video shooting delay caused by manual focusing, and thus improve the quality of search content.

[0098] Figure 4 is a structural diagram of a video image processing device according to an embodiment of the present disclosure, and the device can be set in an electronic device. Specifically, Figure 4 As shown, the video image processing device 400 includes: a target shooting object recognition module 410, a reference focus object determination module 420 and a focus adjustment module 430. Among them:

[0099] The target object recognition module 410 is used to obtain the shooting picture in real time to recognize the target object during the video shooting process;

[0100] A reference focus object determination module 420 is used to determine a reference focus object in a current shooting picture according to a target shooting object and a shooting state of a shooting device when it is determined that a focus processing condition is met according to the recognition results of at least two consecutive shooting pictures;

[0101] The focus adjustment module 430 is used to determine a target focus point according to the image position of a reference focus object in the current shooting image, and adjust the focus of the shooting device according to the target focus point.

[0102] Optionally, the reference focus object determination module 420 includes:

[0103] The first focus processing condition determination unit is used to determine that the focus processing condition is met when it is determined that there is a first target shooting object in motion in the video clip formed by the at least two consecutive shooting pictures based on the recognition results of the at least two consecutive shooting pictures.

[0104] Optionally, the reference focus object determination module 420 includes:

[0105] a first reference focus object determining unit, configured to determine a reference focus object according to a moving direction of the photographing device and a layout of the plurality of first target photographing objects in the current photographing picture if the photographing device is in a moving state and there are a plurality of first target photographing objects in the current photographing picture;

[0106] a second reference focus object determining unit, configured to determine a single first target photographing object as a reference focus object if the photographing device is in a moving state and there is only one first target photographing object in a current photographing picture;

[0107] The third reference focus object determination unit is configured to determine all first target shooting objects in the current shooting picture as reference focus objects if the shooting device is in a stationary state.

[0108] Optionally, the first reference focus object determination unit includes:

[0109] A direction vector construction subunit, used to construct direction vectors between the target shooting objects according to the layout of the plurality of first target shooting objects in the current shooting picture;

[0110] A target direction vector determination subunit, used to match each direction vector with the moving direction of the shooting device, and determine a target direction vector that is relatively consistent with the moving direction;

[0111] The first reference focus object determination subunit is used to determine the end point target shooting object pointed to by the target direction vector as the reference focus object.

[0112] Optionally, the reference focus object determination module 420 includes:

[0113] The second focusing processing condition determination unit is used to determine that the focusing processing condition is satisfied when it is determined, based on the recognition results of at least two consecutive shooting pictures, that the target shooting objects in the video clip formed by the at least two consecutive shooting pictures are all second target shooting objects in a stationary state.

[0114] Optionally, the reference focus object determination module 420 includes:

[0115] a fourth reference focus object determination unit, configured to determine the reference focus object according to the moving direction of the shooting device and the positional relationship between each second target shooting object and the center point of the picture in the current shooting picture if the shooting device is in a moving state;

[0116] The fifth reference focus object determination unit is used to determine the second target shooting object closest to the center point of the current shooting picture as the reference focus object if the shooting device is in a stationary state.

[0117] Optionally, the fourth reference focus object determination unit includes:

[0118] The candidate reference focusing object screening subunit is used to respectively determine the distance value between each second target shooting object in the current shooting picture and the center point of the picture, and screen the candidate reference focusing objects whose distance value is less than or equal to a preset distance threshold;

[0119] The area determination subunit is used to determine the source moving area and the target moving area in the current shooting picture according to the moving direction of the shooting device if the number of candidate reference focusing objects is multiple; wherein the moving direction is from the source moving area to the target moving area;

[0120] A second reference focus object determination subunit is used to determine the candidate reference focus object located in the target moving area as the reference focus object;

[0121] The third reference focus object determination subunit is configured to determine a single candidate reference focus object as the reference focus object if the number of candidate reference focus objects is single.

[0122] The video picture processing device provided in the embodiment of the present disclosure can execute the video picture processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0123] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0124] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0125] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0126] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0127] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as a processing method for a video screen. For example, in some embodiments, the processing method for a video screen may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the processing method for the video screen described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform a processing method for a video screen by any other appropriate means (e.g., by means of firmware).

[0128] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0130] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0132] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0133] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0134] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0135] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for processing a video image, comprising: During the video shooting process, the shooting picture is obtained in real time to identify the target shooting object; When it is determined that the focusing processing condition is met according to the recognition results of at least two consecutive shooting pictures, a reference focusing object is determined in the current shooting picture according to the number and / or motion state of the target shooting objects and the shooting state of the shooting device; the shooting state includes whether the shooting device is in a moving state or a stationary state; Determining a target focus point according to the image position of the reference focus object in the current shooting image, and adjusting the focus of the shooting device according to the target focus point; Wherein, determining that the focusing processing condition is met according to the recognition results of at least two consecutive shooting pictures includes: When it is determined, based on the recognition results of the at least two consecutive shooting pictures, that a first target shooting object in motion exists in the video segment formed by the at least two consecutive shooting pictures, it is determined that the focusing processing condition is met; Wherein, determining the reference focusing object in the current shooting picture according to the target shooting object and the shooting state of the shooting device includes: if the shooting device is in a moving state and there are multiple first target shooting objects in the current shooting picture, determining the reference focusing object according to the moving direction of the shooting device and the layout of the multiple first target shooting objects in the current shooting picture; Determining a reference focus object according to the moving direction of the shooting device and the layout of the plurality of first target shooting objects in the current shooting picture includes: Constructing direction vectors between the first target shooting objects according to the layout of the plurality of first target shooting objects in the current shooting picture; Matching each of the direction vectors with the moving direction of the photographing device, and determining a target direction vector that is relatively consistent with the moving direction; The end point target shooting object pointed by the target direction vector is determined as the reference focusing object.

2. The method according to claim 1, wherein: Determining a reference focus object in a current shooting picture according to the target shooting object and the shooting state of the shooting device includes: If the photographing device is in a moving state, and there is only one first target photographing object in the current photographing picture, determining the single first target photographing object as a reference focusing object; If the photographing device is in a stationary state, all first target photographing objects in the current photographing picture are determined as reference focus objects.

3. The method according to claim 1, wherein: Determining, based on the recognition results of at least two consecutive captured images, whether a focusing processing condition is satisfied includes: When it is determined based on the recognition results of at least two consecutive shooting pictures that all the target shooting objects in the video segment formed by the at least two consecutive shooting pictures are the second target shooting objects in a stationary state, it is determined that the focusing processing condition is met.

4. The method according to claim 3, wherein: Determining a reference focus object in a current shooting picture according to the target shooting object and the shooting state of the shooting device includes: If the photographing device is in a moving state, determining a reference focusing object according to the moving direction of the photographing device and the positional relationship between each second target photographing object in the current photographing picture and the center point of the picture; If the photographing device is in a stationary state, the second target photographing object which is closest to the center point of the picture in the current photographing picture is determined as the reference focusing object.

5. The method according to claim 4, wherein: Determining a reference focus object according to the moving direction of the shooting device and the positional relationship between each second target shooting object in the current shooting picture and the center point of the picture includes: Determine the distance between each second target shooting object in the current shooting picture and the center point of the picture respectively, and select the candidate reference focusing objects whose distance value is less than or equal to the preset distance threshold; If the number of the candidate reference focusing objects is multiple, determining a source moving area and a target moving area in the current shooting picture according to the moving direction of the shooting device; wherein the moving direction is from the source moving area to the target moving area; Determine the candidate reference focus object located in the target moving area as the reference focus object; If the number of the candidate reference focus objects is single, the single candidate reference focus object is determined as the reference focus object.

6. A video image processing device, comprising: The target object recognition module is used to obtain the shooting picture in real time to recognize the target object during the video shooting process; A reference focus object determination module, configured to determine a reference focus object in a current shooting picture according to the number and / or motion state of the target shooting objects and the shooting state of the shooting device when it is determined that the focusing processing condition is met according to the recognition results of at least two consecutive shooting pictures; the shooting state includes whether the shooting device is in a moving state or a stationary state; A focus adjustment module, used to determine a target focus point according to the image position of the reference focus object in the current shooting image, and adjust the focus of the shooting device according to the target focus point; Wherein, the reference focus object determination module includes: A first focusing processing condition determination unit is used to determine that a focusing processing condition is satisfied when it is determined, based on the recognition result of at least two consecutive shooting pictures, that a first target shooting object in motion exists in a video segment formed by at least two consecutive shooting pictures; Wherein, the reference focus object determination module includes: a first reference focus object determining unit, configured to determine a reference focus object according to a moving direction of the shooting device and a layout of the plurality of first target shooting objects in the current shooting picture if the shooting device is in a moving state and there are a plurality of first target shooting objects in the current shooting picture; Wherein, the first reference focus object determination unit includes: A direction vector construction subunit, used to construct direction vectors between the first target shooting objects according to the layout of the multiple first target shooting objects in the current shooting picture; a target direction vector determination subunit, used to match each of the direction vectors with the moving direction of the photographing device, and determine a target direction vector that is relatively consistent with the moving direction; The first reference focus object determination subunit is used to determine the end point target shooting object pointed to by the target direction vector as the reference focus object.

7. The device according to claim 6, wherein: Reference focus object determination module, comprising: a second reference focus object determining unit, configured to determine a single first target photographing object as a reference focus object if the photographing device is in a moving state and there is only one first target photographing object in the current photographing picture; The third reference focus object determination unit is configured to determine all first target photographing objects in the current photographing picture as reference focus objects if the photographing device is in a stationary state.

8. The device according to claim 6, wherein: Reference focus object determination module, comprising: The second focusing processing condition determination unit is used to determine that the focusing processing condition is satisfied when it is determined, based on the recognition results of at least two consecutive shooting pictures, that the target shooting objects in the video clip formed by the at least two consecutive shooting pictures are all second target shooting objects in a stationary state.

9. The device according to claim 8, wherein: Reference focus object determination module, comprising: a fourth reference focus object determination unit, configured to determine a reference focus object according to a moving direction of the shooting device and a positional relationship between each second target shooting object and a center point of a current shooting picture if the shooting device is in a moving state; The fifth reference focus object determination unit is used to determine the second target shooting object closest to the center point of the current shooting picture as the reference focus object if the shooting device is in a stationary state.

10. The device according to claim 9, wherein: The fourth reference focus object determination unit includes: The candidate reference focusing object screening subunit is used to respectively determine the distance value between each second target shooting object in the current shooting picture and the center point of the picture, and screen the candidate reference focusing objects whose distance value is less than or equal to a preset distance threshold; an area determination subunit, for determining a source moving area and a target moving area in the current shooting picture according to a moving direction of the shooting device if the number of the candidate reference focusing objects is multiple; wherein the moving direction is from the source moving area to the target moving area; A second reference focus object determination subunit is used to determine the candidate reference focus object located in the target movement area as the reference focus object; The third reference focus object determination subunit is configured to determine a single candidate reference focus object as the reference focus object if the number of the candidate reference focus objects is single.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images