Image processing method, apparatus, electronic device and storage medium

By identifying and cropping the target image frames and simulating the lens movement, the problems of unsatisfactory and complex operation in the prior art are solved, and efficient and low-cost video mirroring effect are achieved.

WO2024245398A1PCT designated stage expired Publication Date: 2024-12-05BEIJING ZITIAO NETWORK TECH CO LTD

Patent Information

Application Number
PCT/CN2024/096665
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-01
Filing Date
2024-05-31
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

During the video generation process, it is difficult for the existing technology to effectively achieve the effect of mirror-operating. The hand-held bracket requires mirror-operating skills and is costly, and the post-editing operation is complex and time-consuming.

Method used

By identifying the target image frame, the object position and state are determined, the image is cropped and scaled based on this information to simulate lens movement and achieve the mirror effect.

Benefits of technology

It avoids tedious manual mirror movement processes, enriches the image appeal, improves user experience, simplifies the operation process, and reduces costs and time overhead.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024096665_05122024_PF_FP_ABST
    Figure CN2024096665_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the present disclosure are an image processing method, an apparatus, an electronic device and a storage medium. The method comprises: recognizing a target image frame to determine a first recognition object position and a second recognition object state; on the basis of the first recognition object position, determining a target cropping position and, on the basis of the second recognition object state, determining a target cropping size, the target cropping position being a cropping center position when the target image frame is cropped, and the target cropping size being a cropping size when the target image frame is cropped; and, according to the target cropping position and the target cropping size, cropping the target image frame and scaling same to a preset size. Executing the technical solution of the embodiments of the present disclosure can avoid complex manual camera moving processes, enrich picture infectivity, and improve user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Image processing method, device, electronic device and storage medium

[0001] This application claims priority to Chinese Patent Application No. 202310646038.5 filed on June 1, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] Embodiments of the present disclosure relate to an image processing method, apparatus, electronic device, and storage medium. Background Art

[0003] During the video generation process, different camera movements are usually used to process the video. For example, when shooting a dance video, using different camera movements according to body movements can greatly enhance the expressiveness and appeal of the video.

[0004] To add camera movement effects to videos, there are two approaches: one is to use a handheld stand to move the camera during filming to achieve the effect, and the other is to edit existing videos in post-production to create the resulting video with the camera movement effect. However, using a handheld stand to achieve the camera movement effect requires the photographer to possess certain camera movement skills, which is costly to implement. Post-production editing usually requires manual editing, which is complex and time-consuming.

[0005] Summary of the Invention

[0006] The present disclosure provides an image processing method, device, electronic device, and storage medium, which can avoid tedious manual camera movement processes, enrich the appeal of images, and enhance user experience.

[0007] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising:

[0008] By identifying the target image frame, determining the first identification object position and the second identification object state;

[0009] determining a target cropping position based on the first recognized object position and determining a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping the target image frame, and the target cropping size is a cropping size when cropping the target image frame;

[0010] An image is cropped from the target image frame according to the target cropping position and the target cropping size and is scaled to a preset size.

[0011] In a second aspect, an embodiment of the present disclosure further provides an image processing device, the image processing device comprising:

[0012] An image frame recognition module, configured to determine a first recognition object position and a second recognition object state by recognizing a target image frame;

[0013] a cropping position and cropping size determination module, configured to determine a target cropping position based on the first recognized object position and a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping a target image frame, and the target cropping size is a cropping size when cropping the target image frame;

[0014] The image cropping module is used to crop an image from the target image frame and scale the image to a preset size according to the target cropping position and the target cropping size.

[0015] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] a storage device for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method described in any embodiment of the present disclosure.

[0019] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, wherein the computer-readable storage medium stores computer instructions, and the computer-executable instructions are used to execute the image processing method described in any embodiment of the present disclosure when executed by a computer processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0021] FIG1 is a flow chart of an image processing method provided by an embodiment of the present disclosure;

[0022] FIG2a is a flow chart of another image processing method provided by an embodiment of the present disclosure;

[0023] FIG2 b is a flow chart of an image processing method provided by an embodiment of the present disclosure;

[0024] FIG3 is a flowchart of another image processing method provided by an embodiment of the present disclosure;

[0025] FIG4 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure; and

[0026] FIG5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0034] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0035] As an optional but non-limiting implementation, in response to a user's active request, the prompt information may be sent to the user in the form of a pop-up window, for example, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "Agree" or "Disagree" to provide personal information to the electronic device.

[0036] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0037] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0038] FIG1 is a flow chart of an image processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the case of generating a camera movement video. The method can be executed by an image processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC, or a server. As shown in FIG1 , the method includes:

[0039] S110 , determining a first recognition object position and a second recognition object state by recognizing a target image frame.

[0040] The position of the first identified object can be determined based on the position of the target part corresponding to the first identified object identified from the target image frame. The target part may include at least one of the head position, hand position, foot position, hip position, and shoulder position of the first identified object. The state of the second identified object can be the open or closed state of the hands of the second identified object identified in the target image frame. By identifying the target image frame and determining the position of the first identified object and the state of the second identified object, the cropping position and cropping size of the target image frame can be determined, providing a reliable data basis for achieving the effect of simulated lens movement.

[0041] S120. Determine a target cropping position based on the first recognized object position and a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping the target image frame, and the target cropping size is a cropping size when cropping the target image frame.

[0042] This solution can crop a target image frame using a cropping frame, where the cropping center position can be the position of the cropping frame's center in the target image frame. This solution can obtain a target cropping position and target cropping size from the target image frame, directionally capture a fixed-size image at the target cropping position, and continue to determine a new cropping position and cropping size from the image frame following the target image frame. Images cropped from temporally adjacent image frames can be continuously presented to simulate camera effects such as position changes and lens movement.

[0043] S130 : Cropping an image from the target image frame according to the target cropping position and the target cropping size, and scaling the image to a preset size.

[0044] After determining the target cropping position and target cropping size, this solution can crop an image of the target cropping size at the target cropping position in the target image frame. Based on the relationship between the target cropping size and the preset size, each cropped image can be reduced to a preset size or enlarged to a preset size. Images cropped from temporally adjacent image frames are adjusted to the preset size and presented continuously to simulate position changes and lens movement, including zooming in and out at a specific position.

[0045] The technical solution of the embodiment of the present disclosure is to identify the target image frame to determine the first identification object position and the second identification object state; determine the target cropping position based on the first identification object position and determine the target cropping size based on the second identification object state, the target cropping position is the cropping center position when the target image frame is cropped, and the target cropping size is the cropping size when the target image frame is cropped; based on the target cropping position and the target cropping size, the image is cropped from the target image frame and scaled to a preset size. By determining the cropping position and cropping size of the image frame and cropping a cropped image that simulates the lens motion effect from the image frame, it is possible to avoid tedious manual lens movement processes, enrich the appeal of the picture, and enhance the user experience. It solves the problems of unsatisfactory lens movement effects due to technical or conditional limitations, or the complicated and time-consuming operation process caused by imitating lens movement through picture editing.

[0046] Figure 2a is a flow chart of another image processing method provided by an embodiment of the present disclosure. This embodiment of the present disclosure further optimizes the above embodiment based on the above embodiment. This embodiment of the present disclosure can be combined with various optional solutions in one or more of the above embodiments. As shown in Figure 2a, the image processing method provided by the embodiment of the present disclosure may include the following steps:

[0047] S210 , identifying a target part position from a target image frame, wherein the target part position is obtained based on key point detection of the target part in the target image frame.

[0048] Among them, this solution can identify the target part position from the target image frame based on a general key point detection model. The target part position can be at least one of the head position, hand position, foot position, hip position and shoulder position of the identified object in the target image frame.

[0049] As an optional but non-limiting implementation method, the identifying the position of the target part from the target image frame may also include but is not limited to the following process: after completing the key point detection of the target part in the previous image frame, predicting the position and confidence of the target part in the target image frame, the previous image frame and the target image frame are two image frames adjacent in time.

[0050] As shown in FIG2b , this solution can use a general key point detection model to perform key point detection on each image frame in sequence according to the time sequence of the image frames. Specifically, this solution can use a key point detection model to detect the target part position of the previous image frame that is temporally adjacent to the target image frame, and predict the confidence of each target part position, and determine the target part position in the image frame that meets the confidence requirements, so as to complete the key point detection of the target part in the previous image frame. Then, the target part position is detected on the target image frame, and the confidence of each target part position is predicted, and the target part position in the target image frame that meets the confidence requirements is determined. This can achieve the determination of the confident target part position in each image frame, providing a reliable data basis for determining the cropping position of each subsequent image frame.

[0051] S220. Determine a first recognition object position based on the target part position, wherein the first recognition object position is determined based on the hand position and body center position of the first recognition object in the target part position, and the body center position is determined based on positions of other parts except the hand position.

[0052] The first recognition object may be an object that appears in the target image frame, and the number of the first recognition objects is at least one. For each first recognition object, the present solution may determine the position of the first recognition object based on the target part position determined in the aforementioned steps. The position of the first recognition object may be determined based on the hand position or body center position of the first recognition object in the target part position. The body center position may be determined based on the target part positions other than the hand position, such as the head position, foot position, hip position, and shoulder position. This provides a reliable data source for the subsequent step of determining the target cropping position in the target image frame.

[0053] As an optional but non-limiting implementation, the first identified object position includes a first reference center position and a second reference center position in the target image frame, the first reference center position being a hand center position determined based on the hand position in the target part position, and the second reference center position being a center position determined based on at least two other part positions in the target part position except the hand position; or

[0054] The first recognition object position includes a second reference center position in the target image frame, where the second reference center position is a center position determined based on at least two other part positions in the target part position except the hand position.

[0055] As shown in FIG2b , if the trusted target part positions detected in the target image frame of this solution include the hand position, the first recognized object position may include the hand center position determined based on the hand position in the target image frame, i.e., the first reference center position. Furthermore, the center position determined based on at least two other target part positions other than the hand position, i.e., the second reference center position.

[0056] In this solution, if the hand position is not included in the confident target part positions detected in the target image frame, the first recognized object position may include a center position determined based on at least two other target part positions other than the hand position, i.e., a second reference center position. This provides a reliable data basis for determining the target cropping position in the target image frame.

[0057] As an optional but non-limiting implementation, the first reference center position includes at least one of the following: a center point position of a line connecting the left and right hands in the target image frame, the center point position of the line connecting the left and right hands in the target image frame being determined based on the hand positions of the left hand and the right hand of the first identified object in the target part position;

[0058] The center point position of the head-hand line in the target image frame is determined based on the first reference position of the first identified object in the target part position and the head position, and the first reference position is the hand position of the left hand or the hand position of the right hand.

[0059] Among them, if this scheme includes the hand position of the left hand and the hand position of the right hand of the first identified object in the target part position in the target image frame, the first reference center position can be the center point position of the line connecting the left and right hands determined by the hand position of the left hand and the right hand of the first identified object.

[0060] In this solution, if the target part positions in the target image frame only include the left hand position or the right hand position of the first identified subject, and the target part positions include the head position of the first identified subject, the first reference center position can be the center point position of the head-hand line determined by the left hand position and the head position of the first identified subject. Alternatively, the first reference center position can be the center point position of the head-hand line determined by the right hand position and the head position of the first identified subject. This can provide a reliable data basis for determining the target cropping position in the target image frame.

[0061] As an optional but non-limiting implementation, before determining the first identification object position based on the target part position, the following steps may also be performed:

[0062] Step A1: Determine the hand position difference between the target image frame and the second reference image frame, wherein the hand position difference is determined based on the difference between the hand position in the target image frame and the hand position in the second reference image frame, and the second reference image frame is an adjacent image frame that is temporally located before the target image frame.

[0063] Among them, this scheme can respectively determine the hand position in the target image frame and the hand position in the adjacent image frame that is temporally preceding the target image frame, and then determine the difference between the hand position in the target image frame and the hand position in the second reference image frame, and use the ratio of the difference to the interval playback time between the two adjacent image frames in time as the hand position difference between the target image frame and the second reference image frame.

[0064] Step A2: adjusting the hand position of the first recognized object in the target part position based on the hand position difference.

[0065] This solution can adaptively adjust the hand position of the first identified subject in the target part position in the target image frame based on the hand position difference, so that the hand position of the first identified subject in two adjacent image frames is close. This can achieve a smooth transition of the first identified subject's movements in the cropped images of two adjacent image frames, avoiding jitter.

[0066] As an optional but non-limiting implementation, adjusting the hand position of the first identified object in the target part position based on the hand position difference may also include but is not limited to steps B1-B2:

[0067] Step B1: If the hand position difference is greater than a preset position difference threshold, the hand position of the first identified object in the target part position is adjusted by using a weighted average of the hand position in the target image frame and the hand position in the second reference image frame.

[0068] Among them, assuming that the hand position in the target image frame is (x1, y1), and the hand position in the second reference image frame is (x2, y2), if the hand position difference is greater than the preset position difference threshold, it means that the distance between the hand positions in the two adjacent image frames is too large, and there is a high possibility of jitter when outputting the video. This scheme can use the weighted mean of the hand position in the target image frame and the hand position in the second reference image frame ((x1+x2) / 2, (y1+y2) / 2) to adjust the hand position of the first identified object in the target part position.

[0069] Step B2: If the hand position difference is less than or equal to a preset position difference threshold, the hand position of the first identified object in the target part position is not adjusted.

[0070] If the hand position difference is less than or equal to the preset position difference threshold, it indicates that the hand positions in the two adjacent image frames are relatively close, and the likelihood of jitter in the output video is low. Therefore, there is no need to adjust the hand position of the first identified object in the target part position. This allows for a smooth transition of the first identified object's movements in the cropped images of the two adjacent image frames, avoiding jitter.

[0071] S230: Determine a second recognition object state based on the target part position, wherein the second recognition object state is determined based on the hand positions of the left and right hands of the second recognition object in the target part position.

[0072] The second recognized object may be an object appearing in the target image frame, and there may be at least one second recognized object. For each second recognized object, this solution can determine the second recognized object state based on the hand position of the second recognized object's left hand and the hand position of its right hand in the target part position determined in the aforementioned step. This provides a reliable data source for the subsequent step of determining the target cropping size in the target image frame.

[0073] As an optional but non-limiting implementation, the determining of the second recognition object state based on the target part position may also include but is not limited to steps C1-C2:

[0074] Step C1: determining a second reference position of a second identification object from the target part position, wherein the second reference position includes a left hand position and a right hand position.

[0075] Among them, this solution can determine the hand position of the left hand and the hand position of the right hand of the second recognition object, that is, the second reference position, from the target part position of the target image frame.

[0076] Step C2: determining a second recognition object state according to the second reference position, wherein the second recognition object state is determined based on a left-right hand distance between a left hand position and a right hand position of the second recognition object.

[0077] This solution can determine the left-right hand distance between the left and right hands of the second recognition object based on the hand positions of the left and right hands, and then determine the state of the second recognition object based on the left-right hand distance. This can provide a reliable data basis for determining the target cropping size of the target image frame.

[0078] S240: Determine a target cropping position based on the first recognized object position and determine a target cropping size based on the second recognized object state.

[0079] As an optional but non-limiting implementation, determining the target cropping position based on the first recognized object position may also include but is not limited to steps D1-D2:

[0080] Step D1: Determine the relative displacement between the first reference center position and the second reference center position in the first recognition object position.

[0081] Among them, this solution can determine the relative displacement based on the first reference center position and the second reference center position in the first identification object position, including the distance and direction of the offset of the first reference center position relative to the second reference center position.

[0082] Step D2: determining a target cropping position corresponding to the target image frame according to the relative displacement.

[0083] The present solution can determine the target cropping position corresponding to the target image frame based on the relative displacement, thereby providing a reliable data basis for determining the cropping position in the target image frame.

[0084] As an optional but non-limiting implementation, determining the target cropping position corresponding to the target image frame based on the relative displacement may also include but is not limited to steps E1-E2:

[0085] Step E1: If it is detected that the relative displacement is greater than a preset displacement threshold, a target cropping position corresponding to the target image frame is determined according to a reference cropping position of the target image frame and the relative displacement.

[0086] The preset displacement threshold can be adaptively determined based on the size of the first identified object. In this solution, if the relative displacement detected is greater than the preset displacement threshold, the target cropping position corresponding to the target image frame is determined based on the reference cropping position of the target image frame and the relative displacement. For example, a movement distance can be determined based on the relative displacement, and then the target cropping position corresponding to the target image frame is obtained by moving the reference cropping position of the target image frame in the direction of the relative displacement by the movement distance.

[0087] The reference cropping position of the target image frame includes at least one of the center point of the line connecting the left and right hand positions in the first reference image frame, the center point of the line connecting the head and hand positions in the first reference image frame, and the cropping center position used when cropping the image from the second reference image frame. The first reference image frame is an image frame that is temporally preceding the target image frame, and the second reference image frame is an adjacent image frame that is temporally preceding the target image frame. The cropping position of the target image frame can be determined based on the cropping position of the image frame preceding the target image frame, thereby achieving a smooth transition of the movement of the first identified object in the cropped images of the two adjacent image frames to avoid jitter.

[0088] Step E2: If it is detected that the relative displacement is less than or equal to a preset displacement threshold, the reference cropping position of the target image frame is determined as the target cropping position corresponding to the target image frame.

[0089] Among them, the reference cropping position of the target image frame includes at least one of the following: the center point position of the line connecting the left and right hand positions in the first reference image frame, the center point position of the line connecting the head position and the hand position in the first reference image frame, and the cropping center position used when cropping the image from the second reference image frame, the first reference image frame is an image frame that is temporally located before the target image frame, and the second reference image frame is an adjacent image frame that is temporally located before the target image frame.

[0090] Specifically, if the relative displacement detected in this solution is less than or equal to a preset displacement threshold, the reference cropping position of the target image frame can be used as the target cropping position corresponding to the target image frame. This can achieve a smooth transition of the movement of the first identified object in the cropped images of two adjacent image frames, avoiding jitter.

[0091] As an optional but non-limiting implementation, the determining of the target cropping position based on the first recognized object position may also include but is not limited to the following process: determining the second reference center position as the target cropping position corresponding to the target image frame.

[0092] If the target part position of the first identified object in the target image frame is determined not to include a reliable hand position, the second reference center position can be determined as the target cropping position corresponding to the target image frame. This provides a reliable data basis for achieving the effect of simulated lens movement.

[0093] As an optional but non-limiting implementation, determining the target cropping size based on the second recognized object state may also include but is not limited to steps F1-F2:

[0094] Step F1: If the left and right hand distance indicated by the second recognition object state is less than a preset distance threshold, the cropping size to be used when cropping the image in the target image frame is adjusted from the first cropping size to the second cropping size, and the first cropping size is larger than the second cropping size.

[0095] Among them, this scheme can determine the left and right hand distance of the second identification object state indication based on the position difference between the hand position of the left hand and the hand position of the right hand of the second identification object in the target part position determined in the aforementioned steps. If this scheme determines that the left and right hand distance is less than the preset distance threshold, the cropping size to be used when cropping the image in the target image frame is adjusted from the first cropping size to the second cropping size. The first cropping size can be the cropping size used for the previous image frame that is temporally adjacent to the target image frame. The first cropping size should be larger than the second cropping size. The difference between the first cropping size and the second cropping size is the preset cropping step size. Progressive zooming of the lens can be achieved to avoid jitter in the camera movement video.

[0096] Step F2: If the left and right hand distance indicated by the second recognition object state is greater than or equal to a preset distance threshold, the cropping size to be used when cropping the image in the image frame is adjusted from the first cropping size to a third cropping size, and the third cropping size is larger than the first cropping size.

[0097] If the distance between the left and right hands is determined to be greater than or equal to a preset distance threshold, the crop size used when cropping the target image frame is adjusted from the first crop size to a third crop size. The third crop size should be larger than the first crop size, and the difference between the third crop size and the first crop size is the preset cropping step size. This allows for progressive zooming of the camera, preventing jitter in camera-moving videos.

[0098] As an optional but non-limiting implementation, when cropping two temporally adjacent image frames, the cropping size adjustment ratio is set to a preset ratio value, and the preset ratio value determines the scaling speed of the cropped images corresponding to the two adjacent image frames when they are displayed continuously.

[0099] This solution can set the ratio of the cropped sizes of two temporally adjacent image frames to a preset ratio value. The preset ratio value can determine the speed of scaling when the cropped images corresponding to the two adjacent image frames are displayed continuously. This can also be used to adjust the camera movement effect when the image frames are played sequentially.

[0100] S250 : Cropping an image from the target image frame according to the target cropping position and the target cropping size, and scaling the image to a preset size.

[0101] As an optional but non-limiting implementation method, after the image is cropped from the target image frame and scaled to a preset size, it may also include but is not limited to the following process: generating a target image sequence based on the images cropped from different image frames, and the images cropped from different image frames in the target image sequence have the same size when presented continuously.

[0102] Specifically, as shown in Figure 2b, this solution can form a target image sequence by cropping images from different image frames in chronological order. Each image in the target image sequence should be the same size when presented consecutively, which can be a preset size or other size. This allows for the generation of a video with a camera movement effect based on the cropped images from each image frame.

[0103] The technical solution of the embodiment of the present disclosure is to identify the position of a target part from a target image frame, the target part position is obtained based on key point detection of the target part in the target image frame; determine the position of a first recognition object based on the target part position, the first recognition object position is determined based on the hand position and body center position of the first recognition object in the target part position, and the body center position is determined based on the positions of other parts except the hand position; determine the state of a second recognition object based on the target part position, the second recognition object state is determined based on the hand positions of the left and right hands of the second recognition object in the target part position; determine the target cropping position based on the first recognition object position and determine the target cropping size based on the second recognition object state, the target cropping position is the cropping center position when the target image frame is cropped, and the target cropping size is the cropping size when the target image frame is cropped; based on the target cropping position and the target cropping size, crop an image from the target image frame and scale it to a preset size. By determining the cropping position and cropping size of the image frame and cropping a cropped image that simulates the lens motion effect from the image frame, it is possible to avoid the tedious manual camera movement process, enrich the image appeal, and enhance the user experience. It solves the problem of unsatisfactory camera movement due to technical or conditional limitations, or the problem of complex and time-consuming operation procedures caused by imitating camera movement through image editing.

[0104] FIG3 is a flow chart of another image processing method provided by an embodiment of the present disclosure. The present embodiment further optimizes the above embodiment based on the above embodiment. The present embodiment can be combined with various optional solutions in one or more of the above embodiments. As shown in FIG3, the image processing method provided in the embodiment of the present disclosure may include the following steps:

[0105] S310 , determining a first recognition object position and a second recognition object state by recognizing a target image frame.

[0106] S320: Determine a target cropping position based on the first recognized object position and determine a target cropping size based on the second recognized object state.

[0107] S330 , determining whether the cropping area in the target image frame exceeds the boundary of the target image frame, if so, executing S340 ; if not, executing S350 .

[0108] The cropping area in the target image frame is determined based on the target cropping position and the target cropping size.

[0109] As shown in FIG2b , this solution can determine the cropping region of the target image frame based on the target cropping position and the target cropping size, and then determine whether the cropping region of the target image frame exceeds the target image frame boundary based on the position coordinates of the cropping region and the position coordinates of the target image frame boundary. Alternatively, the solution can determine whether the cropping region of the target image frame exceeds the target image frame boundary based on whether the area of ​​the intersection of the cropping region and the target image frame is consistent with the area of ​​the cropping region.

[0110] S340. Adjust the target cropping position on the target image frame, and crop an image from the target image frame according to the target cropping position and the target cropping size at the end of the adjustment. At the end of the adjustment, the cropping area corresponding to the image cropped according to the target cropping position is changed to no longer exceed the boundary of the target image frame.

[0111] Among them, if this scheme determines that the cropping area of ​​the target image frame exceeds the boundary of the target image frame, it means that black edges will be generated after cropping the target image frame, and the value exceeding the boundary of the target image frame needs to be corrected. The target cropping position on the target image frame can be adjusted, for example, by limiting the x and y in the cropping frame (x, y, w, h) determined by the target cropping position and the target cropping size, while keeping w and h unchanged. The target cropping position and target cropping size at the end of the adjustment are obtained, and the image is cropped from the target image frame based on the target cropping position and the target cropping size at the end of the adjustment. The area of ​​the intersection of the cropping area corresponding to the cropped image and the target image frame is exactly the area of ​​the cropping area, that is, the cropping area corresponding to the image cropped according to the target cropping position at the end of the adjustment is changed to no longer exceed the boundary of the target image frame. This can ensure the integrity of the cropped image and provide a reliable data source for the presentation of the camera movement video.

[0112] S350: Cropping an image from the target image frame according to the target cropping position and the target cropping size, and scaling the image to a preset size.

[0113] If the cropping region of the target image frame is determined to not exceed the target image frame boundary, this solution can directly crop an image from the target image frame according to the target cropping position and target cropping size, and then scale the cropped image to a preset size. This provides a reliable data foundation for achieving a simulated camera movement effect when cropping images from temporally adjacent image frames are presented sequentially.

[0114] As an optional but non-limiting implementation, before cropping the image from the target image frame according to the target cropping position and the target cropping size, the steps may also include but are not limited to steps G1-G2:

[0115] Step G1: determining the facial integrity of the cropping area of ​​the target image frame according to the target cropping position, wherein the facial integrity is used to indicate the proportion of the face located in the cropping area relative to the complete face.

[0116] As shown in Figure 2b, before cropping the target image frame based on the target cropping position and target cropping size, this solution also needs to determine the proportion of the face in the cropping area determined by the target cropping position and target cropping size relative to the complete face, namely the facial integrity. This provides a reliable data basis for ensuring that the face falls within the cropping area as much as possible.

[0117] Step G2: If the facial integrity is less than a preset integrity threshold, adjusting the target cropping position on the target image frame so that the facial integrity is no longer less than the preset integrity threshold.

[0118] If this solution determines that the facial integrity is less than a preset integrity threshold, it indicates that the facial area falling within the cropping area is incomplete, and facial cropping will produce an unpleasant visual effect, affecting the presentation of the camera movement effect. The target cropping position on the target image frame can be adaptively adjusted to ensure that the facial integrity is no longer less than the preset integrity threshold, ensuring that the face is in the frame. This provides a reliable data foundation for ensuring the presentation of the camera movement effect.

[0119] As an optional but non-limiting implementation, before cropping the image from the target image frame according to the target cropping position and the target cropping size, the steps may also include but are not limited to steps H1-H2:

[0120] Step H1: Determine the cropping position difference between the target image frame and the second reference image frame, wherein the cropping position difference is determined based on the difference between the cropping center position when the target image frame is cropped and the cropping center position when the second reference image frame is cropped, and the second reference image frame is an adjacent image frame that is temporally located before the target image frame.

[0121] As shown in Figure 2b, this scheme can determine the cropping center position in the target image frame and the cropping center position in the adjacent image frame (i.e., the second reference image frame) that is temporally preceding the target image frame, and determine the cropping position difference between the target image frame and the second reference image frame based on the cropping position of the adjacent image frame.

[0122] Step H2: adjusting the cropping center position used when cropping the target image frame based on the cropping position difference.

[0123] Specifically, if this solution determines that the cropping position difference exceeds the preset movement displacement, it means that the cropping center positions of adjacent image frames differ too much, which will cause the final camera movement video to jitter. The cropping center position when the second reference image frame is cropped and the cropping center position when the target image frame is cropped can be weighted averaged to obtain a new target image frame cropping center position. Based on the new cropping center position, the cropping center position when the target image frame is cropped can be adjusted to slow down the movement speed of the cropping frame, thereby avoiding jitter in the camera movement video. If this solution determines that the cropping position difference does not exceed the preset movement displacement, there is no need to adjust the cropping center position.

[0124] The technical solution of the embodiment of the present disclosure is to identify the target image frame to determine the first identified object position and the second identified object state; determine the target cropping position based on the first identified object position and determine the target cropping size based on the second identified object state, the target cropping position being the cropping center position when the target image frame is cropped, and the target cropping size being the cropping size when the target image frame is cropped; determine whether the cropping area in the target image frame exceeds the target image frame boundary, the cropping area in the target image frame is determined based on the target cropping position and the target cropping size; if it exceeds the target image frame boundary, adjust the target cropping position on the target image frame, and crop the image from the target image frame according to the target cropping position and the target cropping size at the end of the adjustment, and at the end of the adjustment, the cropping area corresponding to the image cropped according to the target cropping position is changed to no longer exceed the target image frame boundary; if it does not exceed the target image frame boundary, crop the image from the target image frame according to the target cropping position and the target cropping size and scale it to a preset size. By determining the cropping position and size of the image frame and cropping it to create a cropped image that simulates camera movement, we can avoid the tedious manual camera movement process, enrich the visual appeal, and enhance the user experience. This solves the problem of unsatisfactory camera movement due to technical or conditional limitations, or the complex and time-consuming process of simulating camera movement through image editing.

[0125] FIG4 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. As shown in FIG4 , the device includes:

[0126] The image frame recognition module 410 is configured to determine a first recognition object position and a second recognition object state by recognizing a target image frame;

[0127] a cropping position and cropping size determining module 420, configured to determine a target cropping position based on the first recognized object position and a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping the target image frame, and the target cropping size is a cropping size when cropping the target image frame;

[0128] The image cropping module 430 is configured to crop an image from the target image frame and scale the image to a preset size according to the target cropping position and the target cropping size.

[0129] In an optional scheme of an embodiment of the present disclosure, optionally, the device further includes a target image sequence generation module, which is used to generate a target image sequence based on images cropped from different image frames after the image is cropped from the target image frame and scaled to a preset size, wherein the images cropped from different image frames in the target image sequence have the same size when presented continuously.

[0130] In an optional scheme of the embodiment of the present disclosure, optionally, the image frame recognition module 410 includes a target part position recognition unit, which is used to identify the target part position from the target image frame, and the target part position is obtained based on key point detection of the target part in the target image frame; a first recognition object position determination unit, which is used to determine the first recognition object position based on the target part position, and the first recognition object position is determined based on the hand position and the body center position of the first recognition object in the target part position, and the body center position is determined based on the positions of other parts except the hand position; a second recognition object state determination unit, which is used to determine the second recognition object state based on the target part position, and the second recognition object state is determined based on the hand positions of the left and right hands of the second recognition object in the target part position.

[0131] In an optional scheme of an embodiment of the present disclosure, the target part position identification unit is optionally used to predict the position and confidence of the target part in the target image frame after completing the key point detection of the target part in the previous image frame, and the previous image frame and the target image frame are two image frames adjacent in time.

[0132] In an optional scheme of an embodiment of the present disclosure, optionally, the first recognized object position includes a first reference center position and a second reference center position in the target image frame, the first reference center position is the hand center position determined based on the hand position in the target part position, and the second reference center position is the center position determined based on at least two other part positions in the target part position except the hand position; or, the first recognized object position includes the second reference center position in the target image frame, and the second reference center position is the center position determined based on at least two other part positions in the target part position except the hand position.

[0133] In an optional scheme of an embodiment of the present disclosure, optionally, the first reference center position includes at least one of the following: the center point position of the line connecting the left and right hands in the target image frame, which is determined based on the left hand position and the right hand position of the first identified object in the target part position; the center point position of the head-hand line in the target image frame, which is determined based on the first reference position and the head position of the first identified object in the target part position, and the first reference position is the hand position of the left hand or the hand position of the right hand.

[0134] In an optional scheme of an embodiment of the present disclosure, optionally, the cropping position and cropping size determination module 420 includes a relative displacement determination unit for determining the relative displacement between the first reference center position and the second reference center position in the first identified object position; and a target cropping position determination unit for determining the target cropping position corresponding to the target image frame based on the relative displacement.

[0135] In an optional scheme of an embodiment of the present disclosure, optionally, a target cropping position determination unit is specifically used to determine the target cropping position corresponding to the target image frame based on the reference cropping position of the target image frame and the relative displacement if it is detected that the relative displacement is greater than a preset displacement threshold; if it is detected that the relative displacement is less than or equal to the preset displacement threshold, the reference cropping position of the target image frame is determined as the target cropping position corresponding to the target image frame; wherein, the reference cropping position of the target image frame includes at least one of the following: the center point position of the line connecting the left and right hand positions in the first reference image frame, the center point position of the line connecting the head position and the hand position in the first reference image frame, and the cropping center position used when cropping the image from the second reference image frame, the first reference image frame is an image frame that is temporally located before the target image frame, and the second reference image frame is an adjacent image frame that is temporally located before the target image frame.

[0136] In an optional solution of the embodiment of the present disclosure, optionally, the cropping position and cropping size determination module 420 is specifically configured to determine the second reference center position as the target cropping position corresponding to the target image frame.

[0137] In an optional scheme of the embodiment of the present disclosure, optionally, the device further includes a hand position difference determination unit for determining the hand position difference between the target image frame and the second reference image frame before determining the position of the first identified object based on the target part position, the hand position difference being determined based on the difference between the hand position in the target image frame and the hand position in the second reference image frame, the second reference image frame being an adjacent image frame temporally preceding the target image frame; and a hand position adjustment unit for adjusting the hand position of the first identified object in the target part position based on the hand position difference.

[0138] In an optional scheme of the embodiment of the present disclosure, the hand position adjustment unit is optionally used to adjust the hand position of the first identified object in the target part position by using the weighted average of the hand position in the target image frame and the hand position in the second reference image frame if the hand position difference is greater than a preset position difference threshold; if the hand position difference is less than or equal to the preset position difference threshold, the hand position of the first identified object in the target part position is not adjusted.

[0139] In an optional scheme of an embodiment of the present disclosure, optionally, the second identification object state determination unit includes a second reference position determination subunit, which is used to determine the second reference position of the second identification object from the target part position, and the second reference position includes the hand position of the left hand and the hand position of the right hand; the second identification object state determination subunit is used to determine the second identification object state based on the second reference position, and the second identification object state is determined based on the left-right hand distance between the hand position of the left hand and the hand position of the right hand of the second identification object.

[0140] In an optional scheme of an embodiment of the present disclosure, optionally, the cropping position and cropping size determination module 420 is specifically used to adjust the cropping size to be used when cropping the image in the target image frame from the first cropping size to the second cropping size if the left-hand and right-hand distance indicated by the second recognition object state is less than a preset distance threshold, and the first cropping size is larger than the second cropping size; if the left-hand and right-hand distance indicated by the second recognition object state is greater than or equal to the preset distance threshold, then the cropping size to be used when cropping the image in the image frame is adjusted from the first cropping size to the third cropping size, and the third cropping size is larger than the first cropping size.

[0141] In an optional solution of the embodiment of the present disclosure, optionally, when cropping two temporally adjacent image frames, a cropping size adjustment ratio is used as a preset ratio value, and the preset ratio value determines the scaling speed when the cropped images corresponding to the two adjacent image frames are continuously displayed.

[0142] In an optional solution of the embodiment of the present disclosure, optionally, the image cropping module 430 includes a judgment unit, configured to determine whether a cropping area in the target image frame exceeds a boundary of the target image frame, the cropping area in the target image frame being determined based on the target cropping position and the target cropping size;

[0143] a target cropping position adjustment unit, configured to adjust the target cropping position on the target image frame if the target cropping position exceeds the target image frame boundary, and crop an image from the target image frame according to the target cropping position and the target cropping size at the end of the adjustment, wherein the cropping area corresponding to the image cropped according to the target cropping position is changed to no longer exceed the target image frame boundary; and an image cropping unit, configured to crop an image from the target image frame according to the target cropping position and the target cropping size if the target image frame boundary is not exceeded.

[0144] In an optional scheme of the embodiment of the present disclosure, optionally, the device also includes a facial integrity determination module, which is used to determine the facial integrity of the cropping area of ​​the target image frame according to the target cropping position before cropping the image from the target image frame based on the target cropping position and the target cropping size, wherein the facial integrity is used to indicate the proportion of the face located in the cropping area relative to the complete face; if the facial integrity is less than a preset integrity threshold, the target cropping position on the target image frame is adjusted so that the facial integrity is no longer less than the preset integrity threshold.

[0145] In an optional scheme of the embodiment of the present disclosure, optionally, the device also includes a cropping center position adjustment module, which is used to determine the cropping position difference between the target image frame and the second reference image frame before cropping the image from the target image frame based on the target cropping position and the target cropping size, the cropping position difference being determined based on the difference between the cropping center position when the target image frame is cropped and the cropping center position when the second reference image frame is cropped, the second reference image frame being an adjacent image frame that is temporally located before the target image frame; the cropping center position used when cropping the target image frame is adjusted based on the cropping position difference.

[0146] The image processing device provided by the embodiments of the present disclosure can execute the image processing method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0147] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0148] FIG5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Referring to FIG5 , a schematic diagram of the structure of an electronic device (such as a terminal device or server in FIG5 ) 500 suitable for implementing an embodiment of the present disclosure is shown below. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG5 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0149] As shown in FIG5 , the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0150] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although FIG5 shows the electronic device 500 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0151] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0152] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0153] The electronic device provided by the embodiment of the present disclosure and the image processing method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0154] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the image processing method provided by the above embodiment is implemented.

[0155] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0156] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0157] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0158] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: determines the first recognition object position and the second recognition object state by recognizing the target image frame;

[0159] determining a target cropping position based on the first recognized object position and determining a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping the target image frame, and the target cropping size is a cropping size when cropping the target image frame;

[0160] Based on the target cropping position and the target cropping size, an image is cropped from the target image frame and scaled to a preset size.

[0161] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0164] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0165] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0166] According to one or more embodiments of the present disclosure, Example 1 provides an image processing method, the method including:

[0167] By identifying the target image frame, determining the first identification object position and the second identification object state;

[0168] determining a target cropping position based on the first recognized object position and determining a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping the target image frame, and the target cropping size is a cropping size when cropping the target image frame;

[0169] Based on the target cropping position and the target cropping size, an image is cropped from the target image frame and scaled to a preset size.

[0170] Example 2: The method according to Example 1, after cropping the image from the target image frame and scaling it to a preset size, further comprising:

[0171] A target image sequence is generated based on images cropped from different image frames, wherein the images cropped from different image frames in the target image sequence have the same size when presented continuously.

[0172] Example 3: According to the method of Example 1, determining the first recognition object position and the second recognition object state by recognizing the target image frame includes:

[0173] Identifying a target part position from the target image frame, wherein the target part position is obtained based on key point detection of the target part in the target image frame;

[0174] Determining a first recognition object position based on the target part position, wherein the first recognition object position is determined based on a hand position and a body center position of the first recognition object in the target part position, and the body center position is determined based on positions of other parts except the hand position;

[0175] A second recognition object state is determined based on the target part position, and the second recognition object state is determined based on the hand positions of the left and right hands of the second recognition object in the target part position.

[0176] Example 4: According to the method of Example 3, identifying the target part position from the target image frame includes:

[0177] After completing key point detection of the target part in the previous image frame, the position and confidence of the target part in the target image frame are predicted, and the previous image frame and the target image frame are two image frames adjacent in time.

[0178] Example 5: According to the method described in Example 3, the first identified object position includes a first reference center position and a second reference center position in the target image frame, the first reference center position is a hand center position determined based on the hand position in the target part position, and the second reference center position is a center position determined based on at least two other part positions in the target part position except the hand position; or

[0179] The first recognition object position includes a second reference center position in the target image frame, where the second reference center position is a center position determined based on at least two other part positions in the target part position except the hand position.

[0180] Example 6: According to the method of Example 5, the first reference center position includes at least one of the following:

[0181] The center point position of the line connecting the left and right hands in the target image frame is determined based on the hand position of the left hand and the hand position of the right hand of the first recognition object in the target part position;

[0182] The center point position of the head-hand line in the target image frame is determined based on the first reference position of the first identified object in the target part position and the head position, and the first reference position is the hand position of the left hand or the hand position of the right hand.

[0183] Example 7: According to the method of Example 5, determining the target cropping position based on the first recognized object position includes:

[0184] determining a relative displacement between the first reference center position and the second reference center position in the first identification object position;

[0185] A target cropping position corresponding to the target image frame is determined according to the relative displacement.

[0186] Example 8: According to the method of Example 7, determining the target cropping position corresponding to the target image frame based on the relative displacement includes:

[0187] If it is detected that the relative displacement is greater than a preset displacement threshold, determining a target cropping position corresponding to the target image frame according to the reference cropping position of the target image frame and the relative displacement;

[0188] If it is detected that the relative displacement is less than or equal to a preset displacement threshold, determining the reference cropping position of the target image frame as the target cropping position corresponding to the target image frame;

[0189] Among them, the reference cropping position of the target image frame includes at least one of the following: the center point position of the line connecting the left and right hand positions in the first reference image frame, the center point position of the line connecting the head position and the hand position in the first reference image frame, and the cropping center position used when cropping the image from the second reference image frame, the first reference image frame is an image frame that is temporally located before the target image frame, and the second reference image frame is an adjacent image frame that is temporally located before the target image frame.

[0190] Example 9: According to the method of Example 5, determining the target cropping position based on the first recognized object position includes:

[0191] The second reference center position is determined as the target cropping position corresponding to the target image frame.

[0192] Example 10: The method according to Example 3, before determining the first identification object position based on the target part position, further comprising:

[0193] determining a hand position difference between a target image frame and a second reference image frame, the hand position difference being determined based on a difference between a hand position in the target image frame and a hand position in a second reference image frame, the second reference image frame being an adjacent image frame temporally preceding the target image frame;

[0194] The hand position of the first recognized object in the target part position is adjusted based on the hand position difference.

[0195] Example 11: According to the method of Example 10, adjusting the hand position of the first recognized object in the target part position based on the hand position difference includes:

[0196] If the hand position difference is greater than a preset position difference threshold, adjusting the hand position of the first identified object in the target part position by using a weighted average of the hand position in the target image frame and the hand position in the second reference image frame;

[0197] If the hand position difference is less than or equal to the preset position difference threshold, the hand position of the first identified object in the target part position is not adjusted.

[0198] Example 12: According to the method of Example 3, determining the second identification object state based on the target part position includes:

[0199] determining a second reference position of a second identification object from the target part position, the second reference position comprising a hand position of a left hand and a hand position of a right hand;

[0200] A second recognition object state is determined according to the second reference position, where the second recognition object state is determined based on a left-right hand distance between a left hand position and a right hand position of the second recognition object.

[0201] Example 13: According to the method of Example 12, determining the target cropping size based on the second recognized object state includes:

[0202] If the left-hand and right-hand distance indicated by the second recognition object state is less than a preset distance threshold, adjusting a cropping size to be used when cropping an image in the target image frame from a first cropping size to a second cropping size, wherein the first cropping size is larger than the second cropping size;

[0203] If the left and right hand distance indicated by the second recognition object state is greater than or equal to a preset distance threshold, the cropping size to be used when cropping the image in the image frame is adjusted from the first cropping size to a third cropping size, and the third cropping size is larger than the first cropping size.

[0204] Example 14: According to the method described in Example 13, when cropping two temporally adjacent image frames, the cropping size adjustment ratio is set to a preset ratio value, and the preset ratio value determines the scaling speed when the cropped images corresponding to the two adjacent image frames are continuously displayed.

[0205] Example 15: According to the method of Example 1, cropping an image from the target image frame according to the target cropping position and the target cropping size includes:

[0206] determining whether a cropping region in the target image frame exceeds a boundary of the target image frame, the cropping region in the target image frame being determined based on the target cropping position and the target cropping size;

[0207] If the target cropping position exceeds the target image frame boundary, adjusting the target cropping position on the target image frame, cropping an image from the target image frame according to the target cropping position and the target cropping size at the end of the adjustment, and changing the cropping area corresponding to the image cropped according to the target cropping position at the end of the adjustment to no longer exceed the target image frame boundary;

[0208] If the target image frame boundary is not exceeded, an image is cropped from the target image frame according to the target cropping position and the target cropping size.

[0209] Example 16: The method according to Example 1 or Example 15, before cropping the image from the target image frame according to the target cropping position and the target cropping size, further comprising:

[0210] determining a facial integrity of a cropping region of the target image frame according to the target cropping position, wherein the facial integrity is used to indicate a ratio of a face located in the cropping region to a complete face;

[0211] If the facial integrity is less than a preset integrity threshold, the target cropping position on the target image frame is adjusted so that the facial integrity is no longer less than the preset integrity threshold.

[0212] Example 17: The method according to Example 1 or Example 15, before cropping the image from the target image frame according to the target cropping position and the target cropping size, further comprising:

[0213] determining a cropping position difference between a target image frame and a second reference image frame, the cropping position difference being determined based on a difference between a cropping center position when the target image frame is cropped and a cropping center position when the second reference image frame is cropped, the second reference image frame being an adjacent image frame temporally preceding the target image frame;

[0214] The cropping center position used when cropping the target image frame is adjusted based on the cropping position difference.

[0215] According to one or more embodiments of the present disclosure, Example 18 further provides an image processing device, the image processing device including:

[0216] An image frame recognition module, configured to determine a first recognition object position and a second recognition object state by recognizing a target image frame;

[0217] a cropping position and cropping size determination module, configured to determine a target cropping position based on the first recognized object position and a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping a target image frame, and the target cropping size is a cropping size when cropping the target image frame;

[0218] The image cropping module is used to crop an image from the target image frame and scale it to a preset size based on the target cropping position and the target cropping size.

[0219] According to one or more embodiments of the present disclosure, Example 19 further provides an electronic device, the electronic device including:

[0220] one or more processors;

[0221] a storage device for storing one or more programs,

[0222] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of Examples 1-17.

[0223] According to one or more embodiments of the present disclosure, Example 20 further provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the image processing method described in any one of Examples 1-17.

[0224] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0225] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0226] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An image processing method, comprising: By identifying the target image frame, determining the position of the first identified object and the state of the second identified object; Determining a target cropping position based on the first recognized object position and determining a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping the target image frame, and the target cropping size is a cropping size when cropping the target image frame; According to the target cropping position and the target cropping size, an image is cropped from the target image frame and scaled to a preset size.

2. The method according to claim 1, wherein: After cropping the image from the target image frame and scaling it to a preset size, the method further includes: A target image sequence is generated based on images cropped from different image frames, wherein the images cropped from different image frames in the target image sequence have the same size when presented continuously.

3. The method according to claim 1, wherein: The step of identifying the target image frame to determine the first identified object position and the second identified object state includes: Identifying a target part position from the target image frame, wherein the target part position is obtained based on key point detection of the target part in the target image frame; Determine a first recognition object position based on the target part position, the first recognition object position is determined based on a hand position and a body center position of the first recognition object in the target part position, and the body center position is determined based on positions of other parts except the hand position; A second recognition object state is determined based on the target part position, and the second recognition object state is determined based on the hand positions of the left and right hands of the second recognition object in the target part position.

4. The method according to claim 3, wherein: The identifying the target part position from the target image frame comprises: After completing key point detection of the target part in the previous image frame, the position and confidence of the target part in the target image frame are predicted, and the previous image frame and the target image frame are two image frames adjacent in time.

5. The method according to claim 3, wherein: The first identified object position includes a first reference center position and a second reference center position in the target image frame, the first reference center position being a hand center position determined based on the hand position in the target part position, and the second reference center position being a center position determined based on at least two other part positions in the target part position except the hand position; or The first recognized object position includes a second reference center position in the target image frame, and the second reference center position is a center position determined based on at least two other part positions in the target part position except the hand position.

6. The method according to claim 5, wherein: The first reference center position includes at least one of the following: The center point position of the line connecting the left and right hands in the target image frame is determined based on the hand position of the left hand and the hand position of the right hand of the first recognition object in the target part position; The center point position of the head-hand line in the target image frame is determined based on the first reference position of the first identified object in the target part position and the head position, and the first reference position is the hand position of the left hand or the hand position of the right hand.

7. The method according to claim 5, wherein: The determining of the target cropping position based on the first recognized object position comprises: determining a relative displacement between the first reference center position and the second reference center position in the first identified object position; A target cropping position corresponding to the target image frame is determined according to the relative displacement.

8. The method according to claim 7, wherein: The step of determining the target cropping position corresponding to the target image frame according to the relative displacement includes: If it is detected that the relative displacement is greater than a preset displacement threshold, determining a target cropping position corresponding to the target image frame according to the reference cropping position of the target image frame and the relative displacement; If it is detected that the relative displacement is less than or equal to a preset displacement threshold, determining the reference cropping position of the target image frame as the target cropping position corresponding to the target image frame; Among them, the reference cropping position of the target image frame includes at least one of the following: the center point position of the line connecting the left and right hand positions in the first reference image frame, the center point position of the line connecting the head position and the hand position in the first reference image frame, and the cropping center position used when cropping the image from the second reference image frame, the first reference image frame is an image frame that is temporally located before the target image frame, and the second reference image frame is an adjacent image frame that is temporally located before the target image frame.

9. The method according to claim 5, wherein: The determining of the target cropping position based on the first recognized object position comprises: The second reference center position is determined as the target cropping position corresponding to the target image frame.

10. The method according to claim 3, wherein: Before determining the first identification object position based on the target part position, the method further includes: Determine a hand position difference between a target image frame and a second reference image frame, wherein the hand position difference is determined based on a difference between a hand position in the target image frame and a hand position in a second reference image frame, wherein the second reference image frame is an adjacent image frame that is temporally preceding the target image frame; The hand position of the first recognized object in the target part position is adjusted based on the hand position difference.

11. The method according to claim 10, wherein: The adjusting the hand position of the first recognized object in the target part position based on the hand position difference includes: If the hand position difference is greater than a preset position difference threshold, the hand position of the first identified object in the target part position is adjusted by using a weighted average of the hand position in the target image frame and the hand position in the second reference image frame; If the hand position difference is less than or equal to a preset position difference threshold, the hand position of the first identified object in the target part position is not adjusted.

12. The method according to claim 3, wherein: The determining of the second recognition object state based on the target part position includes: Determining a second reference position of a second recognition object from the target part position, the second reference position comprising a hand position of a left hand and a hand position of a right hand; A second recognition object state is determined according to the second reference position, and the second recognition object state is determined based on a left-right hand distance between a left hand position and a right hand position of the second recognition object.

13. The method according to claim 12, wherein: The determining the target cropping size based on the second recognized object state comprises: If the left-right hand distance indicated by the second recognition object state is less than a preset distance threshold, adjusting the cropping size to be used when cropping the image in the target image frame from the first cropping size to the second cropping size, the first cropping size being larger than the second cropping size; If the left and right hand distance indicated by the second recognition object state is greater than or equal to the preset distance threshold, Then the cropping size to be used when cropping the image in the image frame is adjusted from the first cropping size to a third cropping size, and the third cropping size is larger than the first cropping size.

14. The method according to claim 13, wherein: When cropping two temporally adjacent image frames, the cropping size adjustment ratio is used as a preset ratio value, and the preset ratio value determines the zooming speed when the cropped images corresponding to the two adjacent image frames are displayed continuously.

15. The method according to claim 1, wherein: The step of cropping an image from the target image frame according to the target cropping position and the target cropping size includes: Determining whether a cropping region in the target image frame exceeds a boundary of the target image frame, wherein the cropping region in the target image frame is determined based on the target cropping position and the target cropping size; If it exceeds the boundary of the target image frame, the target cropping position on the target image frame is adjusted, and an image is cropped from the target image frame according to the target cropping position and the target cropping size at the end of the adjustment, and a cropping area corresponding to the image cropped according to the target cropping position at the end of the adjustment is changed to no longer exceed the boundary of the target image frame; If the target image frame boundary is not exceeded, an image is cropped from the target image frame according to the target cropping position and the target cropping size.

16. The method according to claim 1 or 15, wherein: Before cropping an image from the target image frame according to the target cropping position and the target cropping size, the method further includes: Determine the facial integrity of the cropping area of ​​the target image frame according to the target cropping position, wherein the facial integrity is used to indicate the proportion of the face located in the cropping area relative to the complete face; If the facial integrity is less than a preset integrity threshold, the target cropping position on the target image frame is adjusted so that the facial integrity is no longer less than the preset integrity threshold.

17. The method according to claim 1 or 15, wherein: Before cropping an image from the target image frame according to the target cropping position and the target cropping size, the method further includes: Determining a cropping position difference between a target image frame and a second reference image frame, wherein the cropping position difference is determined based on a difference between a cropping center position when the target image frame is cropped and a cropping center position when the second reference image frame is cropped, wherein the second reference image frame is an adjacent image frame that is temporally located before the target image frame; The cropping center position used when cropping the target image frame is determined based on the cropping position difference. Adjustment.

18. An image processing device, comprising: An image frame recognition module, configured to determine a first recognition object position and a second recognition object state by recognizing a target image frame; a cropping position and cropping size determination module, configured to determine a target cropping position based on the first recognized object position and to determine a target cropping size based on the second recognized object state, wherein the target cropping position is a cropping center position when cropping a target image frame, and the target cropping size is a cropping size when cropping a target image frame; The image cropping module is configured to crop an image from the target image frame and scale it to a preset size according to the target cropping position and the target cropping size.

19. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of claims 1-17.

20. A storage medium comprising computer executable instructions, wherein: When the computer executable instructions are executed by a computer processor, they are used to perform the image processing method according to any one of claims 1 to 17.

Citation Information

Patent Citations

  • Image processing method and device, electronic equipment and storage medium

    CN119068380A

  • Video stream clipping method and electronic equipment

    CN111031178A

  • Alarm video generation method, alarm video display method and alarm video generation device

    CN112948627A

  • Camera for generating alarm video

    CN112954267A

  • Object key point detection method and device, training method and device and computing equipment

    CN114332483A

Cited By

  • Shooting game assisting method

    CN121266136A