An image processing method and apparatus

Abstract

An embodiment of the present application discloses an image processing method, and the method includes: obtaining multiple frames of to-be-processed images in a to-be-processed video for a target object; processing the multiple frames of to-be-processed images to determine target parameters of target pixel points of the multiple frames of to-be-processed images; determining whether the moving speed of the target object conforms to a target set speed based on the target parameters of the target pixel points; in the case where the moving speed of the target object conforms to the target set speed, performing a first processing on the pixel points of the to-be-processed images to determine a target area of the target object; and in the case where the moving speed of the target object does not conform to the target set speed, performing a second processing on the pixel points of the to-be-processed images to determine a target area of the target object. An embodiment of the present application also discloses an image processing device.

Description

Technical Field

[0001] This application relates to image processing technology in the field of image processing, and particularly relates to an image processing method and apparatus. Background Art

[0002] With the development of technology, image editing software is widely used; when editing images in beauty, sticker, etc. software, it is necessary to obtain the part of the image to be edited; when obtaining the part of the image to be edited, even if the person remains stationary, due to the presence of ambient noise, it will cause the beauty effects and stickers to jitter. To avoid the above problems, in related technologies, the jitter is eliminated by taking the average value or weighted value of the pixel points of the edited part of the current frame and the historical frame. However, in the related technology, when the face moves rapidly, the problem of ghosting will occur with this jitter elimination scheme. Summary of the Invention

[0003] To solve the above technical problems, embodiments of this application are expected to provide an image processing method and apparatus, which solve the problem that the jitter elimination scheme in related technologies will cause the problem of ghosting when the face moves rapidly.

[0004] To achieve the above object, the technical solution of this application is realized as follows:

[0005] An image processing method, the method includes:

[0006] Obtain multiple frames of images to be processed in a video to be processed for a target object;

[0007] Process the multiple frames of images to be processed to determine the target parameters of the target pixel points of the multiple frames of images to be processed;

[0008] Based on the target parameters of the target pixel points, determine whether the moving speed of the target object meets a target set speed;

[0009] In the case where the moving speed of the target object meets the target set speed, perform a first process on the pixel points of the image to be processed to determine the target area of the target object;

[0010] In the case where the moving speed of the target object does not meet the target set speed, perform a second process on the pixel points of the image to be processed to determine the target area of the target object.

[0011] In the above solution, the process of processing the multiple frames of images to be processed to determine the target parameters of the pixel points of the target area of the multiple frames of images to be processed includes:

[0012] Perform key point detection on the multi-frame images to be processed to obtain the position information of the key points of the target face in the multi-frame images to be processed; wherein, the target pixel points include the pixel points at the key points.

[0013] In the above solution, determining whether the moving speed of the target object meets the target set speed based on the target parameters of the target pixel points includes:

[0014] Obtain the position information of the key points of the images to be processed corresponding to different frames at an interval of the first number of frames and compare them;

[0015] Based on the comparison result, determine whether the area corresponding to the target key points whose position information of the corresponding key points changes meets the reference area;

[0016] When the area corresponding to the target key points meets the reference area, determine that the moving speed of the target face meets the target set speed;

[0017] When the area corresponding to the target key points does not meet the reference area, determine that the moving speed of the target face does not meet the target set speed.

[0018] In the above solution, when the moving speed of the target object meets the target set speed, performing a first processing on the pixel points of the image to be processed to determine the target area of the target object includes:

[0019] When it is determined that the moving speed of the target face meets the target set speed, determine the target area of the target object based on the position information of the first key points of the image to be processed in the current frame;

[0020] Correspondingly, when the moving speed of the target object does not meet the target set speed, performing a second processing on the pixel points of the image to be processed to determine the target area of the target object includes:

[0021] When it is determined that the moving speed of the target face does not meet the target set speed, determine the target area of the target object based on the position information of the first key points of the image to be processed in the current frame and the position information of the second key points of the image to be processed in the frame before the current frame.

[0022] In the above solution, processing the multi-frame images to be processed to determine the target parameters of the target pixel points of the multi-frame images to be processed includes:

[0023] Analyze the multi-frame images to be processed, and determine the probability value that the pixel points of the multi-frame images to be processed belong to the pixel points of the target part of the image to be processed; wherein, the target pixel points include the pixel points of the multi-frame images to be processed; the target part is determined based on the depth of field information of the image to be processed.

[0024] In the above solution, determining whether the moving speed of the target object meets the target set speed based on the target parameters of the pixel points of the target includes:

[0025] Compare the probability value of the first pixel point in the current frame with a corresponding relationship with the probability value of the second pixel point in the frame that is before the current frame and separated by the second number of frames.

[0026] In the case where the number of pixel points whose comparison result indicates a change in the probability value meets the target number threshold, determine that the moving speed of the target object meets the target set speed;

[0027] In the case where the number of pixel points whose comparison result indicates a change in the probability value does not meet the target number threshold, determine that the moving speed of the target object does not meet the target set speed.

[0028] In the above solution, in the case where the moving speed of the target object meets the target set speed, performing a first process on the pixel points of the image to be processed to determine the target area of the target object includes:

[0029] In the case where it is determined that the moving speed of the target object meets the target set speed, determine the target area of the image to be processed based on the probability value of the first pixel point in the current frame;

[0030] Correspondingly, in the case where the moving speed of the target object does not meet the target set speed, performing a second process on the pixel points of the image to be processed to determine the target area of the target object includes:

[0031] In the case where it is determined that the moving speed of the target object does not meet the target set speed, determine the target area of the image to be processed based on the probability value of the first pixel point in the current frame and the probability value of the second pixel point in the frame before the current frame.

[0032] In the above solution, the method further includes:

[0033] In the case where it is determined that the moving speed of the target face meets the target set speed, delete the position information of the second key points of the images to be processed in the frames before the current frame;

[0034] Or, in the case where it is determined that the moving speed of the target object meets the target set speed, delete the probability values of the second pixel points in the frames before the current frame.

[0035] In the above solution, the method further includes:

[0036] Based on the position information of the target area in the image to be processed, process the target object.

[0037] An image processing device, characterized in that the device includes:

[0038] An acquisition unit, configured to acquire multiple frames of images to be processed in a video to be processed for a target object;

[0039] A first determination unit, configured to process the multiple frames of images to be processed and determine target parameters of target pixel points of the multiple frames of images to be processed;

[0040] A second determination unit, configured to determine whether the moving speed of the target object conforms to a target set speed based on the target parameters of the target pixel points;

[0041] A processing unit, configured to, when the moving speed of the target object conforms to the target set speed, perform a first processing on pixel points of the image to be processed to determine a target area of the target object;

[0042] The processing unit is further configured to, when the moving speed of the target object does not conform to the target set speed, perform a second processing on pixel points of the image to be processed to determine a target area of the target object.

[0043] The image processing method and device provided by the embodiments of the present application acquire multiple frames of images to be processed in a video to be processed for a target object; process the multiple frames of images to be processed and determine target parameters of target pixel points of the multiple frames of images to be processed; determine whether the moving speed of the target object conforms to a target set speed based on the target parameters of the target pixel points; when the moving speed of the target object conforms to the target set speed, perform a first processing on pixel points of the image to be processed to determine a target area of the target object; when the moving speed of the target object does not conform to the target set speed, perform a second processing on pixel points of the image to be processed to determine a target area of the target object; in this way, the moving speed of the target object can be determined according to the target parameters obtained by processing multiple frames of images to be processed, and then different processing methods can be selected according to the moving speed of the target object to process pixel points to determine the target area of the target object, solving the problem that in the related art, when the face moves rapidly, there will be afterimages in the solution for eliminating jitter, thereby achieving the effect of effectively eliminating jitter while avoiding the appearance of afterimage phenomena. Description of the Drawings

[0044] Figure 1Schematic flowchart of an image processing method provided by an embodiment of the present application;

[0045] Figure 2 Schematic flowchart of another image processing method provided by an embodiment of the present application;

[0046] Figure 3 Schematic flowchart of yet another image processing method provided by an embodiment of the present application;

[0047] Figure 4 Schematic structural diagram of an image processing apparatus provided by an embodiment of the present application;

[0048] Figure 5 Schematic structural diagram of an image processing device provided by an embodiment of the present application. Detailed implementation manners

[0049] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application.

[0050] An embodiment of the present application provides an image processing method, which can be applied to an electronic device, such as Figure 1 As shown, the method includes the following steps:

[0051] S101. Obtain multiple frames of images to be processed in the video to be processed for the target object.

[0052] Among them, the target object can be an object with a target face, and the multiple frames of images to be processed can be each frame image (i.e., all frame images) in the video to be processed, or can be partial frame images in the video to be processed.

[0053] In an embodiment of the present application, the electronic device can be a device with image acquisition and processing capabilities, and the electronic device directly obtains multiple frames of images to be processed including the target face from the video to be processed.

[0054] S102. Process the multiple frames of images to be processed to determine the target parameters of the target pixel points of the multiple frames of images to be processed.

[0055] Among them, processing the multiple frames of images to be processed can be processing the pixel points in the multiple frames of images to be processed; the target pixel points can be the pixel points in the multiple frames of images to be processed or the key points of the target face in the multiple frames of images to be processed; the target parameters of the target pixel points can be the position information of the key points in the multiple frames of images to be processed or the probability value of the pixel points.

[0056] In an embodiment of the present application, the electronic device analyzes and processes the target face or target object in the multiple frames of images to be processed, so as to determine the position information of the key points in the multiple frames of images to be processed or the probability value of the pixel points.

[0057] S103. Determine whether the moving speed of the target object meets the target set speed based on the target parameters of the target pixel points.

[0058] Among them, the moving speed of the target object can be the moving speed of the target face in multiple frames of images to be processed, and the target set speed can be the speed set according to the specific scenario; in a feasible implementation, the target set speed can be a speed threshold.

[0059] In the embodiments of the present application, the electronic device determines whether the moving speed of the target face meets the speed threshold set in the current scenario according to the position information of the key points of the target face or the probability value of the pixel points of the image to be processed. In a feasible implementation, the moving speed of the target face is compared with the speed threshold. If the moving speed of the target face is greater than or equal to the speed threshold, it indicates that the moving speed of the target face meets the target set speed; if the moving speed of the target face is less than the speed threshold, it indicates that the moving speed of the target face does not meet the target set speed.

[0060] S104. When the moving speed of the target object meets the target set speed, perform a first process on the pixel points of the image to be processed to determine the target area of the target object.

[0061] In the embodiments of the present application, when the moving speed of the target object meets the target set speed, it indicates that the target object has moved quickly; among them, the first process can be a process related only to the pixel points of the current frame of the image to be processed; it should be noted that the target area can be determined according to the position information of the key points of the target face in the current frame of the image to be processed or the probability value of the pixel points of the current frame of the image to be processed.

[0062] Among them, the target area can refer to the area of the image to be processed that needs to be image - repaired. The target area can include the area where the key points of the target face are located, or the area where the target object is located in the image to be processed. In a feasible implementation, the area of the target face can be the eyes, forehead, nose, and mouth of the target face, etc.

[0063] S105. When the moving speed of the target object does not meet the target set speed, perform a second process on the pixel points of the image to be processed to determine the target area of the target object.

[0064] In the embodiments of the present application, the second process can determine the target area of the target object according to the position information of the key points of the current frame of the image to be processed and the position information of the key points of the frames of the image to be processed before the current frame, or can determine the target area of the target object according to the probability value of the pixel points of the current frame of the image to be processed and the probability value of the key points of the frames of the image to be processed before the current frame.

[0065] The image processing method provided by the embodiment of the present application obtains multiple frames of images to be processed in the video to be processed for the target object; processes the multiple frames of images to be processed to determine the target parameters of the target pixel points of the multiple frames of images to be processed; based on the target parameters of the target pixel points, determines whether the moving speed of the target object meets the target set speed; in the case where the moving speed of the target object meets the target set speed, performs a first processing on the pixel points of the image to be processed to determine the target area of the target object; in the case where the moving speed of the target object does not meet the target set speed, performs a second processing on the pixel points of the image to be processed to determine the target area of the target object; in this way, the moving speed of the target object can be determined according to the target parameters obtained by processing the multiple frames of images to be processed, and then different processing methods can be selected according to the moving speed of the target object to process the pixel points to determine the target area of the target object, solving the problem that the anti-shake solution in the related art will have ghosting when the face moves rapidly, thus achieving the effect of effectively eliminating jitter while avoiding the appearance of ghosting.

[0066] Based on the foregoing embodiments, an embodiment of the present application provides an image processing method. Referring to Figure 2 as shown, the method includes the following steps:

[0067] S201. An electronic device obtains multiple frames of images to be processed in the video to be processed for the target object.

[0068] S202. The electronic device performs key point detection on the multiple frames of images to be processed to obtain the position information of the key points of the target face in the multiple frames of images to be processed.

[0069] Among them, the target pixel points include the pixel points at the key points.

[0070] In the embodiment of the present application, the position information of the key points may be the coordinate information of the key points; in a feasible implementation manner, the electronic device performs key point detection on each frame of the image to be processed to obtain the coordinate information of the key points of the target face in each frame of the image to be processed.

[0071] S203. The electronic device obtains the position information of the key points of the images to be processed corresponding to different frames separated by the first number of frames and compares them.

[0072] Among them, the different frames separated by the first number of frames can be two non - adjacent frames, and the times corresponding to the two non - adjacent frames in the video to be processed are also different; that is, it is necessary to compare and process the position information of the key points of the images to be processed of the frames with a certain time difference. Among them, the first number of frames can be one frame or multiple frames. The comparison of the key - point position information can be to compare the coordinate information of the key points of the target face; in a feasible implementation, a matrix corresponding to it can be generated according to the position information of the key points of the target face in the images to be processed corresponding to different frames, and then the coordinate information of the key points of the target face in different frames is compared by comparing the elements in the matrices corresponding to different frames.

[0073] S204. Based on the comparison result, the electronic device determines whether the area corresponding to the target key points whose position information of the key points with corresponding relationships has changed meets the reference area.

[0074] Among them, the reference area can be a sub - area of the target face. The sub - area of the target face can be one sub - area or multiple sub - areas. The target key points can be the key points whose position information has changed in the image to be processed. It should be noted that the reference area can be a specific area on the target face or an area on the target face whose ratio to the overall area of the target face is greater than a ratio threshold.

[0075] In the embodiment of the present application, the difference between the position information of the key points with corresponding relationships is compared, and the number of key points of the target face in multiple frames of images to be processed whose difference is greater than the target difference threshold is obtained; if the number of key points of the target face whose difference is greater than the target difference threshold is greater than or equal to a certain number threshold, it is determined that the area corresponding to the target key points whose position information has changed meets the reference area; if the number of key points of the target face whose difference is greater than the target difference threshold is less than a certain number threshold, it is determined that the area corresponding to the target key points whose position information has changed does not meet the reference area.

[0076] Among them, the target difference threshold can be a target difference threshold set according to the specific application scenario, and the number threshold can also be a number threshold set according to the specific application scenario.

[0077] It should be noted that after S204, S205 - 206 can be executed, or S207 - 208 can be executed.

[0078] S205. When the area corresponding to the target key points meets the reference area, the electronic device determines that the moving speed of the target face meets the target set speed.

[0079] In the embodiment of the present application, if the number of key points of the target face whose difference is greater than the target difference threshold is greater than or equal to a certain number threshold, it is determined that the moving speed of the target face meets the target set speed.

[0080] S206. When the electronic device determines that the moving speed of the target face meets the target set speed, it determines the target area of the target object based on the position information of the first key points of the to-be-processed image of the current frame.

[0081] Among them, the situation where the moving speed of the target face meets the target set speed can be that the moving speed of the target face is very fast, that is, the target face is in a fast-moving state of motion.

[0082] In the embodiment of the present application, when the electronic device determines that the target face is moving fast, it directly determines the target area of the target face according to the coordinate information of the first key points of the to-be-processed image of the current frame, which can avoid the phenomenon of ghosting. In a feasible implementation manner, when the electronic device determines that the target face is moving fast, it determines the target area of the target face according to the coordinate information of the eyes in the to-be-processed image of the current frame.

[0083] S207. When the area corresponding to the target key point does not meet the reference area, the electronic device determines that the moving speed of the target face does not meet the target set speed.

[0084] In the embodiment of the present application, if the number of key points of the target face with a difference greater than the target difference threshold is less than a certain number threshold, it is determined that the moving speed of the target face does not meet the target set speed.

[0085] S208. When the electronic device determines that the moving speed of the target face does not meet the target set speed, it determines the target area of the target object based on the position information of the first key points of the to-be-processed image of the current frame and the position information of the second key points of the to-be-processed image of the frame before the current frame.

[0086] Among them, the situation where the moving speed of the target face does not meet the target set speed can be that the target face does not move fast, or the target face is in a static state. The to-be-processed image of the frame before the current frame can be the to-be-processed image adjacent to the current frame, or the to-be-processed image not adjacent to the current frame (i.e., with an interval), which is not limited here.

[0087] In the embodiment of the present application, when the electronic device determines that the target face does not move fast, it determines the target area of the target face according to the coordinate information of the first key points and the coordinate information of the second key points. In a feasible implementation manner, as in the related art, the average value or weighted value is used for processing and the target area of the target face is determined. Since the target face does not move fast at this time, using the average value or weighted value of multiple to-be-processed images for processing will not result in the phenomenon of ghosting.

[0088] It should be noted that the descriptions of the same or corresponding steps in this embodiment and other embodiments can be referred to the descriptions in other embodiments, and will not be repeated here.

[0089] The image processing method provided by the embodiments of this application can determine the moving speed of the target object according to the target parameters obtained by processing multiple frames of images to be processed, and then select different processing methods to process pixel points according to the moving speed of the target object to determine the target area of the target object, solving the problem that in the related art, when the face moves rapidly, there will be ghosting in the anti-shake solution, thereby realizing the effective elimination of jitter while avoiding the appearance of ghosting.

[0090] Based on the foregoing embodiments, an embodiment of this application provides an image processing method. Refer to Figure 3 as shown, the method includes the following steps:

[0091] S301. The electronic device obtains multiple frames of images to be processed in the video to be processed for the target object.

[0092] S302. The electronic device analyzes the multiple frames of images to be processed and determines the probability value that the pixel points of the multiple frames of images to be processed belong to the pixel points of the target part of the image to be processed.

[0093] Among them, the target pixel points include the pixel points of the multiple frames of images to be processed, and the target part is determined based on the depth-of-field information of the image to be processed. In a feasible implementation manner, the target part can be the foreground part or the background part of the image to be processed.

[0094] In the embodiments of this application, the probability value refers to the probability that the pixel points of the image to be processed are located in the foreground part of the image to be processed, or the probability that the pixel points of the image to be processed are located in the background part of the image to be processed. In this embodiment, an example where the target part refers to the foreground part of the image to be processed is used for explanation.

[0095] S303. The electronic device compares the probability value of the first pixel point of the current frame with the corresponding relationship with the probability value of the second pixel point of the frame that is two frames before the current frame and separated by the second number of frames.

[0096] Among them, the first pixel point can be the pixel point of the current frame of the image to be processed, and the second pixel point can be the pixel point of the image to be processed that is two frames before the current frame and separated by the second number of frames.

[0097] In an embodiment of the present application, the probability value of the first pixel point refers to the probability that the pixel point of the image to be processed in the current frame is located in the foreground part, and the probability value of the second pixel point refers to the probability that the pixel point of the image to be processed before the image to be processed in the current frame and separated by the second number of frames is located in the foreground part. It should be noted that the second number of frames can be one frame or multiple frames, and the first number of frames and the second number of frames can be the same or different.

[0098] It should be noted that after S303, S304-305 can be executed, or S306-307 can be executed.

[0099] S304. When the number of pixel points whose comparison result represents a change in probability value meets the target number threshold, the electronic device determines that the moving speed of the target object meets the target set speed.

[0100] Among them, the target number threshold can be a number threshold set according to a specific scenario.

[0101] In an embodiment of the present application, when the number of pixel points whose probability value changes is greater than or equal to the target number threshold, it is determined that the moving speed of the target face meets the target set speed.

[0102] S305. When the electronic device determines that the moving speed of the target object meets the target set speed, it determines the target area of the image to be processed based on the probability value of the first pixel point in the current frame.

[0103] In an embodiment of the present application, since when the target face moves quickly, using the weighted value or average value of the probability values of the pixel points of multiple images to be processed to determine the target area will result in ghosting. Therefore, when the target face in the image to be processed moves quickly, directly using the probability value of the first pixel point of the image to be processed in the current frame to determine the target area of the image to be processed can avoid the occurrence of ghosting.

[0104] S306. When the number of pixel points whose comparison result represents a change in probability value does not meet the target number threshold, the electronic device determines that the moving speed of the target object does not meet the target set speed.

[0105] In an embodiment of the present application, when the number of pixel points whose probability value changes is less than the target number threshold, it is determined that the moving speed of the target face does not meet the target set speed.

[0106] S307. When the electronic device determines that the moving speed of the target object does not meet the target set speed, it determines the target area of the image to be processed based on the probability value of the first pixel point in the current frame and the probability value of the second pixel point in the frame before the current frame.

[0107] In the embodiment of the present application, when the target face in the image to be processed does not move quickly or is in a static state, the probability value of the first pixel point and the probability value of the second pixel point are used to determine the target area of the image to be processed. Since the target face does not move quickly at this time, using the weighted value or average value of the probability values of the pixel points of multiple frames of the image to be processed to determine the target area will not cause a ghosting phenomenon.

[0108] Based on the foregoing embodiments, in other embodiments of the present application, the method may further include the following steps:

[0109] S308. When the electronic device determines that the moving speed of the target face meets the target set speed, delete the position information of the second key point of the image to be processed in the frames before the current frame.

[0110] In the embodiment of the present application, when the target face is moving quickly, since the position information of the key points of the image to be processed in the frames before the current frame is not required, the electronic device deletes the locally stored position information, which can save the local storage space of the electronic device.

[0111] S309. When the electronic device determines that the moving speed of the target object meets the target set speed, delete the probability value of the second pixel point of the frames before the current frame.

[0112] In the embodiment of the present application, when the target face is moving quickly, since the probability value of the pixel points of the image to be processed in the frames before the current frame is not required, the electronic device deletes the locally stored probability value, which can save the local storage space of the electronic device.

[0113] S310. The electronic device processes the target object based on the position information of the target area in the image to be processed.

[0114] In the embodiment of the present application, the electronic device processes the target face according to the coordinate information of the target area in the image to be processed. In a feasible implementation manner, processing the target face may be performing retouching or matting processing on the target face.

[0115] It should be noted that the description of the same or corresponding steps in this embodiment and other embodiments may refer to the description in other embodiments, and will not be repeated here.

[0116] The image processing method provided by the embodiments of the present application can determine the moving speed of the target object according to the target parameters obtained by processing multiple frames of images to be processed, and then select different processing methods to process the pixel points according to the moving speed of the target object to determine the target area of the target object, solving the problem that the anti-shake solution in the related art will have afterimages when the face moves quickly, thereby achieving the effective elimination of jitter while avoiding the appearance of afterimage phenomena.

[0117] Based on the foregoing embodiments, an embodiment of the present application provides an image processing device, which can be applied to Figures 1 to 3 the image processing method provided by the corresponding embodiment, referring to Figure 4 As shown, the device 4 may include: an acquisition unit 41, a first determination unit 42, a second determination unit 43, and a processing unit 44, where:

[0118] The acquisition unit 41 is configured to acquire multiple frames of images to be processed in the video to be processed for the target object;

[0119] The first determination unit 42 is configured to process multiple frames of images to be processed and determine the target parameters of the target pixel points of the multiple frames of images to be processed;

[0120] The second determination unit 43 is configured to determine whether the moving speed of the target object meets the target set speed based on the target parameters of the target pixel points;

[0121] The processing unit 44 is configured to, when the moving speed of the target object meets the target set speed, perform a first process on the pixel points of the image to be processed to determine the target area of the target object;

[0122] The processing unit 44 is further configured to, when the moving speed of the target object does not meet the target set speed, perform a second process on the pixel points of the image to be processed to determine the target area of the target object.

[0123] In other embodiments of the present application, the first determination unit 42 is further configured to perform the following steps:

[0124] Perform key point detection on multiple frames of images to be processed to obtain the position information of the key points of the target face in the multiple frames of images to be processed; wherein, the target pixel points include the pixel points at the key points.

[0125] In other embodiments of the present application, the second determination unit 43 is further configured to perform the following steps:

[0126] Obtain the position information of the key points of the images to be processed corresponding to different frames separated by the first number of frames and compare them;

[0127] Based on the comparison result, determine whether the region corresponding to the target key point whose position information of the key points with a corresponding relationship has changed satisfies the reference region;

[0128] When the region corresponding to the target key point satisfies the reference region, determine that the moving speed of the target face conforms to the target set speed;

[0129] When the region corresponding to the target key point does not satisfy the reference region, determine that the moving speed of the target face does not conform to the target set speed.

[0130] In other embodiments of the present application, the processing unit 44 is further configured to perform the following steps:

[0131] When it is determined that the moving speed of the target face conforms to the target set speed, determine the target region of the target object based on the position information of the first key point of the image to be processed in the current frame;

[0132] Correspondingly, when it is determined that the moving speed of the target face does not conform to the target set speed, determine the target region of the target object based on the position information of the first key point of the image to be processed in the current frame and the position information of the second key point of the image to be processed in the frame before the current frame.

[0133] In other embodiments of the present application, the first determination unit 42 is further configured to perform the following steps:

[0134] Analyze multiple frames of images to be processed, and determine the probability value of the pixel points of the multiple frames of images to be processed belonging to the pixel points of the target part of the image to be processed; wherein, the target pixel points include the pixel points of the multiple frames of images to be processed; the target part is determined based on the depth of field information of the image to be processed.

[0135] In other embodiments of the present application, the second determination unit 43 is further configured to perform the following steps:

[0136] Compare the probability value of the first pixel point in the current frame with the corresponding relationship, and the probability value of the second pixel point in the frame that is before the current frame and has an interval of the second number of frames;

[0137] When the number of pixel points whose comparison result indicates a change in the probability value satisfies the target number threshold, determine that the moving speed of the target object conforms to the target set speed;

[0138] When the number of pixel points whose comparison result indicates a change in the probability value does not satisfy the target number threshold, determine that the moving speed of the target object does not conform to the target set speed.

[0139] In other embodiments of the present application, the processing unit 44 is further configured to perform the following steps:

[0140] When it is determined that the moving speed of the target object meets the target set speed, determine the target area of the image to be processed based on the probability value of the first pixel point in the current frame;

[0141] Correspondingly, when it is determined that the moving speed of the target object does not meet the target set speed, determine the target area of the image to be processed based on the probability value of the first pixel point in the current frame and the probability value of the second pixel point in the frame before the current frame.

[0142] In other embodiments of the present application, the processing unit 44 is further configured to perform the following steps:

[0143] When it is determined that the moving speed of the target face does not meet the target set speed, delete the position information of the second key point of the image to be processed in the frame before the current frame;

[0144] Alternatively, when it is determined that the moving speed of the target object does not meet the target set speed, delete the probability value of the second pixel point in the frame before the current frame.

[0145] In other embodiments of the present application, the processing unit 44 is further configured to perform the following steps:

[0146] Process the target object based on the position information of the target area in the image to be processed.

[0147] The image processing device provided by the embodiments of the present application can determine the moving speed of the target object according to the target parameters obtained by processing multiple frames of images to be processed, and then select different processing methods according to the moving speed of the target object to determine the target area of the target object, solving the problem that in the related art, when the face moves rapidly, there will be ghosting in the scheme for eliminating jitter, thereby realizing the effective elimination of jitter while avoiding the appearance of ghosting.

[0148] Based on the foregoing embodiments, an embodiment of the present application provides an image processing device, which can be applied to Figures 1 to 3 the image processing method provided by the corresponding embodiment, refer to Figure 5 As shown, the image processing device 5 may include: a processor 51, a memory 52, and a communication bus 53, where:

[0149] The communication bus 53 is used to implement the communication connection between the processor 51 and the memory 52;

[0150] The processor 51 is configured to execute the information determination program stored in the memory 52 to implement the following steps:

[0151] Obtain multiple frames of images to be processed in the video to be processed for the target object;

[0152] Process multiple frames of images to be processed and determine the target parameters of the target pixel points of the multiple frames of images to be processed;

[0153] Based on the target parameters of the target pixel points, determine whether the moving speed of the target object meets the target set speed;

[0154] When the moving speed of the target object meets the target set speed, perform a first processing on the pixel points of the image to be processed to determine the target area of the target object;

[0155] When the moving speed of the target object does not meet the target set speed, perform a second processing on the pixel points of the image to be processed to determine the target area of the target object.

[0156] In other embodiments of the present application, the processor 51 is used to execute the processing of multiple frames of images to be processed stored in the memory 52, and determine the target parameters of the pixel points of the target area of the multiple frames of images to be processed, so as to implement the following steps:

[0157] Perform key point detection on multiple frames of images to be processed to obtain the position information of the key points of the target face in the multiple frames of images to be processed; wherein, the target pixel points include the pixel points at the key points.

[0158] In other embodiments of the present application, the processor 51 is used to execute the determination of whether the moving speed of the target object meets the target set speed based on the target parameters of the target pixel points stored in the memory 52, so as to implement the following steps:

[0159] Obtain the position information of the key points of the images to be processed corresponding to different frames separated by the first number of frames and compare them;

[0160] Based on the comparison result, determine whether the area corresponding to the target key point where the position information of the corresponding key points changes meets the reference area;

[0161] When the area corresponding to the target key point meets the reference area, determine that the moving speed of the target face meets the target set speed;

[0162] When the area corresponding to the target key point does not meet the reference area, determine that the moving speed of the target face does not meet the target set speed.

[0163] In other embodiments of the present application, the processor 51 is used to execute the first processing on the pixel points of the image to be processed when the moving speed of the target object meets the target set speed, and determine the target area of the target object, so as to implement the following steps:

[0164] When it is determined that the moving speed of the target face meets the target set speed, the target area of the target object is determined based on the position information of the first key points of the image to be processed in the current frame;

[0165] Correspondingly, when it is determined that the moving speed of the target face does not meet the target set speed, the target area of the target object is determined based on the position information of the first key points of the image to be processed in the current frame and the position information of the second key points of the image to be processed in the frames before the current frame.

[0166] In other embodiments of the present application, the processor 51 is used to execute the processing of multiple frames of images to be processed stored in the memory 52, and determine the target parameters of the target pixel points of the multiple frames of images to be processed, so as to implement the following steps:

[0167] Analyze multiple frames of images to be processed, and determine the probability value that the pixel points of the multiple frames of images to be processed belong to the pixel points of the target part of the image to be processed; wherein, the target pixel points include the pixel points of the multiple frames of images to be processed; the target part is determined based on the depth of field information of the image to be processed.

[0168] In other embodiments of the present application, the processor 51 is used to execute to determine whether the moving speed of the target object meets the target set speed based on the target parameters of the target pixel points stored in the memory 52, so as to implement the following steps:

[0169] Compare the probability value of the first pixel point in the current frame with the corresponding relationship with the probability value of the second pixel point in the frame that is before the current frame and has an interval of the second number of frames;

[0170] When the number of pixel points whose comparison result indicates a change in probability value meets the target number threshold, it is determined that the moving speed of the target object meets the target set speed;

[0171] When the number of pixel points whose comparison result indicates a change in probability value does not meet the target number threshold, it is determined that the moving speed of the target object does not meet the target set speed.

[0172] In other embodiments of the present application, the processor 51 is used to execute to perform a first processing on the pixel points of the image to be processed to determine the target area of the target object when the moving speed of the target object meets the target set speed, so as to implement the following steps:

[0173] When it is determined that the moving speed of the target object meets the target set speed, the target area of the image to be processed is determined based on the probability value of the first pixel point in the current frame;

[0174] Correspondingly, in the case where it is determined that the moving speed of the target object does not meet the target set speed, based on the probability value of the first pixel point in the current frame and the probability value of the second pixel point in the frame before the current frame, the target area of the image to be processed is determined.

[0175] In other embodiments of the present application, the processor 51 is further configured to implement the following steps:

[0176] In the case where it is determined that the moving speed of the target face does not meet the target set speed, delete the position information of the second key point of the image to be processed in the frame before the current frame;

[0177] Alternatively, in the case where it is determined that the moving speed of the target object does not meet the target set speed, delete the probability value of the second pixel point in the frame before the current frame.

[0178] In other embodiments of the present application, the processor 51 is further configured to implement the following steps:

[0179] Based on the position information of the target area in the image to be processed, process the target object.

[0180] The image processing device provided by the embodiments of the present application can determine the moving speed of the target object according to the target parameters obtained by processing multiple frames of images to be processed, and then select different processing methods to process the pixel points according to the moving speed of the target object to determine the target area of the target object, solving the problem that the anti-jitter solution in the related art will produce ghosting when the face moves quickly, thereby realizing the effective elimination of jitter while avoiding the appearance of ghosting.

[0181] Based on the foregoing embodiments, an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement Figures 1 to 3 The steps in the image processing method provided by the corresponding embodiment.

[0182] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0183] It should be noted that the above computer-readable storage medium may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it may also be various electronic devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0184] It should be noted that in this text, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or device including that element.

[0185] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0186] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0187] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0188] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0190] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present application, or directly or indirectly applied to other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. An image processing method, the method comprising: Obtaining multiple frames of to-be-processed images in a to-be-processed video for a target object; Processing the multiple frames of to-be-processed images to determine target parameters of target pixel points in the multiple frames of to-be-processed images; Based on the target parameters of the target pixel points, determining whether the moving speed of the target object meets a target set speed; When the moving speed of the target object meets the target set speed, performing a first processing on pixel points of the to-be-processed image to determine a target area of the target object; the first processing is a processing related only to pixel points of the current frame of the to-be-processed image; wherein, the moving speed of the target object meeting the target set speed indicates that the moving speed of the target human face in the to-be-processed image is greater than or equal to a speed threshold; When the moving speed of the target object does not meet the target set speed, performing a second processing on pixel points of the to-be-processed image to determine a target area of the target object.

2. The method according to claim 1, wherein The processing the multiple frames of to-be-processed images to determine target parameters of pixel points in the target area of the multiple frames of to-be-processed images includes: Performing key point detection on the multiple frames of to-be-processed images to obtain position information of key points of a target human face in the multiple frames of to-be-processed images; wherein, the target pixel points include pixel points at the key points.

3. The method according to claim 2, wherein The determining whether the moving speed of the target object meets the target set speed based on the target parameters of the target pixel points includes: Obtaining and comparing position information of key points of to-be-processed images corresponding to different frames separated by a first number of frames; Based on the comparison result, determining whether an area corresponding to a target key point where the position information of the key points with a corresponding relationship changes meets a reference area; When the area corresponding to the target key point meets the reference area, determining that the moving speed of the target human face meets the target set speed; When the area corresponding to the target key point does not meet the reference area, determining that the moving speed of the target human face does not meet the target set speed.

4. The method according to claim 3, wherein The performing a first processing on pixel points of the to-be-processed image to determine a target area of the target object when the moving speed of the target object meets the target set speed includes: When it is determined that the moving speed of the target human face meets the target set speed, determining the target area of the target object based on position information of a first key point of the current frame of the to-be-processed image; Correspondingly, the performing a second processing on pixel points of the to-be-processed image to determine a target area of the target object when the moving speed of the target object does not meet the target set speed includes: When it is determined that the moving speed of the target human face does not meet the target set speed, determining the target area of the target object based on position information of a first key point of the current frame of the to-be-processed image and position information of a second key point of a to-be-processed image of a frame before the current frame.

5. The method according to claim 1, wherein Processing the multi-frame images to be processed to determine the target parameters of the target pixel points of the multi-frame images to be processed, including: Analyzing the multi-frame images to be processed to determine the probability values of the pixel points of the multi-frame images to be processed belonging to the pixel points of the target part of the image to be processed; wherein, the target pixel points include the pixel points of the multi-frame images to be processed; the target part is determined based on the depth of field information of the image to be processed.

6. The method according to claim 5, wherein Based on the target parameters of the target pixel points, determining whether the moving speed of the target object meets the target set speed, including: Comparing the probability value of the first pixel point in the current frame with a corresponding relationship with the probability value of the second pixel point in the frame that is before the current frame and separated by the second number of frames; In the case where the number of pixel points whose comparison result indicates a change in the probability value meets the target number threshold, determining that the moving speed of the target object meets the target set speed; In the case where the number of pixel points whose comparison result indicates a change in the probability value does not meet the target number threshold, determining that the moving speed of the target object does not meet the target set speed.

7. The method according to claim 6, wherein In the case where the moving speed of the target object meets the target set speed, performing a first process on the pixel points of the image to be processed to determine the target area of the target object, including: In the case where it is determined that the moving speed of the target object meets the target set speed, determining the target area of the image to be processed based on the probability value of the first pixel point in the current frame; Correspondingly, in the case where the moving speed of the target object does not meet the target set speed, performing a second process on the pixel points of the image to be processed to determine the target area of the target object, including: In the case where it is determined that the moving speed of the target object does not meet the target set speed, determining the target area of the image to be processed based on the probability value of the first pixel point in the current frame and the probability value of the second pixel point in the frame before the current frame.

8. The method according to claim 4 or 7, characterized in that, The method further includes: In the case where it is determined that the moving speed of the target face meets the target set speed, deleting the position information of the second key points of the images to be processed in the frames before the current frame; Or, in the case where it is determined that the moving speed of the target object meets the target set speed, deleting the probability values of the second pixel points in the frames before the current frame.

9. The method according to claim 1, wherein The method further includes: Processing the target object based on the position information of the target area in the image to be processed.

10. An image processing apparatus, characterized in that, The apparatus includes: An acquisition unit, configured to acquire multi-frame images to be processed in a video to be processed for a target object; A first determination unit, configured to process the multi-frame images to be processed to determine the target parameters of the target pixel points of the multi-frame images to be processed; A second determination unit, configured to determine whether the moving speed of the target object meets the target set speed based on the target parameters of the target pixel points; A processing unit, configured to, in the case where the moving speed of the target object meets the target set speed, perform a first process on the pixel points of the image to be processed to determine the target area of the target object; The first processing is processing related only to the pixel points of the image to be processed in the current frame; wherein, the moving speed of the target object conforms to the target set speed, indicating that the moving speed of the target human face in the image to be processed is greater than or equal to the speed threshold; The processing unit is further configured to, when the moving speed of the target object does not conform to the target set speed, perform a second processing on the pixel points of the image to be processed to determine the target area of the target object.

Images