Image processing method and device, model training method and device, equipment and storage medium

Abstract

The application discloses an image processing method and device, a model training method and device, and a storage medium, and belongs to the technical field of image processing. The image processing method comprises the following steps: acquiring a first image, wherein the first image comprises a target object; identifying a plurality of key points corresponding to the target object in the first image to obtain key point information corresponding to the target object, wherein the key point information comprises a key point position and a key point confidence of each key point in the plurality of key points; identifying at least one blurred region in the first image according to the key point information to obtain a region position and a region blur degree corresponding to the at least one blurred region; and performing a blur removal processing on the first image based on the region position and the region blur degree to obtain a second image.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to an image processing method, a model training method, an apparatus, a device, and a storage medium. Background Technology

[0002] With the widespread use of image capture functions in electronic devices, users often need to use electronic devices to capture moving targets, such as people or animals jumping or running. In this case, the images captured by electronic devices often have blurriness and ghosting, resulting in poor image capture quality.

[0003] To eliminate blurring caused by the movement of a target object in an image, the shutter speed is usually increased during the image shooting process to help capture a clear image. However, this method can only partially eliminate the blurring of moving objects, and it also introduces very serious noise, resulting in poor image quality. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method, model training method, apparatus, device, and storage medium that can solve the problem of poor image quality.

[0005] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0006] Obtain a first image, which includes the target object;

[0007] Identify multiple key points in the first image that correspond to the target object, and obtain key point information corresponding to the target object, wherein the key point information includes the key point position and key point confidence of each of the multiple key points;

[0008] Based on the key point information, at least one blurred region in the first image is identified, and the region position and region blur degree corresponding to the at least one blurred region are obtained respectively.

[0009] Based on the location and ambiguity of the region, the first image is subjected to blur removal processing to obtain the second image.

[0010] Secondly, embodiments of this application provide a model training method, the method comprising:

[0011] Obtain a sample image and its corresponding image label, wherein the sample image includes a first object, and the image label includes a region location label and a region blur label of the blurred region in the sample image;

[0012] Identify multiple key points in the sample image corresponding to the first object to obtain key point information corresponding to the first object, wherein the key point information includes the key point position and key point confidence of each of the multiple key points;

[0013] The sample image and the key point information are input into the target neural network model, and the target neural network model identifies at least one blurred region in the sample image based on the key point information, thereby obtaining the region location and region blur degree corresponding to the at least one blurred region.

[0014] Based on the location of the region and the ambiguity of the region, the target neural network model is trained according to the image label to obtain the ambiguity recognition model.

[0015] Thirdly, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0016] An image acquisition module is used to acquire a first image, wherein the first image includes a target object;

[0017] A key point recognition module is used to identify multiple key points in the first image corresponding to the target object, and obtain key point information corresponding to the target object, wherein the key point information includes the key point position and key point confidence of each of the multiple key points;

[0018] The region recognition module is used to identify at least one blurred region in the first image based on the key point information, and to obtain the region position and region blur degree corresponding to the at least one blurred region respectively.

[0019] An image processing module is used to perform blur removal processing on the first image based on the region location and the region blur level to obtain a second image.

[0020] Fourthly, embodiments of this application provide a model training apparatus, the apparatus comprising:

[0021] The label acquisition module is used to acquire sample images and their corresponding image labels. The sample images include a first object, and the image labels include a region location label and a region blur label of a blurred region in the sample images.

[0022] The first recognition module is used to recognize multiple key points in the sample image corresponding to the first object and obtain key point information corresponding to the first object, wherein the key point information includes the key point position and key point confidence of each of the multiple key points;

[0023] The second recognition module is used to input the sample image and the key point information into the target neural network model, and the target neural network model identifies at least one blurred region in the sample image based on the key point information, and obtains the region position and region blur degree corresponding to the at least one blurred region respectively.

[0024] The model training module is used to train the target neural network model according to the image label based on the region location and the region ambiguity, so as to obtain the ambiguity recognition model.

[0025] Fifthly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the methods described in the first aspect and / or the second aspect.

[0026] In a sixth aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first and / or second aspects.

[0027] In a seventh aspect, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the methods described in the first aspect and / or the second aspect.

[0028] Eighthly, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as described in the first aspect and / or the second aspect.

[0029] In this embodiment, by identifying the key point positions and key point confidence levels of multiple key points corresponding to the target object in the first image, and using these key point positions and key point confidence levels, the region position and region blur level corresponding to at least one blurred region in the first image can be identified. Thus, when performing blur removal processing on the first image based on the region position and region blur level, the blurred region in the first image can be accurately located, and the blurred region can be accurately blurred without introducing excessive noise, thereby improving the effect of image deblurring and improving image quality. Attached Figure Description

[0030] Figure 1 This is a flowchart of an image processing method provided in one embodiment of this application;

[0031] Figure 2 This is an architecture diagram of image processing provided in this application;

[0032] Figure 3 This is a flowchart of an image processing method provided in another embodiment of this application;

[0033] Figure 4a This is a schematic diagram of the key points of the first image provided in this application;

[0034] Figure 4b This is a schematic diagram of the heatmap of key points provided in this application;

[0035] Figure 5 This is a schematic diagram showing the location of the target region in the first image provided in this application;

[0036] Figure 6 This is a schematic diagram of the processing procedure of the ambiguity model provided in this application;

[0037] Figure 7 This is a flowchart of a model training method provided in one embodiment of this application;

[0038] Figure 8 This is a structural block diagram of an image processing apparatus provided in one embodiment of this application;

[0039] Figure 9 This is a structural block diagram of a model training device provided in one embodiment of this application;

[0040] Figure 10 This is a structural block diagram of an electronic device provided in one embodiment of this application;

[0041] Figure 11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0043] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0044] The image processing method, model training method, apparatus, device, and storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0045] The image processing method provided in this application can be applied to scenarios involving the processing of images with blurry trailing effects. The following section combines... Figures 1-6 The image processing method provided in the embodiments of this application will be described in detail. It should be noted that the image processing method provided in the embodiments of this application can be executed by an image processing device. In this embodiment, the image processing method provided in the embodiments of this application is illustrated using an image processing device executing the image processing method as an example.

[0046] Figure 1 This is a flowchart of an image processing method provided in one embodiment of this application.

[0047] like Figure 1 As shown, the image processing method may include steps S110-S140, which will be described in detail below.

[0048] S110, Obtain a first image, wherein the first image includes the target object.

[0049] In this embodiment, the first image may be an image captured of a moving target object, which may be a person, animal, object, etc., and is not limited thereto. Furthermore, the first image may be an RGB image containing RGB image data.

[0050] For example, the methods of acquiring the first image include, but are not limited to, acquiring it using an image acquisition device or obtaining it from a preset image storage address.

[0051] S120, identify multiple key points in the first image corresponding to the target object, and obtain key point information corresponding to the target object, wherein the key point information includes the key point position and key point confidence of each key point among the multiple key points.

[0052] Here, a key point recognition model can be set up to identify the key points corresponding to the target object. Based on this, in some embodiments, the above S120 may specifically include:

[0053] The first image is input into the key point recognition model, which identifies multiple key points in the first image that correspond to the target object, thereby obtaining key point information corresponding to the target object.

[0054] In this embodiment, the keypoint recognition model can be a trained model used to identify keypoints contained in a target object. This keypoint recognition model can be a single model, meaning one model is used to identify all keypoints contained in the entire target object. Alternatively, the keypoint recognition model can include multiple models; that is, different keypoint recognition models are used for keypoint recognition in different regions of the target object to improve recognition accuracy and precision.

[0055] For example, the RGB image data of the first image can be input into a keypoint recognition model. This model can then identify keypoints contained within the target object, outputting keypoint information corresponding to that target object. The target object can contain multiple keypoints, and the keypoint information can specifically include the keypoint location and confidence level for each keypoint in the target object. Here, the keypoint confidence level represents the degree of certainty that the keypoint is correctly positioned.

[0056] It should be noted that in this application, the key point locations and key point confidence obtained by using the key point recognition model are used as input features for the ambiguity recognition model in subsequent steps, thereby achieving the purpose of task backtracking, that is, using downstream tasks to backtrack in order to improve the accuracy of the target task.

[0057] S130, based on key point information, identify at least one blurred region in the first image, and obtain the region position and region blur degree corresponding to the at least one blurred region respectively.

[0058] Here, a fuzziness recognition model can be set up to identify the blurred regions in the first image. Based on this, in some embodiments, the above-mentioned S130 may specifically include:

[0059] The first image and key point information are input into the fuzziness recognition model. The fuzziness recognition model identifies at least one fuzzy region in the first image based on the key point information, and obtains the region location and region fuzziness corresponding to the at least one fuzzy region.

[0060] In this embodiment, the fuzziness recognition model can be a trained model for recognizing fuzzy regions contained in an image. This fuzziness recognition model may include multiple convolutional layers. After the image data and key point information of the first image are input as input features to the fuzziness recognition model, the data passes through these multiple convolutional layers, and matrix operations are performed on the features. n =w n (w n-1 (…(w1x+b1))+b n-1 )+b n Each convolutional layer outputs a feature vector, where the feature vector is... n This represents the feature vector output by the nth module. Ultimately, the fuzziness recognition model outputs the region location and fuzziness corresponding to at least one fuzzy region in the first image. The region location can be identified using a mask image, and the fuzziness can be identified using a score.

[0061] In some specific examples, the RGB image data of the first image, the key point locations and key point confidence of the target object in the first image are connected in parallel and input into the fuzziness recognition model. The model performs convolution operations through multiple convolutional layers and outputs a mask map to identify the location of the fuzzy region and a score to identify the fuzziness of each fuzzy region.

[0062] Furthermore, the results output by the fuzziness recognition model can be analyzed. Specifically, the type of each fuzzy region can be distinguished according to the specific value of the fuzziness. For example, the distinction criteria could be that the fuzziness in the range [0, 0.3) is clear, [0.3, 0.8) is moderately fuzzy, and [0.8, 1.0) is heavily fuzzy. Alternatively, the location of the fuzzy region obtained from the fuzziness recognition model output can be the location of the fuzzy region with fuzziness in the ranges [0.3, 0.8) and [0.8, 1.0).

[0063] Thus, compared to existing binary classification network models, the fuzziness recognition model provided in this application embodiment can not only output the fuzziness category to which the image belongs, but also output the location information of the fuzzy region in the image, thereby providing more and more effective data for subsequent fuzz removal processing.

[0064] S140, based on the region location and region ambiguity, perform blur removal processing on the first image to obtain the second image.

[0065] Here, by utilizing the region location and region ambiguity, each blurred region in the first image can be accurately blurred, thereby obtaining a second image with a clear target object. Different image processing models can be used for different regions with varying degrees of ambiguity, and the blur removal process includes, but is not limited to, inputting the first image and a mask image used to identify the region locations of the blurred areas into a preset image processing model for processing.

[0066] Therefore, by identifying the key point positions and key point confidence levels of multiple key points corresponding to the target object in the first image, and using these key point positions and key point confidence levels, it is possible to identify the region position and region blur degree corresponding to at least one blurred region in the first image. Thus, when performing blur removal processing on the first image based on the region position and region blur degree, the blurred region in the first image can be accurately located, and the blurred region can be accurately blurred without introducing excessive noise, thereby improving the effect of image deblurring and improving image quality.

[0067] In addition, in some embodiments, the above-mentioned S140 may specifically include:

[0068] Based on the regional ambiguity, the first image is classified into ambiguities to obtain the target ambiguity type corresponding to the first image;

[0069] Obtain the target image processing model corresponding to the target ambiguity type;

[0070] The first image and the region location are input into the target image processing model, which then performs blur removal on the first image based on the region location to obtain the second image.

[0071] Here, the image processing model can include a blur removal model or a segmented ultra-high definition processing model, and different image processing models can be used for different application scenarios.

[0072] For example, after performing the above-described blur recognition processing on multiple images, the images can be classified according to the highest region blur in each image. For instance, if the highest region blur in the first image is in the range [0.8, 1.0), the target blur type corresponding to the first image can be determined to be severe blur; if the highest region blur in the first image is in the range [0.3, 0.8), the target blur type corresponding to the first image can be determined to be moderate to mild blur.

[0073] In a specific example, taking blur removal as an example, different blur removal models can be set for different types of blur. For example... Figure 2As shown, images with moderate to mild blur can be input into the first blur removal model 21, while images with severe blur can be input into the second blur removal model 22. The first blur removal model 21 can be a classic encoder-decoder structure. The structure of the second blur removal model 22 is similar to that of the first blur removal model 21, but its size is twice that of the first blur removal model 21. That is, the second blur removal model 22 can be a model obtained by concatenating two first blur removal models 21.

[0074] Furthermore, the second blur removal model 22 may also include a wavelet convolution module and a face attention module, giving it stronger blur removal capabilities. The wavelet convolution module can incorporate the wavelet transform algorithm into the convolutional layer, adding 1x1 convolutions before and after the wavelet transform, which helps to better recover features. Additionally, for cases where the target object is a person, a face attention module can be added at the end of the second blur removal model 22. Through the face attention mechanism in this module and the face discrimination loss function during training, difficult-to-reconstruct facial features can be recovered.

[0075] Thus, by inputting the first image and its corresponding blurred region mask (i.e., region location) into the blur removal model corresponding to the target blur type of the first image, a second image without motion blur can be output.

[0076] In another specific example, taking image ultra-high-definition processing as an example, after classifying the image's blurriness based on the blurriness recognition results to obtain the corresponding blurriness type, a segmented ultra-high-definition processing model can be used. Different ultra-high-definition processing models are trained for different blurriness types, which can significantly improve the accuracy of the ultra-high-definition processing model. After obtaining the trained ultra-high-definition processing model, the model can perform targeted ultra-high-definition processing according to the image's blurriness type, outputting the ultra-high-definition result image.

[0077] In addition, in some implementations, when the target object is a person, the above-mentioned key point recognition model may specifically include a face key point recognition sub-model and a human body key point recognition sub-model.

[0078] Based on this, such as Figure 3 As shown, the steps described above, which involve inputting the first image into a keypoint recognition model and using the keypoint recognition model to identify multiple keypoints in the first image that correspond to the target object, thereby obtaining keypoint information corresponding to the target object, may specifically include the following sub-steps:

[0079] S1201, The first image is input into the facial key point recognition sub-model, and the facial key points corresponding to the facial region of the person in the first image are identified by the facial key point recognition sub-model to obtain the facial key point information corresponding to the facial region.

[0080] S1202, the first image is input into the human key point recognition sub-model, and the human key points corresponding to the human body area of ​​the person in the first image are identified by the human key point recognition sub-model to obtain the human key point information corresponding to the human body area.

[0081] S1203, Based on the facial key point information and the human body key point information, determine the key point information corresponding to the person.

[0082] Here, for the face, i.e., the facial region of a person, this application embodiment provides a facial landmark recognition sub-model for identifying landmarks within this region. The facial landmark recognition sub-model can be a 3D dense landmark model with 468 points, and in addition to providing the location of facial landmarks, such as their coordinates, it can also output the confidence score of each facial landmark. The location and confidence score of the facial landmarks can be two branches of this facial landmark recognition sub-model. The confidence score can be used to determine whether the predicted location of the facial landmarks is accurate; when the confidence score is lower than a set threshold, it can be considered that re-prediction is required.

[0083] In some specific examples, when the confidence level of a certain facial landmark output by the facial landmark recognition sub-model is between 0.9 and 1.0, it can be determined that the predicted facial landmark location is highly reliable. However, if the confidence level is between 0.0 and 0.2, it can be determined that the predicted facial landmark location is highly unreliable.

[0084] Furthermore, for the entire human body, i.e., the human body region of a person, this application embodiment provides a human keypoint recognition sub-model for identifying human keypoints within the entire human body region, including the facial region. The human keypoint recognition sub-model can be a human keypoint model with 32 points, and similar to the face keypoint recognition sub-model, it can also output the face keypoint location and confidence score corresponding to each face keypoint. For the sake of brevity, further details are omitted here.

[0085] Thus, by inputting the first image into the facial landmark recognition sub-model and the human landmark recognition sub-model respectively, facial landmark information containing the location and confidence level of facial landmarks, and human landmark information containing the location and confidence level of human landmarks, can be obtained. Based on this, the landmark information corresponding to the person can be determined by comprehensively considering the two types of landmark information.

[0086] In this way, by selecting different key point recognition models for different areas of a person to perform targeted key point recognition, the key point information obtained can be more accurate, thereby improving the accuracy of fuzzy area recognition in subsequent steps.

[0087] Based on this, in some implementations, the above-mentioned S1203 may specifically include:

[0088] Replace the target human body key point information corresponding to the facial region in the human body key point information with the facial key point information to obtain the key point information corresponding to the person.

[0089] Here, since the human body area of ​​a person can include the facial area and the body area, the human body key point information can correspondingly include key point information corresponding to the body area of ​​the person and key point information corresponding to the facial area of ​​the person (that is, target human body key point information).

[0090] Since the key point information output by the facial landmark recognition sub-model is more targeted to the facial region of a person, the target human key point information in the human key point information can be replaced with facial key point information that is more targeted to the facial region. Then, other key point information in the human key point information other than the target human key point information, as well as the facial key point information, can be used as the key point information corresponding to the person.

[0091] In some specific examples, the key point information of 11 key points corresponding to the facial region out of the 32 key points identified by the human key point recognition sub-model can be replaced with the key point information corresponding to the 468 key points identified by the face key point recognition sub-model to obtain the final determined key point information.

[0092] In this way, by replacing key point information, more accurate key point information in each area can be fully utilized to obtain the final key point information, thereby improving the accuracy of key point recognition for people.

[0093] In addition, in some embodiments, the above-mentioned S1201 may specifically include:

[0094] The first image is input into the facial landmark recognition sub-model. The facial landmark recognition sub-model identifies multiple facial landmarks corresponding to the facial region of the person in the first image, and obtains facial landmark information corresponding to the multiple facial landmarks.

[0095] Calculate the ratio of the face region of the person to the area of ​​the first image;

[0096] If the area ratio is less than the preset ratio, the facial region is taken as the target facial key point, and the target facial key point information corresponding to the target facial key point is determined based on the facial key point information corresponding to multiple facial key points.

[0097] The target facial landmark information is determined as the facial landmark information corresponding to the facial region.

[0098] Here, the preset ratio can be, for example, 1 / 4, and the specific value of the preset ratio can be obtained through multiple experiments.

[0099] For example, before or after inputting the first image into the facial landmark recognition sub-model, the ratio of the area of ​​the facial region of the task to the area of ​​the first image can be calculated. If the area of ​​the facial region occupies 1 / 4 or more of the entire first image, that is, the ratio of the area is greater than or equal to 1 / 4, the confidence score of each facial landmark can be calculated separately, i.e., one confidence score corresponds to one facial landmark. If the area of ​​the facial region occupies less than 1 / 4 of the entire first image, that is, the ratio of the area is less than 1 / 4, the entire facial region can be treated as a whole for confidence score calculation, i.e., the entire facial region is treated as a target facial landmark, and the confidence score of the target facial landmark is calculated using the confidence scores of the landmarks corresponding to multiple facial landmarks output by the facial landmark recognition sub-model. The location of the landmark corresponding to the target facial landmark can be the center location of the facial region, and the confidence score of the landmark corresponding to the target facial landmark can be the average of the confidence scores of multiple facial landmarks.

[0100] Thus, by treating the entire facial region as a whole for key point location identification and confidence calculation when the facial area is too small, we can ensure the accuracy of facial key point identification while removing redundant key point information corresponding to overly concentrated facial key points, which facilitates subsequent calculation of blurred areas.

[0101] Based on this, in some implementations, the above-mentioned S1203 may specifically include:

[0102] Determine whether the first image includes the entire body area of ​​the person;

[0103] Given that the first image includes the entire body area of ​​the person, the target human body key point information corresponding to the facial area in the human body key point information is replaced with the target face key point information to obtain the key point information corresponding to the person.

[0104] Here, when the first image contains the entire body area of ​​the person, that is, when the first image is a full-body photo of the person, the target human body key point information corresponding to the facial area in the human body key point information can be directly replaced with the aforementioned target facial key point information. This is because when the image contains the entire body of the person, the facial area of ​​the person will occupy a relatively small proportion, for example, less than 1 / 4 of the entire image. In this case, the key point information of the 11 human body key points corresponding to the facial area in the human body key point information can be replaced with the aforementioned target facial key point information in order to perform subsequent fuzziness recognition processing.

[0105] In addition, to improve the accuracy of fuzziness recognition, in some embodiments, the steps described above—inputting the first image and key point information into the fuzziness recognition model, and having the fuzziness recognition model identify at least one fuzzy region in the first image based on the key point information, and obtaining the region location and region fuzziness corresponding to each of the at least one fuzzy region—may specifically include:

[0106] Obtain the head angle information corresponding to the target object in the first image, wherein the head angle information includes at least one of the rotation angle, pitch angle and tilt angle;

[0107] Generate a key point heatmap based on key point information and head angle information;

[0108] The first image and the key point heatmap are input into the fuzziness recognition model. The fuzziness recognition model identifies at least one fuzzy region in the first image based on the key point heatmap, and obtains the region location and region fuzziness corresponding to the at least one fuzzy region.

[0109] Here, the head angle information can be obtained by performing head pose recognition on the target object in the first image. The head angle information may include at least one of the head's yaw, pitch, and roll angles.

[0110] For example, the keypoint location, keypoint confidence, and head angle information corresponding to each keypoint can be substituted into the following formula (1) to generate a keypoint heatmap. The generated keypoint heatmap is the same size as the original first image, and the entire image is first set to 0 pixel values. Then, the pixel values ​​of the keypoints in the entire image are calculated at the corresponding positions of the keypoints of the target object.

[0111]

[0112] In the above formula (1), F(x,y) is the key point heatmap, xk and yk refer to the coordinates of the key point positions, x and y represent all the coordinate points in this image, and θ (including θ) x and θ y ) is the weighting coefficient, which can be set to 1.5. I can be the key point confidence level, and A can be the weight determined based on the head angle information.

[0113] Specifically, when any angle in the head angle information is greater than or equal to 75 degrees, the value of A can be 0.5; otherwise, the value of A can remain at 1. Calculations across the entire image reveal a keypoint region with the highest value at its center, very close to 1. The value decreases as the keypoint spreads outwards, eventually reaching 0 in distant areas. For example, regarding the keypoint heatmap... Figure 4a The image shown, containing key point information, can be generated through the above process, as follows: Figure 4b The heatmap shown is a key point heatmap.

[0114] Thus, by incorporating head angle information and combining it with weights determined based on the head angle information, the confidence level of key points can be adjusted by weighting. This avoids the problem of low confidence level of key points when the target object is in a large pose, thereby improving the accuracy of fuzzy recognition.

[0115] Based on this, in order to further improve the accuracy of fuzziness recognition, before the steps described above—inputting the first image and the key point heatmap into the fuzziness recognition model, and having the fuzziness recognition model identify at least one fuzzy region in the first image based on the key point heatmap, and obtaining the region location and region fuzziness corresponding to each of the at least one fuzzy region—the image processing method provided in this application embodiment may further include:

[0116] Based on the key point position of each of the multiple key points, the location of the target region corresponding to the target object in the first image is determined.

[0117] Accordingly, the steps described above, which involve inputting the first image and the keypoint heatmap into the fuzziness recognition model, and having the fuzziness recognition model identify at least one fuzzy region in the first image based on the keypoint heatmap, to obtain the region location and fuzziness corresponding to each of the at least one fuzzy region, may specifically include:

[0118] The first image, the key point heatmap, and the target region location are input into the fuzziness recognition model. The fuzziness recognition model identifies at least one fuzzy region in the first image based on the key point heatmap and the target region location, and obtains the region location and region fuzziness corresponding to the at least one fuzzy region.

[0119] Here, the methods for determining the location of the target area corresponding to the target object include, but are not limited to, determining the location of the bounding rectangle corresponding to the multiple key points based on the key point location of each key point among multiple key points, and using it as the location of the target area.

[0120] For example, for such Figure 4a The key point locations shown can be determined as follows: Figure 5 The bounding rectangle shown contains all the key points, which can be used to determine the general area of ​​the target object. The position of the bounding rectangle is also the position of the target area.

[0121] In some specific examples, such as Figure 6 As shown, the RGB original image of the first image, the key point heatmap (heatmap1), and the target region location can be concatenated in parallel and then simultaneously input into the fuzziness recognition model. After multiple convolution operations in the fuzziness recognition model, matrix operations are used to identify the feature. n =w n (w n-1 (…(w1x+b1))+b n-1 )+b n Each convolutional layer outputs a feature vector, where the feature vector is... n This represents the feature vector output by the nth convolutional layer. The final output includes a score representing the ambiguity of each blurred region and a mask representing the region's location. Furthermore, the scores and mask can be combined, for example, by multiplication, to obtain a heatmap containing both the location and ambiguity of the blurred regions.

[0122] Thus, by determining the location of the target area, the position of the main body of the target object in the first image can be further emphasized, thereby improving the attention of the fuzziness recognition model to the main body of the target object.

[0123] Figure 7 This is a flowchart of a model training method provided in one embodiment of this application.

[0124] like Figure 7 As shown, the model training method may include the following steps:

[0125] S710, Obtain a sample image and its corresponding image label. The sample image includes a first object, and the image label includes the region location label and the region blur label of the blurred region in the sample image.

[0126] S720, identify multiple key points in the sample image corresponding to the first object, and obtain key point information corresponding to the first object, wherein the key point information includes the key point position and key point confidence of each key point among the multiple key points;

[0127] S730, the sample image and key point information are input into the target neural network model, and the target neural network model identifies at least one blurred region in the sample image based on the key point information, and obtains the region position and region blur degree corresponding to the at least one blurred region respectively;

[0128] S740, based on regional location and regional ambiguity, trains a target neural network model according to image labels to obtain an ambiguity recognition model.

[0129] Here, the training process of the fuzziness recognition model in this embodiment is roughly the same as the usage process of the fuzziness recognition model described above. The difference is that the first image input during training is a sample image with sample labels, and after the fuzziness recognition model outputs a fuzzy region heatmap, the L1 loss function can be used for iteration to adjust the model parameters of the fuzziness recognition model. Specifically, the fuzzy region heatmap can be compared with the fuzziness category and fuzzy region position of the corresponding sample label to reduce the gap between the model output and the sample label, and training is carried out until the model converges. After training is completed, in actual use, it is only necessary to obtain the key point information and pose information of the image to be processed, and then input them into the fuzziness recognition model according to the image processing method logic provided in the aforementioned embodiment until the fuzziness recognition model predicts the fuzzy region position and the corresponding fuzziness type, and then directly connect it to the image processing model for fuzz removal processing to obtain a clear image.

[0130] Thus, by acquiring sample images and their corresponding image labels, and then using the key point information corresponding to the first object in the identified sample images as input features of the target neural network model, and combining the image labels to train the target neural network model, a fuzziness recognition model that can identify the location and fuzziness of the corresponding region is obtained. This allows users to easily use the fuzziness recognition model to accurately locate the fuzzy region in the image and perform accurate fuzz removal processing on the fuzzy region without introducing too much noise, thereby providing a model basis for improving the effect of image deblurring and improving image quality.

[0131] It should be noted that the application scenarios described in the above embodiments of this disclosure are for the purpose of more clearly illustrating the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of this disclosure. As those skilled in the art will know, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this disclosure are also applicable to similar technical problems.

[0132] Based on the same inventive concept, this application also provides an image processing apparatus. The following is in conjunction with... Figure 8 and Figure 9The image processing apparatus and model training apparatus provided in the embodiments of this application will be described in detail.

[0133] Figure 8 This is a structural block diagram of an image processing apparatus provided in one embodiment of this application.

[0134] like Figure 8 As shown, the image processing apparatus 800 may include:

[0135] Image acquisition module 801 is used to acquire a first image containing the target object;

[0136] The key point recognition module 802 is used to recognize multiple key points in the first image corresponding to the target object, and obtain key point information corresponding to the target object, wherein the key point information includes the key point position and key point confidence of each of the multiple key points;

[0137] The region recognition module 803 is used to identify at least one blurred region in the first image based on the key point information, and to obtain the region position and region blur degree corresponding to the at least one blurred region respectively.

[0138] Image processing module 804 is used to process the first image based on the region location and the region blur, to obtain a second image.

[0139] The image processing apparatus 800 described above will now be described in detail as follows:

[0140] In one embodiment, the key point recognition module 802 includes:

[0141] The first recognition submodule is used to input the first image into the key point recognition model, and to identify multiple key points in the first image corresponding to the target object through the key point recognition model, so as to obtain key point information corresponding to the target object.

[0142] In one embodiment, the target object includes a person, and the key point recognition model includes a face key point recognition sub-model and a human body key point recognition sub-model.

[0143] The first identification submodule includes:

[0144] The first recognition unit is used to input the first image into the facial key point recognition sub-model, and to recognize the facial key points corresponding to the facial region of the person in the first image through the facial key point recognition sub-model, so as to obtain facial key point information corresponding to the facial region.

[0145] The second recognition unit is used to input the first image into the human body key point recognition sub-model, and use the human body key point recognition sub-model to identify the human body key points corresponding to the human body area of ​​the person in the first image, and obtain human body key point information corresponding to the human body area.

[0146] The key point determination unit is used to determine the key point information corresponding to the person based on the facial key point information and the human body key point information.

[0147] In one embodiment, the key point determination unit includes:

[0148] The first replacement subunit is used to replace the target human body key point information corresponding to the facial region in the human body key point information with the face key point information to obtain key point information corresponding to the person.

[0149] In one embodiment, the first identification unit includes:

[0150] The first recognition subunit is used to input the first image into the facial key point recognition submodel, and to identify multiple facial key points corresponding to the facial region of the person in the first image through the facial key point recognition submodel, so as to obtain facial key point information corresponding to the multiple facial key points.

[0151] The proportion calculation subunit is used to calculate the ratio of the facial region of the person to the area of ​​the first image;

[0152] The key point processing subunit is used to take the facial region as the target facial key point when the ratio of the areas is less than a preset ratio, and to determine the target facial key point information corresponding to the target facial key point based on the facial key point information corresponding to the plurality of facial key points.

[0153] The first determining subunit is used to determine the target facial key point information as facial key point information corresponding to the facial region.

[0154] In one embodiment, the key point determination unit includes:

[0155] The second determining subunit is used to determine whether the first image includes the entire body area of ​​the person;

[0156] The second replacement subunit is used to replace the target human body key point information corresponding to the facial region in the human body key point information with the target face key point information when it is determined that the first image includes the entire body area of ​​the person, so as to obtain key point information corresponding to the person.

[0157] In one embodiment, the region identification module 803 includes:

[0158] The second recognition submodule is used to input the first image and the key point information into the fuzziness recognition model, and the fuzziness recognition model identifies at least one fuzzy region in the first image based on the key point information, and obtains the region position and region fuzziness corresponding to the at least one fuzzy region respectively.

[0159] In one embodiment, the second identification submodule includes:

[0160] An angle acquisition unit is used to acquire head angle information corresponding to the target object in the first image, wherein the head angle information includes at least one of rotation angle, pitch angle and tilt angle;

[0161] A heatmap generation unit is used to generate a heatmap of key points based on the key point information and the head angle information.

[0162] The region identification unit is used to input the first image and the key point heatmap into the fuzziness identification model, and the fuzziness identification model identifies at least one fuzzy region in the first image based on the key point heatmap, and obtains the region location and region fuzziness corresponding to the at least one fuzzy region respectively.

[0163] In one embodiment, the second identification submodule further includes:

[0164] The location region unit is used to determine the target region position in the first image corresponding to the target object based on the key point position of each of the plurality of key points before the first image and the key point heatmap are input to the fuzziness recognition model, and the fuzziness recognition model identifies at least one fuzzy region in the first image based on the key point heatmap to obtain the region position and region fuzziness corresponding to the at least one fuzzy region respectively.

[0165] The region identification unit includes:

[0166] The second recognition subunit is used to input the first image, the key point heatmap, and the target region location into the fuzziness recognition model. The fuzziness recognition model identifies at least one fuzzy region in the first image based on the key point heatmap and the target region location, and obtains the region location and region fuzziness corresponding to the at least one fuzzy region.

[0167] In one embodiment, the image processing module 804 includes:

[0168] The image classification submodule is used to classify the first image based on the region ambiguity to obtain the target ambiguity type corresponding to the first image;

[0169] The model acquisition submodule is used to acquire the target image processing model corresponding to the target ambiguity type;

[0170] The blur removal submodule is used to input the first image and the region location into the target image processing model, and then use the target image processing model to perform blur removal processing on the first image according to the region location to obtain the second image.

[0171] Therefore, by identifying the key point positions and key point confidence levels of multiple key points corresponding to the target object in the first image, and using these key point positions and key point confidence levels, it is possible to identify the region position and region blur degree corresponding to at least one blurred region in the first image. Thus, when performing blur removal processing on the first image based on the region position and region blur degree, the blurred region in the first image can be accurately located, and the blurred region can be accurately blurred without introducing excessive noise, thereby improving the effect of image deblurring and improving image quality.

[0172] Figure 9 This is a structural block diagram of a model training device provided in one embodiment of this application.

[0173] like Figure 9 As shown, the model training device 900 may include:

[0174] The label acquisition module 901 is used to acquire a sample image and its corresponding image label. The sample image includes a first object, and the image label includes a region location label and a region blur label of a blurred region in the sample image.

[0175] The first recognition module 902 is used to recognize multiple key points in the sample image corresponding to the first object and obtain key point information corresponding to the first object, wherein the key point information includes the key point position and key point confidence of each of the multiple key points;

[0176] The second recognition module 903 is used to input the sample image and the key point information into the target neural network model, and the target neural network model recognizes at least one blurred region in the sample image based on the key point information, and obtains the region position and region blur degree corresponding to the at least one blurred region respectively.

[0177] The model training module 904 is used to train the target neural network model according to the image label based on the region location and the region ambiguity, so as to obtain the ambiguity recognition model.

[0178] Thus, by acquiring sample images and their corresponding image labels, and then using the key point information corresponding to the first object in the identified sample images as input features of the target neural network model, and combining the image labels to train the target neural network model, a fuzziness recognition model that can identify the location and fuzziness of the corresponding region is obtained. This allows users to easily use the fuzziness recognition model to accurately locate the fuzzy region in the image and perform accurate fuzz removal processing on the fuzzy region without introducing too much noise, thereby providing a model basis for improving the effect of image deblurring and improving image quality.

[0179] The image processing device and model training device in the embodiments of this application can be electronic devices or components within electronic devices, such as integrated circuits or chips. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. The embodiments of this application do not impose specific limitations.

[0180] The image processing device and model training device in the embodiments of this application can be devices with an operating system. The operating system can be Android, iOS, or other possible operating systems, and this application does not specifically limit them.

[0181] The image processing device and model training device provided in the embodiments of this application can achieve... Figures 1 to 7 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0182] Optional, such as Figure 10As shown, this application embodiment also provides an electronic device 1000, including a processor 1001 and a memory 1002. The memory 1002 stores a program or instructions that can be executed by the processor 1001. When the program or instructions are executed by the processor 1001, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0183] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0184] Figure 11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0185] The electronic device 1100 includes, but is not limited to, components such as: radio frequency unit 1101, network module 1102, audio output unit 1103, input unit 1104, sensor 1105, display unit 1106, user input unit 1107, interface unit 1108, memory 1109, and processor 1110.

[0186] Those skilled in the art will understand that the electronic device 1100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0187] The processor 1110 is configured to acquire a first image, the first image including a target object; identify multiple key points in the first image corresponding to the target object, and obtain key point information corresponding to the target object, wherein the key point information includes the key point position and key point confidence of each of the multiple key points; identify at least one blurred region in the first image based on the key point information, and obtain the region position and region blur degree corresponding to the at least one blurred region respectively; and perform blur removal processing on the first image based on the region position and the region blur degree to obtain a second image.

[0188] Therefore, by identifying the key point positions and key point confidence levels of multiple key points corresponding to the target object in the first image, and using these key point positions and key point confidence levels, it is possible to identify the region position and region blur degree corresponding to at least one blurred region in the first image. Thus, when performing blur removal processing on the first image based on the region position and region blur degree, the blurred region in the first image can be accurately located, and the blurred region can be accurately blurred without introducing excessive noise, thereby improving the effect of image deblurring and improving image quality.

[0189] Optionally, the processor 1110 is further configured to input the first image into a key point recognition model, and identify multiple key points in the first image corresponding to the target object through the key point recognition model to obtain key point information corresponding to the target object.

[0190] Optionally, the processor 1110 is further configured to input the first image into the facial key point recognition sub-model, identify facial key points corresponding to the facial region of the person in the first image through the facial key point recognition sub-model, and obtain facial key point information corresponding to the facial region; input the first image into the human body key point recognition sub-model, identify human body key points corresponding to the human body region of the person in the first image through the human body key point recognition sub-model, and obtain human body key point information corresponding to the human body region; and determine key point information corresponding to the person based on the facial key point information and the human body key point information.

[0191] Optionally, the processor 1110 is further configured to replace the target human body key point information corresponding to the facial region in the human body key point information with the face key point information to obtain key point information corresponding to the person.

[0192] Optionally, the processor 1110 is further configured to input the first image into the facial landmark recognition sub-model, identify multiple facial landmarks corresponding to the facial region of the person in the first image through the facial landmark recognition sub-model, and obtain facial landmark information corresponding to the multiple facial landmarks; calculate the ratio of the area of ​​the person's facial region to the area of ​​the first image; if the area ratio is less than a preset ratio, take the facial region as the target facial landmark, and determine the target facial landmark information corresponding to the target facial landmark based on the facial landmark information corresponding to the multiple facial landmarks; and determine the target facial landmark information as the facial landmark information corresponding to the facial region.

[0193] Optionally, the processor 1110 is further configured to determine whether the first image includes the entire body area of ​​the person; if it is determined that the first image includes the entire body area of ​​the person, the target human body key point information corresponding to the facial area in the human body key point information is replaced with the target facial key point information to obtain key point information corresponding to the person.

[0194] Optionally, the processor 1110 is further configured to input the first image and the key point information into a fuzziness recognition model, wherein the fuzziness recognition model identifies at least one fuzzy region in the first image based on the key point information, and obtains the region location and region fuzziness corresponding to the at least one fuzzy region respectively.

[0195] Optionally, the processor 1110 is further configured to acquire head angle information corresponding to the target object in the first image, wherein the head angle information includes at least one of rotation angle, pitch angle, and tilt angle; generate a key point heatmap based on the key point information and the head angle information; input the first image and the key point heatmap to the fuzziness recognition model, and have the fuzziness recognition model identify at least one fuzzy region in the first image based on the key point heatmap to obtain the region position and region fuzziness corresponding to the at least one fuzzy region respectively.

[0196] Optionally, the processor 1110 is further configured to determine the location of a target region in the first image corresponding to the target object based on the key point location of each of the plurality of key points; input the first image, the key point heatmap, and the target region location to the fuzziness recognition model, and have the fuzziness recognition model identify at least one fuzzy region in the first image based on the key point heatmap and the target region location, thereby obtaining the region location and region fuzziness corresponding to the at least one fuzzy region respectively.

[0197] Optionally, the processor 1110 is further configured to classify the first image according to the region ambiguity to obtain a target ambiguity type corresponding to the first image; obtain a target image processing model corresponding to the target ambiguity type; input the first image and the region location into the target image processing model, and perform ambiguity removal processing on the first image according to the region location through the target image processing model to obtain the second image.

[0198] Therefore, by selecting different key point recognition models for different areas of a person to perform targeted key point recognition, the key point information obtained can be more accurate, thereby improving the accuracy of fuzzy area recognition in subsequent steps.

[0199] Additionally, the processor 1110 can also be used to acquire sample images and their corresponding image labels, wherein the sample images include a first object, and the image labels include a region location label and a region ambiguity label for a blurred region in the sample images; identify multiple key points in the sample images corresponding to the first object to obtain key point information corresponding to the first object, wherein the key point information includes the key point location and key point confidence of each of the multiple key points; input the sample images and the key point information into a target neural network model, and the target neural network model identifies at least one blurred region in the sample images based on the key point information to obtain the region location and region ambiguity corresponding to the at least one blurred region; and train the target neural network model according to the image labels based on the region location and the region ambiguity to obtain an ambiguity recognition model.

[0200] Thus, by acquiring sample images and their corresponding image labels, and then using the key point information corresponding to the first object in the identified sample images as input features of the target neural network model, and combining the image labels to train the target neural network model, a fuzziness recognition model that can identify the location and fuzziness of the corresponding region is obtained. This allows users to easily use the fuzziness recognition model to accurately locate the fuzzy region in the image and perform accurate fuzz removal processing on the fuzzy region without introducing too much noise, thereby providing a model basis for improving the effect of image deblurring and improving image quality.

[0201] It should be understood that, in this embodiment, the input unit 1104 may include a graphics processing unit (GPU) 11041 and a microphone 11042. The GPU 11041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1106 may include a display panel 11061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0202] The memory 1109 can be used to store software programs and various data. The memory 1109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1109 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0203] Processor 1110 may include one or more processing units; optionally, processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1110.

[0204] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0205] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0206] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0207] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0208] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0209] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0210] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0211] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a first image, the first image comprising a target object; identifying a plurality of key points in the first image corresponding to the target object to obtain key point information corresponding to the target object, wherein the key point information comprises a key point position and a key point confidence of each key point in the plurality of key points; identifying at least one blur region in the first image according to the key point information to obtain a region position and a region blur degree corresponding to the at least one blur region, respectively; performing blur removal processing on the first image based on the region position and the region blur degree to obtain a second image; the identifying at least one blur region in the first image according to the key point information to obtain a region position and a region blur degree corresponding to the at least one blur region, respectively, comprises: inputting the first image and the key point information into a blur degree identification model, and identifying at least one blur region in the first image according to the key point information by the blur degree identification model to obtain a region position and a region blur degree corresponding to the at least one blur region, respectively.

2. The method of claim 1, wherein, the identifying a plurality of key points in the first image corresponding to the target object to obtain key point information corresponding to the target object comprises: inputting the first image into a key point identification model to identify a plurality of key points in the first image corresponding to the target object by the key point identification model to obtain key point information corresponding to the target object.

3. The method of claim 2, wherein, The target object comprises a person, and the key point identification model comprises a face key point identification sub-model and a human body key point identification sub-model. the inputting the first image into a key point identification model to identify a plurality of key points in the first image corresponding to the target object by the key point identification model to obtain key point information corresponding to the target object comprises: inputting the first image into the face key point identification sub-model to identify face key points corresponding to a face region of the person in the first image by the face key point identification sub-model to obtain face key point information corresponding to the face region; inputting the first image into the human body key point identification sub-model to identify human body key points corresponding to a human body region of the person in the first image by the human body key point identification sub-model to obtain human body key point information corresponding to the human body region; determining key point information corresponding to the person according to the face key point information and the human body key point information.

4. The method of claim 3, wherein, the determining key point information corresponding to the person according to the face key point information and the human body key point information comprises: replacing target human body key point information corresponding to the face region in the human body key point information with the face key point information to obtain the key point information corresponding to the person.

5. The method of claim 3, wherein, the inputting the first image into the face key point identification sub-model to identify face key points corresponding to a face region of the person in the first image by the face key point identification sub-model to obtain face key point information corresponding to the face region comprises: input the first image into the face key point recognition sub-model, identify a plurality of face key points corresponding to a face region of the person in the first image through the face key point recognition sub-model, and obtain face key point information corresponding to the plurality of face key points; calculate a ratio of an area of the face region of the person to the first image; in a case where the ratio of the area is less than a preset proportion, take the face region as a target face key point, and determine target face key point information corresponding to the target face key point based on the face key point information corresponding to the plurality of face key points; determine whether the first image includes all body regions of the person; 6. The method of claim 5, wherein, in a case where it is determined that the first image includes all body regions of the person, replace target body key point information corresponding to the face region in the body key point information with the target face key point information to obtain key point information corresponding to the person. input the first image and the key point information into a blur degree recognition model, identify at least one blurred region in the first image according to the key point information through the blur degree recognition model, and obtain a region position and a region blur degree corresponding to the at least one blurred region, respectively, including: obtain head angle information corresponding to the target object in the first image, wherein the head angle information includes at least one of a roll angle, a pitch angle, and a yaw angle; 7. The method of claim 1, wherein, generate a key point heat map according to the key point information and the head angle information; input the first image and the key point heat map into the blur degree recognition model, identify at least one blurred region in the first image according to the key point heat map through the blur degree recognition model, and obtain a region position and a region blur degree corresponding to the at least one blurred region, respectively. before inputting the first image and the key point heat map into the blur degree recognition model, identifying at least one blurred region in the first image according to the key point heat map through the blur degree recognition model, and obtaining a region position and a region blur degree corresponding to the at least one blurred region, respectively, the method further includes: determine a target region position corresponding to the target object in the first image according to a key point position of each key point in the plurality of key points; 8. The method of claim 7, wherein, input the first image and the key point heat map into the blur degree recognition model, identify at least one blurred region in the first image according to the key point heat map through the blur degree recognition model, and obtain a region position and a region blur degree corresponding to the at least one blurred region, respectively, including: ​ ​ input the first image, the key point heat map and the target region position to the blur degree identification model, and identify at least one blurred region in the first image according to the key point heat map and the target region position by the blur degree identification model, to obtain a region position and a region blur degree corresponding to the at least one blurred region respectively.

9. The method of claim 1, wherein, The blur removal processing on the first image based on the region position and the region blur degree to obtain a second image, comprising: classifying the blur degree of the first image according to the region blur degree to obtain a target blur degree type corresponding to the first image; obtaining a target image processing model corresponding to the target blur degree type; inputting the first image and the region position into the target image processing model, and performing blur removal processing on the first image according to the region position by the target image processing model to obtain the second image.

10. A model training method, comprising: comprising: obtaining a sample image and an image label corresponding thereto, the sample image comprising a first object, and the image label comprising a region position label and a region blur degree label of a blurred region in the sample image; identifying a plurality of key points corresponding to the first object in the sample image to obtain key point information corresponding to the first object, wherein the key point information comprises a key point position and a key point confidence of each key point in the plurality of key points; inputting the sample image and the key point information into a target neural network model, and identifying at least one blurred region in the sample image according to the key point information by the target neural network model to obtain a region position and a region blur degree corresponding to the at least one blurred region respectively; training the target neural network model according to the image label based on the region position and the region blur degree to obtain a blur degree identification model.

11. An image processing apparatus characterized by comprising: comprising: an image acquisition module configured to acquire a first image, the first image comprising a target object; a key point identification module configured to identify a plurality of key points corresponding to the target object in the first image to obtain key point information corresponding to the target object, wherein the key point information comprises a key point position and a key point confidence of each key point in the plurality of key points; a region identification module configured to identify at least one blurred region in the first image according to the key point information to obtain a region position and a region blur degree corresponding to the at least one blurred region respectively; an image processing module configured to perform blur removal processing on the first image based on the region position and the region blur degree to obtain a second image; the region identification module is specifically configured to input the first image and the key point information into a blur degree identification model, and identify at least one blurred region in the first image according to the key point information by the blur degree identification model to obtain a region position and a region blur degree corresponding to the at least one blurred region respectively.

12. A model training apparatus, comprising: comprising: A label obtaining module is configured to obtain a sample image and an image label corresponding to the sample image, the sample image including a first object, and the image label including a region position label and a region blur degree label of a blur region in the sample image; A first identification module is configured to identify a plurality of key points corresponding to the first object in the sample image, to obtain key point information corresponding to the first object, wherein the key point information includes a key point position and a key point confidence of each key point in the plurality of key points; A second identification module is configured to input the sample image and the key point information into a target neural network model, to identify at least one blur region in the sample image according to the key point information by using the target neural network model, and to obtain a region position and a region blur degree corresponding to the at least one blur region, respectively. A model training module is configured to train the target neural network model according to the image label based on the region position and the region blur degree, to obtain a blur degree identification model.

13. An electronic device, comprising: A processor and a memory are included, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the method according to any one of claims 1-10.

14. A readable storage medium, characterized by, The readable storage medium stores programs or instructions, the programs or instructions being executed by the processor to implement the steps of the method according to any one of claims 1-10.

Images