Image data processing method, device, equipment and medium

Abstract

The embodiments of the present application provide an image data processing method, apparatus, device, and medium, including: outputting a first classification result of a sample image through an initial image recognition model, generating a first activation map based on the first classification result and the sample convolution feature of the sample image; performing data transformation on the sample image to obtain a deformed image, outputting a second classification result of the deformed image through the initial image recognition model, generating a second activation map based on the second classification result and the deformed convolution feature of the deformed image; determining a similarity loss result based on the first activation map and the second activation map, and determining a classification loss result based on the first classification result, the second classification result, and the key point category labels of the sample image; and correcting the network parameters of the initial image recognition model based on the similarity loss result and the classification loss result to generate a target image recognition model. The present application can improve image processing efficiency and the positioning accuracy of the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an image data processing method, apparatus, device, and medium. Background Art

[0002] Human posture estimation can detect the positions of various human joints and bones in images or videos, and has very wide application value in film animation, virtual reality, video surveillance, motion recognition and other fields.

[0003] Current human pose estimation algorithms train models by outputting predicted position information for each key point on the human body and calculating the distance loss between the predicted and true positions. This model training relies on labeled data: the images or videos used to train the model must be labeled with the true position information for each key point on the human body. The closer the predicted position information output by the model is to the true position information, the better the model's recognition performance. This indicates that the images and videos used for model training are limited. The true position information for each key point on the human body must be labeled. However, labeling images or videos is extremely time-consuming and labor-intensive, resulting in low image or video processing efficiency. Furthermore, the human body is inherently flexible and may exhibit complex poses in real applications (which may not appear in the images or videos used to train the model). Consequently, the trained model's prediction accuracy for key points on the human body is low. Summary of the Invention

[0004] The embodiments of the present application provide an image data processing method, apparatus, device, and medium, which can improve image processing efficiency and enhance the positioning accuracy of a model.

[0005] An embodiment of the present application provides an image data processing method, including:

[0006] Obtaining a sample image, outputting a first classification result corresponding to the sample image through an initial image recognition model, and generating a first activation map based on the first classification result and sample convolution features of the sample image; the first classification result is determined by target pose features corresponding to a sample object in the sample image, and the first activation map is used to represent position information of key points of the sample object in the sample image;

[0007] Performing data transformation on the sample image to obtain a deformed image, outputting a second classification result corresponding to the deformed image through the initial image recognition model, and generating a second activation map based on the second classification result and the deformed convolution features of the deformed image; the second classification result is determined by the deformed posture features corresponding to the sample object in the deformed image, and the second activation map is used to represent the position information of the key points of the sample object in the deformed image;

[0008] Determining a similarity loss result of the initial image recognition model based on the first activation map and the second activation map, and determining a classification loss result of the initial image recognition model based on the first classification result, the second classification result, and the key point category labels carried by the sample image;

[0009] Based on the similarity loss results and the classification loss results, the network parameters of the initial image recognition model are modified to generate a target image recognition model; the target image recognition model is used to predict the object part category and positioning results corresponding to the key points in the source image.

[0010] In one aspect, an embodiment of the present application provides an image data processing device, including:

[0011] a first generation module, configured to obtain a sample image, output a first classification result corresponding to the sample image through an initial image recognition model, and generate a first activation map based on the first classification result and a sample convolution feature of the sample image; the first classification result is determined by a target posture feature corresponding to a sample object in the sample image, and the first activation map is used to represent position information of key points of the sample object in the sample image;

[0012] a second generation module for performing data transformation on the sample image to obtain a deformed image, outputting a second classification result corresponding to the deformed image through the initial image recognition model, and generating a second activation map based on the second classification result and the deformed convolution features of the deformed image; the second classification result is determined by the deformed posture features corresponding to the sample object in the deformed image, and the second activation map is used to represent position information of key points of the sample object in the deformed image;

[0013] a loss result determination module, configured to determine a similarity loss result of the initial image recognition model based on the first activation map and the second activation map, and to determine a classification loss result of the initial image recognition model based on the first classification result, the second classification result, and the key point category labels carried by the sample image;

[0014] The parameter correction module is used to correct the network parameters of the initial image recognition model based on the similarity loss results and the classification loss results to generate a target image recognition model; the target image recognition model is used to predict the object component category and positioning results corresponding to the key points in the source image.

[0015] The first generation module includes:

[0016] A feature extraction unit is used to input the sample image into the initial image recognition model, and obtain the target posture feature corresponding to the sample object in the sample image according to the initial image recognition model;

[0017] A feature classification unit, configured to identify the target posture features according to the classifier in the initial image recognition model, and obtain a first classification result corresponding to the sample image;

[0018] a feature mapping unit, configured to obtain a sample convolution feature for the sample image output by the target convolution layer in the initial image recognition model, and perform a product operation on the first classification result and the sample convolution feature to obtain a candidate activation map corresponding to the sample image;

[0019] The upsampling processing unit is configured to perform upsampling processing on the candidate activation map to obtain a first activation map having the same image size as the sample image.

[0020] The feature extraction unit includes:

[0021] The global classification subunit is used to obtain the global posture features corresponding to the sample objects in the sample images in the initial image recognition model, and output the global classification results corresponding to the global posture features through the classifier in the initial image recognition model;

[0022] The block processing subunit is used to perform a product operation on the global classification result and the sample convolution feature to obtain a global mapping map corresponding to the sample image, and to perform block processing on the sample image according to the global mapping map to obtain M local area images; M is a positive integer;

[0023] A local feature extraction subunit is used to sequentially input the M local region images into the initial image recognition model, and obtain local posture features corresponding to the M local region images in the initial image recognition model;

[0024] The feature combination subunit is used to perform feature combination on the global posture feature and the local posture features corresponding to the M local area images to obtain the target posture feature corresponding to the sample object in the sample image.

[0025] The initial image recognition model includes N residual components, each residual component includes one or more convolutional layers, and N is a positive integer;

[0026] The global classification subunit is specifically used for:

[0027] Get the input feature of the i-th residual component among N residual components; when i is 1, the input feature of the i-th residual component is the sample image, and i is a positive integer less than N;

[0028] According to one or more convolutional layers in the i-th residual component, the input features of the i-th residual component are convolved to obtain candidate convolution features;

[0029] Combine the candidate convolutional features and the input features of the i-th residual component to obtain the residual output features of the i-th residual component, and use the residual output features of the i-th residual component as the input features of the (i+1)-th residual component; the i-th residual component is connected to the (i+1)-th residual component;

[0030] The residual output feature of the Nth residual component is determined as the global posture feature corresponding to the sample object in the sample image.

[0031] Among them, the number of global posture features is K, K is a positive integer;

[0032] The global classification subunit is specifically used for:

[0033] Count the feature averages corresponding to the K global posture features, and combine the feature averages corresponding to the K global posture features into a global feature vector;

[0034] According to the activation function in the initial image recognition model, the global feature vector is converted into the feature vector to be classified;

[0035] The feature vector to be classified is input into the classifier in the initial image recognition model, and the classifier in the initial image recognition model outputs the global classification result corresponding to the feature vector to be classified.

[0036] Among them, the loss result determination module includes:

[0037] a data transformation unit, configured to perform data transformation on the second activation map to obtain a deformed activation map;

[0038] The similarity constraint unit is used to perform similarity constraint on the first activation map and the deformed activation map to determine a similarity loss result of the initial image recognition model.

[0039] Among them, the loss result determination module includes:

[0040] a sample loss determination unit, configured to obtain a first error between the first classification result and the key point category label carried by the sample image, and determine a sample loss result of the initial image recognition model based on the first error;

[0041] a deformation loss determining unit, configured to obtain a second error between the second classification result and the key point category label, and determine a deformation loss result of the initial image recognition model based on the second error;

[0042] The classification loss determination unit is used to determine the classification loss result of the initial image recognition model according to the sample loss result and the deformation loss result.

[0043] Among them, the parameter correction module includes:

[0044] A total loss determination unit, configured to determine a total model loss result corresponding to the initial image recognition model based on the similarity loss result and the classification loss result;

[0045] The network parameter adjustment unit is used to correct the network parameters of the initial image recognition model by minimizing the total loss result of the model, and determine the initial image recognition model including the corrected network parameters as the target image recognition model.

[0046] The device further comprises:

[0047] The object component classification module is used to obtain a source image, obtain the object posture features corresponding to the target object in the source image through the target image recognition model, and identify the object component classification results corresponding to the object posture features; the object component classification results are used to characterize the object part categories corresponding to the key points of the target object;

[0048] An object map generation module is used to generate an object part map based on the object part classification results and the object convolution features of the source image;

[0049] The positioning result determination module is used to obtain the pixel average value corresponding to the object part mapping image, determine the positioning result of the key points in the target object in the source image based on the pixel average value, and determine the posture estimation result corresponding to the target object in the source image based on the object part category and the positioning result.

[0050] Among them, the object component classification module includes:

[0051] A global object classification unit is configured to input a source image into a target image recognition model, obtain global object features corresponding to a target object in the source image in the target image recognition model, and output a global object classification result corresponding to the global object features based on a classifier in the target image recognition model;

[0052] A global map generation unit is used to obtain the object convolution features for the source image output by the target convolution layer in the target image recognition model, and multiply the global object classification result and the object convolution features to obtain a global object map corresponding to the source image;

[0053] A component feature acquisition unit is used to perform block processing on the source image according to the global object map to obtain M object component region images, and obtain object component features corresponding to the M object component region images according to the target image recognition model; M is a positive integer;

[0054] The component feature combination unit is used to combine the global object feature and the object component features corresponding to the M object component region images into an object posture feature.

[0055] The device further comprises:

[0056] The audit module is used to determine that the audit result of the source image in the content audit system is a passed audit result when the posture estimation result is the same as the posture of the target object in the content audit system, and to set access rights for the content audit system for the object corresponding to the source image.

[0057] In one aspect, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory is connected to the processor, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method provided in the above aspect of the embodiment of the present application.

[0058] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded and executed by a processor, so that a computer device with a processor executes the method provided in the above aspect of the embodiment of the present application.

[0059] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in the above aspect.

[0060] The embodiment of the present application can extract the target posture features in the sample image through the initial image recognition model, obtain the first classification result of the target posture features by classifying and identifying the target posture features, and generate a first activation map based on the first classification result and the sample convolution features of the sample image; at the same time, the sample image can also be transformed to obtain a deformed image, and the deformed posture features in the deformed image can be extracted through the initial image recognition model, and a second activation map is generated based on the second classification result of the deformed posture features and the deformed convolution features of the deformed image; and then similarity constraints (i.e., similarity loss results) can be imposed on the first activation map and the second activation map, so that the target image recognition model trained in this way can improve the positioning accuracy of the key points in the image; in addition, when training the initial image recognition model, there is no need to mark the position information of each key point of the sample object in the sample image, that is, the sample image is obtained from a wider source, and the labeling operation of the key point positions of the sample image can be reduced, thereby improving the image processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0062] Figure 1 This is a schematic diagram of a network architecture provided by an embodiment of the present application;

[0063] Figure 2 This is a schematic diagram of a human body posture estimation scenario provided by an embodiment of the present application;

[0064] Figure 3 This is a flow chart of an image data processing method provided in an embodiment of the present application;

[0065] Figure 4 This is a schematic diagram of data transformation of a sample image provided by an embodiment of the present application;

[0066] Figure 5 This is a training diagram of an initial image recognition model provided in an embodiment of the present application;

[0067] Figure 6 This is a flow chart of an image data processing method provided in an embodiment of the present application;

[0068] Figure 7 This is a schematic diagram of a global average pooling process provided in an embodiment of the present application;

[0069] Figure 8 This is a training diagram of an initial image recognition model provided in an embodiment of the present application;

[0070] Figure 9 This is a flow chart of an image data processing method provided in an embodiment of the present application;

[0071] Figure 10 This is a schematic diagram of a scene for human posture estimation provided by an embodiment of the present application;

[0072] Figure 11 is a structural diagram of an image data processing device provided in an embodiment of the present application;

[0073] Figure 12 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0075] The present application relates to computer vision technology (Computer Vision, CV). Computer vision is a science that studies how to make machines "see". More specifically, it refers to machine vision that uses cameras and computers to replace human eyes to identify, track and measure targets, and further performs graphic processing so that the computer processing becomes an image that is more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. The present application relates to human pose estimation (Human Pose Estimation) under computer vision technology. Human pose estimation is an important task in computer vision and an indispensable step for computers to understand human movements and behaviors. Human pose estimation can be converted into a prediction problem for key points of the human body, such as predicting the position coordinates of each key point of the human body in the image, and predicting the human skeleton in the image based on the positional relationship between each key point of the human body.

[0076] See Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided by an embodiment of the present application. Figure 1 As shown, the network architecture may include a server 10d and a user terminal cluster, and the user terminal cluster may include one or more user terminals, and the number of user terminals is not limited here. Figure 1 As shown, the user terminal cluster may specifically include user terminal 10a, user terminal 10b and user terminal 10c, etc. Among them, server 10d may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. User terminal 10a, user terminal 10b and user terminal 10c, etc. may include: smart phones, tablet computers, laptop computers, PDAs, mobile internet devices (MID), wearable devices (such as smart watches, smart bracelets, etc.) and smart TVs and other smart terminals. As Figure 1As shown, the user terminal 10a, the user terminal 10b and the user terminal 10c can respectively establish a network connection with the server 10d, so that each user terminal can exchange data with the server 10d through the network connection.

[0077] like Figure 1 The server 10d shown can obtain the source image that needs to be recognized, and use the target image recognition model to output the object component category and positioning results corresponding to each key point of the target object in the source image. The pose estimation result corresponding to the target object in the source image can be determined based on the positioning results and object component category. The server 10d can also train the initial image recognition model using a large number of sample images carrying key point category labels. The initial image recognition model after training is called the target image recognition model. Figure 1 Any user terminal in the user terminal cluster shown can be used to display the pose estimation result of the source image.

[0078] Among them, the above-mentioned initial image recognition model may refer to an image recognition model that has not yet completed training during the training process, the image recognition model here can be used to perform human posture estimation on images or videos, and the target image recognition model may refer to the initial image recognition model after the training is completed; the above-mentioned source image and sample image may both refer to human body images, and the target object in the source image and the sample object in the sample image may both refer to human bodies, various animals, etc.; each key point may refer to each joint point of the human body. It should be noted that the sample image in this application may refer to a human body image that only carries key point category labels, that is, when training the initial image recognition model, the training process for each key point category can be considered as supervised learning (each sample image has a label indicating its true key point category label), and the training process for each key point position can be considered as unsupervised learning (each sample image does not carry a true position label). Therefore, the entire training process of the initial image recognition model can be considered as a weakly supervised learning, that is, the sample image only carries partial label information (key point category label). Optionally, the human posture estimation of the picture or video can be performed by the server 10d, or by Figure 1 The process may be performed by any one of the user terminals shown.

[0079] See Figure 2 , Figure 2 This is a schematic diagram of a human posture estimation scenario provided by an embodiment of the present application. Taking the application of human posture estimation in behavior recognition scenario as an example, Figure 2 As shown, the server (for example, the above Figure 1The server 10d) shown in FIG. 1 can obtain video 20a. Video 20a can be human motion captured by a capture device, or can be a behavioral video downloaded directly from the internet. The capture device can be a video camera or a video camera of various types. The server can perform frame processing on video 20a to obtain multiple frames of image data. For example, images 20b, 20c, and 20d can be extracted from video 20a. These images 20b, 20c, and 20d can represent a series of human motions.

[0080] The server can obtain an image recognition model 20e, which can be a pre-trained human posture estimation model. The image recognition model 20e can be used to predict the category and location results of each human key point in the video 20a. The image recognition model 20e can be a convolutional neural network model. This application does not limit the type of the image recognition model 20e. The training process of the image recognition model 20e can be referred to as follows Figure 3 Corresponding embodiment.

[0081] The server can input picture 20b, picture 20c and picture 20d into the image recognition model 20e in sequence, and through the image recognition model 20e, it can obtain the posture feature 1 corresponding to picture 20b, the posture feature 2 corresponding to picture 20c and the posture feature 3 corresponding to picture 20c, and then the posture features corresponding to each picture can be input into the classifier 20f associated with the image recognition model 20e in sequence, and through the classifier 20f, it can output the classification result 1 corresponding to posture feature 1, the classification result 2 corresponding to posture feature 2, and the classification result 3 corresponding to posture feature 3 in sequence. For the classification result 1 corresponding to image 20b, the server can multiply the classification result 1 with the convolutional features output by the last convolutional layer in the image recognition model 20e to generate a feature image 20g (which can be called the class activation map (CAM) corresponding to image 20b. CAM is a tool for visualizing image features). The above classification result 1 can be used as the weight of the convolutional features output by the last convolutional layer in the image recognition model 20e. By weighting the classification result 1 and the convolutional features output by the last convolutional layer, the above feature image 20g can be obtained. The feature image 20g can refer to the result of visualizing the convolutional features output by the last convolutional layer. The feature image 20g can be used to represent the image pixel area of ​​interest to the image recognition model 20e (for example, area 20p in the feature image 20g). Similarly, for the classification result 2 corresponding to image 20c and the classification result 3 corresponding to image 20d, the server can use the same operation as above to obtain the feature image 20h corresponding to image 20c and the feature image 20i corresponding to image 20d.

[0082] The server can calculate the positioning results of each human key point in image 20b based on the pixel average of feature image 20g. Based on classification result 1 and the positioning results of each human key point in image 20b, the server can obtain a pose estimation result 20j corresponding to image 20b. This pose estimation result 20j can be used to represent the human skeleton in image 20b. Similarly, the server can calculate the positioning results of each human key point in image 20c based on the pixel average of feature image 20h. Based on classification result 2 and the positioning results of each human key point in image 20c, the server can obtain a pose estimation result 20k corresponding to image 20c. This pose estimation result 20k can be used to represent the human skeleton in image 20c. Based on the pixel average of feature image 20i, the server can calculate the positioning results of each human key point in image 20d. Based on classification result 3 and the positioning results of each human key point in image 20d, the server can obtain a pose estimation result 20m corresponding to image 20d. This pose estimation result 20m can be used to represent the human skeleton in image 20d. The server may determine that the behavior recognition result corresponding to the video 20a is: running 20n according to the posture estimation result 20j, the posture estimation result 20k, and the posture estimation result 20m.

[0083] In an embodiment of the present application, by performing human posture estimation on each picture frame in video 20a, the posture estimation results corresponding to each picture frame can be obtained, and then the behavior recognition results of video 20a can be determined based on the posture estimation results corresponding to each picture frame; in other words, the human posture estimation result is an indispensable step in the behavior recognition scenario, and the effect of human posture estimation will directly affect the accuracy of behavior recognition.

[0084] See Figure 3 , Figure 3 FIG is a flow chart of an image data processing method provided by an embodiment of the present application. It is understandable that the image data processing method can be executed by a computer device, which can be a server (for example, Figure 1 The server 10d in the corresponding embodiment), or a user terminal (for example, Figure 1 Any user terminal in the user terminal cluster shown), or a computer program (including program code); Figure 3 As shown, the image data processing method may include the following steps S101 to S104:

[0085] Step S101, obtain a sample image, output a first classification result corresponding to the sample image through an initial image recognition model, and generate a first activation map based on the first classification result and the sample convolution feature of the sample image; the first classification result is determined by the target posture feature corresponding to the sample object in the sample image, and the first activation map is used to represent the position information of the key points of the sample object in the sample image.

[0086] Specifically, a computer device can obtain sample images for training an initial image recognition model. The sample images can be grayscale images or RGB images. The RGB color model is a color standard, representing the colors of the three channels of red, green, and blue. There can be multiple sample images, each of which can contain a sample object. Each sample image can carry a keypoint category label, which can be used to indicate the category corresponding to each keypoint of the sample object contained in the sample image. The sample objects contained in the sample image can include, but are not limited to, humans and animals (e.g., monkeys, gorillas, dogs, etc.). The keypoints of the sample objects can refer to joints of the human body or animal. Different types of sample objects can correspond to different keypoints, and each type of sample object can correspond to a specific number of keypoints. For example, when the sample image is a human image, the keypoints of the sample object contained in the sample image can include 18 joints of the head, shoulders, upper limbs, lower limbs, etc. Connecting these keypoints can describe the human body posture in the sample image. The keypoint category labels carried by the sample image can include human head joints, human upper limb joints, human lower limb joints, etc.

[0087] The computer device can obtain an initialized image recognition model, that is, an initial image recognition model. During the training of the initial image recognition model, a single sample image or a batch of sample images can be input for each training. The present application does not limit the number of sample images input for each training. For any one of the multiple sample images, the computer device can input the sample image into the initial image recognition model. The initial image recognition model can obtain the target posture feature corresponding to the sample object in the sample image. The target posture feature can be used to describe the posture of the sample object in the sample image. The classifier of the initial image recognition model can recognize the target posture feature to obtain a first classification result corresponding to the target posture feature. Then, based on the first classification result and the sample convolution feature corresponding to the sample image, a first activation map can be generated. The first activation map can be used to characterize the position information of each key point of the sample object in the sample image. The sample convolution feature can refer to the convolution feature for the sample image output by the target convolution layer in the initial image recognition model. The target convolution layer can refer to the last convolution layer in the initial image recognition model.

[0088] The initial image recognition model may be a convolutional neural network, or a combined network model of a convolutional neural network and other neural networks (eg, a recurrent neural network). The initial image recognition model may include, but is not limited to: MobileNet V1 (a lightweight convolutional neural network), MobileNet V2 (an improvement on the aforementioned MobileNet V1, also a lightweight convolutional neural network), PoseNet (a visual positioning model that can locate the posture information of the human body through color images), ResNet (Residual Neural Network), DenseNet (Dense Convolutional Network), LSTM (Long Short-Term Memory), RNN (Rerrent Neural Network), GRU (Gate Recurrent Unit), or a combination model of any one or more of the above networks. The initial image recognition model may also be a network model designed based on actual needs. For example, the designed initial image recognition model can be used to extract multi-scale posture features (that is, the above-mentioned target posture features may include posture features of different sizes). These multi-scale posture features can be used to characterize sample objects of different scales, so that the initial image recognition model has better performance. This application does not limit the type of the initial image recognition model.

[0089] Optionally, after the computer device obtains the target pose feature corresponding to the sample object in the sample image according to the initial image recognition model and identifies the first classification result corresponding to the target pose feature, it can obtain the sample convolution feature for the sample image output by the target convolution layer in the initial image recognition model, and multiply the first classification result and the sample convolution feature to obtain a candidate activation map corresponding to the sample image; wherein, the target convolution layer can refer to the last convolution layer in the initial image recognition model. Since the size of the sample convolution feature output by the last convolution layer is smaller than the size of the sample image, that is, the size of the candidate activation map is smaller than the size of the sample image, the candidate activation map can be upsampled to obtain a first activation map having the same image size as the sample image. wherein, the first classification result can include probability values ​​of the key points of the sample object belonging to each class, and the probability values ​​in the first classification result can be considered as weights corresponding to the sample convolution features output by the target convolution layer. The sample convolution features are weighted based on the first classification result to visualize the pixel area of ​​interest of the initial image recognition model. The first activation map can be referred to as a class activation map (CAM). Optionally, the first activation map may also refer to an image obtained by superimposing the upsampling result and the sample image.

[0090] In step S102, data transformation is performed on the sample image to obtain a deformed image. A second classification result corresponding to the deformed image is outputted by the initial image recognition model. A second activation map is generated based on the second classification result and the deformed convolution features of the deformed image. The second classification result is determined by the deformed posture features corresponding to the sample object in the deformed image. The second activation map is used to represent the position information of the key points of the sample object in the deformed image.

[0091] Specifically, the computer device can perform data transformation on the sample data to obtain a deformed image corresponding to the sample image, where the number of deformed images can be one or more, and different deformed images can be obtained by performing different data transformations on the sample image. The data transformation may include perspective transformation (also known as projection transformation) and affine transformation, which may include but is not limited to: rotation, translation, scaling, shearing, reflection, and any combination of the above transformations in any order and number of times. This application does not limit the type of data transformation.

[0092] For one or more deformed images obtained through data transformation, the computer device can sequentially input the one or more deformed images into an initial image recognition model. The initial image recognition model can then obtain deformed posture features corresponding to the sample object in the deformed image. The classifier of the initial image recognition model can then recognize the deformed posture features to obtain a second classification result corresponding to the deformed image. A second activation map can then be generated based on the second classification result and the deformed convolutional features of the deformed image. In other words, the computer device's processing of the deformed image using the initial image recognition model is the same as the processing of the sample image described above, and will not be further described here. When the computer device uses a single transformation method to transform the sample image, a deformed image corresponding to the sample image can be obtained, and the initial image recognition model can generate a second activation map corresponding to the deformed image. When the computer device uses multiple transformation methods to transform the sample image, multiple deformed images corresponding to the sample image can be obtained, and the initial image recognition model can generate second activation maps corresponding to each of the multiple deformed images, i.e., one deformed image can correspond to one second activation map.

[0093] See Figure 4 , Figure 4 This is a data transformation diagram of a sample image provided by an embodiment of the present application. Figure 4 As shown, after the computer device obtains the sample image 30a, it can perform translation and reduction operations on the sample object in the sample image 30a to obtain a deformed image 30b corresponding to the sample image 30a; or, by performing a reflection operation on the sample image 30a, it can obtain a deformed image 30c corresponding to the sample image 30a; or, by performing a rotation operation on the sample image 30a, it can obtain a deformed image 30d corresponding to the sample image 30a; or, by performing a scaling operation on the sample image 30a, it can obtain a deformed image 30e corresponding to the sample image 30a. The computer device can sequentially input the deformed images 30b, 30c, 30d, and 30e into the initial image recognition model, and the initial image recognition model can obtain the second activation maps corresponding to each of the above deformed images.

[0094] Step S103: Determine the similarity loss result of the initial image recognition model based on the first activation map and the second activation map, and determine the classification loss result of the initial image recognition model based on the first classification result, the second classification result, and the key point category labels carried by the sample image.

[0095] Specifically, the computer device can determine the similarity loss result of the initial image recognition model based on the first activation map corresponding to the sample image and the second activation map corresponding to the deformed image after data transformation. The process of determining the above-mentioned similarity loss result may include: the computer device can perform the same data transformation on the second activation map to obtain a deformed activation map, and then perform similarity constraints (also known as consistency constraints) on the first activation map and the deformed activation map to determine the similarity loss result of the initial image recognition model. It can be understood that the sample image and the sample objects in each deformed image are the same object. Theoretically, the first activation map corresponding to the sample image and the deformed activation map should have the same position information. Therefore, when training the initial image recognition model, similarity constraints can be performed on the first activation map image and the deformed activation map, so that the model can learn the geometrically invariant position information of different key points to improve the positioning accuracy of the model.

[0096] Since the sample image carries the key point category labels corresponding to the sample objects it contains, the sample objects contained in the sample image and the deformed image after data transformation are the same. Therefore, the true categories of the key points in the sample object and the deformed image should be the same, and the true category of the key points is the key point category label carried by the sample image. The computer device can obtain a first difference between the first classification result and the key point category label, and determine the sample loss result of the initial image recognition model based on the first difference. The first difference can refer to the distance between the first classification result and the key point category label; similarly, a second difference between the second classification result and the key point category label can be obtained, and the deformation loss result of the initial image recognition model can be determined based on the second difference. The second difference can refer to the distance between the second classification result and the key point category label; the classification loss result of the initial image recognition model is determined based on the sample loss result and the deformation loss result. For example, the sample loss result L1 and the deformation loss result L2 can be added, and the added result (L1+L2) can be used as the classification loss result of the initial image recognition model; or a coefficient a can be set for the sample loss result L1, and a coefficient b can be set for the deformation loss result L2, and (a*L1+b*L2) can be used as the classification loss result of the initial image recognition model; this application does not limit the combination form of the sample loss result and the deformation loss result.

[0097] In step S104, based on the similarity loss result and the classification loss result, the network parameters of the initial image recognition model are modified to generate a target image recognition model; the target image recognition model is used to predict the object component category and positioning result corresponding to the key points in the source image.

[0098] Specifically, the computer device can correct the network parameters of the initial image recognition model based on the similarity loss result and the classification loss result, and determine the trained initial image recognition model as the target image recognition model.

[0099] Optionally, the computer device can determine the total model loss result corresponding to the initial image recognition model based on the similarity loss result and the classification loss result. The total model loss result can refer to the sum of the similarity loss result and the classification loss result, or can refer to the result obtained by multiplying the similarity loss result and the classification loss result by their respective corresponding coefficients and then adding them together. By minimizing the total model loss result of the initial image recognition model, the network parameters of the initial image recognition model can be corrected, and the initial image recognition model containing the corrected network parameters can be determined as the target image recognition model. The target image recognition model can refer to the trained initial image recognition model. In other words, by minimizing the total loss result of the model, the network parameters of the initial image recognition model are continuously trained, that is, the network parameters of the initial image recognition model are continuously adjusted. When the number of training times of the initial image recognition model reaches a preset maximum number of iterations, or the training of the initial image recognition model reaches convergence, the network parameters at this time can be saved, and the initial image recognition model containing the network parameters is determined as the target image recognition model; the target image recognition model can be used to predict the object component category and positioning result corresponding to the key point in the source image, and the object component category and positioning result can be used to determine the source image (for example, the above Figure 2 The pose estimation result corresponding to the target object in the picture 20b in the corresponding embodiment (for example, the above Figure 2 The posture estimation result 20j in the corresponding embodiment).

[0100] See Figure 5 , Figure 5 This is a training diagram of an initial image recognition model provided in an embodiment of the present application. Figure 5As shown, after obtaining a sample image 40a, the computer device can input the sample image 40a into a residual network 40b (here, the residual network 40b can be a ResNet network, which can be considered as the initial image recognition model mentioned above). The residual network 40b can obtain a target posture feature 40c. Recognizing the target posture feature 40c can obtain a first classification result 40d corresponding to the sample image 40a. The first classification result 40d is multiplied by the sample convolution feature output by the last convolution layer in the residual network 40b to obtain a feature image 40e (i.e., the first activation map mentioned above). The number of feature images 40e can be one or more, which is the same as the number of key point categories of the sample object contained in the sample image 40a. That is, the feature image 40e can include activation maps corresponding to each key point category in the sample image 40a. The computer device can determine the sample loss result 40g of the residual network 40b based on the first difference between the first classification result 40d and the key point category label 40f carried by the sample image 40a.

[0101] Furthermore, the computer device can rotate the sample object in sample image 40a to obtain a deformed image 40h. This deformed image 40h can then be input into the residual network 40b. The residual network 40b can then obtain a deformed posture feature 40i. Recognizing the deformed posture feature 40i can yield a second classification result 40j corresponding to the sample image 40a. Multiplying the second classification result 40j with the deformed convolutional feature output by the last convolutional layer in the residual network 40b can yield a feature image 40k (i.e., the aforementioned second activation map). Feature image 40k can include activation maps corresponding to each keypoint category in the deformed image 40h. The computer device can determine a deformation loss result 40m for the residual network 40b based on a second difference between the second classification result 40j and the keypoint category label 40f.

[0102] The computer device can also impose similarity constraints on the feature image 40e and the feature image 40k, and determine the similarity loss result 40n of the residual network 40b. The total model loss result of the residual network 40b can be determined through the sample loss result 40g, the deformation loss result 40m and the similarity loss result 40n. By minimizing and optimizing the total model loss result, the network parameters of the residual network 40b are continuously trained until the number of training reaches a preset maximum number of iterations (or the training reaches convergence) to obtain a trained target image recognition model.

[0103] In an embodiment of the present application, the target posture features in the sample image are extracted by the initial image recognition model, and the target posture features are classified and recognized to obtain a first classification result of the target posture features, and a first activation map is generated based on the first classification result and the sample convolution features of the sample image; at the same time, the sample image can also be transformed to obtain a deformed image, and the deformed posture features in the deformed image are extracted by the initial image recognition model, and a second activation map is generated based on the second classification result of the deformed posture features and the deformed convolution features of the deformed image; and then a similarity constraint (i.e., a similarity loss result) can be imposed on the first activation map and the second activation map, so that the target image recognition model trained in this way can improve the positioning accuracy of the key points in the image; in addition, when training the initial image recognition model, there is no need to mark the position information of each key point of the sample object in the sample image, that is, the sample image is obtained from a wider source, and the labeling operation of the key point positions of the sample image can be reduced, thereby improving the image processing efficiency.

[0104] See Figure 6 , Figure 6 : is a flowchart of an image data processing method provided by an embodiment of the present application. It can be understood that the image data processing method can be executed by a computer device, which can be a server, a user terminal, or a computer program (including program code); Figure 6 As shown, the image data processing method may include the following steps S201 to S208:

[0105] Step S201: obtain a sample image, obtain global posture features corresponding to the sample object in the sample image in the initial image recognition model, and output a global classification result corresponding to the global posture features through the classifier in the initial image recognition model.

[0106] Specifically, after the computer device obtains the sample image, the sample image can be input into the initial image recognition model. The initial image recognition model can obtain the global posture feature corresponding to the sample object in the sample image, and the global posture feature can be used to describe the overall posture of the sample object in the sample image; the global posture feature is identified through the classifier of the initial image recognition model to obtain the global classification result corresponding to the global posture feature.

[0107] In the following, the initial image recognition model is a residual network ResNet as an example. The initial image recognition model may include N residual components, each residual component may include one or more convolutional layers, where N is a positive integer, such as N can be 1, 2, ... The above-mentioned global posture feature extraction process may include: the computer device may obtain the input features of the i-th residual component among the N residual components, when i is 1, the input features of the i-th residual component may be the above-mentioned sample image, and i may be a positive integer less than N; optionally, before the N residual components of the initial image recognition model, the initial image recognition model may further include one or more independent convolutional layers, and the input features of the first convolutional layer (i is 1) may be the convolution features output by the sample image after passing through one or two independent convolutional layers in the initial image recognition model.

[0108] After one or more convolutional layers in the i-th residual component, the input features of the i-th residual component are convolved to obtain candidate convolution features, and then the candidate convolution features and the input features of the i-th candidate residual component can be combined (for example, the combination here can be feature addition) to obtain the residual output features of the i-th residual component, and the residual output features of the i-th residual component are used as the input features of the (i+1)-th residual component, and the residual output features of the N-th residual component are determined as the global posture features corresponding to the sample object in the sample image; wherein the i-th residual component is connected to the (i+1)-th residual component. Optionally, when the size of the candidate convolution features is inconsistent with the size of the input features of the i-th candidate residual component, the input features of the i-th candidate residual component can be linearly transformed so that the size of the transformed features is the same as the size of the candidate convolution features, and then the transformed features can be added to the candidate convolution features to obtain the residual output features of the i-th residual component. In other words, the N residual components in the initial image recognition model are connected in sequence, and the residual output features of the previous residual component (for example, the i-th residual component) can be used as the input features of the next residual component (the (i+1)-th residual component), and finally the residual output features of the last residual component (the N-th residual component) can be used as the global posture features corresponding to the sample object in the sample image.

[0109] Optionally, the number of global posture features is K, where K can be a positive integer. K here can be considered to be the number of channels of the global posture features, such as 1, 2, etc. The computer device can calculate the feature averages corresponding to the K global posture features and combine the feature averages corresponding to the K global posture features into a global feature vector. The global feature vector can then be converted into a feature vector to be classified based on the activation function in the initial image recognition model. The feature vector to be classified is input into the classifier of the initial image recognition model, and the classifier of the initial image recognition model outputs a global classification result corresponding to the feature vector to be classified. In other words, the computer device can perform global average pooling on the K target posture features, converting each global posture feature into a numerical value. That is, the K target posture features can be converted into a K-dimensional global feature vector. After passing the global feature vector through the sigmoid activation function, it is input into the classifier to obtain a global classification result. Global average pooling can preserve spatial information in the sample image, thereby improving the positioning accuracy of the model.

[0110] See Figure 7 , Figure 7 This is a schematic diagram of a global average pooling process provided by an embodiment of the present application. Figure 7 As shown, the size of the global posture feature 50a can be expressed as: 4*4*3, that is, the number K of global posture features 50a can be 3, the width W is 4, and the height H is 4; by performing global average pooling processing on the global posture feature 50a, a global feature vector 50b can be obtained, and the size of the global feature vector 50b can be expressed as: 1*1*3, that is, each 4*4 feature map in the global posture feature 50a can be converted into a numerical value, that is, a 1*1 value, after the statistical feature average value is obtained.

[0111] Step S202: Multiply the global classification result and the sample convolution feature to obtain a global map corresponding to the sample image, and divide the sample image into blocks according to the global map to obtain M local area images; M is a positive integer.

[0112] Specifically, the computer device may perform a product operation on the global classification result and the sample convolution feature of the sample image to obtain a global map corresponding to the sample image, wherein the global map may include a class activation map (CAM) corresponding to each key point of the sample object in the sample image; wherein the generation process of the global map can refer to the above Figure 3 The process of generating the first activation map in step S101 of the corresponding embodiment will not be described in detail here.

[0113] Furthermore, the computer device can use the class activation map of each key point as prior information for the region location and perform block processing (cropping) on ​​the sample image to obtain M local region images, where M can be a positive integer, such as 1, 2, etc. In other words, the sample image can be cropped according to the global map to obtain the local region images corresponding to each component.

[0114] Step S203 : inputting the M local region images into the initial image recognition model in sequence, and obtaining local posture features corresponding to the M local region images in the initial image recognition model.

[0115] Specifically, the computer device can re-input the M cropped local region images into the initial image recognition model. The initial image recognition model can then obtain more fine-grained features, namely, local pose features corresponding to each local region image. The local pose features can be used to represent the pose of each component of the sample object contained in the sample image. The processing of a single local region image using the initial image recognition model can be referenced to the processing of the sample image in step S201 above and will not be further elaborated here.

[0116] Step S204 : performing feature combination on the global posture feature and the local posture features corresponding to the M local area images to obtain the target posture feature corresponding to the sample object in the sample image.

[0117] Specifically, the computer device can combine the global pose features learned by the initial image recognition model with the M local pose features. For example, the global pose features and the M local pose features can be concatenated to obtain the target pose features corresponding to the sample object in the sample image. The target pose features can include both the local pose features of the sample object's components and the global pose features of the sample object. By introducing component-aware block learning into the initial image recognition model, the granularity of the target pose features can be enhanced, thereby improving the model's positioning accuracy.

[0118] Step S205 , identifying the target posture features according to the classifier in the initial image recognition model to obtain a first classification result corresponding to the sample image; and generating a first activation map according to the first classification result and the sample convolution features of the sample image.

[0119] In step S206, data transformation is performed on the sample image to obtain a deformed image. A second classification result corresponding to the deformed image is output through the initial image recognition model. A second activation map is generated based on the second classification result and the deformed convolution features of the deformed image. The second classification result is determined by the deformed posture features corresponding to the sample object in the deformed image. The second activation map is used to represent the position information of the key points of the sample object in the deformed image.

[0120] Step S207: Determine the similarity loss result of the initial image recognition model based on the first activation map and the second activation map, and determine the classification loss result of the initial image recognition model based on the first classification result, the second classification result, and the key point category labels carried by the sample image.

[0121] Step S208: Based on the similarity loss result and the classification loss result, the network parameters of the initial image recognition model are modified to generate a target image recognition model; the target image recognition model is used to predict the object component category and positioning result corresponding to the key points in the source image.

[0122] The specific implementation of steps S205 to S208 can be found in the above Figure 3 Steps S101 to S104 in the corresponding embodiment are not described in detail here. Optionally, in the embodiment of the present application, the classification loss result of the initial image recognition model may include a global loss result in addition to the sample loss result and the deformation loss result; wherein the global loss result can be determined by the third difference between the global classification result and the key point category label.

[0123] See Figure 8 , Figure 8 This is a training diagram of an initial image recognition model provided in an embodiment of the present application. Figure 8 As shown in the aforementioned Figure 5 Based on the network structure of the initial image recognition model shown in FIG, block learning based on component perception is added. The embodiment of this application only describes the newly added block learning based on component perception. Figure 5The same network structure as shown is not described in detail in the embodiments of the present application. After the computer device obtains the global posture feature 60c corresponding to the sample image 40a through the residual network 60b, it can then use global average pooling and activation function to process the global posture feature 60c, and classify the processed result to obtain a global classification result. The global classification result is multiplied with the sample convolution feature output by the last convolution layer to obtain a global mapping graph. The sample image 60a can then be divided into blocks according to the global mapping graph to obtain M local area images 60p. The M local area images are sequentially input into the residual network 60b. After the residual network 60b, the local posture features 60q corresponding to the M local area images can be obtained. The M local posture features 60q are combined with the global posture features 60c of the sample image 60a to obtain the target posture features. By identifying the target posture features, a first classification result 60d can be obtained. The first classification result 60d is multiplied by the sample convolution feature output by the last convolution layer in the residual network 60b to obtain a feature image 60e (i.e., the first activation map mentioned above). The first activation map here is more sensitive to the component position of the sample image 60a. The subsequent processing process can be referred to above. Figure 5 Description in the corresponding embodiment.

[0124] In an embodiment of the present application, the target posture features in the sample image are extracted by the initial image recognition model, and the target posture features are classified and recognized to obtain a first classification result of the target posture features, and a first activation map is generated based on the first classification result and the sample convolution features of the sample image; at the same time, the sample image can also be transformed to obtain a deformed image, and the deformed posture features in the deformed image are extracted by the initial image recognition model, and a second activation map is generated based on the second classification result of the deformed posture features and the deformed convolution features of the deformed image; and similarity constraints (i.e., similarity loss results) can be applied to the first activation map and the second activation map. The target image recognition model trained in this way can improve the positioning accuracy of key points in the image; in addition, when training the initial image recognition model, there is no need to mark the position information of each key point of the sample object in the sample image, that is, the sample image is obtained from a wider source, and the labeling operation of the key point position of the sample image can be reduced, thereby improving the image processing efficiency; introducing component-aware block learning in the initial image recognition model can learn more fine-grained features and further obtain a first activation map that is more sensitive to component positions, which can further improve the positioning accuracy of the model.

[0125] See Figure 9 , Figure 9: is a flowchart of an image data processing method provided by an embodiment of the present application. It can be understood that the image data processing method can be executed by a computer device, which can be a server, a user terminal, or a computer program (including program code); Figure 9 As shown, the image data processing method may include the following steps S301 to S303:

[0126] Step S301, obtain a source image, obtain the object posture features corresponding to the target object in the source image through the target image recognition model, and identify the object component classification results corresponding to the object posture features; the object component classification results are used to characterize the object part categories corresponding to the key points of the target object.

[0127] Specifically, after the initial image recognition model is trained, the trained initial image recognition model can be referred to as a target image recognition model. A computer device can obtain a source image, which can contain a target object for which posture estimation is required. The target object can include, but is not limited to, a human body, an animal, etc. The source image is input into the trained target image recognition model. The target image recognition model can obtain the object posture features corresponding to the target object in the source image. The classifier of the target image recognition model can output the object component classification results corresponding to the object posture features. The object component classification results can be used to characterize the object part categories corresponding to the key points of the target object (e.g., human joints). The object posture features can be global object features for the target object extracted by the target image recognition model, or can be fusion features between the global object features and local object features corresponding to the target object. When the object posture feature is the global object feature corresponding to the target object in the source image, it indicates that component-aware block learning is not introduced in the process of feature extraction of the source image using the target image recognition model; when the object posture feature is the fusion feature between the global object feature and the local object feature corresponding to the target object in the source image, it indicates that component-aware block learning is introduced in the process of feature extraction of the source image using the target image recognition model.

[0128] Optionally, if component-aware block learning is introduced in the process of extracting features from the source image using the target image recognition model, the computer device inputs the source image into the target image recognition model, obtains the global object features corresponding to the target object in the source image in the target image recognition model, and outputs the global object classification result corresponding to the global object features according to the classifier in the target image recognition model; obtains the object convolution feature for the source image output by the target convolution layer in the target image recognition model, multiplies the global object classification result and the object convolution feature to obtain the global object map corresponding to the source image (for example, the above Figure 2 The feature image 20g in the corresponding embodiment); the source image is divided into blocks according to the global object map to obtain M object component region images, and the object component features corresponding to the M object component region images are obtained according to the target image recognition model; M is a positive integer; the global object feature and the object component features corresponding to the M object component region images are combined into an object posture feature. The object posture feature extraction process here can refer to the above Figure 6 The process of extracting the target posture features in steps S201 to S204 of the corresponding embodiment will not be described in detail here.

[0129] It should be noted that since the target image recognition model already has the ability to learn geometrically invariant position information, there is no need to process the deformed image after data transformation during the application of the target image recognition model. That is, when using the target image recognition model, there is no need to introduce a series of operations such as data transformation.

[0130] Step S302 : generating an object part map according to the object component classification result and the object convolution feature of the source image.

[0131] Specifically, after obtaining the object part classification result, the computer device can multiply the object part classification result and the object convolution feature of the source image to generate an object part map. Figure 6 The first activation map in the corresponding embodiment is similar and will not be described again here.

[0132] Step S303, obtaining the pixel average value corresponding to the object part map, determining the positioning result of the key points in the target object in the source image based on the pixel average value, and determining the posture estimation result corresponding to the target object in the source image based on the object part category and the positioning result.

[0133] Specifically, the computer device can take the pixel average of the object part mapping image and determine the pixel average as the positioning result of the key points in the target object in the source image. According to the object part category and the positioning result, the object skeleton of the target object in the source image can be determined, and the object skeleton can be used as the posture estimation result corresponding to the target object in the source image.

[0134] In one or more embodiments, the source image may include one or more target objects. After acquiring the source image, the computer device can first detect the target objects contained in the source image through the target image recognition model, and determine the area where the single target object is located in the source image, and then perform posture estimation on the area where the single target object is located, that is, perform feature extraction on the area where the single target object is located, detect all key points contained in the single target object (such as all human joints), and the object part category and positioning results corresponding to each key point. According to the object part category and positioning results of each key point, all detected key points can be connected to obtain the object skeleton corresponding to the single target object, and the object skeleton can be used to represent the posture estimation result of the single target object.

[0135] Optionally, the human posture estimation method proposed in this application (the above-mentioned target image recognition model) can be applied in different application scenarios, such as security monitoring scenarios, human-computer interaction scenarios (for example, virtual reality, human-computer animation, etc.), content review scenarios, auxiliary sports training scenarios, autonomous driving scenarios, game or movie character action design scenarios, etc. In the security monitoring scenario, for the video obtained by the security monitoring system, the target image recognition model can be used to perform human posture estimation on the video frames in the video based on the time series information corresponding to the video, and the behavior of the people in the video can be determined based on the human posture estimation results (that is, the above-mentioned posture estimation results), and then abnormal behaviors such as fighting can be detected, so as to achieve long-term uninterrupted intelligent monitoring, thereby saving the manpower and material resources spent on manual monitoring, that is, saving security monitoring costs. In human-computer interaction scenarios, source images (or videos) of the user can be collected, and the target image recognition model can be used to estimate the human posture of the collected source images (or videos). Based on the human posture estimation results (which can also be understood as human motion information), the machine can be controlled, such as executing a specific instruction based on a specific human motion. In assisted sports training scenarios, source images (or videos) of the user can be collected, and the target image recognition model can be used to estimate the human posture of the collected source images (or videos). Based on the human posture estimation results, it can be determined whether the athlete's movements are standard and which movement postures need improvement. This can provide users who want to exercise with an intelligent, professional sports coach. In game character motion design scenarios, the target image recognition model can be used to estimate human posture and obtain human motion, replacing expensive motion capture equipment and reducing the cost and difficulty of game character motion design.

[0136] Taking the content review scenario as an example, when the posture estimation result is the same as the posture of the target object in the content review system, the review result of the source image in the content review system is determined to be a passed review result, and access rights to the content review system are set for the object corresponding to the source image; after the posture estimation result passes the review in the content review system, the object corresponding to the source image can have the right to access the content review system. Optionally, the posture estimation in the content review system can refer to the posture estimation of the entire human body, or it can refer to the posture estimation of human body parts, and this application does not limit this.

[0137] See Figure 10 , Figure 10 This is a schematic diagram of a scene for human posture estimation provided by an embodiment of the present application. Figure 10 As shown, user A can send a verification request to server 70d via user terminal 70a. After receiving the verification request from user terminal 70a, server 70d can obtain the identity verification method for user A and return the identity verification method to user terminal 70a. A verification box 70b can be displayed on the terminal screen of user terminal 70a. User A can face the verification box 70b in user terminal 70a and perform a specific action (for example, raising a hand, kicking a leg, putting hands on hips, etc.). User terminal 70a can capture the image to be verified 70c in the verification box 70b in real time (which can be considered as the source image mentioned above) and send the captured image 70c to server 70d.

[0138] The server 70d can obtain the image to be verified 70c sent by the user terminal 70a, and obtain the target object posture 70e set in advance by user A in the content review system. The target object posture 70e can serve as the verification information of user A in the content review system. The server 70d can use the target image recognition model to perform posture estimation on the image to be verified 70c to obtain the posture estimation result corresponding to the image to be verified 70c; the posture estimation result corresponding to the image to be verified 70c is compared with the target object posture 70e for similarity. When the similarity between the posture estimation result of the image to be verified 70c and the target object posture 70e is greater than or equal to the similarity threshold (for example, the similarity threshold can be set to 90%), it can be determined that the posture estimation result of the image to be verified 70c is the same as the target object posture 70e, and the user A has passed the review in the content review system. When the similarity between the posture estimation result of the image to be verified 70c and the target object posture 70e is less than the similarity threshold, it can be determined that the posture estimation result of the image to be verified 70c is different from the target object posture 70e, and the user A has failed the review in the content review system, and an action error prompt message is returned to the user terminal 70a. The action error prompt message is used to prompt user A to redo the action for identity review.

[0139] In an embodiment of the present application, the posture of the source image is estimated by using a trained target image recognition model, which can improve the positioning accuracy of key points in the image; in addition, by introducing block learning based on component perception into the target image recognition model, more fine-grained features can be learned, and an object part mapping map that is more sensitive to component positions can be further obtained, which can further improve the positioning accuracy of the model.

[0140] See Figure 11 , Figure 11 This is a schematic diagram of the structure of an image data processing device provided by an embodiment of the present application. Figure 11 As shown, the image data processing device 1 may include: a first generating module 11, a second generating module 12, a loss result determining module 13, and a parameter correcting module 14;

[0141] A first generation module 11 is configured to obtain a sample image, output a first classification result corresponding to the sample image through an initial image recognition model, and generate a first activation map based on the first classification result and a sample convolution feature of the sample image; the first classification result is determined by a target posture feature corresponding to a sample object in the sample image, and the first activation map is used to represent position information of key points of the sample object in the sample image;

[0142] a second generation module 12 configured to perform data transformation on the sample image to obtain a deformed image, output a second classification result corresponding to the deformed image through the initial image recognition model, and generate a second activation map based on the second classification result and the deformed convolution features of the deformed image; the second classification result is determined by the deformed posture features corresponding to the sample object in the deformed image, and the second activation map is used to represent the position information of the key points of the sample object in the deformed image;

[0143] a loss result determination module 13, configured to determine a similarity loss result of the initial image recognition model based on the first activation map and the second activation map, and to determine a classification loss result of the initial image recognition model based on the first classification result, the second classification result, and the key point category labels carried by the sample image;

[0144] The parameter correction module 14 is used to correct the network parameters of the initial image recognition model based on the similarity loss result and the classification loss result to generate a target image recognition model; the target image recognition model is used to predict the object component category and positioning result corresponding to the key points in the source image.

[0145] The specific functional implementation of the first generation module 11, the second generation module 12, the loss result determination module 13, and the parameter correction module 14 can be found in the above Figure 3Steps S101 to S104 in the corresponding embodiment will not be described in detail here.

[0146] In one or more embodiments, the first generation module 11 may include: a feature extraction unit 111 , a feature classification unit 112 , a feature mapping unit 113 , and an upsampling processing unit 114 ;

[0147] The feature extraction unit 111 is used to input the sample image into the initial image recognition model, and obtain the target posture feature corresponding to the sample object in the sample image according to the initial image recognition model;

[0148] A feature classification unit 112 is configured to identify the target posture feature based on the classifier in the initial image recognition model to obtain a first classification result corresponding to the sample image;

[0149] A feature mapping unit 113 is configured to obtain a sample convolution feature for the sample image output by the target convolution layer in the initial image recognition model, and perform a product operation on the first classification result and the sample convolution feature to obtain a candidate activation map corresponding to the sample image;

[0150] The upsampling processing unit 114 is configured to perform upsampling processing on the candidate activation map to obtain a first activation map having the same image size as the sample image.

[0151] The specific functional implementation of the feature extraction unit 111, the feature classification unit 112, the feature mapping unit 113, and the upsampling processing unit 114 can be found in the above Figure 3 Step S101 in the corresponding embodiment will not be described in detail here.

[0152] In one or more embodiments, the feature extraction unit 111 may include: a global classification subunit 1111, a block processing subunit 1112, a local feature extraction subunit 1113, and a feature combination subunit 1114;

[0153] The global classification subunit 1111 is used to obtain the global posture features corresponding to the sample objects in the sample images in the initial image recognition model, and output the global classification results corresponding to the global posture features through the classifier in the initial image recognition model;

[0154] The block processing subunit 1112 is used to perform a product operation on the global classification result and the sample convolution feature to obtain a global map corresponding to the sample image, and to perform block processing on the sample image according to the global map to obtain M local area images; M is a positive integer;

[0155] The local feature extraction subunit 1113 is used to sequentially input the M local region images into the initial image recognition model, and obtain local posture features corresponding to the M local region images in the initial image recognition model;

[0156] The feature combination subunit 1114 is used to perform feature combination on the global posture feature and the local posture features corresponding to the M local area images to obtain the target posture feature corresponding to the sample object in the sample image.

[0157] Optionally, the initial image recognition model includes N residual components, each residual component includes one or more convolutional layers, and N is a positive integer;

[0158] The global classification subunit 1111 can be specifically used for:

[0159] Get the input feature of the i-th residual component among N residual components; when i is 1, the input feature of the i-th residual component is the sample image, and i is a positive integer less than N;

[0160] According to one or more convolutional layers in the i-th residual component, the input features of the i-th residual component are convolved to obtain candidate convolution features;

[0161] Combine the candidate convolutional features and the input features of the i-th residual component to obtain the residual output features of the i-th residual component, and use the residual output features of the i-th residual component as the input features of the (i+1)-th residual component; the i-th residual component is connected to the (i+1)-th residual component;

[0162] The residual output feature of the Nth residual component is determined as the global posture feature corresponding to the sample object in the sample image.

[0163] Optionally, the number of global posture features is K, where K is a positive integer;

[0164] The global classification subunit 1111 is specifically used for:

[0165] Count the feature averages corresponding to the K global posture features, and combine the feature averages corresponding to the K global posture features into a global feature vector;

[0166] According to the activation function in the initial image recognition model, the global feature vector is converted into the feature vector to be classified;

[0167] The feature vector to be classified is input into the classifier in the initial image recognition model, and the classifier in the initial image recognition model outputs the global classification result corresponding to the feature vector to be classified.

[0168] The specific functional implementation of the global classification subunit 1111, the block processing subunit 1112, the local feature extraction subunit 1113, and the feature combination subunit 1114 can be found in the above Figure 6 Step S201 in the corresponding embodiment will not be described in detail here.

[0169] In one or more embodiments, the loss result determination module 13 may include: a data transformation unit 131 , a similarity constraint unit 132 , a sample loss determination unit 133 , a deformation loss determination unit 134 , and a classification loss determination unit 135 ;

[0170] a data transformation unit 131, configured to perform data transformation on the second activation map to obtain a deformed activation map;

[0171] The similarity constraint unit 132 is configured to perform similarity constraint on the first activation map and the deformed activation map to determine a similarity loss result of the initial image recognition model.

[0172] A sample loss determination unit 133 is configured to obtain a first error between the first classification result and the key point category label carried by the sample image, and determine a sample loss result of the initial image recognition model based on the first error;

[0173] a deformation loss determining unit 134, configured to obtain a second error between the second classification result and the key point category label, and determine a deformation loss result of the initial image recognition model based on the second error;

[0174] The classification loss determination unit 135 is used to determine the classification loss result of the initial image recognition model according to the sample loss result and the deformation loss result.

[0175] The specific functional implementation of the data transformation unit 131, the similarity constraint unit 132, the sample loss determination unit 133, the deformation loss determination unit 134, and the classification loss determination unit 135 can be found in the above Figure 3 Step S103 in the corresponding embodiment will not be described in detail here.

[0176] In one or more embodiments, the parameter modification module 14 may include: a total loss determination unit 141 , a network parameter adjustment unit 142 ;

[0177] A total loss determination unit 141 is used to determine a total model loss result corresponding to the initial image recognition model based on the similarity loss result and the classification loss result;

[0178] The network parameter adjustment unit 142 is used to correct the network parameters of the initial image recognition model by minimizing the total loss result of the model, and determine the initial image recognition model including the corrected network parameters as the target image recognition model.

[0179] The specific functional implementation of the total loss determination unit 141 and the network parameter adjustment unit 142 can be found in the above Figure 3 Step S104 in the corresponding embodiment will not be described in detail here.

[0180] In one or more embodiments, the image data processing device 1 may further include: an object component classification module 15, an object map generation module 16, and a positioning result determination module 17;

[0181] The object component classification module 15 is used to obtain a source image, obtain the object posture features corresponding to the target object in the source image through the target image recognition model, and identify the object component classification results corresponding to the object posture features; the object component classification results are used to characterize the object part categories corresponding to the key points of the target object;

[0182] An object map generation module 16 is configured to generate an object part map based on the object part classification result and the object convolution feature of the source image;

[0183] The positioning result determination module 17 is used to obtain the pixel average value corresponding to the object part map, determine the positioning result of the key points in the target object in the source image based on the pixel average value, and determine the posture estimation result corresponding to the target object in the source image based on the object part category and the positioning result.

[0184] The specific functional implementation of the object component classification module 15, the object map generation module 16, and the positioning result determination module 17 can be found in the above Figure 9 Steps S301 to S303 in the corresponding embodiment will not be described in detail here.

[0185] In one or more embodiments, the object component classification module 15 may include: a global object classification unit 151 , a global map generation unit 152 , a component feature acquisition unit 153 , and a component feature combination unit 154 ;

[0186] The global object classification unit 151 is configured to input a source image into a target image recognition model, obtain global object features corresponding to the target object in the source image in the target image recognition model, and output a global object classification result corresponding to the global object features based on a classifier in the target image recognition model;

[0187] A global map generation unit 152 is configured to obtain the object convolution features for the source image output by the target convolution layer in the target image recognition model, and perform a product operation on the global object classification result and the object convolution features to obtain a global object map corresponding to the source image;

[0188] The component feature acquisition unit 153 is configured to perform block processing on the source image according to the global object map to obtain M object component region images, and obtain object component features corresponding to the M object component region images according to the target image recognition model; M is a positive integer;

[0189] The component feature combining unit 154 is configured to combine the global object feature and the object component features corresponding to the M object component region images into an object posture feature.

[0190] The specific functional implementation of the global object classification unit 151, the global map generation unit 152, the component feature acquisition unit 153, and the component feature combination unit 154 can be found in the above Figure 9 Step S301 in the corresponding embodiment will not be described in detail here.

[0191] In one or more embodiments, the image data processing apparatus may further include: an audit module 18 .

[0192] The audit module 18 is used to determine that the audit result of the source image in the content audit system is a passed audit result when the posture estimation result is the same as the posture of the target object in the content audit system, and set access rights for the content audit system for the object corresponding to the source image.

[0193] The specific function implementation of the audit module 18 can be found in the above Figure 9 Step S303 in the corresponding embodiment will not be described in detail here.

[0194] In an embodiment of the present application, the target posture features in the sample image are extracted by the initial image recognition model, and the target posture features are classified and recognized to obtain a first classification result of the target posture features, and a first activation map is generated based on the first classification result and the sample convolution features of the sample image; at the same time, the sample image can also be transformed to obtain a deformed image, and the deformed posture features in the deformed image are extracted by the initial image recognition model, and a second activation map is generated based on the second classification result of the deformed posture features and the deformed convolution features of the deformed image; and similarity constraints (i.e., similarity loss results) can be applied to the first activation map and the second activation map. The target image recognition model trained in this way can improve the positioning accuracy of key points in the image; in addition, when training the initial image recognition model, there is no need to mark the position information of each key point of the sample object in the sample image, that is, the sample image is obtained from a wider source, and the labeling operation of the key point position of the sample image can be reduced, thereby improving the image processing efficiency; introducing component-aware block learning in the initial image recognition model can learn more fine-grained features and further obtain a first activation map that is more sensitive to component positions, which can further improve the positioning accuracy of the model.

[0195] Further, see Figure 12 , Figure 12 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 12 As shown, the computer device 1000 may be a user terminal, for example, Figure 1 The user terminal 10a in the corresponding embodiment may also be a server, for example, Figure 1 The server 10d in the corresponding embodiment will not be limited here. For ease of understanding, this application takes the computer device as an example of a user terminal. The computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1004 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Figure 12As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

[0196] The network interface 1004 in the computer device 1000 can also provide a network communication function, and the optional user interface 1003 can also include a display screen and a keyboard. Figure 12 In the computer device 1000 shown, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an interface for user input; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0197] Obtaining a sample image, outputting a first classification result corresponding to the sample image through an initial image recognition model, and generating a first activation map based on the first classification result and sample convolution features of the sample image; the first classification result is determined by target pose features corresponding to a sample object in the sample image, and the first activation map is used to represent position information of key points of the sample object in the sample image;

[0198] Performing data transformation on the sample image to obtain a deformed image, outputting a second classification result corresponding to the deformed image through the initial image recognition model, and generating a second activation map based on the second classification result and the deformed convolution features of the deformed image; the second classification result is determined by the deformed posture features corresponding to the sample object in the deformed image, and the second activation map is used to represent the position information of the key points of the sample object in the deformed image;

[0199] Determining a similarity loss result of the initial image recognition model based on the first activation map and the second activation map, and determining a classification loss result of the initial image recognition model based on the first classification result, the second classification result, and the key point category labels carried by the sample image;

[0200] Based on the similarity loss results and the classification loss results, the network parameters of the initial image recognition model are modified to generate a target image recognition model; the target image recognition model is used to predict the object part category and positioning results corresponding to the key points in the source image.

[0201] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 3 、 Figure 6 、 Figure 9 The description of the image data processing method in any corresponding embodiment can also be performed as described above. Figure 7 The description of the image data processing device 1 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here either.

[0202] In addition, it should be pointed out that the embodiment of the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the image data processing device 1 mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the above-mentioned Figure 3 、 Figure 6 、 Figure 9 Therefore, the description of the image data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on a computing device for execution, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network. Multiple computing devices distributed at multiple locations and interconnected by a communication network can constitute a blockchain system.

[0203] In addition, it should be noted that: the embodiment of the present application also provides a computer program product or computer program, which may include computer instructions, which may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, so that the computer device performs the above Figure 3 、 Figure 6 、 Figure 9 The description of the image data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated here. For technical details not disclosed in the computer program product or computer program embodiments involved in this application, please refer to the description of the method embodiments of this application.

[0204] It should be noted that for the aforementioned various method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0205] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0206] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0207] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0208] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for processing image data, characterized in that: include: Acquire a sample image, output a first classification result corresponding to the sample image through an initial image recognition model, and generate a first activation map based on the first classification result and sample convolution features of the sample image; the first classification result is determined by target pose features corresponding to a sample object in the sample image, and the first activation map is used to represent position information of key points of the sample object in the sample image; Performing data transformation on the sample image to obtain a deformed image, outputting a second classification result corresponding to the deformed image through the initial image recognition model, and generating a second activation map based on the second classification result and a deformed convolution feature of the deformed image; the second classification result is determined by a deformed posture feature corresponding to the sample object in the deformed image, and the second activation map is used to represent position information of key points of the sample object in the deformed image; Determining a similarity loss result of the initial image recognition model based on the first activation map and the second activation map, and determining a classification loss result of the initial image recognition model based on the first classification result, the second classification result, and the key point category labels carried by the sample image; Based on the similarity loss result and the classification loss result, the network parameters of the initial image recognition model are modified to generate a target image recognition model; the target image recognition model is used to predict the object component category and positioning result corresponding to the key points in the source image.

2. The method according to claim 1, characterized in that Outputting a first classification result corresponding to the sample image through the initial image recognition model, and generating a first activation map according to the first classification result and the sample convolution feature of the sample image, include: Inputting the sample image into the initial image recognition model, and obtaining target posture features corresponding to the sample object in the sample image according to the initial image recognition model; Identifying the target posture feature according to the classifier in the initial image recognition model to obtain a first classification result corresponding to the sample image; Obtaining a sample convolution feature for the sample image output by a target convolution layer in the initial image recognition model, and performing a product operation on the first classification result and the sample convolution feature to obtain a candidate activation map corresponding to the sample image; Upsampling is performed on the candidate activation map to obtain a first activation map having the same image size as the sample image.

3. The method according to claim 2, characterized in that The acquiring, according to the initial image recognition model, target posture features corresponding to the sample object in the sample image includes: In the initial image recognition model, obtaining a global posture feature corresponding to the sample object in the sample image, and outputting a global classification result corresponding to the global posture feature through a classifier in the initial image recognition model; Performing a product operation on the global classification result and the sample convolution feature to obtain a global map corresponding to the sample image, and performing block processing on the sample image according to the global map to obtain M local area images, where M is a positive integer; Inputting the M local area images into the initial image recognition model in sequence, and obtaining local posture features corresponding to the M local area images in the initial image recognition model; The global posture feature is combined with the local posture features corresponding to the M local area images to obtain a target posture feature corresponding to the sample object in the sample image.

4. The method according to claim 3, characterized in that The initial image recognition model includes N residual components, each residual component includes one or more convolutional layers, and N is a positive integer; The step of obtaining, in the initial image recognition model, a global posture feature corresponding to the sample object in the sample image includes: Obtaining an input feature of an i-th residual component among the N residual components; when i is 1, the input feature of the i-th residual component is the sample image, and i is a positive integer less than N; Performing a convolution operation on the input features of the i-th residual component according to one or more convolutional layers in the i-th residual component to obtain candidate convolution features; Combine the candidate convolutional features and the input features of the i-th residual component to obtain the residual output features of the i-th residual component, and use the residual output features of the i-th residual component as the input features of the (i+1)-th residual component; the i-th residual component is connected to the (i+1)-th residual component; The residual output feature of the Nth residual component is determined as the global posture feature corresponding to the sample object in the sample image.

5. The method according to claim 3, characterized in that The number of the global posture features is K, where K is a positive integer; Outputting a global classification result corresponding to the global posture feature through a classifier in the initial image recognition model includes: Counting the feature averages corresponding to the K global posture features, and combining the feature averages corresponding to the K global posture features into a global feature vector; Converting the global feature vector into a feature vector to be classified according to an activation function in the initial image recognition model; The feature vector to be classified is input into the classifier in the initial image recognition model, and the classifier in the initial image recognition model outputs a global classification result corresponding to the feature vector to be classified.

6. The method according to claim 1, characterized in that The determining, based on the first activation map and the second activation map, a similarity loss result of the initial image recognition model includes: performing the data transformation on the second activation map to obtain a deformed activation map; A similarity constraint is performed on the first activation map and the deformed activation map to determine a similarity loss result of the initial image recognition model.

7. The method according to claim 1, characterized in that The determining, based on the first classification result, the second classification result, and the key point category labels carried by the sample image, a classification loss result of the initial image recognition model includes: Obtaining a first error between the first classification result and the key point category label carried by the sample image, and determining a sample loss result of the initial image recognition model according to the first error; Obtaining a second error between the second classification result and the key point category label, and determining a deformation loss result of the initial image recognition model according to the second error; The classification loss result of the initial image recognition model is determined according to the sample loss result and the deformation loss result.

8. The method according to claim 1, characterized in that The method of modifying the network parameters of the initial image recognition model based on the similarity loss result and the classification loss result to generate a target image recognition model includes: Determining a total model loss result corresponding to the initial image recognition model according to the similarity loss result and the classification loss result; By performing minimization optimization processing on the total loss result of the model, the network parameters of the initial image recognition model are corrected, and the initial image recognition model including the corrected network parameters is determined as the target image recognition model.

9. The method according to claim 1, characterized in that Also includes: Acquire a source image, acquire object posture features corresponding to a target object in the source image using a target image recognition model, and identify object component classification results corresponding to the object posture features; the object component classification results are used to characterize object part categories corresponding to key points of the target object; generating an object part map according to the object part classification result and the object convolution feature of the source image; Obtain the pixel average value corresponding to the object part map, determine the positioning result of the key points in the target object in the source image based on the pixel average value, and determine the posture estimation result corresponding to the target object in the source image based on the object part category and the positioning result.

10. The method according to claim 9, characterized in that The step of obtaining the object posture feature corresponding to the target object in the source image through the target image recognition model includes: Inputting the source image into the target image recognition model, obtaining global object features corresponding to the target object in the source image in the target image recognition model, and outputting a global object classification result corresponding to the global object features according to a classifier in the target image recognition model; Obtaining object convolution features for the source image output by a target convolution layer in the target image recognition model, and performing a product operation on the global object classification result and the object convolution features to obtain a global object map corresponding to the source image; The source image is divided into blocks according to the global object map to obtain M object component region images, and object component features corresponding to the M object component region images are obtained according to the target image recognition model; M is a positive integer; The global object feature and the object component features corresponding to the M object component region images are combined into the object posture feature.

11. The method according to claim 9, characterized in that Also includes: When the posture estimation result is the same as the posture of the target object in the content audit system, the audit result of the source image in the content audit system is determined to be a passed audit result, and access rights to the content audit system are set for the object corresponding to the source image.

12. An image data processing device, characterized in that: include: a first generation module, configured to obtain a sample image, output a first classification result corresponding to the sample image through an initial image recognition model, and generate a first activation map based on the first classification result and a sample convolution feature of the sample image; the first classification result is determined by a target posture feature corresponding to a sample object in the sample image, and the first activation map is used to represent position information of key points of the sample object in the sample image; a second generation module, configured to perform data transformation on the sample image to obtain a deformed image, output a second classification result corresponding to the deformed image through the initial image recognition model, and generate a second activation map based on the second classification result and a deformed convolution feature of the deformed image; the second classification result is determined by a deformed posture feature corresponding to the sample object in the deformed image, and the second activation map is used to represent position information of key points of the sample object in the deformed image; a loss result determination module, configured to determine a similarity loss result of the initial image recognition model based on the first activation map and the second activation map, and determine a classification loss result of the initial image recognition model based on the first classification result, the second classification result, and the key point category labels carried by the sample image; A parameter correction module is used to correct the network parameters of the initial image recognition model based on the similarity loss result and the classification loss result to generate a target image recognition model; the target image recognition model is used to predict the object component category and positioning result corresponding to the key points in the source image.

13. A computer device, characterized in that: including memory and processor; The memory is connected to the processor, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The method comprises a computer program / instruction, which implements the method according to any one of claims 1 to 11 when the computer program / instruction is executed by a processor.

Images