Data processing, posture prediction method, device and storage medium

Abstract

The embodiments of the present application provide a data processing, posture prediction method, device and storage medium. In the data processing method, in the process of training a posture prediction model based on an automatically labeled sample image, the labeling error of the posture marked on the sample image can be predicted according to the posture prediction result output by the posture prediction model. Based on the posture marked on the sample image and the calculated labeling error, the posture prediction model can continue to be trained until the loss caused by the labeling error converges to a specified value. Based on this implementation method, the error of the supervisory signal can be compensated to reduce the impact of the labeling error on the sample image on the training process of the posture prediction model, and then the posture prediction model with higher performance obtained by training can be trained to improve the prediction accuracy of the posture prediction model.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a data processing, posture prediction method, device and storage medium. Background Art

[0002] In applications related to facial recognition, head posture judgment is a very important link. Facial images captured when the head posture is better usually contain richer facial features, which is conducive to facial recognition or facial analysis operations.

[0003] The prior art provides a head posture prediction method based on machine learning, in which the head posture on the sample images used for training needs to be annotated. However, the existing automatic annotation method has a low annotation accuracy, which is not conducive to training an algorithm model with high prediction accuracy. Therefore, a new solution needs to be proposed. Summary of the invention

[0004] Multiple aspects of the present application provide a data processing, posture prediction method, device and storage medium to reduce the impact of labeling errors on the model training process and train an algorithm model with higher performance.

[0005] An embodiment of the present application provides a posture prediction method, comprising: responding to a call request from a client to a first interface, determining a model training service resource corresponding to the first interface, wherein the model training service resource includes a sample image; annotating the sample image to obtain a first annotation result; inputting the annotated sample image into a neural network model to obtain an output result of the neural network; performing error fitting according to the output result to calculate the annotation error of the first annotation result; optimizing the neural network model according to the first annotation result and the annotation error to obtain a result model; and returning the result model to the client.

[0006] An embodiment of the present application provides a posture prediction method, comprising: obtaining an image to be processed containing a first object; inputting the image to be processed into a posture prediction model to obtain a posture prediction result of the first object; wherein the posture prediction model is trained based on an automatically annotated sample image and a predicted annotation error of the sample image.

[0007] An embodiment of the present application also provides a data processing method, including: marking a first posture of a first object on a sample image; inputting the sample image into a posture prediction model to obtain a posture prediction result of the first object; performing error fitting based on the posture prediction result to estimate the marking error of the first posture; and optimizing the posture prediction model based on the first posture and the marking error.

[0008] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: execute the posture prediction method or data processing method provided in the embodiment of the present application.

[0009] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the posture prediction method or data processing method provided in the embodiment of the present application.

[0010] In an embodiment of the present application, in the process of training a posture prediction model based on an automatically annotated sample image, the annotation error of the posture annotated on the sample image can be predicted based on the posture prediction result output by the posture prediction model. Based on the posture annotated on the sample image and the calculated annotation error, the posture prediction model can continue to be trained until the loss caused by the annotation error converges to a specified value. Based on this implementation method, the error of the supervision signal can be compensated to reduce the impact of the annotation error on the sample image on the training process of the posture prediction model, and then the posture prediction model with higher performance obtained by training can be trained to improve the prediction accuracy of the posture prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 A flowchart of a data processing method provided for an exemplary embodiment of the present application;

[0013] Figure 2a A flowchart of a data processing method provided for another exemplary embodiment of the present application;

[0014] Figure 2b A schematic diagram of automatically annotating a sample image provided by an exemplary embodiment of the present application;

[0015] Figure 2c A schematic diagram of attitude angles in three spatial dimensions provided for an exemplary embodiment of the present application;

[0016] Figure 2d A schematic diagram of a training posture prediction model provided for an exemplary embodiment of the present application;

[0017] Figure 2e A schematic diagram of a prediction model based on annotation error training provided by an exemplary embodiment of the present application;

[0018] Figure 3aA flowchart of a posture prediction method provided by an exemplary embodiment of the present application;

[0019] Figure 3b A flowchart of a posture prediction method provided by another exemplary embodiment of the present application;

[0020] Figure 4 A schematic diagram of an application scenario example provided for an exemplary embodiment of the present application;

[0021] Figure 5 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0023] In applications related to facial recognition, head posture judgment is a very important link. Facial images captured when the head posture is better usually contain richer facial features, which is conducive to facial recognition or facial analysis operations. For example, in the process of face recognition, the face of a person can be captured when the head posture is better, and face recognition can be performed based on the captured image; or, in the image sequence obtained by tracking and shooting the head of a person, an image with a better head posture is selected, and then face recognition and other subsequent steps are performed based on the selected image.

[0024] In order to accurately predict head posture in practical applications, it is usually necessary to annotate a batch of head posture pictures with high accuracy for algorithm training. However, accurately annotating head posture is a costly task.

[0025] At present, a method of posture annotation with higher accuracy is to place a row of TV walls in front of the subject, and the subject's body and line of sight are aligned directly in front of it and remain motionless. Then, the subject is told by voice to look at a certain screen by just turning his head. At this time, the subject's head posture is the posture angle corresponding to the screen. Usually, in order to ensure that the subject is looking at the correct screen, the content will be displayed on the screen, and the wrong annotation samples will be further filtered out by whether the subject's answers to questions related to the content displayed on the screen are correct or not. However, the cost of this annotation method is relatively high.

[0026] In order to reduce the cost of annotation, there is a method of manually annotating the key points of the face and then automatically calculating the head posture based on the key points. In this implementation, first, several key points of the face can be annotated on the image, and then, algorithms such as 3DMM (3D Morphable Mode) are used to calculate the head posture using the key points. This annotation method reduces the workload to a certain extent, but the cost of annotating several key points of the face is still not negligible. At the same time, the head posture calculated by the algorithm usually has an error of about plus or minus 5 degrees.

[0027] To further reduce the annotation cost, the samples can be automatically annotated. In this implementation, the facial key point prediction model can be used to automatically predict the facial key points on the sample image. Then, an algorithm is used to calculate the head posture using the facial key points. However, in this process, there are errors in the step of automatically predicting the key points, and the algorithm used in the posture calculation introduces errors again, which further increases the error of the posture annotation result.

[0028] In order to solve the technical problems of large labeling error and low labeling accuracy in the above technical solution, an embodiment of the present application provides a data processing method, which will be described below in conjunction with the accompanying drawings.

[0029] Figure 1 A flowchart of a data processing method provided by an exemplary embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0030] Step 101: Mark a first posture of a first object on a sample image.

[0031] Step 102: Input the sample image into a posture prediction model to obtain a posture prediction result of the first object.

[0032] Step 103: performing error fitting according to the posture prediction result to estimate the labeling error of the first posture.

[0033] Step 104: Optimize the posture prediction model according to the first posture and the annotation error.

[0034] In this embodiment, the sample images include images collected in various scenarios for training the posture prediction model. Among them, the object in the sample image can be described as a first object. In the posture annotation scene, the first object can be a component of a human body to be posture recognized, such as the head and limbs of a human body; or, the object can be a component of an animal body to be posture recognized, such as the head and limbs of an animal; or, the object can also be all or part of a component of a device or apparatus to be recognized, such as a robotic arm, a drone, etc., which is not limited in this embodiment.

[0035] The annotation result obtained by annotating the posture of the first object on the sample image can be described as a first posture. The posture can be represented by a posture angle in a three-dimensional space.

[0036] The operation of marking the first posture of the first object on the sample image can be implemented based on the sample automatic marking algorithm. For example, in some embodiments, the key points of the object on the sample image can be automatically predicted using a key point prediction model. Then, a posture recognition algorithm is used to calculate the posture of the object using the predicted key points. In other embodiments, multiple key points of the object can be manually marked on the sample image, and then a posture recognition algorithm is used to calculate the posture of the object using the predicted key points. Alternatively, other methods can be used to mark the posture of the first object on the sample image, which is not limited in this embodiment.

[0037] The posture prediction model is an artificial neural network that analyzes the input image and predicts the posture based on the analysis results. The training process of the posture prediction model is a supervised learning process. In the supervised learning process, the first posture marked on the sample image can be used as the expected output value of the posture prediction model, which is also called the supervisory signal. After the sample image is input into the posture prediction model, under the supervision of the supervisory signal, the posture prediction model can learn the ability to recognize the posture of the first object from the sample image.

[0038] The algorithm used to calculate the posture based on the key points has certain errors, which leads to the difference between the first posture obtained by annotation and the true posture. If the key point estimation model is used to automatically estimate the key points during the annotation process, the annotation error corresponding to the first posture will be further increased. Therefore, in order to reduce the impact of the annotation error on the posture prediction model, in this embodiment, the annotation error can be obtained, and the posture prediction model can be further optimized according to the annotation error.

[0039] Among them, the annotation error of the first posture can be obtained according to the posture prediction result, which will be specifically explained in subsequent embodiments and will not be repeated here.

[0040] When training the posture prediction model based on the annotation error, the parameters of the posture prediction model can be continuously adjusted to gradually reduce the annotation error. It should be understood that when the error loss function converges to a specified range, the posture prediction model can be considered to have good compatibility with the annotation error, and the trained posture prediction model can be output at this time.

[0041] When the posture prediction model is put into use, the image to be processed can be input into the posture prediction model so that the posture prediction model can estimate the posture of the second object in the image to be processed and output the posture prediction result of the posture of the second object.

[0042] It should be understood that in this embodiment, the use of "first" and "second" to limit objects and the marking postures of objects is only used to facilitate description and distinction, and does not represent information such as order, level, quantity, etc.

[0043] In this embodiment, in the process of training the posture prediction model based on the automatically labeled sample image, the labeling error of the posture labeled on the sample image can be predicted according to the posture prediction result output by the posture prediction model. Based on the posture labeled on the sample image and the calculated labeling error, the posture prediction model can continue to be trained until the loss caused by the labeling error converges to a specified value. Based on this implementation method, the error of the supervision signal can be compensated to reduce the impact of the labeling error on the sample image on the training process of the posture prediction model, and then the posture prediction model with higher performance obtained by training can improve the prediction accuracy of the posture prediction model.

[0044] In the above and following embodiments of the present application, optionally, the posture prediction model can be implemented as an artificial neural network model (Neural Networks, NN), such as: Convolutional Neural Networks (CNN), Deep Neural Networks (DNN), Graph Convolutional Neural Networks (GCN), Recurrent Neural Networks (RNN) and Long Short-Term Memory Neural Networks (LSTM), or can be deformed by one or more of the above neural networks, which is not limited in this embodiment.

[0045] The following will be combined Figure 2a , further illustrating the data processing method provided in the embodiments of the present application.

[0046] Figure 2a A flowchart of a data processing method provided for another exemplary embodiment of the present application is shown in FIG. Figure 2aAs shown, the method includes:

[0047] Step 201: Identify posture key points of a first object in a sample image.

[0048] Step 202: Calculate the posture key points of the first object using a 3DMM algorithm to mark the first posture of the first object on the sample image.

[0049] Step 203: input the sample image into a posture prediction model to obtain a posture classification probability of the first object and a second posture identified according to the posture classification probability.

[0050] Step 204: Fit the posture classification probability and the second posture input error to a network.

[0051] Step 205: In the error fitting network, a classification annotation error is calculated according to the posture classification probability, and a regression annotation error is calculated according to the second posture.

[0052] Step 206: construct a classification loss function according to the first posture and the classification annotation error, and construct a regression loss function according to the first posture and the regression annotation error.

[0053] Step 207: iteratively train the posture prediction model and the error fitting network to reduce the classification loss function and the regression loss function.

[0054] Step 208: When the classification loss function converges to a first range and the regression loss function converges to a second range, output the posture prediction model for performing posture prediction on the object in the image to be processed.

[0055] In step 201, a sample image may be input into a key point prediction model, and an output result of the key point prediction model may be obtained to obtain the posture key points of the first object in the sample image.

[0056] Optionally, the key point prediction model can be implemented as MTCNN, Multi-task convolutional neural network or a key point prediction model based on Openpose (a human posture recognition project) framework or other models that can perform key point prediction, which is not limited in this implementation.

[0057] Alternatively, if Figure 2b As shown, when the first object in the sample image is a human head, the posture key points identified from the sample image are N human face key points, where N is a positive integer.

[0058] After acquiring the posture key points of the first object, step 202 may be performed next to calculate the key points using a posture prediction algorithm.

[0059] In step 202, in order to facilitate the distinction from the posture predicted in the subsequent training process, the posture of the first object marked on the sample image is described as the first posture. Optionally, the first posture of the first object can be represented by the posture angles of the first object in three different spatial dimensions. The posture angles in the three spatial dimensions include: pitch angle, yaw angle, and roll angle.

[0060] by Figure 2b Taking the Cartesian coordinate system (XYZ coordinate system) shown in FIG. 1 as an example, pitch refers to the attitude angle generated when the object rotates around the X-axis; yaw refers to the attitude angle generated when the object rotates around the Y-axis; and roll refers to the attitude angle generated when the object rotates around the Z-axis.

[0061] like Figure 2b As shown, taking the first object implemented as a human head as an example, among the posture angles of the human head in three spatial dimensions, pitch can be implemented as: the posture angle of raising / lowering the head, roll can be implemented as the posture angle of shaking the head, and yaw can be implemented as the posture angle of turning the head.

[0062] Continuing to take the first object as a human head as an example, optionally, the posture prediction algorithm can be implemented as a 3DMM algorithm, such as Figure 2c As shown in the figure, 3DMM is a method for 3D reconstruction of human face, which can be used to calculate the mapping relationship between 2D human face key points and 3D human face key points, so as to calculate the posture of the human head in space. After the posture angle of the head is calculated, the posture angle can be annotated on the sample image to generate a sample image annotated with the expected value.

[0063] In this step, for the convenience of description, the posture angles in the three spatial dimensions included in the first posture are marked as: yaw * , pitch * , roll * .

[0064] Affected by the accuracy of the key point recognition algorithm and the accuracy of the 3DMM algorithm, there may be a certain error between the first posture and the actual posture of the first object. Next, a posture prediction model may be trained based on the labeled sample images.

[0065] In step 203, the sample image may be input into the posture prediction model, and the sample image may be calculated according to the current model parameters of the posture prediction model to predict the posture of the first object in the sample image to obtain a posture prediction result.

[0066] The posture prediction result includes: posture classification probability and posture angle identified according to the posture classification probability. The posture angle is a multi-dimensional posture angle. In this embodiment, for ease of description and distinction, the multi-dimensional posture angle identified and output by the posture prediction model is described as the second posture.

[0067] Among them, the posture classification probability refers to the probability that the posture of the first object belongs to each posture category contained in the three spatial dimensions, such as Figure 2d and Figure 2e The p shown yaw , p pitch , p roll Based on this probability, the second posture can be calculated, such as Figure 2d and Figure 2e Yaw, pitch, roll as shown.

[0068] The posture category is obtained by dividing the posture angle range contained in each spatial dimension. For example, the posture angle range contained in the dimension where the X-axis is located can be divided into 90 posture categories, 180 posture categories, or 360 posture categories, etc., which is not limited in this embodiment.

[0069] Taking the first object implemented as a human head as an example, the posture angle range of the human head in the X-axis (horizontal), Y-axis (vertical), and Z-axis (longitudinal) dimensions is 0° to 180°. Optionally, in some embodiments, the posture angle range of 180° corresponding to each dimension can be divided into 180 posture categories, and the angle difference between any two adjacent posture categories is 1°. In other embodiments, the posture angle range of 180° corresponding to each dimension can be divided into 360 posture categories, and the angle difference between any two adjacent posture categories is 0.5°, which is not limited in this embodiment.

[0070] In the following embodiments, the 180° posture angle range corresponding to each dimension is divided into 180 posture categories as an example for exemplary description. In this step, the posture classification probability output by the posture prediction model may include: the probability p of the first object belonging to the 180 posture categories on the dimension where the Y axis is located yaw , the probability p that the posture of the first object belongs to the 180 posture categories on the dimension where the X-axis is located pitch , the probability p that the posture of the first object belongs to the 180 posture categories on the dimension where the Z axis is located roll Among them, p yaw ∈R 1×180 , p pitch ∈R 1×180 , p roll ∈R 1×180 , R represents the matrix, p yaw 、p pitch 、proll The value range belongs to [0, 1].

[0071] Optionally, in the second posture, the posture angle in each spatial dimension can be obtained by weighted calculation based on the probability that the posture of the first object belongs to each posture category and the angle corresponding to each posture category. yaw , p pitch , p roll The corresponding classification probability value is used as the weight to sum the posture categories and obtain the posture angles yaw, pitch, and roll in each spatial dimension. The range of the posture angle is [0°, 180°]. Subtracting a 90° offset from the calculated posture angle can obtain the posture angles of the first object in three directions in space.

[0072] Taking any spatial dimension as an example, based on the probability that the posture of the first object belongs to each posture category in the spatial dimension, an optional implementation method of calculating the posture angle corresponding to the first object in the spatial dimension can be shown as the following formula 1:

[0073]

[0074] Among them, i represents the i-th posture category in the spatial dimension, p i represents the probability of belonging to the i-th posture category, and I represents the calculated posture angle in this spatial dimension.

[0075] By using the above formula 1 to calculate the posture angles in the three spatial dimensions respectively, the posture angle yaw of the first object in the dimension where the Y axis is located, the posture angle pitch of the first object in the dimension where the X axis is located, and the posture angle roll of the first object in the dimension where the Z axis is located can be obtained.

[0076] It is worth noting that before executing step 204, the posture prediction model may be pre-trained according to the output result of the posture prediction model and the supervisory signal marked on the sample image. The pre-training process is as follows: Figure 2d As shown, in Figure 2d In the above example, according to the posture classification probability p yaw , p pitch , p roll and the first pose yaw marked on the sample image * , pitch * , roll * Construct the classification loss function loss1, and predict the second posture yaw, pitch, roll and the first posture yaw output by the posture prediction model * , pitch * , roll *Construct regression loss function loss2.

[0077] Among them, loss1 can be implemented as a cross entropy loss function, and loss2 can be implemented as an average error loss function, which is not limited in this embodiment. Optionally, loss3=loss2+loss2. During pre-training, the parameters of the posture prediction model can be adjusted to reduce the value of loss3.

[0078] Figure 2d The pre-training process shown does not yet introduce the influence of the labeling error on the sample image on the posture prediction model. Based on the above pre-training, steps 204 to 208 can be performed to further train the posture prediction model in combination with the error fitting network to reduce the influence of the labeling error on the sample image on the posture prediction model.

[0079] In step 204, the posture classification probability and the second posture may be input into an error fitting network.

[0080] Optionally, in this embodiment, the error fitting network includes 6 fully connected layers. Among them, there are three fully connected layers corresponding to the output results of the classification probabilities in three spatial dimensions, and there are another three fully connected layers corresponding to the output results of the attitude angles in three spatial dimensions. A typical network structure can be as follows Figure 2e shown.

[0081] like Figure 2e As shown, it can be found in p yaw , p pitch , p roll Then add a fully connected layer: layer_cls1, layer_cls2 and layer_cls3. The parameters of layer_cls1, layer_cls2 and layer_cls3 are w cls1 , w cls2 , w cls3 , w cls1 ∈R 180×180 , w cls2 ∈R 180×180 , w cls3 ∈R 180×180 Layer_cls1, layer_cls2 and layer_cls3 are used to estimate the classification annotation errors in three spatial dimensions, respectively. The classification annotation errors can reflect the impact of the annotation errors on the classification calculation.

[0082] Optionally, in this embodiment, a confusion matrix can be determined according to the first posture and the posture classification probability output by the posture prediction model, and the fully connected layer w can be determined according to the confusion matrix. cls1 , w cls2, w cls3 The initial model parameters. For example, according to yaw * and p yaw The confusion matrix formed by w cls1 Initialize; according to pitch * and p pitch The confusion matrix formed by w cls2 Initialize; according to roll * and p roll The confusion matrix formed by w cls3 Initialize.

[0083] Among them, each column of the confusion matrix represents the predicted category, and the total number of each column represents the number of data predicted to be of this category; each row represents the true category of the data, and the total number of data in each row represents the number of data instances of this category.

[0084] Continue to refer Figure 2e As shown, a fully connected layer layer_reg1, layer_reg2 and layer_reg3 can be added after yaw, pitch and roll respectively. Among them, the parameters of the fully connected layers layer_reg1, layer_reg2 and layer_reg3 are w reg1 , w reg2 , w reg3 , w reg1 ∈R 1×1 , w cls2 ∈R 1×1 , w cls3 ∈R 1×1 Layer_reg1, layer_reg2 and layer_reg3 are used to estimate the regression annotation error, which is used to reflect the impact of the annotation error on the regression calculation.

[0085] Optionally, w cls1 , w cls2 , w cls3 The initialization value can be set to 1.

[0086] Next, step 205 may be performed to calculate the labeling error. Optionally, in this embodiment, the labeling error includes a classification labeling error and a regression labeling error.

[0087] Alternatively, if Figure 2e As shown, in the error fitting network, the classification probability p in the three spatial dimensions can be yaw , p pitch , p rollThe three fully connected layers layer_cls1, layer_cls2 and layer_cls3 in the error fitting network are input respectively, and the posture angles yaw, pitch and roll in three dimensions are input respectively into the three fully connected layers layer_regl, layer_reg2 and layer_reg3 in the error fitting network.

[0088] The following will take any one of the fully connected layers among layer_cls1, layer_cls2 and layer_cls3 as an example to illustrate the optional implementation method of calculating the classification labeling error, and take any one of the fully connected layers among layer_reg1, layer_reg2 and layer_reg3 as an example to illustrate the optional implementation method of calculating the regression labeling error.

[0089] For ease of description, any one of layer_cls1, layer_cls2 and layer_cls3 is described as the first fully connected layer, and the dimension to which the classification probability of the input first fully connected layer belongs is described as the first dimension. Any one of layer_reg1, layer_reg2 and layer_reg3 is described as the second fully connected layer, and the dimension to which the attitude angle of the input second fully connected layer belongs is described as the second dimension.

[0090] For the first fully connected layer in , the classification probability on the first dimension of the input can be calculated according to the model parameters of the first fully connected layer to obtain the classification labeling error of the first dimension.

[0091] For example, Figure 2e As shown, p can be calculated in the fully connected layer layer_cls1 yaw *w cls1 , and obtain the classification labeling error p in the Y-axis direction yaw1 ; In the fully connected layer layer_cls2, calculate p pitch *w cls2 , and obtain the classification labeling error p in the X-axis direction pitch1 ; In the fully connected layer layer_cls3, calculate p roll *w cls3 , and obtain the classification labeling error p in the X-axis direction roll1 .

[0092] For the second fully connected layer, the posture angle on the second dimension of the input can be calculated according to the model parameters of the second fully connected layer to obtain the regression annotation error of the second dimension.

[0093] For example, Figure 2eAs shown, yaw*w can be calculated in the fully connected layer layer_reg1 reg1 , get the regression annotation error yaw1 in the Y-axis direction; in the fully connected layer layer_reg2, calculate pitch*w reg2 , get the regression annotation error pitch1 in the X-axis direction; in the fully connected layer layer_reg3, calculate roll*w reg3 , and obtain the regression annotation error roll1 in the Z-axis direction.

[0094] After the classification labeling error and the regression labeling error are calculated, step 206 may be performed next.

[0095] In steps 206 to 208, a classification loss function may be constructed based on the first posture and the classification labeling error, and a regression loss function may be constructed based on the first posture and the regression labeling error. * , pitch * , roll * and p yaw1 , p pitcM , p roll1 Construct classification loss function loss_cls; according to yaw * , pitch * , roll * The average error loss function loss_reg is constructed with yaw1, pitch1, and roll1. Optionally, the classification loss function can be implemented as a cross entropy loss function for classification calculation, and the regression loss function can be implemented as an average error loss function for regression calculation. An exemplary description will be given below.

[0096] Optionally, the calculation process of the classification loss function may be as shown in the following formula 2:

[0097]

[0098] In Formula 2, n represents the number of training samples, M represents the number of posture categories (M = 180), k represents the kth sample, and j represents the jth posture category; if the posture category predicted by the kth sample is the same as the labeled posture category, then y kj =1, if different, then y kj =0; p kj It represents the probability that the posture of the object in the kth sample output by the error fitting network belongs to the jth category, that is, the classification annotation error p yaw1 , p pitch1 , p roll1 .

[0099] The calculation process of the regression loss function can be shown in the following formula 3:

[0100]

[0101] In formula 3, n represents the number of training samples, y k Represents the posture angle marked in the kth sample, i.e. yaw * , pitch * , roll * ;p k Represents the attitude angle of the kth sample output by the error fitting network, that is, the attitude regression annotation error yaw1, pitch1, roll1.

[0102] Based on the above loss function, the posture prediction model can be iteratively optimized until the classification loss function converges to the first range and the regression loss function converges to the second range. The first range and the second range are set according to actual training requirements and are not limited in this embodiment.

[0103] In some optional embodiments, the classification loss function and the regression loss function can be superimposed to obtain a joint loss function loss fin , which is shown in Formula 4:

[0104] loss fin =loss -cls +loss_reg formula 4

[0105] Based on the joint loss function loss fin When iteratively optimizing the posture prediction model, the joint loss function loss fin When the training converges to the specified value, the training is stopped and the posture prediction model obtained by training is output. When the posture prediction model is put into use, the image to be processed can be input into the posture prediction model so that the posture prediction model predicts the posture of the object in the input image to be processed.

[0106] In this embodiment, on the one hand, automatic labeling is adopted for sample images, thereby reducing the time and labor costs required for labeling. On the other hand, the posture prediction model is trained based on the labeling error of the image sample, which can compensate for the error of the supervision signal and reduce the influence of the labeling error on the sample image on the training process of the posture prediction model. Thus, a posture prediction model with higher performance is obtained through training, thereby improving the prediction accuracy of the posture prediction model.

[0107] An embodiment of the present application also provides a posture prediction method, which can be implemented based on the posture prediction model recorded in the aforementioned embodiments, and will be exemplarily described below with reference to the accompanying drawings.

[0108] Figure 3aA flowchart of a posture prediction method provided by an exemplary embodiment of the present application is shown in FIG. Figure 3a As shown, the method includes:

[0109] Step 301a: Acquire an image to be processed containing a first object.

[0110] Step 302a: input the image to be processed into a posture prediction model to obtain a posture prediction result of the first object; wherein the posture prediction model is trained based on automatically annotated sample images and predicted annotation errors of the sample images.

[0111] Step 302a: display the posture prediction result of the first object.

[0112] In this embodiment, the image to be processed may be an image obtained by photographing an object in a variety of scenarios. The various scenarios may include scenarios that require posture recognition or analysis. For example, in one scenario, when analyzing the posture of a human head, the image to be processed may be an image captured of a person's head. In another scenario, when performing posture analysis on a drone, the image to be processed may be an image frame in a video obtained by tracking and photographing the drone. In yet another scenario, when monitoring the working posture of workshop equipment, the image to be processed may be an image obtained by monitoring and photographing the workshop, which will not be described in detail.

[0113] The posture prediction model is obtained by training based on the automatically labeled sample images and the labeling errors of the sample images. The training process can refer to the records of the above embodiments and will not be repeated here.

[0114] Figure 3b A flowchart of a posture prediction method provided by an exemplary embodiment of the present application is shown in FIG. Figure 3a As shown, the method includes:

[0115] Step 301b: respond to a call request from the client to the first interface, and determine a model training service resource corresponding to the first interface, wherein the model training service resource includes a sample image.

[0116] Step 302b: annotate the sample image to obtain a first annotation result.

[0117] Step 303b: input the labeled sample image into the neural network model to obtain the output result of the neural network.

[0118] Step 304b: performing error fitting according to the output result to calculate the labeling error of the first labeling result.

[0119] Step 305b: Optimize the neural network model according to the first labeling result and the labeling error to obtain a result model.

[0120] Step 306b: Return the result model to the client.

[0121] The execution subject of this embodiment may be a server device, such as a conventional server or a cloud server, wherein the client may be implemented as a mobile phone, computer, tablet computer or other device on the user side.

[0122] In this embodiment, the data processing methods provided in the above embodiments can be encapsulated as model training service resources that can be used by third parties, such as SaaS (Software-as-a-Service) tools. Among them, the SaaS tool can be implemented as a plug-in or an application. The plug-in or application can be deployed on the server device, and third-party users can easily access and use the model training service provided by the model training service resource by calling the corresponding interface on the server.

[0123] For example, in some scenarios, the SaaS tool can be deployed on a cloud server, and third-party users can access the cloud server to use the SaaS tool online. The SaaS tool can provide sample images required for model training, automatically annotate the sample images, and perform model training based on the annotated sample images.

[0124] In this embodiment, the annotation result of the SaaS tool on the sample image is described as the first result. It should be understood that when the function of the neural network model is different, the sample image is different, and the first annotation result obtained by automatically annotating the sample image is also different. For example, when the neural network model is applied to target detection, the first annotation result can be the result of annotating the target position on the sample image containing the target and the category to which the target belongs. For example, when the neural network model is applied to face recognition, the first annotation result can be the result of annotating the facial area on the face sample image and the name of the person to whom the facial area belongs. For another example, when the neural network model is applied to posture prediction, the first annotation result can be the result of annotating the posture of the object on the sample image, which will not be described one by one.

[0125] After the first annotation result is input into the neural network model, the neural network model can perform calculations based on the model parameters using the set algorithm and output the calculation results. Continuing with the above example, when the first annotation result is the target position and its classification, the output result of the neural network model can be the predicted position and predicted classification of the target; when the first annotation result is the facial area and its corresponding character name, the output result of the neural network model can be the predicted result of the character name; when the first annotation result is implemented as the posture of the object, the output result of the neural network model can be the predicted result of the posture of the object.

[0126] In the process of training the neural network model based on automatically annotated sample images, the SaaS tool can estimate the labeling error generated in the automatic labeling process based on the output results of the neural network model, and further optimize the neural network model according to the first labeling result annotated on the sample image and the estimated labeling error to obtain the result model, and return the result model to the client for offline use.

[0127] For example, following the above example, in some optional embodiments, when training a neural network model for posture prediction, a loss function can be constructed according to the posture and annotation error of the object marked on the sample image, and the neural network model can be optimized according to the loss function; when the loss converges to a specified range, the optimized neural network model is output as a posture prediction model, and the posture prediction model is returned to the client as a result model. For details, please refer to the records of the above embodiments, which will not be repeated here.

[0128] In other optional embodiments, when optimizing the neural network model for target detection, the predicted position and predicted classification of the target output by the neural network model during the training process can be input into the error fitting network, and the labeling error introduced when automatically labeling the target position and its classification on the sample image can be determined based on the output of the error fitting network. Based on the labeling error and the labeled target position and its classification, a loss function is constructed, and the neural network model is continuously optimized based on the loss function until the loss function converges to a certain range, thereby obtaining a target detection model, which will not be described in detail.

[0129] It should be noted that, in some optional embodiments, the SaaS tool may determine whether to execute the model training method provided in this embodiment based on the client user's demand for the neural network model. For example, in some scenarios, the user requests to obtain a posture prediction model with higher accuracy, and the SaaS tool may execute steps 304b and 305b when training the neural network model. During the training process, the neural network model is optimized based on the labeling error obtained by fitting and the first labeling result on the sample image, thereby reducing the impact of the labeling error on the neural network model and improving the prediction accuracy of the result model obtained by the final training.

[0130] Alternatively, the SaaS tool can automatically determine whether to use the model training method provided in the embodiment of the present application to train the neural network model based on the application scenario of the result model. For example, in some scenarios, when the result model is used in a small target detection scenario, the SaaS tool can automatically execute the model training method provided in this embodiment to train a high-precision small target detection model, which will not be repeated.

[0131] It is also worth noting that the model training service resources provided in this embodiment can be used to train a variety of different neural network models, for example, one or more CNN models, DNN models, GCN models, RNN models, and LSTM models, or can also be applied to other neural network models obtained by deforming one or more of the above neural networks, which will not be described one by one.

[0132] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 to 203 can be device A; for another example, the execution subject of steps 201 and 202 can be device A, and the execution subject of step 203 can be device B; and so on.

[0133] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 201, 202, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0134] In the above and below embodiments of the present application, the sample image may be an image obtained by photographing a human head, the first object in the sample image includes a human head, and the first posture is implemented as a posture angle of the human head in three-dimensional space marked on the sample image. When training a posture prediction model, multiple face sample images may be obtained, and the head postures may be automatically marked on the face sample images. Based on the marked sample images, a posture prediction model may be trained. During the training process, the marking error may be further fitted according to the posture prediction results output by the posture prediction model. Then, the posture prediction model may be further trained based on the marking error, thereby reducing the influence of the error generated by the automatic marking and the error generated by the training algorithm during the regression calculation process on the model accuracy.

[0135] The posture prediction model provided in the embodiment of the present application can be applied to a variety of different facial recognition and analysis scenarios. For example, in the application scenario of a smart classroom, the head posture of the student can be detected, and the facial image of the student when the head posture is better can be obtained; based on the acquired facial image, facial recognition and analysis can be performed to realize the operation of automatically recording attendance and automatically analyzing the emotions of students in class. For another example, in the in-vehicle assisted driving scenario, the driver's head posture can be detected, and the facial image of the driver when the head posture is better can be obtained; based on the acquired facial image, facial recognition and analysis can be performed to realize automatic driver authentication, automatic matching of the driver's driving mode, fatigue driving analysis and early warning operations, etc. In addition to the above scenarios, it can also be applied to vehicle face recognition scenarios, face payment scenarios, etc., and this embodiment includes but is not limited to this.

[0136] It should be understood that in addition to facial recognition and analysis scenarios, the technical solutions provided in the embodiments of the present application can also be applied to posture recognition of other body parts. For example, in the scenario of automatic proctoring, the head posture and body posture of the students can be detected, and based on the detection results of the head posture and body posture, it is analyzed whether the students have cheated in the examination room. For example, in sports training scenarios or in sports competition scenarios, the posture of the athletes can be detected, and the athletes' sports events can be identified based on the detected postures, and whether the athletes have made illegal actions can be automatically analyzed, etc.

[0137] The following will be combined Figure 4 , taking the smart classroom scene as an example, the posture prediction method provided by the embodiment of the present application is further explained. Figure 4 As shown, a smart classroom management system may include an image acquisition device 41, a server 42, and a terminal device 43. The image acquisition device 41 is deployed in the classroom to capture images of people in the classroom to capture images containing faces. The server 42 may be implemented as a conventional server or a cloud server. The terminal device 43 may be implemented as a user-side device such as a mobile phone, a tablet computer, or a computer.

[0138] In one case, the image acquisition device 41 can continuously monitor and shoot the people in the classroom, and send the captured monitoring video to the server 42. In this case, the image acquisition device 41 can indiscriminately capture facial images of people in a variety of head postures.

[0139] To improve the efficiency of face recognition, after receiving the surveillance video sent by the image acquisition device 43, the server 42 can input the surveillance video into the posture prediction model. The posture prediction model can divide the input surveillance video into frames, perform posture prediction based on multiple face images obtained by framing, and output the posture recognition result of each image. The posture recognition result can be implemented as the posture angle predicted by the server 42 for the face in each image. After the server 42 obtains the posture angle corresponding to the face in each image, it can select an image with a better head posture from the multiple face images obtained by framing the surveillance video according to the posture angle, such as an image with posture angles of about 90° in three spatial dimensions, that is, a frontal face image.

[0140] In another case, the posture prediction model may be located on one side of the image acquisition device 42. During the shooting process, the image acquisition device 41 may recognize the head posture of the characters in the classroom in real time based on the posture prediction model, and capture the facial image with a better head posture according to the recognition result. Then, the image acquisition device 42 may send the captured facial image with a better posture to the server 42, so that the server 42 performs facial recognition or analysis. In this embodiment, the data sent from the image acquisition device 41 to the server 42 has a smaller data volume, which improves the transmission efficiency.

[0141] Based on the above two implementations, after the server 42 obtains the facial image with a better head posture, it can perform facial recognition on the person in the image with a better head posture to obtain a facial recognition result. The facial recognition result may include the name, facial expression, etc. of each person in the classroom. In some cases, after identifying the name of each person, the server 42 can mark the name of the person on the image and send the image marked with the name to the terminal device 43. The terminal device 43 can be held by the administrator of the classroom (for example, a teacher). After receiving the image marked with the name, the terminal device 43 can display the image so that the administrator can understand the student's attendance and the student's classroom mood at a glance.

[0142] Figure 5 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, such as Figure 5 As shown, the electronic device includes: a memory 501 and a processor 502.

[0143] The memory 501 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0144] Among them, the memory 501 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0145] The processor 502 is coupled to the memory 501 and is used to execute the computer program in the memory 501, so as to: mark a first posture of a first object on a sample image; input the sample image into a posture prediction model to obtain a posture prediction result of the first object; perform error fitting according to the posture prediction result to estimate the marking error of the first posture; and optimize the posture prediction model according to the first posture and the marking error.

[0146] Further optionally, the processor 502 is also used to: output the posture prediction model when the loss of the posture prediction model converges to a specified range; input the image to be processed into the posture prediction model to perform posture estimation on a second object in the image to be processed.

[0147] Further optionally, when the processor 502 marks the first posture of the first object on the sample image, it is specifically used to: identify the posture key points of the first object from the sample image; and calculate the posture key points of the first object using a 3DMM algorithm to mark the first posture of the first object on the sample image.

[0148] Further optionally, the posture prediction result includes: a posture classification probability and a second posture identified according to the posture classification probability.

[0149] Further optionally, when the processor 502 performs error fitting based on the posture prediction result to estimate the labeling error of the first posture, it is specifically used to: input the posture classification probability and the second posture into an error fitting network; in the error fitting network, calculate the classification labeling error based on the posture classification probability, and calculate the regression labeling error based on the second posture.

[0150] Further optionally, when the processor 502 inputs the posture classification probability and the second posture into the error fitting network, it is specifically used to: input the classification probabilities on the three spatial dimensions in the posture classification probability into the three fully connected layers in the error fitting network respectively; accordingly, in the error fitting network, according to the posture classification probability, calculate the classification labeling error, including: for the first fully connected layer among the three fully connected layers, according to the model parameters of the first fully connected layer, calculate the classification probability on the input first dimension to obtain the classification labeling error of the first dimension.

[0151] Further optionally, the processor 502 is also used to: determine a confusion matrix according to the first posture and the posture classification probability; and determine initial model parameters of the first fully connected layer according to the confusion matrix.

[0152] Further optionally, when the processor 502 inputs the posture classification probability and the second posture into the error fitting network, it is specifically used to: input the posture angles on the three spatial dimensions in the second posture into the three fully connected layers in the error fitting network respectively; for the second fully connected layer among the three fully connected layers, calculate the posture angle on the input second dimension according to the model parameters of the second fully connected layer, and obtain the regression annotation error of the second dimension.

[0153] Further optionally, when the processor 502 optimizes the posture prediction model according to the first posture and the labeling error, it is specifically used to: construct a classification loss function according to the first posture and the classification labeling error, and construct a regression loss function according to the first posture and the regression labeling error; iteratively train the posture prediction model and the error fitting network until the classification loss function converges to a first range and the regression loss function converges to a second range.

[0154] Further optionally, the classification loss function is a cross entropy loss function, and the regression loss function is a mean error loss function.

[0155] Further optionally, the first object includes: a human head, and the first posture includes: a posture angle of the human head in three-dimensional space.

[0156] Further optionally, the first object includes: a human head, and the first posture includes: a posture angle of the human head in three-dimensional space.

[0157] Further, if Figure 5 As shown, the electronic device also includes: a communication component 503, a display component 504, a power component 505, an audio component 506 and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0158] Among them, the communication component 503 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0159] Among them, the display component 504 includes a screen, and the screen may include a liquid crystal display component (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0160] The power supply component 505 provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0161] In this embodiment, in the process of training the posture prediction model based on the automatically labeled sample image, the labeling error of the posture labeled on the sample image can be predicted according to the posture prediction result output by the posture prediction model. Based on the posture labeled on the sample image and the calculated labeling error, the posture prediction model can continue to be trained until the loss caused by the labeling error converges to a specified value. Based on this implementation method, the error of the supervision signal can be compensated to reduce the impact of the labeling error on the sample image on the training process of the posture prediction model, and then the posture prediction model with higher performance obtained by training can improve the prediction accuracy of the posture prediction model.

[0162] In addition to the execution logic described in the above embodiments, Figure 5 The illustrated electronic device can also execute the following data processing logic: the processor 502 obtains an image to be processed containing a first object; the image to be processed is input into a posture prediction model to obtain a posture prediction result of the first object; wherein the posture prediction model is trained based on automatically annotated sample images and predicted annotation errors of the sample images; and the posture prediction result of the first object is displayed.

[0163] In addition to the execution logic described in the above embodiments, Figure 5 The illustrated electronic device can also execute the following data processing logic: the processor 502 responds to a call request from the client to the first interface, determines the model training service resources corresponding to the first interface, wherein the model training service resources include sample images; annotates the sample images to obtain a first annotation result; inputs the annotated sample images into the neural network model to obtain an output result of the neural network; performs error fitting based on the output result to calculate the annotation error of the first annotation result; optimizes the neural network model based on the first annotation result and the annotation error to obtain a result model; and returns the result model to the client.

[0164] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be performed by an electronic device in the above method embodiment.

[0165] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0170] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0171] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0172] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0173] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: include: In response to a call request from a client to a first interface, determine a model training service resource corresponding to the first interface, wherein the model training service resource includes a sample image; Automatically annotating the sample image to obtain a first annotation result; Inputting the labeled sample image into the neural network model to obtain the output result of the neural network; Calculating a loss function according to the output result and the first labeling result, and pre-training the neural network model according to the loss function; Performing error fitting according to the output result to calculate a labeling error of the first labeling result; According to the first labeling result and the labeling error, continue to optimize the neural network model to obtain a result model; The result model is returned to the client.

2. The method according to claim 1, characterized in that The first annotation result includes: the posture of the object annotated on the sample image, the output result is a posture prediction result, and the result model is a posture prediction model.

3. The method according to claim 2, characterized in that Optimizing the neural network model according to the first labeling result and the labeling error to obtain a result model includes: constructing another loss function according to the posture of the object and the annotation error; The other loss function optimizes the neural network model; When the loss converges to a specified range, the optimized neural network model is output as the posture prediction model.

4. A posture prediction method, characterized in that: include: Acquire an image to be processed containing a first object; Input the image to be processed into a posture prediction model to obtain a posture prediction result of the first object; wherein the posture prediction model is obtained by performing two-stage training based on a sample image with a first posture automatically labeled, a loss function, and a predicted labeling error of the sample image; the loss function is calculated based on the first posture and the posture prediction model for the posture prediction result of the sample image, and the labeling error is obtained by performing error fitting on the posture prediction result of the sample image based on the posture prediction model; The posture prediction result of the first object is displayed.

5. A data processing method, characterized in that: include: automatically annotating a first pose of a first object on a sample image; Inputting the sample image into a posture prediction model to obtain a posture prediction result of the first object; Calculating a loss function according to the posture prediction result and the first posture, and pre-training the posture prediction model according to the loss function; Performing error fitting according to the posture prediction result to estimate the labeling error of the first posture; The posture prediction model continues to be optimized according to the first posture and the annotation error.

6. The method according to claim 5, characterized in that Also includes: When the loss of the posture prediction model converges to a specified range, outputting the posture prediction model; The image to be processed is input into the posture prediction model to perform posture estimation on the second object in the image to be processed.

7. The method according to claim 5, characterized in that Annotating a first posture of a first object on a sample image includes: From the sample image, identifying posture key points of the first object; The posture key points of the first object are calculated using a 3DMM algorithm to mark the first posture of the first object on the sample image.

8. The method according to claim 5, characterized in that The posture prediction result includes: a posture classification probability and a second posture identified according to the posture classification probability.

9. The method according to claim 8, characterized in that Performing error fitting according to the posture prediction result to estimate the labeling error of the first posture includes: Fitting the posture classification probability and the second posture input error to a network; In the error fitting network, a classification annotation error is calculated according to the posture classification probability, and a regression annotation error is calculated according to the second posture.

10. The method according to claim 9, characterized in that Fitting the posture classification probability and the second posture input error to a network, comprising: Inputting the classification probabilities on three spatial dimensions in the posture classification probability into three fully connected layers in the error fitting network respectively; In the error fitting network, the classification labeling error is calculated according to the posture classification probability, including: For the first fully connected layer of the three fully connected layers, the classification probability on the first dimension of the input is calculated according to the model parameters of the first fully connected layer to obtain the classification labeling error of the first dimension.

11. The method according to claim 10, characterized in that Also includes: Determining a confusion matrix based on the first posture and the posture classification probability; According to the confusion matrix, initial model parameters of the first fully connected layer are determined.

12. The method according to claim 9, characterized in that Fitting the posture classification probability and the second posture input error to a network, comprising: Inputting the posture angles in three spatial dimensions of the second posture into three fully connected layers in the error fitting network respectively; For the second fully connected layer among the three fully connected layers, the posture angle on the second dimension of the input is calculated according to the model parameters of the second fully connected layer to obtain the regression annotation error of the second dimension.

13. The method according to any one of claims 9 to 12, characterized in that: Optimizing the posture prediction model according to the first posture and the annotation error includes: constructing a classification loss function according to the first posture and the classification annotation error, and constructing a regression loss function according to the first posture and the regression annotation error; The posture prediction model and the error fitting network are iteratively trained until the classification loss function converges to a first range and the regression loss function converges to a second range.

14. The method according to claim 13, characterized in that The classification loss function is a cross entropy loss function, and the regression loss function is a mean error loss function.

15. The method according to any one of claims 5 to 12, characterized in that: The first object includes: a human head, and the first posture includes: a posture angle of the human head in three-dimensional space.

16. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to: perform the steps in the method according to any one of claims 1-15.

17. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the method described in any one of claims 1 to 15 can be implemented.

Images