Pose recognition model training method and device, electronic equipment, and storage medium

Abstract

The application relates to the field of artificial intelligence, and provides a pose recognition model training method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a pose sample for model training; taking the pose sample as the input of a pose recognition model; performing enhancement processing on the pose sample by using an enhancement network to obtain an enhanced sample, and inputting the pose sample and the enhanced sample into a feature extraction network to obtain target features; performing feature transformation and loss calculation on the target features by using a first prediction network to obtain a first pose loss; performing feature reconstruction and loss calculation on the target features by using a second prediction network to obtain a second pose loss and a feature consistency loss; and performing reverse updating on the parameters of the model according to the first pose loss, the second pose loss and the feature consistency loss, so as to train the pose recognition model. The application solves the problem that the prior art cannot constrain the features of a shielding sample from the algorithmic point of view, and improves the pose recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, electronic device and storage medium for training a pose recognition model. Background Technology

[0002] Human pose estimation is an important research task in the field of computer vision. Its main goal is to accurately locate and estimate the positions of various key points of the human body in images or videos in order to achieve accurate recognition and understanding of human poses and movements.

[0003] With the continuous development of technology, pose estimation algorithms have made significant progress. In normal scenarios, i.e., unoccluded scenes, these algorithms achieve increasingly high recognition accuracy, meeting real-world needs. However, when there are many people and significant occlusion, these algorithms may lose some key points during the recognition process, leading to incorrect pose estimations. To address the pose estimation problem in occluded scenarios, existing algorithms often focus on data augmentation to construct a large number of occluded samples. While this method improves the robustness of the algorithm to some extent, it does not constrain the features of these occluded samples from an algorithmic perspective. Therefore, these algorithms still have certain limitations when dealing with occluded scenarios. Summary of the Invention

[0004] In view of this, embodiments of this application provide a training method, apparatus, electronic device and storage medium for a pose recognition model to solve the problem that the prior art cannot constrain the features of occluded samples from an algorithmic perspective, resulting in unsatisfactory recognition results.

[0005] A first aspect of this application provides a method for training a pose recognition model, comprising:

[0006] A sample set for model training is obtained, which contains multiple pose samples. These pose samples are used as input to the pose recognition model to be trained. The pose recognition model includes an augmentation network, a feature extraction network, a first prediction network, and a second prediction network. The augmentation network is used to perform data augmentation on the pose samples to obtain augmented samples. These augmented samples and the pose samples are then input into the feature extraction network to output target features. The first prediction network is used to perform feature transformation and loss calculation on the target features to obtain the first pose loss. The second prediction network is used to perform feature reconstruction and loss calculation on the target features to obtain the second pose loss and feature consistency loss. The model parameters are then updated in reverse based on the first pose loss, the second pose loss, and the feature consistency loss to train the pose recognition model.

[0007] A second aspect of this application provides a training apparatus for a pose recognition model, comprising:

[0008] The sample module is configured to acquire a sample set for model training, which contains multiple pose samples. The input module is configured to use the pose samples as input to the pose recognition model to be trained, wherein the pose recognition model includes an augmentation network, a feature extraction network, a first prediction network, and a second prediction network. The feature processing module is configured to use the augmentation network to perform data augmentation processing on the pose samples to obtain augmented samples, and input the pose samples and augmented samples into the feature extraction network so that the feature extraction network outputs target features. The first prediction module is configured to use the first prediction network to predict the target features and calculate the loss to obtain the first pose loss. The second prediction module is configured to use the second prediction network to predict the target features and calculate the loss to obtain the second pose loss and the feature consistency loss. The training module is configured to back-update the model parameters based on the first pose loss, the second pose loss, and the feature consistency loss in order to train the pose recognition model.

[0009] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0010] A fourth aspect of this application provides a computer storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0011] The beneficial effects of the embodiments in this application compared with the prior art are:

[0012] By acquiring pose samples for model training and using these samples as input to the pose recognition model to be trained, and then using an augmentation network to perform data augmentation on the pose samples to obtain augmented samples, the pose samples and augmented samples are input into a feature extraction network to output target features. A first prediction network is used to perform feature transformation and loss calculation on the target features to obtain the first pose loss. A second prediction network is used to perform feature reconstruction and loss calculation on the target features to obtain the second pose loss and feature consistency loss. The model parameters are then updated in reverse based on the first pose loss, second pose loss, and feature consistency loss to train the pose recognition model. Since the first pose loss and second loss can optimize the predicted joint positions to be closer to the joint positions of the real labels, and the feature consistency loss ensures that the model can learn feature representations similar to the original human body image when processing occluded human body images, the model can constrain the features of occluded samples from an algorithmic perspective, thereby improving recognition accuracy. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating a training method for a pose recognition model provided in an embodiment of this application;

[0015] Figure 2 This is a schematic diagram of the structure of a training device for a pose recognition model provided in an embodiment of this application;

[0016] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0018] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0019] Furthermore, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes that element.

[0020] The following detailed description, in conjunction with the accompanying drawings, describes a training method, apparatus, electronic device, and storage medium for a pose recognition model according to embodiments of this application.

[0021] Figure 1 This is a flowchart illustrating a training method for a pose recognition model provided in an embodiment of this application.

[0022] like Figure 1 As shown, the method includes:

[0023] S101, Obtain a sample set for model training, wherein the sample set contains multiple pose samples.

[0024] Specifically, this involves acquiring a dataset for model training. This data will be used to train the model to recognize, classify, or perform certain tasks. Pose typically refers to the posture or position of an object or person. It can include various angles, orientations, or positional states. For example, a model that recognizes different hand gestures might include pose samples of different hand postures (such as flat, clenched fist, pointing, etc.). In other words, this dataset contains samples of various postures or positions to facilitate training the model for recognition or classification.

[0025] S102, the pose sample is used as the input to the pose recognition model to be trained, wherein the pose recognition model is equipped with an enhancement network, a feature extraction network, a first prediction network and a second prediction network.

[0026] Specifically, a pose recognition model is a machine learning model used to identify or predict the pose of an object. Through training, the model learns how to accurately predict the pose of an object from the input data. This model specifically includes an augmentation network, a feature extraction network, a first prediction network, and a second prediction network.

[0027] S103, the pose sample is augmented using an augmentation network to obtain augmented samples, and the pose sample and augmented samples are input into the feature extraction network so that the feature extraction network outputs the target features.

[0028] Specifically, data augmentation refers to increasing the quantity or quality of a dataset through various techniques, making it more varied and diverse. This application employs occlusion processing, that is, randomly occluding some key points of pose samples. Since key points in human images (pose samples) (e.g., joints, key lines, etc.) are crucial information for recognizing human posture and movement, randomly occluding these key points increases the complexity of the data. Thus, during training, the model not only learns from unoccluded images but also learns to process occluded images.

[0029] It's important to note that the occlusion intensity should not be excessive. "Occlusion intensity" refers to the degree or number of keypoints obscured. Excessive occlusion can make it difficult for the model to recognize and process the image, as keypoint obscuration can lead to significant loss of image information. Therefore, to ensure effective model learning, keypoints need to be occluded appropriately.

[0030] If the occlusion is too severe, the model will face greater challenges in recognizing and processing images. Since occlusion of key points may prevent the model from accurately recognizing and predicting poses or actions, excessive occlusion can lead to training difficulties or poor performance.

[0031] Furthermore, pose samples and augmented samples are input into the feature extraction network so that the feature extraction network outputs target features, thereby improving feature representation ability, increasing model generalization ability, improving robustness, and accelerating the training and inference process.

[0032] This step utilizes an augmentation network to perform data augmentation on pose samples, obtaining augmented samples, which are key points with random occlusion. This allows the network to effectively learn and process these images. The pose samples and augmented samples are then input into a feature extraction network, enabling the feature extraction network to output target features, thereby improving feature representation capabilities and increasing the model's generalization ability.

[0033] S104, the first prediction network is used to perform feature transformation and loss calculation on the target features to obtain the first pose loss.

[0034] Specifically, loss calculation can measure the distance or error between the feature transformation result and the target feature using a certain loss function. Specific loss functions that can be used include mean squared error loss, cross-entropy loss, etc., without further limitation here.

[0035] The first prediction network can transform the target features, that is, it performs a series of transformation operations on the target features to obtain a new feature representation. This new feature representation can be used for subsequent tasks or analysis (loss calculation). After feature transformation and loss calculation, the first pose loss can be obtained, which can represent the difference between the predicted result and the true pose.

[0036] S105, the second prediction network is used to reconstruct the target features and calculate the loss, resulting in the second pose loss and feature consistency loss.

[0037] Specifically, the second prediction network can be used to reconstruct the target features, that is, to reconstruct or describe the target features using a certain algorithm or model, and then to further predict or analyze them to obtain the second pose loss and feature consistency loss.

[0038] The second pose loss refers to the pose estimation loss based on the second prediction network. Feature consistency loss measures the consistency or similarity between the reconstructed features and the original features (target features). If the reconstructed features are very similar to the original features, the consistency loss is small; conversely, if the differences are large, the loss is large. In essence, feature consistency loss measures whether the features extracted by the feature extraction network from the pose sample and the occluded human image (enhanced sample) are consistent. If the features are consistent, then the occluded image and the original image should have similar feature values ​​at the same spatial location; therefore, this similarity can be used as the feature consistency loss. During training, feature consistency loss guides the network to learn to extract the same key information from the original image and the occluded image, thereby improving the network's pose estimation ability and enabling the model to accurately estimate the pose even when faced with different pose samples and occlusion conditions.

[0039] S106, The parameters of the model are updated in reverse according to the first pose loss, the second pose loss and the feature consistency loss, so as to train the pose recognition model.

[0040] This embodiment acquires pose samples for model training and uses these samples as input to the pose recognition model to be trained. An augmentation network is used to augment the pose samples, resulting in augmented samples. These augmented samples and the pose samples are then input into a feature extraction network, causing the feature extraction network to output target features. A first prediction network is used to perform feature transformation and loss calculation on the target features, yielding a first pose loss. A second prediction network is used to reconstruct the target features and calculate the loss, yielding a second pose loss and a feature consistency loss. The model parameters are then updated in reverse based on the first pose loss, the second pose loss, and the feature consistency loss to train the pose recognition model. Since the first pose loss and the second loss can optimize the predicted joint positions, making them closer to the joint positions in the actual labels, and the feature consistency loss ensures that the model can learn feature representations similar to the original human body image when processing occluded human body images, the model can constrain the features of occluded samples from an algorithmic perspective, thereby improving recognition accuracy.

[0041] In some embodiments, the first prediction network includes an upsampling layer and a first convolutional layer. The first prediction network is used to perform feature transformation and loss calculation on the target features to obtain the first pose loss, including: upsampling the target features using the upsampling layer to obtain upsampled features; transforming the upsampled features using the first convolutional layer to obtain first transformed features; performing a local maximum operation on the first transformed features to obtain a first joint map; and calculating the first pose loss based on the first joint map and the labels corresponding to the pose samples.

[0042] Specifically, the first prediction network of the pose recognition model includes an upsampling layer and a first convolutional layer. The target features extracted by the aforementioned feature extraction network include feature information of pose samples and enhanced samples after pose sample enhancement processing.

[0043] In deep learning, the size of the target feature image obtained above is usually smaller than the size of the original pose sample image. In order to facilitate the processing of target features in subsequent steps and to retain all information related to the pose sample so that it can be accurately located and identified in subsequent steps, it is necessary to use the upsampling layer of the first prediction network to enlarge (upsample) the target features to the same size as the pose sample, and finally obtain the upsampled features after upsampling.

[0044] Furthermore, the convolutional neural network of the first convolutional layer of the first prediction network is used to perform convolutional transformation on the aforementioned upsampled features. Through convolutional transformation, more useful information can be extracted from the upsampled features, further enhancing their representational power. Then, a local maximum operation is performed on the first transformed features obtained by the convolutional transformation. This operation can enhance the edge information of the feature map or other data by taking the local maximum value in a small neighborhood of the input data. That is, by finding the maximum value in the neighborhood of the input and setting other pixel values ​​to zero, the edges and obvious features are highlighted, thus determining which feature point is the best.

[0045] Furthermore, after the aforementioned local maxima operation, some feature points representing the locations of key points can be obtained. Each of these key points has an associated line indicating the relationship between the parts. The first joint map can be calculated based on these feature points. Then, the loss value is calculated based on the first joint map and the true labels corresponding to the pose samples to understand the gap between the model's prediction and the actual target, so as to understand the model's performance, guide model training and optimization, and ultimately improve the model's prediction accuracy and generalization ability through the loss value.

[0046] This embodiment uses an upsampling layer to upsample the target features to obtain upsampled features; a first convolutional layer transforms the upsampled features to obtain first transformed features; then, a local maximum operation is performed on the first transformed features to obtain a first joint map; based on the first joint map and the labels corresponding to the pose samples, the first pose loss is calculated. Since the target features include pose samples and augmented samples, the obtained first pose loss can improve the model's understanding of the occluded parts in the pose samples. The trained model can accurately estimate human pose under occlusion conditions, improving the model's adaptability to complex environments.

[0047] Furthermore, in some embodiments, the second prediction network includes a second convolutional layer, a function layer, and a feature reconstruction module. The second prediction network is used to reconstruct the target features and calculate the loss to obtain a second pose loss and a feature consistency loss. This includes: transforming the target features using the second convolutional layer to obtain a second transformed feature; calculating the second transformed feature using the function layer, and obtaining an attention-enhanced feature based on the calculated feature and the first transformed feature; and inputting the attention-enhanced feature into the feature reconstruction module to obtain the second pose loss and feature consistency loss output by the feature reconstruction module.

[0048] Specifically, the second prediction network includes a second convolutional layer, a function layer, and a feature reconstruction module for feature reconstruction and loss calculation. In the second prediction network, the second convolutional layer transforms the target features to generate second transformed features. The function layer is a layer in the neural network that maps the input to the output by applying a function. In the second prediction network, the function layer calculates the second transformed features and combines them with the first transformed features to generate attention-enhancing features. These attention-enhancing features are obtained by the function layer calculating the second transformed features and combining them with the first transformed features. The function layer can utilize the calculated features to improve attention to key features.

[0049] Furthermore, the feature reconstruction module is a part that utilizes attention-enhanced features to generate output. It receives the attention-enhanced features as input and feeds them into the feature reconstruction module for feature reconstruction. Ultimately, the feature reconstruction module generates a second pose loss and a feature consistency loss. The feature consistency loss is a metric that measures whether the features extracted by the feature extraction network from the pose samples and the enhanced samples are consistent. It can be used to measure the consistency between the output features of the feature reconstruction module and other features. Simultaneously, it can be used to constrain the similarity of the generated features with other features under a certain metric, so that during training, it guides the network to learn to extract the same key information from the pose samples and the enhanced samples, thereby improving the network's pose estimation ability.

[0050] This embodiment utilizes a second convolutional layer to transform the target features, obtaining second transformed features. A function layer is then used to calculate these second transformed features, and based on the calculated features and the first transformed features, attention-enhanced features are derived. These attention-enhanced features are input into a feature reconstruction module, yielding a second pose loss and a feature consistency loss output by the feature reconstruction module. This second pose loss further enhances the model's understanding of occluded portions in the pose samples, and the feature consistency loss constrains the similarity of the generated features to other features under a certain metric, further improving the network's pose estimation capability.

[0051] In addition, in some embodiments, the second transformation feature is calculated using the function layer, and attention enhancement features are obtained based on the calculated features and the first transformation feature. This includes: calculating the weights of the second transformation feature using the sigmoid function of the function layer to obtain weight features; and multiplying the weight features and the first transformation feature to obtain attention enhancement features.

[0052] Specifically, the sigmoid function, also known as the sigmoid function, is a commonly used activation function used to map input values ​​to probability values ​​between 0 and 1. The sigmoid function performs a non-linear transformation on the second transformation feature, enabling it to better adapt to complex feature relationships.

[0053] Specifically, the sigmoid function maps input values ​​to probability values ​​between 0 and 1, resulting in better interpretability and interpretability of the output. In weight calculation, the sigmoid function maps each value of the second transformation feature to a weight value, which represents the importance of that feature in the attention-enhancing feature.

[0054] Furthermore, multiplying the first transformed feature with the second transformed feature enables feature fusion. In deep learning and neural networks, feature transformation and fusion are crucial steps that help improve the model's representational power and generalization performance. By multiplying the first transformed feature with the second transformed feature, feature fusion can be achieved in different feature spaces, thereby better capturing complex patterns and relationships in the data.

[0055] It is understandable that the first and second transformed features represent different feature dimensions or attributes. Through multiplication, feature combinations between different dimensions or attributes can be achieved, enhancing the expressive power of the model. Furthermore, multiplication can also reduce or compress features, helping to decrease model complexity and computational cost.

[0056] In this embodiment, the sigmoid function of the function layer is used to calculate the weights of the second transformed features, which is to achieve a weighted summation of the features. The sigmoid function maps the second transformed features to a weight value, so that each feature can receive a different weight according to its importance. Multiplying the weighted features with the first transformed features realizes the weighted summation process in the attention mechanism, thereby obtaining the attention-enhanced features. The attention mechanism enables the model to better focus on important features related to the target task, thereby improving the model's accuracy and robustness.

[0057] This embodiment calculates the weights of the second transformation features using the S-shaped function of the function layer to obtain weighted features. The weighted features are then multiplied with the first transformation features to obtain attention-enhanced features, which can improve the model's representation ability and generalization performance, enabling the model to better adapt to various complex tasks and data distributions.

[0058] In some embodiments, the feature reconstruction module includes a third convolutional layer, a first branch network, and a second branch network. The attention-enhanced features are input into the feature reconstruction module to obtain a second pose loss and a feature consistency loss output by the feature reconstruction module. This includes: transforming the attention-enhanced features using the third convolutional layer to obtain a third transformed feature; inputting the third transformed feature into the first branch network to obtain the feature consistency loss output by the first branch network; and inputting the third transformed feature into the second branch network to obtain the second pose loss output by the second branch network.

[0059] Specifically, the feature reconstruction module is a module containing a third convolutional layer, a first branch network, and a second branch network. Attention-enhanced features can be input into this module, and the second pose loss and feature consistency loss are calculated using the module's output. The feature reconstruction module obtains the feature consistency loss and the second pose loss. These losses are used to optimize the model, enabling it to better reconstruct features and estimate poses. Through the feature reconstruction module, especially its third convolutional layer, attention-enhanced features can be processed and transformed at a deeper level, thereby extracting richer and more useful feature information. This not only helps improve the model's accuracy in pose recognition but also enhances its generalization ability.

[0060] Furthermore, by inputting the third transformation features into two different branch networks (the first branch network and the second branch network), we can obtain the feature consistency loss and the second pose loss. These two losses can help the model better understand the intrinsic features of the input data and optimize the model's parameters, thereby further improving the model's performance. They also contribute to improving the model's interpretability. By understanding how the feature reconstruction module works and how to utilize different branch networks to calculate the feature consistency loss and the second pose loss, we can gain a deeper understanding of the model's decision-making process.

[0061] In this embodiment, the attention enhancement features are transformed using a third convolutional layer to obtain a third transformed feature. The third transformed feature is then input into a first branch network to obtain the feature consistency loss output by the first branch network. The third transformed feature is then input into a second branch network to obtain the second pose loss output by the second branch network. These losses are used to optimize the model so that it can better reconstruct features and estimate pose.

[0062] Furthermore, in some embodiments, the third transformation feature is input into the first branch network to obtain the feature consistency loss output by the first branch network, including: masking the third transformation feature to obtain masked features; and calculating the feature consistency loss using the masked features and attention enhancement features.

[0063] Specifically, masks are typically used to obscure or ignore certain data, allowing the focus to be on other data components during processing or analysis. In deep learning and other machine learning tasks, masks are primarily used to handle irregular or missing data. For example, when processing time-series data or images, masks can help ignore incomplete or unwanted portions. By ignoring or obscuring incomplete or noisy data, the model can be prevented from being unnecessarily affected during training, allowing it to focus more on learning other useful features and patterns.

[0064] Since the model needs to ensure the consistency of features output by different branch networks, masking helps to emphasize or highlight this consistency. By comparing the masked third transform features and attention-enhanced features, the differences or inconsistencies between them can be calculated, thus yielding the feature consistency loss. This loss can serve as an optimization objective, guiding the model to gradually reduce the differences between different branch networks during the learning process. Simultaneously, it can constrain pose sample features and enhanced sample features, allowing the network to learn better and thereby improving the model's performance and stability.

[0065] The third transformation features are masked to obtain masked features. Using the masked features and attention enhancement features, the feature consistency loss is calculated, which can help the model better handle incomplete or noisy data. At the same time, the introduction of feature consistency loss promotes the model to converge and generalize better during the learning process.

[0066] In addition, in some embodiments, the third transformation feature is input into the second branch network to obtain the second pose loss output by the second branch network, including: using the fourth convolutional layer to perform feature transformation processing on the third transformation feature, and performing a local maximum operation on the transformed feature to obtain the second joint map; and calculating the second pose loss based on the second joint map and the label corresponding to the pose sample.

[0067] Specifically, the convolutional neural network with the fourth convolutional layer performs a convolutional transformation on the aforementioned third transformation feature. Through the convolutional transformation, more useful information can be extracted from the third transformation feature, further enhancing its representational power. Then, a local maximum operation is performed on the first transformation feature obtained by the convolutional transformation. This operation can enhance the edge information of the feature map or other data by taking the local maximum value in a small neighborhood of the input data. That is, by finding the maximum value in the neighborhood of the input and setting other pixel values ​​to zero, the edges and obvious features are highlighted, thus determining which feature point is the best.

[0068] Furthermore, after the aforementioned local maximum operation, some feature points representing the positions of key points can be obtained. Each of these key points has an associated line indicating the relationship between the parts. That is, the second joint map can be calculated based on these feature points. Then, the loss value is calculated based on the second joint map and the real labels corresponding to the pose samples to understand the gap between the model's prediction and the actual target, so as to understand the model's performance, guide model training and optimization, and ultimately improve the model's prediction accuracy and generalization ability through the loss value.

[0069] In this embodiment, the third transformation features are processed by the fourth convolutional layer, and a local maximum operation is performed on the transformed features to obtain the second joint map. Based on the second joint map and the labels corresponding to the pose samples, the second pose loss is calculated. The obtained second pose loss can improve the model's understanding of the occluded parts in the pose samples. The trained model can accurately estimate human pose under occlusion, thus improving the model's adaptability to complex environments.

[0070] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0071] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the process of the embodiments of this application.

[0072] Figure 2 This is a schematic diagram of a training device for a pose recognition model provided in an embodiment of this application. Figure 2 As shown, the device includes:

[0073] The sample module 201 is configured to acquire a sample set for model training, the sample set containing multiple pose samples;

[0074] The input module 202 is configured to take pose samples as input to a pose recognition model to be trained, wherein the pose recognition model includes an enhancement network, a feature extraction network, a first prediction network, and a second prediction network.

[0075] The feature processing module 203 is configured to perform data augmentation processing on the pose sample using an augmentation network to obtain an augmented sample, and input the pose sample and the augmented sample into the feature extraction network so that the feature extraction network outputs the target feature.

[0076] The first prediction module 204 is configured to use the first prediction network to predict the target features and calculate the loss, so as to obtain the first pose loss;

[0077] The second prediction module 205 is configured to use the second prediction network to predict and calculate the target features, and obtain the second pose loss and feature consistency loss.

[0078] Training module 206 is configured to back-update the model's parameters based on the first pose loss, the second pose loss, and the feature consistency loss in order to train the pose recognition model.

[0079] In some embodiments, the first prediction module 204 is further configured to upsample the target features using an upsampling layer to obtain upsampled features; transform the upsampled features using a first convolutional layer to obtain first transformed features; perform a local maximum operation on the first transformed features to obtain a first joint map; and calculate the first pose loss based on the first joint map and the label corresponding to the pose sample.

[0080] In some embodiments, the second prediction module 205 is further configured to transform the target features using a second convolutional layer to obtain second transformed features; calculate the second transformed features using a function layer, and obtain attention-enhanced features based on the calculated features and the first transformed features; input the attention-enhanced features into the feature reconstruction module to obtain the second pose loss and feature consistency loss output by the feature reconstruction module.

[0081] In some embodiments, the second prediction module 205 is further configured to calculate the weights of the second transformation features using the sigmoid function of the function layer to obtain weighted features; and to multiply the weighted features and the first transformation features to obtain attention-enhanced features.

[0082] In some embodiments, the second prediction module 205 is further configured to transform the attention enhancement features using a third convolutional layer to obtain a third transformed feature; input the third transformed feature into a first branch network to obtain the feature consistency loss output by the first branch network; and input the third transformed feature into a second branch network to obtain the second pose loss output by the second branch network.

[0083] In some embodiments, the second prediction module 205 is further configured to perform masking processing on the third transformation features to obtain masked features; and to calculate the feature consistency loss using the masked features and attention enhancement features.

[0084] In some embodiments, the second prediction module 205 is further configured to perform feature transformation processing on the third transformation features using the fourth convolutional layer, and perform local maximum operation on the transformed features to obtain a second joint map; and calculate the second pose loss based on the second joint map and the label corresponding to the pose sample.

[0085] The apparatus provided in this application embodiment can implement all the method steps of the above method embodiments and achieve the same technical effect, which will not be repeated here.

[0086] Figure 3 This is a schematic diagram of the electronic device 3 provided in an embodiment of this application. Figure 3As shown, the electronic device 3 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the various device embodiments described above.

[0087] Electronic device 3 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 3 may include, but is not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or different components.

[0088] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0089] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or RAM of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 3. The memory 302 can also include both internal and external storage units of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0090] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0091] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The readable storage medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0092] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A training method for a pose recognition model, characterized in that, include: Obtain a sample set for model training, wherein the sample set contains multiple pose samples; The pose sample is used as input to the pose recognition model to be trained, wherein the pose recognition model includes an enhancement network, a feature extraction network, a first prediction network, and a second prediction network. The pose sample is augmented using the augmentation network to obtain an augmented sample. The pose sample and the augmented sample are then input into the feature extraction network so that the feature extraction network outputs the target feature. The first joint map is obtained by sequentially upsampling, feature transformation and local maximum operation on the target features using the first prediction network, and the first pose loss is calculated based on the first joint map and the label corresponding to the pose sample. The target features are sequentially subjected to attention enhancement, feature reconstruction, and loss calculation using the second prediction network to obtain a second pose loss and a feature consistency loss. The feature consistency loss is the consistency loss between the masked feature and the attention-enhanced feature, where the masked feature is obtained by masking the third transform feature obtained through convolutional transformation of the attention-enhanced feature. The second pose loss is calculated based on the second joint graph and the labels corresponding to the pose samples. The second joint graph is obtained by sequentially performing feature transformation and local maximum operations on the third transform feature. The parameters of the model are updated in reverse based on the first pose loss, the second pose loss, and the feature consistency loss, so as to train the pose recognition model.

2. The method according to claim 1, characterized in that, The first prediction network includes an upsampling layer and a first convolutional layer. The first joint map is obtained by sequentially upsampling, feature transformation, and local maximum operations on the target features using the first prediction network, and the first pose loss is calculated based on the first joint map and the labels corresponding to the pose samples. The target features are upsampled using the upsampling layer to obtain upsampled features; The upsampled features are transformed using the first convolutional layer to obtain the first transformed features; Perform a local maxima operation on the first transformation feature to obtain the first joint diagram; The first pose loss is calculated based on the first joint diagram and the labels corresponding to the pose samples.

3. The method according to claim 2, characterized in that, The second prediction network includes a second convolutional layer, a function layer, and a feature reconstruction module. The feature reconstruction module includes a third convolutional layer, a first branch network, and a second branch network. The second prediction network is used to sequentially perform attention enhancement processing, feature reconstruction, and loss calculation on the target features to obtain a second pose loss and a feature consistency loss, including: The target features are transformed using the second convolutional layer to obtain the second transformed features; The second transformation feature is calculated using the function layer, and attention enhancement features are obtained based on the calculated features and the first transformation feature. The attention enhancement feature is input into the feature reconstruction module to transform the attention enhancement feature using the third convolutional layer to obtain the third transformation feature. The third transformation feature is input into the first branch network to obtain the feature consistency loss output by the first branch network. The third transformation feature is input into the second branch network to obtain the second pose loss output by the second branch network.

4. The method according to claim 3, characterized in that, The step of calculating the second transformation feature using the function layer, and obtaining attention-enhanced features based on the calculated features and the first transformation feature, includes: The weighted features are obtained by calculating the weights of the second transformation features using the S-shaped function of the function layer. The attention enhancement feature is obtained by multiplying the weight feature and the first transformation feature.

5. A training device for a pose recognition model, characterized in that, include: The sample module is configured to acquire a sample set for model training, the sample set containing multiple pose samples; The input module is configured to take the pose sample as input to a pose recognition model to be trained, wherein the pose recognition model includes an enhancement network, a feature extraction network, a first prediction network, and a second prediction network. The feature processing module is configured to perform data augmentation processing on the pose sample using the augmentation network to obtain an augmented sample, and input the pose sample and the augmented sample into the feature extraction network so that the feature extraction network outputs target features; The first prediction module is configured to use the first prediction network to sequentially perform upsampling, feature transformation and local maximum operation on the target features to obtain a first joint map, and calculate the first pose loss based on the first joint map and the label corresponding to the pose sample. The second prediction module is configured to sequentially perform attention enhancement processing, feature reconstruction, and loss calculation on the target features using the second prediction network to obtain a second pose loss and a feature consistency loss; wherein, the feature consistency loss is the consistency loss between the mask feature and the attention-enhanced feature, and the mask feature is obtained by masking the third transform feature obtained by convolutional transformation of the attention-enhanced feature; the second pose loss is calculated based on the second joint graph and the label corresponding to the pose sample, and the second joint graph is obtained by sequentially performing feature transformation processing and local maximum operation on the third transform feature; The training module is configured to back-update the model's parameters based on the first pose loss, the second pose loss, and the feature consistency loss, in order to train the pose recognition model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

7. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images