A human posture recognition method and device, electronic equipment and storage medium

Abstract

The application discloses a human posture recognition method and device, an electronic device and a storage medium. First, a target feature extraction model is obtained by training through block reorganization using each first human picture in an unmarked training set. Then, a human posture recognition model is constructed according to the target feature extraction model. Next, the human posture recognition model is trained using each second human picture in a marked training set and respective human key point marking information. Finally, the human posture recognition model is used to recognize the human posture in a human picture to be processed. The target feature extraction model is trained, and the human posture recognition model is constructed according to the target feature extraction model, thereby improving the feature extraction capability of the human posture recognition model and solving the problem of poor accuracy of human posture recognition due to confusion of similar parts.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a human posture recognition method, device, electronic device and storage medium. Background Technology

[0002] Human pose recognition, also known as human key point recognition, is a fundamental task in the field of computer vision and has important applications in tasks such as action recognition.

[0003] With the development of computer technology, it is now possible to detect key points of the human body in images containing human figures. Human key point recognition requires locating key body parts such as the head, neck, shoulders, and hands in a given image. Furthermore, with the development of people's needs, the current task of human key point detection not only needs to obtain high-level semantic information containing human texture information, but also needs to collect the correlation between the detected target and the surrounding environment, so as to achieve better recognition results.

[0004] Existing technologies generally rely on deep neural network models for human key point detection. However, existing deep neural network models have poor feature extraction capabilities and often suffer from confusion of similar parts, such as left and right positions, which leads to poor accuracy in human pose recognition. Summary of the Invention

[0005] This application provides a human posture recognition method, device, electronic device, and storage medium to solve the problem that the existing deep neural network models have poor feature extraction capabilities, resulting in poor accuracy of human posture recognition.

[0006] This application provides a human posture recognition method, the method comprising:

[0007] The unlabeled training set is divided into blocks according to a preset number of blocks to obtain each block of the first human image; for each first human image, the blocks of the first human image are recombined to obtain the recombined image corresponding to the first human image; a target feature extraction model is trained based on each first human image and its corresponding recombined image.

[0008] A human pose recognition model is constructed based on the target feature extraction model, and the human pose recognition model is trained based on each second human image in the labeled training set and their corresponding human key point annotation information.

[0009] Obtain the human image to be processed, and identify the human posture in the human image based on the trained human posture recognition model.

[0010] Further, the step of dividing each first human image in the unlabeled training set into blocks according to a preset number of blocks to obtain each block of the first human image; for each first human image, recombining each block of the first human image to obtain a recombined image corresponding to the first human image; training the target feature extraction model based on each first human image and its corresponding recombined image includes:

[0011] The unlabeled training set is divided into blocks according to a preset initial number of blocks, resulting in blocks for each first human image. The blocks of each first human image are then recombined to obtain an initial recombined image corresponding to the first human image. An initial feature extraction model is trained based on the first human images and their corresponding initial recombined images.

[0012] The initial number of blocks is increased by a preset block number step size to obtain the number of candidate blocks; each first human image is divided according to the number of candidate blocks and recombined to obtain the candidate recombined image corresponding to each first human image; the initial feature extraction model is retrained based on each first human image and its corresponding candidate recombined image to obtain the candidate feature extraction model.

[0013] When the number of candidate blocks reaches the preset target number of blocks, the candidate feature extraction model is retrained to obtain the target feature extraction model.

[0014] Furthermore, the step of recombining the various blocks of the first human body image to obtain the recombined image corresponding to the first human body image includes:

[0015] The misclassification probability distribution of each block is determined based on the existing reference human pose recognition model.

[0016] Based on the misclassification probability distribution of each block, the recombination probability of each block is determined;

[0017] Based on the recombination probability of each block, the blocks of the first human image are recombined to obtain the recombined image corresponding to the first human image.

[0018] Further, determining the recombination probability of each block based on the misclassification probability distribution of each block includes:

[0019] The recombination probability of each block is determined based on the misclassification probability distribution and the number of blocks.

[0020] Furthermore, based on the misclassification probability distribution of each block, the recombination probability of each block is determined, including:

[0021] Based on the position information of each block in the first human body image, the foreground and background probability distribution of each block is determined;

[0022] The recombination probability of each block is determined based on the misclassification probability distribution and the foreground / background probability distribution of each block.

[0023] Further, determining the foreground-background probability distribution of each block based on its position information in the first human image includes:

[0024] Based on the position information of each block in the first human body image, determine each foreground block and each background block in each block;

[0025] The recombination weights of each foreground block and each background block are determined respectively; wherein the recombination weight of each foreground block is greater than the recombination weight of each background block;

[0026] Based on the recombination weights of each foreground block and each background block, the foreground and background probability distributions of each block are determined.

[0027] Further, determining the recombination probability of each block based on the misclassification probability distribution and the foreground / background probability distribution of each block includes:

[0028] The recombination probability of each block is determined based on the misclassification probability distribution, the foreground / background probability distribution, the first weighting coefficient corresponding to the misclassification probability distribution, and the second weighting coefficient corresponding to the foreground / background probability distribution.

[0029] On the other hand, this application provides a human posture recognition device, the device comprising:

[0030] The first training module is used to divide each first human image in the unlabeled training set into blocks according to a preset number of blocks, so as to obtain each block of each first human image; to reassemble each block of the first human image to obtain a reassembled image corresponding to the first human image; and to train a target feature extraction model based on each first human image and its corresponding reassembled image.

[0031] The second training module is used to construct a human pose recognition model based on the target feature extraction model, and to train the human pose recognition model based on each second human image in the labeled training set and its corresponding human key point annotation information.

[0032] The human pose recognition module is used to acquire human images to be processed and to identify human poses in the human images to be processed based on the trained human pose recognition model.

[0033] Further, the first training module is specifically used to divide each first human body image in the unlabeled training set into blocks according to a preset initial block number, obtaining each block of each first human body image; for each first human body image, reassemble each block of the first human body image to obtain an initial reassembled image corresponding to the first human body image; train an initial feature extraction model based on each first human body image and its corresponding initial reassembled image; increase the initial block number by a preset block number step size to obtain a candidate block number; divide each first human body image according to the candidate block number and reassemble it to obtain a candidate reassembled image corresponding to each first human body image; retrain the initial feature extraction model based on each first human body image and its corresponding candidate reassembled image to obtain a candidate feature extraction model; when the candidate block number reaches a preset target block number, retrain the candidate feature extraction model to obtain a target feature extraction model.

[0034] Furthermore, the first training module is specifically used to determine the misclassification probability distribution of each block based on an existing reference human pose recognition model; determine the recombination probability of each block based on the misclassification probability distribution of each block; and recombine each block of the first human image based on the recombination probability of each block to obtain a recombined image corresponding to the first human image.

[0035] Furthermore, the first training module is specifically used to determine the recombination probability of each block based on the misclassification probability distribution and the number of blocks.

[0036] Furthermore, the first training module is specifically used to determine the foreground-background probability distribution of each block based on the position information of each block in the first human image; and to determine the recombination probability of each block based on the misclassification probability distribution and the foreground-background probability distribution of each block.

[0037] Further, the first training module is specifically used to determine each foreground block and each background block in each block according to the position information of each block in the first human image; determine the recombination weight of each foreground block and each background block respectively; wherein the recombination weight of each foreground block is greater than the recombination weight of each background block; and determine the foreground-background probability distribution of each block according to the recombination weight of each foreground block and each background block.

[0038] Furthermore, the first training module is specifically used to determine the recombination probability of each block based on the misclassification probability distribution, the foreground-background probability distribution, the first weight coefficient corresponding to the misclassification probability distribution, and the second weight coefficient corresponding to the foreground-background probability distribution.

[0039] In another aspect, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0040] Memory, used to store computer programs;

[0041] When a processor executes a program stored in memory, it implements the steps of the method described in any of the preceding statements.

[0042] In another aspect, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any of the above claims.

[0043] This application provides a human posture recognition method, apparatus, electronic device, and storage medium. The method includes: dividing each first human image in an unlabeled training set into blocks according to a preset number of blocks, obtaining each block of the first human image; recombining the blocks of the first human images to obtain recombined images corresponding to the first human images; training a target feature extraction model based on the first human images and their corresponding recombined images; constructing a human posture recognition model based on the target feature extraction model, and training the human posture recognition model based on each second human image in a labeled training set and their corresponding human key point annotation information; acquiring a human image to be processed, and recognizing the human posture in the human image to be processed based on the trained human posture recognition model.

[0044] The above technical solution has the following advantages or beneficial effects:

[0045] In this application, a target feature extraction model is first obtained by training various first human images from an unlabeled training set through block recombination. Then, a human pose recognition model is constructed based on the target feature extraction model. Next, the human pose recognition model is trained using various second human images from a labeled training set and their corresponding key point annotations. Finally, the human pose recognition model is used to identify the human pose in the image to be processed. On the one hand, by training the target feature extraction model and then constructing the human pose recognition model based on it, the feature extraction capability of the human pose recognition model is improved, solving the problem of confusion between similar parts and poor accuracy in human pose recognition. On the other hand, training the target feature extraction model based on an unlabeled training set fully utilizes massive amounts of unlabeled data to enhance the model's ability to represent the location of target parts, reducing the amount of labeled data required and saving manpower and resources. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A schematic diagram of the human posture recognition process provided in this application;

[0048] Figure 2 A detailed flowchart of human pose recognition provided for this application;

[0049] Figure 3 This application provides a schematic diagram of the image reconstruction process.

[0050] Figure 4 A schematic diagram of the human posture recognition device provided in this application;

[0051] Figure 5 A schematic diagram of the electronic device structure provided in this application. Detailed Implementation

[0052] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0053] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0054] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0055] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0056] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0058] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0059] Figure 1 The diagram provided in this application illustrates the human pose recognition process, which includes the following steps:

[0060] S101: Divide each first human image in the unlabeled training set into blocks according to a preset number of blocks to obtain each block of the first human image; reassemble each block of the first human image to obtain a reassembled image corresponding to the first human image; train a target feature extraction model based on the first human images and their corresponding reassembled images.

[0061] S102: Construct a human pose recognition model based on the target feature extraction model, and train the human pose recognition model based on each second human image in the labeled training set and its corresponding human key point annotation information.

[0062] S103: Obtain the human body image to be processed, and identify the human body posture in the human body image to be processed based on the trained human body posture recognition model.

[0063] The human posture recognition method provided in this application is applied to electronic devices, which may be PCs, tablets, or servers.

[0064] The electronic device stores an unlabeled training set and a preset number of blocks, such as 4, 6, 8, or 9. For each first human image in the unlabeled training set, the first human image is divided according to the preset number of blocks. It should be noted that the first human image can be divided into the preset number of blocks according to any rule; preferably, it can be divided evenly into the preset number of blocks, or empirically based on the number of human body parts or the number of key points in the final task. The first human image is divided according to the preset number of blocks to obtain individual blocks. Then, the blocks of the first human image are recombined to obtain the recombined image corresponding to the first human image. It should be noted that the recombination probability of each block is determined to be random, and the blocks are recombined according to the random probabilities of each block to obtain the recombined image corresponding to the first human image.

[0065] A target feature extraction model is trained based on each first human image and its corresponding reconstructed image. The task of the target feature extraction model is to restore the corresponding first human image from the reconstructed image. After multiple iterations of training, the target feature extraction model is considered complete when the convergence condition is met.

[0066] After the target feature extraction model is trained, a human pose recognition model is constructed based on it. The human pose recognition model includes a feature extraction unit and a human pose recognition unit. The feature extraction unit uses the trained target feature extraction model. An electronic device stores a labeled training set, which includes various second human images and their corresponding keypoint annotations. The human pose recognition unit in the human pose recognition model is trained based on the second human images and their corresponding keypoint annotations in the labeled training set. The human pose recognition model is considered complete when the convergence condition is met.

[0067] When an electronic device acquires a human image to be processed, it inputs the human image to be processed into the trained human pose recognition model, and identifies the human pose in the human image to be processed based on the trained human pose recognition model.

[0068] In this application, a target feature extraction model is first obtained by training various first human images from an unlabeled training set through block recombination. Then, a human pose recognition model is constructed based on the target feature extraction model. Next, the human pose recognition model is trained using various second human images from a labeled training set and their corresponding key point annotations. Finally, the human pose recognition model is used to identify the human pose in the image to be processed. On the one hand, by training the target feature extraction model and then constructing the human pose recognition model based on it, the feature extraction capability of the human pose recognition model is improved, solving the problem of confusion between similar parts and poor accuracy in human pose recognition. On the other hand, training the target feature extraction model based on an unlabeled training set fully utilizes massive amounts of unlabeled data to enhance the model's ability to represent the location of target parts, reducing the amount of labeled data required and saving manpower and resources.

[0069] To reduce the training difficulty of the target feature extraction model and improve its convergence efficiency, this application involves dividing the unlabeled training set into blocks according to a preset number of blocks, resulting in blocks for each first human image; recombining the blocks of each first human image to obtain recombined images; and training the target feature extraction model based on the first human images and their corresponding recombined images.

[0070] The unlabeled training set is divided into blocks according to a preset initial number of blocks, resulting in blocks for each first human image. The blocks of each first human image are then recombined to obtain an initial recombined image corresponding to the first human image. An initial feature extraction model is trained based on the first human images and their corresponding initial recombined images.

[0071] The initial number of blocks is increased by a preset block number step size to obtain the number of candidate blocks; each first human image is divided according to the number of candidate blocks and recombined to obtain the candidate recombined image corresponding to each first human image; the initial feature extraction model is retrained based on each first human image and its corresponding candidate recombined image to obtain the candidate feature extraction model.

[0072] When the number of candidate blocks reaches the preset target number of blocks, the candidate feature extraction model is retrained to obtain the target feature extraction model.

[0073] The preset initial number of blocks is a small number, such as 2 or 3. The unlabeled human images in the training set are divided according to the preset initial number of blocks, resulting in individual blocks for each human image. For each human image, the blocks are recombined to obtain the corresponding initial recombined image. An initial feature extraction model is then trained based on each human image and its corresponding initial recombined image.

[0074] Then, the initial number of blocks is increased by a preset block size step to obtain the candidate block number. The preset block size step is a fixed value, such as 2 or 3. For example, if the initial number of blocks is 2 and the preset block size step is 3, then an initial feature extraction model is trained based on the initial number of blocks of 2. Then, each first human image is divided into 2 + 3 = 5 blocks, and the initial feature extraction model is trained again to obtain the candidate feature extraction model. Then, each first human image is divided into 5 + 3 = 8 blocks, and a new candidate feature extraction model is trained again to obtain a new candidate feature extraction model. If the preset target number of blocks is 8, then the new candidate feature extraction model is the target feature extraction model; otherwise, the number of blocks continues to increase.

[0075] Alternatively, the preset block size step can vary depending on the number of increments. For example, the preset block size step could be 2 for the first increment, 3 for the second increment, and so on. For instance, if the initial block size is 2, an initial feature extraction model is trained based on this initial block size. Then, each first human image is divided into 2+2=4 blocks, and the initial feature extraction model is trained again to obtain a candidate feature extraction model. Then, each first human image is divided into 4+3=7 blocks, and a new candidate feature extraction model is trained again. If the preset target block size is 7, then the new candidate feature extraction model is the target feature extraction model; otherwise, the number of blocks continues to increase.

[0076] In this application, a training method that gradually increases the number of blocks is adopted, which reduces the training difficulty of the target feature extraction model and improves the training efficiency of the target feature extraction model.

[0077] To improve the feature extraction capability of the target feature extraction model, in this application, the step of recombining the various blocks of the first human image to obtain the recombined image corresponding to the first human image includes:

[0078] The misclassification probability distribution of each block is determined based on the existing reference human pose recognition model.

[0079] Based on the misclassification probability distribution of each block, the recombination probability of each block is determined;

[0080] Based on the recombination probability of each block, the blocks of the first human image are recombined to obtain the recombined image corresponding to the first human image.

[0081] In this application, the misclassification probability distribution of each block is determined based on an existing reference human pose recognition model. For example, the probability of block i being misclassified to block j is 20%, the probability of block i being misclassified to block m is 10%, the probability of block i being misclassified to block n is 5%, the probability of block a being misclassified to block i is 12%, and so on. The misclassification probability distribution is determined based on the existing reference human pose recognition model and the localization errors in labeled human images. Specifically, it involves statistically analyzing the probability that key points in block i are incorrectly located in block j.

[0082] Based on the misclassification probability distribution of each block, the recombination probability of each block is determined. Specifically, the misclassification probability distribution of each block can be directly used as the recombination probability of each block. Then, based on the recombination probability of each block, the blocks of the first human image are recombined to obtain the recombined image corresponding to the first human image.

[0083] To more accurately determine the recombination probability of each block, the step of determining the recombination probability of each block based on the misclassification probability distribution of each block includes:

[0084] The recombination probability of each block is determined based on the misclassification probability distribution and the number of blocks.

[0085] The misclassification probability distribution of each block is Perr(i,j), where Perr(i,j) represents the probability that block i is misclassified into block j, and the number of blocks is N. Therefore, the recombination probability of each block is P(i,j) = 1 / N + Perr(i,j).

[0086] In addition, the weight coefficient λ1 is assigned to the number of blocks, and the weight coefficient λ2 is assigned to the misclassification probability distribution. Then the recombination probability of each block is P(i,j)=λ1*(1 / N)+λ2*Perr(i,j).

[0087] To further improve the accuracy of determining the recombination probability of each block, the step of determining the recombination probability of each block based on the misclassification probability distribution of each block includes:

[0088] Based on the position information of each block in the first human body image, the foreground and background probability distribution of each block is determined;

[0089] The recombination probability of each block is determined based on the misclassification probability distribution and the foreground / background probability distribution of each block.

[0090] Specifically, determining the foreground-background probability distribution of each segment based on its position information in the first human image includes:

[0091] Based on the position information of each block in the first human body image, determine each foreground block and each background block in each block;

[0092] The recombination weights of each foreground block and each background block are determined respectively; wherein the recombination weight of each foreground block is greater than the recombination weight of each background block;

[0093] Based on the recombination weights of each foreground block and each background block, the foreground and background probability distributions of each block are determined.

[0094] Electronic devices can pre-determine each foreground block and each background block within each block based on the position information of each block in the first human body image. For example, the blocks in the upper left and upper right corners are background blocks, and the remaining blocks are foreground blocks.

[0095] Determine the recombination weights for each foreground and background block, where the recombination weight of the foreground block is greater than that of the background block. For example, the recombination weight of the foreground block may be 60% and the recombination weight of the background block may be 40%; or the recombination weight of the foreground block may be 70% and the recombination weight of the background block may be 30%, etc. It is necessary to ensure that the sum of the recombination weights of the foreground and background blocks is 1, and that the recombination weight of the foreground block is greater than that of the background block.

[0096] Based on the positional relationship information between human key points and each block in human images from the labeled training set, the foreground and background probability distribution of each block is determined.

[0097] Based on the recombination weights of each foreground and background block, the foreground / background probability distribution of each block is determined. For example, if both block i and block j are foreground blocks, then the foreground / background probability of block i recombining into block j is the recombination weight of the foreground block. Similarly, if both block i and block j are background blocks, then the foreground / background probability of block i recombining into block j is the recombination weight of the background block. Furthermore, if one of block i and block j is a foreground block and the other is a background block, then the foreground / background probability of block i recombining into block j can be the average of the recombination weights of the foreground and background blocks.

[0098] The recombination probability of each block is determined based on the misclassification probability distribution and the foreground / background probability distribution of each block. Specifically, the recombination probability of each block is determined based on the misclassification probability distribution, the foreground / background probability distribution, the first weighting coefficient corresponding to the misclassification probability distribution, and the second weighting coefficient corresponding to the foreground / background probability distribution.

[0099] The misclassification probability distribution of each block is Perr(i,j), the foreground and background probability distribution of each block is Pa(i,j), the first weight coefficient corresponding to the misclassification probability distribution is λ3, and the second weight coefficient corresponding to the foreground and background probability distribution is λ4. Then the recombination probability of each block is P(i,j)=λ3*Perr(i,j)+λ4*Pa(i,j).

[0100] The human pose recognition method provided in this application first undergoes self-supervised learning training based on the task of restoring the location of body parts. This fully utilizes massive amounts of unlabeled data to improve the model's ability to represent the location of target body parts, reducing the amount of labeled data required and saving manpower and resources. The self-supervised training method proposed in this application is specifically tailored to the characteristics of human keypoint detection tasks. It proposes a training data generation method that divides the human body into blocks based on body parts and reconstructs these blocks based on the misclassification probability of existing models. This allows the model to more fully learn the relative positional relationships between different body parts, while increasing the reconstruction ratio of error-prone blocks, improving the model's feature discrimination ability for error-prone parts, and ultimately enhancing the performance of the human keypoint detection model.

[0101] Figure 2 The detailed flowchart of human pose recognition provided for this application includes the following steps:

[0102] S201: Divide each first human image in the unlabeled training set into blocks according to the preset initial number of blocks, and obtain each block of each first human image; reassemble each block of the first human image to obtain the initial reassembled image corresponding to the first human image; train the initial feature extraction model based on each first human image and its corresponding initial reassembled image.

[0103] S202: Increase the initial number of blocks by a preset block number step size to obtain the number of candidate blocks; divide each first human body image according to the number of candidate blocks and reassemble them to obtain the candidate reassembled images corresponding to each first human body image; retrain the initial feature extraction model based on each first human body image and its corresponding candidate reassembled images to obtain the candidate feature extraction model.

[0104] S203: When the number of candidate blocks reaches the preset target number of blocks, the candidate feature extraction model is retrained to obtain the target feature extraction model.

[0105] S204: Construct a human pose recognition model based on the target feature extraction model, and train the human pose recognition model based on each second human image in the labeled training set and its corresponding human key point annotation information.

[0106] S205: Obtain the human image to be processed, and identify the human posture in the human image to be processed based on the trained human posture recognition model.

[0107] Figure 3 The image reconstruction process diagram provided for this application includes the following steps:

[0108] S301: Determine the misclassification probability distribution of each block based on the existing reference human pose recognition model.

[0109] S302: Based on the position information of each block in the first human body image, determine each foreground block and each background block in each block.

[0110] S303: Determine the recombination weights of each foreground block and each background block respectively; wherein the recombination weight of each foreground block is greater than the recombination weight of each background block.

[0111] S304: Determine the foreground and background probability distributions of each foreground block based on the recombination weights of each foreground block and each background block.

[0112] S305: Determine the recombination probability of each block based on the misclassification probability distribution, the foreground / background probability distribution, the first weighting coefficient corresponding to the misclassification probability distribution, and the second weighting coefficient corresponding to the foreground / background probability distribution.

[0113] S306: Based on the recombination probability of each block, recombine each block of the first human image to obtain the recombined image corresponding to the first human image.

[0114] Figure 4 The schematic diagram of the human posture recognition device provided in this application includes:

[0115] The first training module 41 is used to divide each first human body image in the unlabeled training set into blocks according to a preset number of blocks, so as to obtain each block of each first human body image; to reassemble each block of the first human body image to obtain a reassembled image corresponding to the first human body image; and to train a target feature extraction model based on each first human body image and its corresponding reassembled image.

[0116] The second training module 42 is used to construct a human pose recognition model based on the target feature extraction model, and to train the human pose recognition model based on each second human image in the labeled training set and their corresponding human key point annotation information.

[0117] The human pose recognition module 43 is used to acquire a human image to be processed and to recognize the human pose in the human image to be processed based on the trained human pose recognition model.

[0118] The first training module 41 is specifically used to divide each first human body image in the unlabeled training set into blocks according to a preset initial block number, obtaining each block of each first human body image; for each first human body image, reassemble each block of the first human body image to obtain an initial reassembled image corresponding to the first human body image; train an initial feature extraction model based on each first human body image and its corresponding initial reassembled image; increase the initial block number by a preset block number step size to obtain a candidate block number; divide each first human body image according to the candidate block number and reassemble it to obtain a candidate reassembled image corresponding to each first human body image; retrain the initial feature extraction model based on each first human body image and its corresponding candidate reassembled image to obtain a candidate feature extraction model; when the candidate block number reaches a preset target block number, retrain the candidate feature extraction model to obtain a target feature extraction model.

[0119] The first training module 41 is specifically used to determine the misclassification probability distribution of each block based on an existing reference human pose recognition model; determine the recombination probability of each block based on the misclassification probability distribution of each block; and recombine each block of the first human image based on the recombination probability of each block to obtain a recombined image corresponding to the first human image.

[0120] The first training module 41 is specifically used to determine the recombination probability of each block based on the misclassification probability distribution and the number of blocks.

[0121] The first training module 41 is specifically used to determine the foreground and background probability distribution of each block based on the position information of each block in the first human image; and to determine the recombination probability of each block based on the misclassification probability distribution and the foreground and background probability distribution of each block.

[0122] The first training module 41 is specifically used to determine each foreground block and each background block in each block according to the position information of each block in the first human image; to determine the recombination weight of each foreground block and each background block respectively; wherein the recombination weight of each foreground block is greater than the recombination weight of each background block; and to determine the foreground and background probability distribution of each block according to the recombination weight of each foreground block and each background block.

[0123] The first training module 41 is specifically used to determine the recombination probability of each block based on the misclassification probability distribution, the foreground-background probability distribution, the first weight coefficient corresponding to the misclassification probability distribution, and the second weight coefficient corresponding to the foreground-background probability distribution.

[0124] This application also provides an electronic device, such as Figure 5 As shown, it includes: processor 501, communication interface 502, memory 503 and communication bus 504, wherein processor 501, communication interface 502 and memory 503 communicate with each other through communication bus 504.

[0125] The memory 503 stores a computer program, which, when executed by the processor 501, causes the processor 501 to perform any of the above method steps.

[0126] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0127] Communication interface 502 is used for communication between the above-mentioned electronic device and other devices.

[0128] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0129] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0130] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform any of the above method steps.

[0131] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0132] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for human posture recognition, characterized in that, The method includes: The unlabeled training set is divided into blocks according to a preset number of blocks, resulting in individual blocks for each first human image. For each first human image, the misclassification probability distribution of each block is determined based on an existing reference human pose recognition model. Based on the misclassification probability distribution, the recombination probability of each block is determined. Based on the recombination probability, the blocks of the first human image are recombined to obtain a recombined image corresponding to the first human image. A target feature extraction model is trained based on each first human image and its corresponding recombined image. A human pose recognition model is constructed based on the target feature extraction model, and the human pose recognition model is trained based on each second human image in the labeled training set and their corresponding human key point annotation information. Obtain a human image to be processed, and identify the human pose in the human image based on the trained human pose recognition model.

2. The method as described in claim 1, characterized in that, The unlabeled training set is divided into blocks according to a preset number of blocks to obtain the blocks of each first human body image. For each of the first human body images, the blocks of the first human body images are recombined to obtain the recombined images corresponding to the first human body images; The target feature extraction model trained based on the first human body images and their corresponding reconstructed images includes: The unlabeled training set is divided into blocks according to a preset initial number of blocks, resulting in blocks for each first human image. The blocks of each first human image are then recombined to obtain an initial recombined image corresponding to the first human image. An initial feature extraction model is trained based on the first human images and their corresponding initial recombined images. The initial number of blocks is increased by a preset block number step size to obtain the number of candidate blocks; each first human image is divided according to the number of candidate blocks and recombined to obtain the candidate recombined image corresponding to each first human image; the initial feature extraction model is retrained based on each first human image and its corresponding candidate recombined image to obtain the candidate feature extraction model. When the number of candidate blocks reaches the preset target number of blocks, the candidate feature extraction model is retrained to obtain the target feature extraction model.

3. The method as described in claim 1, characterized in that, The step of determining the recombination probability of each block based on the misclassification probability distribution of each block includes: The recombination probability of each block is determined based on the misclassification probability distribution and the number of blocks.

4. The method as described in claim 1, characterized in that, The step of determining the recombination probability of each block based on the misclassification probability distribution of each block includes: Based on the position information of each block in the first human body image, the foreground and background probability distribution of each block is determined; The recombination probability of each block is determined based on the misclassification probability distribution and the foreground / background probability distribution of each block.

5. The method as described in claim 4, characterized in that, The step of determining the foreground-background probability distribution of each segment based on its position information in the first human image includes: Based on the position information of each block in the first human body image, determine each foreground block and each background block in each block; The recombination weights of each foreground block and each background block are determined respectively; wherein the recombination weight of each foreground block is greater than the recombination weight of each background block; Based on the recombination weights of each foreground block and each background block, the foreground and background probability distributions of each block are determined.

6. The method as described in claim 4, characterized in that, The step of determining the recombination probability of each block based on the misclassification probability distribution and the foreground / background probability distribution of each block includes: The recombination probability of each block is determined based on the misclassification probability distribution, the foreground and background probability distribution, the first weighting coefficient corresponding to the misclassification probability distribution, and the second weighting coefficient corresponding to the foreground and background probability distribution.

7. A human posture recognition device, characterized in that, The device includes: The first training module is used to divide each first human image in the unlabeled training set into blocks according to a preset number of blocks, obtaining each block of each first human image; for each first human image, the misclassification probability distribution of each block is determined according to an existing reference human pose recognition model; the recombination probability of each block is determined according to the misclassification probability distribution of each block; the blocks of the first human image are recombined according to the recombination probability of each block to obtain a recombined image corresponding to the first human image; and a target feature extraction model is trained based on each first human image and its corresponding recombined image. The second training module is used to construct a human pose recognition model based on the target feature extraction model, and to train the human pose recognition model based on each second human image in the labeled training set and their corresponding human key point annotation information. The human pose recognition module is used to acquire human images to be processed and to identify human poses in the human images to be processed based on the trained human pose recognition model.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images