Multiple-person pose estimation method, apparatus, device, medium, and product

Abstract

This invention provides a method, apparatus, device, medium, and product for estimating the posture of multiple individuals, which effectively captures information from the distal limbs, significantly simplifies the processing flow, and achieves competitive accuracy. [Solution] The method involves inputting acquired human body image data into a deep aggregation network, outputting general features of the human body image, obtaining candidate center positions and initial features of each body part based on the general features of the human body image, obtaining a first offset amount from the initial features based on an axial decoupling attention perceptron, inputting the first offset amount and initial features into a regressor by interpolation, outputting a second offset amount, obtaining a keypoint offset amount by adding the first and second offset amounts, and obtaining the result of multi-person pose estimation by adding the candidate center position and the keypoint offset amount. The axial decoupling attention perceptron effectively captures contextual information and distal limb information of the human body, significantly simplifying the processing flow.

Description

[Technical Field]

[0001] This application relates to the field of human body imaging, and more particularly to methods, apparatus, devices, media, and products for estimating the poses of multiple people. [Background technology]

[0002] Existing multi-person pose estimation techniques have significant challenges in recognizing the extremities of distant limbs (e.g., hands and feet). Most two-stage multi-person pose estimation methods suffer from inefficiency, primarily because they employ a decoupling method for representing key body points, resulting in a complex inference process. AdaptivePose can simplify the processing flow to some extent and improve accuracy by representing body parts as semantic proxy points. However, due to the limited receptive field of convolutional features, the model struggles to effectively capture information about distal limbs.

[0003] In other words, with conventional technology, the receptive field is limited when processing distal limbs, so proxy points are often located in semantically weak regions, which often affects recognition accuracy and efficiency. Furthermore, conventional technology also suffered from shortcomings in balance accuracy and inference speed. [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] The object of this application is to provide a method, apparatus, device, medium, and product for estimating the posture of multiple people that can effectively capture information about distal limbs, focus on areas with strong semantics, and improve the accuracy and computational efficiency of human body recognition during the process of estimating the posture of multiple people. [Means for solving the problem]

[0005] To achieve the above objective, the present invention provides the following embodiments.

[0006] In the first embodiment, the present application is, The process involves inputting acquired human body image data into a deep aggregation network and outputting general features of the human body image, The steps include obtaining a center point heatmap based on general features of human body images, and extracting candidate center positions from the center point heatmap. The steps include: extracting regression features based on general features of human body images, and obtaining initial features for each body part through channel grouping of the regression features; The first offset amount is obtained by decomposing the initial features along the horizontal and vertical directions based on an axial decoupling attention perceptron, calculating, merging, and predicting independent self-attention. The first offset amount and initial features are input to the regressor through interpolation, and the second offset amount is output. The first step is to add the second offset amount to obtain the key point offset amount, The present invention provides a method for estimating the poses of multiple people, which includes the step of obtaining the result of estimating the poses of multiple people by adding the aforementioned center position candidate and key point offset amount.

[0007] In the second embodiment, the present application is, A general feature output module that inputs the acquired human body image data into a deep aggregation network and outputs the general features of the human body image; a central position candidate extraction module that obtains a central point heat map based on the general features of the human body image and extracts central position candidates from the central point heat map; a channel grouping module that extracts regression features based on the general features of the human body image and obtains initial features of each body part through channel grouping; a first offset amount determination module that decomposes the initial features along the horizontal and vertical directions, calculates, fuses, and predicts independent self-attention based on an axial decoupling attention perceptron to obtain a first offset amount; a second offset amount determination module that inputs the first offset amount and the initial features into a regressor through an interpolation operation and outputs a second offset amount; a key point offset amount determination module that obtains a key point offset amount by adding the first offset amount and the second offset amount; and a pose estimation module that obtains the results of multi-person pose estimation by adding the central position candidate and the key point offset amount. A device for Theme 2 is provided, which includes these modules.

[0008] In a third aspect, the present application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it realizes the steps of the multi-person pose estimation method according to any one of the above items.

[0009] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it realizes the steps of the multi-person pose estimation method according to any one of the above items.

[0010] In a fifth aspect, the present application provides a computer program product including a computer program, where when the computer program is executed by a processor, it includes a computer program that realizes the steps of the method for estimating the postures of multiple persons according to any one of the above items.

Advantages of the Invention

[0011] According to the specific embodiments provided in the present application, the following technical effects are disclosed.

[0012] The method, device, equipment, medium and product for estimating the postures of multiple persons provided in the present application can effectively distinguish different individuals in a multiple-person scene by extracting a center point heat map based on the general features of a human body image, and can accurately extract center position candidates even in a case of severe occlusion. On the other hand, regression features are extracted based on the general features of a human body image, the regression features are channel-grouped, and a self-attention mechanism is executed in the horizontal and vertical directions based on ADAP (Axial Decoupled Attention Perceptron), and by obtaining the fused self-attention features of each body part, the context information of the human body can be effectively captured, and the overall receptive field of each part feature can be enhanced. By adding the center position candidate obtained based on the center point heat map and the key point offset amount predicted by the self-attention features, and combining local position detection and overall perception ability, the accuracy of posture estimation can be improved, and the result of estimating the postures of multiple persons applicable to a multiple-person scene at a long distance or a complex background can be obtained, the information of distal limbs can be effectively captured, the processing flow can be greatly simplified, and a competitive performance in terms of accuracy can be obtained.

Brief Description of the Drawings

[0013] To more clearly explain the embodiments of this application or the technical concepts in the prior art, the drawings necessary for the embodiments will be briefly described below. However, the drawings in the following description are only a few embodiments of this application, and it is clear that a person skilled in the art can obtain other drawings based on these drawings without expending any creative effort.

[0014] [Figure 1] This figure shows the application environment of the multiple person pose estimation method in one embodiment of the present invention. [Figure 2] This figure shows a flowchart of a method for estimating the posture of multiple people according to one embodiment of the present invention. [Figure 3] This figure shows a specific flowchart of the method for estimating the postures of multiple people according to one embodiment of the present invention. [Figure 4] This figure shows the flow of the regression branches by body part in a method for estimating the posture of multiple people according to one embodiment of the present invention. [Figure 5] This figure shows an axial decoupling perceptron according to one embodiment of the present invention. [Figure 6] This figure shows a functional module of a multi-person posture estimation device according to one embodiment of the present invention. [Figure 7] This diagram shows the configuration of a computer device according to one embodiment of the present invention. [Modes for carrying out the invention]

[0015] The technical concepts in the embodiments of this application will be described clearly and completely below with reference to the drawings of the embodiments of this application, although it is clear that the embodiments described are only a part of the embodiments of this application and not all of them. All other embodiments that can be obtained by a person skilled in the art without creative effort based on the embodiments of this application are within the scope of protection of this application.

[0016] To make the above-mentioned objectives, features, and advantages of this application easier to understand, the application will be described in more detail below with reference to drawings and specific embodiments.

[0017] The multiple-person pose estimation method according to the embodiment of the present invention can be applied to the application environment shown in Figure 1. Terminal 102 communicates with server 104 via a network. The data storage system can store data that server 104 needs to process. The data storage system may be installed independently, integrated with server 104, or located on the cloud or another server. Terminal 102 transmits acquired human body image data to server 104, and server 104 receives the human body image data. Server 104 inputs the acquired human body image data into DLA (Deep Layer Aggregation) and outputs general features of the human body image. Based on the general features of the human body image, a center point heatmap is obtained, and center position candidates are extracted from the center point heatmap. Regression features are extracted based on the general features of the human body image, and initial features of each body part are obtained by channel grouping the regression features. The initial features are decomposed along the horizontal and vertical directions based on an axial decoupling attention perceptron, and a first offset amount is obtained by calculating, merging, and predicting independent self-attention. The first step offset and the initial features are input to the regressor by interpolation, and a second offset amount is output. The first offset amount and the second offset amount are added to obtain a keypoint offset amount. The result of multi-person pose estimation is obtained by adding the center position candidate and the keypoint offset amount. The server 104 can feed back the obtained multi-person pose estimation result to the terminal 102. In some embodiments, the multi-person pose estimation method can be implemented using only the server 104 or the terminal 102. For example, the terminal 102 may directly extract center position candidates from the human body image data to be processed, obtain keypoint offset amounts for each body part, and obtain the result of multi-person pose estimation by adding the center position candidates and the keypoint offset amounts. Alternatively, the server 104 may obtain human body image data from a data storage system, extract center position candidates from the human body image data to be processed, obtain keypoint offset amounts for each body part, and obtain the result of multi-person pose estimation by adding the center position candidates and the keypoint offset amounts.

[0018] Of these, terminal 102 may be various desktop computers, laptop computers, smartphones, tablets, Internet of Things devices, or portable wearable devices. Internet of Things devices may include, but are not limited to, smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices may include smartwatches, smart bracelets, headsets, etc. Server 104 may be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0019] In one example embodiment, as shown in Figures 2 and 3, a method for estimating the poses of multiple people is provided, which may be executed by a computer device such as a terminal or a server alone, or by a terminal and a server working together. In this embodiment, the method will be explained using an example in which it is applied to the server 104 in Figure 1, and will include the following steps 100 to 700. Step 100: Input the acquired human body image data into a deep aggregation network and output general features of the human body image. Step 200: Obtain a center point heatmap based on general features of the human body image, and extract candidate center positions from the center point heatmap. Step 300: Extract regression features based on general features of human body images, and obtain initial features for each body part through channel grouping of the regression features. Step 400: The initial features are decomposed horizontally and vertically based on an axial decoupling attention perceptron, and the first offset quantity is obtained by calculating, merging, and predicting independent self-attention. Step 500: Input the first offset amount and initial features into the regressor using interpolation, and output the second offset amount. Step 600: The keypoint offset amount is obtained by adding the first offset amount and the second offset amount. Step 700: Multiple person pose estimation results are obtained by adding the candidate center position and the key point offset amount.

[0020] Steps 100 to 700 described above are performed. In this invention, a deep aggregation network is first used to extract general features of the human body image, and then the image is processed by dividing it into a central localization branch and a part-wise regression branch. In the central localization branch, center point heatmap extraction is performed based on the general features of the human body image, making it easy to effectively distinguish different individuals in a multi-person scene, and enabling accurate extraction of center position candidates even in cases of severe occlusion. In the part-wise regression branch, the body is divided into multiple groups using group convolution to avoid feature interference between body parts and reduce computational complexity. Furthermore, an axial decoupling attention perceptron is introduced, and self-attention mechanisms are applied in the horizontal and vertical directions respectively, effectively capturing contextual information of the distal ends of limbs and strengthening the full receptive field of each part's features. By adding the central position candidates obtained from the central positioning branch and the region-specific regression branch, along with the keypoint offset amount, and combining local position detection and overall sensing capabilities, the accuracy of posture estimation is improved, and posture estimation results for multiple individuals are obtained. As a result, information on distal limbs is effectively captured, the processing flow is greatly simplified, and a competitive representation in terms of accuracy is achieved.

[0021] Selectively, in step 200, the general features of the human body image are extracted through a first convolutional layer to extract central heatmap features and generate a center point heatmap. In step 300, the general features of the human body image are extracted through a second convolutional layer to extract regression features. The first convolutional layer may be a normal convolutional layer, an extended convolutional layer, or a depth-separable convolutional layer, and is used to extract central heatmap features. The second convolutional layer may be a normal convolutional layer or an extended convolutional layer, and is used to extract regression features.

[0022] In order to concentrate features on different kinematic areas and avoid competition between body parts, this invention incorporates a grouping concept in the convolution operation to design macrobranches.

[0023] As shown in Figures 3 and 4, regression features are extracted from general features of a human body image through a convolutional layer, and the regression features are grouped by channels. Following the body segmentation mechanism of the self-adaptive pose estimation method, the entire body is divided into seven parts, and the initial features of each body part specifically include head features, shoulder features, left arm features, right arm features, hip features, left leg features, and right leg features. Group-wise convolution is an operation in which channels are divided into several groups and a convolution process is performed. In this application, the features are divided into seven groups, namely F head F shoulder F left_arm F right_arm F hip F left_leg F right_leg It is divided into seven different body parts, each corresponding to a different body part. For example, hip feature F hip This data is input to a dedicated axial decoupling attention perceptron, ADAPhip, and a corresponding regressor, which localize the key points of the hip. This body part-based grouping design explicitly facilitates feature learning for different body parts. Furthermore, to fully utilize the model's performance, the channel dimension of the body part features must be compatible with the size of the backbone network.

[0024] This invention introduces the concept of grouping into region-specific regression branches, thereby effectively avoiding feature interference between body parts, enhancing the model's expressive power, and simultaneously reducing the computational complexity of the region-specific regression branches.

[0025] In another embodiment of the present invention, in order to analyze the feature changes of body parts in different directions in more detail, capture the differences in semantic information in different directions in human body posture estimation, and improve the accuracy of feature representation, the initial features are decomposed into horizontal and vertical directions based on an axial decoupling attention perceptron, and the first offset amount is obtained by calculating, fusing, and predicting independent self-attention. The above step 400 is replaced with the following steps 401 to 404. Step 401: In the axial decoupling attention perceptron, decompose the initial features along the x-axis and y-axis to obtain an x-axis feature vector and a y-axis feature vector. Step 402: Calculate self-attention independently on the x-axis and y-axis for the x-axis feature vector and y-axis feature vector respectively to obtain an x-axis self-attention feature and a y-axis self-attention feature. Step 403: Obtain a fused self-attention feature by fusing the results of the x-axis self-attention feature and the y-axis self-attention feature through dimensional broadcasting. Step 404: Obtain the first offset amount by performing a prediction on the fused self-attention feature through a convolutional layer.

[0026] In this application, by using an axial decoupling attention perceptron, context information can be efficiently modeled, and the features of limb parts at a distance can be captured more effectively.

[0027] As shown in FIG. 5, the axial decoupling attention perceptron is represented in the form of a two-dimensional feature map of seven body parts, and the feature data F with high spatial resolution reg ∈R C×H×W (F in the overall structure head , F shoulder , F left_arm , F right_arm , F hip , F left_leg , F right_leg obtained by compressing the features along the x-y axis) are respectively converted into an x-axis feature vector and a y-axis feature vector, and each is F reg x ∈R C×l×W and F regy ∈R C×H×l After being expressed, the self-attention calculation is performed independently on the x and y axes, respectively, to obtain the feature F￣ reg x (x-axis self-attentional features), F reg y (Y-axis self-attention feature) is generated. Then, the feature F￣ is transmitted through dimensional broadcasting. reg x , F￣ reg y By fusing these, the fused self-awareness features F￣ of each of the seven body parts are achieved. reg We obtained the following. Here, in Figure 5, x-Attn represents the self-attentional operation of the x-axis feature, and y-Attn represents the self-attentional operation of the y-axis feature.

[0028] The formula for self-awareness calculation is represented by the following equation 1.

[0029]

number

[0030] In a specific example, the first offset amount is obtained by performing predictions on fused self-attention features through a convolutional layer.

[0031] The specific formula is represented by the following number 2.

[0032]

number

[0033] In this invention, an axial decoupling attention perceptron, inputting initial features of seven body parts, enhances the overall receptive field of each part's features by modeling the spatial dependencies in the horizontal and vertical directions, thereby obtaining fused self-attention features. Furthermore, the first offset amount of the proxy points of each body part is set to "off". proxyIt becomes easier to obtain.

[0034] Fusion Self-Attention Feature F reg Compared to the initial features, the transformation and fusion by the axial decoupling attention perceptron allowed for efficient modeling of contextual information of body parts, guiding proxy points to be positioned in semantically strong regions and providing a richer basis for subsequent offset calculations.

[0035] In another embodiment of the present invention, in order to effectively improve the accuracy and robustness of keypoint detection, step 500, which involves inputting a first offset amount and initial features into a regressor by interpolation and outputting a second offset amount, is replaced with the following steps 501 to 502. Step 501: Perform a Bilinear Interpolation (BI) operation on the first offset amount and initial features to obtain the features of the semantic proxy point. Step 502: Input the features of the semantic proxy points into the regressor and output the second offset amount.

[0036] The first offset amount is calculated by using bilinear interpolation on the initial features of each of the seven body parts, corresponding to the feature off of the semantic proxy point for each body part. proxy This process is represented by the following equation 3.

[0037]

number

[0038] By inputting the characteristics of the semantic agent points for each body part into the regressor R2, corresponding keypoint feature data is obtained. The process of predicting corresponding keypoints using the regressor R2 is represented by the following equation 4.

[0039]

number

[0040] The calculation of the first offset amount in this application is based on fused self-attention features, allowing for a preliminary estimation of the approximate location of keypoints. Then, the first offset amount and initial features are combined to obtain semantic proxy point features through bilinear interpolation. This process is equivalent to performing semantic enhancement and spatial adjustment on the initial features. Next, the data is input into a regressor to output a second offset amount. In this step, the key position can be further refined. By adding the two offset amounts, the final keypoint offset amount is obtained. This stepwise calculation method allows for precise adjustment of the keypoint position at each step. Furthermore, this stepwise calculation method can handle keypoint localization in complex scenes more accurately than a one-step keypoint localization method.

[0041] In this application, the specific operation for extracting candidate center positions from a center point heatmap is to obtain candidate center positions by applying NMS (Non-Maximum Suppression) processing to the center point heatmap.

[0042] In the inference phase, a 5 × 5 maximum pooling kernel is applied to the predicted center point heatmap to identify candidate central positions of the human body and retain 20 potential pose candidates. Furthermore, in this application, the human pose is constructed by extracting keypoint offset amounts from the segmented regression branch output.

[0043] The training strategy of this invention involves co-optimizing the center point heatmap, offset amount, and site keypoint loss.

[0044] The central localization branch predicts a center point heatmap that represents the confidence level for each pixel. In this application, the center heatmap is monitored using a method that reduces the strength of the penalty by using pixel-level focal loss.

[0045]

number

[0046] At the same time, this application employs standard L1 loss and OKS (Object Keypoint Similarity) loss to constrain the sum of the two offset amounts (i.e., the keypoint offset amount).

[0047]

number

[0048] Loss weights are calculated as follows: loss term hm Lossoks , and Loss l1 These values ​​are set to 1, 5, and 1 respectively.

[0049] Experimental results show that this invention demonstrates high recognition accuracy and fast inference speed on both the MS COCO and CrowdPose datasets, making it applicable to a variety of application scenarios such as sports analysis, security monitoring, and medical rehabilitation.

[0050] Furthermore, the embodiments of the present application provide a device for realizing the above-described multiple-person pose estimation based on the same concept of the invention. Since the problem-solving means provided by this device are similar to the problem-solving means described in the method described above, specific limitations in the one or more embodiments of the multiple-person pose estimation device shown below can be referred to with the limitations on the multiple-person pose estimation method described above and are therefore omitted here.

[0051] In one example embodiment, as shown in Figure 6, a multi-person pose estimation device is provided that includes a general-purpose feature output module 1, a center position candidate extraction module 2, a channel grouping module 3, a first offset amount determination module 4, a second offset amount determination module 5, a keypoint offset amount determination module 6, and a pose estimation module 7.

[0052] The general-purpose feature output module 1 inputs the acquired human body image data into a deep aggregation network and outputs general-purpose features of the human body image. The center position candidate extraction module 2 obtains a center point heatmap based on the general-purpose features of the human body image and extracts center position candidates from the center point heatmap. The channel grouping module 3 extracts regression features based on the general-purpose features of the human body image and obtains initial features for each body part by channel grouping the regression features. The first offset amount determination module 4 obtains the first offset amount by decomposing the initial features along the horizontal and vertical directions based on an axial decoupling attention perceptron, and performing independent self-attention calculation, fusion, and prediction. The second offset amount determination module 5 inputs the first offset amount and initial features into a regressor by interpolation and outputs the second offset amount. The keypoint offset amount determination module 6 obtains the keypoint offset amount by adding the first offset amount and the second offset amount. The posture estimation module 7 obtains the result of multi-person posture estimation by adding the center position candidates and the keypoint offset amount.

[0053] In one example embodiment, a computer device is provided. This computer device may be a server or a terminal, and its internal configuration may be as shown in Figure 7. The computer device comprises a processor, memory, an input / output interface (I / O), and a communication interface. The processor, memory, and I / O interface are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device is used to provide computing and control functions. The memory of the computer device comprises a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program on the non-volatile storage medium. The database of the computer device is used to store data related to multi-person pose estimation processing. The I / O interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements a multi-person pose estimation method.

[0054] A person skilled in the art will understand that the configuration shown in Figure 7 is merely a block diagram showing a part of the configuration relating to the embodiment of the present application, and does not limit the computer equipment to which the embodiment of the present application applies. Specific computer equipment may consist of more or fewer components than those shown in the figure, or may be a combination of some components, or may have a different component arrangement.

[0055] In one example embodiment, a computer device is provided that further includes a memory in which a computer program is stored and a processor, wherein when the processor executes the computer program, the steps in each embodiment of the above method are realized.

[0056] In one example, a computer-readable storage medium is provided, which, when executed by a processor, stores a computer program that implements the steps in each of the above-described embodiments of the method.

[0057] In one example, a computer program product is provided which, when executed by a processor, includes a computer program that implements the steps in each of the above-described embodiments of the method.

[0058] Furthermore, the user information (including, but not limited to, user device information and user personal information) and data (including, but not limited to, data used for analysis, stored data, and data to be displayed) relating to this application are all information and data for which the user or each party has obtained sufficient authorization, and the collection, use, and processing of the relevant data must comply with the relevant regulations.

[0059] Those skilled in the art will understand that all or part of the processing flows of the methods described in the above embodiments can be implemented by instructing the corresponding hardware with a computer program, which can be stored in a non-volatile computer-readable storage medium and, when executed, executes the flow of each embodiment of the above methods. Optional references to memory, databases, or other media used in each embodiment provided herein may include at least one of non-volatile memory and volatile memory. Examples of non-volatile memory include read-only memory (ROM), magnetic tape, flexible disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random-access memory (ReRAM), magnetoresistive random-access memory (MRAM), ferroelectric random-access memory (PCM), graphene memory, etc. Examples of volatile memory include random-access memory (RAM) and external cache memory, etc. However, RAM may take various forms, such as static random-access memory (SRAM) or dynamic random-access memory (DRAM).

[0060] The databases relating to each embodiment provided in this application may include at least one of a relational database and a non-relational database. Examples of non-relational databases include, but are not limited to, blockchain-based distributed databases. Examples of processors relating to each embodiment provided in this application include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processing units, programmable logic units, quantum computing-based data processing logic units.

[0061] The technical features relating to the above embodiments can be combined in any way. For the sake of simplicity, not all possible combinations of these technical features are described, but as long as the combinations of these technical features are not contradictory, they should be considered to fall within the scope of what is described herein.

[0062] This specification has described the principles and embodiments of the present application using specific examples, but these examples are merely intended to aid in understanding the method and its basic concept. Furthermore, those skilled in the art can make various modifications to the specific embodiments and scope of application based on the concept of the present invention. Therefore, the contents of this specification should not be construed as limiting the present application.

Claims

1. A method for estimating the posture of multiple people, The process involves inputting acquired human body image data into a deep aggregation network and outputting general features of the human body image, The steps include obtaining a center point heatmap based on general features of human body images, and extracting candidate center positions from the center point heatmap. The steps include: extracting regression features based on general features of human body images, and obtaining initial features for each body part by channel grouping the regression features; The first offset amount is obtained by decomposing the initial features along the horizontal and vertical directions based on an axial decoupling attention perceptron, calculating, merging, and predicting independent self-attention. The first offset amount and initial features are input to the regressor through interpolation, and the second offset amount is output. The steps include obtaining the keypoint offset amount by adding the first offset amount and the second offset amount, The method is characterized by including the step of obtaining the result of estimating the poses of multiple people by adding the aforementioned center position candidate and key point offset amount. A method for estimating the posture of multiple individuals.

2. The step of obtaining the first offset amount by decomposing the initial features along the horizontal and vertical directions based on the axial decoupling attention perceptron, calculating, merging, and predicting independent self-attention, specifically involves: In an axial decoupling attention perceptron, the initial features are decomposed along the x and y axes to obtain x-axis and y-axis feature vectors. The steps include obtaining x-axis self-attention features and y-axis self-attention features by independently calculating self-attention for the x-axis and y-axis feature vectors, respectively. The steps include obtaining a fused self-attention feature by fusing the results of the x-axis self-attention feature and the y-axis self-attention feature through dimensional broadcasting, The method is characterized by including the step of obtaining a first offset amount by performing predictions on fused self-attention features through a convolutional layer, The method for estimating the postures of multiple persons according to claim 1.

3. The step of inputting the first offset amount and initial features into the regressor by interpolation and outputting the second offset amount is, specifically, The first offset amount and initial features are subjected to a bilinear interpolation operation to obtain the features of the semantic proxy point, The method is characterized by including the step of inputting the characteristics of a semantic proxy point into a regressor and outputting a second offset amount. The method for estimating the postures of multiple persons according to claim 1.

4. By employing standard L1 loss and target keypoint similarity loss, the keypoint offset amount is constrained. In the following number 6, Loss ｒｅｇ This is the loss term for the offset amount of the keypoint, and Loss ｏｋｓ This is the loss term of OKS, and Loss ｌ１ is the loss term of L1, where di represents the Euclidean distance between the i-th predicted keypoint and the true keypoint, s is the target scale parameter, a value related to the size of the target, ki is the weight coefficient for the i-th keypoint, used to adjust the importance of different keypoints, vi is the visibility label of the keypoint, and δ(v ｉ >0) is characterized by indicating whether or not the key point is visible. A method for estimating the postures of multiple people according to claim 1 or claim 3. [Math 6] 【number】 【number】

5. The step of extracting candidate center positions from the aforementioned center point heatmap is characterized by specifically including the step of obtaining candidate center positions by performing a non-maximal value suppression process on the center point heatmap. The method for estimating the postures of multiple persons according to claim 1.

6. The initial features of each body part are characterized by including, specifically, head features, shoulder features, left arm features, right arm features, hip features, left leg features, and right leg features. The method for estimating the postures of multiple persons according to claim 1.

7. A general-purpose feature output module that inputs acquired human body image data into a deep aggregation network and outputs general-purpose features of the human body image, A center point candidate extraction module that obtains a center point heatmap based on general features of human body images and extracts center point candidate data from the center point heatmap. A channel grouping module that extracts regression features based on general features of human body images and obtains initial features of each body part through channel grouping of the regression features, A first offset amount determination module obtains a first offset amount by decomposing initial features along the horizontal and vertical directions based on an axial decoupling attention perceptron, calculating, merging, and predicting independent self-attention, and A second offset amount determination module inputs the first offset amount and initial features into a regressor through interpolation and outputs the second offset amount, A keypoint offset amount determination module that obtains the keypoint offset amount by adding the first offset amount and the second offset amount, The present invention is characterized by including a posture estimation module that obtains the result of estimating the posture of multiple people by adding the aforementioned center position candidate and key point offset amount. A device for estimating the posture of multiple people.

8. A computer device comprising memory, a processor, and a computer program stored in memory and executable on the processor, The processor executes the computer program and realizes the method for estimating the posture of multiple people as described in any one of claims 1 to 3. Computer equipment.

9. A computer-readable storage medium that stores computer programs, The computer program, when executed by the processor, is characterized in that it realizes the method for estimating the posture of multiple people as described in any one of claims 1 to 3. A computer-readable storage medium.

10. Computer program products, including computer programs, The computer program, when executed by the processor, is characterized in that it realizes the method for estimating the posture of multiple people as described in any one of claims 1 to 3. Computer program products.

Images