Method and apparatus for multi-person 3D human pose estimation based on single image

Abstract

The application discloses a multi-person 3D human posture estimation method and device based on a single image, equipment and a storage medium, which comprises the following steps: acquiring an input image to be estimated, performing feature extraction on the image to be estimated, and generating a feature map, wherein the image to be estimated is a two-dimensional single image comprising multiple persons; based on the feature map, performing human center positioning in the image plane, human center coordinate regression under the camera coordinate system, and relative center human key point offset regression through a predicted center confidence map, a center coordinate map and a relative center human key point offset regression map respectively; and combining the output center confidence map, the center coordinate map and the relative center human key point offset regression map to obtain the 3D human posture estimation result corresponding to each person. The model complexity and the calculation consumption can be reduced, and the processing precision can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a multi-person 3D human pose estimation method and device based on a single image and equipment. BACKGROUND

[0002] 3D human pose estimation can be widely applied in VR / AR, games, sports analysis, virtual fitting and other technologies. Compared with multi-view image-based 3D human pose estimation, single image-based 3D human pose estimation is more friendly to deployment environment, deployment cost and device computing amount, and therefore has a wider application scenario.

[0003] The existing mainstream multi-person 3D human pose estimation method based on a single image is a method based on a deep artificial neural network. This method has a top-down two-stage process: in the first stage, a human body detector is used to detect all the persons and their positions in the image; in the second stage, a single-person pose estimator and a depth estimator are respectively applied to the detected persons to obtain 3D pose estimation results of the persons in space. This two-stage method has high computational consumption, and the time complexity is linearly related to the number of persons in the scene. In the case of an increase in the number of persons in the scene, the model inference time also increases sharply, and it is difficult to apply in real complex scenes. SUMMARY

[0004] In view of this, the purpose of the present application is to provide a multi-person 3D human pose estimation method and device based on a single image and equipment, aiming to solve the problems of high model complexity and large computational consumption.

[0005] To achieve the above-mentioned purpose, the present application provides a multi-person 3D human pose estimation method based on a single image, which comprises:

[0006] obtaining an input image to be estimated, performing feature extraction on the image to be estimated to generate a feature map, wherein the image to be estimated is a two-dimensional single image including multiple persons;

[0007] based on the feature map, performing human center positioning in the image plane, human center coordinate regression in the camera coordinate system and relative center human key point offset regression by predicting a center confidence map, a center coordinate map and a relative center human key point offset regression map respectively;

[0008] combining and processing the output center confidence map, center coordinate map and relative center human key point offset regression map to obtain 3D human pose estimation results corresponding to each person.

[0009] Preferably, the image plane human body center positioning, human body center coordinate regression in the camera coordinate system and relative center human body key point offset regression based on the feature map by predicting the center confidence map, the center coordinate map and the relative center human body key point offset regression map respectively, comprises:

[0010] Based on the binary classification, it is judged whether each pixel in the feature map belongs to the human body center of the corresponding person. The N pixels closest to the two-dimensional projection of the human body center in the image plane are defined as positive sample pixels, and the remaining pixels are defined as negative sample pixels, so as to perform human body center positioning in the image plane by predicting the center confidence map; wherein the confidence of the positive sample pixel is set to 1, and the confidence of the negative sample pixel is set to 0;

[0011] The mapping of the two-dimensional human body center to the three-dimensional human body center is determined by regressing the offset of the positive sample pixel to the human body center, so as to perform human body center coordinate regression in the camera coordinate system by predicting the center coordinate map;

[0012] The three-dimensional human body center is regressed to the position of the human body key point of the corresponding person, the offset of the human body center to the human body key point is determined, and the relative center human body key point offset regression is performed by predicting the relative center human body key point offset regression map.

[0013] Preferably, it further comprises:

[0014] According to The prediction of the center confidence map is optimized, wherein C H represents the center confidence map, represents the target center confidence map.

[0015] Preferably, it further comprises:

[0016] According to

[0017]

[0018] The prediction of the center coordinate map is optimized, wherein U root [p] represents the center coordinate map, represents the target center coordinate map.

[0019] Preferably, it further comprises:

[0020] According to

[0021]

[0022] The prediction of the relative center human body key point offset regression map is optimized, wherein U k [p] represents the relative center human body key point offset regression map, a human key point offset regression map of a target relative center.

[0023] Preferably, further comprising:

[0024] The prediction of the human key point offset regression map of the relative center is recursively updated, a normalizing flow model is used to learn the probability distribution of the human key point position in space, and a maximum likelihood estimation target function is used for optimization.

[0025] Preferably, the output center confidence map, center coordinate map and relative center human key point offset regression map are combined for processing to obtain a 3D human pose estimation result corresponding to each person, including:

[0026] The 3D human pose estimation result corresponding to each person is obtained by selecting pixels with a prediction score value greater than a preset value on the center confidence map as the two-dimensional human center, and adding the values corresponding to the center coordinate map and the relative center human key point offset regression map at the corresponding positions.

[0027] To achieve the above-mentioned purpose, the application further provides a multi-person 3D human pose estimation device based on a single image, the device comprising:

[0028] A feature extraction unit is configured to acquire an input image to be estimated, perform feature extraction on the image to be estimated, and generate a feature map, wherein the image to be estimated is a single image including multiple persons.

[0029] A prediction unit is configured to perform human center positioning in an image plane, human center coordinate regression in a camera coordinate system and relative center human key point offset regression based on the feature map by predicting a center confidence map, a center coordinate map and a relative center human key point offset regression map, respectively.

[0030] A pose estimation unit is configured to combine the output center confidence map, center coordinate map and relative center human key point offset regression map for processing to obtain a 3D human pose estimation result corresponding to each person.

[0031] To achieve the above-mentioned purpose, the application further provides a device comprising a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a multi-person 3D human pose estimation method based on a single image as described in the above embodiments.

[0032] In order to achieve the above object, the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement steps of the multi-person 3D human pose estimation method based on a single image.

[0033] Advantages:

[0034] The above scheme combines the output center confidence map, center coordinate map and relative center human key point offset regression map by inputting a two-dimensional image into the model, directly obtains the 3D human pose estimation result corresponding to each person, does not need an additional human detector and serial single-person pose estimator, decomposes the multi-person 3D human pose estimation into multiple parallel tasks, reduces the model complexity and computation consumption, and improves the processing precision. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0036] Figure 1 A flowchart of a multi-person 3D human pose estimation method based on a single image provided by an embodiment of the present application.

[0037] Figure 2 A network framework diagram of a 3D human pose estimation network provided by an embodiment of the present application.

[0038] Figure 3 A 3D human pose estimation visualization result diagram provided by an embodiment of the present application.

[0039] Figure 4 A structure diagram of a multi-person 3D human pose estimation device based on a single image provided by an embodiment of the present application.

[0040] The realization of the object of the application, the functional features and the advantages will be further illustrated with reference to the embodiments and the drawings. DETAILED DESCRIPTION

[0041] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0042] In the description of the present application, the terms "first", "second" are only used for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features.

[0043] The content of the present application will be described in detail below in combination with embodiments.

[0044] In addition to the top-down method described above, another method in the prior art includes a two-stage bottom-up process: the first stage is to locate all the human key points in the scene, and the locating step does not distinguish between human instances; the second stage is to aggregate the key points belonging to each human through an association clustering algorithm, to form the final multi-person 3D pose. The operation time of this method is less relevant to the number of persons in the scene, but a complex second-stage key point clustering algorithm is required, and the accuracy is usually inferior to the top-down multi-person 3D pose estimation method. In summary, the existing 3D human pose estimation methods all require high computational consumption, and have low accuracy.

[0045] Based on this, the application provides a multi-person 3D human posture estimation method based on a single image, which converts multi-person 3D human posture estimation into human center positioning in an image plane, human center coordinate regression in a camera coordinate system and human key point offset regression relative to the center. In addition, the internal distribution of 3D key point positions in space is modeled by introducing normalizing flow to guide the learning of the regression model, and the relative center human key point offset prediction is continuously optimized through recursive updating, so that the 3D human posture estimation result is more accurate. The method is realized based on a convolutional neural network, and three intermediate outputs are generated in one network forward process: a center confidence map, a center coordinate map and a center-relative joint offset map. The multi-person 3D human posture estimation result can be obtained by combining these intermediate outputs without other complex association clustering methods. The method can reduce model complexity and computational consumption and improve processing accuracy.

[0046] Reference Figure 1 Fig. 1 shows a flowchart of a multi-person 3D human posture estimation method based on a single image according to an embodiment of the application.

[0047] In the embodiment, the method is realized based on a pre-trained 3D human posture estimation network, wherein the network framework of the 3D human posture estimation network includes a feature extraction backbone network, a feature pyramid network, a center confidence prediction subnetwork, a center coordinate prediction subnetwork and a human key point offset regression subnetwork. For details, refer to Fig. 2. Figure 2 Fig. 2 shows a network framework diagram of a 3D human posture estimation network.

[0048] S11, an input to-be-estimated image is acquired, feature extraction is performed on the to-be-estimated image to generate a feature map, wherein the to-be-estimated image is a two-dimensional single image including multiple persons.

[0049] S12, based on the feature map, human center positioning in an image plane, human center coordinate regression in a camera coordinate system and human key point offset regression relative to the center are performed by predicting a center confidence map, a center coordinate map and a center-relative joint offset map, respectively.

[0050] S13, the output center confidence map, center coordinate map and center-relative joint offset map are combined and processed to obtain 3D human posture estimation results corresponding to each person.

[0051] The output center confidence map, center coordinate map, and human keypoint offset regression map relative to the center are combined to obtain the 3D human pose estimation results for each character, including:

[0052] By selecting pixels with predicted scores greater than a preset value on the center confidence map as the two-dimensional human body center, and adding the values ​​corresponding to the center coordinate map and the human body key point offset regression map relative to the center at the corresponding positions, the 3D human body pose estimation results for each person are obtained.

[0053] Furthermore, the prediction of human body center localization in the image plane, human body center coordinate regression in the camera coordinate system, and human body key point offset regression relative to the center of the feature map includes:

[0054] S12-1, Based on binary classification, determine whether each pixel in the feature map belongs to the center of the corresponding person's body. Define the N pixels closest to the center of the person's body in the two-dimensional projection in the image plane as positive sample pixels and the remaining pixels as negative sample pixels, so as to locate the center of the person's body in the image plane by predicting the center confidence map; wherein, the confidence of the positive sample pixels is set to 1 and the confidence of the negative sample pixels is set to 0.

[0055] Furthermore, it also includes:

[0056] according to The prediction of the center confidence map is optimized, wherein C H This represents the central confidence map. This represents the confidence plot of the target center.

[0057] In this embodiment, the person H in the given image i ={j ik =(x ik ,y ik ,d ik )|k∈[1…K]}, where j ik Let j be the 3D coordinate of the k-th keypoint of the i-th character. The 3D coordinates of the human keypoint j are represented by the two-dimensional coordinates (x, y) in the image plane and the depth value d in the camera coordinate system. For each character H, the position of its human body center as the root keypoint (usually set as the pelvic keypoint) is defined as j. root The problem of locating the center of a human body in an image plane is treated as a binary classification problem, that is, determining whether each pixel in the feature map belongs to a certain human body center j. root Among them, the one closest to the center of each human body is j root Two-dimensional projection (x) in the image plane root ,y root Npos Each of the positive sample pixels will be regarded as a positive sample pixel (confidence of 1), and the other pixels are regarded as negative sample pixels (confidence of 0). In this embodiment, the human body center positioning is performed by predicting the center confidence map. Specifically, the predicted center confidence map is C H , and the target center confidence map is The prediction of the center confidence is optimized using the Focal loss, and the formula is as follows:

[0058]

[0059] S12-2, determining the mapping from the two-dimensional human body center to the three-dimensional human body center by regressing the offset of the positive sample pixel to the human body center, so as to regress the human body center coordinates in the camera coordinate system by predicting the center coordinate map.

[0060] Further, it also includes:

[0061] According to

[0062]

[0063] The prediction of the center coordinate map is optimized, wherein U root [p] represents the center coordinate map, and represents the target center coordinate map.

[0064] In this embodiment, for the human body center j root = (x root , y root , d root ) and its corresponding positive sample pixel p = (x p , y p ) in a certain image plane, the algorithm regresses the offset (x root -x p , y root -y p , d root ) from p to the human body center coordinate j root . The mapping from each detected two-dimensional human body center to the three-dimensional human body center is represented by predicting the center coordinate map U root . Specifically, the regression target is set as The algorithm uses the L1 loss to optimize the prediction of the center coordinate regression, and the formula is as follows:

[0065]

[0066] S12-3, regress the three-dimensional human center to the human body key point position of the corresponding person, determine the offset of the human body center to the human body key point, and regress the relative center human body key point offset by predicting the relative center human body key point offset regression graph.

[0067] Further, it also includes:

[0068] According to

[0069]

[0070] The prediction of the relative center human body key point offset regression graph is optimized, wherein U k [p] represents the relative center human body key point offset regression graph, represents the target relative center human body key point offset regression graph.

[0071] In this embodiment, each human body key point position is directly regressed from the 3D human center, and the offset from the human body center j root to the kth human body key point j k is j root -j k =(x root -x k ,y root -y k ,d root -d k ). The relative center human body key point offset regression graph U joint ={U1,…,U K} is predicted, wherein U k encodes the offset from the human body center to the human body key point j k . For each positive sample pixel p of a person H, the target relative center human body key point offset regression graph is The algorithm uses L1 loss to optimize the prediction of center coordinate regression, and the formula is as follows:

[0072]

[0073] Further, it also includes:

[0074] The prediction of the relative center human body key point offset regression graph is recursively updated, the probability distribution of the human body key point position in space is learned by using a normalizing flow model, and the maximum likelihood estimation objective function is used for optimization.

[0075] In this embodiment, for positive sample pixels p, in order to better model the human key point position u = U[p], the predicted human key point offset is recursively updated as follows:

[0076] U[p]←U[p]+U[p+U[p]]

[0077] Further, a normalizing flow model is used to learn the probability distribution of the human key point position. Denote the normalizing flow model parameter as θ, and the learned human key point position distribution as u ~ P(u|θ). The algorithm uses a maximum likelihood estimation objective function to optimize the learning of the human key point position distribution, where is the target human key point position.

[0078]

[0079] In addition, the effectiveness of the above method is verified on a large-scale public multi-person 3D pose benchmark dataset. CMUPanoptic is a large-scale indoor scene multi-person 3D pose dataset containing 65 videos of daily activities captured by multiple cameras. The method is verified on the CMU Panoptic dataset according to the previous evaluation protocol, and the MPJPE (mean per joint position error) is calculated on 9600 frames from four activities (Haggling, Mafia, Ultimatum, Pizza). The experimental results are shown in the following table:

[0080]

[0081] Further, the embodiment outputs the center confidence map C H , the center coordinate map U root and the relative center human key point offset regression map U joint by inputting a two-dimensional image. By selecting pixels with a predicted score greater than a certain threshold on the center confidence map C H as the two-dimensional human center, and taking the corresponding values on the center coordinate map U root and the relative center human key point offset regression map U joint , the individual 3D human pose estimation results are obtained by adding them together. The algorithm uses pose non-maximum suppression to reduce redundant predictions. The visualization results are shown in Figure 3 .

[0082] In summary, the embodiment decomposes multi-person 3D pose estimation into multiple parallel tasks, avoids the serial operation of the previous two-stage method, reduces the model complexity and computational consumption. In addition, the embodiment has higher accuracy than the existing bottom-up method and most top-down methods, and the model inference time is not affected by the number of people in the scene, providing a new solution for the application of multi-person 3D human pose estimation.

[0083] Referring to Figure 4 Fig. 1 shows a structural schematic diagram of a multi-person 3D human pose estimation device based on a single image according to an embodiment of the present application.

[0084] In the embodiment, the device 40 comprises:

[0085] a feature extraction unit 41 configured to acquire an input image to be estimated, perform feature extraction on the image to be estimated, and generate a feature map, wherein the image to be estimated is a single image including multiple persons;

[0086] a prediction unit 42 configured to perform human center positioning in an image plane, human center coordinate regression in a camera coordinate system, and relative center human key point offset regression based on the feature map by predicting a center confidence map, a center coordinate map, and a relative center human key point offset regression map, respectively;

[0087] a pose estimation unit 43 configured to combine the output center confidence map, center coordinate map, and relative center human key point offset regression map to obtain 3D human pose estimation results corresponding to each person.

[0088] Further, the prediction unit 42 comprises:

[0089] a first prediction unit configured to determine whether each pixel in the feature map belongs to the human center of the corresponding person by binary classification, define N pixels closest to the two-dimensional projection of the human center in the image plane as positive sample pixels, and define the remaining pixels as negative sample pixels, to perform human center positioning in the image plane by predicting the center confidence map; wherein the confidence of the positive sample pixels is set to 1, and the confidence of the negative sample pixels is set to 0;

[0090] a second prediction unit configured to determine the mapping of the two-dimensional human center to the three-dimensional human center by regressing the offset of the positive sample pixels to the human center, to perform human center coordinate regression in the camera coordinate system by predicting the center coordinate map;

[0091] a third prediction unit configured to regress the three-dimensional human center to the human key point position of the corresponding person, determine the offset of the human center to the human key point, and regress the relative center human key point offset by predicting the relative center human key point offset regression map.

[0092] Further, the method further comprises:

[0093] According to optimizing the prediction of the center confidence map, wherein C H represents the center confidence map, represents the target center confidence map.

[0094] Further, the method further comprises:

[0095] According to

[0096]

[0097] optimizing the prediction of the center coordinate map, wherein U root [p] represents the center coordinate map, represents the target center coordinate map.

[0098] Further, the method further comprises:

[0099] According to

[0100]

[0101] optimizing the prediction of the relative center human key point offset regression map, wherein U k [p] represents the relative center human key point offset regression map, represents the target relative center human key point offset regression map.

[0102] Further, the method further comprises:

[0103] recursively updating the prediction of the relative center human key point offset regression map, learning the probability distribution of the human key point position in space by using a normalizing flow model, and optimizing by using a maximum likelihood estimation objective function.

[0104] Further, the pose estimation unit 43 is further configured to:

[0105] obtaining the 3D human pose estimation result corresponding to each person by selecting the pixels with a predicted score value greater than a preset value on the center confidence map as the two-dimensional human center, and adding the values corresponding to the center coordinate map and the relative center human key point offset regression map at the corresponding positions.

[0106] The various unit modules of the device 40 can respectively perform the corresponding steps in the above method embodiments, and thus are not described in detail here. For details, please refer to the description of the corresponding steps above.

[0107] The embodiment of the application also provides a device comprising the single-image-based multi-person 3D human pose estimation device as described above, wherein the single-image-based multi-person 3D human pose estimation device can adopt Figure 4 The structure of the embodiment, which can correspondingly perform Figure 1 The technical solutions of the method embodiments shown in the drawings have similar implementation principles and technical effects, and details can be referred to the relevant description in the above embodiments, which will not be described here.

[0108] The device can include a mobile phone, a digital camera, a tablet computer, or other devices with a photographing function, or a device with an image processing function, or a device with an image display function. The device can include a memory, a processor, an input unit, a display unit, a power supply, and the like.

[0109] The memory can be used to store software programs and modules, and the processor can execute various function applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs (such as an image playing function, etc.) required by at least one function, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Correspondingly, the memory can also include a memory controller to provide access to the memory for the processor and the input unit.

[0110] The input unit can be used to receive input digital or character or image information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. Specifically, the input unit of the embodiment can include a camera, a touch-sensitive surface (such as a touch display screen), and other input devices.

[0111] The display unit can be used to display information input by the user or provided to the user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, video and any combination thereof. The display unit can include a display panel, which can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface can cover the display panel, and when the touch-sensitive surface detects a touch operation thereon or adjacent thereto, it transmits to the processor to determine the type of touch event, and then the processor provides corresponding visual output on the display panel according to the type of touch event.

[0112] The embodiment of the present application further provides a computer readable storage medium, which can be the computer readable storage medium contained in the memory in the above-mentioned embodiments, or can exist separately and not be assembled into the device. The computer readable storage medium stores at least one instruction, which is loaded and executed by the processor to realize Figure 1 The computer readable storage medium can be a read-only memory, a disk or an optical disk, etc.

[0113] It should be noted that each of the embodiments in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between each embodiment can be referred to each other. For the device embodiment, the equipment embodiment and the storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the part of the method embodiment.

[0114] Also, in this document, the terms "comprise", "comprising", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or device that includes the element.

[0115] The foregoing description has shown and described preferred embodiments of the application, but it will be understood that the application is not limited to the particular embodiments disclosed, and it will be understood that various other modifications, applications and embodiments can be made without departing from the spirit and scope of the application, and that the scope of the application will be measured by the claims that follow.

Claims

1. A single image based multi-person 3D human pose estimation method, characterized in that, The method comprises: obtaining an input image to be estimated, performing feature extraction on the image to be estimated to generate a feature map, wherein the image to be estimated is a two-dimensional single image including multiple persons; based on the feature map, respectively through the prediction of the center confidence map, the center coordinate map and the relative center human key point offset regression map, the human body center positioning in the image plane, the human body center coordinate regression in the camera coordinate system and the relative center human key point offset regression are carried out; further comprising: based on the binary classification, it is judged whether each pixel in the feature map belongs to the human body center of the corresponding person, the N pixels closest to the two-dimensional projection of the human body center in the image plane are defined as positive sample pixels, and the remaining pixels are defined as negative sample pixels, so as to carry out human body center positioning in the image plane by predicting the center confidence map; wherein the confidence of the positive sample pixel is set to 1, and the confidence of the negative sample pixel is set to 0; determine the mapping of the two-dimensional human body center to the three-dimensional human body center by regression of the offset of the positive sample pixel to the human body center, so as to carry out human body center coordinate regression in the camera coordinate system by predicting the center coordinate map; the three-dimensional human body center is regressed to the human key point position of the corresponding person to determine the offset of the human body center to the human key point, so as to carry out relative center human key point offset regression by predicting the relative center human key point offset regression map; combine the output center confidence map, center coordinate map and relative center human key point offset regression map to obtain the 3D human pose estimation result corresponding to each person; further comprising: select the pixel with a predicted score greater than a preset value on the center confidence map as the two-dimensional human body center, and add the corresponding values of the center coordinate map and the relative center human key point offset regression map at the corresponding position to obtain the 3D human pose estimation result corresponding to each person.

2. The method of claim 1, wherein, Further comprising: According to optimizing the prediction of the center confidence map, wherein C H representing the center confidence map, representing a target center confidence map.

3. The method of claim 1, wherein, Further comprising: According to optimizing the prediction of the center coordinate map, wherein U root [p] represents the center coordinate map, represents the target center coordinate map.

4. The single image based multi-person 3D pose estimation method of claim 1, wherein, Further comprising: According to optimizing the prediction of the relative center human keypoint offset regression map, wherein U k [p] represents the relative center human keypoint offset regression map, represents the target relative center human keypoint offset regression map.

5. The single image based multi-person 3D pose estimation method of claim 1, wherein, Further comprising: The prediction of the relative center human key point offset regression map is recursively updated, the normalizing flow model is used to learn the probability distribution of the human key point position in space, and the maximum likelihood estimation objective function is used for optimization.

6. An apparatus for single image based multi-person 3D human pose estimation, the apparatus comprising: a processor configured to: receive a single image; and determine a 3D human pose for each person in the image. The device comprises: a feature extraction unit configured to obtain an input image to be estimated, perform feature extraction on the image to be estimated to generate a feature map, wherein the image to be estimated is a single image including multiple persons; a prediction unit configured to perform human body center positioning in the image plane, human body center coordinate regression in the camera coordinate system and relative center human key point offset regression by predicting a center confidence map, a center coordinate map and a relative center human key point offset regression map based on the feature map; the prediction unit is further configured to: Based on the binary classification, it is determined whether each pixel in the feature map belongs to the body center of the corresponding person. N pixels closest to the two-dimensional projection of the body center in the image plane are defined as positive sample pixels, and the remaining pixels are defined as negative sample pixels, so as to locate the body center in the image plane by predicting the center confidence map; wherein the confidence of the positive sample pixel is set to 1, and the confidence of the negative sample pixel is set to 0; The mapping of the two-dimensional body center to the three-dimensional body center is determined by regression of the offset of the positive sample pixel to the body center, so as to regress the body center coordinates in the camera coordinate system by predicting the center coordinate map; The three-dimensional body center is regressed to the body key point position of the corresponding person, the offset of the body center to the body key point is determined, and the relative center body key point offset regression is performed by predicting the relative center body key point offset regression map; The posture estimation unit is used for combining and processing the output center confidence map, the center coordinate map and the relative center body key point offset regression map to obtain the 3D human body posture estimation result corresponding to each person; the posture estimation unit is further used for: By selecting the pixel with a predicted score value greater than a preset value on the center confidence map as the two-dimensional body center, and adding the values corresponding to the center coordinate map and the relative center body key point offset regression map at the corresponding positions, the 3D human body posture estimation result corresponding to each person is obtained.

7. An apparatus, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the multi-person 3D human body posture estimation method based on a single image according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the multi-person 3D human body posture estimation method based on a single image according to any one of claims 1 to 5.

Images