Rapid human body posture estimation method and estimation system

By obtaining scene pictures from multiple perspectives, identifying human units and calculating ground positions, combining clustering and appearance feature comparison, triangulation method and minimizing reprojection error optimization, the consumption of identifying the same person target in multiple shooting devices is solved. The problem of high hardware performance has achieved fast and accurate 3D human body posture estimation.

CN119169659APending Publication Date: 2024-12-20GUANGZHOU HAOYANG ELECTRONICS CO LTD

Patent Information

Application Number
CN202411220222.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-20

AI Technical Summary

Technical Problem

The prior art process of identifying the same person's target in scene pictures taken by multiple shooting devices is time-consuming and requires high hardware performance, making it difficult to achieve fast and accurate 3D human posture estimation.

Method used

By obtaining scene pictures from multiple perspectives, identifying human body units and calculating ground positions, initial clustering and appearance feature comparison, combining triangulation and minimizing reprojection error optimization, we quickly construct 3D human body posture.

Benefits of technology

Significant time savings, reduce hardware performance requirements, improve the accuracy and efficiency of 3D human posture estimation, and can handle occlusion and complex backgrounds.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119169659A_ABST
    Figure CN119169659A_ABST
Patent Text Reader

Abstract

The invention discloses a rapid human body posture estimation method and system, and the method comprises the following steps: S1, obtaining scene pictures of a performance site at the same time through a plurality of photographing devices, recognizing a human body unit in each scene picture, and obtaining a bounding box and a human body key point; s3, obtaining the ground position of the human body unit on the ground of the performance site according to the bounding box or the human body key points; s4, performing initialization clustering on all the ground positions, and preliminarily obtaining all human body units corresponding to the same person target in all the scene pictures; and S5, calculating 3D positions of the human body key points based on the human body key points of the human body units corresponding to the same person target in each scene picture. According to the method, initialization clustering analysis is carried out on the ground positions of the human body units on the ground of the performance site, and the human body units with relatively close ground positions in different scene pictures are considered to belong to the same character target, so that cross-view matching is quickly completed, and time is greatly saved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of human body posture estimation, and more specifically, to a fast human body posture estimation method and estimation system. Background Art

[0002] It is common to use multiple cameras to shoot human targets and estimate their 3D human poses (i.e., to analyze 3D human poses through multiple cameras in a collaborative manner). However, one of the major difficulties of this technology is how to accurately identify the same human target from scene images shot by multiple cameras so as to estimate the 3D human pose of the human target. The current cross-view matching algorithm needs to construct a complex matrix and then obtain the matching results through multiple iterations. This operation is time-consuming and requires high hardware performance. Summary of the invention

[0003] In order to overcome at least one of the defects of the above-mentioned prior art, the present invention provides a method for fast human posture estimation, which can quickly identify the same person target from scene pictures taken by multiple shooting devices, thereby completing human posture estimation.

[0004] To solve the above technical problems, the technical solution adopted by the present invention is: a method for fast human posture estimation, comprising the following steps:

[0005] S1. Using multiple shooting devices to simultaneously obtain scene images of the performance venue from multiple perspectives;

[0006] S2, identify the human body units in each scene image and obtain the bounding box and human body key points;

[0007] S3, according to the bounding box or human key points of the human body unit in each scene image, deriving the ground position of the human body unit on the ground of the performance venue;

[0008] S4, initializing the ground position of each human body unit in all scene images and clustering them, and preliminarily obtaining each human body unit corresponding to the same person target in all scene images;

[0009] S5. Based on the human body key points of the human body units corresponding to the same human target in each scene image, the 3D positions of the human body key points are calculated to construct a 3D human body posture.

[0010] The fast human posture estimation method transforms the recognition of human body units in scene pictures taken by multiple shooting devices into an analysis of the ground position of the human body units on the ground of the performance venue. By initializing clustering, human body units with relatively close ground positions in different scene pictures are considered to belong to the same human target, thereby quickly completing cross-view matching, greatly saving time, reducing the requirements for hardware performance, and facilitating the calculation of the 3D positions of human body key points based on the human body key points of the human body units corresponding to the same human target in each scene picture, and constructing 3D human posture.

[0011] Furthermore, in step S3, the ground position of the human body unit on the ground of the performance venue is calculated according to the midpoint of the lower boundary of the bounding box. The bounding box parameters are simple to calculate, and the lower boundary of the bounding box is almost close to the ground. The ground position of the human body unit on the ground of the performance venue is calculated using the points on it, which reduces the projection error and makes the estimation of the ground position of the human body unit more accurate. Ideally, the midpoints of the lower boundaries of the bounding boxes of the same human target in different scene images should overlap, so the human body units belonging to the same human target in the scene image can be judged based on this.

[0012] Furthermore, in step S4, the initial clustering adopts a threshold-based clustering method, and the distances between all ground positions in the same category are less than the first threshold. The threshold-based clustering method is relatively simple and intuitive to implement, does not require complex preprocessing or parameter setting, only needs to define a suitable first threshold distance, has low computational complexity, and does not require the number of clusters to be determined in advance (i.e., the number of human targets does not need to be determined in advance).

[0013] Furthermore, the first threshold is 0 cm-5 cm, so as to avoid mutual interference between different human targets.

[0014] Furthermore, between step S4 and step S5, it also includes comparing the appearance features of the human body units corresponding to the same human target in all the scene images initially obtained, and accurately obtaining the human body units corresponding to the same human target. Due to the existence of occlusion and illumination change problems, the detection of human key points is not 100% accurate, which will lead to human key point matching errors between different human targets. Therefore, after initializing clustering of all ground positions and preliminarily obtaining the human body units corresponding to the same human target, the appearance features of these human body units are compared again to detect whether there is a misjudgment, exclude abnormal human body units, and classify them into the correct human target, so that the 3D human posture can be estimated more accurately.

[0015] Furthermore, when comparing the appearance features of the human body unit, a threshold-based comparison method is used to exclude human body units whose Euclidean distance to other human body units is greater than a second threshold. The threshold-based comparison method is relatively simple and intuitive to implement, and has a low computational complexity.

[0016] Furthermore, the appearance feature is a Reid feature. The object of Reid feature research is the entire characteristics of a person, including clothing, body shape, hairstyle, posture, etc., without using face recognition as a means, and there is no rigid requirement for the clarity, installation position, and angle of the shooting device.

[0017] Furthermore, in step S5, according to the human body key points of the human body units corresponding to the same person target in each scene image, combined with the position and internal parameters of the shooting device, the 3D position of the human body key points is calculated by triangulation to construct the 3D human body posture. The triangulation method is fast and accurate in calculation, can handle occlusion and complex background, and reduce recognition errors through algorithm optimization.

[0018] Furthermore, in step S5, the 3D position is optimized by minimizing the reprojection error using the bundle adjustment method, and then the 3D human posture is constructed. Each 3D position of the calculated human target is optimized by minimizing the reprojection error, and the value that minimizes the comprehensive error between its back projection in the scene image and the corresponding all human key points is obtained as the optimized 3D position, and then the 3D human posture is constructed accordingly.

[0019] The present invention also provides an estimation system, which runs any of the above-mentioned fast human posture estimation methods, including a shooting device for acquiring scene pictures, a front-end data processing device for identifying human units in each scene picture and obtaining bounding boxes and human key points, and a back-end data processing device for completing 3D human posture estimation. The shooting device, the front-end data processing device, and the back-end data processing device work together, each performing its own duties, and finally quickly obtain a 3D human posture.

[0020] Furthermore, the back-end data processing device is connected to the camera device for communication. Whenever the back-end data processing device sends a request instruction to the camera device, each camera device obtains a frame of the scene picture of the performance venue. Compared with the current processing method, each camera device sends a video stream with a timestamp to the data processing center in real time, and the data processing center then screens the synchronization frame by comparing the timestamp. The solution of the prior art will lead to redundant transmission of data, which puts a certain pressure on the communication bandwidth of the data processing center, and the time synchronization of each camera device is not easy to maintain. The present application adopts a processing method in which the camera device feeds back the scene picture according to the request instruction of the back-end data processing device, which can effectively solve the problems of redundant transmission and difficult time synchronization, can achieve millisecond-level synchronization accuracy, and significantly improve the efficiency and accuracy of data synchronization. Furthermore, the back-end data processing device asynchronously sends request instructions to each camera device. After the request instruction is issued, there is no need to wait for the current camera device to return data, and the request instruction can continue to be sent to the next camera device, which speeds up the data acquisition speed of the back-end data processing device.

[0021] Furthermore, the number of front-end data processing devices corresponds to the number of shooting devices, and multiple front-end data processing devices are connected to a back-end data processing device. Each front-end data processing device processes the scene pictures sent by the shooting device connected to it, identifies the human body units in each scene picture in advance and obtains the bounding box and human body key points, and then sends the data to the back-end data processing device, which can effectively reduce the performance requirements of the back-end data processing device and speed up the data processing speed.

[0022] Furthermore, the back-end data processing device has multiple threads. When calculating the 3D positions of the key points of the human body of the same person target, the task is divided into multiple parts and processed in parallel by multiple threads. Compared with the prior art, in which a single thread calculates the 3D position of each key point of the human body one by one, by setting multiple threads in the back-end data processing device and processing the 3D positions of multiple key points of the human body in parallel, the speed of data processing can be accelerated. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of the fast human body posture estimation method of the present invention.

[0024] Figure 2 It is a schematic diagram of steps S1 and S2 of the method for rapid human posture estimation of the present invention.

[0025] Figure 3 It is a schematic diagram of steps S3 and S4 of the method for rapid human posture estimation of the present invention.

[0026] Figure 4 It is a schematic diagram of the appearance feature comparison of the present invention.

[0027] Figure 5 Schematic diagram of step S5 of the method for rapid human posture estimation of the present invention.

[0028] Figure 6 It is a schematic diagram of the structure of the estimation system of the present invention. DETAILED DESCRIPTION

[0029] The drawings are only for illustrative purposes and cannot be construed as limiting the present invention. To better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged, or reduced, and do not represent the size of the actual product. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the drawings. The positional relationships described in the drawings are only for illustrative purposes and cannot be construed as limiting the present invention.

[0030] like Figures 1 to 3 and Figure 5 The present invention provides a method for fast human posture estimation, comprising the following steps:

[0031] S1. Using multiple shooting devices to simultaneously obtain scene images of the performance venue from multiple perspectives;

[0032] S2, identify the human body units in each scene image and obtain the bounding box and human body key points;

[0033] S3, according to the bounding box or human key points of the human body unit in each scene image, deriving the ground position of the human body unit on the ground of the performance venue;

[0034] S4, initializing the ground position of each human body unit in all scene images and clustering them, and preliminarily obtaining each human body unit corresponding to the same person target in all scene images;

[0035] S5. Based on the human body key points of the human body units corresponding to the same human target in each scene image, the 3D positions of the human body key points are calculated to construct a 3D human body posture.

[0036] The fast human posture estimation method transforms the recognition of human body units in scene pictures taken by multiple shooting devices into an analysis of the ground position of the human body units on the ground of the performance venue. By initializing clustering, human body units with relatively close ground positions in different scene pictures are considered to belong to the same human target, thereby quickly completing cross-view matching, greatly saving time, reducing the requirements for hardware performance, and facilitating the calculation of the 3D positions of human body key points based on the human body key points of the human body units corresponding to the same human target in each scene picture, and constructing 3D human posture.

[0037] The floor of the performance venue can be the actual ground, or the floor of the stage, that is, the plane on which the character target stands.

[0038] In this embodiment, the number of the photographing devices is 3 or more, preferably 5.

[0039] In the present invention, the RtmDet detector is first used to extract the bounding box of the human body unit, and then the RtmPose detector is used to extract the human body key points of the human body unit from the bounding box.

[0040] In a preferred embodiment of the present invention, in step S3, the ground position of the human body unit on the ground of the performance venue is calculated according to the midpoint of the lower boundary of the bounding box. The bounding box parameters are simple to calculate, and the lower boundary of the bounding box is almost close to the ground. The ground position of the human body unit on the ground of the performance venue is calculated using the points on it, which reduces the projection error and makes the estimation of the ground position of the human body unit more accurate. The midpoints of the lower boundaries of the bounding boxes of the same human target in different scene pictures should ideally overlap, so the human body units belonging to the same human target in the scene pictures can be judged accordingly.

[0041] In the present application, the ground position of the human body unit on the ground of the performance venue is preferably the position in the world coordinate system, that is, the midpoint (u, v) of the lower boundary of the bounding box in the shooting coordinate system is converted to the world coordinate system to obtain the ground position (x, y, z) of the human body unit on the ground of the performance venue, as follows:

[0042]

[0043] Where s is a scaling factor, K is the intrinsic parameter matrix of the camera with a size of 3×3. [R|t] is the extrinsic parameter matrix with a size of 3×4. In order to extract the position of the feature in the world coordinate system, we consider the ground plane (z=0) as a common reference plane, and formula (1) can be rewritten as formula (2):

[0044]

[0045] According to formula (2), the midpoint (u, v) of the lower boundary of the bounding box of each human unit in the scene image is projected onto the common ground in reality, as follows: Figure 3 shown.

[0046] In a preferred embodiment of the present invention, in step S4, the initial clustering adopts a threshold-based clustering method, and the distances between all ground positions in the same category are less than a first threshold. The threshold-based clustering method is relatively simple and intuitive to implement, does not require complex preprocessing or parameter setting, only needs to define a suitable first threshold distance, has low computational complexity, and does not require predetermining the number of clusters (i.e., does not require predetermining the number of human targets).

[0047] In a preferred embodiment of the present invention, the first threshold is 0 cm-5 cm, so as to avoid mutual interference between different human targets.

[0048] In the present invention, the first threshold is 3 cm.

[0049] In this application, p1, p2, ..., p n represents the ground position of each human unit on the ground, S = {p1, p2, ..., p n} represents the set of all ground positions, ε represents the first threshold, then:

[0050] R p ={(p i ,p j )∈S×S|‖p i ,-p j ‖<ε} (3)

[0051] Among them, ‖p i ,-p j ‖ represents the Euclidean distance between two ground locations, (p i ,p j )∈R p It expresses the ground position p i and ground position p j If the distance between them is less than ε, they belong to the same cluster.

[0052] Through the above clustering process, the set S is divided into k non-overlapping subsets {C1, C2, …, C k} (i.e., preliminarily obtain all human units corresponding to the same person target in all scene images), where any two points in each subset satisfy R p , while the points between different subsets do not meet this condition. Each subset can be expressed as follows:

[0053]

[0054] C k The ground position p and C k The Euclidean distance between any other ground positions p' in is less than ε.

[0055] like Figure 4In a preferred embodiment of the present invention, between step S4 and step S5, it also includes comparing the appearance features of the human body units corresponding to the same human target in all the scene pictures initially obtained, and accurately obtaining the human body units corresponding to the same human target. Due to the existence of occlusion and illumination change problems, the detection of human key points is not 100% accurate, which will lead to mismatching of human key points between different human targets. Therefore, after initializing clustering of all ground positions and preliminarily obtaining the human body units corresponding to the same human target, the appearance features of these human body units are compared again to detect whether there is any misjudgment, exclude abnormal human body units, and classify them into the correct human target, so that the 3D human posture can be estimated more accurately.

[0056] In a preferred embodiment of the present invention, when comparing the appearance features of human body units, a threshold-based comparison method is used to exclude human body units whose Euclidean distances with other human body units are greater than a second threshold, and the excluded human body units are compared with human body units corresponding to other human targets, and are classified as correct human targets. The threshold-based comparison method is simple and intuitive to implement, and has low computational complexity.

[0057] In a preferred embodiment of the present invention, the appearance feature is a Reid feature. The object of Reid feature research is the entire characteristics of a person, including clothing, body shape, hairstyle, posture, etc., without using face recognition as a means, and there is no rigid requirement for the clarity, installation position, and angle of the shooting device.

[0058] In the present invention, FastReid detector is used to extract Reid features.

[0059] In this application, subset C is k The appearance characteristics of each human unit are Г k1 ,G k2 ,…,Г km , then C k ={G k1 ,G k2 ,…,Г km} shows subset C k The appearance feature set of all human units in the scene is expressed as:

[0060] R Г ={(Г ku, G kv )∈C k ×C k |‖Г ku -G kv ‖<τ} (5)

[0061] Here, Gkm的 The subscripts k and m indicate that it is in the kth cluster and the mth appearance feature. ku -G kv ‖ represents the Euclidean distance between two appearance features, and τ represents the second threshold. The human body units that do not meet the second threshold are excluded, and then the excluded human body units or human body units that are not classified (cannot be classified into any human body target) during clustering are classified into the correct human body target by voting based on the appearance features, and the human body unit set R corresponding to the same human body target is accurately obtained. Г .

[0062] Preferably, the second threshold is preferably 0.1.

[0063] like Figure 5 In a preferred embodiment of the present invention, in step S5, according to the human body key points of the human body units corresponding to the same human target in each scene picture, combined with the position and internal parameters of the shooting device, the 3D position of the human body key points is calculated by triangulation to construct the 3D human body posture. The triangulation method is fast and accurate in calculation, can handle occlusion and complex background, and reduce recognition errors through algorithm optimization.

[0064] In this application, the triangulation method is used to calculate the 3D position of the human body key points. Specifically, the preliminary 3D position is first calculated based on the corresponding human body key points on any two human body units of the same human target, and then the reprojection error of each preliminary 3D position in all scene images is compared to find the one with the smallest reprojection error as the 3D position of the human body key point.

[0065] In a preferred embodiment of the present invention, in step S5, the 3D position is optimized by minimizing the reprojection error using the bundle adjustment method, and then the 3D human posture is constructed. Each 3D position of the calculated human target is optimized by minimizing the reprojection error, and the value that minimizes the comprehensive error between its back projection in all scene images and all corresponding human key points is obtained as the optimized 3D position, and then the 3D human posture is constructed accordingly.

[0066] Assume that each person target has N human body key points and M camera devices, and the parameters of each camera device (including internal and external parameters) are P j , each 3D position X i The corresponding human key point in the jth camera is x ij Our goal is to minimize the reprojection error of all 3D positions in all cameras, that is, to minimize the loss function L(P,X), which is expressed as follows:

[0067]

[0068] The percentage of correctly estimated parts (PCP) of the method of the present invention is verified using the Campus dataset and the Shelf dataset, wherein the Campus dataset consists of three people interacting outdoors and is captured by three calibrated cameras. The dataset contains a total of 2000 frames, and the validation sets are 350-470 frames and 650-750 frames respectively; the Shelf dataset consists of four people disassembling a shelf in a small indoor area, which is captured by five cameras, but each view is severely occluded. The dataset contains a total of 3200 frames, of which the 300-600 frames are validation sets; when verifying this solution, the backend data processing device is a laptop computer equipped with an Intel i9 2.20GHz CPU and an NVIDIA RTX4060Laptop GPU, and the shooting device is 3 realsense cameras, two of which are mounted on the ceiling, facing downward toward the center, and one is mounted on a bracket for horizontal observation, running at 30FPS and 1920x720 resolution. In other embodiments, the number of shooting devices can be 2 or more.

[0069] Through ablation experiments on the Campus and Shelf datasets, the effectiveness of appearance feature comparison and minimization of reprojection error optimization in this method is verified. For comparison, we removed the processing steps of appearance feature comparison or minimization of reprojection error optimization and performed 3D human pose estimation. The specific results of the percentage of correctly estimated parts (PCP) are shown in Table 1.

[0070] Campus Character Target 0 Character Target 1 Character Target 2 average No Appearance 97.71 92.45 97.87 96.01 NO BA 93.54 90.16 97.72 93.81 This method 98.54 92.77 97.94 96.42

[0071] Shelf Character Target 0 Character Target 1 Character Target 2 average No Appearance 98.42 92.97 97.20 96.20 NO BA 99.03 92.16 97.52 96.24 This method 98.8 94.1 97.8 96.9

[0072] Table 1: Ablation experiments on the Campus dataset and Shelf dataset.

[0073] No Appearance means that appearance feature comparison is not used to post-process the human body units corresponding to the same human target in all the initially obtained scene images, and NO BA means that the minimization of reprojection error optimization is not performed when constructing 3D human posture.

[0074] Experiments show that when appearance feature comparison is performed (data corresponding to the NO BA row), matching can achieve better performance and can reduce matching errors to a certain extent.

[0075] When minimizing the reprojection error optimization (data corresponding to the No Appearance row), we set the threshold for minimizing the reprojection error optimization to 0.3. When the number of cameras in the scene is relatively small, such as in the Campus dataset (containing 3 cameras), and the dataset has a low resolution, which reduces the accuracy of triangulation reconstruction of 3D human poses, minimizing the reprojection error optimization can significantly improve performance; in the Shelf dataset with a large number of cameras, since it can be observed from more perspectives, even without minimizing the reprojection error optimization, it can still achieve good performance, but for the case of frequent occlusion such as human target 2, adding the minimization of the reprojection error optimization can bring significant performance improvements, so minimizing the reprojection error optimization still helps to improve the accuracy of 3D human pose construction.

[0076] like Figure 6 The present invention also provides an estimation system for running any of the above-mentioned fast human posture estimation methods, including a shooting device for acquiring scene images, a front-end data processing device for identifying human units in each scene image and obtaining bounding boxes and human key points, and a back-end data processing device for completing 3D human posture estimation. The shooting device, the front-end data processing device, and the back-end data processing device work together, each performing its own duties, and finally quickly obtain a 3D human posture.

[0077] In a preferred embodiment of the present invention, the back-end data processing device is connected to the camera device for communication. Whenever the back-end data processing device sends a request instruction to the camera device, each camera device obtains a frame of the scene picture of the performance venue. Compared with the current processing method, each camera device sends a video stream with a timestamp to the data processing center in real time, and the data processing center then screens the synchronization frame by comparing the timestamp. The solution of the prior art will lead to redundant transmission of data, which puts a certain pressure on the communication bandwidth of the data processing center, and the time synchronization of each camera device is not easy to maintain. The present application adopts a processing method in which the camera device feeds back the scene picture according to the request instruction of the back-end data processing device, which can effectively solve the problems of redundant transmission and difficult time synchronization, can achieve millisecond-level synchronization accuracy, and significantly improve the efficiency and accuracy of data synchronization. In a preferred embodiment of the present invention, the back-end data processing device asynchronously sends a request instruction to each camera device. After the request instruction is issued, there is no need to wait for the current camera device to return data, and the request instruction can continue to be sent to the next camera device, which speeds up the data acquisition speed of the back-end data processing device.

[0078] In a preferred embodiment of the present invention, the number of front-end data processing devices corresponds to the number of shooting devices, and multiple front-end data processing devices are connected to a back-end data processing device. Each front-end data processing device processes the scene pictures sent by the shooting device connected to it, identifies the human body units in each scene picture in advance and obtains the bounding box and human body key points, and then sends the data to the back-end data processing device, which can effectively reduce the performance requirements of the back-end data processing device and speed up the data processing speed.

[0079] In a preferred embodiment of the present invention, the back-end data processing device has multiple threads. When calculating the 3D positions of the key points of the human body of the same person target, the task is divided into multiple parts and processed in parallel by multiple threads. Compared with the prior art, in which a single thread calculates the 3D position of each key point of the human body one by one, by setting multiple threads in the back-end data processing device and processing the 3D positions of multiple key points of the human body in parallel, the speed of data processing can be accelerated.

[0080] At present, although the floating-point computing power of the front-end data processing device used in this method is only about one-tenth of the floating-point computing power in existing papers, since the number of front-end data processing devices corresponds to the number of shooting devices, each front-end data processing device processes the scene pictures sent by the shooting device connected to it, identifies the human body units in each scene picture in advance and obtains the bounding box and human body key points, and then sends the data to the back-end data processing device to construct a 3D human body posture. Therefore, the speed of identifying human body key points is comparable to that in existing papers. It can be seen that the use of multiple front-end data processing devices can significantly improve the speed of accelerating the recognition of human body key points.

[0081] In the process of constructing 3D human poses, we significantly improved the reconstruction speed through multi-threaded parallel processing. On the Campus dataset, our method achieved a construction speed of 2.3ms, while on the Shelf dataset, the construction speed reached 4ms. This shows that our multi-threaded parallel processing has a significant advantage in constructing 3D human poses.

[0082] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A fast human posture estimation method, characterized in that: The following steps are involved: S1. Using multiple shooting devices to simultaneously obtain scene images of the performance venue from multiple perspectives; S2, identify the human body units in each scene image and obtain the bounding box and human body key points; S3, according to the bounding box or human key points of the human body unit in each scene image, deriving the ground position of the human body unit on the ground of the performance venue; S4, initializing the ground position of each human body unit in all scene images and clustering them, and preliminarily obtaining each human body unit corresponding to the same person target in all scene images; S5. Based on the human body key points of the human body units corresponding to the same human target in each scene image, the 3D positions of the human body key points are calculated to construct a 3D human body posture.

2. The fast human posture estimation method according to claim 1, characterized in that: In step S3, the ground position of the human body unit on the ground of the performance venue is calculated according to the midpoint of the lower boundary of the bounding box.

3. The fast human posture estimation method according to claim 1, characterized in that: In step S4, the clustering is initialized using a threshold-based clustering method, and the distances between all ground positions in the same category are smaller than a first threshold.

4. The fast human posture estimation method according to claim 3, characterized in that: The first threshold is 0cm-5cm.

5. The fast human posture estimation method according to claim 1, characterized in that: Between step S4 and step S5, it also includes comparing the appearance features of the human body units corresponding to the same human target in all the scene pictures initially obtained, so as to accurately obtain the human body units corresponding to the same human target.

6. The fast human posture estimation method according to claim 5, characterized in that: When comparing the appearance features of human body units, a threshold-based comparison method is used to exclude human body units whose Euclidean distance to other human body units is greater than a second threshold.

7. The fast human posture estimation method according to claim 5, characterized in that: The appearance features are Reid features.

8. The fast human posture estimation method according to claim 1, characterized in that: In step S5, according to the human body key points of the human body units corresponding to the same human target in each scene image, combined with the posture and internal parameters of the shooting device, the triangulation method is used to calculate the 3D position of the human body key points and construct the 3D human body posture.

9. The fast human posture estimation method according to claim 1, characterized in that: In step S5, the 3D position is optimized by minimizing the reprojection error using the bundle adjustment method, and then the 3D human posture is constructed.

10. An estimation system, characterized in that The fast human posture estimation method according to any one of claims 1 to 9 includes a shooting device for acquiring scene images, a front-end data processing device for identifying human body units in each scene image and obtaining bounding boxes and human body key points, and a back-end data processing device for completing 3D human posture estimation.

11. The estimation system according to claim 10, characterized in that The back-end data processing device is in communication connection with the shooting device. Whenever the back-end data processing device sends a request instruction to the shooting device, each shooting device obtains a frame of scene picture of the performance venue.

12. The estimation system according to claim 11, characterized in that The back-end data processing device asynchronously sends request instructions to each shooting device.

13. The estimation system according to claim 10, characterized in that The number of front-end data processing devices corresponds to the number of shooting devices, and multiple front-end data processing devices are commonly connected to a back-end data processing device.

14. The estimation system according to claim 10, characterized in that The back-end data processing device has multiple threads. When calculating the 3D positions of the key points of the human body of the same human target, the task is divided into multiple parts and processed in parallel by multiple threads.

Citation Information

Patent Citations

  • Multi-person 3D human body posture estimation method, device and equipment based on single image

    CN114495164A

  • Recovery machine built-in camera shooting control method, device and system

    CN114945071A

  • Multi-view-angle-based human body positioning method and lighting system

    CN116758267A

  • Methods and apparatus for real-time interactive performances

    WO2024000362A1

  • Human body pose recognition method and device

    WO2024114500A1

Cited By

  • Finger joint point labeling method and device and electronic equipment

    CN120747965A

  • Finger joint marking method and device thereof, and electronic device

    CN120747965B