Panoramic human body posture estimation and evaluation method and system and computer device

Abstract

The invention relates to the technical field of computer vision, in particular to a panoramic human body posture estimation and evaluation method and system and a computer device.The method comprises the steps that S1, an ERP image of a target human body is obtained, and the target human body is placed in the center of the image; s2, obtaining a distortionless projection image of the target human body through a dynamic panoramic image projection method; s3, calculating skeleton coordinate points of the target human body in the projection image through a planar human body posture estimation method; s4, projecting the skeleton coordinate point of the target human body back to the ERP image to obtain a final skeleton prediction point of the ERP image; and S5, calculating according to the evaluation data set to obtain a human body posture estimation evaluation index. According to the method, a new projection method is introduced, the distortion influence of an ERP image is effectively reduced, a plane attitude estimation method is more effectively adapted to a panoramic image, and a special data set for panoramic human body attitude estimation is also provided for verifying the effectiveness of the method.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a panoramic human pose estimation and evaluation method, system, and computer device. Background Technology

[0002] In recent decades, human pose estimation has become a core problem in the field of computer vision, aiming to accurately detect and locate various parts of the human body (usually represented as skeletal keypoints) from images or videos. Its technical capabilities have been widely applied in many fields such as human-computer interaction, motion analysis, virtual reality, security monitoring, and robotics. This technology has undergone a leapfrog development from manual feature extraction to human pose learning-driven approaches. Convolutional neural networks (CNNs) have promoted the robustness of feature extraction, and in recent years, large models based on the Transformer architecture have achieved higher-precision pose prediction by capturing long-range dependencies and global context. With the rapid development of panoramic camera technology, spherical multimedia data is increasingly widely used in fields such as military defense, virtual navigation, and entertainment (e.g., Google Street View uses billions of panoramic images to construct virtual geographic scenes). However, existing research on human pose estimation mainly focuses on perspective image scenes, and research on human pose estimation for panoramic images still has a significant gap. Panoramic images are commonly represented by equal rectangular projection (ERP), which has the advantage of a large field of view (capable of completely detecting everyone in the scene), but it has two major technical challenges: first, targets easily cross image boundaries; second, severe distortion easily occurs as latitude increases, causing pose estimation methods suitable for perspective images to experience a significant performance drop in panoramic scenes, failing to meet the needs of practical applications.

[0003] Especially in VR / AR interaction and panoramic security monitoring scenarios, accurate recognition of human posture directly impacts the interactive experience and monitoring effectiveness. The shortcomings of existing technologies have become a key bottleneck restricting the development of these fields. Currently, the demand for human posture analysis in panoramic images is increasingly urgent, but there is a lack of suitable technical solutions and standard evaluation datasets. Therefore, it is imperative to develop human posture estimation methods for panoramic scenes and construct dedicated datasets to address issues such as panoramic image distortion and target boundary crossing, thereby promoting the practical development of human understanding technology in the field of panoramic vision. Summary of the Invention

[0004] In view of this, the present invention proposes a panoramic human pose estimation and evaluation method, system and computer device, which aims to solve the problems existing in the current technology.

[0005] This invention proposes a panoramic human pose estimation and evaluation method, comprising the following steps: S1. Obtain the ERP image of the target human body and place the target human body in the center of the image; S2. Based on S1, obtain a distortion-free projection image of the target human body through a dynamic panoramic image projection method; S3. Calculate the skeletal coordinates of the target human body in the projected image in S2 using the planar human body pose estimation method. S4. Project the skeletal coordinates of the target human body in S3 back to the ERP image to obtain the final skeletal prediction points of the ERP image. S5. Based on the evaluation dataset, calculate the human posture estimation evaluation index.

[0006] Preferably, step S1 specifically includes: S11. Use the target detection algorithm to obtain the pixel coordinates (u,v) of the center point of the target human body in the ERP image; S12. Based on the mapping relationship between the ERP image and the virtual unit sphere, convert the pixel coordinates (u,v) of the target human body center point in S11 into spherical coordinates (α,β). S13. For the spherical coordinates (α,β) obtained in S12, construct the rotation matrix Ry around the vertical y-axis and the rotation matrix Rx around the horizontal x-axis, and perform rotation on the virtual unit sphere so that the spherical coordinates (α,β) of the target human body are mapped to the origin (0,0) of the spherical coordinate system. S14. Convert the rotated virtual unit sphere in S13 into a 2D ERP image through inverse mapping, denoted as ERProt image, so that the target person is located at the center of the ERProt image; S15. Repeat steps S11 to S12 for the ERProt image in S14 to obtain the spherical coordinates (cα, cβ) of the target human body center point in the ERProt image, as well as the maximum horizontal field of view θ and the maximum vertical field of view occupied by the target human body. And based on the maximum horizontal field of view θ and the maximum vertical field of view Define the target region Ω, and the specific calculation formula is as follows: ; Where W is the original width of the panoramic image; H is the original height of the panoramic image; w is the horizontal coordinate of the center point of the target human body on the panoramic image; and h is the vertical coordinate of the center point of the target human body on the panoramic image.

[0007] Preferably, step S2 specifically includes: S21. Based on the maximum horizontal field of view θ and maximum vertical field of view of the target human body obtained in S15. Dynamically select the appropriate projection sub-scheme: S22, when θ < 90 degrees and When the angle is less than 90 degrees, perspective projection is used to project the target region Ω onto the planar perspective image through the BFoV parameter to obtain the projected image; S23. When 90 degrees ≤ θ < 180 degrees and 90 degrees ≤ When the angle is less than 180 degrees, a disk projection is used to project the target area Ω onto a planar disk image, and then cropping is performed to obtain the projected image. S24, when the maximum horizontal field of view θ and the maximum vertical field of view θ If the conditions of steps S22 and S23 are not met, direct cropping is used to directly crop the original image blocks of the ERP rot image to obtain the projected image.

[0008] Preferably, the planar human pose estimation method in step S3 is the HRNet model, which calculates the coordinates (x, y) of the skeletal key points of the target human body on the projected image, i.e., the skeletal coordinate points.

[0009] Preferably, step S4 specifically includes: transforming the skeletal coordinate points (x, y) in S3 using the inverse transformation of the projection method in S2 to obtain the corresponding latitude and longitude coordinates (φ, θ) on the ERP image, which are the final skeletal prediction points.

[0010] Preferably, the target detection algorithm in step S11 is the YOLOv8 model.

[0011] Preferably, the scale of the target human body in step S2 is 1.2 times the scale of the target human body in step S1; the evaluation dataset in step S5 is the PanoPose panoramic image dataset.

[0012] Preferably, the human posture estimation evaluation index in step S5 includes the prediction accuracy of the target human body, and the calculation formula is as follows: ; in, For the accuracy of prediction of the target human body; S is the spherical distance between the predicted skeletal keypoints and their corresponding ground realities; bbox k is the square root of the area of ​​the target human detection bounding box on a virtual unit sphere; i A constant for controlling the decay at each skeletal key point; The total number of visible skeletal keypoints in the target human body; v i For the visibility of the ground view; i represents the index of the skeletal key points of the target human body; Define A as the predicted skeletal keypoint on the virtual unit sphere, i.e. the final predicted skeletal point, and B as the real skeletal keypoint on the virtual unit sphere. ; ; Calculate the dot product of OA and OB, and then use the inverse cosine function to calculate the distance between points A and B. The specific calculation formula is as follows: ; Where, φ gt The longitude of the actual skeletal key points, -π≤φ gt ≤π;φ pred To predict the longitude of key skeletal points, -π≤φ pred ≤π;θ gt The coordinates of the actual skeletal keypoints, -π / 2 ≤ θ gt ≤π / 2; θ pred To predict the latitude of skeletal key points, -π / 2 ≤ θ pred ≤π / 2; The scale of the target human body in the panoramic image is obtained by calculating the area of ​​each spherical rectangle, i.e., S. bbox The specific calculation formula is as follows: .

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention addresses the issue that traditional human pose estimation methods for perspective images cannot handle panoramic image distortion and target boundary crossing problems. It proposes a multi-scene projection strategy, specifically for normal scenes (maximum horizontal field of view θ < 90 degrees and maximum vertical field of view θ < 90 degrees). (When <90 degrees) The target position is characterized by BFoV, and the resolution dependency is eliminated by utilizing the spherical properties, thus resolving issues related to boundary crossings and distortion; the maximum horizontal field of view θ and the maximum vertical field of view are also considered. When both are within 90°-180°, a circular projection is used to balance the field of view and distortion; in other cases, such as the maximum horizontal field of view θ and the maximum vertical field of view θ, a circular projection is used to balance the field of view and distortion. When the angle exceeds 180°, the pre-processed rectangular projection (ERP) image is directly cropped to achieve full coverage adaptation of the panoramic scene, significantly reduce target distortion, and ensure the consistency of attitude estimation.

[0014] The framework of this invention can adapt mainstream perspective human pose estimation methods to panoramic scenes. By optimizing scale calculation through two-stage object detection and reducing distortion through precise projection clipping, these methods achieve an average performance improvement on the PanoPose dataset, especially in medium-to-high distortion scenes with a performance improvement of over 30%, while also showing improvement in low distortion scenes. It takes into account the generalization of scenes with different distortion levels and reduces the cost of technology migration.

[0015] On the other hand, this application also provides a panoramic human pose estimation and evaluation system, which applies the aforementioned panoramic human pose estimation and evaluation method, including: The ERP branch module is used to extract target human information from panoramic images; Projection module, used to eliminate distortion of the target human body; The calculation module is used to calculate the skeletal coordinates of the target human body and project them back into the ERP image to obtain the final skeletal prediction points. The evaluation module is used to evaluate the final prediction results based on the evaluation dataset.

[0016] Furthermore, this application also provides a computer device, comprising: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the panoramic human pose estimation and evaluation method.

[0017] It is understood that the panoramic human posture estimation and evaluation system and the computer device provided in this application have the same beneficial effects as the panoramic human posture estimation and evaluation method described above, and will not be repeated here. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a panoramic human pose estimation and evaluation method provided in an embodiment of the present invention; Figure 2 This is a result block diagram of a panoramic human pose estimation and evaluation system provided in an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Example 1 See Figure 1 This embodiment provides a panoramic human pose estimation and evaluation method, including the following steps: S1. Obtain the ERP image of the target human body and place the target human body in the center of the image; S2. Based on S1, obtain a distortion-free projection image of the target human body through a dynamic panoramic image projection method; S3. Calculate the skeletal coordinates of the target human body in the projected image in S2 using the planar human body pose estimation method. S4. Project the skeletal coordinates of the target human body in S3 back to the ERP image to obtain the final skeletal prediction points of the ERP image. S5. Based on the evaluation dataset, calculate the human posture estimation evaluation index.

[0021] Understandably, in step S1 of this embodiment, global spatial features of the target human body are extracted through a centering operation on the rotated ERP image. These features clearly define the overall position of the target human body in the panoramic sphere, avoiding the impact of initial position offset on subsequent projection accuracy. In step S2, local detail features of the target human body are extracted through a dynamic projection scheme. These features focus on low-distortion information in the human body region, providing fine texture and contour support for pose estimation. In step S3, the projected image is input into the planar human pose estimation model, and skeletal coordinate points are calculated by combining global and local features. The model uses feature correlation to accurately locate joints. In step S4, the skeletal coordinates are transmitted back to the ERP image through inverse projection mapping, and the spherical and pixel coordinate transformation logic is combined to ensure accurate coordinate correspondence. In step S5, evaluation metrics are calculated based on the PanoPose dataset. The performance is quantified by spherical distance and scale normalization to achieve objective evaluation of the panoramic pose estimation results.

[0022] Preferably, step S1 specifically includes: S11. Use the target detection algorithm to obtain the pixel coordinates (u,v) of the center point of the target human body in the ERP image.

[0023] Understandably, in step S11, the initial center point pixel coordinates (u,v) of the target human body are located through the target detection algorithm, providing a basis for subsequent spherical coordinate transformation.

[0024] S12. Based on the mapping relationship between the ERP image and the virtual unit sphere, convert the pixel coordinates (u,v) of the target human body center point in S11 into spherical coordinates (α,β) to ensure that subsequent rotation operations conform to the geometric characteristics of the sphere.

[0025] Understandably, in step S12, based on the fixed mapping relationship between the ERP image and the virtual unit sphere, the 2D pixel coordinates (u,v) obtained in step S11 are converted into longitude α and latitude β in the 3D spherical coordinate system to ensure that subsequent rotation operations conform to the spherical geometric characteristics.

[0026] S13. Construct rotation matrices and perform spherical rotation. For the spherical coordinates (α,β) obtained in S12, construct rotation matrix Ry around the vertical y-axis and rotation matrix Rx around the horizontal x-axis. Perform rotation on the virtual unit sphere to map the spherical coordinates (α,β) of the target human body to the origin (0,0) of the spherical coordinate system, thus eliminating the latitude and longitude offset of the target.

[0027] Specifically, the core objective of spherical rotation is to adjust the spherical coordinates (α, β) of the target human body to the origin (0, 0) of the spherical coordinate system. This requires two rotations: rotation around the y-axis to eliminate the longitude offset α, and rotation around the x-axis to eliminate the latitude offset β. The rotation matrix is ​​designed following the right-hand rule of 3D spatial rotation to ensure that the relative positions of the spherical points remain unchanged after rotation, with only the overall posture adjusted. Then, matrix operations are used to complete the rotation.

[0028] S14. Convert the rotated virtual unit sphere in S13 into a 2D ERP image through inverse mapping, denoted as ERProt image, so that the target person is located at the center of the ERProt image, thus initially solving the problem of the target crossing the left and right edges of the original ERP image.

[0029] S15. Repeat steps S11 to S12 for the ERProt image in S14 to obtain the spherical coordinates (cα, cβ) of the target human body center point in the ERProt image, as well as the maximum horizontal field of view θ and the maximum vertical field of view occupied by the target human body. And based on the maximum horizontal field of view θ and the maximum vertical field of view Define the target region Ω, and calculate it using the following formula, which is the conversion formula from the center point pixel coordinates to spherical coordinates: ; Where W is the original width of the panoramic image; H is the original height of the panoramic image; w is the horizontal coordinate of the center point of the target human body on the panoramic image; and h is the vertical coordinate of the center point of the target human body on the panoramic image.

[0030] Specifically, the ERP image is essentially a 2D planar image unfolded from a virtual unit sphere (radius r=1) using a rectangular projection method. Its horizontal direction corresponds to the longitude of the sphere (range [-π, π], corresponding to 360°), and its vertical direction corresponds to the latitude of the sphere (range [-π / 2, π / 2], corresponding to 180°). Assuming the ERP image size is 1024×512 pixels, with the center pixel coordinates being (512, 256), the converted longitude α=0 and latitude β=0, perfectly corresponding to the origin (0,0) of the spherical coordinate system, verifying the accuracy of the conversion formula.

[0031] Understandably, step S15 takes the new ERP image obtained in step S14, denoted as the ERP rot image, and performs target detection again to accurately locate the spherical parameters of the target human body, providing a high-accuracy input for the projection method in step S2.

[0032] Preferably, step S2 specifically includes: S21. Based on the maximum horizontal field of view θ and maximum vertical field of view of the target human body obtained in S15. Dynamically select the appropriate projection sub-scheme: S22, when θ < 90 degrees and When the angle is less than 90 degrees, use perspective projection to project the target area Ω onto the planar perspective image through the BFoV parameter to obtain the projected image; S23. When 90 degrees ≤ θ < 180 degrees and 90 degrees ≤ When the angle is less than 180 degrees, use a disk projection method to project the target area Ω onto a planar disk image, and then crop it to obtain the projected image. S24, when the maximum horizontal field of view θ and the maximum vertical field of view θ If the conditions of steps S22 and S23 are not met, direct cropping is used to directly crop the original image blocks of the ERP rot image to obtain the projected image.

[0033] Preferably, the planar human pose estimation method in step S3 is the HRNet model. The coordinates (x, y) of the skeletal key points of the target human body on the projected image are calculated by the HRNet model, that is, the skeletal coordinate points.

[0034] Preferably, step S4 specifically includes: transforming the skeletal coordinate points (x, y) in S3 using the inverse transformation of the projection method in S2 to obtain the corresponding latitude and longitude coordinates (φ, θ) on the ERP image, which are the final skeletal prediction points.

[0035] Preferably, the target detection algorithm in step S11 is the YOLOv8 model.

[0036] Preferably, the scale of the target human body in step S2 is 1.2 times the scale of the target human body in step S1; the evaluation dataset in step S5 is the PanoPose panoramic image dataset.

[0037] It is understood that the panoramic human pose estimation and evaluation method provided in this embodiment first achieves target centering and outputs accurate spherical parameters; then dynamically selects a projection scheme according to the field of view to obtain a distortion-free image; then uses a planar human pose estimator to calculate the human pose coordinates after projection processing, and then inversely maps them back to the original ERP image, thus solving the problem of low pose estimation accuracy caused by target distortion and edge crossing in ERP panoramic images.

[0038] Preferably, the human posture estimation evaluation index in step S5 includes the prediction accuracy of the target human body, and the calculation formula is as follows: ; in, For the accuracy of prediction of the target human body; S is the spherical distance between the predicted skeletal keypoints and their corresponding ground realities; bbox k is the square root of the area of ​​the target human detection bounding box on a virtual unit sphere; i A constant for controlling the decay at each skeletal key point; The total number of visible skeletal keypoints in the target human body; v i For ground-level visibility; i represents the index of the target human's skeletal key points, for example, nose is 0, eyes are 1, and left foot is 15; Define A as the predicted skeletal keypoint on the virtual unit sphere, i.e. the final predicted skeletal point, and B as the real skeletal keypoint on the virtual unit sphere. ; ; Calculate the dot product of OA and OB, and then use the inverse cosine function to calculate the distance between points A and B. The specific calculation formula is as follows: ; Where, φ gt The longitude of the actual skeletal key points, -π≤φ gt ≤π;φ pred To predict the longitude of key skeletal points, -π≤φ pred ≤π;θ gt The coordinates of the actual skeletal keypoints, -π / 2 ≤ θ gt ≤π / 2; θ pred To predict the latitude of skeletal key points, -π / 2 ≤ θ pred ≤π / 2; The scale of the target human body in the panoramic image is obtained by calculating the area of ​​each spherical rectangle, i.e., S. bbox The specific calculation formula is as follows: .

[0039] Calculated and After normalization and using a unit sphere as the unit, consistency and comparability are ensured across different panoramic scenes, and finally the prediction accuracy of the target human body is calculated.

[0040] Specifically, the human pose estimation evaluation index in this embodiment The higher the indicator value, the better the prediction effect.

[0041] Specifically, the PanoPose panoramic image dataset is a dataset specifically designed for panoramic human pose estimation, as proposed in this embodiment. This dataset contains 1500 images, covering various indoor and outdoor environments and multiple human poses. Each human instance in the dataset has been manually annotated with 17 skeletal keypoints.

[0042] Based on the PanoPose panoramic image dataset, spherical distance was used to calculate metrics such as AP (Action Capability) and AR (Average Representation). Real-world testing shows that AP is improved by over 30% in high-distortion scenes, providing a reliable technical approach for panoramic human pose estimation.

[0043] It is understood that the panoramic human pose estimation and evaluation method of the present invention helps to improve the efficiency and accuracy of feature extraction, and can be applied to many application fields such as robot vision, consumer electronics, security, autonomous driving, human-computer interaction, image retrieval, intelligent monitoring, augmented reality, and virtual reality. It also provides a foundation for the realization of more complex tasks such as semantic segmentation and scene understanding.

[0044] Example 2 Combination Figure 2 As shown, this embodiment provides a panoramic human pose estimation and evaluation system, and applies a panoramic human pose estimation and evaluation method, including: The ERP branch module is used to extract target human information from panoramic images; Projection module, used to eliminate distortion of the target human body; The calculation module is used to calculate the skeletal coordinates of the target human body and project them back into the ERP image to obtain the final skeletal prediction points. The evaluation module is used to evaluate the final prediction results based on the evaluation dataset.

[0045] It is understood that the panoramic human pose estimation and evaluation system provided in this embodiment has the same beneficial effects as the panoramic human pose estimation and evaluation method, and will not be described in detail here. Example 3

[0046] This embodiment provides a computer device, including: At least one processor; At least one memory for storing at least one program; When at least one program is executed by at least one processor, a panoramic human pose estimation and evaluation method is implemented by at least one processor.

[0047] It is understood that the computer device provided in this embodiment has the same beneficial effects as a panoramic human pose estimation and evaluation method, and will not be described in detail here.

[0048] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0049] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0050] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0051] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A panoramic human pose estimation and evaluation method, characterized in that, Includes the following steps: S1. Obtain the ERP image of the target human body and place the target human body in the center of the image; S2. Based on S1, obtain a distortion-free projection image of the target human body through a dynamic panoramic image projection method; S3. Calculate the skeletal coordinates of the target human body in the projected image in S2 using the planar human body pose estimation method. S4. Project the skeletal coordinates of the target human body in S3 back to the ERP image to obtain the final skeletal prediction points of the ERP image. S5. Based on the evaluation dataset, calculate the human posture estimation evaluation index.

2. The panoramic human pose estimation and evaluation method according to claim 1, characterized in that, Step S1 specifically includes: S11. Use the target detection algorithm to obtain the pixel coordinates (u,v) of the center point of the target human body in the ERP image; S12. Based on the mapping relationship between the ERP image and the virtual unit sphere, convert the pixel coordinates (u,v) of the target human body center point in S11 into spherical coordinates (α,β). S13. For the spherical coordinates (α,β) obtained in S12, construct the rotation matrix Ry around the vertical y-axis and the rotation matrix Rx around the horizontal x-axis, and perform rotation on the virtual unit sphere so that the spherical coordinates (α,β) of the target human body are mapped to the origin (0,0) of the spherical coordinate system. S14. Convert the rotated virtual unit sphere in S13 into a 2D ERP image through inverse mapping, denoted as the ERP rot image, so that the target person is located at the center of the ERP rot image; S15. Repeat steps S11 to S12 for the ERProt image in S14 to obtain the spherical coordinates (cα, cβ) of the target human body center point in the ERProt image, as well as the maximum horizontal field of view θ and the maximum vertical field of view occupied by the target human body. And based on the maximum horizontal field of view θ and the maximum vertical field of view Define the target region Ω, and the specific calculation formula is as follows: ； Where W is the original width of the panoramic image; H is the original height of the panoramic image; w is the horizontal coordinate of the center point of the target human body on the panoramic image; and h is the vertical coordinate of the center point of the target human body on the panoramic image.

3. The panoramic human pose estimation and evaluation method according to claim 2, characterized in that, Step S2 specifically includes: S21. Based on the maximum horizontal field of view θ and maximum vertical field of view of the target human body obtained in S15. Dynamically select the appropriate projection sub-scheme: S22, when θ < 90 degrees and When the angle is less than 90 degrees, perspective projection is used to project the target region Ω onto the planar perspective image through the BFoV parameter to obtain the projected image; S23. When 90 degrees ≤ θ < 180 degrees and 90 degrees ≤ When the angle is less than 180 degrees, a disk projection is used to project the target area Ω onto a planar disk image, and then cropping is performed to obtain the projected image. S24, when the maximum horizontal field of view θ and the maximum vertical field of view θ If the conditions of steps S22 and S23 are not met, direct cropping is used to directly crop the original image blocks of the ERP rot image to obtain the projected image.

4. The panoramic human pose estimation and evaluation method according to claim 1, characterized in that, The planar human pose estimation method described in step S3 is the HRNet model. The coordinates (x, y) of the skeletal key points of the target human body on the projected image are calculated by the HRNet model, which are the skeletal coordinate points.

5. The panoramic human pose estimation and evaluation method according to claim 4, characterized in that, Step S4 specifically includes: transforming the skeletal coordinate points (x, y) in S3 using the inverse transformation of the projection method in S2 to obtain the corresponding latitude and longitude coordinates (φ, θ) on the ERP image, which are the final skeletal prediction points.

6. The panoramic human pose estimation and evaluation method according to claim 2, characterized in that, The target detection algorithm in step S11 is the YOLOv8 model.

7. The panoramic human pose estimation and evaluation method according to claim 1, characterized in that, The scale of the target human body in step S2 is 1.2 times that of the target human body in step S1; the evaluation dataset in step S5 is the PanoPose panoramic image dataset.

8. The panoramic human pose estimation and evaluation method according to claim 5, characterized in that, The human pose estimation evaluation index in step S5 includes the prediction accuracy of the target human body, calculated using the following formula: ； in, For the accuracy of prediction of the target human body; S is the spherical distance between the predicted skeletal keypoints and their corresponding ground realities; bbox k is the square root of the area of ​​the target human detection bounding box on a virtual unit sphere; i A constant for controlling the decay at each skeletal key point; The total number of visible skeletal keypoints in the target human body; v i For the visibility of the ground view; i represents the index of the skeletal key points of the target human body; Define A as the predicted skeletal keypoint on the virtual unit sphere, i.e. the final predicted skeletal point, and B as the real skeletal keypoint on the virtual unit sphere. ； ； Calculate the dot product of OA and OB, and then use the inverse cosine function to calculate the distance between points A and B. The specific calculation formula is as follows: ； Where, φ gt The longitude of the actual skeletal key points, -π≤φ gt ≤π;φ pred To predict the longitude of key skeletal points, -π≤φ pred ≤π;θ gt The coordinates of the actual skeletal keypoints, -π / 2 ≤ θ gt ≤π / 2; θ pred To predict the latitude of skeletal key points, -π / 2 ≤ θ pred ≤π / 2; The scale of the target human body in the panoramic image is obtained by calculating the area of ​​each spherical rectangle, i.e., S. bbox The specific calculation formula is as follows: 。 9. A panoramic human pose estimation and evaluation system, characterized in that, The panoramic human pose estimation and evaluation method as described in any one of claims 1-8 includes: The ERP branch module is used to extract target human information from panoramic images; Projection module, used to eliminate distortion of the target human body; The calculation module is used to calculate the skeletal coordinates of the target human body and project them back into the ERP image to obtain the final skeletal prediction points. The evaluation module is used to evaluate the final prediction results based on the evaluation dataset.

10. A computer device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the panoramic human pose estimation and evaluation method according to any one of claims 1-8.

Images