Monocular three-dimensional positioning method and device combining physical geometric priori and multi-scale feature fusion network

Abstract

This application discloses a monocular 3D localization method and apparatus that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network, relating to the field of computer vision. The method includes: firstly, constructing a physically interpretable geometric depth baseline using keypoints; then, jointly inputting multi-scale appearance features and geometric features into a lightweight regression network to predict depth residuals; and finally, correcting the geometric depth baseline using these residuals. This fully utilizes the complementary information of pedestrian appearance and geometric priors to achieve precise 3D localization of the pedestrian target. Furthermore, a hierarchical degradation strategy is introduced to address long-tail problems such as pose loss.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a monocular 3D localization method and apparatus that combines Phy-MSFF with physical geometric priors and multi-scale feature fusion networks. Background Technology

[0002] With the continuous development of artificial intelligence and intelligent manufacturing, vision-based environmental perception systems are playing an increasingly important role in fields such as autonomous driving and intelligent robots. Among numerous perception solutions, monocular vision is particularly attractive in resource-constrained or structurally limited applications due to its advantages such as low hardware cost, flexible deployment, small size, and ease of large-scale deployment. However, monocular cameras lack depth information, and how to achieve accurate 3D localization of dynamic targets (especially pedestrian targets) relying solely on 2D images remains an important research direction in the field of computer vision.

[0003] Existing monocular 3D localization methods often exhibit a polarized trend. Traditional purely geometric methods rely on the flat terrain assumption and the geometric relationship of the bounding box. Although computationally simple, they are overly sensitive to pixel noise in the detection box and changes in human pose. While a large number of monocular 3D localization methods based on Transformer, complex end-to-end architecture, and temporal modeling have emerged in recent years, the model inference process lacks physical interpretability and is difficult to deploy in real time on edge devices with limited computing power and power consumption. Summary of the Invention

[0004] Therefore, it is necessary to provide a monocular 3D localization method and device that combines Phy-MSFF with physical geometric priors and multi-scale feature fusion networks to address the above-mentioned technical problems, so as to achieve stable and accurate pedestrian target localization under resource-constrained conditions.

[0005] Firstly, this application provides a monocular 3D localization method that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network. This method includes:

[0006] Collect images including pedestrian targets, with a pre-defined torso height;

[0007] Pose estimation is performed on the image, human body key points and bounding boxes of pedestrian targets are extracted, the torso scale is obtained based on the human body key points, the geometric depth baseline is obtained based on the torso scale and torso height prior, and geometric features are constructed based on the geometric depth baseline.

[0008] Multi-scale appearance features of pedestrian targets are extracted from images using a multi-scale feature fusion network. The multi-scale appearance features are then fused with geometric features to obtain fused features. The depth residual is then obtained based on the fused features.

[0009] The reliability of the pose estimation and multi-scale feature fusion network is evaluated. Based on the evaluation results, depth estimates are dynamically obtained from the geometric depth baseline, depth residual, and bounding box. Based on the depth estimates and bounding boxes, the 3D localization of the pedestrian target is obtained.

[0010] In one embodiment, key points of the human body include the left shoulder, right shoulder, left hip, and right hip;

[0011] Obtaining trunk dimensions based on key human body points includes:

[0012] The center point of the shoulder is obtained based on the coordinates of the left and right shoulders, the center point of the hip is obtained based on the coordinates of the left and right hips, and the trunk dimensions are obtained based on the distance between the center points of the shoulder and the center points of the hips.

[0013] In one embodiment, obtaining the geometric depth baseline based on prior torso scale and torso height includes:

[0014] Obtain camera intrinsic parameters, including focal length and principal point coordinates;

[0015] Based on the principal point coordinates, the center coordinates of the bounding box, and the focal length, obtain the horizontal and vertical viewing angles.

[0016] Based on the prior values ​​of torso scale, torso height, and focal length, the initial value of geometric depth is obtained;

[0017] The initial value of geometric depth is corrected by using the longitudinal viewing angle to obtain the geometric depth baseline.

[0018] In one embodiment, the geometric features include a geometric depth baseline, torso scale, average confidence of keypoints, lateral view skew, longitudinal view skew, and pixel offset of the bounding box center coordinates relative to the principal point coordinates.

[0019] Multi-scale appearance features are fused with geometric features to obtain fused features. The depth residual is then obtained based on these fused features, including:

[0020] Multi-scale appearance features are spliced ​​together along the channel dimension and then concatenated with geometric features to obtain fused features;

[0021] The fused features are input into a regression head composed of fully connected layers to obtain deep residuals.

[0022] In one embodiment, evaluating the reliability of the pose estimation and multi-scale feature fusion network includes:

[0023] The default initial assessment result is at the first level of confidence.

[0024] When invalid multi-scale appearance features or anomalies exist in multi-scale feature fusion network inference, the evaluation is at level two confidence level.

[0025] When the confidence level of a key point is lower than the confidence threshold or the trunk scale is lower than the trunk scale threshold, it is assessed as a level three confidence level.

[0026] In one embodiment, dynamically obtaining a depth estimate from the geometric depth baseline, depth residual, and bounding box based on the evaluation results includes:

[0027] When the assessment result is at the first level of confidence, the depth estimate is the sum of the geometric depth baseline and the depth residual;

[0028] When the assessment result is at the second confidence level, the depth estimate is the geometric depth baseline;

[0029] When the evaluation result is a level 3 confidence level, the depth estimate is obtained from the bounding box.

[0030] Secondly, this application also provides a monocular 3D localization device that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network. The device includes:

[0031] The data input module is used to acquire images including pedestrian targets, with a preset torso height as a prior.

[0032] The physical baseline and residual learning module is used to perform pose estimation on images, extract human key points and bounding boxes of pedestrian targets, obtain the torso scale based on human key points, obtain the geometric depth baseline based on the torso scale and torso height prior, and construct geometric features based on the geometric depth baseline; a multi-scale feature fusion network is used to extract multi-scale appearance features of pedestrian targets from images, fuse multi-scale appearance features with geometric features to obtain fused features, and obtain depth residuals based on fused features.

[0033] The hierarchical degradation strategy module is used to evaluate the reliability of the pose estimation and multi-scale feature fusion network. Based on the evaluation results, the depth estimate is dynamically obtained from the geometric depth baseline, depth residual and bounding box, and the 3D localization of the pedestrian target is obtained based on the depth estimate and bounding box.

[0034] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described monocular 3D localization method combining Phy-MSFF with physical geometric priors and a multi-scale feature fusion network.

[0035] Fourthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in the above-described monocular 3D localization method combining Phy-MSFF with physical geometric priors and a multi-scale feature fusion network.

[0036] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in the above-described monocular 3D localization method combining Phy-MSFF with physical geometric priors and a multi-scale feature fusion network.

[0037] The aforementioned Phy-MSFF monocular 3D localization method and device, which combines physical geometric priors with multi-scale feature fusion networks, first constructs a physically interpretable geometric depth baseline using key points. Then, it inputs multi-scale appearance features and geometric features into a lightweight regression network to predict depth residuals and correct the geometric depth baseline residuals. This fully utilizes the complementary information of pedestrian appearance and geometric priors to achieve precise 3D localization of pedestrian targets. In addition, a hierarchical degradation strategy is introduced to address long-tail problems such as pose loss. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating a monocular 3D localization method that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network in one embodiment.

[0039] Figure 2 The BEV plane error distribution of the Phy-MSFF model in different distance intervals is shown.

[0040] Figure 3 Figures illustrating the qualitative results of monocular 3D positioning at different distance ranges;

[0041] Figure 4 The Recall-EDE curve was plotted using a multi-working-point scanning method;

[0042] Figure 5 This is a block diagram of a monocular 3D localization device that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] This application provides a monocular 3D localization method that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network, such as... Figure 1 As shown, it includes the following steps:

[0045] S101. Collect images including pedestrian targets, with a preset torso height as a prior.

[0046] By using cameras set up on the roadside or mounted on vehicles, road traffic images are captured, including images of pedestrians.

[0047] Because adult heights vary considerably, directly setting a pre-defined total height can lead to significant errors. The torso has a smaller coefficient of variation across a population, making it a more robust reference point. Based on ergonomic data, a pre-defined torso height is established as a priori. rice.

[0048] During the data collection process, if the system has preliminary ranging capabilities (such as through the assumption of ground contact points), it can dynamically correct the prior value to make it more consistent with the actual individuals captured.

[0049] S102. Perform pose estimation on the image, extract human body key points and bounding boxes of pedestrian targets, obtain the torso scale based on human body key points, obtain the geometric depth baseline based on the torso scale and torso height prior, and construct geometric features based on the geometric depth baseline.

[0050] In the imaging principle of a pinhole camera model, the pixel height of the target object in the image... With real physical height Camera focal length and object distance Perspective projection relationship exists:

[0051]

[0052] Therefore, in order to calculate the object distance This refers to geometric depth. In this embodiment, it is necessary to obtain the camera intrinsic parameters and the torso scale in the image.

[0053] First, the camera intrinsic parameter K can be obtained using the traditional checkerboard calibration method, expressed as:

[0054]

[0055] in, , These are the focal lengths in the horizontal and vertical directions, respectively. The coordinates of the main point.

[0056] Secondly, the trained YOLOv8 pedestrian detector is used to detect the entire image, retaining only person-type targets to obtain bounding boxes. Then, the YOLO-Pose algorithm is run to obtain pose information containing 17 key points. In this embodiment, the key points for the left shoulder, right shoulder, left hip, and right hip are extracted from this information. Assuming the coordinates of the left and right shoulders are... , Left and right hip coordinates are , The center point of the shoulder is obtained based on the coordinates of the left and right shoulders, the center point of the hip is obtained based on the coordinates of the left and right hips, and the trunk dimensions are obtained based on the distance between the center points of the shoulder and the center points of the hips.

[0057] Since the torso of a pedestrian is in a fixed vertical posture during walking, in one embodiment, the torso dimensions can be determined by the vertical coordinates of the center points of the shoulders and hips, as follows:

[0058]

[0059] Further, the pixel height of the corresponding target torso can be obtained, i.e., the torso scale. :

[0060]

[0061] Ideally, the projection height and depth of the target torso in the image should satisfy the following relationship:

[0062]

[0063] This is the initial value for the geometric depth.

[0064] Considering that the location of pedestrian targets often deviates from the camera's optical axis, resulting in a viewing angle between the imaging ray and the vertical direction, this invention further introduces longitudinal viewing angle correction to reduce the system deviation caused by the viewing angle. Let the coordinates of the bounding box center point be... Then the vertical deflection angle can be obtained as:

[0065]

[0066] Subsequently, the introduction As a correction factor, the geometric depth baseline after viewpoint correction is obtained:

[0067]

[0068] In addition, although geometric depth baseline While primarily relying on longitudinal perspective correction, the lateral perspective also has a potential impact on spatial distortion. Therefore, this invention simultaneously calculates the lateral perspective deflection angle:

[0069]

[0070] The present invention obtained Based on this, a 7-dimensional geometric eigenvector is further constructed:

[0071]

[0072] in, The average confidence level is for four key human body points: left shoulder, right shoulder, left hip, and right hip. , These are the coordinates of the bounding box center relative to the principal point. Pixel offset.

[0073] In Before being input into the network, the data is normalized through pre-fitting offline to improve the stability of the regression.

[0074] The geometric depth baseline can provide a physically interpretable absolute scale benchmark, which allows subsequent operations to focus primarily on compensating for systematic biases, thereby utilizing geometric priors to constrain the extent of the solution space.

[0075] S103. Use a multi-scale feature fusion network to extract multi-scale appearance features of pedestrian targets from images, fuse multi-scale appearance features with geometric features to obtain fused features, and obtain depth residuals based on fused features.

[0076] If the final depth estimate is obtained solely through analytical geometric derivation, it is difficult to withstand the influence of complex factors such as modeling occlusion, pose changes, keypoint drift, and residual biases in 2D bounding boxes. To address these issues, this invention further introduces the multi-scale feature fusion network MSFF-Net on top of the geometric depth baseline, enabling data-driven repair of the depth estimate.

[0077] For each valid pedestrian target, a torso patch is cropped from the original image with the torso as the region center, and then scaled into three image patches of different scales: a high-resolution image patch of 64×128, a medium-resolution image patch of 32×64, and a low-resolution image patch of 16×32.

[0078] Image patches at three different scales are subjected to uniform normalization and convolutional feature extraction to obtain multi-scale appearance features. , , .

[0079] After multi-scale appearance features are stitched together along the channel dimension, they are combined with geometric features. The layers are cascaded to form the final fused feature vector, which is then input into a regression head consisting of several fully connected layers to output the depth residual. :

[0080]

[0081] Multi-scale appearance features can capture the target's occlusion method, body posture and surrounding context information, and compensate for geometric calculation errors caused by missing key points or small offsets of the torso boundary through implicit learning.

[0082] S104. Evaluate the reliability of the pose estimation and multi-scale feature fusion network. Based on the evaluation results, dynamically obtain depth estimates from the geometric depth baseline, depth residual, and bounding box. Based on the depth estimates and bounding box, obtain the 3D localization of the pedestrian target.

[0083] Considering that occlusion, pose ambiguity, or network uncertainty may lead to the failure of a single branch in practical applications, this invention designs a hierarchical fallback strategy, which divides the output into three-dimensional confidence tiers based on the integrity and reliability of the information:

[0084] Tier 1: When the target keypoints are complete, the network input is valid, and the inference confidence is high, the result of superimposing the physical baseline and the network residuals is used. This is the system's optimal output, balancing physical scale and appearance detail. At this point, depth estimation... for:

[0085]

[0086] Tier 2: When the input image patch of MSFF-Net is invalid (e.g., the cropped region is out of bounds or has an area of ​​0) or the network output variance is too large, the system automatically "downgrades" by discarding residual terms and retaining only the physical baseline. At this point, it degenerates into pure torso geometry localization, which, although lacking detailed corrections, still possesses physical interpretability. At this stage, depth estimation... for:

[0087]

[0088] Tier 3: When target pose estimation fails, the system performs a "safety net" operation, estimating depth using the bounding box height. This is the minimum guarantee of the system, in order to ensure the highest possible detection rate of the target. For example, if the confidence level of the shoulder or hip joint is lower than the confidence threshold (e.g., 0.4) or the trunk scale is lower than the trunk scale threshold, the target pose estimation can be considered to have failed.

[0089] In one embodiment, if the attitude estimation is deemed unreliable in step S2, step S3 can be skipped and the process can proceed directly to step S4, triggering Tier 3 and ensuring the bounding box geometry.

[0090] The final depth estimate is obtained using the method described above. Then, based on the pinhole camera model, the two-dimensional detection center can be... By back-projecting onto the camera coordinate system, the final 3D coordinates (X, Y, Z) can be obtained:

[0091]

[0092] This invention utilizes a monocular 3D localization model (Phy-MSFF, Physics - Multi-Scale Feature Fusion) that combines physical geometry priors with a multi-scale feature fusion network. This model borrows ideas from MSFF but limits them to a local area of ​​the torso to reduce redundancy. At the same time, it uses pose geometry to establish a physical baseline, simplifying the network task to "centimeter-level" residual correction.

[0093] In one embodiment, the complete single-frame inference process of the Phy-MSFF model is as follows:

[0094] S201. Two-dimensional detection and pose estimation. First, the YOLOv8x pedestrian detector is used to detect the entire image, retaining only person-type targets. Then, the YOLO-Pose algorithm is run to obtain pose information containing 17 key points.

[0095] S202, Keypoint Matching and Torso Region Extraction. The pedestrian's pose bounding box and detection box are matched according to the principle of maximizing IoU. If the maximum IoU is less than a threshold or no pose candidate exists, the detection is considered to have failed to match a reliable pose, and the process jumps directly to S204 to execute the Tier 3 safety net strategy. For successfully matched samples, keypoints of the left and right shoulders and left and right hips are extracted to determine the upper and lower boundaries of the torso. Based on this, the torso is expanded outwards according to a preset ratio based on its pixel height to include contextual information, and finally, the corresponding torso region is cropped from the original image.

[0096] S203. Geometric Depth Estimation Based on Scale Prior. Based on the pixel positions of keypoints at the shoulders and hips, the torso pixel height is calculated. This is combined with the prior knowledge of the true torso height and camera intrinsic parameter matrix information. The vertical offset of the detection box center on the image plane is used to estimate the longitudinal viewing angle. The geometric depth is then corrected for viewing angle using a cosine function to obtain the geometric depth baseline. And construct geometric feature vectors containing geometric information. .

[0097] S204 and MSFF-Net depth residual regression. The obtained Torso region is scaled into image patches of three scales to extract multi-scale appearance features of the target object. These features are then stitched together with geometric features and input into MSFF-Net for depth residual regression. If the network inference fails, execute the Tier 1 branch to overlay the depth residual onto the geometric depth baseline; if the network inference fails, execute the Tier 2 branch, which degenerates to outputting only the geometric depth baseline.

[0098] S205. Tier-based Degradation Strategy and Backprojection. Based on the pose, appearance, and reliability of network inference, Tier 1 (network correction), Tier 2 (pose geometry estimation), and Tier 3 (bounding box geometry baseline) are dynamically selected as the output paths for the final depth estimation. Finally, the 2D center is backprojected into 3D coordinates using camera intrinsic parameters. .

[0099] This invention employs a two-stream architecture of "physical geometry + network data". In the geometry stream, a pose estimator extracts human keypoints, which, after matching and filtering, are used to calculate a physically interpretable geometric depth baseline. In the network stream, MSFF-Net is used to extract multi-scale features of the torso, and the geometric feature vectors are fused, focusing on regressing the depth residual between the initial geometric value and the true depth. The final depth estimate is obtained by superimposing both data, achieving fine-grained correction for system biases.

[0100] Furthermore, this invention proposes a reliability-based hierarchical degradation strategy. Considering long-tail problems such as occlusion and pose estimation failure in complex scenes, this invention designs a three-level fallback mechanism at the output end. The system establishes a multi-dimensional reliability criterion including keypoint confidence, ROI (Region of Interest) cropping validity, and network inference state, and dynamically selects the optimal output level based on the criteria: in ideal cases, the fusion result is used first (Tier 1); when the network prediction is unstable, it falls back to the initial pose geometry value (Tier 2); in extreme cases, the system automatically degrades to the geometric estimation based on the detection box height (Tier 3), thereby ensuring the robustness of the system in all scenes.

[0101] To verify the advancement of the method provided in this invention, the monocular 3D localization performance of the Phy-MSFF model was evaluated based on the KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) benchmark experiment.

[0102] This experiment uses the Pedestrian class from the KITTI object detection benchmark for evaluation. The 7481 training images were divided into a training set (Train, 3712 images) and a validation set (Val, 3769 images). To avoid data leakage, all model training only used the Train subset, and the final results were reported on the Val subset. Based on the task requirements, this invention establishes two complementary evaluation protocols:

[0103] 1. Protocol A (Main Assessment): 0-20m Core Area Subset

[0104] On the Pedestrian subset of the KITTI dataset, samples with depth Z ≤ 20m, occlusion level ≤ 2, and truncation degree ≤ 0.5 were selected. This setting aims to evaluate the lower bound of the algorithm's performance under non-ideal observation conditions (i.e., corresponding to Hard level visibility) within the core interactive area of ​​vehicle perception.

[0105] 2. Protocol B (Supplementary Assessment): KITTI Official Difficulty Rating

[0106] Following the official KITTI definition, the dataset is divided into three levels—Easy, Moderate, and Hard—based on bounding box height, occlusion, and truncation levels, serving as a benchmark for comparison with existing works across the entire distance range.

[0107] This experiment follows the general protocol for monocular positioning, specifically the following metrics:

[0108] 1. Average depth error: measures the absolute distance deviation along the camera's optical axis;

[0109] 2. BEV (Browser Equivalent Vehicle) Planar Error: Measures the deviation of a target from its Euclidean distance on the bird's-eye view (XZ) plane;

[0110] 3. Three-dimensional Euclidean distance error: measures the overall straight-line distance between the predicted point and the true point in three-dimensional space.

[0111] In addition, three-dimensional nearest neighbor matching with a threshold of 2.0m was used to calculate the sample recall rate in order to evaluate the system's target capture and coverage capabilities in complex scenarios.

[0112] To adapt to the vehicle-mounted perspective of the KITTI data and enhance adaptability to complex scenes, the front-end detector of the 3D localization module uses YOLOv8x and is jointly trained using a hybrid dataset based on "WiderPerson + CityPersons + KITTI". By mixing and sampling urban street view and vehicle-mounted perspective data, the input resolution is increased to 1280×1280 to maximize the model's generalization ability under different scales and occlusion conditions. Pose estimation uses the officially pre-trained YOLOv8x-pose-p6.

[0113] After data preparation, experimental verification was conducted. First, the coverage of 2D detection and pose estimation on a pedestrian subset within 20m was examined. Then, the role of geometric modeling and MSFF-Net in the depth estimation module was evaluated through ablation experiments. The error distribution of the final model in different distance intervals was further analyzed. Finally, the hierarchical degradation strategy was verified, and the trade-off between accuracy and recall was discussed, along with existing related work.

[0114] The 0-20m pedestrian subset selected from the KITTI validation set contains 1603 Pedestrian-class ground truth (GT) targets. The YOLOv8x pedestrian detector, jointly trained with Widerperson, Citypersons, and KITTI, covers 1047 GT targets on this subset with an IoU ≥ 0.5 matching criterion, resulting in a recall of approximately 0.65. Statistical analysis across distance intervals shows that the detection recall is highest in the 5-10m range (0.76), decreasing significantly in the 10-20m range (0.59), indicating that the coverage limit of the end-to-end 3D localization link is more easily limited by missed detections in 2D detection at medium to long distances.

[0115] Regarding pose coverage, this invention associates the pose bounding boxes and detection boxes output by YOLOv8x-pose-p6 according to the principle of maximum IoU (det↔pose IoU ≥ 0.5), and further applies minimum confidence constraints (0.4) and torso pixel height constraints (5px) to four key points: left and right shoulders and left and right hips, thereby ensuring the reliability of torso geometric modeling. Under the above settings, a total of 656 samples satisfying the strict pose set conditions can be obtained, with a corresponding coverage of approximately 0.41. Under the condition of "detection hit GT", the condition coverage of the strict pose branch (Strict / Tier-1) is approximately 63%. After further enabling the graded degradation strategy, the number of samples that can output 3D results increases to 1047, with a corresponding Fallback|det = 1.000 (of which approximately 391 samples are downgraded to the Bbox-Geom branch due to pose unavailability). Overall, detection recall determines the upper limit of end-to-end coverage, while the tiered degradation strategy mainly improves the effective output coverage under the "detection hit condition", especially in the 10-20 m range, it significantly alleviates the sample loss caused by pose unavailability.

[0116] To better quantify the role of each component in the depth estimation module, this invention allows for configuration switching of the depth estimation model's structure, while maintaining the same detection and pose front-end. The overall performance of the three configurations on a 0-20m pedestrian subset is shown in Table 1.

[0117] Table 1. Comparison of ablation experimental results of each component of the depth estimation module.

[0118]

[0119] Among them, Bbox-Geom is a method that relies solely on bounding box geometry, while Pose-Geom is a method that incorporates pose geometry on top of Bbox-Geom. Matched GTs (number of ground truth matches / proportion) represents the number or proportion of targets that successfully match the ground truth annotations in the prediction results; Recall represents the recall rate; Depth MAE / m (Mean Absolute Depth Error, in meters); EDE MAE (Mean Absolute Edge Depth Error, in meters); EDE Median (Median Edge Depth Error, in meters); BEV MAE (Mean Absolute Error of Bird's-eye View, in meters); BEV Median (Median Error of Bird's-eye View, in meters).

[0120] As shown in Table 1, although Bbox-Geom, as the baseline method, achieved a relatively high recall rate (0.63) on the 0-20 meter pedestrian subset, indicating that it can cover a large proportion of targets relying solely on the geometric information of the detection boxes, Bbox-Geom is susceptible to detection box jitter and deformation, resulting in a significant upper limit to its error level.

[0121] Building upon this, Pose-Geom further introduces geometric constraints on keypoints of the human torso to mitigate the systematic bias of Bbox-Geom under occlusion and scale variation conditions. Overall, Pose-Geom shows some improvement in mean error metrics, with BEV MAE decreasing from 0.7061m to 0.6872m and DED MAE decreasing from 0.7123m to 0.7037. These data indicate that pose geometry can provide more physically meaningful depth constraints in some samples. It should be noted that this scheme employs gating for pose integrity and keypoint quality, thus reducing coverage accordingly. Simultaneously, the median error of Pose-Geom did not decrease synchronously, reflecting that in scenarios involving long distances, occlusion, or unstable keypoint localization, keypoint noise and human scale preset deviations may introduce additional uncertainties, leading to fluctuations in the error distribution of typical samples.

[0122] To further improve localization accuracy on strictly gated samples, Phy-MSFF integrates multi-scale appearance features and regression depth residuals to compensate for systematic errors in physical priors. Compared to Pose-Geom, its Depth MAE is further reduced by approximately 0.12m, with a final BEV MAE of 0.5558m and an EDE Median of only 0.4470. This significant performance improvement demonstrates that the "physical prior guidance + residual learning" strategy successfully uncovers subtle features in images, achieving more stable sub-meter-level high-precision localization on the core sample set. To address the recall bottleneck caused by strict geometric constraints, a tiered degradation strategy is employed for targeted optimization.

[0123] Next, we analyze the 3D error distribution characteristics of Phy-MSFF in different distance ranges and its variation with distance. Phy-MSFF successfully matched 664 ground truths (GTs) of pedestrians within a 20m pedestrian evaluation subset, corresponding to a recall rate of 0.40. In terms of overall metrics, Phy-MSFF's Depth MAE is 0.5122m, EDE MAE is 0.5715m, and EDE Median is 0.4470m; meanwhile, BEV MAE is 0.5558m, and BEV Median is 0.4362m. It can be seen that its errors in planar localization and overall 3D localization are on the same order of magnitude, and the Median is significantly lower than the MAE, indicating that the error is mainly amplified by a small number of long-tailed samples. Given the systematic bias between the center of the KITTI pedestrian bounding box and the pedestrian's true centroid, and the significant differences in individual height, this invention primarily uses depth error and BEV planar error as indicators to evaluate 3D localization performance in subsequent analyses.

[0124] To more intuitively demonstrate the trend of error change with target distance, Figure 2 The BEV error statistics for the three GT depth ranges are presented. It should be noted that... Figure 2Only Tier 1 samples that pass attitude gating and enter the Phy-MSFF residual correction branch are statistically analyzed. This analysis aims to characterize the error patterns of the model on the core sample set where "attitude information is available." As shown in the figure, in the 0-5m close-range segment, the BEV MAE is approximately 0.356m, and the BEV Median is only 0.367m, indicating that the model can achieve planar positioning accuracy on the order of 30cm for close-range pedestrians. In the 5-10m medium-range segment, the BEV MAE is approximately 0.500m, and the BEV Median is approximately 0.410m, with the overall error still controlled at around 50cm. In the 10-20m relatively long-range segment, the BEV MAE increases to approximately 0.744m, and the BEV Median is approximately 0.676m, reflecting that the monocular method is affected by factors such as decreased resolution, severe occlusion, and attitude instability in long-range scenes, resulting in a certain increase in error, but the overall error still remains within the sub-meter range.

[0125] also, Figure 3 It provides frame-by-frame visualization results for key areas ranging from 5 to 20 meters. Figure 3 The image shows the qualitative results of monocular 3D localization at different distances, demonstrating the frame-by-frame localization performance in scenes of 5-10m (top two rows) and 10-20m (bottom two rows). The data in the image are labeled as the BEV plane localization error for the target. It can be observed that in the 5-10m range, the model achieves near-perfect accuracy in locating pedestrians; in the 10-20m range, despite challenges such as reduced pixel ratio and texture blurring, Phy-MSFF still maintains a controllable error level on most samples.

[0126] In summary, Phy-MSFF exhibits relatively stable error distribution characteristics within the near-to-medium distance range of 0-20m. On Tier 1 samples, the BEV Median in the 5-10m range can be stably controlled at around 0.4m; however, in the 10-20m range, the BEV Median increases to 0.676m, demonstrating the impact of increasing distance on monocular 3D localization accuracy. Considering the overall error differences, this indicates that Phy-MSFF can provide high-precision planar localization on most samples, while there is still room for improvement on challenging samples such as those from long distances, those with occlusion, and those with unstable attitudes. This provides direction for further improvements in keypoint quality and gating robustness.

[0127] Considering that pose estimation and multi-scale clipping may fail locally in practical applications, simply treating these targets as invalid would significantly reduce the coverage of the 3D localization module. To address this issue, this invention further designs a hierarchical degradation strategy based on Phy-MSFF: when both the torso pose and clipping are valid and MSFF-Net inference is normal, Phy-MSFF is used as the final depth estimate; when the clipped region is invalid or MSFF-Net inference is abnormal, it degenerates to using only the initial torso geometry as the final depth estimate; when the target fails to match a reliable torso pose, it further degenerates to a simple geometric depth based on the bounding box height and camera intrinsics. Only when all of the above processing methods are ineffective are samples considered invalid.

[0128] Table 2 compares the overall performance of Phy-MSFF and the system after introducing the hierarchical degradation strategy on the 0-20m subset. It can be seen that after introducing the hierarchical degradation strategy, the number of matching ground truths increased from 664 to 1051, and the recall rate improved by more than 57%. This phenomenon indicates that the fallback branch can effectively take over samples in scenarios such as unavailable poses, failed pruning, and inference anomalies, enabling the system to provide 3D localization results for a larger proportion of targets.

[0129] Table 2. Impact of graded degradation strategy on 3D localization accuracy and recall.

[0130]

[0131] Meanwhile, after introducing the tiered degradation strategy, Depth MAE increased from 0.5122m to 0.6582m, EDE MAE from 0.5715m to 0.7146m, BEV MAE from 0.5558m to 0.7029m, and BEV Median from 0.4362m to 0.5277m. While there was some loss in accuracy, most metrics still outperformed Bbox-Geom. This phenomenon can be explained: the newly added samples mainly come from more difficult instances, which rely more on geometric backoff branches when attitude and appearance information is insufficient, thus increasing the overall error, but still remaining within an acceptable range for engineering applications. Overall, the tiered degradation strategy provides a trade-off between coverage and accuracy that better meets the needs of practical systems.

[0132] To clarify the source of the coverage bottleneck, this invention further introduces Oracle Boxes as an upper bound diagnostic, directly inputting the ground truth box into the backend to evaluate the depth estimation module's capability under "ideal detection input." Experiments show that under Oracle Boxes conditions, the system can achieve a matching recall of 1.0, while the error remains at a reasonable level (DepthMAE: 0.6601m, EDE MAE: 0.7629m, BEV MAE: 0.7477m). This indicates that the main factors limiting end-to-end coverage stem from front-end detection and pose availability, rather than insufficient backend depth regression capabilities. Based on these conclusions, the tiered degradation strategy, by providing a robust backoff path under unstable front-end conditions, effectively alleviates the coverage decline problem, providing a clear direction for further improving front-end quality or optimizing backoff branches.

[0133] The above experimental analysis is mainly based on the evaluation subset for near-to-medium distance scenarios. That is, in the original KITTI pedestrian annotation, only targets that meet the conditions of both truncation and occlusion and are within a forward distance of 20m are retained, in order to focus on the typical application scope of this invention.

[0134] To facilitate indirect comparison between this method and existing monocular pedestrian 3D localization methods under a unified protocol, this invention further supplements the evaluation of the Easy, Moderate, and Hard subsets of the Pedestrian category, while fully adhering to the official KITTI difficulty classification. Specifically, no additional distance upper limit is applied; instead, samples are divided into three difficulty levels—Easy, Moderate, and Hard—based solely on the target bounding box width, occlusion level, and truncation degree specified by the official KITTI classification. Depth MAE, EDE MAE, and BEV MAE are calculated for each difficulty level. Simultaneously, to reflect the usability of the end-to-end system, this invention reports recall for each difficulty subset to characterize the overall trade-off between accuracy and coverage.

[0135] As shown in Table 3, under the official KITTI difficulty settings, the EDE MAE of Phy-MSFF on the Easy, Moderate, and Hard subsets are 0.5449m, 0.5545m, and 0.5707m, respectively, with corresponding recall rates of 0.45, 0.36, and 0.30. It can be seen that the recall rate gradually decreases with increasing difficulty, indicating that the end-to-end coverage is significantly constrained by the front-end detector and pose availability. However, on samples where pose and geometric validity meet the gating conditions, the EDE MAE remains stable in the range of 0.5–0.6 m, demonstrating the stable accuracy of the depth estimation backend of this invention on core available samples.

[0136] Table 3. Performance evaluation of 3D localization under KITTI's official difficulty rating

[0137]

[0138] After further introducing a tiered degradation strategy, the recall rates for Easy / Moderate / Hard improved to 0.78, 0.69, and 0.59, respectively, resulting in a significant increase in end-to-end coverage. Simultaneously, EDE MAE increased to 0.6792m, 0.8309m, and 0.8659m, respectively. These phenomena indicate that the fallback branch can effectively take over difficult samples with unavailable pose or insufficient appearance information, thereby improving coverage. However, because the newly added samples contain a higher proportion of distant, occluded, and low-resolution targets, the overall system error is increased, but it still remains at the sub-meter level under the official three difficulty levels. Overall, Phy-MSFF (Fallback) achieves a more "engineering-oriented" trade-off under the standard evaluation protocol—trading a certain degree of accuracy for more stable output.

[0139] In the research direction of monocular pedestrian 3D localization, the MonoLoco series of works has formed a relatively typical technical route and evaluation paradigm. MonoLoco uses 2D human keypoints as input, combines camera imaging geometry and height statistical priors, estimates the 3D position of pedestrians in the camera coordinate system, and provides uncertainty modeling. At the same time, from the task perspective, it points out that depth estimation under monocular conditions has inherent fuzziness, making it difficult to compress the average error to an extremely low level in complex road scenes. MonoLoco++ further improves the localization performance by improving the network structure and representation method. The above studies collectively show that the end-to-end performance of 3D localization tasks depends not only on the lower bound of the back-end localization error, but also on the visibility and sample coverage of the 2D detection front-end under difficult conditions.

[0140] To provide an intuitive understanding of the error magnitudes of different methods, Table 4 summarizes the results of representative works and the method provided by this invention under the KITTI difficulty level. It should be noted that existing works may differ in aspects such as front-end detection / pose models, training data, threshold selection, and evaluation protocols. Therefore, Table 4 is primarily used as a reference to present error magnitudes and trends, and is not a strict numerical alignment under the same protocol.

[0141] Table 4 Comparison results between Phy-MSFF and MonoLoco series

[0142]

[0143] Both the method of this invention (Ours) and the reproduced MonoLoco++-style were evaluated within the KITTI official difficulty standard (inclusive) and the range of 0–20m, and a strict 1-to-1 matching strategy was adopted; the error metric was the mean localization error (ALE, m). The values ​​in parentheses represent the recall rate on the corresponding subset: where Phy-MSFF is the high-confidence sample that passed the reliability check, Fallback is the end-to-end output of the entire system, and MonoLoco++-style is the sample with valid pose input. Other literature data are directly cited from the original papers, and due to differences in front-end detectors and evaluation protocols, they are for qualitative reference only and not for rigorous horizontal comparison.

[0144] Under the same front-end conditions, this invention further constructs a MonoLoco++-style back-end as a control method. Compared with Phy-MSFF, it only replaces the back-end localization module while maintaining the same detector, the same keypoint front-end, and consistent matching and statistical methods, thus allowing for a more direct comparison of the impact of different back-end strategies on localization error. Control experiments show that, under the same front-end input, the method of this invention can consistently achieve lower localization errors at multiple operating points.

[0145] The aforementioned advantages do not rely on the accidental result of a specific threshold point. Under the same two-dimensional front-end and consistent matching rules, this invention constructs a MonoLoco++-style back-end as a comparison, and uses a multi-working-point scanning method to draw the following... Figure 4 The Recall-EDE curve is shown. Figure 4 This section compares the R-EDE curves at different detection confidence thresholds (<=20m, one-to-one matching). Figure 4 (a) EDE MAE as a function of recall_det; Figure 4(b) EDE Median variation with recall_det. Both methods use the same front-end and matching protocol, only replacing the back-end positioning module. Specifically, by scanning different det_conf (0.05, 0.08, 0.10, 0.12, 0.15, 0.18, 0.20, 0.25, 0.30), recall_det varies from 0.6957 to 0.6123 within this range. The results show that within the same recall_det working range, the EDE MAE of the complete system of this invention is approximately 0.7012-0.7616m, and the EDE Median is approximately 0.5329-0.5499m; while the EDE MAE of the MonoLoco++-style control is approximately 1.2867-1.3371m, and the EDE Median is approximately 1.0360-1.0753m. The two differences exist stably at multiple operating points, and the overall curves show a consistent advantage relationship, which further illustrates that the error reduction of the method of the present invention does not depend on the randomness of a specific threshold point, but has better positioning performance throughout the entire process of coverage change.

[0146] It should be noted that the purpose of this control experiment is to compare the relative differences between different backends under the same frontend conditions, rather than to reproduce the optimal absolute values ​​of MonoLoco++ under specific pose frontend and evaluation settings in the literature. Since the keypoint frontend used in this invention is YOLOv8x-pose-p6, which differs from the PifPaf used in the literature in terms of keypoint distribution, noise morphology, and visibility statistics, the backend regression head is more sensitive to changes in input distribution. A shift in the absolute error magnitude is expected, but under strict comparison with the frontend's controlled variables, the backend of this invention can still consistently achieve lower EDE errors, thus more directly verifying the effectiveness of the "torso physical geometry modeling + MSFF residual correction + graded degradation mechanism".

[0147] Furthermore, the system design of this invention emphasizes a "trade-off of interpretable accuracy and coverage." When keypoints and geometric conditions meet reliability constraints, the system prioritizes outputting high-precision results generated by the Phy-MSFF branch. When pose is missing or geometry is unstable, the system uses a tiered degradation strategy to backtrack to a more robust geometric estimation branch to increase the range of output samples, thereby ensuring end-to-end output continuity. It should be noted that, under the current front-end output distribution and gating threshold settings, the intermediate layer backtracking branch (Tier2) is triggered less frequently, which is consistent with its design goal of "conservative effectiveness condition": Tier2 is used to cover boundary samples where keypoint quality is in the middle range but still has a certain degree of geometric observability; when keypoint quality further deteriorates, the system tends to backtrack directly to the bounding box geometry branch to ensure stable output. This phenomenon does not affect the conclusions of this invention regarding the effectiveness of the tiered mechanism, but rather reflects the clear division of labor of the gate control strategy in different input quality ranges.

[0148] Therefore, the relationship between this invention and existing work can be summarized in the following two points: First, under strict comparison with the same front end, the method of this invention consistently reduces 3D Euclidean localization error at multiple operating points, demonstrating the effectiveness of the back-end modeling and residual learning strategies; second, from the perspective of system links, it reveals that end-to-end coverage is mainly constrained by detection recall, keypoint visibility, and instance alignment quality, and constructs an interpretable "accuracy-coverage" trade-off mechanism through a hierarchical degradation strategy. The key direction for further improving end-to-end performance in the future lies in enhancing the visibility and stability of front-end detection and keypoint estimation under difficult conditions, and the method of this invention can serve as a stable back-end localization module, providing a lower error and more controllable engineering trade-off for the overall system after front-end enhancement.

[0149] In summary, this invention effectively improves detection consistency and positioning accuracy when relying solely on a monocular camera.

[0150] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this invention, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages of other steps.

[0151] Based on the same inventive concept, this application also provides a monocular 3D localization device that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the monocular 3D localization device that combine Phy-MSFF with physical geometric priors and a multi-scale feature fusion network provided below can be found in the limitations of the monocular 3D localization method that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network described above, and will not be repeated here.

[0152] In one embodiment, such as Figure 5 As shown, a monocular 3D localization device combining Phy-MSFF physical geometric priors and multi-scale feature fusion networks is provided, comprising:

[0153] The data input module is used to acquire images including pedestrian targets, with a preset torso height as a prior.

[0154] The physical baseline and residual learning module is used to perform pose estimation on images, extract human key points and bounding boxes of pedestrian targets, obtain the torso scale based on human key points, obtain the geometric depth baseline based on the torso scale and torso height prior, and construct geometric features based on the geometric depth baseline; a multi-scale feature fusion network is used to extract multi-scale appearance features of pedestrian targets from images, fuse multi-scale appearance features with geometric features to obtain fused features, and obtain depth residuals based on fused features.

[0155] The hierarchical degradation strategy module is used to evaluate the reliability of the pose estimation and multi-scale feature fusion network. Based on the evaluation results, the depth estimate is dynamically obtained from the geometric depth baseline, depth residual and bounding box, and the 3D localization of the pedestrian target is obtained based on the depth estimate and bounding box.

[0156] The modules in the aforementioned monocular 3D localization device, which combines physical geometric priors with a multi-scale feature fusion network using Phy-MSFF, can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0157] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in all of the above method embodiments.

[0158] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in all of the above method embodiments.

[0159] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in all of the above method embodiments.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0161] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0162] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A monocular 3D localization method combining Phy-MSFF with physical geometric priors and multi-scale feature fusion networks, characterized in that, The method includes: Collect images including pedestrian targets, with a pre-defined torso height; Pose estimation is performed on the image, human body key points and bounding boxes of the pedestrian target are extracted, the torso scale is obtained based on the human body key points, the geometric depth baseline is obtained based on the torso scale and torso height prior, and geometric features are constructed based on the geometric depth baseline. Multi-scale appearance features of the pedestrian target are extracted from the image using a multi-scale feature fusion network. The multi-scale appearance features are then fused with the geometric features to obtain fused features. The depth residual is then obtained based on the fused features. The reliability of the pose estimation and the multi-scale feature fusion network is evaluated. Based on the evaluation results, the depth estimate is dynamically obtained from the geometric depth baseline, the depth residual and the bounding box. Based on the depth estimate and the bounding box, the 3D localization of the pedestrian target is obtained.

2. The method according to claim 1, characterized in that: The key points of the human body include the left shoulder, right shoulder, left hip, and right hip; The process of obtaining trunk dimensions based on the key human body points includes: The center point of the shoulder is obtained based on the coordinates of the left and right shoulders, the center point of the hip is obtained based on the coordinates of the left and right hips, and the trunk dimensions are obtained based on the distance between the center points of the shoulder and the center points of the hips.

3. The method according to claim 1, characterized in that, The process of obtaining the geometric depth baseline based on the prior knowledge of the torso scale and torso height includes: Obtain camera intrinsic parameters, including focal length and principal point coordinates; Based on the principal point coordinates, the center coordinates of the bounding box, and the focal length, obtain the lateral and longitudinal viewing angles. Based on the aforementioned torso dimensions, torso height prior, and focal length, the initial value of geometric depth is obtained; The initial value of the geometric depth is corrected using the longitudinal viewing angle to obtain the geometric depth baseline.

4. The method according to claim 3, characterized in that: The geometric features include the geometric depth baseline, the torso scale, the average confidence of the keypoints, the lateral view angle, the longitudinal view angle, and the pixel offset of the center coordinates of the bounding box relative to the principal point coordinates.

5. The method according to claim 1, characterized in that, The step of fusing the multi-scale appearance features with the geometric features to obtain fused features, and obtaining the depth residual based on the fused features, includes: The multi-scale appearance features are spliced ​​together along the channel dimension and then concatenated with the geometric features to obtain the fused features; The fused features are input into a regression head composed of fully connected layers to obtain the depth residual.

6. The method according to claim 1, characterized in that, The evaluation of the reliability of the pose estimation and the multi-scale feature fusion network includes: The default initial assessment result is at the first level of confidence. When there are invalid multi-scale appearance features or anomalies in the multi-scale feature fusion network inference, the evaluation is at level two confidence level. When the confidence level of the key point is lower than the confidence threshold or the trunk scale is lower than the trunk scale threshold, it is evaluated as a level three confidence level.

7. The method according to claim 6, characterized in that, The process of dynamically obtaining depth estimates from the geometric depth baseline, the depth residual, and the bounding box based on the evaluation results includes: When the evaluation result is at the first level of confidence, the depth estimate is the sum of the geometric depth baseline and the depth residual; When the evaluation result is at the second confidence level, the depth estimate is the geometric depth baseline; When the evaluation result is a level 3 confidence level, the depth estimation is obtained based on the bounding box.

8. A monocular 3D localization device that combines Phy-MSFF with physical geometric priors and a multi-scale feature fusion network, characterized in that, The device includes: The data input module is used to acquire images including pedestrian targets, with a preset torso height as a prior. The physical baseline and residual learning module is used to perform pose estimation on the image, extract human key points and bounding boxes of the pedestrian target, obtain the torso scale based on the human key points, obtain the geometric depth baseline based on the torso scale and torso height prior, construct geometric features based on the geometric depth baseline, extract multi-scale appearance features of the pedestrian target from the image using a multi-scale feature fusion network, fuse the multi-scale appearance features with the geometric features to obtain fused features, and obtain depth residuals based on the fused features. The hierarchical degradation strategy module is used to evaluate the reliability of the pose estimation and the multi-scale feature fusion network. Based on the evaluation results, it dynamically obtains the depth estimate from the geometric depth baseline, the depth residual and the bounding box, and obtains the 3D localization of the pedestrian target based on the depth estimate and the bounding box.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images