Monocular 3D target detection method and system, electronic device and storage medium

Abstract

The present application relates to the technical field of computer vision, and specifically proposes a monocular 3D target detection method, system, electronic device and storage medium, wherein the monocular 3D target detection method forms an end-to-end, physics common sense driven monocular 3D detection framework by constructing parallel depth and height dual geometric feature encoders, designing a cascaded dual geometric guide decoder, introducing a differentiable geometric consistency design, and adopting an adaptive depth fusion mechanism with a stop gradient, thereby effectively solving the problems of lack of geometric consistency and serious scale drift in monocular 3D detection, significantly improving the perception accuracy and reliability of autonomous driving, and providing a new technical path for the deep fusion of deep learning and geometric vision.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a monocular 3D target detection method, system, electronic device, and storage medium. Background Technology

[0002] Autonomous driving technology has become a key pillar for revolutionizing the modern transportation system. The environmental perception module is regarded as the visual center of the autonomous driving system. Monocular 3D (3-dimensional) target detection is the core perception technology of the environmental perception module. By performing high-precision detection and spatiotemporal positioning of traffic participants, it has built the underlying foundation for the stable operation of autonomous driving.

[0003] However, in monocular 3D object detection, when a 3D scene is compressed to a 2D image plane, depth information is irreversibly lost. Countless 3D objects with different physical sizes and depths may produce identical 2D (2 Dimensions) projections on the imaging plane, resulting in inherent ambiguity in accurately recovering depth based solely on visual representation. This has become a key bottleneck affecting the performance of monocular 3D detection. Summary of the Invention

[0004] This invention aims to address at least the inherent ambiguity in accurately recovering depth solely from visual representations, a key bottleneck affecting the performance of monocular 3D detection in related technologies. Therefore, the purpose of this invention is to propose a monocular 3D target detection method, system, electronic device, and storage medium.

[0005] In a first aspect, the present invention proposes a monocular 3D target detection method, comprising the following steps: inputting a monocular input image acquired by a camera into a backbone network for feature extraction, generating a multi-scale visual feature map, and obtaining the intrinsic parameter matrix of the camera; based on the multi-scale visual feature map, generating a deep embedding feature map and a height embedding feature map respectively using parallel deep feature learning branches and height feature learning branches; and using a cascaded dual geometrically guided decoder, with the height embedding feature map, the deep embedding feature map, and the multi-scale visual feature map as interactive objects, sequentially updating the object query vector used to characterize the target to be detected through height-guided localization, depth refinement calibration, and visual feature extraction, generating a target carrying... An updated object query vector with target geometric and semantic information is generated. Based on the updated object query vector, the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar used to characterize the confidence uncertainty of the direct regression depth are regressed and predicted. The geometric depth is derived using the focal length parameter in the intrinsic parameter matrix and the pinhole imaging principle. The direct regression depth and the geometric depth are adaptively weighted and fused using the uncertainty scalar. A stopping gradient operation is applied to the two depths before fusion. The fused depth and the 3D spatial attributes predicted based on the updated object query vector are combined to output the 3D bounding box, thus obtaining the final 3D target detection result of the monocular input image.

[0006] According to an embodiment of the present invention, the monocular 3D target detection method first inputs the monocular input image acquired by the camera into the backbone network for feature extraction, generating a multi-scale visual feature map and obtaining the camera's intrinsic parameter matrix. Then, based on the multi-scale visual feature map, parallel deep feature learning branches and height feature learning branches are constructed to generate a depth embedding feature map and a height embedding feature map, respectively. Next, using a cascaded dual geometrically guided decoder, with the height embedding feature map, depth embedding feature map, and multi-scale visual feature map as interaction objects, the object query vector is sequentially updated through height-guided localization, depth refinement calibration, and visual feature extraction to generate an updated object query vector. Then, based on the updated object query vector, the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the predicted target are regressed, and the focal length parameter in the intrinsic parameter matrix is ​​combined with... The geometric depth is derived from the pinhole imaging principle. Finally, an uncertainty scalar is used to adaptively weight and fuse the direct regression depth and geometric depth. A stopping gradient operation is applied to the two depths before fusion. Combining the fused depth with the 3D spatial attributes predicted by the updated object query vector, a 3D bounding box is output to obtain the final 3D object detection result of the monocular input image. By constructing a parallel depth and height dual geometric feature encoder, designing a cascaded dual geometric guided decoder, introducing a differentiable geometric consistency design, and adopting an adaptive depth fusion mechanism with stopping gradient, an end-to-end, physics-driven monocular 3D detection framework is formed. This effectively solves the problems of lack of geometric consistency and severe scale drift in monocular 3D detection, significantly improves the perception accuracy and reliability of autonomous driving, and provides a new technical path for the deep integration of deep learning and geometric vision.

[0007] In addition, the monocular 3D target detection method according to embodiments of the present invention may also have the following additional technical features: Furthermore, the step of inputting the monocular input image acquired by the camera into the backbone network for feature extraction, generating a multi-scale visual feature map, and obtaining the intrinsic parameter matrix of the camera includes: acquiring a monocular RGB input image using the camera; inputting the monocular RGB input image into the backbone network for feature extraction, and using a feature pyramid network for multi-scale fusion to output a multi-scale visual feature map; reading the intrinsic parameter matrix of the camera and extracting the vertical focal length parameter from it.

[0008] Further, based on the multi-scale visual feature map, the generation of a deep embedding feature map and a height embedding feature map using parallel deep feature learning branches and height feature learning branches respectively includes: the deep feature learning branch uses a linearly increasing discretization strategy to divide the continuous depth interval into multiple non-uniform depth intervals, predicts the first probability distribution of each pixel belonging to each depth interval through a convolutional layer, and then uses a learnable depth prototype vector to perform a weighted summation of the first probability distribution to generate a deep embedding feature map; the height feature learning branch uses a uniform discretization strategy to divide the physical height range into multiple uniform intervals, predicts the second probability distribution of each pixel belonging to each height interval through a convolutional layer, then uses a learnable height prototype vector to perform a weighted summation of the second probability distribution, and superimposes the multi-scale visual feature map as a residual connection to generate a height embedding feature map.

[0009] Furthermore, the cascaded dual geometric guided decoder includes multiple cascaded decoder layers; the step of using the cascaded dual geometric guided decoder, with the high-embedded feature map, the deep-embedded feature map, and the multi-scale visual feature map as interaction objects, to sequentially update the object query vector representing the target to be detected through height-guided localization, depth refinement calibration, and visual feature extraction, generating an updated object query vector carrying target geometric and semantic information, includes: performing the following operations sequentially in each decoder layer of the cascaded dual geometric guided decoder: using the object query vector of the current decoder layer as the query, and using the high-embedded feature map... The feature map is used as the key and value for cross-attention calculation, and a set of height key indices is selected to achieve height-guided localization. The object query vector after height-guided localization is used as the query, and the deep embedding feature map is used as the key and value for cross-attention calculation, and a set of depth key indices is selected to achieve depth refinement calibration. The object query vector after depth refinement calibration is used as the query, and the multi-scale visual feature map is used as the key and value. The union of the set of height key indices and the set of depth key indices is used as the geometric index set. Cross-attention calculation is performed only on the region specified by the geometric index set to achieve visual feature extraction. The updated object query vector is output.

[0010] Further, the step of regressing and predicting the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar representing the confidence uncertainty of the direct regression depth based on the updated object query vector, and deriving the geometric depth using the focal length parameter in the intrinsic parameter matrix in conjunction with the pinhole imaging principle, includes: based on the updated object query vector, regressing and predicting the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the target through parallel regression of the detection head; using the vertical focal length parameter in the intrinsic parameter matrix, converting the ratio of the 3D physical height to the 2D bounding box height into geometric depth according to the method that the geometric depth is equal to the vertical focal length parameter multiplied by the 3D physical height divided by the 2D bounding box height.

[0011] Further, the step of adaptively weighting and fusing the direct regression depth and the geometric depth using the uncertainty scalar, applying a stopping gradient operation to the two depths before fusion, and combining the fused depth with the 3D spatial attributes predicted based on the updated object query vector to output a 3D bounding box includes: mapping the uncertainty scalar to generate dynamic gating weights; applying a stopping gradient operation to the direct regression depth and the geometric depth respectively; using the dynamic gating weights to perform a weighted summation of the two depths after the stopping gradient operation to obtain the final fused depth; combining the final fused depth with the 3D center point projection coordinates, 3D physical dimensions, and orientation yaw angle predicted based on the updated object query vector, combining the inverse solution of the intrinsic parameter matrix to obtain the absolute 3D center point, and then outputting a 3D bounding box after 3D nonmaximum suppression.

[0012] Furthermore, after deriving the geometric depth, the monocular 3D target detection method further includes: constructing a consistency loss function between the direct regression depth and the geometric depth, and adding the consistency loss function to the total loss for end-to-end error backpropagation.

[0013] Secondly, based on the same inventive concept, this invention also proposes a monocular 3D target detection system, comprising: a feature extraction module, used to input monocular input images acquired by a camera into a backbone network for feature extraction, generating multi-scale visual feature maps, and obtaining the intrinsic parameter matrix of the camera; a dual geometric feature encoding module, used to generate a depth embedding feature map and a height embedding feature map respectively based on the multi-scale visual feature maps using parallel deep feature learning branches and height feature learning branches; and a decoding module, used to utilize a cascaded dual geometrically guided decoder, with the height embedding feature map, the depth embedding feature map, and the multi-scale visual feature map as interactive objects, sequentially performing height-guided localization, depth refinement calibration, and visual feature extraction on the object query vector representing the target to be detected. The system progressively updates the object query vector, generating an updated object query vector carrying the geometric and semantic information of the target. A geometric consistency module is used to regress and predict the target's 2D bounding box height, 3D physical height, direct regression depth, and an uncertainty scalar representing the confidence uncertainty of the direct regression depth based on the updated object query vector. It also uses the focal length parameter in the intrinsic parameter matrix and the pinhole imaging principle to derive the geometric depth. A fusion output module is used to adaptively weight and fuse the direct regression depth and the geometric depth using the uncertainty scalar. A stopping gradient operation is applied to the two depth paths before fusion. Combining the fused depth with the 3D spatial attributes predicted by the updated object query vector, a 3D bounding box is output, resulting in the final 3D target detection result of the monocular input image.

[0014] The monocular 3D target detection system provided in the second aspect has the same technical features as the monocular 3D target detection method provided in the first aspect, and therefore also has similar technical effects as the first aspect, which will not be described in detail here.

[0015] Thirdly, based on the same inventive concept, the present invention also proposes an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store a computer program; and the processor is used to execute the computer program stored in the memory to implement the steps of the monocular 3D target detection method as described in the above embodiments of the present invention.

[0016] The electronic device provided in the third aspect has the same technical features as the monocular 3D target detection method provided in the first aspect, and therefore also has similar technical effects as the first aspect, which will not be described in detail here.

[0017] Fourthly, based on the same inventive concept, the present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the monocular 3D target detection method as described in the above embodiments of the present invention.

[0018] The computer-readable storage medium provided in the fourth aspect has the same technical features as the monocular 3D target detection method provided in the first aspect, and therefore also has similar technical effects as the first aspect, which will not be described in detail here. Attached Figure Description

[0019] Figure 1 It is based on the MonoDETR framework diagram in the existing technology; Figure 2 This is a diagram of an end-to-end dual geometric feature learning and detection framework according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the overall network architecture of an end-to-end dual geometric feature learning and detection framework according to an embodiment of the present invention; Figure 4 This is a flowchart of a monocular 3D target detection method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a parallel dual geometric feature encoding process according to a specific embodiment of the present invention; Figure 6 This is a comparison diagram of the discretized mapping curves of the deep LID strategy and the high UD strategy according to a specific embodiment of the present invention; Figure 7 This is a schematic diagram of the cascaded attention interaction mechanism inside a dual geometry-guided decoder according to a specific embodiment of the present invention; Figure 8 This is a schematic diagram of the gradient flow direction and gradient stopping strategy of a differentiable geometric consistency module according to a specific embodiment of the present invention. Figure 9 This is a schematic diagram comparing the degradation mitigation risk caused by not using the stopping gradient with the robust calibration process after using the stopping gradient, according to a specific embodiment of the present invention. Figure 10 This is a schematic diagram of the structure of an adaptive deep fusion module based on object query vectors according to a specific embodiment of the present invention; Figure 11 This is a schematic diagram illustrating the statistical distribution and trend analysis of the gating weight as a function of target distance according to a specific embodiment of the present invention; Figure 12 This is a structural block diagram of a monocular 3D target detection system according to an embodiment of the present invention; Figure 13This is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0020] Figure label: 100 - Monocular 3D target detection system; 110 - Feature extraction module; 120 - Dual geometric feature encoding module; 130 - Decoding module; 140 - Geometric consistency module; 150 - Fusion output module. Detailed Implementation

[0021] The embodiments of the present invention are described in detail below, and the embodiments described with reference to the accompanying drawings are exemplary.

[0022] Current mainstream monocular 3D object detection methods generally treat depth estimation as a purely data-driven regression task. The drawback is that they seriously ignore the strict geometric projection rules in the imaging process and fail to effectively utilize the strong geometric prior that the physical height of rigid objects (such as vehicles) in traffic scenes has extremely high intra-class stability. This lack of geometric logic makes it easy for the model to violate physical common sense when learning the statistical mapping between image features and depth values. Especially when dealing with distant or occluded targets, the phenomenon of "scale drift" frequently occurs - that is, the predicted depth contradicts the actual physical size of the object. This problem directly threatens the robustness and safety of autonomous driving perception systems and has become a key bottleneck restricting the performance of monocular 3D detection.

[0023] To address the above problems, this invention provides a monocular 3D target detection method, system, electronic device, and storage medium, as described below. Figures 1-13 This invention describes a monocular 3D target detection method, system, electronic device, and storage medium according to embodiments of the present invention.

[0024] It should be noted that the monocular 3D target detection method of this invention is implemented based on the following end-to-end dual geometric feature learning and detection framework, which will be briefly introduced below.

[0025] Figure 1 It is based on the existing MonoDETR (Depth-guided Transformer for Monocular 3D Object Detection) framework diagram. Figure 2 This is a diagram of an end-to-end dual geometric feature learning and detection framework according to an embodiment of the present invention, such as... Figure 1 and Figure 2As shown, most existing monocular 3D detection methods (such as MonoDETR) employ a single depth-guided mechanism. However, due to the pathological nature of monocular vision, the depth values ​​obtained by direct regression often face a severe scale drift challenge, meaning that there is a lack of strict physical consistency between the depth values ​​predicted by the model and the visual appearance of the object in the image.

[0026] Furthermore, traditional single-depth-branch methods tend to predict a visually reasonable depth, but often ignore the constraints of perspective projection laws. In contrast, the framework of this invention introduces parallel physical height branches and GCM (Geometric Consistency Module), and uses the intra-class stability of the height of rigid objects as geometric anchors to achieve end-to-end self-calibration of depth.

[0027] Figure 3 This is a schematic diagram of the overall network architecture of an end-to-end dual geometric feature learning and detection framework according to an embodiment of the present invention. Figure 3 In the diagram, the left side shows the input monocular image, the middle section represents the core architecture of the proposed dual-geometry-guided decoder, and the right side shows the detection head and GCM (Geometric Consistency Module). This ultimately achieves 3D bounding box generation without using NMS (3-dimensional Non-Maximum Suppression). Specifically, the architecture consists of several core subsystems: 1. Feature extraction backbone: Utilizing a deep residual network to extract multi-scale basic visual features from the original image; 2. Dual-Geometry Encoder: Containing parallel visual, depth, and height branches, mapping semantic features, depth distribution, and height priors to a unified high-dimensional embedding space; 3. Dual-Geometry Guided Decoder: Employing a cascaded attention mechanism to guide the object query vector to sequentially aggregate information from height, depth, and visual features.

[0028] To avoid redundancy, the specific composition and principles of the overall network architecture of this end-to-end dual geometric feature learning and detection framework will be explained in detail in the subsequent specific embodiments.

[0029] Based on the aforementioned end-to-end dual geometric feature learning and detection framework, the following describes the monocular 3D target detection method of this invention.

[0030] Figure 4 This is a flowchart of a monocular 3D target detection method according to an embodiment of the present invention. Figure 4As shown, a monocular 3D target detection method according to an embodiment of the present invention includes the following steps: Step S1: Input the monocular input image acquired by the camera into the backbone network for feature extraction, generate multi-scale visual feature maps, and obtain the camera's intrinsic parameter matrix.

[0031] In a specific embodiment, a camera can be used to acquire monocular RGB (Red-Green-Blue) input images; the monocular RGB input images are input into a backbone network for feature extraction, and a feature pyramid network can be used for multi-scale fusion to output a multi-scale visual feature map containing rich contextual semantics; at the same time, the intrinsic parameter matrix of the camera is read to extract projection parameters such as vertical focal length parameters.

[0032] Specifically, in this embodiment of the invention, the monocular input image acquired by the camera is first input into the backbone network for feature extraction, generating a multi-scale visual feature map, and obtaining the camera's intrinsic parameter matrix; thereby providing basic visual features and necessary physical projection parameters for subsequent 3D reconstruction, constructing a preliminary bridge from pixels to physical space, and enhancing the network's ability to perceive targets of different sizes (from large targets nearby to small targets far away) through multi-scale feature fusion, providing rich and hierarchical visual input for subsequent geometric feature encoding.

[0033] Step S2: Based on the multi-scale visual feature map, generate deep embedding feature map and high embedding feature map respectively by using parallel deep feature learning branch and high feature learning branch.

[0034] In a specific embodiment, the deep feature learning branch, for example, uses a linearly increasing discretization strategy to divide the continuous depth interval into multiple non-uniform depth intervals, predicts the probability distribution of each pixel belonging to each depth interval through convolutional layers, and then uses a learnable depth prototype vector to perform a weighted summation of the probability distribution to generate a deep embedding feature map.

[0035] In a specific embodiment, the height feature learning branch, for example, adopts a uniform discretization strategy to divide the physical height range into multiple uniform intervals, predicts the probability distribution of each pixel belonging to each height interval through a convolutional layer, then uses a learnable height prototype vector to perform a weighted summation of the probability distribution, and superimposes the multi-scale visual feature map as a residual connection to generate a height embedding feature map.

[0036] Specifically, in this embodiment of the invention, based on multi-scale visual feature maps, parallel deep feature learning branches and height feature learning branches are constructed to generate deep embedding feature maps and height embedding feature maps, respectively. The depth branch adopts non-uniform discretization, which is more in line with the perspective rule of "precise near, blurred far". The height branch adopts uniform discretization and superimposed residual connections to enhance the stability of the height prior. By introducing the physical height of rigid objects with high intra-class stability as an independent geometric anchor point, a learning architecture parallel to the depth branch is constructed. This effectively injects the network with three-dimensional spatial priors that are minimally affected by changes in viewpoint and distance, decoupling depth and height features from the source. This helps to improve the robustness of coarse localization of targets in three-dimensional space in complex road environments and reduce the false negative rate caused by changes in visual appearance.

[0037] Step S3: Using a cascaded dual geometrically guided decoder, with high-embedded feature maps, deep-embedded feature maps, and multi-scale visual feature maps as interactive objects, the object query vector used to represent the target to be detected is sequentially updated by high-guided localization, deep refinement calibration, and visual feature extraction, generating an updated object query vector carrying the target's geometric and semantic information.

[0038] In a specific embodiment, the following steps can be performed sequentially in each decoder layer of the cascaded dual geometrically guided decoder: using the object query vector of the current layer as the query and the height embedding feature map as the key and value for cross-attention calculation to achieve height-guided localization; using the object query vector after height-guided localization as the query and the depth embedding feature map as the key and value for cross-attention calculation to achieve depth refinement calibration; using the object query vector after depth refinement calibration as the query and the multi-scale visual feature map as the key and value, and using the union of height and depth key indices as the geometric index set for cross-attention calculation to achieve visual feature extraction.

[0039] Specifically, this embodiment of the invention then utilizes a cascaded dual geometrically guided decoder, using the height-embedded feature map, depth-embedded feature map, and multi-scale visual feature map as interactive objects, to sequentially update the object query vector through height-guided localization, depth refinement calibration, and visual feature extraction, generating an updated object query vector. This forces the target query vector to execute a coarse-to-fine decoupled update path of "height-guided coarse localization → depth refinement calibration → visual feature extraction," enabling the network to utilize multimodal geometric information in a targeted manner, significantly improving the model's ability to extract and refine the spatial boundaries of objects. The cascaded design ensures a reasonable cognitive order where physical priors (height) take precedence over distance estimation (depth), and distance estimation takes precedence over appearance features (visual). This fundamentally solves the technical pain point of mutual interference between multi-dimensional features in existing networks.

[0040] Step S4: Based on the updated object query vector, regress the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar used to characterize the confidence uncertainty of the direct regression depth of the target, and derive the geometric depth using the focal length parameter in the intrinsic parameter matrix in conjunction with the pinhole imaging principle.

[0041] In a specific embodiment, based on the updated object query vector, the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the target can be predicted by parallel regression of the detection head. Using the vertical focal length parameter in the intrinsic parameter matrix, the ratio of 3D physical height to 2D bounding box height is converted into geometric depth according to the pinhole imaging principle (the inverse geometric law of pinhole imaging).

[0042] Specifically, the embodiments of the present invention then, based on the updated object query vector, regress the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the target, and derive the geometric depth using the focal length parameter in the intrinsic parameter matrix in conjunction with the pinhole imaging principle. Thus, for the size drift phenomenon that has long been faced and is the most intractable problem in the field of monocular 3D detection, a differentiable geometric consistency design based on pinhole imaging prior is innovatively proposed. This transforms the rigorous inverse geometric law of pinhole imaging in the physical world into a mathematical operation inherent in the network, and combines implicit depth regression with explicit geometric derivation, providing a physical anchor point for subsequent self-supervised constraints. The introduction of the uncertainty scalar provides a confidence basis for adaptive fusion.

[0043] Step S5: Using the uncertainty scalar, adaptively weightedly fuse the direct regression depth and geometric depth, and apply a stopping gradient operation to the two depths before fusion. Combine the fused depth with the 3D spatial attributes predicted by the updated object query vector to output the 3D bounding box and obtain the final 3D object detection result of the monocular input image.

[0044] In a specific embodiment, the uncertainty scalar can be mapped to generate dynamic gating weights. Then, stopping gradient operations are applied to the direct regression depth and geometric depth respectively. Next, the two depths after the stopping gradient operation are weighted and summed using the dynamic gating weights to obtain the final fused depth. Then, the final fused depth is combined with the 3D center point projection coordinates, 3D physical size, and orientation yaw angle predicted by the updated object query vector. The absolute 3D center point is obtained by combining the inverse solution of the camera intrinsic parameter matrix. Finally, the 3D bounding box is output after 3D nonmaximum suppression.

[0045] Specifically, the embodiments of the present invention utilize an uncertainty scalar to adaptively weight and fuse the direct regression depth and geometric depth, and apply a stopping gradient operation to the two depths before fusion. Combining the fused depth with the 3D spatial attributes predicted by the updated object query vector, a 3D bounding box is output to obtain the final 3D object detection result of the monocular input image. Thus, in long-tail scenes such as distant targets or severe occlusion, the adaptive gating depth fusion mechanism that prevents gradient interference brings excellent robustness. It utilizes uncertainty for dynamic complementary advantages, and creatively introduces a stopping gradient operation to cut off the negative gradient backpropagation of unreliable features to the front-end feature extraction branch in the fusion calculation, avoiding feature assimilation during network training. This significantly improves the training convergence stability of the entire network architecture and the comprehensive detection index under complex road conditions.

[0046] Therefore, the monocular 3D target detection method according to embodiments of the present invention first inputs the monocular input image acquired by the camera into the backbone network for feature extraction, generating a multi-scale visual feature map and obtaining the camera's intrinsic parameter matrix. Then, based on the multi-scale visual feature map, parallel deep feature learning branches and height feature learning branches are constructed to generate a depth embedding feature map and a height embedding feature map, respectively. Next, using a cascaded dual geometrically guided decoder, with the height embedding feature map, depth embedding feature map, and multi-scale visual feature map as interaction objects, the object query vector is sequentially updated through height-guided localization, depth refinement calibration, and visual feature extraction to generate an updated object query vector. Then, based on the updated object query vector, the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the predicted target are regressed, and the focal length parameter in the intrinsic parameter matrix is ​​used to... This paper derives the geometric depth based on the pinhole imaging principle, and then uses an uncertainty scalar to adaptively weight and fuse the direct regression depth and geometric depth. A stopping gradient operation is applied to the two depth paths before fusion. Combining the fused depth with the 3D spatial attributes predicted by the updated object query vector, a 3D bounding box is output, yielding the final 3D object detection result for the monocular input image. By constructing a parallel depth and height dual geometric feature encoder, designing a cascaded dual geometric guided decoder, introducing a differentiable geometric consistency design, and employing an adaptive depth fusion mechanism with stopping gradients, an end-to-end, physically common-sense-driven monocular 3D detection framework is formed. This effectively solves the problems of lack of geometric consistency and severe scale drift in monocular 3D detection, significantly improving the perception accuracy and reliability of autonomous driving, and providing a new technical path for the deep integration of deep learning and geometric vision.

[0047] In one embodiment of the present invention, step S1 involves inputting a monocular input image acquired by a camera into a backbone network for feature extraction, generating a multi-scale visual feature map, and obtaining the camera's intrinsic parameter matrix. This includes: acquiring a monocular RGB input image using a camera; inputting the monocular RGB input image into a backbone network for feature extraction, and using a feature pyramid network for multi-scale fusion to output a multi-scale visual feature map; and reading the camera's intrinsic parameter matrix to extract the vertical focal length parameter from it.

[0048] In a specific embodiment, a monocular camera deployed in front of the vehicle can acquire real-time monocular RGB raw image data of a real road scene, and simultaneously read the corresponding camera calibration intrinsic parameter matrix (such as the camera's focal length, optical center coordinates, etc.) to provide the necessary parameter basis for geometric derivation in three-dimensional space. Subsequently, the monocular RGB image data is input into a pre-trained backbone network (such as ResNet (Residual Network) or DLA-34 (DeepLayer Aggregation 34-layer Network) for shallow and deep visual feature extraction, and combined with FPN (Feature Pyramid Network) for multi-scale feature fusion processing. Finally, a set of multi-scale visual feature maps containing different resolutions and rich contextual semantics is output, denoted as... This provides a high-quality visual input source for subsequent decoupling learning of depth and height features; simultaneously, it obtains the intrinsic parameter matrix of the current camera and extracts the camera focal length. Projection parameters, etc.

[0049] Specifically, the feature pyramid network integrates shallow high-resolution geometric details with deep strong semantic information through top-down paths and lateral connections to generate multi-scale feature maps, enabling the network to detect targets of different sizes simultaneously; the focal length parameter in the camera intrinsic matrix is ​​the core parameter of the pinhole imaging model, which determines the projection ratio from three-dimensional space to two-dimensional image.

[0050] Specifically, this embodiment of the invention utilizes a camera to acquire monocular RGB input images, inputs these images into a backbone network for feature extraction, and employs a feature pyramid network for multi-scale fusion to output multi-scale visual feature maps. By reading the camera's intrinsic parameter matrix, the vertical focal length parameter is extracted. This ensures the accurate acquisition of the physical parameters required for subsequent geometric derivation, laying a data foundation for geometric consistency constraints. Multi-scale feature fusion enables the network to have good response capabilities for both distant small targets and close-range large targets, while the extraction of the vertical focal length parameter provides accurate physical input for subsequent geometric depth, ensuring the accuracy of geometric derivation.

[0051] In one embodiment of the present invention, step S2, based on a multi-scale visual feature map, utilizes parallel deep feature learning branches and height feature learning branches to generate a deep embedding feature map and a height embedding feature map, respectively. This includes: the deep feature learning branch employs a linearly increasing discretization strategy to divide a continuous depth range into multiple non-uniform depth ranges; predicts a first probability distribution for each pixel belonging to each depth range using a convolutional layer; and then uses a learnable depth prototype vector to perform a weighted summation of the first probability distribution to generate a deep embedding feature map; the height feature learning branch employs a uniform discretization strategy to divide the physical height range into multiple uniform ranges; predicts a second probability distribution for each pixel belonging to each height range using a convolutional layer; then uses a learnable height prototype vector to perform a weighted summation of the second probability distribution; and superimposes the multi-scale visual feature map as a residual connection to generate a height embedding feature map.

[0052] In a specific embodiment, a Linear Increasing Discretization (LID) strategy is employed to divide a continuous depth interval into multiple non-uniform depth intervals. A convolutional layer predicts the probability distribution of each pixel belonging to each depth interval, and a learnable depth prototype vector is used to perform a weighted summation of these probability distributions to generate a depth embedding feature map. Specifically, the linear increasing discretization strategy means that the boundary values ​​of the depth intervals increase quadratically with the interval index, resulting in denser depth intervals for nearby objects and sparser intervals for distant objects. This aligns with the physical characteristics of monocular depth estimation: depth changes of nearby targets are more sensitive to pixel coordinates and require more refined modeling; depth estimation of distant targets inherently has lower accuracy, so sparse partitioning is sufficient.

[0053] In a specific embodiment, a Uniform Discretization (UD) strategy is employed to divide the physical height range into multiple uniform intervals. A convolutional layer predicts the probability distribution of each pixel belonging to each height interval. Then, a learnable height prototype vector is used to weight and sum these probability distributions, and the multi-scale visual feature map is superimposed as a residual connection to generate a height embedding feature map. Specifically, the uniform discretization strategy is suitable for the physical property of height because the actual height distribution of rigid objects is relatively concentrated and stable (e.g., the height of a car is typically between 1.4m and 1.6m), and uniform division can evenly cover the entire possible height range. The residual connection superimposes the original visual features onto the height features, preserving image texture information and avoiding the loss of details during height feature transformation.

[0054] Figure 5 This is a schematic diagram of a parallel dual geometric feature encoding process according to a specific embodiment of the present invention. Figure 5 In the middle, the top represents the deep feature learning branch, and the bottom represents the high-level feature learning branch. Figure 6 This is a comparison chart of the discretized mapping curves of a deep LID strategy and a high UD strategy according to a specific embodiment of the present invention. Figure 6 In the image, the left side shows the discretized mapping curve of the depth LID strategy, and the right side shows the discretized mapping curve of the height UD strategy. Figure 5 and Figure 6 As shown, in a specific embodiment, to address the problem in the prior art that a single visual feature is difficult to directly and accurately recover three-dimensional depth information, the specific embodiment of the present invention utilizes multi-scale visual feature maps (denoted as...). Building upon this foundation, a parallel dual-geometry architecture is constructed, which specifically comprises a deep feature learning branch and a high-order feature learning branch: In the deep feature learning branch, due to the "smaller near, larger far" error tolerance of monocular depth estimation, the deep feature learning branch employs a linearly increasing discretization strategy to divide the continuous depth space. Divided into The depth interval, the first Boundary values ​​of each depth interval The calculation formula is as follows: In the formula, These are the minimum depth range and the maximum depth range, respectively.

[0055] Utilizing lightweight convolutional layers for multi-scale visual feature maps Process the data and output the unnormalized depth log odds. The position of each pixel is calculated using the Softmax function. Probability distribution of each depth range : In the formula, pixel position The network output of the first The unnormalized log-probability of each depth interval pixel position The network output of the first The unnormalized log-probability of each depth interval It is an exponential function.

[0056] Subsequently, a set of learnable deep prototype vectors is introduced. ( (where the number of feature channels is used), and the prototype vector is weighted and summed using the above probability distribution to generate the final dense pixel positions. Deep embedding feature map : .

[0057] This depth feature map explicitly introduces perspective depth cues and geometric continuity across the entire scene, providing distance decoupling features along the camera's line of sight for subsequent 3D target localization.

[0058] In the height feature learning branch, the strong intra-class stability of rigid objects (such as vehicles and traffic signs) in the real physical world is fully utilized. Since their sensitivity to viewpoint changes and distance perturbations is much smaller than that of depth, the height feature learning branch is constructed independently of the depth branch in parallel. A uniform discretization strategy is used to select the physical height range of interest. Divided into The uniform interval, the first The center value of each height interval The calculation formula is as follows: In the formula, These are the minimum physical height and the maximum physical height, respectively.

[0059] The pixel-level height class log odds are predicted using independent convolutional layers, and a probability distribution map of the physical height is generated using the Softmax function. : In the formula, () represents the convolution function.

[0060] Introduce a set of learnable high-prototype vectors In the formula, C is the feature channel dimension. Combined with the residual connection mechanism, the pixel position is... Multiscale visual feature maps The resulting dense pixel locations are superimposed onto the probability-weighted height features. High embedding feature map : In the formula, For the first k The height prototype vector corresponding to each height category.

[0061] As an absolute geometric anchor point independent of perspective depth, the height feature map fundamentally eliminates the three-dimensional scale degradation problem caused by perspective projection, providing a robust physical prior constraint for subsequent decoding networks to perform explicit and decoupled target localization in three-dimensional space.

[0062] Specifically, this invention employs a linearly increasing discretization strategy in the deep feature learning branch to divide a continuous depth range into multiple non-uniform depth ranges. A convolutional layer predicts the first probability distribution of each pixel belonging to each depth range, and then a learnable depth prototype vector is used to perform a weighted summation of the first probability distributions to generate a depth embedding feature map. Conversely, a uniform discretization strategy in the height feature learning branch divides the physical height range into multiple uniform ranges. A convolutional layer predicts the second probability distribution of each pixel belonging to each height range, and then a learnable height prototype vector is used to perform a weighted summation of the second probability distributions. Multi-scale visual feature maps are then superimposed as residual connections to generate a height embedding feature map. Thus, by using linearly increasing discretization and uniform discretization strategies to process depth and height separately, the depth features better conform to the perspective rule of "smaller objects appear smaller in the distance," while the height features serve as absolutely stable geometric anchors. This effective decoupling of depth and height features fundamentally eliminates the risk of 3D scale degradation caused by perspective projection, providing the decoder with mutually independent geometric feature representations with clear physical meaning.

[0063] In one embodiment of the present invention, the cascaded dual geometry-guided decoder includes multiple cascaded decoder layers; step S3 utilizes the cascaded dual geometry-guided decoder, using a high-level embedding feature map, a deep-level embedding feature map, and a multi-scale visual feature map as interaction objects, to sequentially update the object query vector used to characterize the target to be detected through height-guided localization, depth refinement calibration, and visual feature extraction, generating an updated object query vector carrying target geometric and semantic information, including: performing the following operations sequentially in each decoder layer of the cascaded dual geometry-guided decoder: using the object query vector of the current decoder layer as the query, using a high-level embedding feature map, a deep-level embedding feature map, and a multi-scale visual feature map as interaction objects, to sequentially update the object query vector carrying target geometric and semantic information through height-guided localization, depth refinement calibration, and visual feature extraction, generating an updated object query vector. Cross-attention calculation is performed using the embedded feature map as the key and value, and a set of height key indices is selected to achieve height-guided localization. Cross-attention calculation is performed using the height-guided localized object query vector as the query and the deep embedded feature map as the key and value, and a set of depth key indices is selected to achieve depth refinement calibration. Cross-attention calculation is performed using the depth-refinement calibrated object query vector as the query and the multi-scale visual feature map as the key and value, and the union of the height key index set and the depth key index set is used as the geometric index set. Cross-attention calculation is performed only on the regions specified by this geometric index set to achieve visual feature extraction. The updated object query vector is output.

[0064] In a specific embodiment, cross-attention calculation is performed using the object query vector of the current layer as the query and the highly embedded feature map as the key and value. A set of highly key indices is selected to achieve highly guided localization. Specifically, the cross-attention mechanism generates attention weights by calculating the similarity between the query vector and all keys, and then aggregates the corresponding values ​​in a weighted manner. Selecting the "key index set" means sampling according to the attention weights and retaining only a few positions with the strongest response, thereby achieving sparsity processing and reducing computational complexity.

[0065] In a specific embodiment, the object query vector after height-guided localization is used as the query, and the depth embedding feature map is used as the key and value for cross-attention calculation. A set of depth key indexes is selected to achieve depth refinement calibration. Specifically, this step is performed after height-guided localization, meaning that the query vector already carries prior height information. When interacting with depth features, it can more effectively refine the depth estimation within a reasonable height region.

[0066] In a specific embodiment, the object query vector after depth refinement and calibration is used as the query, and the multi-scale visual feature map is used as the key and value. The union of the height key index set and the depth key index set is used as the geometric index set. Cross-attention calculation is performed only on the regions specified by this geometric index set to achieve visual feature extraction. Specifically, the union mechanism of the geometric index set is equivalent to a geometric mask, forcing the network to extract visual features only on image regions that simultaneously satisfy both height reasonableness and depth saliency, effectively filtering background noise and geometrically unreasonable regions.

[0067] Figure 7 This is a schematic diagram of the cascaded attention interaction mechanism inside a dual geometry-guided decoder according to a specific embodiment of the present invention. Figure 7 The diagram illustrates the cascaded attention interaction mechanism of a single decoder layer within the Dual-Geometry Guided Decoder. For example... Figure 7 As shown, in a specific embodiment, a cascaded dual-geometry guided decoder is designed. Through a multi-layered cascaded Transformer decoder architecture, an explicit decoupling and coarse-to-fine strategy of "high-level guided coarse localization - deep refinement - visual feature extraction" is adopted to progressively update a set of initialized object query vectors.

[0068] Before the decoder performs feature interaction, the object query vector is first initialized. Defined as content feature part With location features The summation; to accelerate network convergence and provide initial 3D spatial priors, through a set of... A set of learnable 3D reference points To generate the initial location code, the initial object query vector. It is expressed as follows: In the formula, This represents the position encoding function. The 3D reference point is dynamically updated in each layer of the decoder based on the prediction results, thereby gradually approaching the true 3D center point of the object.

[0069] In each decoder layer, the object query vector undergoes state evolution sequentially through three execution sub-modules: height-guided localization, depth refinement, and visual feature extraction, all cascaded in series. 1. Height-guided Localization: Uses the object query vector input from the current layer. As a query, using a high-resolution feature map Cross-attention is performed using physical height features as keys and values. Leveraging the robustness and intra-class stability of physical height features independent of perspective distance, the object query vector is quickly focused in 3D space onto a target region conforming to a specific physical size distribution, achieving preliminary coarse localization in 3D space. To reduce computational overhead and filter background noise, a Top-K sampling (first K samples) strategy is employed, selecting a set of height-critical indices based on attention weights. The highly guided update process is represented as follows: In the formula, ( ) represents layer normalization, ( ) represents multi-head attention computation. () represents sparse sampling. This formula injects the prior physical scale of the object into the query vector, thereby effectively excluding background areas with obviously unreasonable physical heights in deep features.

[0070] 2. Depth Refinement Submodule: This module generates the object query vector output after the height-guided localization step. As a query, using deep feature maps Cross-attention is performed as both key and value; using perspective depth cues, the initial target localization results are fine-tuned and calibrated along the camera's depth axis, enabling the query vector to distinguish visually similar objects with different depths; similarly, the set of key indices with the strongest depth response is selected through depth attention weights. The query vector is updated by sampling the corresponding deep features, and the calculation formula is as follows: .

[0071] 3. Visual Feature Extraction Submodule: Uses object query vectors calibrated with both height and depth geometric features. As a query, using multi-scale visual feature maps Cross-attention interaction is performed as keys and values; at this point, the object query vector has accurate spatial boundaries and three-dimensional scale orientation; to ensure the consistency of the feature space and achieve efficient sparse sampling, the geometric saliency index generated in the first two steps is reused, and the height index is used. and depth index The union of the sets is denoted as the geometric index set. Cross-attention is performed only on reasonable regions specified by the geometric index set. The extraction formula for fine-grained visual appearance features (such as the target's category semantics, specific 3D size, and orientation angle) is expressed as follows: .

[0072] This sparse extraction mechanism based on geometric indexes acts as a geometric mask, forcing the network to focus only on image regions that conform to physical laws in terms of physical height and perspective depth, thus significantly suppressing background clutter interference.

[0073] The updated object query vector is output at each layer of the decoder. Then, a shared detection head is used to predict the spatial offset of the target relative to the current reference point. Through an iterative dynamic reference point update mechanism, coarse-to-fine positioning is achieved; the new reference point coordinates... The update formula is expressed as follows: In the formula, This represents the Sigmoid activation function. This represents its corresponding inverse function (Logit transform). This represents the coordinates of the reference point from the previous round. This represents a multilayer perceptron prediction network.

[0074] Through the collaborative interaction of cascaded attention mechanism and dynamic reference point update, object query can gradually lock onto the true three-dimensional physical center of the object as the number of layers increases, completely avoiding feature coupling and mutual interference caused by mixed learning of height, depth and visual semantic features in a single attention layer.

[0075] Specifically, in each decoder layer of the cascaded dual geometrically guided decoder, the following steps are performed sequentially: Using the object query vector of the current layer as the query and the height embedding feature map as the key and value, cross-attention calculation is performed to select a set of height key indices for height-guided localization; using the height-guided localized object query vector as the query and the depth embedding feature map as the key and value, cross-attention calculation is performed to select a set of depth key indices for depth refinement calibration; using the depth-refinement calibrated object query vector as the query and the multi-scale visual feature map as the key and value, the union of the height key indices and the depth key indices is used as the geometric index set, and cross-attention calculation is performed only on the regions specified by this geometric index set to achieve visual feature extraction; the updated object query vector is output. Thus, through the serially cascaded cross-attention mechanism and the use of the Top-K strategy to select key indices for sparse interaction, computational overhead is significantly reduced. Simultaneously, the network is forced to follow a cognitive logic from geometry (height) to distance (depth) and then to appearance (vision), improving the efficiency and accuracy of feature learning. The geometric index filtering mechanism plays an implicit spatial attention role, enabling the network to focus on the most informative regions.

[0076] In one embodiment of the present invention, step S4, based on the updated object query vector, regresses and predicts the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar used to characterize the confidence uncertainty of the direct regression depth, and derives the geometric depth using the focal length parameter in the intrinsic parameter matrix in conjunction with the pinhole imaging principle. This includes: based on the updated object query vector, regressing and predicting the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the target in parallel using the detection head; and using the vertical focal length parameter in the intrinsic parameter matrix, converting the ratio of the 3D physical height to the 2D bounding box height into geometric depth according to the method that the geometric depth is equal to the vertical focal length parameter multiplied by the 3D physical height divided by the 2D bounding box height.

[0077] In one embodiment of the present invention, after regressing and predicting the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the target, and using the focal length parameter in the intrinsic parameter matrix to derive the geometric depth in combination with the pinhole imaging principle, the method further includes: constructing a consistency loss function between the direct regression depth and the geometric depth, and adding the consistency loss function to the total loss for end-to-end error backpropagation.

[0078] Figure 8 This is a schematic diagram of the gradient flow direction and gradient stopping strategy of a differentiable geometric consistency module according to a specific embodiment of the present invention. Figure 8The diagram illustrates the gradient flow direction and gradient stopping strategy of the differentiable geometrical consistency module (GCM). Figure 8 As shown, the feature map is input to two parallel branches: the regression head is the student branch, used to predict the direct regression depth Zreg; the geometry head is used to predict the 3D physical height H3d and the 2D bounding box height h2d; the computation module calculates the geometrically derived depth Zgeo using the pinhole camera model formula, and this branch is labeled TeacherSignal to represent the teacher signal; subsequently, Log-L1 loss, representing the log-smooed L1 loss function, is used to calculate the consistency loss between the student branch output Zreg and Zgeo; the gradient update direction is backpropagated from the loss to the regression head; the gradient stop operation is located after the geometry head and the computation module, cutting off the gradient backpropagation from the geometrically derived depth Zgeo to the height prediction branch, preventing depth prediction errors from interfering with the learning of the height branch.

[0079] Figure 9 This is a schematic diagram comparing the degradation mitigation risk caused by not using a stopping gradient with the robust calibration process after using a stopping gradient, according to a specific embodiment of the present invention. Figure 9 In the image, the left side shows the degenerate solution risk due to not using the stopping gradient (Without SG, Degenerate Solution), and the right side shows the robust calibration process after using the stopping gradient (With SG, Robust Calibration). Figure 9 As shown in the left-hand flow, during backpropagation of the loss, the gradient will be passed to h. proj (Projected height), due to the gradient magnitude and 1 / h proj 2 Proportional, when h proj When the gradient is small, h becomes maximum, leading to h proj Collapse (e.g., → 0 or ∞); in the right-hand flow, the Stop Gradient (SG) operation cuts off the loss pair h. proj Gradient backpropagation, Z reg Converging to Z geo (Already calibrated).

[0080] like Figure 8 and Figure 9 As shown in the specific embodiment, in order to eliminate the severe scale drift of three-dimensional dimensions and depth caused by minor changes in visual appearance during perspective projection, a differentiable GCM (Geometric Consistency Module) is constructed, and its specific execution flow is as follows: 1. Parametric Regression: The object query vector is input into the parallel prediction branch of the detection head. The detection head, through fully connected layers and linear mapping layers, regresses and predicts the height of the 2D bounding box of the target on the two-dimensional plane of the image. 3D physical height of the target itself and the direct regression depth of the target .

[0081] 2. Geometric Depth Derivation: Based on the ideal pinhole camera projection imaging principle, there is a strict inverse perspective geometric law between the target's 3D physical height, the projected height on the image, and the absolute depth of the scene. The geometric consistency module uses differentiable mathematical formulas to correlate discrete 2D detection parameters with 3D spatial dimensions across space, deriving the target's geometric depth online. Its mathematical formula is expressed as follows: In the formula, Given the known focal length parameters of the current camera along the vertical direction. By reading the camera intrinsic parameter matrix, this formula explicitly transforms the implicit black-box deep reasoning process into a geometric quantitative calculation based on physical common sense.

[0082] 3. Constructing a geometric consistency loss function: During the network training phase, in order to force the 3D physical parameters obtained by the network regression to match the 2D perspective results shown by the image, and to overcome the lack of robustness of purely data-driven models, a self-supervised geometric consistency loss function was designed. It utilizes smoothness The loss function for the direct regression depth depth of geometric derivation The spatial solution differences between them are measured, and the loss function expression is as follows: By using geometric consistency loss By introducing a total training loss for end-to-end error backpropagation, implicit depth regression is transformed into explicit geometric and physical constraints. This mechanism forces the feature learning network to automatically correct any prediction biases that violate the pinhole perspective principle during parameter updates, effectively eliminating the significant size drift and positioning errors caused by minute visual deformations. Specifically, the total training loss function... include: = +λ× In the formula, Here, λ represents the base loss function for monocular 3D object detection, where λ is the weighting coefficient for the geometric consistency loss. This is presented as an example, not a limitation. It can be any of the following forms: (1) = + In the formula, For classifying losses, The regression loss is for the target 3D attributes (including 3D center point coordinates, 3D size, and orientation angle); (2) = + + + In the formula, For heatmap loss, For 2D center point deviation loss, For 2D size loss, For 3D attribute regression loss; (3) = + + + + + In the formula, , , , and These correspond to 2D bounding box regression loss, target confidence loss, classification loss, center point offset loss, depth loss, size loss, and orientation angle loss, respectively. (4) Other well-known basic loss functions for monocular 3D target detection in this field.

[0083] Geometric consistency loss As a newly added self-supervised constraint term, it is added to the basic loss function and together with any of the above basic loss functions, it constitutes the total training loss function and participates in end-to-end error backpropagation.

[0084] Specifically, this embodiment of the invention, based on the updated object query vector, predicts the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the target through parallel regression using a detection head. Utilizing the vertical focal length parameter in the intrinsic parameter matrix, the ratio of the 3D physical height to the 2D bounding box height is converted into geometric depth according to the formula that geometric depth equals the vertical focal length parameter multiplied by the 3D physical height divided by the 2D bounding box height. This quantifies the perspective projection law of the physical world into a differentiable mathematical formula, realizing explicit inference from two-dimensional image metrics (pixel height) and three-dimensional physical priors (actual height) to scene depth (geometric depth). This derivation process is fully differentiable and can participate in end-to-end backpropagation, enabling the network to directly learn depth estimates that conform to physical laws during training.

[0085] Furthermore, after deriving the geometric depth, this embodiment of the invention also constructs a consistency loss function between the direct regression depth and the geometric depth, and incorporates the consistency loss function into the total loss for end-to-end error backpropagation. The introduced geometric consistency loss function, as a self-supervised physical constraint, can drive the network to self-calibrate without additional labeling information, making the depth of direct regression continuously approach the geometric depth derived based on physical laws. This mechanism overcomes the deficiency of pure data-driven models lacking physical common sense, transforming implicit depth regression into explicit physical law constraints, and fundamentally suppressing scale drift.

[0086] In one embodiment of the present invention, to expand the scope of patent protection and prevent others from bypassing this technology through simple mathematical substitution or feature dimension replacement, several alternative technical solutions that can also achieve the purpose of the present invention are provided for the technical solution of depth self-calibration and self-supervision constraint based on geometric consistency module: As an alternative, the geometric consistency self-supervised constraint based on pinhole imaging in this invention can be replaced by a reverse 2D perspective reprojection constraint mechanism. Specifically, instead of directly calculating the absolute geometric depth using a formula, this invention reverses the process by using the depth directly regressed by the network to project the 3D spatial features back onto a 2D plane. In this approach, combined with the camera focal length, the network-predicted 3D physical height is projected onto the 2D image plane using direct regression depth, deriving the 2D projected height of the target. Subsequently, a self-supervised reprojection loss function is constructed by calculating the difference between this projected height and the 2D bounding box height extracted by the network. This method also establishes a closed-loop physical alignment between the 3D parameters and 2D visual features within the network, effectively suppressing the size drift caused by perspective projection and achieving the same inventive objective.

[0087] As a second alternative, the equivalent perspective derivation basis in the geometric consistency module can be replaced or extended to other multi-dimensional 3D physical anchor points of the target, rather than being limited to physical height. Specifically, this mechanism can use the physical width of the target as a stable anchor point, and derive the consistency of the depth space through the proportional relationship between the 3D physical width and the 2D bounding box width; or, more precisely, construct the eight spatial corner points of the complete 3D bounding box using the 3D center point, 3D size, and orientation angle of the network regression, and then use the regression depth to project its perspective onto the 2D image plane, and then construct the consistency loss by calculating the spatial alignment error between these projected 2D corner points and the actual detected target key points in the image; this modified scheme also follows the strict 3D to 2D perspective geometry rules, and realizes mutual supervision of the network's intrinsic parameters.

[0088] In addition to the aforementioned solutions, the geometric consistency constraint mechanism can also be transformed into an equivalent self-supervised computation in the probability distribution domain or disparity domain. For example, during the network prediction stage, the scalar prediction structure of depth can be modified to output a probability distribution model of the target depth (such as the mean and variance of the predicted Gaussian distribution), and then the KL divergence (Kullback-Leibler Divergence) or negative log-likelihood loss between this predicted distribution and the depth pseudo-label derived using geometric formulas can be calculated as a consistency constraint. Alternatively, to avoid drastic fluctuations in the absolute depth value calculation at extremely long distances, the core derivation formula can be transformed into the disparity domain, which is inversely proportional to depth. The disparity substitution can be calculated using the ratio of 2D height to 3D physical height, and the loss can be aligned with the disparity value directly regressed by the network. These variant techniques based on probability models or disparity transformations all adhere to the core concept of using the prior physical laws of pinhole imaging to constrain network feature learning, and can also successfully achieve the technical objectives of this invention.

[0089] In one embodiment of the present invention, step S5 utilizes an uncertainty scalar to adaptively weight and fuse the direct regression depth and geometric depth, and applies a stopping gradient operation to the two depths before fusion. Combining the fused depth with the 3D spatial attributes predicted based on the updated object query vector, a 3D bounding box is output. This includes: generating dynamic gating weights by mapping the uncertainty scalar; applying stopping gradient operations to the direct regression depth and geometric depth respectively; using the dynamic gating weights to perform a weighted summation of the two depths after the stopping gradient operation to obtain the final fused depth; combining the final fused depth with the 3D center point projection coordinates, 3D physical dimensions, and orientation yaw angle predicted based on the updated object query vector, and combining this with the inverse solution of the camera intrinsic parameter matrix to obtain the absolute 3D center point, and then outputting the 3D bounding box after 3D nonmaximum suppression.

[0090] Figure 10 This is a schematic diagram of an adaptive deep fusion module based on object query vectors according to a specific embodiment of the present invention. Figure 10 In the context of object query (Q) obj As input features to this module, the regression head is used to predict the direct regression depth Z. reg The geometry head is used to predict the 3D physical height H and the 2D projected height h. proj Using the calculation formula Z=H×f / h proj Derivation of geometric depth Zgeo The gated network (MLP + Sigmoid, multilayer perceptron with Sigmoid activation function) takes the object query vector as input and outputs dynamic gating weights α. Then, it uses the fusion formula Z... final =α×Z reg +(1 α)×Zgeo The regression depth and geometric depth are adaptively weighted and summed to obtain the final fusion depth Z. final .

[0091] Figure 11 This is a schematic diagram illustrating the statistical distribution and trend analysis of the gating weight as a function of target distance according to a specific embodiment of the present invention. Figure 10 and Figure 11 As shown, in a specific embodiment, during the inference or testing phase, for a distant target (whose 2D bounding box height is...) Extremely small, causing tiny perturbations in the denominator to lead to a greater depth in geometric derivation. (Severe fluctuations) or severely occluded targets (incomplete visual appearance information leads to direct depth regression) For complex scenarios (inaccurate), an uncertainty-based gated adaptive deep fusion mechanism is designed. This mechanism dynamically adjusts the fusion weights of direct regression depth and geometric derivation depth. The specific execution flow is as follows: 1. Uncertainty estimation: The detection head predicts the target depth during regression. Simultaneously, a scalar of uncertainty about depth is predicted in parallel. Uncertainty scalar It reflects the current network's confidence in the direct regression depth results. Its value is predicted by the uncertainty branch in the detector head, thus providing data guidance for the subsequent construction of dynamic adaptive gating weights.

[0092] 2. Stop gradient operation: When the direct regression depth is reduced... depth of geometric derivation Before inputting into the deep fusion network, explicit SG operations are performed on each network to truncate gradient backpropagation during the optimization process: This results in a regression depth after gradient truncation. Geometry depth after gradient descent stops By applying the stopping gradient operation to the two input depth values, the gradient generated by the fusion layer during training can be completely blocked from being propagated back to the front-end feature network. This design can effectively prevent the joint optimization of the fusion weights from forcing the height and depth features to compromise with each other, thereby perfectly protecting the specificity of the independent height feature learning branch and the depth feature learning branch, and ensuring the independence and stability of the network when extracting deep and high features.

[0093] 3. Deep Gated Fusion: Utilizing Uncertainty Scalars Dynamic gating weights are generated by calculating the Sigmoid activation function (S-shaped growth curve) through a mapping layer. (in Subsequently, dynamic gating weights were used. An adaptive weighted summation is performed on the regression depth and geometric depth after gradient truncation to obtain the final fused depth that takes into account the advantages of both depth cues. Its mathematical formula is expressed as follows: This mechanism can dynamically adjust the complementary ratio of the two according to the actual scene in which the target is located. When the geometric derivation depth fluctuates due to 2D height perturbation, it automatically tilts towards the regression depth. When the regression depth fails due to occlusion, it automatically amplifies the geometric depth weight that conforms to physical common sense, thus realizing the complementary advantages of depth estimation across the entire distance range.

[0094] 4. Final 3D bounding box output: The final blending depth obtained The projected coordinates of the target's 3D center point, 3D physical dimensions (length, width, height), and orientation yaw angle predicted by the detection head are analytically combined. The projected center point is then converted to a 3D world coordinate system using the camera intrinsic parameter matrix to calculate the absolute 3D center point position of the target. Finally, the target's category confidence is combined with the overlapping box filtering using the 3D NMS (3-dimensional Non-Maximum Suppression) method to output the final high-precision monocular 3D target detection bounding box result.

[0095] Specifically, this embodiment of the invention generates dynamic gating weights by mapping an uncertainty scalar; applies stopping gradient operations to both the direct regression depth and the geometric depth; uses the dynamic gating weights to perform a weighted sum of the two depths after the stopping gradient operation to obtain the final fused depth; combines the final fused depth with the 3D center point projection coordinates, 3D physical dimensions, and orientation yaw angle predicted by the updated object query vector, and uses the inverse solution of the camera intrinsic matrix to obtain the absolute 3D center point, and then outputs the 3D bounding box through 3D non-maximum suppression; thus, by applying stopping gradient operations to the depth before fusion, it effectively prevents the deep fusion network from destroying the independence of the front-end height and depth feature branches during training, ensuring the overall stable convergence of the network; the dynamic gating weights enable the model to adaptively choose between trusting the regression depth or the geometric depth according to the scene: when the target is clear and the network confidence is high, it tends to regress the depth; when the target is far away, small, or occluded, the geometrically derived depth provides more stable physical constraints, ensuring that the final output detection box has high quality and uniqueness.

[0096] In summary, the key features of the monocular 3D object detection method provided by this invention lie in the physical-level explicit decoupling and geometric prior self-supervision of multi-dimensional features in monocular 3D object detection. The technical solution breaks through the black-box limitation of the severe coupling between depth and visual features in traditional networks, innovatively introducing the "physical height" of rigid objects, which possesses high intra-class stability, as an independent spatial anchor point, constructing a dual geometric feature learning architecture that parallels height and depth. Based on this, a cascaded decoder that strictly follows the physical spatial cognitive logic is designed, forcing the target query vector to follow a process of "height-guided coarse localization, depth-refined calibration, and depth-guided coarse localization." The visual feature extraction process is explicitly decoupled and updated from coarse to fine order. Meanwhile, to address the size drift issue caused by perspective projection, a differentiable geometric consistency module based on pinhole imaging principles is constructed. By aligning the geometric depth with the direct regression depth through online derivation, an end-to-end self-supervised depth loop is formed. Furthermore, to cope with complex occlusion or long-distance scenes, an adaptive gating depth fusion mechanism based on uncertainty is introduced. By cleverly cutting off the reverse interference of fusion computation on front-end feature extraction through stopping gradient operations, the purity and stability of feature learning in each branch of the network are fundamentally guaranteed.

[0097] In other words, the monocular 3D target detection method provided by this invention has the following advantages compared with the prior art: (1) This invention constructs a parallel dual geometric feature learning architecture by introducing the "physical height" of rigid objects, which has great intra-class stability, as an independent geometric anchor point. Existing monocular detection technology usually relies on visual cues or a single depth estimate, which is very easy to face the serious problem of loss of depth information and scale ambiguity caused by perspective projection. The technical solution of this invention effectively injects the network with three-dimensional spatial priors that are minimally affected by changes in viewpoint and distance. This explicit physical constraint greatly improves the robustness of coarse localization of the target in three-dimensional space in complex road environments and significantly reduces the false alarm rate caused by changes in visual appearance.

[0098] (2) The cascaded dual geometric guided decoder designed in this invention fundamentally solves the technical pain point of mutual interference of multi-dimensional features in existing networks. Existing technologies often mix height, depth and visual semantic features in a single attention mechanism. This "black box" feature coupling not only reduces learning efficiency but also easily leads to the superposition of localization errors. This invention forces the target query vector to perform a coarse-to-fine decoupled update path of "height-guided coarse localization, depth-refined calibration, and visual feature extraction", which enables the network to make targeted use of multimodal geometric information and greatly improves the model's ability to extract the spatial boundary of objects and restore the fineness.

[0099] (3) In response to the long-standing and most challenging size drift phenomenon in the field of monocular 3D detection, this invention innovatively proposes a differentiable geometric consistency module based on pinhole imaging priors. Traditional networks mostly directly and brute-force regress 3D parameters, relying entirely on data-driven approaches and lacking physical common sense constraints, which makes them prone to failure when dealing with unseen scenes. This invention transforms the rigorous inverse geometric law of pinhole imaging in the physical world into a self-supervised closed loop inherent in the network, derives the geometric depth online, and forces it to be consistent with the depth directly regressed by the network. This method transforms implicit depth regression into explicit physical law constraints, effectively eliminating the huge 3D positioning error caused by small deformations in visual appearance, and significantly improving the absolute depth estimation accuracy and generalization ability of the model across the entire distance range.

[0100] (4) In the face of long-tail extreme scenarios such as distant targets, severe occlusion or image truncation, the present invention brings excellent robustness through an adaptive gating depth fusion mechanism that prevents gradient interference. Existing methods often completely collapse when encountering unclear or unreliable image features. The present invention not only utilizes the uncertainty of prediction to dynamically complement the advantages of the two depth sources, but also creatively introduces a stopping gradient operation in the technical solution. This design ingeniously cuts off the negative gradient backpropagation of unreliable features to the front-end feature extraction branch during the fusion calculation process, avoids feature assimilation during network training, and thus greatly improves the training convergence stability of the entire network architecture and the comprehensive detection index under complex road conditions.

[0101] A further embodiment of the present invention discloses a monocular 3D target detection system. Figure 12 This is a structural block diagram of a monocular 3D target detection system according to an embodiment of the present invention. Figure 12 As shown, in one embodiment of the present invention, the monocular 3D target detection system 100 includes: a feature extraction module 110, a dual geometric feature encoding module 120, a decoding module 130, a geometric consistency module 140, and a fusion output module 150.

[0102] Specifically, the feature extraction module 110 is used to input the monocular input image acquired by the camera into the backbone network for feature extraction, generate multi-scale visual feature maps, and obtain the camera's intrinsic parameter matrix.

[0103] The dual geometric feature encoding module 120 is used to construct parallel deep feature learning branches and high feature learning branches based on multi-scale visual feature maps, and generate deep embedding feature maps and high embedding feature maps respectively.

[0104] The decoding module 130 is used to utilize a cascaded dual geometrically guided decoder, with high-embedded feature map, deep-embedded feature map and multi-scale visual feature map as interactive objects, to sequentially update the object query vector through height-guided localization, depth refinement calibration and visual feature extraction, and generate an updated object query vector.

[0105] The geometric consistency module 140 is used to regress the 2D bounding box height, 3D physical height, direct regression depth and uncertainty scalar of the target based on the updated object query vector, and to derive the geometric depth using the focal length parameter in the intrinsic parameter matrix in combination with the pinhole imaging principle.

[0106] The fusion output module 150 is used to adaptively weight and fuse the direct regression depth and geometric depth using an uncertainty scalar, and to apply a stopping gradient operation to the two depths before fusion. It combines the fused depth with the 3D spatial attributes predicted by the updated object query vector to output a 3D bounding box, thus obtaining the final 3D object detection result of the monocular input image.

[0107] It should be noted that the specific implementation of the monocular 3D target detection system 100 in this embodiment of the invention is similar to the specific implementation of the monocular 3D target detection method described in the above embodiment of the invention, and therefore has similar technical effects. For details, please refer to the description of the monocular 3D target detection method section. To reduce redundancy, it will not be repeated here.

[0108] Further embodiments of the present invention also disclose an electronic device, Figure 13 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Figure 13 As shown, in one embodiment of the present invention, the electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; when the processor executes the computer programs stored in the memory, it implements the monocular 3D target detection method as described in any of the above embodiments of the present invention.

[0109] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0110] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0111] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0112] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0113] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0114] It should be noted that the specific implementation of the electronic device in the embodiments of the present invention is similar to the specific implementation of the monocular 3D target detection method described in the above embodiments of the present invention, and therefore has similar technical effects. For details, please refer to the description of the monocular 3D target detection method section. In order to reduce redundancy, it will not be repeated here.

[0115] Further embodiments of the present invention disclose a computer-readable storage medium storing a computer program that, when executed by a processor, implements the monocular 3D target detection method as described in any of the above embodiments of the present invention.

[0116] It should be noted that the specific implementation of the computer-readable storage medium in the embodiments of the present invention is similar to the specific implementation described in the monocular 3D target detection method of the above embodiments of the present invention, and therefore has similar technical effects. For details, please refer to the description of the monocular 3D target detection method section. To reduce redundancy, it will not be repeated here.

[0117] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0118] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A monocular 3D target detection method, characterized in that, Includes the following steps: The monocular input image captured by the camera is input into the backbone network for feature extraction, generating a multi-scale visual feature map, and the intrinsic parameter matrix of the camera is obtained. Based on the multi-scale visual feature map, deep embedding feature map and high embedding feature map are generated by using parallel deep feature learning branches and height feature learning branches, respectively. Using a cascaded dual geometrically guided decoder, with the height embedding feature map, the depth embedding feature map and the multi-scale visual feature map as interaction objects, the object query vector used to characterize the target to be detected is sequentially updated by height-guided localization, depth refinement calibration and visual feature extraction, generating an updated object query vector carrying the target's geometric and semantic information. Based on the updated object query vector, the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar used to characterize the confidence uncertainty of the direct regression depth of the target are regressed and predicted. The geometric depth is derived by using the focal length parameter in the intrinsic parameter matrix and combining the pinhole imaging principle. Using the uncertainty scalar, the direct regression depth and the geometric depth are adaptively weighted and fused, and a stopping gradient operation is applied to the two depths before fusion. Combining the fused depth with the 3D spatial attributes predicted based on the updated object query vector, a 3D bounding box is output to obtain the final 3D object detection result of the monocular input image. The cascaded dual geometry-guided decoder includes multiple cascaded decoder layers. The method utilizes a cascaded dual geometrically guided decoder, using the height-embedded feature map, the depth-embedded feature map, and the multi-scale visual feature map as interaction objects. It sequentially updates the object query vector representing the target to be detected through height-guided localization, depth refinement calibration, and visual feature extraction, generating an updated object query vector carrying target geometric and semantic information. This includes: The following operations are performed sequentially in each decoder layer of the cascaded dual geometry-guided decoder: Using the object query vector of the current decoder layer as the query and the high-embedded feature map as the key and value, cross-attention calculation is performed to select a set of high-key indexes and achieve height-guided localization. Using the height-guided localization object query vector as the query and the depth embedding feature map as the key and value, cross-attention calculation is performed, and a set of depth key indexes is selected to achieve depth refinement calibration; Using the object query vector after depth refinement and calibration as the query, and the multi-scale visual feature map as the key and value, the union of the height key index set and the depth key index set is used as the geometric index set. Cross attention is calculated only on the region specified by the geometric index set to achieve visual feature extraction. Output the updated object query vector.

2. The monocular 3D target detection method according to claim 1, characterized in that, The process of inputting the monocular input image acquired by the camera into the backbone network for feature extraction, generating multi-scale visual feature maps, and obtaining the intrinsic parameter matrix of the camera includes: Capture monocular RGB input images using a camera; The monocular RGB input image is input into the backbone network for feature extraction, and multi-scale fusion is performed using the feature pyramid network to output a multi-scale visual feature map. Read the camera's intrinsic parameter matrix and extract the vertical focal length parameter from it.

3. The monocular 3D target detection method according to claim 1, characterized in that, Based on the multi-scale visual feature map, deep embedding feature maps and high embedding feature maps are generated using parallel deep feature learning branches and height feature learning branches, respectively, including: The deep feature learning branch uses a linearly increasing discretization strategy to divide the continuous depth interval into multiple non-uniform depth intervals. It predicts the first probability distribution of each pixel belonging to each depth interval through a convolutional layer, and then uses the learnable deep prototype vector to perform a weighted summation of the first probability distribution to generate a deep embedding feature map. The height feature learning branch uses a uniform discretization strategy to divide the physical height range into multiple uniform intervals. It predicts the second probability distribution of each pixel belonging to each height interval through a convolutional layer, and then uses the learnable height prototype vector to perform a weighted summation of the second probability distribution. The multi-scale visual feature map is then superimposed as a residual connection to generate a height embedding feature map.

4. The monocular 3D target detection method according to claim 1, characterized in that, The process involves regressing the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar representing the confidence uncertainty of the direct regression depth based on the updated object query vector, and deriving the geometric depth using the focal length parameter in the intrinsic parameter matrix in conjunction with the pinhole imaging principle, including: Based on the updated object query vector, the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar of the target are predicted by parallel regression of the detection head. Using the vertical focal length parameter in the intrinsic parameter matrix, the ratio of the 3D physical height to the 2D bounding box height is converted into geometric depth according to the method that the geometric depth is equal to the vertical focal length parameter multiplied by the 3D physical height divided by the 2D bounding box height.

5. The monocular 3D target detection method according to claim 1, characterized in that, The process involves using the uncertainty scalar to adaptively weight and fuse the direct regression depth and the geometric depth, applying a stopping gradient operation to the two depths before fusion, and combining the fused depth with the 3D spatial attributes predicted based on the updated object query vector to output a 3D bounding box, including: The uncertainty scalar is mapped to generate dynamic gating weights; A stopping gradient operation is applied to both the direct regression depth and the geometric depth. The two depths after the gradient operation is stopped are weighted and summed using the dynamic gating weights to obtain the final fused depth. The final fusion depth is combined with the 3D center point projection coordinates, 3D physical size, and orientation yaw angle predicted based on the updated object query vector. The absolute 3D center point is obtained by inverse solving the intrinsic parameter matrix, and then the 3D bounding box is output after 3D nonmaximum suppression.

6. The monocular 3D target detection method according to claim 1, characterized in that, After deriving the geometric depth, the monocular 3D target detection method further includes: Construct a consistency loss function between the direct regression depth and the geometric depth, and add the consistency loss function to the total loss for end-to-end error backpropagation.

7. A monocular 3D target detection system, characterized in that, include: The feature extraction module is used to input the monocular input image acquired by the camera into the backbone network for feature extraction, generate multi-scale visual feature maps, and obtain the intrinsic parameter matrix of the camera. The dual geometric feature encoding module is used to generate a deep embedding feature map and a high embedding feature map respectively based on the multi-scale visual feature map and by using parallel deep feature learning branches and height feature learning branches. The decoding module is used to utilize a cascaded dual geometrically guided decoder, with the height embedding feature map, the depth embedding feature map and the multi-scale visual feature map as interaction objects, to sequentially update the object query vector used to characterize the target to be detected by height-guided localization, depth refinement calibration and visual feature extraction, thereby generating an updated object query vector carrying target geometric and semantic information. The geometric consistency module is used to regress the 2D bounding box height, 3D physical height, direct regression depth, and uncertainty scalar for characterizing the confidence uncertainty of the direct regression depth based on the updated object query vector, and to derive the geometric depth using the focal length parameter in the intrinsic parameter matrix in combination with the pinhole imaging principle. The fusion output module is used to adaptively weight and fuse the direct regression depth and the geometric depth using the uncertainty scalar, and apply a stopping gradient operation to the two depths before fusion. Combining the fused depth with the 3D spatial attributes predicted by the updated object query vector, it outputs a 3D bounding box to obtain the final 3D object detection result of the monocular input image. The cascaded dual geometry-guided decoder includes multiple cascaded decoder layers. The decoding module utilizes a cascaded dual geometrically guided decoder, using the height-embedded feature map, the depth-embedded feature map, and the multi-scale visual feature map as interaction objects. It sequentially updates the object query vector representing the target object through height-guided localization, depth refinement calibration, and visual feature extraction, generating an updated object query vector carrying target geometric and semantic information, including: The following operations are performed sequentially in each decoder layer of the cascaded dual geometry-guided decoder: Using the object query vector of the current decoder layer as the query and the high-embedded feature map as the key and value, cross-attention calculation is performed to select a set of high-key indexes and achieve height-guided localization. Using the height-guided localization object query vector as the query and the depth embedding feature map as the key and value, cross-attention calculation is performed, and a set of depth key indexes is selected to achieve depth refinement calibration; Using the object query vector after depth refinement and calibration as the query, and the multi-scale visual feature map as the key and value, the union of the height key index set and the depth key index set is used as the geometric index set. Cross attention is calculated only on the region specified by the geometric index set to achieve visual feature extraction. Output the updated object query vector.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the monocular 3D target detection method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the monocular 3D target detection method according to any one of claims 1-6.

Images