Improved monocular 3D detection method, device, equipment and medium

Abstract

The invention provides an improved monocular 3D detection method and device, equipment and a medium, and the method comprises the steps: obtaining a monocular picture, processing the monocular picture through employing an improved direct regression monocular 3D target detection algorithm, and obtaining a feature map and a preliminary detection result; the detection algorithm adopts a distance ratio loss function based on the ratio of a loss value to a true value as depth regression loss; selecting a key target based on the preliminary detection result, and constructing a local BEV space taking the key target as a center according to a preset BEV grid size; based on the feature map, performing refined single target regression on the key target in the local BEV space to obtain a refined key target result; and fusing the refined key target result with the preliminary detection result to obtain a final detection result. On the basis of a low-precision model, local BEV space conversion is carried out on the key target, and the regression precision of the key target is improved on the premise of increasing a small amount of calculation.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an improved monocular 3D detection method, device, equipment and medium. Background Art

[0002] Autonomous vehicles (AVs) rely primarily on technologies such as artificial intelligence (AI), visual computing, radar, GPS positioning, and structured road vehicle-infrastructure collaboration to enable environmental perception, path planning, and autonomous control. These capabilities are enabled by embedded edge computing terminals. Since 2020, the concept of Level 2 and Level 3 autonomous driving has become widespread within the industry. Unlike traditional human-driven vehicles, a core feature of autonomous vehicles is the application and dominance of AI technology. The driving process is achieved through computer-generated automated driving, which involves continuously collecting various driving information, analyzing it, and self-learning through machine learning or deep learning. This is a system engineering approach to computer-generated autonomous driving. Safe driving under all road conditions and circumstances, coupled with timely response to extreme situations, ensures the safety of the driver, the vehicle, and all other road users.

[0003] Currently, autonomous driving is a key area for the deployment and implementation of deep learning algorithms. Autonomous driving algorithms are undergoing rapid iteration and update, with a variety of novel algorithms being proposed in areas such as object detection, lane recognition, object tracking, semantic segmentation, dynamic object trajectory prediction, and multi-task learning. The core autonomous driving algorithm technology framework consists of three components: environmental perception, decision planning, and control execution. In recent years, with the increasing market acceptance of advanced assisted driving solutions, the requirements for autonomous driving perception algorithms have also become increasingly stringent. Monocular 3D is increasingly widely used, and the deployment of monocular 3D models based on low-computing chips is currently a hot topic.

[0004] The monocular 3D algorithms currently proposed in the industry often use high-precision models such as transformers and sparse projections, which are not conducive to deployment on ordinary chips, or use dense projections to BEV space for target prediction, which requires huge computational complexity. Currently, models that are easy to deploy with less computational complexity often have model accuracy issues. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide an improved monocular 3D detection method, device, equipment and medium. The embodiments of the present application are an improved two-stage model. Based on a low-precision model, a small BEV space conversion is performed on the key target, thereby improving the regression accuracy of the key target while increasing the amount of calculation less.

[0006] The present application provides the following technical solution: an improved monocular 3D detection method, comprising:

[0007] Obtain a monocular image and process the monocular image using an improved direct regression monocular 3D object detection algorithm to obtain a feature map and preliminary detection results; wherein the improved direct regression monocular 3D object detection algorithm uses a distance ratio loss function based on the ratio of the loss value to the true value as the depth regression loss;

[0008] Selecting a preset key target based on the preliminary detection result, and constructing a local BEV space centered on the key target according to a preset local BEV grid space size;

[0009] Based on the feature map, performing refined single-target regression on the key target in the local BEV space to obtain a refined key target result;

[0010] The refined key target results are fused with the preliminary detection results to obtain the final high-precision monocular 3D detection results.

[0011] According to one embodiment of the present application, the method further includes:

[0012] In the model detection head part of the improved direct regression monocular 3D object detection algorithm, a regression branch is added as a lateral distance regression head, so that the model directly learns and outputs the lateral offset of the target in the vehicle coordinate system.

[0013] According to an embodiment of the present application, the distance ratio loss function is: L1 loss function = abs(Z1-Z2) / Z1; Z1 is the target true value, Z2 is the target predicted value, and Z1-Z2 is the loss value.

[0014] According to an embodiment of the present application, based on the feature map, performing refined single-target regression on the key target in the local BEV space includes:

[0015] Through the perspective projection relationship, the BEV features of the local area corresponding to the local BEV space are obtained in the feature map, and single target regression is performed according to the BEV features of the local area to obtain the refined key target results.

[0016] According to an embodiment of the present application, obtaining the BEV features of the local area corresponding to the local BEV space in the feature map through a perspective projection relationship includes:

[0017] Using the intrinsic and extrinsic parameters of the monocular camera, a mapping relationship between image pixel coordinates and BEV grid coordinates of the local BEV space is established;

[0018] Obtaining the image pixel position corresponding to each coordinate point in the local BEV space on the feature map through the mapping relationship;

[0019] A feature vector of the image pixel position is extracted from the feature map by bilinear interpolation, and the feature vector is assigned to the local BEV space to obtain the BEV feature of the local area.

[0020] According to an embodiment of the present application, selecting a preset key target based on the preliminary detection result includes:

[0021] Based on the preliminary detection results and actual application requirements, key targets that require refined processing are screened out according to set screening criteria; the set screening criteria include distance, category, confidence level and location.

[0022] According to one embodiment of the present application, the preset local BEV grid space size is 1 / 10 of the full-map BEV grid space size.

[0023] The present application also provides an improved monocular 3D detection device, comprising:

[0024] A preliminary detection module is used to obtain a monocular image and process the monocular image using an improved direct regression monocular 3D object detection algorithm to obtain a feature map and preliminary detection results; wherein the improved direct regression monocular 3D object detection algorithm uses a distance ratio loss function based on the ratio of the loss value to the true value as the depth regression loss;

[0025] A local BEV space construction module is used to select a key target based on the preliminary detection result, and construct a local BEV space centered on the key target according to a preset local BEV grid space size;

[0026] A refined regression module, configured to perform refined single-target regression on the key target in the local BEV space based on the feature map to obtain a refined key target result;

[0027] The detection result output module is used to fuse the refined key target results with the preliminary detection results to obtain the final high-precision monocular 3D detection results.

[0028] The present application also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned improved monocular 3D detection method when executing the computer program.

[0029] The present application also provides a computer-readable storage medium storing a computer program for executing the above-mentioned improved monocular 3D detection method.

[0030] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least the following: the embodiments of the present invention adopt an improved two-stage model, which performs a small BEV space conversion on the key targets based on the low-precision model, thereby improving the regression accuracy of the key targets with a relatively small increase in computational complexity. Since the model does not use a special structure and the overall computational complexity is relatively small, it can be deployed on a low-cost, low-computing-power chip while meeting the model accuracy; and the model solution proposed in the embodiments of the present invention has a small computational load, a simple structure, and is easy to deploy. It has high accuracy for key targets and good practicality. The embodiments of the present invention achieve higher positioning accuracy in the local area, greatly reduce the computational complexity while ensuring the accuracy of close-range key targets, and improve the practicality and deployability of the perception module of the assisted driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 3D detection method according to an embodiment of the present invention;

[0033] Figure 2 3D detection device according to an embodiment of the present invention;

[0034] Figure 3 It is a structural diagram of the computer device of the present invention. DETAILED DESCRIPTION

[0035] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0036] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the features in the following embodiments and embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of this application.

[0037] like Figure 1 As shown, an embodiment of the present invention provides an improved monocular 3D detection method, including:

[0038] 101. Obtain a monocular image, and process the monocular image using an improved direct regression monocular 3D object detection algorithm to obtain a feature map and preliminary detection results; wherein the improved direct regression monocular 3D object detection algorithm uses a distance ratio loss function based on the ratio of the loss value to the true value as the depth regression loss;

[0039] 102. Select a preset key target based on the preliminary detection results, and construct a local BEV space centered on the key target according to a preset local BEV grid space size;

[0040] 103. Based on the feature map, perform refined single-target regression on the key target in the local BEV space to obtain a refined key target result;

[0041] 104. Fuse the refined key target results with the preliminary detection results to obtain the final high-precision monocular 3D detection results.

[0042] In some embodiments of the present invention, the method further includes: adding a regression branch as a lateral distance regression head to the model detection head part of the improved direct regression monocular 3D target detection algorithm, so that the model directly learns and outputs the lateral offset of the target in the vehicle coordinate system.

[0043] In this embodiment of the present invention, a monocular image is input into a direct regression monocular 3D object detection algorithm (features directly regress the depth, angle and other information of the 3D box without converting to BEV space). In order to improve the quality of preliminary detection, a range ratio loss function and a lateral range head design are proposed.

[0044] While the typical distance loss function uses the L1 loss function, this approach results in high overall depth regression accuracy, but compromises accuracy for closer objects. Assisted driving often focuses on closer critical targets, requiring higher depth prediction accuracy for nearby objects. Therefore, to address the short-range accuracy limitations of the traditional L1 loss in depth prediction, a new "distance ratio loss function abs(Z1-Z2) / Z1" is proposed, where Z1 is the true target value, Z2 is the predicted target value, and Z1-Z2 is the loss value. Its core concept is to convert absolute error into relative error relative to the true distance. Its key characteristic is that the loss value is inversely proportional to distance: for the same absolute error |Z1-Z2|, the smaller the true distance Z1 (the closer the object), the smaller the denominator Z1 and the larger the calculated loss value; the larger the true distance Z1 (the farther the object), the larger the denominator Z1 and the smaller the calculated loss value.

[0045] The distance loss function is calculated as abs(Z1-Z2) / Z1. By making the loss inversely proportional to distance, the loss caused by prediction errors at close range is significantly amplified. The model optimization process prioritizes reducing these large loss values. This allows the model to devote more attention and resources to improving the prediction accuracy of close-range targets, thereby enhancing the practicality and safety of the assisted driving system.

[0046] In addition, the general direct regression monocular 3D detection model obtains the target center point in the vehicle coordinate system by predicting the target depth and the target center point in the image, and performing an inverse perspective transformation using the internal and external parameters of the camera. It does not separately predict the lateral distance of the target, and the regression effect for occluded and truncated targets often has significant problems. Therefore, this embodiment designs a lateral distance regression head to directly regress the lateral distance, which significantly improves the lateral distance of the target for truncated and occluded objects (which can be determined by 2D and 3D center points).

[0047] Traditional methods are highly sensitive to errors in the model's predicted target depth Z (the distance along the optical axis in the camera coordinate system) and the 2D center point (u_c, v_c) on the image plane. The calculation of the lateral distance Y depends on both the prediction accuracy of the 2D center point v_c and the depth Z. The design of the lateral distance regression head in this embodiment adds an independent regression branch, allowing the model to directly learn and predict the target's lateral distance Y in the ego-vehicle coordinate system, thereby bypassing the error-prone geometric calculation chain and no longer relying solely on the predicted v_c and Z to calculate the lateral distance Y through a formula. This significantly improves the lateral positioning accuracy of truncated / occluded targets. The model directly learns the lateral distance Y using contextual information, effectively solving the problem of inaccurate lateral positioning of occluded and truncated targets caused by reliance on error-prone 2D center point and depth calculations in traditional monocular 3D detection. This reduces over-reliance on the prediction accuracy of a single 2D center point v_c, decouples the impact of depth error Z on lateral distance Y, and improves the robustness and practicality of the overall detection system in complex scenarios. Moreover, in this embodiment, adding a lightweight regression head has little impact on the overall model complexity and is easy to implement and deploy.

[0048] In some embodiments of the present invention, selecting preset key targets based on the preliminary detection results includes: based on the preliminary detection results and actual application requirements, according to set screening criteria, screening out key targets that require refined processing; the set screening criteria include distance, category, confidence and location.

[0049] In specific implementation, based on the preliminary detection results and application requirements (such as paying more attention to close-range targets in assisted driving), key targets that require refined processing are screened. The set screening criteria may include: (1) distance, which can select targets with an estimated depth Z less than a certain threshold (such as 30 meters); (2) category: select categories that are critical to safety (such as pedestrians, vehicles, and cyclists); (3) confidence: select targets with a high initial detection confidence; (4) location: select targets located on the vehicle's front path. In specific implementation, only a small number of key targets need to be subjected to subsequent refined processing with a larger computational workload, avoiding high-precision calculations for all detected targets, which can significantly improve efficiency.

[0050] In some embodiments of the present invention, based on the feature map, a refined single target regression is performed on the key target in the local BEV space, including: obtaining the BEV features of the local area corresponding to the local BEV space in the feature map through a perspective projection relationship, performing single target regression based on the BEV features of the local area, and obtaining the refined key target result.

[0051] In some embodiments of the present invention, the BEV features of the local area corresponding to the local BEV space are obtained in the feature map through the perspective projection relationship, including: using the intrinsic parameters and extrinsic parameters of the monocular camera to establish a mapping relationship between the image pixel coordinates and the BEV grid coordinates of the local BEV space; obtaining the image pixel position corresponding to each coordinate point in the local BEV space on the feature map through the mapping relationship; extracting the feature vector of the image pixel position from the feature map through bilinear interpolation, assigning the feature vector to the local BEV space, and obtaining the BEV features of the local area.

[0052] In specific implementation, the spatial position of the ego-vehicle of the key target is known. With this position as the center point, a small BEV space (N, N, C) is constructed, where (N, N) is the spatial size of the BEV grid and C is the number of BEV feature channels. (The size of N here, since the approximate position of the target is known, only 1 / 10 of the spatial size of the full-image BEV grid is required to achieve high accuracy.) Through perspective projection, the required BEV features are obtained from the stored image features. Single-target regression is then performed on the BEV features. This eliminates the need for anchor construction, resulting in less computational effort and the introduction of special structures such as transformers, making deployment easier. The computational effort can be reduced to 1 / 10 or even more of the full-image BEV. Small BEV grids are more likely to achieve high resolution (the physical size represented by the unit grid is smaller), thereby achieving higher positioning accuracy within a local area. While ensuring accuracy for key targets at close range, this significantly reduces computational complexity, improving the practicality and deployability of the assisted driving system's perception module.

[0053] In the improved direct regression monocular 3D target detection algorithm of the embodiment of the present invention, by designing a distance loss ratio function, the model focuses more on the regression of close-range key targets, and the depth accuracy of nearby objects is higher, thereby improving the overall practicality of the model. The lateral distance prediction of truncated objects and occluded objects is improved by the design of a lateral range head. On the other hand, the embodiment of the present invention is based on the design of a small BEV network based on preliminary detection results. Without introducing a special algorithm structure that is difficult to deploy and increasing a small amount of computation, it can significantly improve the angle, center point, length, width, height and other attributes of key targets.

[0054] like Figure 2 As shown, an embodiment of the present invention further provides an improved monocular 3D detection device 200, comprising:

[0055] A preliminary detection module 201 is configured to obtain a monocular image and process the monocular image using an improved direct regression monocular 3D object detection algorithm to obtain a feature map and preliminary detection results; wherein the improved direct regression monocular 3D object detection algorithm uses a distance ratio loss function based on the ratio of the loss value to the true value as the depth regression loss;

[0056] A local BEV space construction module 202 is configured to select a key target based on the preliminary detection result and construct a local BEV space centered on the key target according to a preset local BEV grid space size;

[0057] A refined regression module 203 is configured to perform refined single-target regression on the key target in the local BEV space based on the feature map to obtain a refined key target result;

[0058] The detection result output module 204 is used to fuse the refined key target result with the preliminary detection result to obtain a final high-precision monocular 3D detection result.

[0059] In some embodiments of the present invention, the preliminary detection module 201 is also used to: add a regression branch as a lateral distance regression head in the model detection head part of the improved direct regression monocular 3D target detection algorithm, so that the model directly learns and outputs the lateral offset of the target in the vehicle coordinate system.

[0060] In some embodiments of the present invention, the refined regression module 203 obtains the BEV features of the local area corresponding to the local BEV space in the feature map through a perspective projection relationship, performs single target regression based on the BEV features of the local area, and obtains the refined key target results. Specifically, the refined regression module 203 uses the intrinsic and extrinsic parameters of the monocular camera to establish a mapping relationship between image pixel coordinates and the BEV grid coordinates of the local BEV space; obtains the image pixel position corresponding to each coordinate point in the local BEV space on the feature map through the mapping relationship; extracts the feature vector of the image pixel position from the feature map through bilinear interpolation, assigns the feature vector to the local BEV space, and obtains the BEV features of the local area.

[0061] In some embodiments of the present invention, the local BEV space construction module 202 selects key targets that require refined processing based on the preliminary detection results and actual application requirements in accordance with set screening criteria; the set screening criteria include distance, category, confidence and location.

[0062] In one embodiment, a computer device is provided, such as Figure 3As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory 301 and executable on the processor 302. When the processor 302 executes the computer program, the improved monocular 3D detection method described above is implemented.

[0063] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0064] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned improved monocular 3D detection method.

[0065] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include transitory media such as modulated data signals and carrier waves.

[0066] Obviously, those skilled in the art should understand that the various modules or steps of the above-mentioned embodiments of the present invention can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices. Alternatively, they can be implemented using program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into separate integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module for implementation. Thus, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0067] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An improved monocular 3D detection method, characterized in that: include: Obtain a monocular image and process the monocular image using an improved direct regression monocular 3D object detection algorithm to obtain a feature map and preliminary detection results; wherein the improved direct regression monocular 3D object detection algorithm uses a distance ratio loss function based on the ratio of the loss value to the true value as the depth regression loss; Selecting a preset key target based on the preliminary detection result, and constructing a local BEV space centered on the key target according to a preset local BEV grid space size; Based on the feature map, performing refined single-target regression on the key target in the local BEV space to obtain a refined key target result; The refined key target results are fused with the preliminary detection results to obtain the final high-precision monocular 3D detection results.

2. The improved monocular 3D detection method according to claim 1, characterized in that: The method further comprises: In the model detection head part of the improved direct regression monocular 3D object detection algorithm, a regression branch is added as a lateral distance regression head, so that the model directly learns and outputs the lateral offset of the target in the vehicle coordinate system.

3. The improved monocular 3D detection method according to claim 1, characterized in that: include: The distance ratio loss function is: L1 loss function = abs(Z1-Z2) / Z1; Z1 is the target true value, Z2 is the target predicted value, and Z1-Z2 is the loss value.

4. The improved monocular 3D detection method according to claim 1, characterized in that: Based on the feature map, performing refined single-target regression on the key target in the local BEV space includes: Through the perspective projection relationship, the BEV features of the local area corresponding to the local BEV space are obtained in the feature map, and single target regression is performed according to the BEV features of the local area to obtain the refined key target results.

5. The improved monocular 3D detection method according to claim 4, characterized in that: Obtaining the BEV features of the local area corresponding to the local BEV space in the feature map through the perspective projection relationship includes: Using the intrinsic and extrinsic parameters of the monocular camera, a mapping relationship between image pixel coordinates and BEV grid coordinates of the local BEV space is established; Obtaining the image pixel position corresponding to each coordinate point in the local BEV space on the feature map through the mapping relationship; A feature vector of the image pixel position is extracted from the feature map by bilinear interpolation, and the feature vector is assigned to the local BEV space to obtain the BEV feature of the local area.

6. The improved monocular 3D detection method according to claim 1, characterized in that: Based on the preliminary test results, preset key targets are selected, including: Based on the preliminary detection results and actual application requirements, key targets that require refined processing are screened out according to set screening criteria; the set screening criteria include distance, category, confidence level and location.

7. The improved monocular 3D detection method according to claim 1, characterized in that: The preset local BEV grid space size is 1 / 10 of the full-map BEV grid space size.

8. An improved monocular 3D detection device, characterized in that: include: A preliminary detection module is used to obtain a monocular image and process the monocular image using an improved direct regression monocular 3D object detection algorithm to obtain a feature map and preliminary detection results; wherein the improved direct regression monocular 3D object detection algorithm uses a distance ratio loss function based on the ratio of the loss value to the true value as the depth regression loss; A local BEV space construction module is used to select a key target based on the preliminary detection result, and construct a local BEV space centered on the key target according to a preset local BEV grid space size; A refined regression module, configured to perform refined single-target regression on the key target in the local BEV space based on the feature map to obtain a refined key target result; The detection result output module is used to fuse the refined key target results with the preliminary detection results to obtain the final high-precision monocular 3D detection results.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the improved monocular 3D detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program for executing the improved monocular 3D detection method according to any one of claims 1 to 7.

Images