BEV target detection method and system for automatic driving and medium
By adopting the attention mechanism of the convolutional neural network optimized by the Crown Porcupine and the detection model of the random inertial weight in the BEV perception algorithm, the problems of the BEV perception algorithm in model convergence and accuracy generalization are solved, and the operation efficiency and safety of autonomous driving vehicles are improved.
Patent Information
- Application Number
- CN202411777101.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-03-14
AI Technical Summary
The BEV perception algorithm has difficulty in model convergence and the difference in accuracy and generalization, especially the significant difference with the 2D perception algorithm, making it difficult to maintain the advantages of the original BEV perception algorithm.
The prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine is used to predict the 3D detection feature points of the target, and the BEV detection model of the target based on the random inertial weight and differential variation learning factor is constructed to detect the target.
The accuracy indicators and generalization performance of the BEV perception solution are improved, the gap between accuracy and generalization performance with the monocular perception solution is narrowed, and the operation efficiency and safety of autonomous driving vehicles are improved.
Smart Images
Figure CN119625274A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving technology, and in particular to a BEV target detection method, system and medium for autonomous driving. Background Art
[0002] In recent years, with the development of technology, vehicle-mounted sensors such as lidar, millimeter-wave radar, cameras, ultrasonic radar, etc. have been widely used in intelligent driving perception technology.
[0003] Traditional 2D perception algorithms are highly accurate and mature in technology, but covering a 360-degree view of the vehicle body often requires multiple cameras. The representation of different spatial features is different, which makes post-processing complex, and a single view has a large occlusion blind spot.
[0004] In order to integrate the representation of multiple sensors and unify features, BEV perception algorithms have become a hot topic of research. Such algorithms map multi-sensor features to the BEV space to obtain 360-degree perception results around the vehicle.
[0005] While the BEV perception algorithm has greatly improved the upper limit of perception capabilities, it also has two pain points: the commonly used detr-base solution has difficulty in model convergence; there are significant differences in generalization and accuracy compared with the 2D perception algorithm. How to align the accuracy of the 2D perception algorithm while maintaining the original advantages of the BEV perception algorithm has become a research difficulty. Summary of the invention
[0006] In view of the above problems, the present invention provides a BEV target detection method, system and medium for autonomous driving, which not only helps to improve the accuracy index and generalization performance, narrow the gap between BEV perception scheme and monocular perception scheme in accuracy and generalization performance, but also improves the operating efficiency and safety of autonomous driving vehicles.
[0007] In order to achieve the above-mentioned purpose and other related purposes, the technical solution provided by the present invention is as follows: a method for BEV target detection for autonomous driving, the method comprising:
[0008] U1. Collecting image data information of multiple viewing angles of the target and preprocessing it to obtain image data information of multiple viewing angles of the target after preprocessing;
[0009] U2. Based on the data information of the images of multiple perspectives of the preprocessed target, a prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine is used to predict the 3D detection feature points of the target, and the data information of the predicted 3D detection feature points of the target is obtained;
[0010] U3. Based on the data information of the predicted 3D detection feature points of the target, a BEV detection model of the target based on random inertia weight and differential variation learning factor is constructed, and the target is detected to obtain the detection data information of the target;
[0011] U4. Based on the detection data information of the target, a loss value function F of the target is established, and the loss value of the detection result of the target is calibrated to obtain data information of the loss value of the detection result of the target.
[0012] Furthermore, the method further comprises:
[0013] U5. Based on the data information of the loss value of the detection result of the target, a preset threshold is set. If the loss value of the detection result of the target is less than the preset threshold, the detection requirement is met. If the loss value of the detection result of the target is greater than the preset threshold, the detection requirement is not met, and the process returns to step U2 until the loss value of the detection result of the target is less than the preset threshold.
[0014] Furthermore, in step U2, the prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crested porcupine is used to predict the 3D detection feature points of the target, including:
[0015] U21. Input the data information of the preprocessed target images from multiple perspectives into the convolutional neural network for training and learning, initialize the weights and biases of the convolutional neural network, and obtain the data information of the weights and biases of the initialized convolutional neural network;
[0016] U22. Based on the weight and bias data information of the initialized convolutional neural network, the crested porcupine population is initialized, population parameters are determined, and data information of the initialized crested porcupine population is obtained;
[0017] U23. Based on the data information of the initialized crested porcupine population, establish the fitness function Q of the population individuals,
[0018]
[0019] Among them, x is the data information of the initialized crested porcupine population, α 1 , α 2 and α 3 is the fitness adjustment factor of the population individuals, and the fitness values of the population individuals are calculated to obtain the data information of the fitness values of the population individuals;
[0020] U24. Based on the data information of the fitness values of the individuals in the population, establish the optimization function W of the target,
[0021]
[0022] Among them, y is the data information of the fitness value of the population individual, β 1 , β 2 and β 3 The weights and biases of the convolutional neural network are optimized to obtain the optimized convolutional neural network.
[0023] Furthermore, the prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine to predict the 3D detection feature points of the target also includes:
[0024] U25. Based on the optimized convolutional neural network, the data information of the preprocessed target images from multiple perspectives is input, and the convolution features of the target image are extracted to obtain the data information of the convolution features of the target image;
[0025] U26. Based on the data information of the convolution feature of the target image, a prediction function R based on the attention mechanism is established.
[0026]
[0027] Among them, z is the data information of the convolution feature of the target image, δ 1 ,δ 2 and δ 3 is the attention mechanism factor of the target image, predicts the 3D detection feature points of the target, and obtains the data information of the predicted 3D detection feature points of the target.
[0028] Furthermore, the attention mechanism factor δ of the target image 1 ,δ 2 and δ 3 for,
[0029]
[0030] Among them, z is the data information of the convolution feature of the target image.
[0031] Furthermore, the optimization factor β of the population individuals 1 , β 2 and β 3 The constraint function f is,
[0032]
[0033] The value range of the constraint function f is (2,3).
[0034] Furthermore, in step U3, the BEV detection model of the target based on the random inertia weight and the differential variation learning factor is constructed, and the target is detected, including:
[0035] U31. Based on the data information of the predicted 3D detection feature points of the target, construct a target detection sequence function G based on random inertia weights,
[0036]
[0037] Among them, a is the data information of the 3D detection feature points of the predicted target, γ 1 , γ 2 and γ 3 is a random inertia weight, and the 3D detection feature points of the target are serialized to obtain data information of the 3D detection feature sequence of the target;
[0038] U32. Based on the data information of the 3D detection feature sequence of the target, a BEV detection function H based on the differential variation learning factor is established.
[0039]
[0040] Where b is the data information of the 3D detection feature sequence of the target, η 1 , η 2 and η 3 is the differential variation learning factor;
[0041] U33. Based on the BEV detection function H of the differential variation learning factor, the target is detected to obtain the detection data information of the target.
[0042] Furthermore, the target is detected to obtain the detection data information F of the target.
[0043] Among them, r is the detection data information of the target, λ 1 , 2 and λ 3 The detection factor of the target.
[0044] In order to achieve the above-mentioned objectives and other related objectives, the present invention also provides a system for BEV target detection for autonomous driving, including a computer device, which is programmed or configured to execute the steps of any one of the methods for BEV target detection for autonomous driving.
[0045] In order to achieve the above-mentioned purpose and other related purposes, the present invention also provides a computer-readable storage medium, on which is stored a computer program programmed or configured to execute any one of the BEV target detection methods for autonomous driving.
[0046] The present invention has the following positive effects:
[0047] 1. The present invention predicts the 3D detection feature points of the target by adopting a prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine, and combines the construction of a BEV detection model of the target based on random inertia weights and differential variation learning factors to detect the target, which not only helps to improve the accuracy index and generalization performance, narrow the gap in accuracy and generalization performance between the BEV perception scheme and the monocular perception scheme, but also improves the operating efficiency and safety of the autonomous driving vehicle.
[0048] 2. The present invention establishes the loss value function F of the target and calibrates the loss value of the target detection result, which is not only beneficial to the convergence of the detr-base type solution model, but also solves the problem that the difference in accuracy and generalization cannot be completely compensated and the degree of integration with the existing monocular perception technology is insufficient. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 It is a schematic diagram of the method flow of the present invention;
[0050] Figure 2 A schematic diagram of a prediction algorithm based on the attention mechanism of a convolutional neural network optimized for crested porcupines of the present invention;
[0051] Figure 3 A schematic diagram of a process for constructing a BEV detection model based on a random inertia weight and a differential variation learning factor according to the present invention;
[0052] Figure 4 It is a schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION
[0053] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0054] Example 1: Figure 1 As shown, a method for BEV target detection for autonomous driving, the method comprising:
[0055] U1. Collecting image data information of multiple viewing angles of the target and preprocessing it to obtain image data information of multiple viewing angles of the target after preprocessing;
[0056] U2. Based on the data information of the images of multiple perspectives of the preprocessed target, a prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine is used to predict the 3D detection feature points of the target, and the data information of the predicted 3D detection feature points of the target is obtained;
[0057] U3. Based on the data information of the predicted 3D detection feature points of the target, a BEV detection model of the target based on random inertia weight and differential variation learning factor is constructed, and the target is detected to obtain the detection data information of the target;
[0058] U4. Based on the detection data information of the target, a loss value function F of the target is established, and the loss value of the detection result of the target is calibrated to obtain data information of the loss value of the detection result of the target.
[0059] In this embodiment, the method further includes:
[0060] U5. Based on the data information of the loss value of the detection result of the target, a preset threshold is set. If the loss value of the detection result of the target is less than the preset threshold, the detection requirement is met. If the loss value of the detection result of the target is greater than the preset threshold, the detection requirement is not met, and the process returns to step U2 until the loss value of the detection result of the target is less than the preset threshold.
[0061] In this embodiment, if Figure 2 As shown, in step U2, the prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine is used to predict the 3D detection feature points of the target, including:
[0062] U21. Input the data information of the preprocessed target images from multiple perspectives into the convolutional neural network for training and learning, initialize the weights and biases of the convolutional neural network, and obtain the data information of the weights and biases of the initialized convolutional neural network;
[0063] U22. Based on the weight and bias data information of the initialized convolutional neural network, the crested porcupine population is initialized, population parameters are determined, and data information of the initialized crested porcupine population is obtained;
[0064] U23. Based on the data information of the initialized crested porcupine population, establish the fitness function Q of the population individuals,
[0065]
[0066] Among them, x is the data information of the initialized crested porcupine population, α 1 , α 2 and α 3is the fitness adjustment factor of the population individuals, and the fitness values of the population individuals are calculated to obtain the data information of the fitness values of the population individuals;
[0067] U24. Based on the data information of the fitness values of the individuals in the population, establish the optimization function W of the target,
[0068]
[0069] Among them, y is the data information of the fitness value of the population individual, β 1 , β 2 and β 3 The weights and biases of the convolutional neural network are optimized to obtain the optimized convolutional neural network.
[0070] In this embodiment, the prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine to predict the 3D detection feature points of the target also includes:
[0071] U25. Based on the optimized convolutional neural network, the data information of the preprocessed target images from multiple perspectives is input, and the convolution features of the target image are extracted to obtain the data information of the convolution features of the target image;
[0072] U26. Based on the data information of the convolution feature of the target image, a prediction function R based on the attention mechanism is established.
[0073]
[0074] Among them, z is the data information of the convolution feature of the target image, δ 1 ,δ 2 and δ 3 is the attention mechanism factor of the target image, predicts the 3D detection feature points of the target, and obtains the data information of the predicted 3D detection feature points of the target.
[0075] In this embodiment, the attention mechanism factor δ of the target image 1 ,δ 2 and δ 3 for,
[0076]
[0077] Among them, z is the data information of the convolution feature of the target image.
[0078] In this embodiment, the optimization factor β of the population individuals is 1 , β 2 and β 3 The constraint function f is,
[0079]
[0080] The value range of the constraint function f is (2,3).
[0081] Example 2: Based on the method of BEV target detection for autonomous driving in Example 1, the present invention is further illustrated and described below.
[0082] like Figure 1 As shown, a method for BEV target detection for autonomous driving, the method comprising:
[0083] U1. Collecting image data information of multiple viewing angles of the target and preprocessing it to obtain image data information of multiple viewing angles of the target after preprocessing;
[0084] U2. Based on the data information of the images of multiple perspectives of the preprocessed target, a prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine is used to predict the 3D detection feature points of the target, and the data information of the predicted 3D detection feature points of the target is obtained;
[0085] U3. Based on the data information of the predicted 3D detection feature points of the target, a BEV detection model of the target based on random inertia weight and differential variation learning factor is constructed, and the target is detected to obtain the detection data information of the target;
[0086] U4. Based on the detection data information of the target, a loss value function F of the target is established, and the loss value of the detection result of the target is calibrated to obtain data information of the loss value of the detection result of the target.
[0087] In this embodiment, if Figure 3 As shown, in step U3, the BEV detection model of the target based on the random inertia weight and the differential variation learning factor is constructed, and the target detection includes:
[0088] U31. Based on the data information of the predicted 3D detection feature points of the target, construct a target detection sequence function G based on random inertia weights,
[0089]
[0090] Among them, a is the data information of the 3D detection feature points of the predicted target, γ 1 , γ 2 and γ 3 is a random inertia weight, and the 3D detection feature points of the target are serialized to obtain data information of the 3D detection feature sequence of the target;
[0091] U32. Based on the data information of the 3D detection feature sequence of the target, a BEV detection function H based on the differential variation learning factor is established.
[0092]
[0093] Where b is the data information of the 3D detection feature sequence of the target, η 1 , η 2 and η 3 is the differential variation learning factor;
[0094] U33. Based on the BEV detection function H of the differential variation learning factor, the target is detected to obtain the detection data information of the target.
[0095] In this embodiment, the target is detected to obtain the detection data information F of the target.
[0096]
[0097] Among them, r is the detection data information of the target, λ 1 , 2 and λ 3 The detection factor of the target.
[0098] In this embodiment, the present invention provides a system for BEV target detection for autonomous driving, including a computer device programmed or configured to execute the steps of any one of the methods for BEV target detection for autonomous driving.
[0099] like Figure 4 As shown in the figure, the monocular 3D detection branch is added mainly for the detr-base type solution: 1) The images of multiple cameras are combined and sent to the image feature extractor after scaling, cropping and other processing.
[0100] 2) The image feature extractor obtains multiple image features. The image feature extractor is not limited to CNN and transformer backbone networks. The FPN pyramid network can be used to enhance image feature extraction.
[0101] 3) Use a monocular detection method such as yolox to extract 3D targets from multiple image features obtained in step 2
[0102] 4) Use the 3D target obtained in step 3 and the true value to calculate loss function 1.
[0103] 5) Use the 3D target in step 3 to generate target query 1 through the target query generator (the specific formula is as follows), and optionally use the method of adding noise to the true value to generate target query 2, which is combined with the randomly generated target query 3 to generate the final target query.
[0104] Function 1: C 3d =K i I i {x,y,z,1} T ;
[0105] Function 2: Q = PosEmbed(C 3d )+MLP(C score ,F C2d )
[0106] Where K i and I i are the external and internal parameters of camera i respectively. Function 1 is used to transform the monocular 3D target to BEV coordinates. PosEmbed is generally composed of sine transform and MLP layer, and ContextEmbed is also an MLP layer. Function 2 is used to obtain target query 1.
[0107] 6) Using the image features obtained in step 2 and the target query in step 4, the decoder features are obtained through the decoder.
[0108] 7) Use the 3D detection head to obtain the final 3D target result.
[0109] 8) Use the 3D target obtained in step 7 and the true value to calculate loss function 2.
[0110] 9) The final target loss is obtained by weighting loss function 1 and loss function 2.
[0111] In this embodiment, the present invention provides a computer-readable storage medium, on which is stored a computer program programmed or configured to execute any one of the BEV target detection methods for autonomous driving.
[0112] Any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).
[0113] In summary, the present invention not only helps to improve the accuracy index and generalization performance, narrow the gap between the BEV perception solution and the monocular perception solution in accuracy and generalization performance, but also improves the operating efficiency and safety of autonomous driving vehicles.
[0114] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for BEV target detection for autonomous driving, characterized in that: The method comprises: U1. Collecting image data information of multiple viewing angles of the target and preprocessing it to obtain image data information of multiple viewing angles of the target after preprocessing; U2. Based on the data information of the images of multiple perspectives of the preprocessed target, a prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine is used to predict the 3D detection feature points of the target, and the data information of the predicted 3D detection feature points of the target is obtained; U3. Based on the data information of the predicted 3D detection feature points of the target, a BEV detection model of the target based on random inertia weight and differential variation learning factor is constructed, and the target is detected to obtain the detection data information of the target; U4. Based on the detection data information of the target, a loss value function F of the target is established, and the loss value of the detection result of the target is calibrated to obtain data information of the loss value of the detection result of the target.
2. The method for BEV target detection for autonomous driving according to claim 1, characterized in that: The method further comprises: U5. Based on the data information of the loss value of the detection result of the target, a preset threshold is set. If the loss value of the detection result of the target is less than the preset threshold, the detection requirement is met. If the loss value of the detection result of the target is greater than the preset threshold, the detection requirement is not met, and the process returns to step U2 until the loss value of the detection result of the target is less than the preset threshold.
3. The method for BEV target detection for autonomous driving according to claim 1, characterized in that: In step U2, the prediction algorithm based on the attention mechanism of the convolutional neural network optimized by the crown porcupine is used to predict the 3D detection feature points of the target, including: U21. Input the data information of the preprocessed target images from multiple perspectives into the convolutional neural network for training and learning, initialize the weights and biases of the convolutional neural network, and obtain the data information of the weights and biases of the initialized convolutional neural network; U22. Based on the weight and bias data information of the initialized convolutional neural network, the crested porcupine population is initialized, population parameters are determined, and data information of the initialized crested porcupine population is obtained; U23. Based on the data information of the initialized crested porcupine population, establish the fitness function Q of the population individuals, Among them, x is the data information of the initialized crested porcupine population, α1, α2 and α3 are the fitness adjustment factors of the population individuals, and the fitness values of the population individuals are calculated to obtain the data information of the fitness values of the population individuals; U24. Based on the data information of the fitness values of the individuals in the population, establish the optimization function W of the target, Among them, y is the data information of the fitness value of the population individual, β1, β2 and β3 are the optimization factors of the population individual, and the weights and biases of the convolutional neural network are optimized to obtain the optimized convolutional neural network.
4. The method for BEV target detection for autonomous driving according to claim 3, characterized in that: The prediction algorithm using the attention mechanism of the convolutional neural network optimized by the crown porcupine to predict the 3D detection feature points of the target also includes: U25. Based on the optimized convolutional neural network, the data information of the preprocessed target images from multiple perspectives is input, and the convolution features of the target image are extracted to obtain the data information of the convolution features of the target image; U26. Based on the data information of the convolution feature of the target image, a prediction function R based on the attention mechanism is established. Among them, z is the data information of the convolution feature of the target image, δ1, δ2 and δ3 are the attention mechanism factors of the target image, and the 3D detection feature points of the target are predicted to obtain the data information of the predicted 3D detection feature points of the target.
5. The method for BEV target detection for autonomous driving according to claim 4, characterized in that: The attention mechanism factors δ1, δ2 and δ3 of the target image are, Among them, z is the data information of the convolution feature of the target image.
6. The method for BEV target detection for autonomous driving according to claim 3, characterized in that: The constraint function f of the optimization factors β1, β2 and β3 of the individuals in the population is, The value range of the constraint function f is (2,3).
7. The method for BEV target detection for autonomous driving according to claim 1, characterized in that: In step U3, the BEV detection model of the target based on the random inertia weight and the differential variation learning factor is constructed, and the target is detected including: U31. Based on the data information of the 3D detection feature points of the predicted target, a target detection sequence function G based on random inertia weight is constructed. Among them, a is the data information of the predicted 3D detection feature points of the target, γ1, γ2 and γ3 are random inertia weights, and the 3D detection feature points of the target are serialized to obtain the data information of the 3D detection feature sequence of the target; U32. Based on the data information of the 3D detection feature sequence of the target, a BEV detection function H based on the differential variation learning factor is established. Among them, b is the data information of the target's 3D detection feature sequence, η1, η2 and η3 are differential variation learning factors; U33. Based on the BEV detection function H of the differential variation learning factor, the target is detected to obtain the detection data information of the target.
8. The method for BEV target detection for autonomous driving according to claim 1, characterized in that: The target is detected to obtain the detection data information F of the target, Among them, r is the detection data information of the target, and λ1, λ2 and λ3 are the detection determining factors of the target.
9. A system for BEV target detection for autonomous driving, comprising a computer device, characterized in that: The computer device is programmed or configured to execute the steps of the method for BEV target detection for autonomous driving as described in any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program programmed or configured to execute the BEV target detection method for autonomous driving as described in any one of claims 1 to 8.