A 3D object detection system and method based on neural architecture search

Abstract

The application belongs to the technical field of 3D target detection based on image and point cloud fusion, and specifically provides a 3D target detection system and method based on neural architecture search, comprising: a sparse BEV feature extraction module for obtaining point cloud BEV features and image BEV features; a BEV feature mixing module for constructing a neural architecture search space by using a DARTS technology; a loss calculation module for obtaining a calculation loss of the neural architecture search space; a NASFusion model construction module for performing 3D target detection based on 3D sparse features and the neural architecture search space, combining the calculation loss, and constructing a NASFusion model; and a target detection module for performing target detection on a target scene to be detected based on the NASFusion model and obtaining a target detection result. The technical scheme of the application realizes the best balance between precision and efficiency of the model.

Description

Technical Field

[0001] This invention belongs to the field of 3D object detection technology based on image and point cloud fusion, specifically relating to a 3D object detection system and method based on neural architecture search. Background Technology

[0002] 3D object detection based on image and point cloud fusion is a key task in autonomous driving. To improve vehicle perception capabilities, many studies have explored ways to enhance detection accuracy.

[0003] 3D object detection based on multimodal perception is a core technology for environmental perception in autonomous driving. In recent years, with the successful application of the Transformer architecture in computer vision, detection methods based on image and point cloud fusion have made significant progress. Mainstream methods such as IS-FUSION have significantly improved detection accuracy by designing hierarchical scene fusion (HSF) and instance-guided fusion (IGF) modules to achieve multi-granularity feature fusion in the BEV space. However, these methods generally face the problem of low computational efficiency: on the one hand, they need to process a large number of query features (usually 100-400); on the other hand, the complex fusion modules bring a heavy computational burden.

[0004] Current research on efficiency optimization mainly focuses on two directions: feature selection optimization and network architecture optimization. Regarding feature selection, a progressive query pruning method has been proposed, which iteratively removes redundant queries with low classification scores, significantly reducing computational complexity. In terms of architecture optimization, PVNAS innovatively combines point-based and voxel-based methods and introduces neural architecture search techniques to design efficient PVConv basic units. Nevertheless, existing methods still have significant shortcomings in optimizing the efficiency of multimodal feature fusion, especially in real-time-critical autonomous driving scenarios.

[0005] To address this, we propose NASFusio, an end-to-end, neural architecture search-based method for efficient 3D object detection, ensuring that the model achieves the best balance between accuracy and efficiency. Summary of the Invention

[0006] This invention provides a 3D target detection system and method based on neural architecture search, aiming to solve the problems of low real-time performance and heavy computational burden in existing methods.

[0007] A 3D object detection system based on neural architecture search includes:

[0008] The sparse BEV feature extraction module is used to extract 3D sparse features from the original point cloud and original image of the acquired preset scene, and to extract BEV features based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features.

[0009] The BEV feature fusion module is used to add point cloud BEV features and image BEV features as initial nodes and use DARTS technology to construct the neural architecture search space.

[0010] The loss calculation module is used to embed a computational loss supervision function into the neural architecture search space to obtain the computational loss of the neural architecture search space.

[0011] The NASFusion model building module is used to perform 3D object detection based on 3D sparse features and neural architecture search space, and to build the NASFusion model by combining the computational loss.

[0012] The target detection module is used to perform target detection on the scene to be detected based on the NASFusion model and obtain the target detection result.

[0013] Preferably, the sparse BEV feature extraction module includes:

[0014] The voxel output unit is used to obtain original point cloud voxels and original image voxels based on the original point cloud and original image of the acquired preset scene.

[0015] The 3D sparse feature extraction unit is used to extract 3D sparse features from the original point cloud voxels and the original image voxels respectively, so as to obtain the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels.

[0016] A shared sparse feature extraction unit is used to obtain 2D BEV sparse features based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels.

[0017] The BEV feature extraction unit is used to extract 2D BEV sparse features using two BEV sparse feature extractors, obtaining independent point cloud BEV features and image BEV features respectively.

[0018] Preferably, the BEV feature mixing module includes:

[0019] An initial search space construction unit is used to add the point cloud BEV features and the image BEV features as initial nodes, and then combine them with node convolution operations to obtain a preset number of ordered nodes; based on the ordered nodes, an initial search space is constructed.

[0020] An architecture selection unit, used in the DARTS-based method, transforms the discrete architecture selection problem in the initial search space into a continuous optimization problem using a softmax function, thereby obtaining the search architecture of the initial search space;

[0021] The search optimization unit is used to calculate the training loss during the search process and to optimize the search architecture and search architecture parameters using the Adam method.

[0022] An enhancement unit is used to update the optimized search architecture parameters based on the indices of the first two maximum values ​​in the initial search space, and obtain the final optimized search architecture parameters.

[0023] A neural architecture search space acquisition unit is used to obtain the neural architecture search space based on the optimized search architecture and the finally optimized search architecture parameters.

[0024] Preferably, in the ordered nodes, each node has q candidate operations, and the ordered nodes are represented as N = {n1, ..., n}. p}, where the following formula applies to each node:

[0025]

[0026] in, and o j They are respectively by and The resulting vector, a binary variable Indicates whether to select node n i and n j Candidate operations between

[0027] Preferably, the discrete architecture selection problem in the initial search space is represented by the following formula:

[0028]

[0029] A continuous optimization problem is represented by the following formula:

[0030]

[0031] st||α j ||0=1,1≤j≤p,

[0032] Where α represents the search architecture, It is by The resulting vector, L train F(ω,θ,α) represents the training loss, ω and θ represent the search architecture parameters, and F(ω,θ,α) represents the detection result. It is node n i and n j The weight of the k-th candidate operation between them.

[0033] This invention also provides a 3D object detection method based on neural architecture search, wherein the system includes:

[0034] 3D sparse features are extracted from the original point cloud and original image of the preset scene, and BEV features are extracted based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features.

[0035] The point cloud BEV features and image BEV features are added together and used as the initial nodes. The neural architecture search space is constructed using DARTS technology.

[0036] A computational loss supervision function is embedded into the neural architecture search space to obtain the computational loss of the neural architecture search space;

[0037] Based on 3D sparse features and neural architecture search space, 3D object detection is performed, and the NASFusion model is constructed by combining the aforementioned computational loss.

[0038] The target scene to be detected is detected based on the NASFusion model, and the target detection result is obtained.

[0039] Preferred methods for obtaining point cloud BEV features and image BEV features include:

[0040] Based on the collected original point cloud and original image of the preset scene, the original point cloud voxels and the original image voxels are obtained.

[0041] 3D sparse features are extracted from the original point cloud voxels and the original image voxels respectively to obtain the 3D sparse features of the original point cloud voxels and the original image voxels.

[0042] Based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels, 2D BEV sparse features are obtained.

[0043] Two BEV sparse feature extractors are used to extract 2D BEV sparse features, obtaining independent point cloud BEV features and image BEV features respectively.

[0044] Preferred methods for constructing the neural architecture search space include:

[0045] The point cloud BEV features and the image BEV features are added together to form the initial nodes. Then, a predetermined number of ordered nodes are obtained by combining node convolution operations. Based on the ordered nodes, an initial search space is constructed.

[0046] The DARTS-based method uses the softmax function to transform the discrete architecture selection problem in the initial search space into a continuous optimization problem, thereby obtaining the search architecture of the initial search space.

[0047] Calculate the training loss during the search process, and use the Adam method to optimize the search architecture and its parameters;

[0048] Based on the indices of the two largest values ​​in the initial search space, the optimized search architecture parameters are updated to obtain the final optimized search architecture parameters.

[0049] The neural architecture search space is based on the optimized search architecture and the final optimized search architecture parameters.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention uses a neural architecture search BEV feature fusion (NASF) module to replace the traditional single-modality BEV feature processing method. It uses differentiable neural architecture search technology to optimize the model structure, and automatically optimizes the most suitable feature fusion module without the need for additional reinforcement learning methods, thereby effectively improving the detection accuracy of the model.

[0051] The computational cost of the Neural Architecture Search (NASF) BEV Feature Hybridization module is incorporated into the computational supervision function (FLS) as part of the loss, thereby constructing a lightweight BEV feature extraction space in conjunction with the NASF method. This BEV hybrid feature module outperforms traditional single-modal BEV feature extraction methods in terms of both computational cost and parameter count, and achieves superior detection accuracy compared to the baseline model, ultimately achieving a balance between detection accuracy and speed. Attached Figure Description

[0052] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is an architecture diagram of a 3D object detection system based on neural architecture search according to an embodiment of the present invention;

[0054] Figure 2 This is a visual comparison diagram of NASFusion and SFD on the KITTI validation set for the vehicle category, according to an embodiment of the present invention.

[0055] Figure 3 This is a visual comparison diagram of NASFusion on the KITTI validation set for three categories according to an embodiment of the present invention; where (a) is the SFD result; and (b) is the NASFusion result.

[0056] Figure 4This is a schematic diagram showing the visualization comparison of BEV feature maps of NASFusion and SFD on the KITTI validation set according to an embodiment of the present invention; wherein, (a) is the BEV feature map processed by the NAF module in NASFusion; (b) is the BEV feature map processed by the 2D BEV feature extraction network in SFD; (c) is the BEV feature map of NASFusion; and (d) is the BEV feature map of SFD. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Example 1

[0060] NASFusion is an end-to-end, high-efficiency 3D object detection system based on neural architecture search. It includes a sparse BEV feature extraction module, a BEV feature mixing module, a loss calculation module, a NASFusion model building module, and an object detection module. BEV (Bird's Eye View) refers to a bird's-eye view.

[0061] The sparse BEV feature extraction module is used to extract 3D sparse features from the original point cloud and original image of the acquired preset scene, and to extract BEV features based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features.

[0062] A further implementation method includes a sparse BEV feature extraction module comprising:

[0063] The voxel output unit is used to obtain original point cloud voxels and original image voxels based on the original point cloud and original image of the acquired preset scene.

[0064] The 3D sparse feature extraction unit is used to extract 3D sparse features from the original point cloud voxels and the original image voxels, respectively, to obtain the 3D sparse features of the original point cloud voxels. and the 3D sparse features of the original image voxels f pv and f′ pq They are F Vand F Q The elements, where C V and C Q It is F V and F Q The number of channels, (X,Y,Z) is the grid size of the voxel feature output layer. Corresponding to F... V and F Q The positions are V l and Q l p v ∈V l and p q ∈Q l They are F V and F Q The non-empty voxel features are located in 3D space, and their coordinates are and

[0065] A shared sparse feature extraction unit is used to obtain 2D BEV sparse features based on the 3D sparse features of the original point cloud voxels and the original image voxels. Specifically, in order to project the 3D sparse features into the 2D BEV, the voxel feature output layer features of the two modalities are passed through the shared sparse feature layer to generate 2D BEV sparse features. and

[0066] The BEV feature extraction unit is used to extract 2D BEV sparse features using two BEV sparse feature extractors, obtaining independent point cloud BEV features and image BEV features respectively.

[0067] In this embodiment, 2D BEV sparse features and In the diagram, the positions corresponding to R and P are R and P, respectively. l and P l ,and and These represent the locations of non-empty voxel features in the BEV of the point cloud and the image, respectively. Where C... R It is the number of channels in R, C P Let P be the number of channels, and (X,Y) be the grid size of the sparse BEV feature. The formula for calculating BEV feature extraction is as follows:

[0068]

[0069] Among them, R l and P l These represent the corresponding positions of R and P, respectively. v and p q Representing F V and F QThe position of non-empty voxel features in 3D space, V l and Q l They respectively represent the corresponding F V and F Q The location, and their coordinates are and and Representing F respectively V and F Q elements,

[0070]

[0071] The BEV Feature Hybridization Module (NASF Module) is used to add the point cloud BEV features R and the image BEV features P as the initial node, and to construct the neural architecture search space using DARTS technology.

[0072] A further implementation method includes a BEV feature mixing module comprising:

[0073] The initial search space construction unit uses the sum of point cloud BEV features and image BEV features as the initial node. Combined with node convolution operations, this yields a preset number (N) of ordered nodes. Specifically, the initial node is the summed feature. Subsequent nodes utilize features summed from preceding nodes after some convolution operations, such as... Figure 1 As shown, the index "0" represents the initial node. Based on the ordered nodes, an initial search space is constructed; a further implementation method is that each node in the ordered nodes has q candidate operations, and the ordered nodes are represented as N = {n1,...,n...}. p}, where the following formula applies to each node:

[0074]

[0075] in, and o j They are respectively by and The resulting vector, through the transpose vector δ j and o j

[0076] The dot product simplifies the mathematical expression of node calculations. (Binary variables) Indicates whether to select node n i and n j Candidate operations between

[0077] Architecture selection unit, used for DARTS (Differentiable Architecture Search) based methods, uses the softmax function to initialize the search space. The discrete architecture selection problem is transformed into a continuous optimization problem, yielding the search architecture in the initial search space. A further implementation method is that the discrete architecture selection problem in the initial search space is expressed by the following formula:

[0078]

[0079]

[0080] A continuous optimization problem is represented by the following formula:

[0081]

[0082] st||α j ||0=1,1≤j≤p

[0083] Where α represents the search architecture, It is by The resulting vector, L train F(ω,θ,α) represents the training loss, ω and θ represent the search architecture parameters, and F(ω,θ,α) represents the detection result. It is node n i and n j The weight of the k-th candidate operation between them.

[0084] The search optimization unit is used to calculate the training loss L during the search process. train The Adam (gradient descent) method is used to optimize the search architecture and its parameters. Specifically, this embodiment adopts an end-to-end training strategy, selecting the first 10 epochs as the search phase to find valuable BEV feature extraction space, and the following 40 epochs as the enhancement phase to fix the BEV feature extraction space and enhance the detection performance of the model.

[0085] The enhancement unit updates the optimized search architecture parameters based on the indices of the two largest values ​​in the initial search space, thus obtaining the final optimized search architecture parameters. Specifically, in the model enhancement phase, after the model has been trained for ε epochs, a fixed search space is selected. The indices α of the first two maximum values ​​in 1≤i≤j≤p * Continue training the entire network, updating only ω and θ, and finally save the optimized parameters ω. * and θ * .

[0086] The neural architecture search space acquisition unit is used to obtain the neural architecture search space based on the optimized search architecture and the parameters of the final optimized search architecture.

[0087] The loss calculation module is used to embed a computational loss supervision function (FLS, the FLOPs loss supervision function) into the neural architecture search space to obtain the computational loss of the neural architecture search space. Embedding the computational loss supervision function not only improves detection accuracy compared to the baseline model but also significantly reduces the model's computational cost.

[0088] Specifically, to further achieve a balance between accuracy and speed, this invention also designs a computationally supervised loss function. During the search space process, add The loss function, which tends to optimize the model towards a more lightweight structure, has the following supervised loss formula:

[0089]

[0090] L FLOPs =abs(FLOPs) nas -FLOPs t )γ,

[0091] L train =βL FLOPs +(1-β)L baseline ,

[0092] Among them, FLOPs nas It is the sum of FLOPs calculated by the neural architecture search BEV feature fusion module based on the search structure α. It is node n i and n j The number of FLOPs required for the k-th candidate operation between; It is node n i and n j The weight of the k-th candidate operation between; FLOPs t It is the lightweight computational cost to achieve the target, γ is the attenuation coefficient, and L baseline It is the loss function of the baseline model, and β is L. FLOPs The weights involved in the overall loss calculation of the model. After completing the spatial search, this invention sets the gradients that do not participate in the calculation of parameter α, therefore only L needs to be calculated. baseline To update ω and θ, i.e., L train =L baseline .

[0093] The NASFusion model building module is used for 3D object detection using a detection head based on 3D sparse features and a neural architecture search space. Combined with the computational complexity loss, it constructs the NASFusion model. First, it replaces the traditional single-modal BEV feature processing method with an end-to-end model training and enhancement module based on a differentiable neural architecture search method for BEV feature mixing, automatically learning the most suitable BEV feature mixing module. Second, it proposes an embedded computational complexity loss supervision function, incorporating the computational complexity of the search space into the loss function for joint optimization, constructing a faster neural architecture search BEV feature mixing module.

[0094] The target detection module is used to perform target detection on the scene to be detected based on the NASFusion model and obtain the target detection result.

[0095] Example 2

[0096] This invention also provides a 3D object detection method based on neural architecture search, and an application system comprising:

[0097] 3D sparse features are extracted from the original point cloud and original image of the preset scene, and BEV features are extracted based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features.

[0098] Using the summed point cloud BEV features and image BEV features as initial nodes, a neural architecture search space is constructed using DARTS technology.

[0099] By embedding a computational loss supervision function into the neural architecture search space, the computational loss of the neural architecture search space can be obtained.

[0100] Based on 3D sparse features and neural architecture search space, 3D object detection is performed, and the NASFusion model is constructed by combining the aforementioned computational loss.

[0101] The target scene to be detected is detected based on the NASFusion model, and the target detection result is obtained.

[0102] A further implementation method for obtaining point cloud BEV features and image BEV features includes:

[0103] Based on the collected original point cloud and original image of the preset scene, the original point cloud voxels and the original image voxels are obtained.

[0104] 3D sparse features were extracted from the original point cloud voxels and the original image voxels respectively to obtain the 3D sparse features of the original point cloud voxels and the original image voxels.

[0105] Based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels, 2D BEV sparse features are obtained.

[0106] Two BEV sparse feature extractors are used to extract 2D BEV sparse features, obtaining independent point cloud BEV features and image BEV features respectively.

[0107] A further implementation method involves constructing a neural architecture search space, including:

[0108] The point cloud BEV features and image BEV features after addition are used as initial nodes. By combining node convolution operations, a preset number of ordered nodes are obtained. Based on the ordered nodes, an initial search space is constructed.

[0109] The DARTS-based method uses the softmax function to transform the discrete architecture selection problem in the initial search space into a continuous optimization problem, thereby obtaining the search architecture in the initial search space.

[0110] Calculate the training loss during the search process, and use the Adam method to optimize the search architecture and search architecture parameters;

[0111] Based on the indices of the two largest values ​​in the initial search space, update the optimized search architecture parameters to obtain the final optimized search architecture parameters.

[0112] The neural architecture search space is based on the optimized search architecture and the final optimized search architecture parameters.

[0113] Example 3

[0114] This embodiment provides a specific application process of the 3D object detection system (NASFusion) based on neural architecture search.

[0115] The NASFusion algorithm first fuses image and point cloud features in the BEV space based on a differentiable neural architecture search method, called the Neural Architecture Search BEV Feature Fusion (NASF) module, and then optimizes the model using a gradient-based approach. The most suitable operation space for the BEV feature fusion module is searched in the first 10 epochs, and then enhanced through training in the remaining 40 epochs, effectively improving the model's detection accuracy. Furthermore, a computational complexity loss supervision function (FLS) incorporates the computational complexity of the search space into the loss function for joint optimization, promoting a lightweight approach to optimizing the neural architecture search operation space. Finally, after fixing the search space of the neural architecture search BEV feature fusion module, end-to-end model fine-tuning is performed to improve detection performance.

[0116] Figure 2This display compares the detection results of NASFusion and SFD (baseline model) on the KITTI validation set for the car category. Blue boxes represent NASFusion detection results, red boxes represent ground truth bounding boxes, and green boxes represent the detection results of the baseline model SFD. The first row shows the NASFusion results, SFD results, and ground truth bounding boxes on the same image; the second row shows the NASFusion results and ground truth bounding boxes on the same image; and the third row shows the SFD results and ground truth bounding boxes on the same image. Figure 4 It can be seen that the NASFusion algorithm of this invention exhibits good detection capabilities in the case of distance and occlusion.

[0117] Figure 3 This visualization shows the detection results of the NASFusion model on the KITTI validation set for three categories, demonstrating the detection sensitivity of the NASFusion algorithm for pedestrians and bicycles. The blue boxes represent the detection results of NASFusion, the red boxes represent the ground truth boxes, and the green boxes represent the detection results of the baseline model SFD. Figure 3 (a) shows the SFD results; Figure 3 (b) shows the NASFusion results; this figure demonstrates the sensitivity of the NASFusion algorithm for detecting pedestrian and bicycle categories.

[0118] Figure 4 This is a schematic diagram showing a visual comparison of BEV feature maps of NASFusion and SFD on the KITTI validation set according to an embodiment of the present invention; wherein, Figure 4 (a) is the BEV feature map after processing by the NAF module in NASFusion; Figure 4 (b) is the BEV feature map after processing by the 2D BEV feature extraction network in SFD; Figure 4 (c) is the BEV feature map in NASFusion classification; Figure 4 (d) is the BEV feature map in SFD classification; the above feature maps are visualized by channel summation, which shows that the NAF module in NASFusion can effectively enhance the representation ability of target features.

[0119] This invention proposes a high-efficiency 3D target detection system, NASFusion, based on neural architecture search, which achieves good detection performance on the large-scale autonomous driving KITTI dataset. It demonstrates excellent detection performance, particularly on the easy detection difficulty level for the car category in the KITTI validation set. Specifically, on the easy detection difficulty level, the BEV mAP(R40) is improved by +2.47%, and the BEV mAP(R11) reaches 97.47%, representing a +7.19% improvement compared to the baseline. Through comparison of detection speed, the BEV feature extraction module replaced by the NASFusion algorithm outperforms the baseline model in both computational cost and parameter count, with a 56.4% reduction in parameters and a 15.34% reduction in computational cost. Furthermore, the complete NASFusion model also outperforms the baseline model in both computational cost and parameter count. Therefore, the NASFusion algorithm of this invention achieves an effective balance between detection accuracy and speed, improving not only detection accuracy but also outperforming the baseline model in detection speed, making it of significant research value.

[0120] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A 3D object detection system based on neural architecture search, characterized in that, include: The sparse BEV feature extraction module is used to extract 3D sparse features from the original point cloud and original image of the acquired preset scene, and to extract BEV features based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features. The BEV feature fusion module is used to add point cloud BEV features and image BEV features as initial nodes and use DARTS technology to construct the neural architecture search space. The loss calculation module is used to embed a computational loss supervision function into the neural architecture search space to obtain the computational loss of the neural architecture search space. The NASFusion model building module is used to perform 3D object detection based on 3D sparse features and neural architecture search space, and to build the NASFusion model by combining the computational loss. The target detection module is used to perform target detection on the scene to be detected based on the NASFusion model and obtain the target detection result.

2. The system according to claim 1, characterized in that, The sparse BEV feature extraction module includes: The voxel output unit is used to obtain original point cloud voxels and original image voxels based on the original point cloud and original image of the acquired preset scene. The 3D sparse feature extraction unit is used to extract 3D sparse features from the original point cloud voxels and the original image voxels respectively, so as to obtain the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels. A shared sparse feature extraction unit is used to obtain 2D BEV sparse features based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels. The BEV feature extraction unit is used to extract 2D BEV sparse features using two BEV sparse feature extractors, obtaining independent point cloud BEV features and image BEV features respectively.

3. The system according to claim 1, characterized in that, The BEV feature mixing module includes: An initial search space construction unit is used to add the point cloud BEV features and the image BEV features as initial nodes, and then combine them with node convolution operations to obtain a preset number of ordered nodes; based on the ordered nodes, an initial search space is constructed. An architecture selection unit, used in the DARTS-based method, transforms the discrete architecture selection problem in the initial search space into a continuous optimization problem using a softmax function, thereby obtaining the search architecture of the initial search space; The search optimization unit is used to calculate the training loss during the search process and to optimize the search architecture and search architecture parameters using the Adam method. An enhancement unit is used to update the optimized search architecture parameters based on the indices of the first two maximum values ​​in the initial search space, and obtain the final optimized search architecture parameters. A neural architecture search space acquisition unit is used to obtain the neural architecture search space based on the optimized search architecture and the finally optimized search architecture parameters.

4. The system according to claim 3, characterized in that, In the ordered nodes, each node has q candidate operations, and the ordered nodes are represented as N = {n1, ..., n}. p }, where the following formula applies to each node: in, and o j They are respectively due to and The resulting vector, a binary variable Indicates whether to select node n i and n j Candidate operations between 5. The system according to claim 4, characterized in that, The discrete architecture selection problem in the initial search space is represented by the following formula: A continuous optimization problem is represented by the following formula: st||a j ||0=1,1≤j≤p, Where α represents the search architecture, It is by The resulting vector, L train F(ω,θ,α) represents the training loss, ω and θ represent the search architecture parameters, and F(ω,θ,α) represents the detection result. It is node n i and n j The weight of the k-th candidate operation between them.

6. A 3D object detection method based on neural architecture search, using the system described in any one of claims 1-5, characterized in that, include: 3D sparse features are extracted from the original point cloud and original image of the preset scene, and BEV features are extracted based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features. The point cloud BEV features and image BEV features are added together and used as the initial nodes. The neural architecture search space is constructed using DARTS technology. A computational loss supervision function is embedded into the neural architecture search space to obtain the computational loss of the neural architecture search space; Based on 3D sparse features and neural architecture search space, 3D object detection is performed, and the NASFusion model is constructed by combining the aforementioned computational loss. The target scene to be detected is detected based on the NASFusion model, and the target detection result is obtained.

7. The method according to claim 6, characterized in that, Methods for obtaining point cloud BEV features and image BEV features include: Based on the collected original point cloud and original image of the preset scene, the original point cloud voxels and the original image voxels are obtained. 3D sparse features are extracted from the original point cloud voxels and the original image voxels respectively to obtain the 3D sparse features of the original point cloud voxels and the original image voxels. Based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels, 2D BEV sparse features are obtained. Two BEV sparse feature extractors are used to extract 2D BEV sparse features, obtaining independent point cloud BEV features and image BEV features respectively.

8. The method according to claim 6, characterized in that, Methods for constructing neural architecture search spaces include: The point cloud BEV features and the image BEV features are added together to form the initial nodes. Then, a predetermined number of ordered nodes are obtained by combining node convolution operations. Based on the ordered nodes, an initial search space is constructed. The DARTS-based method uses the softmax function to transform the discrete architecture selection problem in the initial search space into a continuous optimization problem, thereby obtaining the search architecture of the initial search space. Calculate the training loss during the search process, and use the Adam method to optimize the search architecture and its parameters; Based on the indices of the two largest values ​​in the initial search space, the optimized search architecture parameters are updated to obtain the final optimized search architecture parameters. The neural architecture search space is based on the optimized search architecture and the final optimized search architecture parameters.

Images