3D target detection system and method based on neural architecture search

Abstract

The invention belongs to the technical field of 3D target detection based on image and point cloud fusion, and particularly provides a 3D target detection system and method based on neural architecture search, and the system comprises a sparse BEV feature extraction module which is used for obtaining point cloud BEV features and image BEV features; the BEV feature mixing module is used for constructing a neural architecture search space by adopting a DARTS technology; the loss calculation module is used for obtaining the calculated amount loss of the neural architecture search space; the NASFusion model building module is used for carrying out 3D target detection based on the 3D sparse features and a neural architecture search space, and building an NASFusion model in combination with the calculated amount loss; and the target detection module is used for performing target detection on the to-be-detected target scene based on the NASFusion model to obtain a target detection result. According to the technical scheme, the optimal balance between the precision and the efficiency of the model is achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of 3D target detection based on image and point cloud fusion, and particularly relates to a 3D target detection system and method based on neural architecture search. BACKGROUND

[0002] 3D target detection based on image and point cloud fusion is a key task of automatic driving. In order to improve the perception ability of vehicles, many studies explore how to improve the detection accuracy.

[0003] 3D target detection based on multi-modal perception is a core technology of environmental perception in automatic driving. In recent years, with the successful application of the Transformer architecture in the field of computer vision, detection methods based on image and point cloud fusion have made significant progress. Mainstream methods such as IS-FUSION achieve multi-granularity feature fusion in the BEV space by designing hierarchical scene fusion (HSF) and instance-guided fusion (IGF) modules, significantly improving detection accuracy. However, such methods generally face the problem of low computational efficiency: on the one hand, a large number of query features (usually 100-400) need to be processed, and on the other hand, complex fusion modules bring heavy computational burden.

[0004] In view of efficiency optimization, current research mainly develops in two directions: feature selection optimization and network architecture optimization. In terms of feature selection, some people have proposed a progressive query pruning method, which significantly reduces the computational complexity by iteratively removing redundant queries with low classification scores. In terms of architecture optimization, PVNAS innovatively combines point-based and voxel-based methods and introduces neural architecture search technology to design an efficient PVConv basic unit. Despite this, existing methods still have obvious shortcomings in the efficiency optimization of multi-modal feature fusion, especially in the context of real-time requirements in automatic driving scenarios.

[0005] Therefore, an end-to-end 3D target efficient detection method NASFusio based on neural architecture search is proposed to ensure the best balance between accuracy and efficiency. SUMMARY

[0006] The application provides a 3D target detection system and method based on neural architecture search, aiming to solve the problems of low real-time performance and heavy computational burden in existing methods.

[0007] A 3D target detection system based on neural architecture search comprises:

[0008] A sparse BEV feature extraction module is used to extract 3D sparse features from the collected original point cloud and original image of a preset scene, and to extract BEV features based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features.

[0009] a BEV feature mixing module, configured to add the point cloud BEV feature and the image BEV feature to obtain an initial node, and use a DARTS technology to construct a neural architecture search space;

[0010] a loss calculation module, configured to embed a calculation amount loss supervision function into the neural architecture search space to obtain a calculation amount loss of the neural architecture search space;

[0011] a NASFusion model construction module, configured to perform 3D target detection based on 3D sparse features and the neural architecture search space, and construct a NASFusion model in combination with the calculation amount loss;

[0012] a target detection module, configured to perform target detection on a target scene to be detected based on the NASFusion model to obtain a target detection result.

[0013] Preferably, the sparse BEV feature extraction module comprises:

[0014] a voxel output unit, configured to obtain original point cloud voxels and original image voxels based on the acquired original point cloud and original image of a preset scene;

[0015] a 3D sparse feature extraction unit, configured to perform 3D sparse feature extraction on the original point cloud voxels and the original image voxels respectively to obtain 3D sparse features of the original point cloud voxels and 3D sparse features of the original image voxels;

[0016] a shared sparse feature extraction unit, configured to obtain 2D BEV sparse features based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels;

[0017] a BEV feature extraction unit, configured to perform feature extraction on the 2D BEV sparse features by using two BEV sparse feature extractors to obtain independent point cloud BEV features and image BEV features respectively.

[0018] Preferably, the BEV feature mixing module comprises:

[0019] an initial search space construction unit, configured to add the point cloud BEV feature and the image BEV feature to obtain an initial node, and obtain a preset number of ordered nodes in combination with a node convolution operation; and construct an initial search space based on the ordered nodes;

[0020] an architecture selection unit, configured to use a softmax function to convert a discrete architecture selection problem in the initial search space into a continuous optimization problem based on a DARTS method to obtain a search architecture of the initial search space;

[0021] The search optimization unit is configured to calculate a training loss in a search process, and optimize the search architecture and search architecture parameters by using an Adam method.

[0022] The enhancement unit is configured to update the optimized search architecture parameters based on indexes of the first two maximum values in the initial search space, and obtain final optimized search architecture parameters.

[0023] The neural architecture search space acquisition unit is configured to acquire the neural architecture search space based on the optimized search architecture and the final optimized search architecture parameters.

[0024] Preferably, each node in the ordered node has q candidate operations, and the ordered node is represented as N = {n1,..., nq}, wherein the following formula is used for each node: p

[0025]

[0026] wherein, and o j are vectors formed by and , and binary variable represents whether the candidate operation between nodes n i and n j is selected.

[0027] Preferably, the discrete architecture selection problem in the initial search space is represented by the following formula:

[0028]

[0029] The continuous optimization problem is represented by the following formula:

[0030]

[0031] s.t.||α j ||0=1,1≤j≤p,

[0032] wherein, α represents a search architecture, is a vector formed by , L train represents a training loss, ω and θ represent search architecture parameters, and F(ω, θ, α) represents a detection result. is a proportion weight of the kth candidate operation between nodes n i and n j .

[0033] The application further provides a 3D target detection method based on neural architecture search, and application of the system, comprising: ​

[0034] Perform 3D sparse feature extraction on the original point cloud and original image of the collected preset scene, and perform BEV feature extraction based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features;

[0035] The point cloud BEV features and image BEV features are added together as the initial node, and the DARTS technology is used to construct the neural architecture search space;

[0036] Embedding a computational loss supervision function into the neural architecture search space to obtain the computational loss of the neural architecture search space;

[0037] Based on 3D sparse features and neural architecture search space, 3D object detection is performed, and the NASFusion model is constructed in combination with the aforementioned computational loss.

[0038] The target scene to be detected is detected based on the NASFusion model to obtain a target detection result.

[0039] Preferably, the method for obtaining point cloud BEV features and image BEV features includes:

[0040] Based on the collected original point cloud and original image of the preset scene, original point cloud voxels and original image voxels are obtained;

[0041] Performing 3D sparse feature extraction on the original point cloud voxels and the original image voxels respectively to obtain 3D sparse features of the original point cloud voxels and 3D sparse features of the original image voxels;

[0042] Based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels, 2D BEV sparse features are obtained;

[0043] Two BEV sparse feature extractors are used to extract 2D BEV sparse features to obtain independent point cloud BEV features and image BEV features respectively.

[0044] Preferably, the method for constructing a neural architecture search space includes:

[0045] Adding the point cloud BEV features and the image BEV features as the initial node, combining the node convolution operation to obtain a preset number of ordered nodes; constructing the initial search space based on the ordered nodes;

[0046] Based on the DARTS method, a softmax function is used to convert the discrete architecture selection problem in the initial search space into a continuous optimization problem to obtain the search architecture of the initial search space;

[0047] calculating a training loss in a search process, optimizing the search architecture and search architecture parameters by using an Adam method;

[0048] updating the optimized search architecture parameters based on the indexes of the first two maximum values in the initial search space, to obtain final optimized search architecture parameters;

[0049] based on the optimized search architecture and the final optimized search architecture parameters, the neural architecture search space.

[0050] Compared with the prior art, the beneficial effects of the present application are: the present application uses a neural architecture search BEV feature mixing (NASFusion) module to replace the traditional single-modal-based BEV feature processing method, and uses a differentiable neural architecture search technology to optimize the model structure, without the need for additional reinforcement learning methods to automatically optimize the most suitable feature fusion module, effectively improving the detection accuracy of the model.

[0051] The comprehensive computation of the neural architecture search BEV feature mixing (NASFusion) module is added as part of the loss in the computational loss supervision function (FLS), and a lightweight BEV feature extraction space is constructed by combining the neural architecture search method. The computation and parameter amount of the BEV mixed feature module are better than those of the traditional single-modal BEV feature extraction method, and the detection accuracy is better than that of the baseline model, finally achieving a balance between detection accuracy and speed. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0053] Figure 1 The figure is a 3D target detection system architecture based on neural architecture search of the embodiment of the present application.

[0054] Figure 2 The figure is a visual comparison diagram of the NASFusion and SFD on the vehicle class of the KITTI validation set of the embodiment of the present application.

[0055] Figure 3 The figure is a visual comparison diagram of the NASFusion on the three classes of the KITTI validation set of the embodiment of the present application; wherein (a) is the SFD result; (b) is the NASFusion result.

[0056] Figure 4Figure 1 is a schematic diagram of BEV feature map visualization comparison of NASFusion and SFD on KITTI validation set according to an embodiment of the present application; wherein (a) is the BEV feature map processed by the NASF module in NASFusion; (b) is the BEV feature map processed by the 2D BEV feature extraction network in SFD; (c) is the classification BEV feature map in NASFusion; (d) is the classification BEV feature map in SFD. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0058] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0059] Embodiment one

[0060] A 3D target detection system (NASFusion) based on neural architecture search is an end-to-end 3D target efficient detection system based on neural architecture search, which comprises a sparse BEV feature extraction module, a BEV feature mixing module, a loss calculation module, a NASFusion model construction module and a target detection module. BEV (Bird's Eye View) is a bird's eye view.

[0061] The sparse BEV feature extraction module is used for 3D sparse feature extraction of the collected original point cloud and original image of the preset scene, and BEV feature extraction based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features.

[0062] Further embodiments are that the sparse BEV feature extraction module comprises:

[0063] The voxel output unit is used for obtaining original point cloud voxels and original image voxels based on the collected original point cloud and original image of the preset scene.

[0064] The 3D sparse feature extraction unit is used for 3D sparse feature extraction of the original point cloud voxels and the original image voxels respectively to obtain 3D sparse features of the original point cloud voxels and 3D sparse features of the original image voxels f pv and f′ pq are F Vand F Q Elements, where C V and C Q It's F V and F Q The number of channels, (X, Y, Z) is the grid size of the voxel feature output layer. Corresponding to F V and F Q The positions are V l and Q l , p v ∈V l and p q ∈Q l F V and F Q The locations of the non-empty voxel features in 3D space, and their coordinates are and

[0065] The shared sparse feature extraction unit is used to obtain 2D BEV sparse features based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels. Specifically, in order to project the 3D sparse features into the 2D BEV, the voxel feature output layer features of the two modalities are generated through the shared sparse feature layer. and

[0066] The BEV feature extraction unit is used to extract 2D BEV sparse features using two BEV sparse feature extractors to obtain independent point cloud BEV features and image BEV features respectively.

[0067] In this embodiment, 2D BEV sparse features and In the figure, the positions corresponding to R and P are R l and P l ,and and are the locations of non-empty voxel features in the BEV of point clouds and images, respectively. R is the number of channels of R, C P is the number of channels of P, and (X,Y) is the grid size of sparse BEV features. The calculation formula for BEV feature extraction is as follows:

[0068]

[0069] Among them, R l and P l Represent the corresponding positions of R and P respectively, p v and p q F V and F QThe location of the non-empty voxel feature in 3D space, V l and Q l Respectively represent the corresponding F V and F Q , and their coordinates are and and Represents F V and F Q Elements,

[0070]

[0071] The BEV feature mixing module (NASF module) is used to add the point cloud BEV feature R and the image BEV feature P as the initial node, and use the DARTS technology to construct the neural architecture search space.

[0072] In a further embodiment, the BEV characteristic mixing module includes:

[0073] The initial search space construction unit is used to add the point cloud BEV features and the image BEV features as the initial node, and combine the node convolution operation to obtain a preset number (N) of ordered nodes. Specifically, the initial node is the added feature. The subsequent nodes are the features added after some convolution operations on the previous nodes, such as Figure 1 As shown, the sequence number "0" represents the initial node. Based on the ordered nodes, the initial search space is constructed; a further implementation method is that in the ordered nodes, each node has q candidate operations, and the ordered nodes are represented as N = {n1,...,n p}, where for each node there is the following formula:

[0074]

[0075] in, and o j They are respectively and The vector formed by transposing the vector δ j and o j

[0076] The dot product of simplifies the mathematical expression of node calculation. Indicates whether to select node n i and n j Candidate operations between

[0077] The architecture selection unit is used for the DARTS (Differentiable Architecture Search) method, which uses the softmax function to convert the initial search space The discrete architecture selection problem in the initial search space is converted into a continuous optimization problem to obtain the search architecture of the initial search space. A further implementation method is that the discrete architecture selection problem in the initial search space is expressed by the following formula:

[0078]

[0079]

[0080] The continuous optimization problem is expressed as follows:

[0081]

[0082] st||α j ||0=1,1≤j≤p

[0083] Among them, α represents the search architecture, is The vector formed, L train represents the training loss, ω and θ represent the search architecture parameters, and F(ω,θ,α) represents the detection result. is node n i and n j The weight of the kth candidate operation between .

[0084] Search optimization unit, used to calculate the training loss L during the search process train , the Adam (gradient descent) method is used to optimize the search architecture and search architecture parameters; specifically, this embodiment adopts an end-to-end training strategy, selects the first 10 epochs as the search stage, searches for valuable BEV feature extraction space, and the next 40 epochs as the enhancement stage, fixes the BEV feature extraction space to enhance the detection performance of the model.

[0085] The enhancement unit is used to update the optimized search architecture parameters based on the indexes of the first two maximum values ​​in the initial search space to obtain the final optimized search architecture parameters; specifically, in the model enhancement stage, after the model has been trained for ε epochs, a fixed search space is selected. The indices α of the first two maximum values ​​in 1≤i≤j≤p * , continue to train the entire network and only update ω and θ, and finally save the optimized parameters ω * and θ * .

[0086] The neural architecture search space acquisition unit is used for the neural architecture search space based on the optimized search architecture and the finally optimized search architecture parameters.

[0087] The loss calculation module is used for embedding a calculation amount loss supervision function (FLS, the FLOPs loss supervision function) into the neural architecture search space to obtain the calculation amount loss of the neural architecture search space.

[0088] Specifically, to further realize the balance between precision and speed, the application also designs a calculation amount supervision loss function In the process of searching the space, a loss function is added to make the model tend to optimize to a lighter structure, and the loss supervision formula is as follows:

[0089]

[0090] L FLOPs =abs(FLOPs nas -FLOPs t )γ,

[0091] L train =βL FLOPs +(1-β)L baseline ,

[0092] Wherein, FLOPs nas is the total FLOPs calculated by the BEV feature mixing module according to the search structure alpha, is the FLOPs required by the kth candidate operation between nodes n i and n j ; is the proportion weight of the kth candidate operation between nodes n i and n j ; FLOPs t is the target lightweight calculation amount value, gamma is the attenuation coefficient, L baseline is the loss function of the baseline model, beta is the weight of L FLOPs participating in the calculation of the overall loss of the model. After completing the space search, the application sets no gradient of the parameter alpha participating in the calculation, so only L baseline is needed to update omega and theta, that is, L train =L baseline .

[0093] The NASFusion model construction module is configured to perform 3D target detection by using a detection head based on 3D sparse features and a neural architecture search space, and construct a NASFusion model in combination with the calculation loss.

[0094] The target detection module is configured to perform target detection on a target scene to be detected based on the NASFusion model, and obtain a target detection result.

[0095] Embodiment Two

[0096] The application further provides a 3D target detection method based on neural architecture search, and an application system, comprising:

[0097] 3D sparse feature extraction is performed on the collected original point cloud and original image of the preset scene, and BEV feature extraction is performed based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features.

[0098] The added point cloud BEV features and image BEV features are used as initial nodes, and a neural architecture search space is constructed by using DARTS technology.

[0099] A calculation loss supervision function is embedded into the neural architecture search space to obtain the calculation loss of the neural architecture search space.

[0100] 3D target detection is performed based on 3D sparse features and a neural architecture search space, and a NASFusion model is constructed in combination with the calculation loss.

[0101] Target detection is performed on a target scene to be detected based on the NASFusion model, and a target detection result is obtained.

[0102] Further embodiments are that the method for obtaining the point cloud BEV features and the image BEV features comprises:

[0103] Based on the collected original point cloud and original image of the preset scene, original point cloud voxels and original image voxels are obtained.

[0104] 3D sparse feature extraction is performed on the original point cloud voxels and the original image voxels respectively to obtain 3D sparse features of the original point cloud voxels and 3D sparse features of the original image voxels.

[0105] Based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels, 2D BEV sparse features are obtained;

[0106] Two BEV sparse feature extractors are used to extract the 2D BEV sparse features, and independent point cloud BEV features and image BEV features are obtained.

[0107] Further embodiments are directed to a method for constructing a neural architecture search space, comprising:

[0108] The added point cloud BEV features and image BEV features are used as initial nodes, combined with node convolution operations, and a preset number of ordered nodes are obtained; based on the ordered nodes, an initial search space is constructed;

[0109] Based on the DARTS method, the initial search space is converted into a continuous optimization problem using a softmax function, and a search architecture of the initial search space is obtained;

[0110] The training loss in the search process is calculated, and the Adam method is used to optimize the search architecture and the search architecture parameters;

[0111] Based on the indexes of the first two maximum values in the initial search space, the optimized search architecture parameters are updated, and the final optimized search architecture parameters are obtained;

[0112] Based on the optimized search architecture and the final optimized search architecture parameters, a neural architecture search space is obtained.

[0113] Embodiment three

[0114] The embodiment provides a specific application process of a 3D target detection system (NASFusion) based on neural architecture search.

[0115] The NASFusion algorithm first fuses image and point cloud features in the BEV space based on a differentiable neural architecture search method, referred to as a neural architecture search BEV feature mixing (NASF) module, and uses a gradient-based method to optimize the model. The operation space of the most suitable BEV mixed feature module is searched in the first 10 epochs, and training enhancement is performed in the last 40 epochs to effectively improve the detection accuracy of the model; in addition, the computational complexity loss supervision function (FLS) includes the computational complexity of the search space in the loss function for joint optimization, which promotes the model to optimize the operation space of the neural architecture search in the direction of lightweight, and after fixing the search space of the neural architecture search BEV feature mixing module, the model is fine-tuned in an end-to-end manner to improve the detection performance.

[0116] Figure 2The detection comparison results of NASFusion and SFD (baseline model) on the vehicle class of KITTI validation set are shown, and the blue box represents the detection result of NASFusion, the red box represents the true box, and the green box represents the detection result of the baseline model SFD. The first row shows the NASFusion result, the SFD result and the true box in the same image, the second row shows the NASFusion result and the true box in the same image, and the third row shows the SFD result and the true box in the same image. From the above, it can be seen that the NASFusion algorithm of the present application has good detection ability in the far distance and occlusion. Figure 4 It can be seen that the algorithm NASFusion of the present application has good detection ability in the far distance and occlusion.

[0117] Figure 3 The detection visualization results of the three classes of the NASFusion model on the KITTI validation set are shown, which shows that the algorithm NASFusion of the present application has good detection sensitivity for the pedestrian and bicycle classes. Among them, the blue box represents the detection result of NASFusion, the red box represents the true box, and the green box represents the detection result of the baseline model SFD. Among them, Figure 3 (a) is the SFD result; Figure 3 (b) is the NASFusion result; the figure shows that the algorithm NASFusion has good detection sensitivity for the pedestrian and bicycle classes.

[0118] Figure 4 The BEV feature map visualization comparison diagram of the embodiment NASFusion and SFD of the present application on the KITTI validation set is shown; among them, Figure 4 (a) is the BEV feature map processed by the NASF module in NASFusion; Figure 4 (b) is the BEV feature map processed by the 2D BEV feature extraction network in SFD; Figure 4 (c) is the classification BEV feature map in NASFusion; Figure 4 (d) is the classification BEV feature map in SFD; the above feature maps are visualized by channel summation, which shows that the NASF module in NASFusion can effectively enhance the representation ability of target features.

[0119] The 3D target efficient detection system NASFusion based on neural architecture search provided in the application achieves good detection performance on a large-scale automatic driving KITTI data set. On the KITTI verification set of the automobile category, especially on the simple detection difficulty, good detection performance is achieved. Among them, on the simple detection difficulty, BEV mAP(R40) is increased by +2.47%, BEVmAP(R11) is 97.47%, compared with the baseline, the detection result is improved by +7.19%. Through the comparison of the detection speed result, the BEV feature extraction module replaced by the algorithm NASFusion of the chapter is superior to the baseline model in the calculation amount and the parameter amount, the parameter amount is reduced by 56.4%, and the calculation amount is reduced by 15.34%. In addition, the calculation amount and the parameter amount of the complete model NASFusion are superior to the baseline model. Therefore, the algorithm NASFusion of the application realizes the effective balance of the detection accuracy and the speed, not only the detection accuracy is improved, but also the detection speed is superior to the baseline model, and has important research value.

[0120] The above-described embodiments are only descriptions of the preferred modes of the application and do not limit the scope of the application. Without departing from the design spirit of the application, various modifications and improvements to the technical solutions of the application made by those skilled in the art shall fall within the protection scope determined by the claims of the application.

Claims

1. A 3D object detection system based on neural architecture search, characterized in that: include: The sparse BEV feature extraction module is used to extract 3D sparse features from the original point cloud and original image of the collected preset scene, and perform BEV feature extraction based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features; The BEV feature mixing module is used to add the point cloud BEV features and image BEV features as the initial node and use the DARTS technology to build a neural architecture search space; A loss calculation module is used to embed a computational loss supervision function into the neural architecture search space to obtain the computational loss of the neural architecture search space; A NASFusion model building module is used to perform 3D object detection based on 3D sparse features and a neural architecture search space, and to build a NASFusion model in combination with the aforementioned computational loss. The target detection module is used to perform target detection on the target scene to be detected based on the NASFusion model to obtain a target detection result.

2. The system according to claim 1, wherein: The sparse BEV feature extraction module includes: A voxel output unit, configured to obtain original point cloud voxels and original image voxels based on the collected original point cloud and original image of the preset scene; a 3D sparse feature extraction unit, configured to perform 3D sparse feature extraction on the original point cloud voxels and the original image voxels, respectively, to obtain 3D sparse features of the original point cloud voxels and 3D sparse features of the original image voxels; A shared sparse feature extraction unit is used to obtain 2D BEV sparse features based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels; The BEV feature extraction unit is used to extract 2D BEV sparse features using two BEV sparse feature extractors to obtain independent point cloud BEV features and image BEV features respectively.

3. The system according to claim 1, wherein: The BEV characteristic hybrid module includes: An initial search space construction unit is used to add the point cloud BEV features and the image BEV features as the initial node, combine the node convolution operation, obtain a preset number of ordered nodes; and construct the initial search space based on the ordered nodes; An architecture selection unit, configured to convert the discrete architecture selection problem in the initial search space into a continuous optimization problem using a softmax function based on a DARTS method, and obtain a search architecture in the initial search space; A search optimization unit, used to calculate the training loss during the search process and optimize the search architecture and search architecture parameters using the Adam method; an enhancement unit, configured to update the optimized search architecture parameters based on the indexes of the first two maximum values ​​in the initial search space to obtain final optimized search architecture parameters; A neural architecture search space acquisition unit is used to obtain the neural architecture search space based on the optimized search architecture and the finally optimized search architecture parameters.

4. The system according to claim 3, characterized in that In the ordered nodes, each node has q candidate operations, and the ordered nodes are represented as N={n1, ..., n p }, where for each node there is the following formula: in, and o j There are and The vector formed, binary variable Indicates whether to select node n i and n j Candidate operations between 5. The system according to claim 4, characterized in that The discrete architecture selection problem in the initial search space is formulated as follows: The continuous optimization problem is expressed as follows: st||a j ||0=1,1≤j≤p, Among them, α represents the search architecture, is The vector formed, L train represents the training loss, ω and θ represent the search architecture parameters, and F(ω,θ,α) represents the detection result. is node n i and n j The weight of the kth candidate operation between .

6. A 3D object detection method based on neural architecture search, applying the system according to any one of claims 1 to 5, characterized in that: include: Perform 3D sparse feature extraction on the original point cloud and original image of the collected preset scene, and perform BEV feature extraction based on the extracted 3D sparse features to obtain point cloud BEV features and image BEV features; The point cloud BEV features and image BEV features are added together as the initial node, and the DARTS technology is used to construct the neural architecture search space; Embedding a computational loss supervision function into the neural architecture search space to obtain the computational loss of the neural architecture search space; Based on 3D sparse features and neural architecture search space, 3D object detection is performed, and the NASFusion model is constructed in combination with the aforementioned computational loss. The target scene to be detected is detected based on the NASFusion model to obtain a target detection result.

7. The method according to claim 6, characterized in that Methods for obtaining point cloud BEV features and image BEV features include: Based on the collected original point cloud and original image of the preset scene, original point cloud voxels and original image voxels are obtained; Performing 3D sparse feature extraction on the original point cloud voxels and the original image voxels respectively to obtain 3D sparse features of the original point cloud voxels and 3D sparse features of the original image voxels; Based on the 3D sparse features of the original point cloud voxels and the 3D sparse features of the original image voxels, 2D BEV sparse features are obtained; Two BEV sparse feature extractors are used to extract 2D BEV sparse features to obtain independent point cloud BEV features and image BEV features respectively.

8. The method according to claim 6, characterized in that Methods for constructing a neural architecture search space include: Adding the point cloud BEV features and the image BEV features as the initial node, combining the node convolution operation to obtain a preset number of ordered nodes; constructing the initial search space based on the ordered nodes; Based on the DARTS method, a softmax function is used to convert the discrete architecture selection problem in the initial search space into a continuous optimization problem to obtain the search architecture of the initial search space; Calculating the training loss during the search process and optimizing the search architecture and search architecture parameters using the Adam method; Based on the indexes of the first two maximum values ​​in the initial search space, updating the optimized search architecture parameters to obtain final optimized search architecture parameters; Based on the optimized search architecture and the finally optimized search architecture parameters, the neural architecture search space.

Images