Vehicle intelligent driving system automatic upgrading method, device, equipment, medium and program product

Abstract

The application provides an automatic upgrading method and device of a vehicle intelligent driving system, electronic equipment, a computer storage medium and a computer program product, the vehicle intelligent driving system comprises a pre-trained basic model and is used for realizing an auxiliary driving function, the automatic upgrading method of the vehicle intelligent driving system comprises the following steps: acquiring vehicle data by using a data acquisition device of a vehicle, the vehicle data comprising vehicle driving data and vehicle environment data; preprocessing the vehicle data to generate preprocessed vehicle data; inputting the preprocessed vehicle data into the pre-trained basic model, outputting a result to realize the auxiliary driving function, and saving the preprocessed vehicle data; when the amount of the saved preprocessed vehicle data reaches a preset amount, upgrading the architecture of the pre-trained basic model to obtain a preliminary upgraded model; and training and optimizing the preliminary upgraded model by using the saved preprocessed vehicle data to obtain a target upgraded model to replace the pre-trained basic model.

Description

Technical Field

[0001] This application relates to the field of intelligent driving of vehicles, and in particular to automatic upgrade methods, devices, electronic devices, computer storage media and computer program products for intelligent driving systems of vehicles. Background Technology

[0002] In recent years, artificial intelligence has been significantly applied in Level 2 driver assistance systems, especially in perception and recognition modules. Through algorithms such as deep learning, vehicles can more accurately identify lane lines, traffic signs, pedestrians, and other vehicles, thereby improving the performance of functions such as adaptive cruise control and lane keeping assist.

[0003] However, for higher levels of autonomous driving (such as Level 4), the biggest challenge in applying large models lies in the efficiency and effectiveness of data acquisition. While massive amounts of data are the foundation for training high-performance models, high-quality, diverse data that covers rare corner cases is often difficult to obtain. To accelerate the commercialization of Level 4 autonomous driving, innovative strategies utilizing self-regulating and automatically deployed models must be adopted to improve data acquisition efficiency. Summary of the Invention

[0004] This application was made to solve the above-mentioned problems, and its purpose is to provide an automatic upgrade method, device, electronic device, computer storage medium and computer program product for a vehicle intelligent driving system. The automatic upgrade method for the vehicle intelligent driving system combines a pre-trained model and a large model, which improves data acquisition efficiency and ensures efficient training of data at different stages.

[0005] According to a first aspect of this application, an automatic upgrade method for a vehicle intelligent driving system is provided. The vehicle intelligent driving system includes a pre-trained base model for implementing assisted driving functions. The method includes: acquiring vehicle data using a vehicle data acquisition device, the vehicle data including vehicle driving data and vehicle environmental data; preprocessing the vehicle data to generate preprocessed vehicle data; inputting the preprocessed vehicle data into the pre-trained base model, outputting a result to implement assisted driving functions, and saving the preprocessed vehicle data; when the amount of saved preprocessed vehicle data reaches a preset amount, upgrading the architecture of the pre-trained base model to obtain a preliminary upgrade model; and training and optimizing the preliminary upgrade model using the saved preprocessed vehicle data to obtain a target upgrade model to replace the pre-trained base model.

[0006] In one possible implementation of the first aspect above, preprocessing the vehicle data to generate preprocessed vehicle data includes: data cleaning and format standardization of the collected vehicle data.

[0007] In one possible implementation of the first aspect described above, the data acquisition device includes at least a camera device and a lidar sensor.

[0008] In one possible implementation of the first aspect mentioned above, the pre-trained base model is a fusion neural network model based on the Transformer architecture.

[0009] In one possible implementation of the first aspect above, the architecture of the pre-trained base model is upgraded to obtain a preliminary upgraded model, which includes: expanding the number of layers of the pre-trained base model; increasing the attention head dimension of the pre-trained base model; and adding learnable gating weights to the fusion layer.

[0010] In one possible implementation of the first aspect above, the architecture of the pre-trained basic model is upgraded to obtain a preliminary upgraded model, which further includes: performing cross-modal fusion at layers P3-P5 of the feature pyramid respectively; and performing bilinear upsampling on the voxel features collected by the lidar sensor.

[0011] In one possible implementation of the first aspect above, training and optimizing the preliminary upgrade model using the saved preprocessed vehicle data to obtain a target upgrade model to replace the pre-trained base model includes: generating a first training dataset based on the saved preprocessed vehicle data; training the preliminary upgrade model according to the first training dataset to obtain training results; and evaluating the training results from multiple evaluation dimensions and iteratively adjusting based on the evaluation results to obtain the target upgrade model.

[0012] In one possible implementation of the first aspect mentioned above, a distributed training method is used during training.

[0013] In one possible implementation of the first aspect above, during training, training enhancement is performed using any one or more of the following methods: randomly masking a preset proportion of input data to force the model to learn complementary features; injecting noise into the point cloud data acquired by the lidar sensor; adding fog effects to the images acquired by the camera device; and randomly reversing the time series of the training data.

[0014] In one possible implementation of the first aspect mentioned above, multiple evaluation dimensions include: perception accuracy, real-time performance, and coverage of extreme scenarios.

[0015] In one possible implementation of the first aspect described above, a first training dataset is generated based on the saved preprocessed vehicle data, and multiple first training datasets with driving scenario focus are generated based on the saved preprocessed vehicle data.

[0016] In one possible implementation of the first aspect above, multiple target upgrade models are obtained by training multiple first training datasets, and the multiple target upgrade models are adapted to different driving scenarios respectively.

[0017] One possible implementation of the first aspect mentioned above includes: selecting an appropriate target upgrade model based on the user's high-frequency driving scenarios and deploying it.

[0018] According to a second aspect of this application, an automatic upgrade device for a vehicle intelligent driving system is provided. The intelligent driving system includes a pre-trained basic model for implementing assisted driving functions, comprising: a data acquisition module for acquiring vehicle data using a vehicle data acquisition device, the vehicle data including vehicle driving data and vehicle environmental data; a preprocessing module for preprocessing the vehicle data to generate preprocessed vehicle data; an output module for inputting the preprocessed vehicle data into the pre-trained basic model, outputting results to implement assisted driving functions, and saving the preprocessed vehicle data; an upgrade module for upgrading the architecture of the pre-trained basic model when the amount of saved preprocessed vehicle data reaches a preset amount, to obtain a preliminary upgrade model; and a training module for training and optimizing the preliminary upgrade model using the saved preprocessed vehicle data to obtain a target upgrade model to replace the pre-trained basic model.

[0019] According to a third aspect of this application, an electronic device is provided, the electronic device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded by the processor and executing the method described in one aspect above.

[0020] According to a fourth aspect of this application, a computer storage medium is provided, the storage medium storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded by a processor and executing the method described in one aspect above.

[0021] According to a fifth aspect of this application, a computer program product is provided, comprising at least one instruction or at least one program segment, wherein the at least one instruction or the at least one program segment is loaded by a processor and executes the method described in one aspect above. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the automatic upgrade method for a vehicle intelligent driving system provided in this application embodiment.

[0023] Figure 2 This is a structural block diagram of the automatic upgrade device for the vehicle intelligent driving system provided in the embodiments of this application.

[0024] Figure 3 This is a structural block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0026] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0027] One embodiment of this application provides an automatic upgrade method, device, electronic device, computer storage medium, and computer program product for a vehicle intelligent driving system. It can efficiently acquire data by combining with commercially available intelligent driving scenarios and continuously improve the performance of large models by utilizing self-learning and self-training methods, thereby providing users with an continuously upgraded vehicle intelligent driving system.

[0028] This specification provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server products, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0029] Figure 1 This is a flowchart illustrating the automatic upgrade method for a vehicle intelligent driving system provided in this application embodiment. The intelligent driving system equipped in this vehicle employs artificial intelligence technology to achieve Level 2 assisted driving functions. For example, by applying artificial intelligence technology to the perception and recognition module, the vehicle can collect environmental information through sensors such as cameras and radar, and analyze it using deep learning algorithms to identify key elements such as traffic lights, lane lines, other vehicles, and pedestrians. The automatic upgrade method for this vehicle intelligent driving system includes the following steps.

[0030] S101: Vehicle data is acquired using the vehicle's data acquisition device, including vehicle driving data and vehicle environmental data.

[0031] Data acquisition devices used for data collection include, for example, environmental sensors, which can be used to detect the vehicle and its surrounding environment to obtain vehicle driving data and vehicle environmental data. These environmental sensors include, for example, millimeter-wave radar, lidar, camera devices, infrared devices, and vehicle speed sensors. For instance, a camera device can detect the vehicle's lane and adjacent lanes, as well as the position, speed, and acceleration of all environmental objects around the vehicle. Millimeter-wave radar can detect the relative distance between the vehicle and other vehicles.

[0032] In some embodiments, the data acquisition device may include a vehicle positioning device. The positioning device allows the vehicle's location on a map to be obtained. Based on this information, the vehicle's lane and adjacent lanes can also be determined.

[0033] In some embodiments, the data acquisition device may include a vehicle communication device. By means of the communication device, information about the vehicle and its surrounding environment can be acquired. For example, information about the vehicle and its surrounding environment, such as the speed of the other vehicles, can be received from other vehicles via vehicle-to-vehicle communication. Alternatively, information can be received from roadside devices and / or remote servers via communication between the vehicle and a roadside device and / or a remote server.

[0034] S102: Preprocess the vehicle data to generate preprocessed vehicle data.

[0035] The collected vehicle data is preprocessed, including data cleaning and format standardization.

[0036] Specifically, since the collected data typically contains redundant information and noise, filtering algorithms (such as Kalman filtering) can be used to eliminate noise interference from the sensors before further processing, and outliers in radar point clouds or abnormal pixels in camera images can be identified. Abnormal trajectory points caused by sensor aging or environmental interference (such as rain and snow) can also be corrected, improving data reliability. Furthermore, since data is collected using multiple different sensors, heterogeneous data from different sensors (such as cameras, LiDAR, and millimeter-wave radar) needs to be converted into a unified format (such as point cloud coordinates or image matrices) to facilitate subsequent fusion processing. Timestamp alignment of the raw data is also necessary to address the time synchronization issues caused by differences in sampling frequencies among multiple sensors.

[0037] S103: Input the preprocessed vehicle data into the pre-trained base model, output the results to realize the assisted driving function, and save the preprocessed vehicle data.

[0038] In this embodiment, the intelligent driving system equipped in the vehicle employs artificial intelligence technology, including a pre-trained base model. This pre-trained base model has been well-trained and can achieve Level 2 assisted driving functions. Therefore, pre-processed vehicle data is first input into the pre-trained base model, and the output result enables low-level assisted driving functions. Simultaneously, the pre-processed vehicle data is saved to continuously accumulate data. Here, the pre-processed vehicle data can be stored in the cloud, allowing for the rapid collection of real-world vehicle data covering all road scenarios.

[0039] In some embodiments, the pre-trained base model is a fusion neural network model based on the Transformer architecture. This fusion neural network model based on the Transformer architecture fuses data acquired by the camera device and the LiDAR sensor; therefore, the data acquisition device in this case includes at least a camera device and a LiDAR sensor.

[0040] S104: When the amount of preprocessed vehicle data saved reaches the preset amount, the architecture of the pre-trained basic model is upgraded to obtain the initial upgraded model.

[0041] Whether an upgrade to the architecture of the pre-trained base model is needed to obtain a model suitable for a high-level autonomous driving system is determined by the amount of training data. Therefore, when the amount of pre-processed vehicle data stored in the cloud reaches a preset amount, it indicates that the data preparation for the model upgrade is complete and the upgrade can proceed. This model upgrade can be performed in the cloud. The preset amount can be set according to design needs; it can be a fixed value or a value that increases with a certain gradient.

[0042] Taking a fusion neural network model based on the Transformer architecture as an example, upgrades to the model architecture can include:

[0043] Expand the number of layers in the pre-trained base model;

[0044] Increase the attention head dimension of the pre-trained base model;

[0045] Add learnable gating weights to the fusion layer.

[0046] For example, the model layer depth can be increased to 8 layers or more, thereby increasing the number of intermodal feature interactions. The number of layers added here can be based on the amount of preprocessed vehicle data saved, and this application does not impose any limitations on this. For example, the multi-head dimension of the pre-trained base model can be increased to 128, expanding the cross-modal feature fusion capacity. Of course, the attention head dimension here is just an example, and this application does not impose any limitations on it. Furthermore, learnable gating weights can be added to the fusion layer to suppress the influence of noisy modalities.

[0047] In addition to the aforementioned model architecture upgrades, model architecture upgrades may also include:

[0048] Cross-modal fusion was performed at layers P3-P5 of the feature pyramid; and

[0049] Bilinear upsampling is performed on the voxel features acquired by the lidar sensor.

[0050] By performing cross-modal fusion at layers P3-P5 of the feature pyramid, perception capabilities can be optimized, balancing the recognition accuracy of fine-grained features with the reliability of recognizing large obstacles at long distances. In addition, compared with traditional single-stage fusion, the computational load is reduced by 30%, and the robustness of the model is improved.

[0051] Furthermore, to match the resolution of image features, bilinear upsampling can be performed on the voxel features acquired by the LiDAR sensor. First, the sparse voxel features are extracted through methods such as 3D convolution or voxel feature encoding. Then, a bilinear interpolation algorithm is used to interpolate the voxel features in three dimensions according to a preset upsampling ratio.

[0052] Furthermore, this application embodiment uses a fusion neural network model based on the Transformer architecture as an example for illustration, but other models can also be used, such as the BEVFormer model that fuses multiple camera devices or a combined model.

[0053] S105: Use the saved preprocessed vehicle data to train and optimize the initial upgrade model to obtain the target upgrade model to replace the pre-trained base model.

[0054] Specifically, training and optimizing the initial upgrade model using the saved preprocessed vehicle data may include the following sub-steps:

[0055] (1) Generate the first training dataset based on the saved preprocessed vehicle data.

[0056] (2) Train the initial upgrade model based on the first training dataset to obtain the training results.

[0057] Because of the massive amount of training data, distributed training can be used, which distributes different parts of the model to different computing devices for training. For example, data parallelism or model parallelism strategies can be used to support training of larger-scale models and datasets and improve training efficiency.

[0058] (3) Evaluate the training results from multiple evaluation dimensions, and make iterative adjustments based on the evaluation results to obtain the target upgrade model.

[0059] The training results are the training results obtained after training the initial upgrade model using the first training dataset. They may include the intermediate upgrade model obtained after adjusting the model parameters of the initial upgrade model, as well as the performance indicators of the intermediate upgrade model and the prediction results of the intermediate upgrade model on the test set.

[0060] The training results are evaluated from multiple evaluation dimensions, such as perception accuracy, real-time performance, and coverage of extreme scenes. The performance indicators and prediction results of the training results are evaluated from multiple dimensions to obtain an evaluation result.

[0061] Then, the intermediate upgrade model is iteratively adjusted based on the evaluation results. The above evaluation results can be used to guide the construction of training data in the model fine-tuning training process. The second training dataset can be reconstructed based on the first training dataset, thereby guiding the iterative training in the model fine-tuning process.

[0062] During the initial and iterative training of the upgraded model, one or more of the following methods may be used for training enhancement:

[0063] Randomly mask a preset proportion of input data to force the model to learn complementary features;

[0064] Noise is injected into the point cloud data acquired by the LiDAR sensor, and fog effects are added to the images acquired by the camera device; and

[0065] Randomly reverse the time series of training data.

[0066] By randomly masking a predetermined proportion of input data (i.e., "mask modeling"), the model can be forced to uncover undisturbed complementary features in the data by reconstructing the masked portions, thereby improving the model's understanding of the data's essential structure. Injecting noise or simulating fog effects into LiDAR point clouds and camera images can enhance model robustness; by appropriately controlling noise parameters, the model's generalization ability in complex environments can be improved without compromising data semantics. Randomly reversing the time series of training data can enhance the model's understanding of time-series data by changing the order of the time dimension.

[0067] In some embodiments, multiple first training datasets with driving scenario focuses can be generated based on the saved preprocessed vehicle data. For example, multiple first training datasets with driving scenario focuses can be generated based on scenario dimensions such as road type, traffic flow, special terrain, weather conditions, and key risk types. Examples include scenarios such as (urban roads, high traffic flow, rain and snow), (highway roads, low traffic flow, predominantly sunny), and (rural roads, low traffic flow, frequent day-night cycles, steep slopes). Furthermore, scenario dimensions can also include real-time requirements, cost sensitivity, etc. Here, "driving scenario focus" indicates that a certain driving scenario has a higher weight in the training dataset.

[0068] Then, the initial upgrade model is trained and optimized using multiple first training datasets with different driving scenario focuses, resulting in multiple target upgrade models adapted to different driving scenarios. Here, adapting to a specific driving scenario can be considered as achieving optimal performance under that scenario.

[0069] When deploying high-level autonomous driving models for individual users, a suitable target upgrade model can be selected and deployed based on the user's high-frequency driving scenarios. These high-frequency driving scenarios can be obtained through analysis of the user's driving records or based on the user's submitted requirements. For example, if a logistics company needs to deploy autonomous freight vehicles in a mountainous area with heavy snowfall in winter and high cost sensitivity, a suitable target upgrade model can be matched based on these requirements.

[0070] In some embodiments, a suitable target upgrade model can be selected and deployed based on the user's actual high-frequency driving scenarios and long-tail driving scenarios.

[0071] The automatic upgrade method for vehicle intelligent driving systems provided in this application leverages the self-learning and self-training characteristics of models. In commercial applications of intelligent driving, by combining real-world driving data and employing a self-supervised learning strategy, the model can continuously learn and optimize its performance from large amounts of data without manual annotation. This iterative learning process not only effectively improves the model's perception, prediction, and decision-making capabilities but also empowers users to continuously upgrade intelligent driving services, adapting to complex and ever-changing driving environments and ultimately enhancing user experience and commercial value.

[0072] Figure 2 This is a structural block diagram of the automatic upgrade device for a vehicle intelligent driving system provided in an embodiment of this application. Figure 2As shown, the automatic upgrade device 200 for a vehicle intelligent driving system includes a pre-trained basic model for implementing assisted driving functions. The automatic upgrade device 200 includes: a data acquisition module 201 for acquiring vehicle data using a vehicle data acquisition device, the vehicle data including vehicle driving data and vehicle environmental data; a preprocessing module 202 for preprocessing the vehicle data to generate preprocessed vehicle data; an output module 203 for inputting the preprocessed vehicle data into the pre-trained basic model, outputting results to implement assisted driving functions, and saving the preprocessed vehicle data; an upgrade module 204 for upgrading the architecture of the pre-trained basic model when the amount of saved preprocessed vehicle data reaches a preset amount, obtaining a preliminary upgrade model; and a training module 205 for training and optimizing the preliminary upgrade model using the saved preprocessed vehicle data to obtain a target upgrade model to replace the pre-trained basic model.

[0073] The specific implementation methods of the above modules are consistent with the specific implementation methods of each step in the embodiment of the automatic upgrade method for the vehicle intelligent driving system, and will not be repeated here.

[0074] Figure 3 This is a structural block diagram of the electronic device provided in the embodiments of this application. For example... Figure 3 As shown, this application also provides an electronic device 300, which includes a processor 301, a memory 302, a communication interface 303, and a bus 304. The processor 301, memory 302, and communication interface 303 are interconnected via the bus 304.

[0075] The memory 302 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store programs, which are then executed by the processor 301 when the program stored in the memory 302 is run. Figure 1 The steps of the method shown.

[0076] The processor 301 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, used to execute relevant programs to implement this application. Figure 3 The method shown.

[0077] The processor 301 can also be an integrated circuit chip with signal processing capabilities. In the implementation process, the embodiments of this application... Figure 1 Each step of the method can be accomplished through integrated logic circuits in the hardware of the processor 301 or through instructions in software form.

[0078] The processor 301 described above can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0079] The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 302. The processor 301 reads the information in memory 302 and, in conjunction with its hardware, completes the functions required by the units included in the device of this application. For example, it can execute... Figure 1 The various steps / functions of the illustrated embodiment.

[0080] The communication interface 303 can use, but is not limited to, transceivers to enable communication between the electronic device 300 and other devices or communication networks.

[0081] Bus 304 may include a pathway for transmitting information between various components of electronic device 300 (e.g., processor 301, memory 302, communication interface 303).

[0082] This application also provides a computer storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded by a processor and executed by the methods described in the above embodiments.

[0083] This application also provides a computer program product, including at least one instruction or at least one program segment, characterized in that the at least one instruction or the at least one program segment is loaded by a processor and executes the method described in the above embodiments.

[0084] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0085] Those skilled in the art will recognize that the modules, units, and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] Although this application has been described with reference to specific embodiments, those skilled in the art should recognize that the scope of the invention involved in this application is not limited to the technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An automatic upgrade method for a vehicle intelligent driving system, characterized in that, The vehicle intelligent driving system includes a pre-trained base model for implementing assisted driving functions, and the method includes: Vehicle data is acquired using a vehicle data acquisition device, and the vehicle data includes vehicle driving data and vehicle environmental data. The vehicle data is preprocessed to generate preprocessed vehicle data; The preprocessed vehicle data is input into the pre-trained base model, the output results are used to realize the assisted driving function, and the preprocessed vehicle data is saved. When the amount of preprocessed vehicle data saved reaches a preset amount, the architecture of the pre-trained basic model is upgraded to obtain a preliminary upgraded model; and The initial upgrade model is trained and optimized using the saved preprocessed vehicle data to obtain the target upgrade model to replace the pre-trained base model.

2. The automatic upgrade method for a vehicle intelligent driving system as described in claim 1, characterized in that, The vehicle data is preprocessed to generate preprocessed vehicle data, which includes: The collected vehicle data is cleaned and its format is standardized.

3. The automatic upgrade method for a vehicle intelligent driving system as described in claim 1 or 2, characterized in that, The data acquisition device includes at least a camera device and a lidar sensor.

4. The automatic upgrade method for a vehicle intelligent driving system as described in claim 3, characterized in that, The pre-trained base model is a fusion neural network model based on the Transformer architecture.

5. The automatic upgrade method for a vehicle intelligent driving system as described in claim 4, characterized in that, The architecture of the pre-trained base model is upgraded to obtain a preliminary upgraded model, which includes: Expand the number of layers in the pre-trained base model; Increase the attention head dimension of the pre-trained base model; and Add learnable gating weights to the fusion layer.

6. The automatic upgrade method for a vehicle intelligent driving system as described in claim 5, characterized in that, The architecture of the pre-trained base model is upgraded to obtain a preliminary upgraded model, which also includes: Cross-modal fusion was performed at layers P3-P5 of the feature pyramid; and Bilinear upsampling is performed on the voxel features acquired by the lidar sensor.

7. The automatic upgrade method for a vehicle intelligent driving system as described in claim 4, characterized in that, The initial upgrade model is trained and optimized using the saved preprocessed vehicle data to obtain a target upgrade model to replace the pre-trained base model, including: The first training dataset is generated based on the saved preprocessed vehicle data; The initial upgrade model is trained using the first training dataset to obtain training results; and The training results are evaluated from multiple evaluation dimensions, and iterative adjustments are made based on the evaluation results to obtain the target upgraded model.

8. The automatic upgrade method for a vehicle intelligent driving system as described in claim 7, characterized in that, During training, a distributed training approach is adopted.

9. The automatic upgrade method for a vehicle intelligent driving system as described in claim 7 or 8, characterized in that, During training, one or more of the following methods can be used for training enhancement: Randomly mask a preset proportion of input data to force the model to learn complementary features; Noise is injected into the point cloud data acquired by the lidar sensor, and fog effects are added to the images acquired by the camera device. as well as Randomly reverse the time series of training data.

10. The automatic upgrade method for a vehicle intelligent driving system as described in claim 7, characterized in that, Multiple evaluation dimensions include: perception accuracy, real-time performance, and coverage of extreme scenarios.

11. The automatic upgrade method for a vehicle intelligent driving system as described in claim 7, characterized in that, The first training dataset is generated based on the saved preprocessed vehicle data. Multiple first training datasets with driving scenario focuses are generated based on the saved preprocessed vehicle data.

12. The automatic upgrade method for a vehicle intelligent driving system as described in claim 11, characterized in that, Multiple target upgrade models are obtained by training multiple first training datasets, and these multiple target upgrade models are adapted to different driving scenarios.

13. The automatic upgrade method for a vehicle intelligent driving system as described in claim 12, characterized in that, include: Select the appropriate target upgrade model based on the user's high-frequency driving scenarios and deploy it.

14. An automatic upgrade device for a vehicle intelligent driving system, characterized in that, The intelligent driving system includes a pre-trained base model for implementing assisted driving functions, including: The data acquisition module is used to acquire vehicle data using the vehicle's data acquisition device. The vehicle data includes vehicle driving data and vehicle environmental data. The preprocessing module is used to preprocess the vehicle data to generate preprocessed vehicle data. The output module is used to input the preprocessed vehicle data into the pre-trained basic model, output the results to realize the assisted driving function, and save the preprocessed vehicle data. An upgrade module is used to upgrade the architecture of the pre-trained basic model when the amount of pre-processed vehicle data stored reaches a preset amount, thereby obtaining a preliminary upgraded model; and The training module is used to train and optimize the initial upgrade model using the saved preprocessed vehicle data to obtain the target upgrade model to replace the pre-trained base model.

15. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded by the processor and executed as described in any one of claims 1 to 13.

16. A computer storage medium, characterized in that, The storage medium stores at least one instruction or at least one program, which is loaded by a processor and executed according to any one of claims 1 to 13.

17. A computer program product comprising at least one instruction or at least a program segment, characterized in that, The at least one instruction or the at least one program segment is loaded by the processor and executed as described in any one of claims 1 to 13.

Images