An intelligent driving system, an autonomous learning method thereof, and a storage medium

Abstract

The application discloses an intelligent driving system and an autonomous learning method thereof and a storage medium, relates to the technical field of intelligent driving, and comprises the following steps: constructing a feature extraction network model, extracting current driving scene features, and acquiring a decision vector based on a pre-established scene feature-decision mapping relationship; designing and training an integrated neural network model to acquire migration features; constructing a self-encoding neural network model to obtain new data features, and after the new data features, the migration features and the decision vector are fused, the fused feature expression is optimized to obtain an optimized self-encoding neural network model, and new new data features are output; and based on an online learning algorithm and an incremental updating algorithm, the feature extraction network model, the integrated neural network model and the self-encoding neural network model are updated and optimized through real-time monitoring and dynamic adjustment. The application can enhance the adaptability to new scenes and the real-time response and self-adjustment capability to complex environments.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, specifically to an intelligent driving system and its autonomous learning method and storage medium. Background Technology

[0002] In recent years, autonomous driving technology has been developing rapidly. With research in areas such as environmental perception, path planning, and decision control, autonomous driving systems have continuously improved in terms of perception accuracy, decision efficiency, and environmental adaptability. However, traditional autonomous driving systems still face a series of challenges when dealing with large-scale and complex environmental data. Existing systems typically rely on classical data processing and machine learning algorithms for environmental perception, path planning, and decision control. However, in practical applications, with the diversification of driving scenarios and the dramatic increase in the amount of data collected by sensors, traditional methods are prone to computational bottlenecks when processing this massive amount of data, leading to prolonged system response time and even the inability to make timely decisions.

[0003] Currently, efficiently processing and analyzing terabytes of data has become a significant bottleneck for system performance. Therefore, efficiently processing massive amounts of data and achieving high-precision perception and decision-making in complex environments has become a key issue in the field of autonomous driving.

[0004] In related technologies, based on the continuous development of deep learning technology, end-to-end learning methods are gradually being applied. By using a unified deep neural network, driving strategies can be learned directly from raw sensor data, eliminating the need for manual feature extraction and independent model design in the middle.

[0005] However, the above methods still perform poorly in the face of complex and ever-changing environments, and have problems such as poor environmental adaptability, insufficient perception accuracy and low decision-making efficiency. Summary of the Invention

[0006] This application provides an intelligent driving system and its autonomous learning method and storage medium, which can solve the technical problem of poor environmental adaptability in the prior art.

[0007] Firstly, this application provides an autonomous learning method for an intelligent driving system, the method comprising:

[0008] A multi-scale, multi-view feature extraction network model is constructed to extract the features of the current driving scene based on real-time driving data, and to obtain the decision vector based on the pre-established scene feature-decision mapping relationship.

[0009] Design and train an integrated neural network model that integrates adaptive feature selection and multi-scale feature extraction to obtain transfer features based on the aforementioned real-time driving data;

[0010] An autoencoder neural network model is constructed to compress and represent the real-time driving data to obtain new data features. The new data features, transfer features and decision vectors are then fused and the fused feature expression is optimized to obtain an optimized autoencoder neural network model and output new new data features.

[0011] Using the new data features as input, and based on online learning algorithms and incremental update algorithms, the aforementioned feature extraction network model, ensemble neural network model, and autoencoder neural network model are updated and optimized through real-time monitoring and dynamic adjustment.

[0012] Design and train an ensemble neural network model that integrates adaptive feature selection and multi-scale feature extraction, specifically including:

[0013] The parallel convolutional structure of the Inception module is combined with the residual connections of ResNet to form the basic network architecture;

[0014] In the above basic network architecture, dilated causal convolutional layers are introduced and a gated spatiotemporal attention mechanism is integrated to construct an integrated neural network architecture that integrates adaptive feature selection and multi-scale feature extraction.

[0015] The above-mentioned ensemble neural network architecture was pre-trained using a large-scale image dataset and then fine-tuned using a driving scenario database to obtain a trained ensemble neural network model.

[0016] Constructing an autoencoder neural network model specifically includes:

[0017] Design an unsupervised neural network with an encoder-decoder structure, where the encoder is used to compress the input data into a low-dimensional latent space, and the decoder is used to map the low-dimensional latent representation back to the original data space.

[0018] Based on a driving scenario database, the above-mentioned unsupervised neural network is trained by minimizing the reconstruction error to obtain the above-mentioned autoencoder neural network model.

[0019] Multi-scale, multi-view feature extraction network models include:

[0020] A multi-scale, multi-view coding module is used to synchronously process sensor data of different scales and perspectives and output basic coding features;

[0021] The global and local information extraction module is used to input the basic encoded features and extract features, and output a scene feature representation with global and local consistency.

[0022] The cross-attention fusion module is used to input the scene feature representation, perform feature fusion, and output a comprehensive feature representation;

[0023] A multi-layer encoder is used to input the comprehensive feature representation and extract feature representations from low to high levels in a hierarchical manner, outputting generalized driving scene features.

[0024] In conjunction with the first aspect, in one implementation method, a scene feature-decision mapping relationship is pre-established, specifically including:

[0025] Based on a driving scenario database, a multi-scale, multi-view feature extraction network model is used to output generalized driving scenario features. The aforementioned generalized driving scenario features are general feature representations that are extracted and formed by the multi-scale, multi-view feature extraction network model and have global and local consistency and are applicable to diverse driving scenarios.

[0026] Based on the generalized driving scenario characteristics mentioned above, and using self-supervised learning, training and feedback are conducted through simulated driving scenarios to optimize the correlation between scenario features and driving decisions, and to establish a scenario feature-decision mapping relationship.

[0027] In conjunction with the first aspect, in one implementation, the aforementioned new data features, transfer features, and decision vectors are fused, specifically including:

[0028] The aforementioned new data features, transfer features, and decision vectors are deeply fused through adaptive weight allocation to obtain fused features.

[0029] In conjunction with the first aspect, in one implementation method, feature deep fusion is achieved through adaptive weight allocation, specifically including:

[0030] The reconstruction reliability of the newly generated features, the model confidence of the transferred features, and the scene matching degree of the decision vector are obtained respectively.

[0031] The above reconstruction reliability, model confidence, and scene matching degree are normalized to obtain the weights of new data features, transfer features, and decision vectors, respectively.

[0032] Based on the aforementioned weights, the features of new data, migration features, and decision vectors are weighted and fused.

[0033] In conjunction with the first aspect, in one implementation method, optimizing the fusion feature representation specifically includes:

[0034] Based on the aforementioned fusion characteristics, the autoencoder neural network model is optimized by minimizing the reconstruction error, resulting in an optimized autoencoder neural network model. This optimized model is then used to output new emerging data features.

[0035] In conjunction with the first aspect, in one implementation, before extracting the current driving scenario features based on real-time driving data, the method further includes:

[0036] Multimodal data is collected and preprocessed to obtain the real-time driving data.

[0037] Secondly, this application provides an intelligent driving system that implements the above-mentioned autonomous learning method, the system comprising:

[0038] The knowledge mapping acquisition module is used to construct a multi-scale, multi-view feature extraction network model, extract the features of the current driving scene based on real-time driving data, and obtain the decision vector based on the pre-established scene feature-decision mapping relationship.

[0039] The transfer feature acquisition module is used to design and train an integrated neural network model that integrates adaptive feature selection and multi-scale feature extraction to acquire transfer features based on the aforementioned real-time driving data.

[0040] The feature fusion module is used to construct an autoencoder neural network model, compress and represent the real-time driving data to obtain new data features, fuse the new data features, transfer features and decision vectors, optimize the fused feature expression to obtain an optimized autoencoder neural network model, and output new new data features.

[0041] The update and optimization module is used to update and optimize the feature extraction network model, ensemble neural network model, and autoencoder neural network model by taking the new data features as input and using online learning algorithms and incremental update algorithms, through real-time monitoring and dynamic adjustment.

[0042] Thirdly, this application provides a computer-readable storage medium storing a self-learning program, wherein when the self-learning program is executed by a processor, it implements the steps of the self-learning method described above.

[0043] The beneficial effects of the technical solutions provided in this application include:

[0044] By establishing a scene feature-decision mapping relationship, decision vectors corresponding to real-time driving data are obtained. An ensemble neural network model integrating adaptive feature selection and multi-scale feature extraction is designed and trained. Transfer features are obtained based on real-time driving data. An autoencoder neural network model, online learning algorithm, and incremental update algorithm are introduced in combination with a dynamic feedback mechanism. The model's adaptability to new scenes is enhanced through feature transfer and fusion processes. Continuous optimization is achieved using incremental updates and online learning. The dynamic feedback mechanism ensures the system's real-time response and self-adjustment capability to complex environments. This constructs an efficient closed-loop data processing system and solves the technical problem of poor environmental adaptability in autonomous driving technology. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating an embodiment of the autonomous learning method for an intelligent driving system according to this application;

[0046] Figure 2 This is a schematic diagram of the multi-scale, multi-view feature extraction network model structure in the embodiments of this application;

[0047] Figure 3 This is a diagram of the Inception-ResNet network structure in the embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the functional modules of an embodiment of the intelligent driving system of this application. Detailed Implementation

[0049] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0050] In a first aspect, embodiments of this application provide an autonomous learning method for an intelligent driving system.

[0051] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the autonomous learning method for the intelligent driving system of this application. The aforementioned autonomous learning method for the intelligent driving system includes:

[0052] S1. Construct a multi-scale, multi-view feature extraction network model to extract features of the current driving scene based on real-time driving data, and obtain decision vectors based on the pre-established scene feature-decision mapping relationship;

[0053] S2. Design and train an integrated neural network model that integrates adaptive feature selection and multi-scale feature extraction to obtain transfer features based on the above real-time driving data;

[0054] S3. Construct an autoencoder neural network model, compress and represent the above real-time driving data to obtain new data features, and fuse the above new data features, transfer features and decision vectors to optimize the fused feature expression, so as to obtain an optimized autoencoder neural network model and output new new data features.

[0055] S4. Using the new data features as input, and based on online learning algorithms and incremental update algorithms, the above feature extraction network model, ensemble neural network model, and autoencoder neural network model are updated and optimized through real-time monitoring and dynamic adjustment.

[0056] In this embodiment, a multi-scale, multi-view feature extraction network model is constructed to extract features of the current driving scene based on real-time driving data, and a decision vector is obtained based on a pre-established scene feature-decision mapping relationship. An ensemble neural network model integrating adaptive feature selection and multi-scale feature extraction is designed and trained to obtain transfer features based on the aforementioned real-time driving data. An autoencoder neural network model is constructed to compress and represent the aforementioned real-time driving data to obtain new data features. The aforementioned new data features, transfer features, and decision vectors are then fused, and the fused feature expression is optimized to obtain an optimized autoencoder neural network model, which outputs new new data features. Using these new new data features as input, the feature extraction network model, ensemble neural network model, and autoencoder neural network model are updated and optimized through real-time monitoring and dynamic adjustment based on online learning algorithms and incremental update algorithms.

[0057] By establishing a scene feature-decision mapping relationship, decision vectors corresponding to real-time driving data are obtained. An ensemble neural network model integrating adaptive feature selection and multi-scale feature extraction is designed and trained. Transfer features are obtained based on real-time driving data. An autoencoder neural network model, online learning algorithm, and incremental update algorithm are introduced in combination with a dynamic feedback mechanism. The model's adaptability to new scenes is enhanced through feature transfer and fusion processes. Continuous optimization is achieved using incremental updates and online learning. The dynamic feedback mechanism ensures the system's real-time response and self-adjustment capability to complex environments, thereby constructing an efficient closed-loop data processing system.

[0058] In this embodiment, the above-mentioned multi-scale, multi-view feature extraction network model includes:

[0059] A multi-scale, multi-view coding module is used to synchronously process sensor data of different scales and perspectives and output basic coding features;

[0060] The global and local information extraction module is used to input the basic encoded features and extract features, and output a scene feature representation with global and local consistency.

[0061] The cross-attention fusion module is used to input the scene feature representation, perform feature fusion, and output a comprehensive feature representation;

[0062] A multi-layer encoder is used to input the comprehensive feature representation and extract feature representations from low to high levels in a hierarchical manner, outputting generalized driving scene features.

[0063] Furthermore, in one embodiment, before extracting the current driving scenario features based on real-time driving data, the method further includes:

[0064] Multimodal data is collected and preprocessed to obtain the real-time driving data.

[0065] In this embodiment, driving scenario data is collected from multiple sources of sensors as multimodal data, and the multimodal data is cleaned, denoised, and preprocessed to ensure the accuracy and format consistency of the data, providing high-quality input for subsequent processing.

[0066] In this embodiment, when constructing a multi-scale, multi-view feature extraction network model, a Transformer network architecture is first designed and built, and the encoding structure of the input layer is optimized to enable it to simultaneously process input data from different scales and perspectives, obtaining basic encoded features. Secondly, based on the basic encoded features, global information details and key local information of the driving scene are deeply extracted to form a scene feature representation with global and local consistency, providing a reliable basis for subsequent decision-making and planning. Then, a cross-attention mechanism is designed to fuse data from different sources, i.e., the aforementioned scene feature representations, dynamically adjusting the contribution of different data sources to scene understanding, allowing different types of data to complement each other and improving the accurate understanding of complex driving scenarios. Finally, based on the above data fusion, a multi-layer Transformer encoder is used for hierarchical feature extraction to capture deep semantics and complex relationships in the scene, generating general driving scene features with strong generalization capabilities. This process lays the foundation for the model's human-like self-learning knowledge base, enabling the system to quickly adapt and make accurate decisions when facing complex driving environments.

[0067] Furthermore, in one embodiment, before obtaining the decision vector in step S1 above, a scene feature-decision mapping relationship is pre-established. Pre-establishing the scene feature-decision mapping relationship specifically includes:

[0068] First, based on the driving scenario database, a multi-scale, multi-view feature extraction network model is used to output generalized driving scenario features.

[0069] Then, based on the generalized driving scenario features mentioned above, and using self-supervised learning, training and feedback are conducted through simulated driving scenarios to optimize the correlation between scenario features and driving decisions, so as to establish a scenario feature-decision mapping relationship.

[0070] Among them, the aforementioned generalized driving scene features, also known as generalized driving scene features, refer to the generalized feature representation formed by extracting global and local information in a driving scene layer by layer through a multi-scale and multi-view feature extraction network model. This generalized driving scene feature aims to capture the deep semantics and complex relationships in the scene, thereby maintaining the consistency of feature expression under different environmental, time and weather conditions and eliminating the interference of environmental differences on scene cognition.

[0071] In this embodiment, by combining self-supervised learning technology, a self-learning mechanism for the feature extraction network model is established through continuous scene training and feedback. In the simulated driving environment, the understanding of scene features is gradually improved, and a feature-knowledge mapping relationship for common scenes is established, that is, a scene feature-decision mapping relationship, thus initially forming the knowledge foundation for human-like self-learning of the model.

[0072] Furthermore, in one embodiment, step S2 above, designing and training an integrated neural network model that integrates adaptive feature selection and multi-scale feature extraction, specifically includes:

[0073] First, the parallel convolutional structure of the Inception module is combined with the residual connections of ResNet to form the basic network architecture.

[0074] Then, dilated causal convolutional layers are introduced into the above basic network architecture, and a gated spatiotemporal attention mechanism is integrated to construct an integrated neural network architecture that integrates adaptive feature selection and multi-scale feature extraction.

[0075] Finally, the above-mentioned ensemble neural network architecture was pre-trained using a large-scale image dataset and fine-tuned using a driving scenario database to obtain a trained ensemble neural network model.

[0076] In this embodiment, migration features can be obtained from real-time driving data using a trained ensemble neural network model.

[0077] In this embodiment, firstly, a neural network integrating the Inception module and residual connections is constructed. Parallel convolutional kernels are used to extract local details and global contextual information from the image, while residual connections ensure the training stability of the deep network. Secondly, dilated convolutions are added to the network to capture long-distance dependencies in driving scenarios, and causal convolutions are combined to process temporal data, preventing future information leakage and laying the foundation for spatiotemporal modeling. Then, a gated spatiotemporal attention mechanism is designed based on spatiotemporal convolutions, enabling the network to adaptively focus on key features. By dynamically calculating feature weights through a gating function, the model can adaptively allocate computational resources to key regions in the image, improving perception accuracy.

[0078] In this embodiment, the stability of image feature extraction is improved by using a pre-trained Inception-ResNet network and introducing residual connections. Simultaneously, a gated spatiotemporal attention mechanism module based on dilated causal convolution is designed to ensure hierarchical extraction of global and local features in the driving scene, enabling the model to adaptively adjust attention under different driving conditions and improve perception accuracy.

[0079] Furthermore, in one embodiment, step S3 above, constructing an autoencoder neural network model, specifically includes:

[0080] First, we design an unsupervised neural network with an encoder-decoder structure, where the encoder is used to compress the input data into a low-dimensional latent space, and the decoder is used to map the low-dimensional latent representation back to the original data space.

[0081] Then, based on the driving scenario database, the above unsupervised neural network is trained by minimizing the reconstruction error to obtain the above autoencoder neural network model.

[0082] In this embodiment, a encoder-decoder structure is used to compress and expand real-time driving data, effectively removing redundant information while extracting core features. Compared to related technologies that rely directly on high-dimensional data processing, this embodiment reduces data dimensionality while retaining key information, significantly improving computational efficiency.

[0083] Furthermore, in this embodiment, step S3 above, which involves feature fusion of the newly generated data features, transfer features, and decision vector, specifically includes:

[0084] The aforementioned new data features, transfer features, and decision vectors are deeply fused through adaptive weight allocation to obtain fused features.

[0085] In this embodiment, feature transfer technology is used to deeply fuse the decision vector with the features of newly generated data, fully leveraging the value of new data while utilizing existing data. This method of jointly utilizing information not only improves the accuracy of data processing but also achieves efficient resource utilization, providing strong technical support for real-time decision-making and optimization of intelligent driving systems and effectively improving data utilization.

[0086] Preferably, deep feature fusion is achieved through adaptive weight allocation, specifically including:

[0087] First, the reconstruction reliability of the newly generated features, the model confidence of the transferred features, and the scene matching degree of the decision vector are obtained respectively.

[0088] Then, the reconstruction reliability, model confidence, and scene matching degree are normalized to obtain the weights of the new data features, transfer features, and decision vectors, respectively.

[0089] Finally, based on the aforementioned weights, the new data features, migration features, and decision vectors are weighted and fused.

[0090] In other embodiments, the new data features, migration features and decision vectors can be weighted and fused according to the pre-set weights of the new data features, the migration features, and the decision vectors.

[0091] Based on the above embodiments, in this embodiment, the optimized fusion feature representation specifically includes:

[0092] Based on the aforementioned fusion characteristics, the autoencoder neural network model is optimized by minimizing the reconstruction error, resulting in an optimized autoencoder neural network model. This optimized model is then used to output new emerging data features.

[0093] In this embodiment, an unsupervised neural network model is introduced, utilizing an encoder-decoder structure to compress and expand newly generated driving data. This data is then efficiently fused with pre-trained transfer features and decision vectors. This significantly improves the model's ability to extract and represent environmental features, and enhances its ability to quickly adapt to new environments and scenarios, avoiding the need for extensive retraining with labeled data in related technologies. By employing unsupervised learning, reliance on manual annotation is reduced, enabling rapid feature transfer and adaptation in unfamiliar driving environments. This significantly improves the flexibility and practicality of the intelligent driving system, meeting the real-world needs of complex and ever-changing scenarios.

[0094] The self-learning method in this embodiment is applicable to intelligent driving systems. By integrating model and multimodal data-driven approaches, the intelligent driving system can exhibit stronger adaptability and decision-making capabilities in complex and dynamic driving environments. This significantly improves the system's perception accuracy, decision-making speed, and operational efficiency. It is applicable to various hardware and scenarios, reduces deployment difficulty and development costs, and has broad engineering application prospects, providing an innovative and efficient solution for the development of intelligent driving technology.

[0095] Specifically, the self-learning method in this embodiment includes:

[0096] Step A1: Based on the established driving scenario database, a Transformer network architecture with multi-scale and multi-view encoding is introduced. This multi-scale and multi-view encoding captures global and local information of the driving scenario, and a cross-attention mechanism is used to achieve efficient fusion of multi-modal data. General features of the generalized driving scenario are extracted, and a human-like self-learning knowledge base is established. Specifically, this includes the following steps:

[0097] Step A101: Multimodal Data Acquisition and Preprocessing: Driving scenario data covering various weather, lighting, and road conditions is collected using multi-source sensors such as cameras, LiDAR, and millimeter-wave radar. The collected data is divided into two categories: offline training data and online real-time data. The collected offline training data forms a driving scenario database and is used in the complete training process of subsequent steps. The online real-time data is not used in offline training but is needed to obtain subsequent new data features, transfer features, and decision vectors. Preprocessing of the collected data includes data cleaning, denoising, and fine-grained annotation to ensure data quality. Simultaneously, data augmentation techniques (rotation, scaling, flipping) are used to expand data diversity, providing the model with high-quality and diverse input data.

[0098] Step A102, Multi-scale and Multi-view Coding Design: Based on the offline training data collected in step A101, a feature extraction network model capable of simultaneously processing multi-view and multi-scale information is constructed, such as... Figure 2 As shown, a multi-scale, multi-view coding module is designed, which includes a multi-scale coding module and a multi-view coding module. The multi-scale coding module extracts global information (such as road layout and traffic flow) and local information (such as vehicles, pedestrians, and obstacles) from the scene in layers. The multi-view coding module of the Transformer network architecture integrates data from different sensors or perspectives (such as front camera, side camera, etc.) for alignment and stitching to form basic coding features that have both multi-scale dimensions and cross-view correlation.

[0099] Step A103, Global and Local Information Extraction: Based on the basic coding features output from A102, the global and local information extraction module further extracts the details of global information (such as road topology association and traffic flow dynamic changes) and key content of local information (such as vehicle movement trends, pedestrian behavior characteristics, and obstacle attributes) of the driving scene. Finally, a scene feature representation with global and local consistency is formed, providing a reliable basis for subsequent decision-making and planning.

[0100] Step A104, Cross-Attention Fusion Mechanism: Design a cross-attention fusion module. Based on the scene feature representation output in step A103, extract the feature representation of each data source and realize the dynamic fusion of multimodal data. During the fusion process, the weights of the data sources are adaptively adjusted according to their importance to generate a unified scene feature representation containing multimodal information, i.e., a comprehensive feature representation, which accurately reflects scene information and improves the ability to understand complex driving scenarios.

[0101] Step A105, Multi-level Feature Extraction and Generalization: Based on the comprehensive feature representation, the multi-level encoder, i.e., the multi-level Transformer encoder, further extracts feature representations from low level to high level, including low-level features such as edges and textures and high-level features such as object categories and scene semantics, to generate general features applicable to diverse driving scenarios, i.e., generalized driving scenario features, so that the model has strong generalization ability under different environmental, time and weather conditions.

[0102] Step A106: Using the generalized driving scenario features with strong generalization ability output in Step A105 as the knowledge base, and combining self-supervised learning technology, a pre-training task is designed to predict safe driving decisions based on scenario features to drive the alignment of features and decisions. Through continuous training and dynamic feedback in simulated driving scenarios, the understanding and modeling of scenario features are continuously optimized. A common scenario feature-knowledge mapping relationship is established, i.e., a scenario feature-decision mapping relationship (such as "rainy road surface - deceleration decision"), laying the knowledge foundation for subsequent optimization of human-like self-learning capabilities. By continuously accumulating scenario data and optimizing feature understanding, the ability to adapt to and optimize complex dynamic environments is gradually achieved. During training, self-iterative optimization reduces reliance on manual annotation while improving adaptability to unseen environments.

[0103] At this point, based on the real-time driving data obtained after online real-time data preprocessing, the current driving scenario features can be extracted, and the decision vector can be obtained based on the established scenario feature-decision mapping relationship.

[0104] Step A2: Design and pre-train an Inception-ResNet network integrating a gated spatiotemporal attention mechanism, introduce residual connections to stabilize training, and utilize dilated causal convolutions to enhance spatiotemporal modeling capabilities, outputting transferable feature vectors for subsequent feature fusion. This includes the following steps:

[0105] Step A201: Design the Inception-ResNet basic network architecture to achieve multi-scale feature extraction. The hierarchical structure of this architecture is shown in the attached figure. Figure 3As shown, from top to bottom, the architecture includes an input layer, a backbone module, multiple parallel convolutional residual modules (Inception-ResNet modules), a reduction module, a global average pooling layer, a random deactivation layer, and a classification output layer. The parallel convolutional residual modules A, B, and C are combinations of parallel convolutions and residual connections designed for different feature dimensions, used to extract multi-scale features at different depths. The reduction module is used to downsample the feature maps to reduce computational cost. This architecture combines the parallel convolutional structure of the Inception module with the residual connections of ResNet: the Inception module uses convolutional kernels of different sizes (1x1, 3x3, 5x5) for parallel processing and concatenation to capture multi-scale features; the residual connections directly add the input of each sub-module to its output to form "skip connections," effectively preventing gradient vanishing as the number of layers increases during deep network training. Specifically, after the input image passes through the backbone module, it is processed by parallel convolutional residual modules and reduction modules of different depths to extract multi-scale feature representations; then it enters the global average pooling layer for feature dimensionality reduction; next, it is processed by a random deactivation layer with a retention ratio of 0.8 (Dropout=0.8) to prevent model overfitting; finally, the classification result is output by the Softmax classification output layer.

[0106] Step A202: Introduce a dilated causal convolutional layer to expand the network's receptive field and ensure temporal causality. The output is a feature sequence encoded by spatiotemporal convolution, providing input to the gated spatiotemporal attention module. A dilation factor is introduced into the convolution operation to expand the receptive field, enabling the convolution to capture a wider range of contextual information while maintaining computational efficiency. To ensure the model does not leak information from future frames and only relies on data from the current time step and earlier, the dilated causal convolution calculation formula is as follows:

[0107]

[0108] in, Let be the output feature value at time t, and k be the size of the convolution kernel. Let be the weight parameters at the i-th position of the convolution kernel. The input feature sequence for the convolution is the multi-scale feature representation output from step A201, and d is the dilation factor, which controls the size of the receptive field.

[0109] Step A203: Design a gating spatiotemporal attention mechanism to enable the network to adaptively focus on key features. Building upon spatiotemporal convolution, an attention mechanism is further introduced to optimize feature selection. The sigmoid function is used as the gating function to filter effective features and suppress redundant information. Through the gating mechanism and self-attention, importance weights are calculated for each spatial location and temporal frame, weighting the features and enhancing the model's attention to key regions. This allows the network to adaptively focus on important visual areas, improving the accuracy of driving scene perception.

[0110] Step A204: Pre-train an ensemble neural network architecture on a large-scale image dataset to obtain general image features. Pre-train the network on a large-scale image dataset (ImageNet) to learn general, basic visual feature representations from massive amounts of natural images, and obtain basic image features such as edges, textures, and shapes, laying the foundation for subsequent adaptive driving tasks.

[0111] Step A205: Fine-tune the network on driving scenario data to output transfer features for subsequent fusion. Based on the network pre-trained in step A204, transfer the network weights to the driving scenario task as initialization parameters, and train and fine-tune it on the offline training data obtained in step A101 to optimize the network's discriminative ability in driving scenarios, making it adaptable to driving tasks, and finally extract highly discriminative transfer feature vectors for subsequent feature fusion.

[0112] Step S3: Introduce an unsupervised neural network model, utilize an encoder-decoder structure to compress and expand the new driving data, combine pre-trained transfer features and human-like driving knowledge mapping, and fuse them with features from the new environment to enhance the model's ability to quickly adapt to new scenarios. This specifically includes the following steps:

[0113] Step A301: Construct an unsupervised neural network model with an encoder-decoder structure. The encoder compresses the input data through a multi-layer neural network, mapping it to a low-dimensional latent space. Specifically, the input x is mapped by the encoder z to obtain a low-dimensional latent representation, i.e.: This process learns the core features of the data and removes noise and redundant information. The latent space has a relatively small z-dimensional dimension, which is beneficial for extracting the essential features of the data.

[0114] The decoder maps the low-dimensional latent representation z back to the original data space, restoring the original distribution of the data and preserving key information, gradually reconstructing the data through multiple layers of deconvolution: , This is the decoded output data.

[0115] The encoder-decoder minimizes the loss function. The training process aims to effectively compress and restore the input data while preserving key information. The loss function uses the reconstruction error (mean squared error, MSE) to measure the difference between the compressed and original data.

[0116]

[0117] Where N is the total number of samples. For the i-th original input data, This is the reconstructed data after being compressed by the encoder and restored by the decoder.

[0118] Step A302: Using the unsupervised neural network model constructed in step A301, feature extraction and encoding are performed on the input data, i.e., the real-time driving data, to obtain new data features. The encoded offline training data yields generalized driving scenario features for model training, while the real-time driving data is used for encoding and feature fusion, and also serves as the input for the subsequent online learning portion.

[0119] Step A303: Based on the image features extracted from the pre-trained and fine-tuned Inception-ResNet network in step A2, i.e., the transfer features. Combined with the knowledge vector obtained in step A106, i.e., the decision vector The features of the newly generated data are deeply fused with those of the new data through an adaptive weight allocation mechanism to generate an enhanced fused feature representation:

[0120] .

[0121] in, , , The weight parameters control the contribution of different features, and are adaptively allocated based on the credibility of the three types of features as follows:

[0122] Reliability of reconstructed features of new data ,in, This refers to the reconstruction error of newly generated data, i.e., real-time driving data, which is calculated by the encoder-decoder structure during the reconstruction of the input data. The reconstruction error sensitivity coefficient is denoted as , where a smaller reconstruction error indicates a more reliable representation of the features of the new data, and its weight should be increased.

[0123] Model confidence of transfer features ,in, This refers to the transfer feature vector extracted after fine-tuning in step A2. The feature prototype vectors of the driving scenario are more consistent, and the greater the weight is;

[0124] Scene matching , among which, Here, C represents the scene matching sensitivity coefficient, and C is the feature vector of the current driving scene. The most matching human-like driving knowledge vector is retrieved from the pre-established scene feature-decision mapping relationship. That is, the more it matches human-like driving experience, the greater the contribution of the knowledge vector.

[0125] Finally, the three factors are normalized using softmax to obtain the fusion weights, ensuring a balance in feature contributions:

[0126] .

[0127] Step A304: By optimizing the fused feature representation, train an unsupervised neural network with an encoder-decoder structure and feature fusion weights to enable the fused features to accurately represent key driving information, thereby enhancing the network's ability to represent features in new scenarios. During training, the fused features... The input decoder reconstructs the data. By minimizing the reconstruction error and introducing feature consistency constraints, the network parameters and feature fusion weights are optimized, allowing the network to gradually adapt to diverse driving scenarios and improve the generalization ability of features in different scenarios, ensuring good performance in changing environments. The feature consistency constraint is implemented using L2 regularization to ensure semantic consistency between the fused features, newly generated data features, transfer features, and decision vectors, avoiding the loss of key driving information. After training, fixed network parameters and fusion weights are obtained, and new newly generated data features are generated from real-time data for subsequent online phases.

[0128] Step A4: Based on Bayesian online learning and incremental model update algorithms, through dynamic parameter updates and feedback adjustment mechanisms, the intelligent driving system can rapidly adapt and learn under large-scale data, completing an efficient closed loop of data processing. This specifically includes the following steps:

[0129] Step A401: Employ a Bayesian online learning algorithm to dynamically model real-time data (i.e., newly generated data features), progressively updating model parameters to achieve rapid response to environmental changes. Specifically, use new, newly generated data features as input and update model parameters online to adapt the model to real-time driving data. Model initialization: Assume the current model is... , where θ represents the model parameters, including trainable parameters in the feature extraction network model, trainable parameters in the ensemble neural network model, and trainable parameters in the autoencoder neural network model.

[0130] Each time new data is received At that time, the posterior distribution is calculated based on Bayes' theorem for data updates:

[0131] .

[0132] in, It is the likelihood function, representing the data. The probability given parameter θ It is the prior distribution. Through Bayesian inference, the model parameters θ are continuously updated based on new data:

[0133] .

[0134] At each update step, the model learns from the new data and adjusts its parameters to better adapt to the current environment.

[0135] Step A402: Combine new data with historical data features through an incremental model update algorithm. That is, only some model parameters are updated each time new data arrives, instead of training the entire model from scratch. This allows for continuous optimization of the model with limited computing resources and avoids the catastrophic forgetting problem (i.e., the model forgets the original knowledge when learning new knowledge), ensuring continuous and stable optimization of the model.

[0136] Historical data is weighted, with more recent data receiving higher weights and older data receiving less weight. A decay factor is set. To gradually reduce the impact of historical data:

[0137] .

[0138] in, Weights for the features of newborn data. This is the attenuation factor.

[0139] In this embodiment, an experience replay strategy is adopted, in which a certain amount of historical data is kept in a buffer pool and used periodically for model training to prevent the learning of new data from overwriting the experience of old data.

[0140] Step A403: Design a dynamic feedback mechanism. Based on real-time monitoring of the driving environment and system performance indicators, generate feedback signals to guide model updates and optimization strategies. Specifically, real-time monitoring of the driving environment uses sensors to collect driving data, extracts environmental features and analyzes their changes, defines system performance indicators (control accuracy, reaction time, driving stability), and generates feedback signals by calculating system errors and deviations.

[0141] .

[0142] in, As a feedback signal, For systematic error, For performance metrics.

[0143] Based on real-time error and performance evaluation, the strength and type of feedback signals are dynamically adjusted. The learning rate, training strategy, and optimization objective of the model are dynamically adjusted according to the feedback signals, thereby achieving dynamic model optimization.

[0144] Step A404: By continuously collecting data, extracting features, updating the model, and generating feedback, the intelligent driving system achieves self-adaptation and self-learning. The entire process forms an efficient closed-loop data processing flow, continuously iterating to improve the adaptive and self-learning capabilities of the intelligent driving system.

[0145] In this embodiment, Bayesian online learning and incremental model update algorithms are employed to achieve dynamic modeling and parameter optimization of real-time data in large-scale data environments. Incremental updates avoid a large amount of redundant computation and significantly reduce time and resource costs. Furthermore, the Bayesian learning method maintains high robustness and accuracy when dealing with uncertain data and effectively avoids the catastrophic forgetting problem common in model updates. This enables the system to continuously learn from new data and optimize performance, maintaining high learning capacity and adaptability in the face of dynamically changing driving environments.

[0146] Secondly, by monitoring the driving environment and system performance indicators in real time, feedback signals are used to guide model updates and optimizations. This constructs an efficient closed-loop process from data acquisition and feature extraction to model updates and feedback adjustments, enabling the system to automatically adjust its performance according to environmental changes, significantly improving the system's perception and response efficiency in complex scenarios. Simultaneously, the introduction of the feedback mechanism can quickly identify and correct potential problems in the system, further enhancing the robustness and stability of the intelligent driving system.

[0147] The method in this embodiment is based on a multi-scale, multi-view coding and cross-attention fusion approach for intelligent driving scene feature extraction and self-learning. It integrates multimodal data, deep learning, and self-supervised learning. Multi-scale, multi-view coding technology extracts global and local features of the driving scene, and a cross-attention mechanism achieves efficient fusion of multimodal data. Furthermore, self-supervised learning constructs a human-like self-learning knowledge system, enhancing the system's environmental understanding and self-optimization capabilities. Through end-to-end design and optimization, it achieves efficient coordination of the intelligent driving system from environmental perception to decision-making and control. Simultaneously, it possesses high practicality and portability, applicable to large-scale driving data environments without relying on complex hardware. Its core algorithm structure is simple yet efficient, easily integrated into existing intelligent driving platforms. It maintains stable performance under various hardware conditions and scenarios. This high adaptability and flexibility make it an ideal choice for practical engineering applications.

[0148] Secondly, embodiments of this application also provide an intelligent driving system that implements the above-described autonomous learning method.

[0149] In one embodiment, reference is made to Figure 4 , Figure 4 This is a functional module diagram of an embodiment of the intelligent driving system of this application. The intelligent driving system includes: a knowledge mapping acquisition module, a transfer feature acquisition module, a feature fusion module, and an update and optimization module.

[0150] The aforementioned knowledge mapping acquisition module is used to construct a multi-scale, multi-view feature extraction network model, extract the features of the current driving scene based on real-time driving data, and obtain the decision vector based on the pre-established scene feature-decision mapping relationship.

[0151] The aforementioned transfer feature acquisition module is used to design and train an integrated neural network model that integrates adaptive feature selection and multi-scale feature extraction to acquire transfer features based on the aforementioned real-time driving data.

[0152] The aforementioned feature fusion module is used to construct an autoencoder neural network model, compress and represent the real-time driving data to obtain new data features, and then fuse the new data features, transfer features and decision vectors to optimize the fused feature expression, so as to obtain an optimized autoencoder neural network model and output new new data features.

[0153] The aforementioned update and optimization module is used to update and optimize the feature extraction network model, ensemble neural network model, and autoencoder neural network model by taking the new data features as input and based on online learning algorithms and incremental update algorithms, through real-time monitoring and dynamic adjustment.

[0154] Furthermore, in one embodiment, the knowledge mapping acquisition module is also used for:

[0155] Based on a driving scenario database, a multi-scale, multi-view feature extraction network model is used to output generalized driving scenario features.

[0156] Based on the generalized driving scenario characteristics mentioned above, and using self-supervised learning, training and feedback are conducted through simulated driving scenarios to optimize the correlation between scenario features and driving decisions, and to establish a scenario feature-decision mapping relationship.

[0157] Furthermore, in one embodiment, the migration feature acquisition module described above is used for:

[0158] The parallel convolutional structure of the Inception module is combined with the residual connections of ResNet to form the basic network architecture;

[0159] In the above basic network architecture, dilated causal convolutional layers are introduced and a gated spatiotemporal attention mechanism is integrated to construct an integrated neural network architecture that integrates adaptive feature selection and multi-scale feature extraction.

[0160] The aforementioned ensemble neural network architecture was pre-trained using a large-scale image dataset and then fine-tuned using a driving scenario database to obtain a trained ensemble neural network model.

[0161] Furthermore, in one embodiment, the feature fusion module described above is used for:

[0162] Design an unsupervised neural network with an encoder-decoder structure, where the encoder is used to compress the input data into a low-dimensional latent space, and the decoder is used to map the low-dimensional latent representation back to the original data space.

[0163] Based on a driving scenario database, the above-mentioned unsupervised neural network is trained by minimizing the reconstruction error to obtain the above-mentioned autoencoder neural network model.

[0164] Furthermore, in one embodiment, the feature fusion module is also used for:

[0165] The aforementioned new data features, transfer features, and decision vectors are deeply fused through adaptive weight allocation to obtain fused features.

[0166] Furthermore, in one embodiment, the feature fusion module is also used for:

[0167] The reconstruction reliability of the newly generated features, the model confidence of the transferred features, and the scene matching degree of the decision vector are obtained respectively.

[0168] The above reconstruction reliability, model confidence, and scene matching degree are normalized to obtain the weights of new data features, transfer features, and decision vectors, respectively.

[0169] Based on the aforementioned weights, the features of new data, migration features, and decision vectors are weighted and fused.

[0170] Furthermore, in one embodiment, the feature fusion module is also used for:

[0171] Based on the aforementioned fusion characteristics, the autoencoder neural network model is optimized by minimizing the reconstruction error, resulting in an optimized autoencoder neural network model. This optimized model is then used to output new emerging data features.

[0172] Furthermore, in one embodiment, the knowledge mapping acquisition module is also used for:

[0173] Multimodal data is collected and preprocessed to obtain the real-time driving data.

[0174] The functions of each module in the above-mentioned intelligent driving system correspond to the steps in the above-mentioned autonomous learning method embodiment of the intelligent driving system, and their functions and implementation processes will not be described in detail here.

[0175] Thirdly, embodiments of this application also provide a computer-readable storage medium.

[0176] The present application has a computer-readable storage medium storing a self-learning program, wherein when the self-learning program is executed by a processor, it implements the steps of the self-learning method described above.

[0177] The method implemented when the self-learning program is executed can be referred to in the various embodiments of the self-learning method of this application, and will not be repeated here.

[0178] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0179] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0180] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0181] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0182] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0183] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0184] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An autonomous learning method for an intelligent driving system, characterized by, The method includes: A multi-scale, multi-view feature extraction network model is constructed to extract the features of the current driving scene based on real-time driving data, and to obtain the decision vector based on the pre-established scene feature-decision mapping relationship. Design and train an integrated neural network model that integrates adaptive feature selection and multi-scale feature extraction to obtain transfer features based on the real-time driving data; An autoencoder neural network model is constructed to compress and represent the real-time driving data to obtain new data features. The new data features, transfer features and decision vectors are then fused and the fused feature expression is optimized to obtain an optimized autoencoder neural network model and output new new data features. Using the new data features as input, the feature extraction network model, ensemble neural network model, and autoencoder neural network model are updated and optimized based on online learning algorithms and incremental update algorithms through real-time monitoring and dynamic adjustment; Design and train an ensemble neural network model that integrates adaptive feature selection and multi-scale feature extraction, specifically including: The parallel convolutional structure of the Inception module is combined with the residual connections of ResNet to form the basic network architecture; In the basic network architecture, dilated causal convolutional layers are introduced and a gated spatiotemporal attention mechanism is integrated to construct an integrated neural network architecture that integrates adaptive feature selection and multi-scale feature extraction. The integrated neural network architecture is pre-trained using a large-scale image dataset and then fine-tuned using a driving scenario database to obtain a trained integrated neural network model. Constructing an autoencoder neural network model specifically includes: Design an unsupervised neural network with an encoder-decoder structure, where the encoder is used to compress the input data into a low-dimensional latent space, and the decoder is used to map the low-dimensional latent representation back to the original data space. Based on a driving scenario database, the unsupervised neural network is trained by minimizing the reconstruction error to obtain the autoencoder neural network model; The multi-scale, multi-view feature extraction network model includes: A multi-scale, multi-view coding module is used to synchronously process sensor data of different scales and perspectives and output basic coding features; The global and local information extraction module is used to input the basic encoded features and extract features, and output a scene feature representation with global and local consistency. The cross-attention fusion module is used to input the scene feature representation, perform feature fusion, and output a comprehensive feature representation; A multi-layer encoder is used to input the comprehensive feature representation and extract feature representations from low to high levels in a hierarchical manner, outputting generalized driving scene features. 2.The intelligent driving system autonomous learning method of claim 1, wherein, Pre-establish scene feature-decision mapping relationships, specifically including: Based on a driving scenario database, a multi-scale, multi-view feature extraction network model is used to output generalized driving scenario features. The generalized driving scenario features are general feature representations that are extracted and formed by the multi-scale, multi-view feature extraction network model and have global and local consistency and are applicable to diverse driving scenarios. Based on the generalized driving scenario features, and using self-supervised learning, training and feedback are conducted through simulated driving scenarios to optimize the correlation between scenario features and driving decisions, and to establish a scenario feature-decision mapping relationship. 3.The intelligent driving system autonomous learning method of claim 1, wherein, The feature fusion of the newly generated data features, transfer features, and decision vectors specifically includes: The newly generated data features, transfer features, and decision vectors are deeply fused through adaptive weight allocation to obtain fused features. 4.The intelligent driving system autonomous learning method of claim 3, wherein, Deep feature fusion is achieved through adaptive weight allocation, specifically including: The reconstruction reliability of the newly generated data features, the model confidence of the transfer features, and the scene matching degree of the decision vector are obtained respectively. The reconstruction reliability, model confidence, and scene matching degree are normalized to obtain the weights of the new data features, transfer features, and decision vectors, respectively. Based on the weights, the new data features, migration features, and decision vectors are weighted and fused. 5.The intelligent driving system autonomous learning method of claim 3, wherein, Optimize the fusion feature representation, specifically including: Based on the fusion features, the autoencoder neural network model is optimized by minimizing the reconstruction error to obtain an optimized autoencoder neural network model, which is used to output new emerging data features. 6.The intelligent driving system autonomous learning method of claim 1, wherein, Before extracting features of the current driving scenario from real-time driving data, the following steps are also included: Multimodal data is collected and preprocessed to obtain the real-time driving data.

7. An intelligent driving system implementing the autonomous learning method of claim 1, characterized in that, The system includes: The knowledge mapping acquisition module is used to construct a multi-scale, multi-view feature extraction network model, extract the features of the current driving scene based on real-time driving data, and obtain the decision vector based on the pre-established scene feature-decision mapping relationship. The transfer feature acquisition module is used to design and train an integrated neural network model that integrates adaptive feature selection and multi-scale feature extraction to acquire transfer features based on the real-time driving data. The feature fusion module is used to construct an autoencoder neural network model, compress and represent the real-time driving data to obtain new data features, fuse the new data features, transfer features and decision vectors, optimize the fused feature expression to obtain an optimized autoencoder neural network model, and output new new data features. The update and optimization module is used to update and optimize the feature extraction network model, ensemble neural network model, and autoencoder neural network model by taking the new data features as input and based on online learning algorithms and incremental update algorithms, through real-time monitoring and dynamic adjustment.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a self-learning program, wherein when the self-learning program is executed by a processor, it implements the steps of the self-learning method as described in any one of claims 1 to 6.

Images