Intelligent vehicle extreme test scene generation method and system based on deep learning

Abstract

The invention discloses an intelligent vehicle extreme test scene generation method based on deep learning. The method comprises the following steps: building a hybrid generative adversarial network scene; collecting and preprocessing data; performing dynamic confrontation training; according to the extreme test simulation, an intelligent vehicle extreme test scene generation system based on deep learning comprises a hybrid generative adversarial network model, a data input module, a preprocessing module, an edge cloud module and an interaction module. A double-channel confrontation mechanism enables the risk controllability of a generated scene to be high, and dynamic confrontation training is carried out; a real multi-source data set is enhanced and labeled, the integrity of point cloud data is high through a point cloud missing region feature completion algorithm, and dynamic risk entry is effectively simulated by introducing a social force model into dynamic obstacle modeling; the multi-modal risk assessment network improves the assessment accuracy.

Description

Technical Field

[0001] The present invention relates to the field of smart car testing technology, and specifically to a method and system for generating extreme test scenarios for smart cars based on deep learning. Background Art

[0002] Smart cars are the product of the combination of the latest scientific and technological achievements such as electronic computers and the modern automobile industry.

[0003] An intelligent vehicle is a comprehensive system that integrates environmental perception, planning and decision-making, and multi-level assisted driving. It uses computers, modern sensors, information fusion, communications, artificial intelligence, and automatic control technologies, and is a typical high-tech complex. Research on intelligent vehicles is mainly focused on improving the safety and comfort of automobiles, as well as providing an excellent human-vehicle interaction interface.

[0004] An intelligent vehicle is a general vehicle that has advanced sensors (such as radars, cameras, etc.), controllers, actuators and other devices added to it. Through the on-board environmental perception system and information terminal, it can exchange information with people, vehicles, roads, etc., so that the vehicle has intelligent environmental perception capabilities, can automatically analyze the safety and danger status of vehicle driving, and enable the vehicle to reach the destination according to the wishes of the person, ultimately achieving the purpose of replacing human operation.

[0005] In general, smart cars are a new generation of cars that are equipped with advanced sensing systems, decision-making systems, and execution systems, and use new technologies such as information communication, the Internet, big data, cloud computing, and artificial intelligence. They have partial or full autonomous driving capabilities and are gradually transforming from simple transportation tools to smart mobile spaces.

[0006] Smart car technology is different from the generally referred to autonomous driving technology. It refers to the autonomous driving of cars achieved by using a variety of sensors and smart road technologies.

[0007] In the Chinese invention patent with application publication number CN115935642A, a method and system for automatically generating extreme test scenarios for smart cars based on accident information are disclosed. The method is as follows: obtain the investigation information text of the actual traffic accident, pre-process it first, and then use natural language processing technology to extract the scene static elements of the actual traffic accident, including scene road network information, weather environment information, scene start and end point information and collision basic information; initialize the scene to be generated based on the extracted scene static elements, and use the deep deterministic gradient strategy algorithm to search for the dynamic parameter combination set of the scene to be generated; use the extracted scene static elements of the actual traffic accident as the static elements of the scene to be generated, and combine them with the searched dynamic parameter combination set to generate an extreme test scenario for smart car traffic accidents similar to the actual traffic accident. The present invention supports the construction of a library of extreme test scenarios for smart cars and accelerates its testing.

[0008] However, the above-mentioned technology mainly studies the extraction of extreme scenarios of existing accidents, focusing on the clustering extraction of key types of scenarios. Some scenario difference information will be ignored, and the final degree of automation in the construction process of extreme scenarios is not high. However, the training process for extreme test scenarios is relatively scarce, which can easily lead to problems such as lack of resources for extreme scenarios, low efficiency in generating extreme scenarios, and incomplete coverage.

[0009] To this end, the present invention provides a method and system for generating extreme test scenarios for smart vehicles based on deep learning. Summary of the invention

[0010] 1. Technical issues to be resolved In view of the shortcomings of the prior art, the present invention provides a method and system for generating extreme test scenarios for smart cars based on deep learning. By fusing a generative adversarial network with a variational autoencoder, a hybrid adversarial network model is generated, and the collected multi-source data set of real accidents is enhanced and labeled to improve the integrity of the multi-source data set, generate physically reasonable and risk-intensive diversified scenarios, and cover the long-tail distribution, thereby solving the technical problems recorded in the background technology.

[0011] (II) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for generating extreme test scenarios for smart cars based on deep learning, comprising the following steps: Build a hybrid generative adversarial network scenario: Use the generative adversarial network and variational autoencoder to fuse and generate a hybrid generative adversarial network model; Data collection and preprocessing: Collect real multi-source data sets, enhance and annotate them, and generate dense data sets; Dynamic adversarial training: The generated dense data set is input into the hybrid adversarial generative network model for training. High-risk extreme scenarios are generated through a dual-channel adversarial mechanism for dynamic adversarial training. Simulated extreme testing: Input various data of the smart car into the trained adversarial generative network model to conduct tests in extreme environments.

[0012] Furthermore, the dynamic adversarial training also includes the following steps: Multimodal dynamic risk assessment: A dual-stream network is used to process the physical and logical constraints of the scene respectively. The risk factors are then weighted and integrated by introducing an attention mechanism. Then, risk interpretability analysis is performed to generate quantifiable risk indicators. Finally, the quantifiable risk indicators are input into the hybrid adversarial generative network model to promote the generation of high-risk scenarios. Evolutionary algorithm optimization and simulation verification: Multi-objective optimization is performed on extreme scenarios, and then verified through a digital twin closed loop, and the verification results are input into the hybrid adversarial generative network model.

[0013] Furthermore, the hybrid generative adversarial network model performs probabilistic modeling of the latent space through the variational autoencoder; Assume that the input data set is dense , output potential distribution parameters ; Sampling to obtain latent variables ; Control the regularization strength of the latent space through KL divergence constraints; , in, ; in, Expressed as the divergence function of the space, is represented as the regularization coefficient, , Represents the conditional distribution defined by the encoder, given an input dense dataset , latent variables The posterior distribution of Represented as a standard normal distribution: the mean is 0 and the covariance matrix is ​​the identity matrix , It is expressed as an indicator to measure the difference between two probability distributions. Represented as the mean vector output by the encoder network, Denoted as the standard deviation vector of the encoder network output.

[0014] Furthermore, the hybrid adversarial generative network model includes a generator, a discriminator and a risk assessment module; Generator is used to input latent variables , output generation scene ; The discriminator is used to evaluate: physical plausibility and risk level ; The adversarial loss function is calculated as follows: , The total loss function is a weighted sum of multiple objectives: ; in, Expressed as the total loss function of the generative adversarial network, Expressed as the expected value of the calculation, Represented as the discriminator for dense data sets The probability of discrimination, It is represented as the generator outputting the generated scene according to the latent variable, and the latent variable is the noise vector. Represented as the discriminator to generate samples The probability of discrimination, Represented as a dense dataset score term, is represented as the generated data score term, and They are respectively represented as GAN adversarial loss weight, KL divergence loss weight and risk loss weight, and the initial fixed weights are set to , Expressed as the total loss function, Expressed as the divergence function of the space, It is expressed as risk adversarial loss function.

[0015] Furthermore, the steps of data collection and preprocessing are as follows: Multi-source data collection: extract accident clips from the real road test database; collection parameters include: sensor data, environmental parameters and driving behavior; Preprocess the data: filter the collected data, detect abnormal data in the data, remove and replace the abnormal data, and then normalize the data to reduce the dimension of the data; Data enhancement and annotation: Perform 3D reconstruction and physical parameter interpolation on the accident fragment to generate a dense data set. The steps are as follows; Extract features from the accident fragments and generate feature maps based on the extracted features; Then, point cloud computing of the accident fragment is realized, and the point cloud distribution of the missing area is predicted; First, the accident fragment is divided into regions, and the missing regions are supplemented by feature maps near the missing regions. The feature maps are first normalized to generate uniform reference points in the missing regions, and then the predicted point cloud data points are generated by the mean or variance of the feature maps. Environmental variable expansion: Randomly change weather parameters in the reconstructed scene; add dynamic obstacles to enhance and annotate the data.

[0016] Furthermore, the characteristic map of the missing region is completed: Point cloud with missing areas , the missing regions are marked as , local feature map of each point ; Local feature aggregation: For each point , in radius Aggregate K nearest neighbor features within the neighborhood: , in, Expressed as The neighborhood point index of Expressed as The neighboring points of Expressed as a learnable feature transformation function, Indicates that the maximum pooling retains significant features, Represented as the local feature map of the missing area, Expressed as The local feature map of the neighborhood points, It is expressed as the operation function of a multi-layer perceptron; Local feature maps of missing areas Combine the mean or variance to generate predicted point cloud data points.

[0017] Furthermore, the calculation of adding dynamic obstacles is as follows: Dynamic Obstacle Modeling: , in, Represented as a dynamic obstacle model, Expressed as the additional driving force of the dynamic obstacle model, The social repulsion force expressed as a dynamic obstacle model, is represented as a random perturbation modulo the dynamic obstacle.

[0018] Furthermore, the specific steps of the multimodal dynamic risk assessment are as follows: The feature matrix of the scenario generated by the hybrid adversarial generative network model is input into the multimodal dynamic risk assessment network, which contains information on physical and logical flows, and outputs a comprehensive risk value. ; Physical flow processing: Use 3D convolutional layers to extract spatiotemporal features and check whether there are acceleration changes beyond the safe range; Logical flow processing: traffic participants are represented as graph nodes, and the relative movement and position relationship between the vehicle and other traffic participants are predicted; Multi-head attention mechanism: 4 attention heads are calculated in parallel, focusing on different modal information, calculating attention weights, and generating comprehensive risk values. ; , in, =64 is the key vector dimension, 4 attention heads are calculated in parallel, Represented as the query key value calculation value, which is the comprehensive risk value , Represented as the query vector, the feature vector of the input sequence, represents the information of a certain position in the current sequence, Represented as a key vector, another feature vector associated with the query vector, which is used to measure the relevance or matching degree of the query vector to itself. Represented as a value vector, another feature vector associated with the query vector and the key vector, usually carrying important information, Represented as a scaling factor, used in attention calculations to prevent the dot product of the query vector and the key vector from being too large. Expressed as a function, it converts the attention weights into a probability distribution, ensuring that the sum of the weights of all attention heads is 1. Denoted as attention weight, is the product of two matrices, the result is a matrix, the scaling factor Normalize it; The steps of risk interpretability analysis are as follows: The comprehensive risk value Enter and set high-risk scenarios =0.85, output interpretability report; Counterfactual reasoning: Through gradient back propagation, the analysis model is based on the input comprehensive risk value Draw high-risk conclusions, calculate feature importance, and identify key risk sources; Risk heat map generation: mark high-risk areas in the feature space and use heat maps to show the distribution of risk factors; , in, Indicates a high-risk area. Respectively represented as input comprehensive risk value The eigenvalues ​​in Expressed as an integral variable, it controls the degree of disturbance, Expressed as risk function versus feature gradient.

[0019] A deep learning-based smart car extreme test scenario generation system, including a hybrid adversarial generative network model, a data input module, a preprocessing module, an edge cloud module, and an interaction module; The hybrid generative adversarial network model uses a generative adversarial network and a variational autoencoder for fusion, and then performs dynamic adversarial training to generate extreme test scenarios and perform extreme tests on smart vehicles; The data input module is used to input multi-source data sets into the hybrid adversarial generative network model, perform dynamic adversarial training, and simulate extreme test scenarios; and realize the input of various data of the smart car to perform extreme scenario test simulation; The preprocessing module is used to perform noise reduction, anomaly elimination, replacement and normalization on the multi-source data set, and enhance and annotate the multi-source data set to generate a dense data set; The edge cloud module is used to generate and train a hybrid adversarial generative network model based on a multi-source data set, and is used to perform extreme scenario test simulations after inputting various data of the smart car, and to generate results of the extreme scenario tests, and transmit them to the interactive module through the Internet of Things or the 5G network; The interactive module is used to display the scenes simulated by the extreme scenario test of the smart car, and to realize control adjustment, set according to requirements, and receive and display the results of the extreme scenario test.

[0020] Furthermore, the calculation process of the noise reduction is as follows: For multi-source datasets , use windows of different scales to perform median filtering, and then perform weighted fusion on the filtering results of different scales; , in, Represents the output data after filtering; Indicates the number of scales; Representation scale The weights are used to perform weighted averaging of filtering results of different scales; Indicated in scale The result of median filtering the data is as follows: Representative scale The corresponding neighborhood window; The calculation of the abnormal elimination and replacement is as follows: The Z-score of the data is obtained by calculating the difference between the data and the mean and dividing it by the standard deviation; , in, is the value of the multi-source dataset, is the mean of the multi-source dataset, is the standard deviation of the multi-source dataset, Points with values ​​greater than 3 or less than -3 are considered outliers and are removed; The vacancies removed by outliers are filled by the mean, and the mean is calculated as follows: , in, is the value of non-missing data, is the number of non-missing data; The calculation of the normalization process is as follows: , in, is the minimum value in the multi-source dataset, is the maximum value in the multi-source dataset, Represented as normalized data, the multi-source datasets are scaled to the range of [0, 1].

[0021] (III) Beneficial effects The present invention provides a method and system for generating extreme test scenarios for smart cars based on deep learning, which has the following beneficial effects: When the present invention is used, a deep coupling model is implemented by using a generative adversarial network and a variational autoencoder, and the regularized distribution of scene elements is realized through the latent space probability modeling of the variational autoencoder. Combined with the adversarial generation capability of the generative adversarial network, the rapid generation of extreme scenes is realized, and the diversity of extreme scenes can also be improved. The dual-channel adversarial mechanism makes the risk controllability of the generated scenes higher, and generates high-risk extreme scenes for dynamic adversarial training. In addition, the real multi-source data sets are enhanced and annotated, and the point cloud missing area feature completion algorithm is used, and the local feature aggregation and prediction generation mechanism are used to make the point cloud data more complete. The social force model is introduced into the dynamic obstacle modeling, which can effectively generate high-risk extreme scenarios and effectively simulate dynamic risk entry, thereby improving the authenticity of high-risk extreme scenarios. In addition, a multimodal risk assessment network is constructed. Through dual-stream processing of physical and logical flows, combined with a four-head attention machine, dynamic weighted fusion of risk factors is achieved, which can effectively reduce the false alarm rate and improve the assessment accuracy. The risk heat map generation module realizes the visual positioning of key risk sources. Digital twin closed-loop verification uses evolutionary algorithms to achieve closed-loop iteration of scenario generation and verification, improving test efficiency. Combined with the three-dimensional visualization interface of the interactive module, it supports real-time parameter adjustment and risk tracing analysis, greatly improving test design efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the steps of a method for generating extreme test scenarios for smart cars based on deep learning according to the present invention; Figure 2 This is a schematic diagram of the system structure of a deep learning-based smart car extreme test scenario generation system of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 The present invention provides a method for generating extreme test scenarios for smart cars based on deep learning, comprising the following steps: Build a hybrid generative adversarial network scenario: Use the generative adversarial network and variational autoencoder to fuse and generate a hybrid generative adversarial network model; Data collection and preprocessing: Collect real multi-source data sets, enhance and annotate them, and generate dense data sets; Dynamic adversarial training: The generated dense data set is input into the hybrid adversarial generative network model for training. High-risk extreme scenarios are generated through a dual-channel adversarial mechanism for dynamic adversarial training. Simulated extreme testing: Input various data of the smart car into the trained adversarial generative network model to conduct tests in extreme environments.

[0025] In this embodiment, preferably, the dynamic adversarial training further includes the following steps: Multimodal dynamic risk assessment: A dual-stream network is used to process the physical and logical constraints of the scene respectively. The risk factors are then weighted and integrated by introducing an attention mechanism. Then, risk interpretability analysis is performed to generate quantifiable risk indicators. Finally, the quantifiable risk indicators are input into the hybrid adversarial generative network model to promote the generation of high-risk scenarios. Evolutionary algorithm optimization and simulation verification: Multi-objective optimization is performed on extreme scenarios, and then the results are verified through a digital twin closed loop, and the verification results are input into the hybrid adversarial generative network model; It should be noted that the dual-stream network architecture realizes multi-dimensional feature decoupling analysis through parallel processing of physical constraint flow and logical constraint flow. The physical flow detects hard indicators such as acceleration mutation rate, and the logical flow predicts vehicle-pedestrian interaction trajectory, which improves the risk assessment dimension by 2.3 times compared with the single-stream network. The weighted fusion of the four-head attention mechanism improves the accuracy of identifying key risk factors. In addition, the risk interpretability analysis locates key risk sources through counterfactual reasoning, guides the GAN generator to focus on high-risk features, and improves the efficiency of high-risk scenario generation. The probability of vehicle collision in the generated scenario reaches 0.87, and the heat map visualization system enables engineers to quickly locate potential risk points.

[0026] In this embodiment, preferably, the hybrid generative adversarial network model performs probabilistic modeling of the latent space through the variational autoencoder; Assume that the input data set is dense , output potential distribution parameters ; Sampling to obtain latent variables ; Control the regularization strength of the latent space through KL divergence constraints; , in, ; in, Expressed as the divergence function of the space, is represented as the regularization coefficient, , Represents the conditional distribution defined by the encoder, given an input dense dataset , latent variables The posterior distribution of Represented as a standard normal distribution: the mean is 0 and the covariance matrix is ​​the identity matrix , It is expressed as an indicator to measure the difference between two probability distributions. Represented as the mean vector output by the encoder network, Represented as the standard deviation vector of the encoder network output; It should be noted that the KL divergence constraint forces the latent variables It obeys the standard normal distribution, realizes decoupled feature expression, improves the independence between dimensions of the latent space, and supports single variable adjustment of scene characteristics; enhances generation controllability and improves the coverage of variable sampling space; and suppresses overfitting, so that the regularization coefficient can be dynamically adjusted to make the generated scenes diverse and efficient; the parameterized design of the encoder network output mean and standard deviation realizes the bidirectional mapping from dense data sets to latent space, reduces reconstruction errors, and samples high-risk potential areas in a targeted manner, which increases the probability of collision scene generation.

[0027] In this embodiment, preferably, the hybrid adversarial generative network model includes a generator, a discriminator and a risk assessment module; Generator is used to input latent variables , output generation scene ; The discriminator is used to evaluate: physical plausibility and risk level ; The adversarial loss function is calculated as follows: , The total loss function is a weighted sum of multiple objectives: ; in, Expressed as the total loss function of the generative adversarial network, Expressed as the expected value of the calculation, Represented as the discriminator for dense data sets The probability of discrimination, It is represented as the generator outputting the generated scene according to the latent variable, and the latent variable is the noise vector. Represented as the discriminator to generate samples The probability of discrimination, Represented as a dense dataset score term, is represented as the generated data score term, and They are respectively represented as GAN adversarial loss weight, KL divergence loss weight and risk loss weight, and the initial fixed weights are set to , Expressed as the total loss function, Expressed as the divergence function of the space, Expressed as risk adversarial loss function; It should be noted that the dual-channel discrimination evaluation system detects hard indicators through a 3D convolutional network, reduces the misjudgment rate, and the adversarial loss weight is adaptively adjusted with the training rounds. The KL divergence weight implements temperature coefficient annealing to reduce the occurrence rate of mode collapse. In addition, the risk perception generation mechanism increases the probability of generating high-risk scenarios, and the risk loss function design makes the high-risk value of the generated scenarios account for a high proportion. The weight strategy of the loss function collaborative optimization of the multi-objective dynamic balance achieves a reduction in the gradient conflict rate and an increase in the training convergence speed. In addition, the KL divergence regularization constrains the potential space distribution, which improves the physical rationality of the generated scenarios. Risk-Adverse Loss Function The calculation formula is as follows: , in, It is represented as the output probability of the pre-trained risk assessment model, and the risk value ∈ [0,1]; the negative sign indicates that the risk value needs to be maximized, forcing the generator to generate high-risk scenarios.

[0028] Dynamic weight adjustment Strategy: Dynamic adjustment based on reinforcement learning (PPO algorithm) ; Reward function design: , in, =0.6, =0.4 is used to balance risk and diversity, the lambda value is updated every 100 iterations, and the learning rate is 0.001. Represented as the risk assessment value of the generated scenario, Expressed as the diversity evaluation value of the generated scene, A high risk value.

[0029] In this embodiment, preferably, the steps of data collection and preprocessing are as follows: Multi-source data collection: extract accident clips from the real road test database; collection parameters include: sensor data, environmental parameters and driving behavior; Preprocess the data: filter the collected data, detect abnormal data in the data, remove and replace the abnormal data, and then normalize the data to reduce the dimension of the data; Data enhancement and annotation: Perform 3D reconstruction and physical parameter interpolation on the accident fragment to generate a dense data set. The steps are as follows; Extract features from the accident fragments and generate feature maps based on the extracted features; Then, point cloud computing of the accident fragment is realized, and the point cloud distribution of the missing area is predicted; First, the accident fragment is divided into regions, and the missing regions are supplemented by feature maps near the missing regions. The feature maps are first normalized to generate uniform reference points in the missing regions, and then the predicted point cloud data points are generated by the mean or variance of the feature maps. Environmental variable expansion: randomly change weather parameters in the reconstructed scene; add dynamic obstacles to enhance and annotate data; It should be noted that local feature aggregation can be used to complete missing areas, generate sudden obstacles through the social force model, improve the coverage of dangerous scenes, support random combination of weather parameters, generate scene diversity, automate feature extraction, generate feature maps, and make accident fragment reconstruction more efficient.

[0030] In this embodiment, preferably, the characteristic map of the missing region is completed: Point cloud with missing areas , the missing regions are marked as , local feature map of each point ; Local feature aggregation: For each point , in radius Aggregate K nearest neighbor features within the neighborhood: , in, Expressed as The neighborhood point index of Expressed as The neighboring points of Expressed as a learnable feature transformation function, Indicates that the maximum pooling retains significant features, Represented as the local feature map of the missing area, Expressed as The local feature map of the neighborhood points, It is expressed as the operation function of a multi-layer perceptron; Local feature maps of missing areas Combine the mean or variance to generate predicted point cloud data points; It should be noted that in the missing area, for each point, its local feature information is extracted to form a feature map, which may include multiple features such as the point's coordinates, normal vector, distance, etc.; for each point in the missing area, a radius neighborhood is defined, and the K nearest neighbor features of the neighborhood points are collected. By aggregating these features, the local geometry and shape information around the neighborhood points can be captured; through feature aggregation and generative prediction, combined with statistical methods and deep learning models, the problem of point cloud data completion is effectively solved, and multimodal features can be captured to generate reasonable completion data.

[0031] In this embodiment, preferably, the calculation of adding dynamic obstacles is as follows: Dynamic Obstacle Modeling: , in, Represented as a dynamic obstacle model, Expressed as the additional driving force of the dynamic obstacle model, The social repulsion force expressed as a dynamic obstacle model, Expressed as random perturbations of the dynamic obstacle modulus; It should be noted that social repulsion and random disturbances simulate the uncertainty and diversity in complex traffic scenarios and generate more realistic obstacle behaviors. The additional driving force predicts the future behavior of obstacles through speed and acceleration, enhancing the forward-looking and predictive capabilities of the model.

[0032] In this embodiment, preferably, the specific steps of the multimodal dynamic risk assessment are as follows: The feature matrix of the scenario generated by the hybrid adversarial generative network model is input into the multimodal dynamic risk assessment network, which contains information on physical and logical flows, and outputs a comprehensive risk value. ; Physical flow processing: Use 3D convolutional layers to extract spatiotemporal features and check whether there are acceleration changes beyond the safe range; Logical flow processing: traffic participants are represented as graph nodes, and the relative movement and position relationship between the vehicle and other traffic participants are predicted; Multi-head attention mechanism: 4 attention heads are calculated in parallel, focusing on different modal information, calculating attention weights, and generating comprehensive risk values. ; , in, =64 is the key vector dimension, 4 attention heads are calculated in parallel, Represented as the query key value calculation value, which is the comprehensive risk value , Represented as the query vector, the feature vector of the input sequence, represents the information of a certain position in the current sequence, Represented as a key vector, another feature vector associated with the query vector, which is used to measure the relevance or matching degree of the query vector to itself. Represented as a value vector, another feature vector associated with the query vector and the key vector, usually carrying important information, Represented as a scaling factor, used in attention calculations to prevent the dot product of the query vector and the key vector from being too large. Expressed as a function, it converts the attention weights into a probability distribution, ensuring that the sum of the weights of all attention heads is 1. Denoted as attention weight, is the product of two matrices, the result is a matrix, the scaling factor Normalize it; The steps of risk interpretability analysis are as follows: The comprehensive risk value Enter and set high-risk scenarios =0.85, output interpretability report; Counterfactual reasoning: Through gradient back propagation, the analysis model is based on the input comprehensive risk value Draw high-risk conclusions, calculate feature importance, and identify key risk sources; Risk heat map generation: mark high-risk areas in the feature space and use heat maps to show the distribution of risk factors; , in, Indicates a high-risk area. Respectively represented as input comprehensive risk value The eigenvalues ​​in Expressed as an integral variable, it controls the degree of disturbance, Expressed as risk function versus feature The gradient of It should be noted that multimodal dynamic risk assessment generates comprehensive and multi-dimensional risk assessment results through the processing of physical flow and logical flow combined with a multi-head attention mechanism. Dynamic obstacle modeling simulates the behavioral logic of dynamic obstacles through parameters such as physical properties, social exclusion, frontal driving force and random disturbances to ensure the authenticity and flexibility of modeling. Risk interpretability analysis further provides interpretability support for high-risk scenarios through counterfactual reasoning and heat map generation, helping to optimize model and scenario configuration; it not only improves the model's ability to capture dynamic obstacles and risk factors, but also ensures the reliability and safety of the system in complex traffic scenarios through multimodal fusion and interpretability analysis. It has high advantages and practical value.

[0033] refer to Figure 2 , a deep learning-based smart car extreme test scenario generation system, including a hybrid adversarial generative network model, a data input module, a preprocessing module, an edge cloud module and an interaction module; The hybrid generative adversarial network model uses a generative adversarial network and a variational autoencoder for fusion, and then performs dynamic adversarial training to generate extreme test scenarios and perform extreme tests on smart vehicles; The data input module is used to input multi-source data sets into the hybrid adversarial generative network model, perform dynamic adversarial training, and simulate extreme test scenarios; and realize the input of various data of the smart car to perform extreme scenario test simulation; The preprocessing module is used to perform noise reduction, anomaly elimination, replacement and normalization on the multi-source data set, and enhance and annotate the multi-source data set to generate a dense data set; The edge cloud module is used to generate and train a hybrid adversarial generative network model based on a multi-source data set, and is used to perform extreme scenario test simulations after inputting various data of the smart car, and to generate results of the extreme scenario tests, and transmit them to the interactive module through the Internet of Things or the 5G network; The interactive module is used to display the scenes simulated by the extreme scenario test of the smart car, and to realize control adjustment, set according to requirements, and receive and display the results of the extreme scenario test.

[0034] In this embodiment, preferably, the calculation process of the noise reduction is as follows: For multi-source datasets , use windows of different scales to perform median filtering, and then perform weighted fusion on the filtering results of different scales; , in, Represents the output data after filtering; Indicates the number of scales; Representation scale The weights are used to perform weighted averaging of filtering results of different scales; Indicated in scale The result of median filtering the data is as follows: Representative scale The corresponding neighborhood window; The calculation of the abnormal elimination replacement is as follows: The Z-score of the data is obtained by calculating the difference between the data and the mean and dividing it by the standard deviation; , in, is the value of the multi-source dataset, is the mean of the multi-source dataset, is the standard deviation of the multi-source dataset, Points with values ​​greater than 3 or less than -3 are considered outliers and are removed; The vacancies removed by outliers are filled by the mean, and the mean is calculated as follows: , in, is the value of non-missing data, is the number of non-missing data; The calculation of the normalization process is as follows: , in, is the minimum value in the multi-source dataset, is the maximum value in the multi-source dataset, Represented as normalized data, the multi-source data set is scaled to the range of [0, 1]; It should be noted that the combination of median filtering and weighted fusion can effectively remove noise and retain the effective components of the signal. Windows of different scales capture noise of different frequencies, which improves the robustness of noise reduction. The weighted average of the filtering results of different scales balances the noise reduction effects of different scales and avoids the shortcomings of single filtering. Z-score standardization makes outliers prominent and easy to remove quickly. Mean filling is simple and effective, avoiding complex interpolation methods and suitable for large-scale data processing. Eliminating outliers and filling gaps significantly improves data quality and provides more complete data for subsequent analysis. Scaling the data to the range of [0,1] prevents gradient explosion or disappearance problems and stabilizes the training process. The normalized data makes the comparison of different features more intuitive and facilitates model training and tuning. The combination of denoising, anomaly removal and normalization handles data noise, anomalies and scale problems and improves data quality.

[0035] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired means (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, DVD ), or semiconductor media. The semiconductor media may be a solid state drive.

[0036] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0037] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0038] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only some logical function divisions. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0039] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0040] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0041] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U Disk, mobile hard disk, read-only memory ( read - onlymemory , ROM )、Random Access Memory( RandomAccessMemory , RAM ), magnetic disks, optical disks, and other media that can store program codes.

[0042] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0043] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for generating extreme test scenarios for smart cars based on deep learning, characterized in that: The following steps are involved: Build a hybrid generative adversarial network scenario: Use the generative adversarial network and variational autoencoder to fuse and generate a hybrid generative adversarial network model; Data collection and preprocessing: Collect real multi-source data sets, enhance and annotate them, and generate dense data sets; Dynamic adversarial training: The generated dense data set is input into the hybrid adversarial generative network model for training. High-risk extreme scenarios are generated through a dual-channel adversarial mechanism for dynamic adversarial training. Simulated extreme testing: Input various data of the smart car into the trained adversarial generative network model to conduct tests in extreme environments.

2. The method for generating extreme test scenarios for smart cars based on deep learning according to claim 1, characterized in that: The dynamic adversarial training also includes the following steps: Multimodal dynamic risk assessment: A dual-stream network is used to process the physical and logical constraints of the scene respectively. The risk factors are then weighted and integrated by introducing an attention mechanism. Then, risk interpretability analysis is performed to generate quantifiable risk indicators. Finally, the quantifiable risk indicators are input into the hybrid adversarial generative network model to promote the generation of high-risk scenarios. Evolutionary algorithm optimization and simulation verification: Multi-objective optimization is performed on extreme scenarios, and then verified through a digital twin closed loop, and the verification results are input into the hybrid adversarial generative network model.

3. The method for generating extreme test scenarios for smart cars based on deep learning according to claim 1, characterized in that: The hybrid generative adversarial network model performs probabilistic modeling of the latent space through the variational autoencoder; Assume that the input data set is dense , output potential distribution parameters ; Sampling to obtain latent variables ; Control the regularization strength of the latent space through KL divergence constraints; ， in, ; in, Expressed as the divergence function of the space, is represented as the regularization coefficient, , Represents the conditional distribution defined by the encoder, given an input dense dataset , latent variables The posterior distribution of Represented as a standard normal distribution: the mean is 0 and the covariance matrix is ​​the identity matrix , It is expressed as an indicator to measure the difference between two probability distributions. Represented as the mean vector output by the encoder network, Denoted as the standard deviation vector of the encoder network output.

4. The method for generating extreme test scenarios for smart cars based on deep learning according to claim 3 is characterized in that: The hybrid adversarial generative network model includes a generator, a discriminator and a risk assessment module; Generator is used to input latent variables , output generation scene ; The discriminator is used to evaluate: physical plausibility and risk level ; The adversarial loss function is calculated as follows: ， The total loss function is a weighted sum of multiple objectives: ； in, Expressed as the total loss function of the generative adversarial network, Expressed as the expected value of the calculation, Represented as the discriminator for dense data sets The probability of discrimination, It is represented as the generator outputting the generated scene according to the latent variable, and the latent variable is the noise vector. Represented as the discriminator to generate samples The probability of discrimination, Represented as a dense dataset score term, is represented as the generated data score term, and They are respectively represented as GAN adversarial loss weight, KL divergence loss weight and risk loss weight, and the initial fixed weights are set to , Expressed as the total loss function, Expressed as the divergence function of the space, It is expressed as risk adversarial loss function.

5. The method for generating extreme test scenarios for smart cars based on deep learning according to claim 1, characterized in that: The steps of data collection and preprocessing are as follows: Multi-source data collection: extract accident clips from the real road test database; collection parameters include: sensor data, environmental parameters and driving behavior; Preprocess the data: filter the collected data, detect abnormal data in the data, remove and replace the abnormal data, and then normalize the data to reduce the dimension of the data; Data enhancement and annotation: Perform 3D reconstruction and physical parameter interpolation on the accident fragment to generate a dense data set. The steps are as follows; Extract features from the accident fragments and generate feature maps based on the extracted features; Then, point cloud computing of the accident fragment is realized, and the point cloud distribution of the missing area is predicted; First, the accident fragment is divided into regions, and the missing regions are supplemented by feature maps near the missing regions. The feature maps are first normalized to generate uniform reference points in the missing regions, and then the predicted point cloud data points are generated by the mean or variance of the feature maps. Environmental variable expansion: Randomly change weather parameters in the reconstructed scene; add dynamic obstacles to enhance and annotate the data.

6. The method for generating extreme test scenarios for smart cars based on deep learning according to claim 5, characterized in that: The characteristic map of the missing area is completed: Point cloud with missing areas , the missing regions are marked as , local feature map of each point ; Local feature aggregation: For each point , in radius Aggregate K nearest neighbor features within the neighborhood: ， in, Expressed as The neighborhood point index of Expressed as The neighboring points of Expressed as a learnable feature transformation function, Indicates that the maximum pooling retains significant features, Represented as the local feature map of the missing area, Expressed as The local feature map of the neighborhood points, It is expressed as the operation function of a multi-layer perceptron; Local feature maps of missing areas Combine the mean or variance to generate predicted point cloud data points.

7. The method for generating extreme test scenarios for smart cars based on deep learning according to claim 5, characterized in that: The calculation of adding dynamic obstacles is as follows: Dynamic Obstacle Modeling: ， in, Represented as a dynamic obstacle model, Expressed as the additional driving force of the dynamic obstacle model, The social repulsion force expressed as a dynamic obstacle model, is represented as a random perturbation modulo the dynamic obstacle.

8. The method for generating extreme test scenarios for smart cars based on deep learning according to claim 2, characterized in that: The specific steps of the multimodal dynamic risk assessment are as follows: The feature matrix of the scenario generated by the hybrid adversarial generative network model is input into the multimodal dynamic risk assessment network, which contains information on physical and logical flows, and outputs a comprehensive risk value. ; Physical flow processing: Use 3D convolutional layers to extract spatiotemporal features and check whether there are acceleration changes beyond the safe range; Logical flow processing: traffic participants are represented as graph nodes, and the relative movement and position relationship between the vehicle and other traffic participants are predicted; Multi-head attention mechanism: 4 attention heads are calculated in parallel, focusing on different modal information, calculating attention weights, and generating comprehensive risk values. ; ， in, =64 is the key vector dimension, 4 attention heads are calculated in parallel, Represents the calculated value of the query key value, which is the comprehensive risk value , Represented as the query vector, the feature vector of the input sequence, represents the information of a certain position in the current sequence, Represented as a key vector, another feature vector associated with the query vector, which is used to measure the relevance or matching degree of the query vector to itself. Represented as a value vector, another feature vector associated with the query vector and the key vector, usually carrying important information, Represented as a scaling factor, used in attention calculations to prevent the dot product of the query vector and the key vector from being too large. Expressed as a function, it converts the attention weights into a probability distribution, ensuring that the sum of the weights of all attention heads is 1. Denoted as attention weight, is the product of two matrices, the result is a matrix, the scaling factor Normalize it; The steps of risk interpretability analysis are as follows: The comprehensive risk value Enter and set high-risk scenarios =0.85, output interpretability report; Counterfactual reasoning: Through gradient back propagation, the analysis model is based on the input comprehensive risk value Draw high-risk conclusions, calculate feature importance, and identify key risk sources; Risk heat map generation: mark high-risk areas in the feature space and use heat maps to show the distribution of risk factors; ， in, Indicates a high-risk area. Respectively represented as input comprehensive risk value The eigenvalues ​​in Expressed as an integral variable, it controls the degree of disturbance, Expressed as risk function versus feature gradient.

9. A system for generating extreme test scenarios for smart cars based on deep learning, characterized in that: It includes a hybrid adversarial generative network model, a data input module, a preprocessing module, an edge cloud module and an interaction module; The hybrid generative adversarial network model uses a generative adversarial network and a variational autoencoder for fusion, and then performs dynamic adversarial training to generate extreme test scenarios and perform extreme tests on smart vehicles; The data input module is used to input multi-source data sets into the hybrid adversarial generative network model, perform dynamic adversarial training, and simulate extreme test scenarios; and realize the input of various data of the smart car to perform extreme scenario test simulation; The preprocessing module is used to perform noise reduction, anomaly elimination, replacement and normalization on the multi-source data set, and enhance and annotate the multi-source data set to generate a dense data set; The edge cloud module is used to generate and train a hybrid adversarial generative network model based on a multi-source data set, and is used to perform extreme scenario test simulations after inputting various data of the smart car, and to generate results of the extreme scenario tests, and transmit them to the interactive module through the Internet of Things or the 5G network; The interactive module is used to display the scenes simulated by the extreme scenario test of the smart car, and to realize control adjustment, set according to requirements, and receive and display the results of the extreme scenario test.

10. The system for generating extreme test scenarios for smart cars based on deep learning according to claim 9, characterized in that: The calculation process of the noise reduction is as follows: For multi-source datasets , use windows of different scales to perform median filtering, and then perform weighted fusion on the filtering results of different scales; ， in, Represents the output data after filtering; Indicates the number of scales; Representation scale The weights are used to perform weighted averaging of filtering results of different scales; Indicated in scale The result of median filtering the data is as follows: Representative scale The corresponding neighborhood window; The calculation of the abnormal elimination replacement is as follows: The Z-score of the data is obtained by calculating the difference between the data and the mean and dividing it by the standard deviation; ， in, is the value of the multi-source dataset, is the mean of the multi-source dataset, is the standard deviation of the multi-source dataset, Points with values ​​greater than 3 or less than -3 are considered outliers and are removed; The vacancies removed by outliers are filled by the mean, and the mean is calculated as follows: ， in, is the value of non-missing data, is the number of non-missing data; The calculation of the normalization process is as follows: ， in, is the minimum value in the multi-source dataset, is the maximum value in the multi-source dataset, Represented as normalized data, the multi-source datasets are scaled to the range of [0, 1].

Images