Driving condition data enhancement method and related equipment

Abstract

The invention discloses a driving condition data enhancement method and related equipment, and relates to the technical field of data processing.The method comprises the steps that real-time driving data of a driver in the current automatic driving state are obtained, and the real-time driving data comprise chassis control data, picture data and vehicle state data; based on a shadow mode, comparing the real-time driving data with theoretical driving data to obtain driving extreme working condition data, the theoretical driving data being data obtained through an automatic driving model; and performing data enhancement on the driving extreme working condition data based on a target diffusion model to generate driving working condition enhanced data conforming to target quality. According to the method, data enhancement in a shadow mode is realized based on the diffusion model, and the scene data set is expanded, so that the adaptive capacity and decision accuracy of the automatic driving system are improved, and the purpose of remarkably improving efficiency, utilization rate and quality is realized.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a driving condition data enhancement method and related equipment. Background Art

[0002] With the continuous development of autonomous driving technology, autonomous driving systems have higher and higher requirements for the collection and processing of environmental data. However, existing data collection methods have certain limitations, such as low data quality and insufficient reliability, which affect the adaptability and decision-making accuracy of autonomous driving systems.

[0003] Shadow Mode is an advanced data collection and analysis technology designed to improve the performance and safety of autonomous driving systems. This mode silently monitors the driving environment while the vehicle is in motion, collecting sensor data and decision information without interfering with actual driving. In complex or uncommon traffic situations, Shadow Mode may not fully capture all potential variables, and for extreme or rare situations, Shadow Mode may not provide enough learning data.

[0004] Data augmentation refers to the process of generating new data samples by performing a series of transformations on the original data to expand the data set and improve the generalization and robustness of the model. However, existing data augmentation techniques can often only generate data that is similar to the original data in a certain dimension and cannot fully capture the full picture of the data distribution.

[0005] Therefore, it is necessary to propose a driving condition data enhancement method to solve the above problems. Summary of the invention

[0006] A series of simplified concepts are introduced in the Summary of the Invention section, which will be further described in detail in the Detailed Description of the Invention section. The Summary of the Invention section of this application does not mean to attempt to define the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the scope of protection of the claimed technical solution.

[0007] This application specifically includes the following aspects:

[0008] In a first aspect, the present application proposes a driving condition data enhancement method, comprising:

[0009] Acquire real-time driving data of the driver in the current automatic driving state, wherein the real-time driving data includes chassis control data, image data and vehicle state data;

[0010] Comparing the real-time driving data with theoretical driving data based on the shadow mode to obtain extreme driving condition data, wherein the theoretical driving data is data obtained through an automatic driving model;

[0011] The extreme driving condition data is enhanced based on a target diffusion model to generate driving condition enhanced data that meets the target quality.

[0012] In a feasible implementation manner, the real-time driving data is compared with the theoretical driving data based on the shadow mode to obtain the driving extreme condition data, including:

[0013] Calculating data deviation amplitudes of different data types respectively according to the real-time driving data and the theoretical driving data;

[0014] If the data deviation amplitude of the current data type is greater than the preset value, it is determined that there is a deviation between the theoretical driving data corresponding to the current data type and the real-time driving data;

[0015] If the data deviation amplitude of all data types exceeds a preset threshold, the current scene is determined to be an extreme operating condition, and the current driving data is recorded as the extreme driving condition data.

[0016] In a feasible implementation manner, data enhancement is performed on the extreme driving condition data based on a target diffusion model, including:

[0017] Performing preliminary data diffusion on the image data;

[0018] Performing data enhancement on the chassis control data under road condition constraints;

[0019] The vehicle state data is subjected to data enhancement under vehicle state constraints.

[0020] In a feasible implementation manner, the image data is initially diffused, and the specific formula is:

[0021]

[0022] Among them, a is the preset diffusion coefficient, ε is continuous Gaussian noise, x is image data, and t is time.

[0023] In a feasible implementation manner, the chassis control data is enhanced under the road condition constraint, and the specific formula is:

[0024]

[0025] Among them, x is the image data; ε θ is a neural network model for denoising, and the parameters θ are obtained through training; σ t is the preset variance; z t is the preset random Gaussian noise; line is the preset value of the line feature; tReduce the amount of image tearing or reduce line discontinuities.

[0026] In a feasible implementation manner, data enhancement is performed on the vehicle state data under vehicle state constraints, and the specific formula is:

[0027]

[0028] Among them, x is the image data; a is the preset diffusion coefficient; ε θ is a neural network model for denoising, and the parameters θ are obtained through training; σ t is the preset variance; z t is the preset random Gaussian noise; ∑H t It is the cumulative value of all status data.

[0029] In a feasible implementation manner, data enhancement is performed on the extreme driving condition data based on a target diffusion model to generate enhanced driving condition data that meets the target quality, further comprising: data enhancement of actual driving conditions is performed on the enhanced extreme driving condition data based on an objective function, wherein the specific formula of the objective function is:

[0030]

[0031] Among them, L(θ) is the objective function; ε is continuous Gaussian noise; x is the image data; t is time; ||.|| is the norm, the subscript 2 represents the norm of L2, and the superscript 2 represents the square.

[0032] In a second aspect, the present application proposes a driving condition data enhancement system, comprising:

[0033] A data acquisition module is used to obtain real-time driving data of the driver in the current automatic driving state, wherein the real-time driving data includes chassis control data, image data and vehicle status data;

[0034] A data comparison module, used for comparing the real-time driving data with theoretical driving data based on a shadow mode to obtain extreme driving condition data, wherein the theoretical driving data is data obtained through an automatic driving model;

[0035] The data enhancement module is used to enhance the extreme driving condition data based on a target diffusion model to generate enhanced driving condition data that meets the target quality.

[0036] In a third aspect, an electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the steps of the driving condition data enhancement method of any one of the first aspects described above when executing the computer program stored in the memory.

[0037] In a fourth aspect, the present application further proposes a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the driving condition data enhancement method of any one of the first aspects are implemented.

[0038] In summary, the driving condition data enhancement method proposed in this application realizes data enhancement under the shadow mode based on the diffusion model, expands the scene data set, thereby improving the adaptability and decision-making accuracy of the autonomous driving system, and achieving the goal of significantly improving efficiency, utilization, and quality.

[0039] The driving condition data enhancement method proposed in this application and other advantages, objectives and features of this application will be partially reflected through the following description, and will also be partially understood by technical personnel in this field through research and practice of this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present specification. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0041] Figure 1 A schematic diagram of a driving condition data enhancement method provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of a process framework of a driving condition data enhancement method provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of functional modules of a driving condition data enhancement system provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of the structure of an electronic device for enhancing driving condition data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to better understand the technical solutions provided by the embodiments of this specification, the technical solutions of the embodiments of this specification are described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.

[0046] In this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the statement "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. The term "more than two" includes two or more than two situations.

[0047] In the prior art, the generation and amplification of road data under various weather conditions are mainly achieved through text descriptions. Therefore, there are problems such as text descriptions alone cannot accurately represent the driving behavior and environment at the time; the road data is generated and amplified only for various weather conditions, and the scope of data set enhancement is limited; the amount of data generated by autonomous driving itself is extremely large, and increasing the data through a diffusion model will result in greater resource consumption.

[0048] Based on the above problems, the present application provides a driving condition data enhancement method to solve the limitations of existing autonomous driving data processing.

[0049] See also Figure 1 , is a schematic flow chart of a driving condition data enhancement method provided in an embodiment of the present application, which may specifically include:

[0050] S110, obtaining real-time driving data of the driver in the current automatic driving state, wherein the real-time driving data includes chassis control data, image data, and vehicle state data.

[0051] For example, chassis control data, image data and vehicle status data are collected. These data cover driving behavior, vehicle perception, vehicle regulation and control, vehicle positioning and other vehicle driving data, thereby depicting complex real driving scenarios in detail.

[0052] Specifically, chassis control data also includes steering wheel angle: records the angle and speed at which the driver turns the steering wheel, and is one of the most direct data in driving behavior; accelerator pedal and brake pedal: including the depth and duration of pedaling, analyzing the intensity of acceleration and braking; throttle and brake opening: reflects the driver's intention to accelerate and decelerate; driving speed: the real-time speed of the vehicle.

[0053] Vehicle perception data also includes: Camera: Captures visual information of the surrounding environment for identifying road signs, traffic lights, pedestrians, vehicles, etc.; Radar: Detects the distance and speed of surrounding objects by emitting radio waves and receiving reflected waves; Lidar (laser radar): Acquires high-precision three-dimensional environmental information by emitting lasers and measuring the time difference of reflected light; Inertial navigation: Provides vehicle position, speed and direction information.

[0054] Vehicle status data also includes: power-on status: determining the continuous state of the driving process; kinetic energy recovery status: estimating the driving status based on the changes in kinetic energy recovery; positioning status: judging the location status of the vehicle, etc.; vehicle communication: exchanging information with other vehicles or infrastructure to improve traffic flow and safety.

[0055] S120. Compare the real-time driving data with the theoretical driving data based on the shadow mode to obtain extreme driving condition data, wherein the theoretical driving data is data obtained through an automatic driving model.

[0056] For example, the use of shadow mode for data collection solves the problem of autonomous driving data collection cost and efficiency, and obtains high-quality data for extreme scenarios. Originally, the verification of a scene required the collection of data from drivers for up to a month, and the collection cost and efficiency could not meet the needs of rapid iteration and update of intelligent driving functions.

[0057] S130 . Performing data enhancement on the extreme driving condition data based on a target diffusion model to generate driving condition enhanced data that meets target quality.

[0058] For example, the process of data enhancement is similar to "restoring" a photo. Through data enhancement, the corresponding degree of noise can be removed and the original clear image can be reconstructed as much as possible.

[0059] In summary, the driving condition data enhancement method proposed in the embodiment of the present application realizes data enhancement under the shadow mode through the diffusion model, expands the scene data set, thereby improving the adaptability and decision-making accuracy of the autonomous driving system, and achieving significant improvements in efficiency, utilization, and quality.

[0060] In some examples, the real-time driving data is compared with the theoretical driving data based on the shadow mode to obtain the driving extreme condition data, including:

[0061] Calculating data deviation amplitudes of different data types respectively according to the real-time driving data and the theoretical driving data;

[0062] If the data deviation amplitude of the current data type is greater than the preset value, it is determined that there is a deviation between the theoretical driving data corresponding to the current data type and the real-time driving data;

[0063] If the data deviation amplitude of all data types exceeds a preset threshold, the current scene is determined to be an extreme operating condition, and the current driving data is recorded as the extreme driving condition data.

[0064] For example, the shadow mode of this application mainly compares the above-mentioned real-time driving data with the theoretical driving data, that is, compares the driver's actual control data with the data inferred by the autonomous driving model. Excluding sensor and data accuracy issues, it is set that the deviation of numerical data exceeds 5% and is considered to exist. The calculation formula is as follows:

[0065]

[0066] When θ>5%, it is determined that there is a deviation between the data and the driver's actual operation.

[0067] When the deviation of all operation data accounts for more than a certain threshold of the total data type, P is initially set to 35% in this embodiment. Different weights W and thresholds P are adjusted accordingly according to the data volume distribution. The calculation formula is as follows:

[0068]

[0069] Among them, Ni represents each data field, and Wi is the weight assigned according to the characteristics of each field. When P exceeds 35%, it is determined that the result of the current model reasoning is quite different from the current driver operation. The scene at this time is judged as an "extreme working condition" and data recording and saving are started. The vehicle side adopts direct deployment of domain control data lossless high compression to ensure zero signal loss, reduce upload pressure, and save storage, computing, network and other resources for subsequent diffusion model enhancement. At the same time, the vehicle-cloud collaboration, verification and retransmission mechanism ensure the integrity of the data.

[0070] Specifically, the data collection technology for extreme working conditions in shadow mode and the scope of quantitative data coverage are framed. If the deviation between the results of model reasoning and the actual data of the driver exceeds 35% (configurable ratio) through quantification, the corresponding data will be regarded as extreme working conditions. The entire quantification process covers chassis control data, vehicle perception data, and vehicle status data. Through flexible configuration strategies, we focus on efficient and high-quality data set collection, optimize storage and computing resources, easily identify specific trigger scenarios, and accumulate a rich driving scenario library.

[0071] In some examples, data enhancement is performed on the extreme driving condition data based on a target diffusion model, including:

[0072] Performing preliminary data diffusion on the image data;

[0073] Performing data enhancement on the chassis control data under road condition constraints;

[0074] The vehicle state data is subjected to data enhancement under vehicle state constraints.

[0075] For example, in the data enhancement stage, the denoising process adds constraints in stages. Through a series of intermediate state images with different noise levels, the perception information, control information, vehicle status and other information are integrated into the denoising process in steps and stages. The model can generate images that meet specific conditional constraints. Conditional diffusion achieves high-quality image reconstruction by learning how to remove noise of different degrees. Generate effective, high-quality enhanced images that meet the required driving behavior and vehicle conditions from pure noise images. The generalization ability and robustness of the autonomous driving algorithm are greatly improved.

[0076] In some examples, the image data is initially diffused, and the specific formula is:

[0077]

[0078] Among them, a is the preset diffusion coefficient, ε is continuous Gaussian noise, x is image data, and t is time.

[0079] Exemplarily, the prior art is to fuse the original image data with different degrees of noise through noise sampling or multi-noise sampling technology to generate a series of intermediate state images with different noise levels. Then, the corresponding degree of noise is removed from the intermediate state image through conditional diffusion, and the original clear image is reconstructed as much as possible. The present application also adopts noise sampling technology, and the specific process is: according to a predefined noise level parameter (for example, from 0 to 1), a plurality of different noise level values ​​are sampled. For each noise level value, according to the formula of the diffusion model, the original data (such as an image) is weightedly fused with a random Gaussian noise to obtain a corresponding intermediate state image as the preliminary diffusion data. The specific calculation formula is as shown in formula (3) above.

[0080] In some examples, the chassis control data is enhanced under road condition constraints, and the specific formula is:

[0081]

[0082] Among them, x is the image data; ε θ is a neural network model for denoising, and the parameters θ are obtained through training; σ t is the preset variance; z t is the preset random Gaussian noise; line is the preset value of the line feature; tReduce the amount of image tearing or reduce line discontinuities.

[0083] Exemplarily, the existing data enhancement technology is to achieve denoising of intermediate state images through conditional diffusion, thereby achieving high-quality image generation. However, the "conditions" constraints of the conditional diffusion process are relatively few, and the final generated image deviates more and more from the semantics contained in the original image as the denoising process progresses. Although the richness is improved, the usability will be greatly reduced. The data enhancement under road conditions provided by this application increases the regularization term of line-related features, so that relevant information such as road boundaries, lane lines, and traffic signs can be enhanced during the conditional diffusion process. Sufficient usability is guaranteed in the generated rich images. For example, the road image of left-turn driving will give priority to generating enhanced images such as right turns, straight driving, and continuous turns, and can also be enhanced by converting dotted lines to solid lines and solid lines to dotted lines on the ground. It is greatly guaranteed that the enhanced data can be used for research and development related to autonomous driving. The specific calculation formula is the above formula (4).

[0084] In the above formula (4), λ line It is the preset value for line features to prevent excessive regularization. t It is to ensure that the image does not have too much tearing or line discontinuity. line The larger the value is, the stronger the corresponding line information is. In the process of removing noise, the line features are retained first. The enhanced image data focuses more on the changes related to lines such as roads or lane lines, which greatly improves the usability of the enhanced data. On the contrary, the line features are weakened, which is close to the ordinary diffusion model.

[0085] In some examples, the vehicle state data is enhanced under vehicle state constraints, and the specific formula is:

[0086]

[0087] Among them, x is the image data; a is the preset diffusion coefficient; ε θ is a neural network model for denoising, and the parameters θ are obtained through training; σ t is the preset variance; z t is the preset random Gaussian noise; ∑H t It is the cumulative value of all status data.

[0088] Exemplarily, this embodiment combines the data of the real-time operating status of the vehicle including charging and discharging, GPS positioning status, vehicle communication status and other data, and reflects the subtle differences in real vehicle usage scenarios through status data. Adding subtle differences in the diffusion algorithm model can generate enhanced data for related vehicle usage scenarios.

[0089] The data processing methods are different for different states, including: the charging and discharging state is converted into a 1-0 value according to the discharge as 1 and the charge as 0, mainly to obtain the scenarios of kinetic energy usage and kinetic energy recovery during acceleration and deceleration of the vehicle during driving; the GPS positioning state is divided into levels 1-5 according to the signal strength. Places with strong signals (4-5 grids) represent open scenes with fewer obstructions, and places with weak signals (1-3 grids) are likely to be in tunnels, under bridges, or scenes with more obstructions; the vehicle communication state is converted into a 0-1 value according to whether it is connected to the network. Even if there are more scenes in urban areas and the vehicle communication state remains connected to the network, it is also representative for scenes in areas without network.

[0090] Through the processed data values, further constraint conditions are enhanced. The specific calculation formula is as follows: t It is the cumulative value of all states. When added to the original noise term, the diffusion model will incorporate the vehicle state information during the denoising and enhancement process. The enhanced image scene will be more accurate and consistent with the scene during vehicle driving.

[0091] In some examples, data enhancement is performed on the extreme driving condition data based on a target diffusion model to generate enhanced driving condition data that meets the target quality, and further includes: data enhancement of actual driving conditions is performed on the enhanced extreme driving condition data based on an objective function, wherein a specific formula of the objective function is:

[0092]

[0093] Among them, L(θ) is the objective function; ε is continuous Gaussian noise; x is the image data; t is time; ||.|| is the norm, the subscript 2 represents the norm of L2, and the superscript 2 represents the square.

[0094] For example, data enhancement is achieved through end-to-end learning, which maps from noisy data to target results. After improvement and enhancement, this has become one of the mainstream solutions in the field of generative models and has demonstrated very powerful generation capabilities. A large amount of unlabeled data can be used for pre-training to avoid the expensive cost of manual labeling. Secondly, the task objectives are universal, which helps the model learn the underlying laws and representations of the data. Specifically, the above formula (6) is used as the objective function to achieve data enhancement close to the actual driving conditions.

[0095] Specifically, in the above formula (6), the model is required to go from the intermediate state x t and time t predict the true noise ε, that is, to remove x to the maximum extent tThe noise part in the image can be obtained through optimization methods such as gradient descent to obtain appropriate model parameters θ. Its goal is to remove the corresponding degree of noise from the intermediate state image and reconstruct the original clear image as much as possible. This process is a bit like "repairing" a photo. If it is a slight "defacement", the model only needs to remove a little noise to restore the result very close to the original image; if it is a serious "defacement", the model needs to remove a lot of noise through the learned knowledge and intelligently complete the damaged image content and details. During the training process, the diffusion model repeatedly observes a large number of intermediate state images and their corresponding noise level labels, and learns how to effectively remove different degrees of noise by optimizing the objective function. The objective function measures the difference between the denoising result of the model and the real image, and adjusts the model according to the difference. After sufficient training, the inverse mapping of the diffusion process can be mastered, that is, starting from any intermediate state, according to the given noise level conditions, the corresponding denoised image is generated, and finally the enhanced data close to the real working condition is obtained and uploaded to the cloud. This application fully exploits the advantages of the diffusion model itself and the rich processed data to build a richer, more diverse, and more robust training data set, thereby effectively improving the generalization ability of the intelligent driving system.

[0096] In summary, the driving condition data enhancement method provided in this application is as follows: Figure 2 As shown in the figure, based on the shadow mode, the driver's real-time driving data is compared with the theoretical data obtained through driving model inference, so as to record the extreme driving condition data, compress this condition data, and save the compressed condition data to a local small file. Based on the diffusion model, the image data is initially diffused, and the chassis control data and vehicle status data are constrained to complete the data enhancement under the real road conditions and the convergence to the real state. Finally, the data enhancement close to the real driving condition is completed through the objective function, and the results are uploaded to the cloud database.

[0097] Furthermore, the present application first compares the driving behavior of the driver with the decision generated by the model algorithm in the shadow mode of the car. Once the two are inconsistent, the scene is judged as an "extreme working condition" or a scene to be analyzed, which triggers the data to be transmitted back. The data transmitted back is detailed driving data on the real road, which cannot be described by ordinary text. Secondly, the car collects a large amount of rich data such as pictures, point clouds, and videos during driving. With the increase of the mileage of collection, the scenes involved will become more and more abundant, and the scope of data set enhancement will be wider. At the same time, in the process of diffusion enhancement, the control data of the chassis and the real state data of the vehicle are used as constraints to obtain enhanced data that is closer to the real scene. Finally, an efficient compression algorithm is used, and the transmitted data is concentrated on the "few and precise" working condition data. At the same time, the overall computing power occupancy is maintained at 10-15% of the vehicle computing power, and sufficient performance is retained to directly use the diffusion model, which can obtain high-quality enhanced data generated in near real time. Make full use of a small amount of data resources and computing resources to obtain the maximum data enhancement.

[0098] It should be noted that the above embodiments are only the best examples and are not limitations on the implementation methods of the present application.

[0099] Furthermore, the present application also proposes a driving condition data enhancement system, which is applied to any of the above-mentioned driving condition data enhancement methods, specifically as follows: Figure 3 FIG. 1 is a functional module diagram of a driving condition data enhancement system proposed in this application, including:

[0100] A data acquisition module 21 is used to obtain real-time driving data of the driver in the current automatic driving state, wherein the real-time driving data includes chassis control data, image data and vehicle state data;

[0101] A data comparison module 22, configured to compare the real-time driving data with theoretical driving data based on a shadow mode to obtain extreme driving condition data, wherein the theoretical driving data is data obtained through an automatic driving model;

[0102] The data enhancement module 23 is used to enhance the extreme driving condition data based on a target diffusion model to generate enhanced driving condition data that meets the target quality.

[0103] like Figure 4 As shown, an embodiment of the present application also provides an electronic device 300, including a processor 310, a memory 320, and a computer program 321 stored in the memory 320 and executable on the processor, and when the processor 310 executes the computer program 321, the steps of any of the above-mentioned methods for enhancing driving condition data are implemented.

[0104] Since the electronic device introduced in this embodiment is a device used to implement a driving condition data enhancement method in the embodiment of the present application, based on the method introduced in the embodiment of the present application, technical personnel in this field can understand the specific implementation method of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present application is not introduced in detail here. As long as the equipment used by technical personnel in this field to implement the method in the embodiment of the present application is within the scope of protection of this application.

[0105] In the specific implementation process, when the computer program 321 is executed by the processor, it can achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0106] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-readable program code.

[0108] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0111] An embodiment of the present application also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device executes the process of the driving condition data enhancement method.

[0112] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0114] In the several embodiments provided in the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0115] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0118] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0119] Although the preferred embodiments of this specification have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of this specification.

[0120] Obviously, those skilled in the art can make various changes and modifications to this specification without departing from the spirit and scope of this specification. Thus, if these modifications and variations of this specification fall within the scope of the claims of this specification and their equivalents, this specification is also intended to include these modifications and variations.

Claims

1. A driving condition data enhancement method, characterized in that: include: Acquire real-time driving data of the driver in the current automatic driving state, wherein the real-time driving data includes chassis control data, image data and vehicle state data; Comparing the real-time driving data with theoretical driving data based on the shadow mode to obtain extreme driving condition data, wherein the theoretical driving data is data obtained through an automatic driving model; The extreme driving condition data is enhanced based on a target diffusion model to generate driving condition enhanced data that meets the target quality.

2. The driving condition data enhancement method according to claim 1, characterized in that: The real-time driving data is compared with the theoretical driving data based on the shadow mode to obtain driving extreme condition data, including: Calculating data deviation amplitudes of different data types respectively according to the real-time driving data and the theoretical driving data; If the data deviation amplitude of the current data type is greater than the preset value, it is determined that there is a deviation between the theoretical driving data corresponding to the current data type and the real-time driving data; If the data deviation amplitude of all data types exceeds a preset threshold, the current scene is determined to be an extreme operating condition, and the current driving data is recorded as the extreme driving condition data.

3. The driving condition data enhancement method according to claim 1, characterized in that: The extreme driving condition data is enhanced based on a target diffusion model, including: Performing preliminary data diffusion on the image data; Performing data enhancement on the chassis control data under road condition constraints; The vehicle state data is subjected to data enhancement under vehicle state constraints.

4. The driving condition data enhancement method according to claim 3, characterized in that: The image data is initially diffused, and the specific formula is: Among them, a is the preset diffusion coefficient, ε is continuous Gaussian noise, x is image data, and t is time.

5. The driving condition data enhancement method according to claim 3, characterized in that: The chassis control data is enhanced under the constraints of road conditions. The specific formula is: Among them, x is the image data; ε θ is a neural network model for denoising, and the parameters θ are obtained through training; σ t is the preset variance; z t is the preset random Gaussian noise; line is the preset value of the line feature; t Reduce the amount of image tearing or reduce line discontinuities.

6. The driving condition data enhancement method according to claim 3, characterized in that: The vehicle state data is enhanced under the vehicle state constraint, and the specific formula is: Among them, x is the image data; a is the preset diffusion coefficient; ε θ is a neural network model for denoising, and the parameters θ are obtained through training; σ t is the preset variance; z t is the preset random Gaussian noise; ∑H t It is the cumulative value of all status data.

7. The driving condition data enhancement method according to claim 3, characterized in that: The method further comprises: performing data enhancement on the extreme driving condition data based on the target diffusion model to generate driving condition enhanced data that meets the target quality, and further comprising: performing data enhancement of actual driving conditions on the enhanced extreme driving condition data based on the objective function, wherein the specific formula of the objective function is: Among them, L(θ) is the objective function; ε is continuous Gaussian noise; x is the image data; t is time; ||.|| is the norm, the subscript 2 represents the norm of L2, and the superscript 2 represents the square.

8. A driving condition data enhancement system, characterized in that: include: A data acquisition module is used to obtain real-time driving data of the driver in the current automatic driving state, wherein the real-time driving data includes chassis control data, image data and vehicle status data; A data comparison module, used for comparing the real-time driving data with theoretical driving data based on a shadow mode to obtain extreme driving condition data, wherein the theoretical driving data is data obtained through an automatic driving model; The data enhancement module is used to enhance the extreme driving condition data based on a target diffusion model to generate enhanced driving condition data that meets the target quality.

9. An electronic device, comprising: A memory and a processor, wherein the processor is used to implement the steps of the driving condition data enhancement method as described in any one of claims 1 to 7 when executing a computer program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the driving condition data enhancement method according to any one of claims 1 to 7 are implemented.

Images