Autonomous driving simulation method, device and server

Abstract

The application provides an automatic driving simulation method and device and a server, and relates to the technical field of automatic driving, and comprises the following steps: performing data analysis and processing on steering wheel information to obtain a feature vector; performing double condition judgment processing on the feature vector through a dynamic scene generator to generate a first long tail training scene, so as to obtain a first automatic driving simulation training scene; performing collision risk monitoring processing on a vehicle in the first automatic driving simulation training scene through a double-mode controller, and switching the automatic driving simulation training mode to a manual driving simulation training mode when a risk value is greater than a preset risk threshold; inputting structured takeover data into a generative adversarial network model to obtain a second long tail training scene, and combining the second long tail training scene with the first automatic driving simulation training scene to obtain a second automatic driving simulation training scene, so as to re-perform automatic driving simulation training on the vehicle. The application can significantly reduce simulation cost and improve simulation accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to an automatic driving simulation method, device and server. BACKGROUND

[0002] With the rapid development of computer technology, automatic driving has attracted widespread attention. At present, the related technology proposes that professional hardware simulation system and pre-programmed scene library can be used for automatic driving simulation, or software pure virtual automatic generation of a large number of long-tail scenes can be used. However, the hardware cost of the professional hardware simulation system is high, and the scenes in the pre-programmed scene library are rigid and cannot dynamically respond to human factors input. The pure virtual automatic generation of a large number of long-tail scenes lacks real-time human-computer interaction interface. Therefore, the existing simulation scheme generally has the problems of high simulation cost and low simulation precision. SUMMARY

[0003] Therefore, the purpose of the present application is to provide an automatic driving simulation method, device and server, which can significantly reduce the simulation cost and improve the simulation precision.

[0004] In a first aspect, an embodiment of the present application provides an automatic driving simulation method, which comprises: obtaining steering wheel information of a vehicle in an automatic driving simulation training mode, and performing data analysis and processing on the steering wheel information to obtain a feature vector; performing double condition judgment processing on the feature vector through a dynamic scene generator to generate a first long-tail training scene, and combining the first long-tail training scene with an original automatic driving simulation training scene to obtain a first automatic driving simulation training scene; performing collision risk monitoring processing on the vehicle in the first automatic driving simulation training scene through a double-mode controller, and switching the automatic driving simulation training mode to a manual driving simulation training mode when the risk value is greater than a preset risk threshold; inputting structured takeover data of the vehicle in the manual driving simulation training mode into a generative adversarial network model to obtain a second long-tail training scene, and combining the second long-tail training scene with the first automatic driving simulation training scene to obtain a second automatic driving simulation training scene, so as to retrain the vehicle based on the second automatic driving simulation training scene.

[0005] In an embodiment, the step of performing data analysis and processing on the steering wheel information to obtain the feature vector comprises: converting the steering wheel information into a structured JSON data packet by using a software development kit; obtaining a steering angle velocity and a jitter frequency by performing difference calculation processing and fast Fourier transform processing on the structured JSON data packet, and determining the steering angle velocity and the jitter frequency as the feature vector.

[0006] In an embodiment, the step of generating the first long-tail training scene by the dynamic scene generator through double conditional judgment processing of the feature vector includes: real-time monitoring of the steering angular velocity and the jitter frequency in the feature vector; when the steering angular velocity is detected to be greater than a preset steering angular velocity threshold, generating a pedestrian ghost head long-tail scene of a preset travel speed at a preset distance in front of the vehicle; when the jitter frequency is detected to be greater than a preset jitter frequency threshold, generating a vehicle skid long-tail scene at the vehicle position; and determining the pedestrian ghost head long-tail scene and the vehicle skid long-tail scene as the first long-tail training scene.

[0007] In an embodiment, the step of generating the vehicle skid long-tail scene at the vehicle position includes: adjusting the tire model friction coefficient of the vehicle through the physical engine built in the Unreal Engine, generating a preset radius of oil stain road surface decal special effects at the vehicle position while reducing the tire model friction coefficient, to obtain the vehicle skid long-tail scene.

[0008] In an embodiment, the step of monitoring the collision risk of the vehicle in the first automatic driving simulation training scene by the dual-mode controller includes: obtaining the obstacle distance and the collision time between the vehicle and the obstacle in the first automatic driving simulation training scene; and sending the obstacle distance and the collision time to the risk value calculation model of the dual-mode controller for collision risk value calculation processing to obtain the risk value.

[0009] In an embodiment, the step of switching the automatic driving simulation training mode to the manual driving simulation training mode when the risk value is greater than a preset risk threshold includes: when the risk value is greater than the preset risk threshold and the duration is greater than a preset time threshold, determining that the automatic driving simulation training result is vehicle automatic driving failure, switching the simulation mode of the vehicle from the automatic driving simulation training mode to the manual driving simulation training mode, and simultaneously performing pulse vibration and light high alert.

[0010] In an embodiment, before the step of obtaining the second long-tail training scene by inputting the structured takeover data of the vehicle in the manual driving simulation training mode into the generative adversarial network model, includes: obtaining the steering wheel information, the vehicle speed, and the road surface friction coefficient of the vehicle in the manual driving simulation training mode; generating a trajectory point sequence according to the steering wheel information, the vehicle speed, and the road surface friction coefficient; and performing differential encoding processing on the trajectory point sequence after encapsulation to obtain the structured takeover data.

[0011] In a second aspect, the embodiments of the present application also provide an automatic driving simulation device, the device comprising: a data analysis module, which obtains steering wheel information of a vehicle in an automatic driving simulation training mode and performs data analysis processing on the steering wheel information to obtain a feature vector; a first long-tail training scene generation module, which performs double condition judgment processing on the feature vector through a dynamic scene generator to generate a first long-tail training scene, and combines the first long-tail training scene with an original automatic driving simulation training scene to obtain a first automatic driving simulation training scene; a risk monitoring module, which performs collision risk monitoring processing on the vehicle in the first automatic driving simulation training scene through a double-mode controller, and switches the automatic driving simulation training mode to a manual driving simulation training mode when a risk value is greater than a preset risk threshold; and a second long-tail training scene generation module, which inputs structured takeover data of the vehicle in the manual driving simulation training mode into a generative adversarial network model to obtain a second long-tail training scene, and combines the second long-tail training scene with the first automatic driving simulation training scene to obtain a second automatic driving simulation training scene, so as to re-perform automatic driving simulation training on the vehicle based on the second automatic driving simulation training scene.

[0012] In a third aspect, the embodiments of the present application also provide a server, comprising a processor and a memory, the memory storing computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the method of any one of the first aspect.

[0013] In a fourth aspect, the embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium storing computer executable instructions, and the computer executable instructions, when called and executed by a processor, cause the processor to implement the method of any one of the first aspect.

[0014] The embodiments of the present application bring the following beneficial effects:

[0015] The embodiment of the application provides an automatic driving simulation method, device and server, the method obtains the steering wheel information of a vehicle in an automatic driving simulation training mode, carries out data analysis and processing on the steering wheel information, obtains a feature vector, carries out double condition judgment processing on the feature vector through a dynamic scene generator, generates a first long tail training scene, combines the first long tail training scene with an original automatic driving simulation training scene, obtains a first automatic driving simulation training scene, then carries out collision risk monitoring processing on the vehicle in the first automatic driving simulation training scene through a double mode controller, and when the risk value is greater than a preset risk threshold, the automatic driving simulation training mode is switched to a manual driving simulation training mode, finally, structured takeover data of the vehicle in the manual driving simulation training mode is input into a generative adversarial network model, a second long tail training scene is obtained, the second long tail training scene is combined with the first automatic driving simulation training scene, a second automatic driving simulation training scene is obtained, and the vehicle is retrained based on the second automatic driving simulation training scene. The embodiment of the application can significantly reduce the simulation cost and improve the simulation precision.

[0016] Other features and advantages of the present application will be set forth in the descriptions below, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and attained by the structure particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more apparent, the following preferred embodiments are specifically described with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0019] Figure 1 A flowchart of an automatic driving simulation method provided by the embodiment of the present application is shown in the figure;

[0020] Figure 2 A specific flowchart of an automatic driving simulation method provided by the embodiment of the present application is shown in the figure;

[0021] Figure 3 A structural diagram of an automatic driving simulation device provided by the embodiment of the present application is shown in the figure;

[0022] Figure 4A structural schematic diagram of a server is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described below in connection with the embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0024] At present, with the rapid development of computer technology, automatic driving has attracted widespread attention. An automatic driving car refers to a car that realizes autonomous driving through artificial intelligence, sensors, navigation positioning, pattern recognition, machine vision, intelligent control and other technologies. The core goal is to replace human driving decisions through technology, and it has become a cutting-edge research direction in the fields of artificial intelligence, automotive engineering and transportation systems.

[0025] Compared with traditional manual driving cars, automatic driving cars have higher complexity and need to install laser radars, cameras, millimeter wave radars, domain controllers, IMUs and other devices. Before reaching a high level of safety, they cannot be directly tested on open roads and must rely on simulation devices for simulation testing. Automatic driving simulation technology is a computer virtual technology that simulates actual traffic systems and plays an important role in the entire process of automatic driving development, testing, landing and operation.

[0026] Automatic driving long-tail scenarios refer to extreme or special scenarios that occur with low probability but have high complexity and are difficult to predict during automatic driving. Such scenarios are diverse and include abnormal traffic behavior, rare objects, extreme weather conditions, complex road environments, special vehicle or pedestrian behavior, etc. Automatic driving long-tail scenarios pose many challenges to automatic driving systems. Since these rare events are often omitted in data sets, artificial intelligence models have difficulty learning and generalizing to these situations during training, affecting the reliability and safety of the system.

[0027] Currently, automatic driving long-tail scene simulation technology mainly relies on the following three schemes. One is to use professional hardware simulation systems, such as six-degree-of-freedom motion platforms to simulate vehicle bumps, and to use PreScan and other software for CAN protocol coupling, which has high hardware costs. The second is to use a pre-programmed scene library, which presets scene parameters (obstacle position, weather parameter), and the scene generation is rigid and cannot dynamically respond to human input. The third is to use NVIDIA Drive Sim to automatically generate a large number of long-tail scenes in a virtual environment, but this scheme has no real-time human-computer interaction interface. Based on this, the automatic driving simulation method provided by the embodiment can simulate automatic driving long-tail scenes based on consumer-level hardware, thereby significantly reducing the simulation cost and improving the simulation accuracy.

[0028] Referring to Figure 1 The flowchart of the automatic driving simulation method is shown in the figure, and the method mainly includes the following steps S102 to S108:

[0029] In step S102, the steering wheel information of the vehicle in the automatic driving simulation training mode is obtained, and the steering wheel information is subjected to data analysis and processing to obtain a feature vector.

[0030] In an embodiment, before the simulation training starts, an external simulation steering wheel needs to be connected through a USB interface, and the driver program corresponding to the external simulation steering wheel needs to be installed on the system. In addition, the steering wheel base also needs to be connected to the accelerator, brake, clutch and other pedal components, so as to form a complete driving control structure. Then, the steering wheel is initialized in the Unreal Engine environment, the steering angle range parameter is read, the force feedback intensity is set to zero, and the physical parameter is preset to define the minimum friction coefficient of the tire, thereby laying a foundation for dynamic scene modification.

[0031] In actual application, the steering wheel G923 can be connected to the PC host through the USB3.0 interface, the LogitechG HUB driver program is installed, the steering wheel base is connected to the pedal components (accelerator / brake / clutch) through the DIN 4-pin interface, a complete driving control unit is formed, a custom C++ plug-in is loaded in the Unreal Engine 5.5 environment, the LogiSteeringInitialize(false) function is called to initialize the steering wheel, the steering angle range parameter (1080°) is read and stored in the configuration file WheelSettings.ini, the force feedback intensity is set to zero, and the physical parameter preset file PhysicsParams.ini defines the minimum friction coefficient of the tire as 0.25 (corresponding to an oil-stained road).

[0032] In step S104, the feature vector is subjected to double condition judgment processing by the dynamic scene generator to generate a first long-tail training scene, and the first long-tail training scene is combined with the original autonomous driving simulation training scene to obtain a first autonomous driving simulation training scene.

[0033] In an embodiment, the dynamic scene generator is one of the core modules of the autonomous driving simulation system, responsible for dynamically generating or adjusting a virtual traffic scene according to real-time input or preset conditions to simulate rare or dangerous long-tail situations in the real world (such as pedestrian ghost head, sudden slippery road surface, etc.).

[0034] In step S106, the vehicle in the first autonomous driving simulation training scene is subjected to collision risk monitoring processing by the dual-mode controller, and when the risk value is greater than a preset risk threshold, the autonomous driving simulation training mode is switched to the manual driving simulation training mode.

[0035] In an embodiment, the dual-mode controller includes: mode A: autonomous driving effectiveness and mode B: driver takeover response, the data of mode A is derived from vehicle sensors and virtual scenes, used to monitor the risk value and determine whether to trigger the conversion of the manual driving simulation training mode, the data of mode B is derived from the actual signals of the steering wheel and the pedal, used to judge whether to open manual intervention according to the change of the opening of the steering angle or the throttle brake pedal.

[0036] In step S108, the structured takeover data of the vehicle in the manual driving simulation training mode is input into the generative adversarial network model to obtain a second long-tail training scene, and the second long-tail training scene is combined with the first autonomous driving simulation training scene to obtain a second autonomous driving simulation training scene, so as to re-train the vehicle based on the second autonomous driving simulation training scene.

[0037] In an embodiment, the structured takeover data can be used to construct a takeover database, and a generative adversarial network (GAN) can be trained based on the takeover database, wherein the input vector z of the generator of the generative adversarial network is spliced from three parts: 1, normalized obstacle avoidance trajectory Traj_norm (512-dimensional vector), 2, maximum steering angle velocity Steer_max (scalar), and 3, delay time Δt from warning to takeover (scalar). The generator network structure includes two fully connected layers (1024→2048 nodes) and Tanh activation function, and outputs a 256×256 matrix representing the height field data of the collapsed road surface. The newly generated long-tail scene of the trained generative adversarial network is injected into the event library and dynamically loaded as a triggerable event in subsequent simulation, realizing the closed-loop iteration of human behavior-scene variation-algorithm optimization.

[0038] The automatic driving simulation method provided by the embodiment of the present application can simulate the long-tail scene of automatic driving based on consumer-grade hardware, thereby significantly reducing the simulation cost and improving the simulation accuracy.

[0039] Referring to Figure 2 The embodiment of the present application also provides an implementation of improving the simulation accuracy without additional hardware cost by simulating the long-tail scene of automatic driving. For details, please refer to (1) to (5) as follows:

[0040] (1) The steering wheel information is converted into a structured JSON data packet by using a software development kit, and then the structured JSON data packet is processed by differential calculation and fast Fourier transform to obtain the steering angular velocity and the jitter frequency, and the steering angular velocity and the jitter frequency are determined as the feature vector.

[0041] In an implementation, the original steering wheel signal can be collected according to a first preset time interval, the steering wheel data is converted into a structured JSON data packet by using a Logitech SDK, and the key operation features are calculated: the steering angular velocity is obtained by a differential algorithm, and the jitter frequency is calculated by using fast Fourier transform (FFT) analysis to obtain the main oscillation frequency, and then the output feature vector is used as the input source of the scene trigger.

[0042] In actual application, the data analysis module collects the original steering wheel signal at a period of 50 ms, converts the USB HID data stream into a structured JSON data packet by using a Logitech SDK, and calculates the key operation features: the steering angular velocity is obtained by a differential algorithm (the difference between the current steering angle and the previous sampling divided by the time interval 0.05 seconds, in units of ° / s), and the jitter frequency is calculated by using fast Fourier transform (FFT) analysis on 20 angle sampling points (sampling rate 20 Hz) in the last 1 second to obtain the main oscillation frequency, and the output feature vector is in the format of { "angular_vel": 215.3, "jitter_freq": 2.4}, which is used as the input source of the scene trigger.

[0043] (2) Real-time monitoring and processing of steering angular velocity and jitter frequency in feature vector: When the steering angular velocity is detected to be greater than the preset steering angular velocity threshold, a pedestrian ghosting long tail scene with a preset travel speed is generated at a preset distance in front of the vehicle; when the jitter frequency is detected to be greater than the preset jitter frequency threshold, a vehicle skidding long tail scene is generated at the vehicle position; then the pedestrian ghosting long tail scene and the vehicle skidding long tail scene are determined as the first long tail training scene. The friction coefficient of the vehicle's tire model can be adjusted by the physics engine built into Unreal Engine. While reducing the friction coefficient of the tire model, an oil stain road decal effect with a preset radius is generated at the vehicle position to obtain the vehicle skidding long tail scene.

[0044] In one implementation, the dynamic scene generator receives feature vectors in real time and performs a dual conditional judgment: when the steering angular velocity > At that time, in front of the vehicle The generation rate at a distance of meters is The jitter frequency is measured in m / s for pedestrian ghosting events, and a single trigger is ensured by an anti-repetition flag; when the jitter frequency > At Hz, the friction coefficient of the Pacejka tire model is modified by calling the Unreal Engine Chaos physics engine, and a radius is generated at the vehicle's position. The effect is a rice-gray oil stain decal. Both of the above operations achieve millisecond-level response through the Unreal Engine Blueprint node UScenarioTriggerComponent::TickComponent().

[0045] In practical applications, the dynamic scene generator receives feature vectors in real time and performs dual condition judgments: when the steering angular velocity is >200° / s, a "Pedestrian_Running" pedestrian ghosting event with a speed of 3m / s is generated 5 meters in front of the vehicle (world coordinates FVector(500,0,0)), and the anti-repetition flag bEventActive is used to ensure single triggering; when the jitter frequency is >2.0Hz, the Unreal Engine Chaos physics engine APISetTireFrictionScale(0.25f) is called to modify the friction coefficient of the Pacejka tire model, and at the same time, an oil stain road decal effect with a radius of 2 meters is generated at the vehicle position.

[0046] (3) Obtain the obstacle distance and collision time between the vehicle and the obstacle in the first autonomous driving simulation training scene; send the obstacle distance and collision time to the risk value calculation model of the dual-mode controller for collision risk value calculation processing to obtain the risk value. Further, when the risk value is greater than the preset risk threshold and the duration is greater than the preset time threshold, it is determined that the autonomous driving simulation training result is vehicle autonomous driving failure, the simulation mode of the vehicle is switched from the autonomous driving simulation training mode to the manual driving simulation training mode, and the pulse vibration and light high alert are simultaneously performed.

[0047] In an embodiment, the dual-mode controller continuously monitors the collision risk value, which is dynamically calculated by the following formula:

[0048]

[0049] wherein TTC is the collision time, and Distance is the obstacle distance. When the risk value is greater than 0.95 and lasts for more than a second preset time threshold (e.g., 500 ms), it is determined that autonomous driving has failed. The system immediately activates the secondary takeover warning: output 15 Hz pulse vibration (100 ms period, 80% intensity) through the Logitech SDK function logiPlayPulseForce(0, 15, 100), and turn on the steering wheel LED red light at the same time. After the warning signal is issued, the time stamp of the driver's steering operation, the steering wheel angle and the vehicle speed are recorded in real time.

[0050] (4) After obtaining the steering wheel information of the vehicle in the manual driving simulation training mode, as well as the vehicle speed and the road friction coefficient, generate a sequence of trajectory points according to the steering wheel information, the vehicle speed and the road friction coefficient, and perform differential encoding processing after encapsulating the sequence of trajectory points to obtain structured takeover data.

[0051] In an embodiment, the emergency obstacle avoidance operation is performed after the driver feels the force feedback warning. The system captures the steering wheel turning angle (range -540°~540°) at a third preset time threshold (for example, 100 ms) sampling interval, generates a trajectory point sequence in combination with the vehicle speed and the road friction coefficient, and encapsulates the trajectory data in Protobuf format, with fields including timestamp (ms), steering angle (°), speed (m / s), and friction coefficient. The change amount of adjacent trajectory points is compressed through Delta Encoding difference encoding, and then stored through the Zstandard algorithm to achieve a 5:1 compression ratio, forming a structured takeover database. Delta Encoding is a data compression technique, and its core idea is that instead of storing original data, the difference between adjacent data is stored, that is, the first value is kept as is, and each subsequent value only stores the difference from the previous value. When decoding, the original data can be recovered by simply accumulating the difference. By using difference values instead of original values, continuous data can be efficiently stored to store the driver takeover trajectory.

[0052] (5) The structured takeover data is used to train a generative adversarial network, and a second long-tail training scene is generated according to the collected human behavior information, so as to expand the event library. The original autonomous driving simulation training scene, the first long-tail training scene, and the second long-tail training scene are combined as a new original autonomous driving simulation training scene, and a new round of vehicle autonomous driving simulation training mode is started.

[0053] In summary, the present application can overcome the problems of high cost of existing autonomous driving scene simulation professional hardware simulation system, rigid pre-programmed scene library, inability to dynamically respond to human input, and lack of real-time human-computer interaction interface. Based on consumer-grade hardware, the present application simulates autonomous driving long-tail scenes, and further generates new autonomous driving long-tail scenes through a closed-loop iteration of human behavior-scene variation-algorithm optimization, which are dynamically loaded as triggerable events in subsequent simulations, thereby significantly reducing simulation costs and improving simulation accuracy.

[0054] For the autonomous driving simulation method provided by the foregoing embodiments, the present application provides an autonomous driving simulation device, which is shown in Figure 3 The device includes the following parts:

[0055] The data analysis module 302 obtains the steering wheel information of the vehicle in the autonomous driving simulation training mode, and performs data analysis and processing on the steering wheel information to obtain a feature vector.

[0056] The first long-tail training scene generation module 304 generates a first long-tail training scene through double condition judgment processing of the feature vector by the dynamic scene generator, and combines the first long-tail training scene with the original autonomous driving simulation training scene to obtain a first autonomous driving simulation training scene.

[0057] The risk monitoring module 306 performs collision risk monitoring processing on the vehicle in the first autonomous driving simulation training scene through the double-mode controller, and switches the autonomous driving simulation training mode to the manual driving simulation training mode when the risk value is greater than a preset risk threshold.

[0058] The second long-tail training scene generation module 308 inputs the structured takeover data of the vehicle in the manual driving simulation training mode into the generative adversarial network model to obtain a second long-tail training scene, and combines the second long-tail training scene with the first autonomous driving simulation training scene to obtain a second autonomous driving simulation training scene, so as to re-perform autonomous driving simulation training on the vehicle based on the second autonomous driving simulation training scene.

[0059] The above-mentioned autonomous driving simulation device provided by the embodiments of the present application can significantly reduce the simulation cost and improve the simulation accuracy.

[0060] In an implementation form, when performing the step of performing data analysis processing on the steering wheel information to obtain the feature vector, the data analysis module 302 is further configured to: convert the steering wheel information into a structured JSON data packet by using a software development kit; obtain the steering angular velocity and the jitter frequency by performing difference calculation processing and fast Fourier transform processing on the structured JSON data packet, and determine the steering angular velocity and the jitter frequency as the feature vector.

[0061] In an implementation form, when performing the step of generating the first long-tail training scene through double condition judgment processing of the feature vector by the dynamic scene generator, the first long-tail training scene generation module 304 is further configured to: perform real-time monitoring processing on the steering angular velocity and the jitter frequency in the feature vector; when it is detected that the steering angular velocity is greater than a preset steering angular velocity threshold, generate a pedestrian ghost head long-tail scene with a preset advancing speed in front of the vehicle at a preset distance; when it is detected that the jitter frequency is greater than a preset jitter frequency threshold, generate a vehicle skidding long-tail scene at the vehicle position; and determine the pedestrian ghost head long-tail scene and the vehicle skidding long-tail scene as the first long-tail training scene.

[0062] In an embodiment, when the step of generating the vehicle skid long tail scene at the vehicle position is performed, the first long tail training scene generation module 304 is further configured to: adjust the tire model friction coefficient of the vehicle by the built-in physics engine of Unreal Engine, generate a predetermined radius of oil stain road sticker special effects at the vehicle position while reducing the tire model friction coefficient, and obtain the vehicle skid long tail scene.

[0063] In an embodiment, when the step of monitoring the collision risk of the vehicle in the first automatic driving simulation training scene by the dual-mode controller is performed, the risk monitoring module 306 is further configured to: obtain the obstacle distance and the collision time between the vehicle and the obstacle in the first automatic driving simulation training scene; and send the obstacle distance and the collision time to the risk value calculation model of the dual-mode controller for collision risk value calculation processing to obtain the risk value.

[0064] In an embodiment, when the step of switching the automatic driving simulation training mode to the manual driving simulation training mode when the risk value is greater than the predetermined risk threshold is performed, the risk monitoring module 306 is further configured to: when the risk value is greater than the predetermined risk threshold and the duration is greater than the predetermined time threshold, determine that the automatic driving simulation training result is vehicle automatic driving failure, switch the simulation mode of the vehicle from the automatic driving simulation training mode to the manual driving simulation training mode, and simultaneously perform the pulse vibration and light high alert.

[0065] In an embodiment, before the step of inputting the structured takeover data of the vehicle in the manual driving simulation training mode into the generative adversarial network model to obtain the second long tail training scene is performed, the second long tail training scene generation module 308 is further configured to: obtain the steering wheel information of the vehicle in the manual driving simulation training mode, and the vehicle speed and the road friction coefficient; generate a trajectory point sequence according to the steering wheel information, the vehicle speed and the road friction coefficient; and perform differential encoding processing on the encapsulated trajectory point sequence to obtain the structured takeover data.

[0066] The device provided by the embodiments of the present application has the same implementation principle and technical effects as the foregoing method embodiments. For brevity, the part not mentioned in the device embodiment part can be referred to the corresponding content in the foregoing method embodiments.

[0067] The server provided by the embodiments of the present application includes a processor and a storage device. The storage device stores a computer program, and the computer program performs the method of any one of the embodiments when executed by the processor.

[0068] Figure 4A structural schematic diagram of a server provided by the embodiment of the present application is shown in the figure, the server 100 comprises: a processor 40, a memory 41, a bus 42 and a communication interface 43, the processor 40, the communication interface 43 and the memory 41 are connected through the bus 42; the processor 40 is used for executing the executable module stored in the memory 41, for example, a computer program.

[0069] The memory 41 can contain a high-speed random access memory (RAM), and can also include a non-volatile memory, for example, at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 43 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0070] The bus 42 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 4 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0071] The memory 41 is used for storing a program, and the processor 40 executes the program after receiving an execution instruction. The method executed by the device defined by the flow process disclosed in any of the foregoing embodiments of the present application can be applied to the processor 40 or realized by the processor 40.

[0072] The processor 40 can be an integrated circuit chip with signal processing capability. In implementation, each step of the above method can be completed by integrated logic circuit of hardware in the processor 40 or by instructions in the form of software. The processor 40 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 41, and the processor 40 reads the information in the memory 41, and combines the hardware to complete the steps of the above method.

[0073] The computer program product of the readable storage medium provided by the embodiments of the present application comprises a computer readable storage medium storing program codes, and the program codes comprise instructions for executing the method described in the foregoing method embodiments. For specific implementation, reference can be made to the foregoing method embodiments, which will not be described here.

[0074] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0075] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit them. The protection scope of the present application is not limited to this. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily think of changes to the technical solutions recorded in the foregoing embodiments within the technical range disclosed by the present application, or make equivalent replacements to some technical features. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An autonomous driving simulation method, characterized in that, The method includes: The steering wheel information of the vehicle in the autonomous driving simulation training mode is obtained, and the steering wheel information is processed by data parsing to obtain a feature vector; The feature vector is processed by a dynamic scene generator to generate a first long-tail training scene. The first long-tail training scene is then combined with the original autonomous driving simulation training scene to obtain the first autonomous driving simulation training scene. The dual-mode controller monitors the collision risk of vehicles in the first autonomous driving simulation training scenario and switches the autonomous driving simulation training mode to the manual driving simulation training mode when the risk value is greater than the preset risk threshold. The structured takeover data of the vehicle in the manual driving simulation training mode is input into the generative adversarial network model to obtain the second long-tail training scenario. The second long-tail training scenario is then combined with the first autonomous driving simulation training scenario to obtain the second autonomous driving simulation training scenario. Based on the second autonomous driving simulation training scenario, the vehicle is retrained for autonomous driving simulation. The step of generating a first long-tail training scene by performing dual condition judgment processing on the feature vector through a dynamic scene generator includes: real-time monitoring of the steering angular velocity and jitter frequency in the feature vector; when the steering angular velocity is detected to be greater than a preset steering angular velocity threshold, generating a long-tail scene of a pedestrian peeking out at a preset speed at a preset distance in front of the vehicle; when the jitter frequency is detected to be greater than a preset jitter frequency threshold, generating a long-tail scene of vehicle skidding at the vehicle's position; and determining the long-tail scene of the pedestrian peeking out and the long-tail scene of vehicle skidding as the first long-tail training scene. The step of generating a long tail scene of vehicle skidding at the vehicle location includes: adjusting the friction coefficient of the vehicle's tire model using the physics engine built into Unreal Engine, and generating an oil stain road decal effect with a preset radius at the vehicle location while reducing the friction coefficient of the tire model, so as to obtain the long tail scene of vehicle skidding. The step of monitoring and processing the collision risk of the vehicle in the first autonomous driving simulation training scenario through the dual-mode controller includes: obtaining the obstacle distance and collision time between the vehicle and the obstacle in the first autonomous driving simulation training scenario; sending the obstacle distance and the collision time to the risk value calculation model of the dual-mode controller for collision risk value calculation processing to obtain the risk value.

2. The autonomous driving simulation method according to claim 1, characterized in that, The steps of parsing and processing the steering wheel information to obtain feature vectors include: The steering wheel information was converted into a structured JSON data packet using a software development kit; By performing differential calculation and fast Fourier transform on the structured JSON data packet, the steering angular velocity and jitter frequency are obtained, and the steering angular velocity and jitter frequency are determined as the feature vector.

3. The autonomous driving simulation method according to claim 1, characterized in that, When the risk value exceeds a preset risk threshold, the steps to switch from autonomous driving simulation training mode to manual driving simulation training mode include: When the risk value exceeds a preset risk threshold and the duration exceeds a preset time threshold, the autonomous driving simulation training result is determined to be a failure of vehicle autonomous driving. The vehicle's simulation mode is then switched from the autonomous driving simulation training mode to the manual driving simulation training mode, and pulse vibration and high-alert light warnings are issued simultaneously.

4. The autonomous driving simulation method according to claim 1, characterized in that, Before the step of inputting the structured takeover data of the vehicle in the manual driving simulation training mode into the generative adversarial network model to obtain the second long-tail training scenario, the following steps are included: Acquire steering wheel information, vehicle speed, and road friction coefficient of the vehicle in the manual driving simulation training mode; A sequence of trajectory points is generated based on the steering wheel information, the vehicle speed, and the road surface friction coefficient. The trajectory point sequence is encapsulated and then differentially encoded to obtain the structured takeover data.

5. An autonomous driving simulation device, characterized in that, The device includes: The data parsing module acquires the steering wheel information of the vehicle in the autonomous driving simulation training mode, and performs data parsing processing on the steering wheel information to obtain a feature vector; The first long-tail training scene generation module performs dual condition judgment processing on the feature vector through a dynamic scene generator to generate a first long-tail training scene, and combines the first long-tail training scene with the original autonomous driving simulation training scene to obtain the first autonomous driving simulation training scene. The risk monitoring module monitors the collision risk of vehicles in the first autonomous driving simulation training scenario through a dual-mode controller, and switches the autonomous driving simulation training mode to the manual driving simulation training mode when the risk value is greater than the preset risk threshold. The second long-tail training scenario generation module inputs the structured takeover data of the vehicle in the manual driving simulation training mode into the generative adversarial network model to obtain the second long-tail training scenario. The second long-tail training scenario is then combined with the first autonomous driving simulation training scenario to obtain the second autonomous driving simulation training scenario. Based on the second autonomous driving simulation training scenario, the vehicle is retrained for autonomous driving simulation. The step of generating a first long-tail training scene by performing dual condition judgment processing on the feature vector through a dynamic scene generator includes: real-time monitoring of the steering angular velocity and jitter frequency in the feature vector; when the steering angular velocity is detected to be greater than a preset steering angular velocity threshold, generating a long-tail scene of a pedestrian peeking out at a preset speed at a preset distance in front of the vehicle; when the jitter frequency is detected to be greater than a preset jitter frequency threshold, generating a long-tail scene of vehicle skidding at the vehicle's position; and determining the long-tail scene of the pedestrian peeking out and the long-tail scene of vehicle skidding as the first long-tail training scene. The step of generating a long tail scene of vehicle skidding at the vehicle location includes: adjusting the friction coefficient of the vehicle's tire model using the physics engine built into Unreal Engine, and generating an oil stain road decal effect with a preset radius at the vehicle location while reducing the friction coefficient of the tire model, so as to obtain the long tail scene of vehicle skidding. The step of monitoring and processing the collision risk of the vehicle in the first autonomous driving simulation training scenario through the dual-mode controller includes: obtaining the obstacle distance and collision time between the vehicle and the obstacle in the first autonomous driving simulation training scenario; sending the obstacle distance and the collision time to the risk value calculation model of the dual-mode controller for collision risk value calculation processing to obtain the risk value.

6. A server, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method described in any one of claims 1 to 4.

Images