Automatic driving control system, method and equipment

Abstract

The invention discloses an automatic driving control system, method and device, and relates to the field of intelligent driving, the system comprises a cloud server side and a vehicle-mounted terminal, the cloud server side is used for obtaining sample driving data marked with decision labels; obtaining an end-to-end automatic driving model; predicting sample driving data through an automatic driving model to obtain automatic driving decision data corresponding to the sample driving data; training an automatic driving model based on the difference between the automatic driving decision data and the decision label; sending the trained automatic driving model to the vehicle-mounted terminal; the vehicle-mounted terminal is used for receiving the automatic driving model; acquiring driving data; analyzing the driving data through an automatic driving model, and outputting automatic driving control data; and controlling automatic driving of the vehicle based on the automatic driving control data. Perception, decision making and other processes are integrated in the same model, and the end-to-end automatic driving control process is achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent driving, and particularly to a control system, method, and device for autonomous driving. Background Art

[0002] Autonomous driving refers to a technology that enables a vehicle to automatically drive without driver intervention by means of devices such as sensors, controllers, and actuators mounted on the vehicle. Its core purpose is to improve traffic safety, enhance travel efficiency, and improve the travel experience. End-to-end autonomous driving is an application of deep learning in the field of autonomous driving. Through a deep learning network, raw data such as visual images and radar signals collected by sensors are directly converted into control instructions for vehicle steering, acceleration, braking, etc., to achieve autonomous vehicle driving.

[0003] In related technologies, the mainstream solutions for end-to-end autonomous driving include the perception-decision model-based solution, which splits the autonomous driving control process into two stages: perception and decision-making. In the perception stage, data is collected through cameras and radars, processed using complex algorithms, and targets such as roads, vehicles, and pedestrians are identified, and key information such as position and speed is extracted to obtain the perception result. For example, the camera image is processed by a convolutional neural network to identify the contour and distance of the vehicle ahead. In the decision-making stage, based on the perception result and combined with factors such as traffic rules, a driving strategy is determined. For example, it is judged whether to maintain a vehicle distance, accelerate, or decelerate according to the distance and speed of the vehicle ahead.

[0004] However, the perception link and the decision-making link are relatively independent, which is very likely to cause time delay and loss of original details during information transmission, resulting in deviations and untimely decisions in complex scenarios, affecting the safety of autonomous driving. Summary of the Invention

[0005] The embodiments of the present application provide a control system, method, and device for autonomous driving, which can implement data perception, decision control, etc. through one model, improve the decision-making and control efficiency, and ensure the safety during the autonomous driving process. The technical solutions are as follows:

[0006] On the one hand, a control system for autonomous driving is provided. The system includes:

[0007] A cloud server, configured to obtain sample driving data labeled with decision labels, where the sample driving data is data related to autonomous driving control collected during the vehicle driving process; obtain an end-to-end autonomous driving model; predict the sample driving data through the autonomous driving model to obtain autonomous driving decision data corresponding to the sample driving data; train the autonomous driving model based on the difference between the autonomous driving decision data and the decision labels; and send the trained autonomous driving model to the in-vehicle terminal;

[0008] The in-vehicle terminal is used to receive the autonomous driving model; collect driving data, where the driving data is data related to autonomous driving control collected by the in-vehicle terminal during the driving process; analyze the driving data through the autonomous driving model and output autonomous driving control data; and control the vehicle for autonomous driving based on the autonomous driving control data.

[0009] In an optional embodiment, the cloud server is further configured to extract a data feature representation of the sample driving data through the autonomous driving model; analyze the data feature representation to obtain the autonomous driving decision data corresponding to the sample driving data.

[0010] In an optional embodiment, the cloud server is further configured to extract a data feature representation of the sample driving data through the encoder in the autonomous driving model; decode the data feature representation through the decoder in the autonomous driving model to obtain the autonomous driving decision data corresponding to the sample driving data.

[0011] In an optional embodiment, the driving data includes at least one of the following: at least one frame of vehicle-acquired image; radar sensing data; lidar sensing data; temperature sensing data; humidity sensing data; acceleration data; motion sensing data; steering wheel control data; power pedal control data.

[0012] In an optional embodiment, the in-vehicle terminal is further configured to collect first sample driving data, where the first sample driving data is data related to autonomous driving control collected by the in-vehicle terminal during the vehicle driving process in a first time period; and send the first sample driving data to the cloud server.

[0013] The cloud server is further configured to receive the first sample driving data; obtain environmental data, where the environmental data is used to describe the road conditions during the vehicle driving process; determine the decision label based on the environmental data, where the decision label is used to indicate decision data corresponding to the type of the road conditions; and obtain the sample driving data based on the decision label and the first sample driving data.

[0014] In an optional embodiment, the cloud server is further configured to analyze the first sample driving data to obtain first sample change data corresponding to the first sample driving data, where the first sample change data refers to data whose change amplitude within a preset time interval in the first sample driving data meets the preset requirements; and determine the decision label based on the first sample change data.

[0015] In an optional embodiment, the cloud server is further configured to control a three-dimensional vehicle model to travel in a virtual scene through a simulation of an autonomous driving program, where the simulation of the autonomous driving program is used to simulate the vehicle operation process, and the three-dimensional vehicle model is a model obtained by three-dimensional modeling of a vehicle; when the three-dimensional vehicle model is simulated to travel, model driving data of the three-dimensional vehicle model is collected, where the model driving data is data related to autonomous driving control; the model driving data is analyzed by the autonomous driving model to output model control data; the verification result of controlling the three-dimensional vehicle model to autonomously drive based on the model control data is displayed through the simulation of the autonomous driving program; and when the verification result meets the model training requirements, the trained autonomous driving model is sent to the in-vehicle terminal.

[0016] In an optional embodiment, the virtual scene includes a three-dimensional model of obstacle elements, and the obstacle elements include at least one of obstacle elements, moving objects, and roads with restricted driving.

[0017] The cloud server is further configured to send the trained autonomous driving model to the in-vehicle terminal when the verification result indicates that the three-dimensional vehicle model and the three-dimensional model of the obstacle elements meet the preset autonomous driving safety requirements.

[0018] On the other hand, a control method for autonomous driving is provided, and the method includes:

[0019] Obtain sample driving data labeled with decision labels, where the sample driving data is data related to autonomous driving control collected during the vehicle's travel.

[0020] Obtain an end-to-end autonomous driving model.

[0021] Predict the sample driving data through the autonomous driving model to obtain autonomous driving decision data corresponding to the sample driving data.

[0022] Train the autonomous driving model based on the difference between the autonomous driving decision data and the decision labels.

[0023] Send the trained autonomous driving model to the in-vehicle terminal.

[0024] Wherein, after receiving the autonomous driving model, the in-vehicle terminal collects driving data, where the driving data is data related to autonomous driving control collected during the in-vehicle terminal's travel; analyzes the driving data through the autonomous driving model to output autonomous driving control data; and controls the vehicle to autonomously drive based on the autonomous driving control data.

[0025] On the other hand, a control device for autonomous driving is provided, and the device includes:

[0026] An acquisition module, configured to acquire sample driving data labeled with decision labels, where the sample driving data is data related to autonomous driving control collected during the vehicle's driving process;

[0027] The acquisition module is further configured to acquire an end-to-end autonomous driving model;

[0028] A prediction module, configured to predict the sample driving data through the autonomous driving model to obtain autonomous driving decision data corresponding to the sample driving data;

[0029] A training module, configured to train the autonomous driving model based on the difference between the autonomous driving decision data and the decision labels;

[0030] A sending module, configured to send the trained autonomous driving model to an in-vehicle terminal;

[0031] Wherein, after receiving the autonomous driving model, the in-vehicle terminal acquires driving data, where the driving data is data related to autonomous driving control collected during the in-vehicle terminal's driving process; analyzes the driving data through the autonomous driving model, and outputs autonomous driving control data; and controls the vehicle to drive autonomously based on the autonomous driving control data.

[0032] On the other hand, a computer device is provided, where the computer device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the control method for autonomous driving as described in any one of the embodiments of the present application above.

[0033] On the other hand, a computer-readable storage medium is provided, where at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the control method for autonomous driving as described in any one of the embodiments of the present application above.

[0034] On the other hand, a computer program product or a computer program is provided, where the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the control method for autonomous driving as described in any one of the above embodiments.

[0035] The beneficial effects brought by the technical solutions provided in the embodiments of the present application at least include:

[0036] By collecting data during vehicle driving to train an autonomous driving model, integrating processes such as perception and decision-making in the same model, it avoids information transmission delays and lack of information details caused by independence between the perception stage and the decision-making stage, can output decision results in a timely manner, improve decision-making accuracy and the safety level of autonomous driving control, and achieve an end-to-end autonomous driving control process. As a whole, the autonomous driving model does not have the coordination problem between multiple independent modules, and when the vehicle-mounted terminal is offline, it can also obtain the vehicle autonomous driving strategy through the pre-deployed autonomous driving model to achieve safe driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following-described drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0038] Figure 1 is a schematic diagram of a control system for autonomous driving provided by an exemplary embodiment of the present application;

[0039] Figure 2 is a schematic diagram of the architecture of a control system for autonomous driving provided by an exemplary embodiment of the present application;

[0040] Figure 3 is a schematic diagram of the architecture of an end-to-end model (autonomous driving model) provided by an exemplary embodiment of the present application;

[0041] Figure 4 is a schematic diagram of the structure of an encoder provided by an exemplary embodiment of the present application;

[0042] Figure 5 is a schematic diagram of the structure of a decoder provided by an exemplary embodiment of the present application;

[0043] Figure 6 is a flowchart of a control method for autonomous driving provided by an exemplary embodiment of the present application;

[0044] Figure 7 is a block diagram of the structure of a control device for autonomous driving provided by an exemplary embodiment of the present application;

[0045] Figure 8 is a block diagram of the structure of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail in conjunction with the drawings.

[0047] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0048] It should be noted that the information and data involved in this application are all information and data authorized by users or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0049] Autopilot is a technology that enables a vehicle to drive automatically without driver intervention through devices such as sensors, controllers, and actuators mounted on the vehicle. Its core lies in fusing multi-sensor data to perceive environmental information and generating control commands by combining decision-making and planning algorithms, thereby improving traffic safety, traffic efficiency, and user experience.

[0050] End-to-end autopilot is an application of deep learning in the field of autopilot. It directly maps the raw data collected by sensors (such as camera images, millimeter-wave radar signals, etc.) to vehicle control signals (steering angle, acceleration, braking force, etc.) using a deep neural network. In related technologies, one of the mainstream implementation methods is the perception-decision model-based architecture. This architecture divides the autopilot control process into two independent stages: perception and decision-making. In the perception stage, algorithms such as convolutional neural networks are used to process the raw data to extract structured features such as object detection boxes and lane line coordinates. In the decision-making stage, based on the above feature data and combined with preset traffic rules, a driving strategy is generated.

[0051] However, this architecture has inherent defects: the perception module and the decision-making module are decoupled through intermediate features, resulting in time delay and semantic information loss during the information transmission process, making it difficult for the decision-making module to obtain the complete environmental context. Moreover, the serial processing architecture increases the system response time, leading to deviations and delays in decision-making in complex scenarios and affecting the overall performance of the system.

[0052] This application provides a control system and method for autopilot, which can implement the above end-to-end autopilot process only through a pre-trained autopilot model, integrating processes such as perception and decision-making in the same model, avoiding information transmission time delay and information detail loss caused by independence between the perception stage and the decision-making stage, ensuring the accuracy and timeliness of the decision result, and achieving safe driving.

[0053] Secondly, the control system for autonomous driving involved in the embodiments of the present application will be described. Schematically, please refer to Figure 1 , the system includes a cloud server 110 and an in-vehicle terminal 120, and there is a communication connection between the cloud server 110 and the in-vehicle terminal 120.

[0054] Among them, the cloud server 110 is responsible for training the autonomous driving model and sending the trained model to the in-vehicle terminal 120. The in-vehicle terminal 120 deploys the autonomous driving model. When the vehicle is driving autonomously, the autonomous driving model can provide control data for guiding the vehicle to drive.

[0055] Optionally, the cloud server 110 obtains sample driving data labeled with decision labels, and the sample driving data is data related to autonomous driving control collected during the vehicle's driving process.

[0056] The cloud server 110 obtains an end-to-end autonomous driving model. At this time, the end-to-end autonomous driving model is a model framework to be trained, and the sample driving data is used to train this model.

[0057] In some embodiments, the sample driving data is not labeled with decision labels. When the cloud server 110 receives the sample driving data, it first classifies and cleans the sample driving data. For example, the sample driving data is divided according to the data source (sensor collection, radar collection, camera collection, etc.) to facilitate the extraction of decision-related information. For example, if the sample driving data indicates that the current vehicle speed is V1 and the speed limit of the road where the vehicle is located is V2, and V2 < V1, then the information that the vehicle is speeding and needs to decelerate can be extracted from the sample driving data.

[0058] After obtaining the preprocessed sample driving data, the decision labels can be automatically labeled by the cloud server 110 or manually labeled to obtain the sample driving data labeled with decision labels.

[0059] In some embodiments, data mining and data augmentation processing can also be performed on the sample driving data through relevant algorithms or technologies to extract the information hidden in the sample driving data, enhance the diversity of the data, and make the sample driving data more comprehensive and accurate.

[0060] Exemplarily, the sample driving data is predicted by the autonomous driving model to obtain the autonomous driving decision data corresponding to the sample driving data. The autonomous driving decision data is used to reflect the decision result after the autonomous driving model analyzes the autonomous driving-related data collected during the vehicle's driving process and is used to indicate the control operations that need to be executed when the vehicle is driving autonomously.

[0061] The cloud server 110 trains the autonomous driving model based on the difference between the autonomous driving decision data and the decision label. The decision label is a pre-annotated reference decision result. That is, the decision label is used as a benchmark to evaluate whether the autonomous driving model has the ability to make accurate decisions.

[0062] When the difference between the autonomous driving decision data and the decision label does not meet the preset training requirements, loss training is performed on the autonomous driving model based on the backpropagation of the difference between the two. The above processes of prediction and difference training are repeated until the autonomous driving model can output autonomous driving decision data that meets the training requirements, and then the training is stopped.

[0063] The cloud server 110 verifies the autonomous driving model through a simulated autonomous driving program to confirm that the autonomous driving model can output accurate decision results to achieve safe autonomous driving. The simulated autonomous driving program can simulate the scenario of a vehicle driving autonomously on roads with various road conditions. By collecting the driving data of the simulated vehicle and inputting it into the autonomous driving model, the autonomous driving model analyzes the input data and outputs a decision result, and controls the simulated vehicle based on the decision result. If the driving situation of the simulated vehicle meets the preset safety requirements (for example, avoiding obstacles, obeying traffic rules, controlling speed, etc.), the verification passes, and the cloud server 110 sends the trained autonomous driving model to the in-vehicle terminal 120; otherwise, the verification fails, and the cloud server 110 needs to obtain more sample driving data to train the autonomous driving model until the verification passes.

[0064] The in-vehicle terminal 120 receives the autonomous driving model, deploys the autonomous driving model locally, and controls the vehicle based on the decision result given by the autonomous driving model during the autonomous driving process of the vehicle.

[0065] Optionally, the in-vehicle terminal 120 collects driving data, which is data related to autonomous driving control collected by the in-vehicle terminal 120 during the driving process. The driving data is analyzed by the autonomous driving model to output autonomous driving control data, and the vehicle is controlled to drive autonomously based on the autonomous driving control data. The autonomous driving control data indicates the way to control the vehicle to drive, and is used to instruct the in-vehicle terminal 120 to coordinate the operation modes of various components and functional systems in the vehicle.

[0066] Exemplarily, the in-vehicle terminal 120 controls the vehicle to start the engine and start driving. After collecting driving data through various sensors deployed on the vehicle, the driving data is input into the autonomous driving model for prediction and reasoning to obtain autonomous driving control data for guiding the vehicle to drive autonomously. The in-vehicle terminal 120 controls the vehicle to drive autonomously based on the autonomous driving control data. At this time, the driving state of the vehicle changes, and the corresponding driving data is updated.

[0067] During the vehicle driving process, the in-vehicle terminal 120 repeats the above process based on the real-time collected driving data to ensure the safety during the vehicle's autonomous driving process.

[0068] After the in-vehicle terminal 120 collects the driving data, it will store the driving data locally and synchronously upload it to the cloud server 110, so that the cloud server 110 can further improve or update the autonomous driving model based on the actual driving situation of the vehicle, enabling the autonomous driving model to cover more driving scenarios and output accurate autonomous driving control data.

[0069] That is, the system provides a data-driven end-to-end autonomous driving solution, which can integrate links such as perception, prediction, decision-making and planning, and vehicle control in the same autonomous driving model.

[0070] Exemplarily, as Figure 2 shown, Figure 2 is a schematic diagram of the architecture of the autonomous driving control system provided by an exemplary embodiment of the present application.

[0071] The autonomous driving control system 200 can integrate the perception module 210, the prediction module 220, the decision-making and planning module 230, and the control module 240 into a complete modular system.

[0072] For example, the following system functions are integrated in the perception module 210: lane line detection system, 3D object detection system, obstacle detection system, traffic light detection system, traffic sign detection system. The following system functions are integrated in the prediction module 220: vehicle behavior prediction system, pedestrian behavior prediction system, Agent (intelligent agent, an autonomous and intelligent entity) behavior prediction system. The following system functions are integrated in the decision-making and planning module 230: path planning system, behavior decision-making system, motion control system. The following system functions are integrated in the control module 240: vehicle steering control system, vehicle braking control system, vehicle power control system.

[0073] Specifically, the above-mentioned perception, decision-making, planning and other modules are integrated into a full-stack TransformX (data transformation layer) end-to-end model (autonomous driving module). By joint training, the characteristics of each module are retained, and the integration of perception and decision-making is achieved. The signals of the vehicle body sensors are input into the end-to-end model in real time by the autonomous driving control system, and the model outputs the control instructions of the in-vehicle terminal 120, which can correspondingly control the autonomous driving mode of the vehicle.

[0074] As Figure 3 shown, Figure 3 is a schematic diagram of the architecture of the end-to-end model (autonomous driving model).

[0075] In TransformX, X represents the number of Encoders 310 and Decoders 320 in the end-to-end model. The specific value of X is designed according to the actual project.

[0076] The Encoder 310 and the Decoder 320 are important components in deep learning. The Encoder 310 extracts and transforms the input data, encoding the input information into an encoded vector or a feature vector. The Decoder 320 receives the encoded vector output by the Encoder 310 and decodes it back to the required output form, making the output data meet the requirements of the autonomous driving control task.

[0077] The input data (Inputs) is input into the model, and the multi-layer Encoder 310 performs feature extraction one by one to obtain the feature representation. Then, the feature representation is input into the Decoder 320 for decoding one by one, and finally the output data (Outputs) is obtained as the output result of the model, which is used to guide the autonomous driving of the vehicle.

[0078] In Figure 3 each Encoder 310 and Decoder 320 has the same structure but different weight parameters. For each Encoder 310, it can be internally divided into two layers: the Attention Layer (used to capture and interact with information from different perspectives of the input sequence) and the FFNN (Feed Forward Neural Network, used to perform further feature transformation and non-linear mapping on the output of the Attention Layer), as well as two computational units, namely the Addition fusion operator (referred to as ADD for short) and the Norm operator (hereinafter referred to as the ADD and Norm operators).

[0079] Schematically, as Figure 4 shown, Figure 4 is a schematic diagram of the structure of an encoder.

[0080] The input data (Inputs) passes through the Attention Layer 410, the ADD and Norm operators 420, the Feed Forward Neural Network 430, and the ADD and Norm operators 420 in sequence, and finally outputs the result. Among them, the encoder includes two output paths (out1 and out2) for different processing tasks.

[0081] For each Decoder 320, it also contains an Attention Layer and a Feed Forward Neural Network (FFNN) internally, and an Encoder-Decoder Attention layer (used to establish the association between the encoder output and the decoder input) is inserted, enabling the Decoder 320 to focus on each input feature layer.

[0082] Schematic, such as Figure 5 shown Figure 5 is a schematic structural diagram of a decoder.

[0083] The initial input data (Output) sequentially passes through the attention layer 510, ADD and Norm operators 520, and the encoder-decoder attention layer 530. At this time, the output results of the two output paths of the encoder are input to the encoder-decoder attention layer 530, and the two sets of data continue to pass through the feed-forward neural network 540, ADD and Norm operators 520, and finally the output result (Outs) is obtained.

[0084] Among them, the core algorithm module of the end-to-end is the algorithm design of the attention layer and the feed-forward neural network in the encoder and decoder. The most core formula of the Attention mechanism is shown as Formula 1 below.

[0085] Formula 1:

[0086] Among them, Q is the Query (query vector), K is the Key (key vector), V is the Value (value vector), and d k is the dimension size of K; Q, K, and V are linearly transformed from the same input matrix In_X, as shown in Formula 2 below.

[0087] Formula 2:

[0088] Among them, W Q , W k and W v are three trainable parameter matrices. The input matrix In_X is multiplied by W Q , W k , W v respectively to generate Q, K, and V, which can be regarded as having undergone a linear transformation. The Attention layer does not directly use In_X, but uses these three matrices generated by matrix multiplication, enabling the model to learn different representations and enhancing the learning ability of the model.

[0089] The FFNN network layer unit generally consists of two operator units: the activation function of the first layer is ReLU (Rectified Linear Unit, rectified linear unit), as shown in Formula 3 below; the second layer is a linear activation function, as shown in Formula 4.

[0090] Formula 3: max(0, In_M * W1 + b1)

[0091] Formula Four: FFNN(In_M) = max(0, In_M * W1 + b1) * W2 + b2

[0092] Among them, In_M is the intermediate layer result after being processed by the Attention layer and serves as the input of the FFNN module. W1 is the weight matrix of the first layer, b1 is the bias vector of the first layer. The role of the ReLU function is to introduce non-linearity, enabling the model to learn more complex patterns. When the input is less than 0, the output is 0, and when the input is greater than 0, the output is equal to the input. W2 is the weight matrix of the second layer, b2 is the bias vector of the second layer. Through the calculations of these two layers, the features output by the Attention layer are further transformed and processed to obtain the output of the final FFNN layer for subsequent task processing.

[0093] It should be noted that the above cloud server 110 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0094] In some embodiments, the above server can also be implemented as a node in a blockchain system.

[0095] In summary, the automatic driving control system provided by this application trains an automatic driving model by collecting data during the vehicle driving process, integrates processes such as perception and decision-making in the same model, avoids situations such as information transmission delay and information detail loss caused by independence between the perception stage and the decision-making stage, can output decision results in a timely manner, improves decision accuracy and the safety level of automatic driving control, and realizes an end-to-end automatic driving control process. As a whole, the automatic driving model does not have the coordination problem between multiple independent modules, and when the in-vehicle terminal is offline, it can also obtain the vehicle automatic driving strategy through the pre-deployed automatic driving model to achieve safe driving.

[0096] Combined with the above noun introduction and application scenarios, the automatic driving control method provided by this application is described. This method can be executed by the cloud server or the in-vehicle terminal, or jointly executed by the cloud server and the in-vehicle terminal. In the embodiments of this application, it is described by taking the example that this method is jointly executed by the cloud server and the in-vehicle terminal, as Figure 6 shown, Figure 6 is a flowchart of the automatic driving control method provided by an exemplary embodiment of this application. This method includes the following steps.

[0097] Step 611, the cloud server obtains sample driving data labeled with decision tags.

[0098] The sample driving data is data related to autonomous driving control collected during the vehicle's driving process, and the sample driving data is used as training data for subsequent training of the autonomous driving model.

[0099] Optionally, the in-vehicle terminal collects first sample driving data, which is data related to autonomous driving control collected by the in-vehicle terminal during the vehicle's driving process in the first time period, and sends the first sample driving data to the cloud server.

[0100] Exemplarily, the first time period is a time period before the current moment, and the sample driving data is collected based on the driving process of the first vehicle in the first time period. Here, the in-vehicle terminal refers to the in-vehicle terminal of the first vehicle.

[0101] For example, the first time period is set to the past 24 hours to obtain sufficient first sample driving data.

[0102] The first vehicle is an autonomous driving test vehicle undergoing road tests, and the first vehicle is in the autonomous driving mode. Multiple in-vehicle cameras, radars (such as millimeter-wave radars, lidar), various sensors (such as temperature sensors, humidity sensors, acceleration sensors, motion sensors, etc.), and a data recording module in the vehicle electronic control system are installed on the first vehicle, jointly constituting the data acquisition system of the in-vehicle terminal, which is used to collect autonomous driving data generated during the driving process of the first vehicle as the first sample data.

[0103] Exemplarily, the first sample driving data includes at least one of the following data: (1) at least one frame of vehicle acquisition image; (2) radar sensing data; (3) lidar sensing data; (4) temperature sensing data; (5) humidity sensing data; (6) acceleration data; (7) motion sensing data; (8) steering wheel control data; (9) power pedal control data.

[0104] For example, the first sample driving data includes vehicle acquisition images collected by the in-vehicle camera to indicate whether there are obstacles in front of the vehicle, the type of the vehicle, the actions of pedestrians, etc.

[0105] For example, the first sample driving data includes radar sensing data collected by the in-vehicle radar. The in-vehicle radar emits electromagnetic waves and receives reflected waves to obtain information such as the distance, speed, and angle of the target object.

[0106] The cloud server receives the first sample driving data, and based on the first sample driving data, analyzes it by means of automatically generating labels and / or manually annotating labels to generate decision labels, and annotates the first sample driving data based on the decision labels to obtain the sample driving data.

[0107] Optionally, the process of obtaining the decision label is as follows.

[0108] Obtain environmental data, which is used to describe the road conditions during the vehicle driving process.

[0109] Among them, the environmental data includes the data collected by the on-vehicle terminals / data collection devices of other vehicles in the environment where the first vehicle is located (for example, cameras, sensors, etc. deployed on the road).

[0110] In some embodiments, the environmental data further includes the traffic data of the environment where the first vehicle is located obtained by the cloud server through networking.

[0111] Traffic data refers to the data related to traffic lights and traffic signs when the first vehicle is in a traffic section. For example, when the first vehicle is at a traffic light intersection, the traffic data includes the color change of the traffic light; when the first vehicle is on a speed-limited road, the traffic data includes the maximum speed and the minimum speed of the speed-limited road.

[0112] Determine the decision label based on the environmental data, and the decision label is used to indicate the decision data corresponding to the type of road conditions. Obtain the sample driving data based on the decision label and the first sample driving data.

[0113] Exemplarily, after obtaining the environmental data and the first sample driving data, perform preprocessing such as data cleaning and normalization on the data. Analyze the preprocessed data, judge the driving scenario of the first vehicle (for example, urban road scenario, whether there are obstacles in the front road, whether at a turning intersection), and extract key features such as the speed change rate and the lane line position. Formulate driving decision rules based on the driving scenario and features, and set priorities for various driving decision rules respectively. For example, the priority of emergency braking is higher than that of ordinary lane keeping rules. Then determine the corresponding driving actions based on the driving decision rules, and classify the driving actions to obtain different decision labels.

[0114] In some embodiments, the cloud server analyzes the first sample driving data to obtain the first sample change data corresponding to the first sample driving data. The first sample change data refers to the data in the first sample driving data whose change amplitude within a preset time interval meets the preset requirements. Determine the decision label based on the first sample change data.

[0115] Exemplarily, for each type of data in the first sample driving data (such as vehicle speed data, steering wheel angle data, etc.), a corresponding change amplitude threshold is set as the basis for obtaining the first sample change data.

[0116] For example, the change amplitude threshold of the vehicle speed data is 25 km / h, and the change amplitude threshold of the steering wheel angle data is 70°. When there is any data in the first sample driving data whose change amplitude within the preset time interval exceeds the preset change amplitude threshold, the first sample change data is obtained.

[0117] Among them, the division method of the preset time interval is determined based on the maximum value of the change curve of each type of data in the first sample driving data. For example, the vehicle speed reaches the maximum value at the first moment. Based on the first moment, the vehicle speeds at the second moment and the third moment at a preset time interval from the first moment are obtained. The second moment is the moment before the first moment, and the third moment is the moment after the first moment.

[0118] If the difference between the vehicle speeds at at least one moment between the second moment and the third moment and the maximum value of the vehicle speed reaches the preset change amplitude threshold, the vehicle speed data collected during the time period between the second moment and the third moment is determined as the first sample change data.

[0119] Exemplarily, the first sample driving data also includes manual intervention data. The manual intervention data refers to the vehicle driving data generated by the driver's manual operation during the automatic driving process. The manual intervention data can reflect the driver's decision-making results. Taking the manual intervention data as the first sample change data can obtain a more accurate and comprehensive decision label.

[0120] Based on the analysis of the first sample change data, the corresponding vehicle control operation type is obtained from the change situation of the first sample change data, and a decision label is obtained. For example, in the first sample change data, the vehicle speed drops from the first speed of 60 km / h to the second speed of 20 km / h, and the corresponding decision label is: the vehicle brakes and decelerates. For example, in the first sample change data, the change in the vehicle steering wheel angle is 45° in the clockwise direction, and the corresponding decision label is: the vehicle turns right.

[0121] Step 612, the cloud server obtains an end-to-end automatic driving model.

[0122] The end-to-end automatic driving model is an integrated architecture model that directly converts sensor data into vehicle control commands. The automatic driving model contains multiple groups of encoders and decoders, which can perform hierarchical feature processing on the input data and output decision results through two branches: longitudinal control (such as control of vehicle acceleration) and lateral control (such as control of steering wheel angle).

[0123] Step 613: The cloud server predicts the sample driving data through the autonomous driving model to obtain the autonomous driving decision data corresponding to the sample driving data.

[0124] Optionally, the cloud server extracts the data feature representation of the sample driving data through the autonomous driving model, analyzes the data feature representation, and obtains the autonomous driving decision data corresponding to the sample driving data.

[0125] Exemplarily, the cloud server extracts the data feature representation of the sample driving data through the encoder in the autonomous driving model, and decodes the data feature representation through the decoder in the autonomous driving model to obtain the autonomous driving decision data corresponding to the sample driving data.

[0126] The autonomous driving model realizes the full-process decision control through the encoder-decoder architecture: the encoder first performs spatio-temporal calibration and feature fusion on the sample driving data, extracts environmental semantic features (such as lane lines, obstacles) through the attention layer and the feed-forward neural network, and captures long-distance dependencies through the Transformer to form an abstract feature vector containing spatial position and dynamic trend; based on these features, the decoder decodes the feature representation output by the encoder through the attention layer, the feed-forward neural network, and the encoder-decoder attention layer, and adopts a dual-branch structure to generate longitudinal (acceleration / deceleration) and lateral (steering) control commands respectively. The longitudinal branch predicts the acceleration through time series analysis, and the lateral branch plans the steering angle using the attention mechanism. The architecture of this autonomous driving model combines multi-modal feature fusion with the autonomous learning ability of the deep neural network, significantly improving the decision accuracy and robustness in complex scenarios.

[0127] Step 614: The cloud server trains the autonomous driving model based on the difference between the autonomous driving decision data and the decision label.

[0128] Exemplarily, the data formats of the automatic decision data and the decision label are the same, and both are used to indicate the way of controlling the vehicle's autonomous driving by the in-vehicle terminal. Using the pre-trained feature extraction model, the feature extraction is respectively performed on the automatic decision data and the decision label to obtain the first feature representation corresponding to the automatic decision data (expressed as an n-dimensional feature vector, where n is a positive integer) and the second feature representation corresponding to the decision label (with the same dimension as the first feature representation).

[0129] By calculating the similarity between the first feature representation and the second feature representation, the difference between the autonomous driving decision data and the decision label is determined.

[0130] For example, calculate the cosine similarity between the first feature representation and the second feature representation. The value of the cosine similarity is inversely correlated with the degree of difference. That is, the higher the calculated similarity value, the smaller the difference between the autonomous driving decision data and the decision label.

[0131] Perform loss training on the autonomous driving model based on the difference between the autonomous driving decision data and the decision label until the autonomous driving model can output autonomous driving decision data whose difference from the decision label meets the requirements of loss training.

[0132] For example, obtain the preset loss function F(x) = ax + b, where x refers to the cosine similarity between the first feature representation and the second feature representation, and a and b are preset weights, which are constants. When the value calculated by F(x) based on the cosine similarity is lower than the preset first loss value, stop training the autonomous driving model.

[0133] Optionally, the obtained autonomous driving model after training can be initially used for vehicle autonomous driving decisions. Before deploying the autonomous driving model on the in-vehicle terminal, the three-dimensional vehicle operation process can be simulated through a simulation program, and the autonomous driving model is used to output driving decision data to guide the three-dimensional vehicle autonomous driving.

[0134] Exemplarily, the cloud server controls the three-dimensional vehicle model to drive in the virtual scene through the simulated autonomous driving program. The simulated autonomous driving program is used to simulate the vehicle operation process, and the three-dimensional vehicle model is a model obtained after three-dimensional modeling of the vehicle.

[0135] When the three-dimensional vehicle model is simulated to drive, collect the model driving data of the three-dimensional vehicle model. The model driving data is data related to autonomous driving control.

[0136] Among them, the data format of the model driving data is the same as that of the sample driving data, and is used to describe the driving situation of the three-dimensional vehicle model in the virtual scene.

[0137] Analyze the model driving data through the autonomous driving model and output the model control data. The data format of the model control data is the same as that of the driving decision data.

[0138] Among them, display the verification result of controlling the three-dimensional vehicle model to drive autonomously based on the model control data through the simulated autonomous driving program. When the verification result meets the model training requirements, send the trained autonomous driving model to the in-vehicle terminal.

[0139] The process of the simulated autonomous driving program simulating the three-dimensional vehicle model to drive autonomously will be displayed on the terminal screen in the form of an animation. The preset model training requirements refer to that the verification result indicates that the three-dimensional vehicle model safely avoids obstacles in the virtual scene and abides by traffic rules.

[0140] Exemplarily, the virtual scene contains a three-dimensional model with obstacle elements, and the obstacle elements include at least one of obstacle elements, moving objects, and roads with restricted driving.

[0141] An obstacle element refers to a fixed or moving physical entity that directly obstructs the passage path of a three-dimensional vehicle model in a virtual scene. For example, static obstacles include construction tools (barriers), and dynamic obstacles include other vehicle models present in the same virtual scene.

[0142] A moving entity refers to an entity with the ability to perform autonomous movement behaviors. For example, pedestrians walking or riding non-motor vehicles, or animals. Moving entities affect the vehicle road conditions through dynamic behaviors.

[0143] A road with restricted driving is a section where the traffic capacity needs to be maintained within a specified speed range due to structural or environmental conditions. For example, on roads near schools, the vehicle speed limit must not exceed a preset speed threshold to avoid safety hazards caused by excessive driving speed.

[0144] When the verification result indicates that the three-dimensional vehicle model and the three-dimensional model of the obstacle element meet the preset requirements for autonomous driving safety, the cloud server sends the trained autonomous driving model to the in-vehicle terminal.

[0145] Exemplarily, the verification result shows that the three-dimensional vehicle model and the three-dimensional model of the obstacle element exhibit collaborative capabilities that meet the preset safety requirements in a dynamic interaction scenario. For example, in a collision avoidance test, the autonomous driving model can timely identify and respond to obstacles based on the model driving data, and ensure a safe distance between the three-dimensional vehicle model and the obstacle through emergency braking and steering operations; in path planning verification, in a restricted road environment (such as a construction section or a narrow bridge), the autonomous driving model outputs decision data to control the three-dimensional vehicle model to avoid conflicts by dynamically adjusting the driving trajectory and speed; in extreme environment simulations (such as waterlogged or icy roads), it effectively maintains the stability of the three-dimensional vehicle model and completes hazard avoidance operations.

[0146] Step 615, the cloud server side sends the trained autonomous driving model to the in-vehicle terminal.

[0147] There is a communication connection between the cloud server side and the in-vehicle terminal, and the installation / deployment file of the autonomous driving model is sent to the in-vehicle terminal through the communication connection.

[0148] Step 621, the in-vehicle terminal receives the autonomous driving model.

[0149] After receiving the autonomous driving model, the in-vehicle terminal deploys the autonomous driving model locally based on the guidance of the deployment file, enabling the in-vehicle terminal to make real-time decisions for vehicle autonomous driving when connected to the network, and the autonomous driving model can also output corresponding driving decision data in the offline situation.

[0150] Step 622, the in-vehicle terminal collects driving data.

[0151] Driving data is data related to autonomous driving control collected by an in-vehicle terminal during driving. Driving data is the input data for an autonomous driving model and the basis for the autonomous driving model to judge the current environment and vehicle state of the vehicle, so as to make driving decisions and control the vehicle to drive safely and autonomously.

[0152] Optionally, the driving data includes at least one of the following:

[0153] 1. At least one frame of vehicle-captured images: Images captured by on-vehicle cameras equipped on the vehicle. For example, the images captured by the front camera of the vehicle show the information of the road ahead, other vehicles, pedestrians, etc., which are used to indicate whether there are obstacles ahead of the vehicle, the types of vehicles, the actions of pedestrians, etc., and help the system plan the driving route.

[0154] 2. Radar sensing data: By emitting electromagnetic waves and receiving reflected waves through a radar, information such as the distance, speed, and angle of a target object can be obtained. For example, a millimeter-wave radar can monitor the distance and relative speed of other vehicles or objects around the vehicle in real time, reflecting the speed change of other vehicles.

[0155] 3. LiDAR sensing data: Using laser beams to scan the surrounding environment can construct a high-precision three-dimensional environmental map. In complex road conditions, LiDAR can accurately sense the positions and shapes of surrounding buildings, trees, road boundaries, etc., providing more accurate environmental information for the autonomous driving model.

[0156] 4. Temperature sensing data: Temperature information of relevant parts of the vehicle or the surrounding environment collected by a temperature sensor. For example, too high a battery temperature may affect the vehicle performance and safety.

[0157] 5. Humidity sensing data: Humidity information of the surrounding environment obtained by a humidity sensor. In weather with high humidity, the road surface may be slippery, affecting the braking distance of the vehicle.

[0158] 6. Acceleration data: The acceleration conditions of the vehicle during acceleration, deceleration, or turning in the driving process measured by an acceleration sensor.

[0159] 7. Motion sensing data: Covers information such as the driving speed, direction, and attitude of the vehicle. These data can enable the autonomous driving system to clearly understand the current motion state of the vehicle, such as whether the vehicle is driving straight, the angle and speed of turning, etc.

[0160] 8. Steering wheel control data: Records the operation information of the driver on the steering wheel, including the rotation angle, rotation speed, etc. In the autonomous driving mode, it can also be used to monitor the control of the steering wheel by the vehicle's autonomous driving system and the operation records when the driver takes over the driving when necessary.

[0161] 9. Power pedal control data: It contains control information of the accelerator pedal and the brake pedal, such as the depression depth and depression speed of the pedal. For example, corresponding operation information is obtained when the driver intervenes in the power pedal.

[0162] Step 623, the in-vehicle terminal analyzes the driving data through the autonomous driving model and outputs autonomous driving control data.

[0163] The format of the driving control data is the same as that of the autonomous driving decision data. The autonomous driving model analyzes the driving data through an encoder to extract the data feature representation of the driving data; and decodes the data feature representation of the driving data through a decoder to obtain the autonomous driving control data corresponding to the driving data. The autonomous driving control data is used to guide the in-vehicle terminal to control the vehicle for autonomous driving.

[0164] Step 624, the in-vehicle terminal controls the vehicle for autonomous driving based on the autonomous driving control data.

[0165] Exemplarily, when the vehicle is driving on a construction section, the driving data includes the straight-line distance between the vehicle and the construction site. The autonomous driving control data obtained by the autonomous driving model based on the driving data is a detour trajectory data, indicating that when the distance between the vehicle and the construction site is about 5 meters, the in-vehicle terminal controls the vehicle to automatically drive along the driving path indicated by the detour trajectory data, bypass the construction site, and achieve a safe autonomous driving process.

[0166] In summary, the method provided in this application trains the autonomous driving model by collecting data during the vehicle driving process, integrates processes such as perception and decision-making in the same model, avoids situations such as information transmission delay and information detail loss caused by independence between the perception stage and the decision-making stage, can output decision results in a timely manner, improves the decision accuracy and the safety level of autonomous driving control, and realizes an end-to-end autonomous driving control process. As a whole, the autonomous driving model does not have the coordination problem between multiple independent modules. When the in-vehicle terminal is offline, it can also obtain the vehicle autonomous driving strategy through the pre-deployed autonomous driving model to achieve safe driving.

[0167] Figure 7 It is the structural block diagram of the control device for autonomous driving provided by an exemplary embodiment of this application. As Figure 7 shown, the device includes the following parts.

[0168] An acquisition module 710, configured to acquire sample driving data labeled with decision labels, where the sample driving data is data related to autonomous driving control collected during the vehicle driving process;

[0169] The acquisition module 710 is further configured to acquire an end-to-end autonomous driving model;

[0170] A prediction module 720, configured to predict the sample driving data through the autonomous driving model to obtain autonomous driving decision data corresponding to the sample driving data;

[0171] A training module 730, configured to train the autonomous driving model based on the difference between the autonomous driving decision data and the decision label;

[0172] A sending module 740, configured to send the trained autonomous driving model to an in-vehicle terminal;

[0173] Wherein, after receiving the autonomous driving model, the in-vehicle terminal collects driving data, and the driving data is data related to autonomous driving control collected by the in-vehicle terminal during driving; analyzes the driving data through the autonomous driving model, and outputs autonomous driving control data; and controls the vehicle to drive autonomously based on the autonomous driving control data.

[0174] In summary, the control device for autonomous driving provided in this application trains an autonomous driving model by collecting data during the driving process of the vehicle, integrates processes such as perception and decision-making in the same model, avoids situations such as information transmission delay and lack of information details caused by independence between the perception stage and the decision-making stage, can output decision results in a timely manner, improves decision accuracy and the safety level of autonomous driving control, and realizes an end-to-end autonomous driving control process. As a whole, the autonomous driving model does not have the coordination problem between multiple independent modules, and when the in-vehicle terminal is offline, it can also obtain the vehicle autonomous driving strategy through the pre-deployed autonomous driving model to achieve safe driving.

[0175] It should be noted that: the control device for autonomous driving provided in the above embodiment is only illustrated by the division of the above function modules. In actual application, the above functions can be allocated to different function modules according to needs, that is, the internal structure of the device is divided into different function modules to complete all or part of the functions described above. In addition, the control device for autonomous driving provided in the above embodiment and the embodiment of the control method for autonomous driving belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0176] Figure 8The block diagram of a computer device 800 provided by an exemplary embodiment of the present application is shown. The computer device 800 may be: a smart phone, a tablet computer, a Moving Picture Experts Group Audio Layer III (MP3) player, a Moving Picture Experts Group Audio Layer IV (MP4) player, a laptop computer, or a desktop computer. The computer device 800 may also be referred to by other names such as a user device, a portable terminal, a laptop terminal, a desktop terminal, etc.

[0177] Generally, the computer device 800 includes: a processor 801 and a memory 802.

[0178] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the Central Processing Unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a Graphics Processing Unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an Artificial Intelligence (AI) processor, and the AI processor is used to process computational operations related to machine learning.

[0179] The memory 802 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 802 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 801 to implement the control method for autonomous driving provided by the method embodiments of the present application.

[0180] In some embodiments, the computer device 800 further includes some other components 803, and the types and quantities of the other components 803 can be selected based on the functional requirements of the computer device 800. Those skilled in the art can understand that Figure 8 the structure shown in does not constitute a limitation on the computer device 800, and it may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component layout.

[0181] Optionally, the computer-readable storage medium may include: Read Only Memory (ROM), Random Access Memory (RAM), Solid State Drives (SSD), or optical discs, etc. Among them, the random access memory may include Resistance Random Access Memory (ReRAM) and Dynamic Random Access Memory (DRAM). The serial numbers of the embodiments of the present application are only for description and do not represent the superiority or inferiority of the embodiments.

[0182] The embodiments of the present application also provide a computer device, which includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the control method for autonomous driving as described in any one of the above embodiments of the present application.

[0183] The embodiments of the present application also provide a computer-readable storage medium, in which at least one instruction, at least one program, a code set, or an instruction set is stored, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the control method for autonomous driving as described in any one of the above embodiments of the present application.

[0184] The embodiments of the present application also provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the control method for autonomous driving as described in any one of the above embodiments.

[0185] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, an optical disc, etc.

[0186] The above are only alternative embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. An automatic driving control system, characterized in that: The system comprises: A cloud server is used to obtain sample driving data marked with decision labels, wherein the sample driving data is data related to autonomous driving control collected by the vehicle during driving; obtain an end-to-end autonomous driving model; predict the sample driving data through the autonomous driving model to obtain autonomous driving decision data corresponding to the sample driving data; train the autonomous driving model based on the difference between the autonomous driving decision data and the decision label; and send the trained autonomous driving model to the vehicle terminal; The vehicle-mounted terminal is used to receive the autonomous driving model; collect driving data, where the driving data is data related to autonomous driving control collected by the vehicle-mounted terminal during driving; analyze the driving data through the autonomous driving model and output autonomous driving control data; and control the vehicle's autonomous driving based on the autonomous driving control data.

2. The system according to claim 1, characterized in that The cloud server is also used to extract the data feature representation of the sample driving data through the autonomous driving model; analyze the data feature representation to obtain the autonomous driving decision data corresponding to the sample driving data.

3. The system according to claim 2, characterized in that The cloud server is also used to extract the data feature representation of the sample driving data through the encoder in the autonomous driving model; decode the data feature representation through the decoder in the autonomous driving model to obtain the autonomous driving decision data corresponding to the sample driving data.

4. The system according to any one of claims 1 to 3, characterized in that: The driving data includes at least one of the following: At least one frame of vehicle acquisition image; Radar sensor data; LiDAR sensor data; Temperature sensor data; Humidity sensor data; Acceleration data; Motion sensor data; Steering wheel control data; Power pedal control data.

5. The system according to any one of claims 1 to 3, characterized in that: The vehicle-mounted terminal is further used to collect first sample driving data, where the first sample driving data is data related to automatic driving control collected by the vehicle-mounted terminal during vehicle driving in a first time period; and send the first sample driving data to a cloud server; The cloud server is further configured to receive the first sample driving data; obtain environmental data, the environmental data being used to describe the road conditions during vehicle driving; and determine the decision tag based on the environmental data, the decision tag being used to indicate the decision data corresponding to the type of the road condition; The sample driving data is obtained based on the decision label and the first sample driving data.

6. The system according to claim 5, characterized in that The cloud server is also used to analyze the first sample driving data to obtain first sample change data corresponding to the first sample driving data, where the first sample change data refers to data in the first sample driving data whose change amplitude within a preset time interval meets preset requirements; and determine the decision label based on the first sample change data.

7. The system according to any one of claims 1 to 3, characterized in that: The cloud server is also used to control the three-dimensional vehicle model to travel in a virtual scene through a simulated automatic driving program, the simulated automatic driving program is used to simulate the vehicle operation process, and the three-dimensional vehicle model is a model obtained after three-dimensional modeling of the vehicle; when the three-dimensional vehicle model simulates driving, the model driving data of the three-dimensional vehicle model is collected, and the model driving data is data related to automatic driving control; the model driving data is analyzed by the automatic driving model, and the model control data is output; the verification result of controlling the automatic driving of the three-dimensional vehicle model based on the model control data is displayed through the simulated automatic driving program; when the verification result meets the model training requirements, the trained automatic driving model is sent to the vehicle-mounted terminal.

8. The system according to claim 7, characterized in that The virtual scene includes a three-dimensional model of an obstacle element, wherein the obstacle element includes at least one of an obstacle element, a moving subject, and a road with restricted driving; The cloud server is also used to send the trained automatic driving model to the vehicle terminal when the verification result indicates that the three-dimensional vehicle model and the three-dimensional model of the obstacle element meet the preset automatic driving safety requirements.

9. A control method for automatic driving, characterized in that: Executed by a cloud server, the method includes: Acquire sample driving data marked with decision labels, where the sample driving data is data related to automatic driving control collected during driving of the vehicle; Obtain an end-to-end autonomous driving model; Predicting the sample driving data by using the autonomous driving model to obtain autonomous driving decision data corresponding to the sample driving data; training the autonomous driving model based on a difference between the autonomous driving decision data and the decision label; Sending the trained autonomous driving model to the vehicle terminal; Among them, after the vehicle-mounted terminal receives the automatic driving model, it collects driving data, and the driving data is data related to automatic driving control collected by the vehicle-mounted terminal during driving; the driving data is analyzed by the automatic driving model, and the automatic driving control data is output; and the vehicle automatic driving is controlled based on the automatic driving control data.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the automatic driving control method as described in claim 9.

Images