Model training method, vehicle control method and related device

Abstract

A model training method, a vehicle control method and a related device can be applied to the field of artificial intelligence, and the method comprises the following steps: obtaining road condition information of a target vehicle; according to the road condition information, target information is obtained through a first neural network model, and the target information is driving intention prediction of the target vehicle, prediction of a driving path or prediction of an interaction behavior of the target vehicle and the environment; and according to the target information, updating the first neural network model through an expert system or a result obtained by processing the road condition information and the label corresponding to the road condition information by the expert system. According to the method, the gradient for updating the AI model is guided and determined by utilizing the output of the expert system, which is equivalent to closed-loop training of the AI model, so that the precision of the AI model is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a model training method, a vehicle control method and related devices. Background Art

[0002] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0003] Key technologies in the field of autonomous driving include perception, decision-making, planning, and control. Among them, the vehicle's autonomous decision-making ability plays a key role in the intelligence and safety of the entire autonomous driving system. Compared with expert systems, data-driven AI methods have the ability to make human-like decisions in complex scenarios and are the development trend of autonomous driving technology. However, data-driven AI methods are not explainable and need to be used in conjunction with regulatory expert systems for a considerable period of time now and in the future.

[0004] The current mainstream data-driven methods are mostly open-loop learning methods (such as imitation learning). The idea of ​​this method is relatively direct: learn an end-to-end mapping model from observations to posterior trajectory outputs from massive human driving data. However, this method makes it difficult to consider the actual closed-loop effect of the model's output when learning the model, resulting in a decrease in the actual application effect. Summary of the invention

[0005] The present application provides a model training method, a vehicle control method and related devices, which can perform closed-loop training on the AI ​​model in the vehicle's regulation and control system, thereby improving the accuracy of the AI ​​model.

[0006] In a first aspect, the present application provides a model training method, specifically, obtaining road condition information of a target vehicle; obtaining target information through a first neural network model based on the road condition information, the target information being a prediction of the target vehicle's driving intention, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment; and updating the first neural network model based on the target information through an expert system or the result obtained by processing the road condition information and labels corresponding to the road condition information by the expert system.

[0007] Since driving information is often inaccurate (for example, it is not optimal in terms of safety or comfort), an expert module is needed to provide a safety net and correct the driving information. In existing implementations, during the training of AI models, driving information is often used as a label to update the AI ​​model, and the output of the subsequent expert system is not introduced. Therefore, the labels used to update the first neural network model have a lower confidence level in essence, resulting in lower processing accuracy of the trained model.

[0008] Among them, the label corresponding to the road condition information can be data in the actual driving process (which can be a driving process in a simulation environment or a driving process in an actual physical environment) (for example, it can be a prediction of driving intention, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment), and this data can be used as the true value corresponding to the target information in the training sample.

[0009] In an embodiment of the present application, the expert system or the output of the expert system is utilized to participate in the training of the first neural network model, which is equivalent to closed-loop training of the first neural network model, so that the first neural network model has the characteristics of a regulatory expert system, thereby improving the accuracy of the first neural network model.

[0010] Among them, the expert system can be implemented based on rules or based on neural networks.

[0011] In one possible implementation, when determining the loss for updating the first neural network model, the loss is determined based on the label corresponding to the target information and the road condition information, and the loss is adjusted based on the feasibility of the label. The feasibility of the label can be obtained based on the processing results of the expert system on the road condition information and the label corresponding to the road condition information.

[0012] The feasibility of the label corresponding to the road condition information can be considered as an evaluation of the label. Therefore, the feasibility can be introduced into the training process of the first neural network model. Specifically, the loss can be determined based on the target information and the corresponding label (since the label may not be accurate, the loss may also be inaccurate. If the AI ​​model is updated based on inaccurate losses, the training accuracy of the model will be poor). Therefore, in the embodiment of the present application, the loss is adjusted by the feasibility, and based on the adjusted loss (the adjusted loss is equivalent to introducing the output of the expert system).

[0013] In a possible implementation, the feasibility indicates whether the label corresponding to the road condition information is feasible or not.

[0014] In a possible implementation, the feasibility indicates whether the safety or comfort of the vehicle itself meets the requirements when the target vehicle performs corresponding driving control according to the label. For example, the feasibility can be represented by 1 or 0, where 1 indicates feasible and 0 indicates infeasible. For example, the feasibility indicates whether the label is feasible or infeasible. When the feasibility indicates that the label is feasible, it can be determined that the AI ​​model needs to be updated, and when the feasibility indicates that the label is infeasible, it can be determined that the AI ​​model does not need to be updated, or the absolute size of the update gradient can be reduced accordingly.

[0015] In a possible implementation, the feasibility is a feasibility score; the loss is adjusted by the feasibility, including: using the feasibility score as a weight to adjust the loss. The feasibility score of the label can be determined according to the processing result of the expert system, that is, the degree of satisfaction of the vehicle's own safety or comfort when the target vehicle performs corresponding driving control according to the label. For example, the feasibility can indicate the feasibility score of the label, and the feasibility score can be used as a weight to adjust the loss. When the feasibility score is large, it can be considered that the accuracy of the label is high, and thus the update gradient obtained based on the feasibility score is also large. When the feasibility score is small, it can be considered that the accuracy of the label is low, and thus the update gradient obtained based on the feasibility score is also small.

[0016] In a possible implementation, the feasibility is related to the following information: the safety or comfort of the target vehicle when the target vehicle is controlled to travel according to the tag.

[0017] In one possible implementation, the expert system is a second neural network model, that is, when the first neural network model is trained, the expert system is parameterized (the parameterized expert system is the second neural network model); the control instructions of the target vehicle can be determined through the second neural network model based on the road condition information and the target information; the target vehicle is controlled to interact with the environment according to the control instructions, the interaction results are determined, and the first neural network model is updated according to the interaction results.

[0018] For example, a reward value can be determined based on the interaction result; based on the reward value, an update gradient of the first neural network model is determined, and the update gradient is used to update the first neural network model.

[0019] In the case where the expert system is a non-neural network, the reward value corresponding to the output of the expert system cannot be directly used to determine the update gradient of the first neural network model (the reason is that the expert system is a non-neural network and cannot perform gradient feedback, and the expert system based on the strategy algorithm often has a large number of state machine transitions, random sampling, optimization and other modules. If the characteristics are learned through random exploration, it is very slow, which greatly increases the difficulty of online learning interaction). If the expert system is parameterized during the training process, the output of the expert system can be used to determine the reward value, and when the gradient is fed back, it can be passed to the first neural network model. On the one hand, the update gradient corresponding to the first neural network model can be obtained, and the update gradient is obtained based on the output of the expert system, which is equivalent to introducing the output obtained by the expert system into the training process of the AI ​​model, thereby improving the processing accuracy of the trained model.

[0020] In a second aspect, the present application provides a vehicle control method, the method comprising: obtaining road condition information of a target vehicle; according to the road condition information, obtaining target information through an updated first neural network model obtained by the first aspect or any possible implementation method of the first aspect, the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment;

[0021] According to the road condition information and the target information, a control instruction of the target vehicle is determined through an expert system.

[0022] In a third aspect, the present application provides a model training device, the device comprising:

[0023] An acquisition module is used to obtain the road condition information of the target vehicle;

[0024] A processing module, used to obtain target information through a first neural network model according to the road condition information, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment;

[0025] An updating module is used to update the first neural network model according to the target information through an expert system or a result obtained by the expert system processing the road condition information and the label corresponding to the road condition information.

[0026] In one possible implementation, the update module is specifically used to: determine the loss based on the target information and the label; adjust the loss by the feasibility of the label to obtain an adjusted loss, wherein the feasibility of the label is obtained based on the processing result of the expert system on the road condition information and the label corresponding to the road condition information; and update the first neural network model based on the adjusted loss.

[0027] In a possible implementation, the feasibility indicates whether the label corresponding to the road condition information is feasible or not.

[0028] In a possible implementation, the feasibility is a feasibility score; the updating module is specifically used to: use the feasibility score as a weight to adjust the loss.

[0029] In a possible implementation, the feasibility is related to the following information: the safety or comfort of the target vehicle when the target vehicle is controlled to travel according to the tag.

[0030] In one possible implementation, the expert system is a second neural network model; the update module is specifically used to: determine the control instructions of the target vehicle through the second neural network model based on the road condition information and the target information; control the target vehicle and the environment around the target vehicle to interact according to the control instructions, and determine the interaction results; and update the first neural network model according to the interaction results.

[0031] In one possible implementation, the update module is also used to update the second neural network model according to the control instruction and the label corresponding to the control instruction, where the label corresponding to the control instruction is obtained by processing the road condition information and the target information through a rule-based expert system.

[0032] In a fourth aspect, the present application provides a vehicle control device, which includes: an acquisition module for acquiring road condition information of a target vehicle; a processing module for obtaining target information based on the road condition information through an updated first neural network model obtained according to the first aspect or any possible implementation method of the first aspect, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment; and based on the road condition information and the target information, a control instruction for the target vehicle is determined through an expert system.

[0033] In a fifth aspect, an embodiment of the present application provides a computing device which may include a memory, a processor, and a bus system, wherein the memory is used to store programs, and the processor is used to execute programs in the memory to execute the first aspect or any possible implementation method of the first aspect, as well as the second aspect or any possible implementation method of the second aspect.

[0034] In a sixth aspect, an embodiment of the present application provides a vehicle, comprising a sensor and a vehicle control device as in the fourth aspect; the sensor is used to collect road condition information of the vehicle.

[0035] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, it enables the computer to execute the above-mentioned first aspect or any possible implementation method of the first aspect, as well as the above-mentioned second aspect or any possible implementation method of the second aspect.

[0036] In an eighth aspect, an embodiment of the present application provides a computer program product, comprising code, which, when executed, is used to implement the above-mentioned first aspect or any possible implementation method of the first aspect, as well as the above-mentioned second aspect or any possible implementation method of the second aspect.

[0037] In a ninth aspect, the present application provides a chip system, which includes a processor for supporting a computing device to implement the functions involved in the above aspects, such as sending or processing the data involved in the above methods; or information. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for executing the device or training the device. The chip system can be composed of a chip, or it can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A structural diagram of the main framework of artificial intelligence;

[0039] Figure 2 A schematic diagram of an automatic driving device provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0041] Figure 4 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0042] Figure 5 A flowchart of a model training method provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of a lane-changing behavior of a vehicle in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of determining a reward value provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of pre-training provided in an embodiment of the present application;

[0046] Fig. 9 A flowchart of a model training method provided in an embodiment of the present application;

[0047] Fig. 10A A schematic diagram of the structure of a model training device provided in an embodiment of the present application;

[0048] Fig. 10B A schematic diagram of the structure of a vehicle control device provided in an embodiment of the present application;

[0049] Fig.11 A schematic diagram of a terminal device provided in an embodiment of the present application;

[0050] Fig.12 A schematic diagram of a server provided in an embodiment of the present application;

[0051] Fig.13 A schematic diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present invention are described below in conjunction with the accompanying drawings in the embodiments of the present invention. The terms used in the implementation mode of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0055] First, the overall workflow of the artificial intelligence system is described. Figure 1 , Figure 1The figure shows a structural diagram of the main framework of artificial intelligence. The following is an explanation of the above artificial intelligence theme framework from the two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has undergone a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the industrial ecology process of the system.

[0056] (1) Infrastructure

[0057] The infrastructure provides computing power support for the AI ​​system, enables communication with the outside world, and supports it through the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips (CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0058] (2) Data

[0059] The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0060] (3) Data processing

[0061] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0062] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0063] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0064] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0065] (4) General ability

[0066] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0067] (5) Smart products and industry applications

[0068] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical applications. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0069] This application can be applied to the autonomous driving module of a vehicle.

[0070] In one possible implementation, the vehicle may be an internal combustion engine vehicle using an engine as a power source, a hybrid vehicle using an engine and an electric motor as power sources, an electric vehicle using an electric motor as a power source, and the like.

[0071] In the embodiment of the present application, the vehicle may include a driving device 100 having a driving function.

[0072] Reference Figure 2 , Figure 2 1 is a functional block diagram of a driving device 100 with an automatic driving function provided in an embodiment of the present application. In one embodiment, the driving device 100 is configured to be in a fully or partially automatic driving mode. For example, the driving device 100 can control itself while being in the automatic driving mode, and can determine the current state of the automatic driving device and its surrounding environment through human operation, determine the possible behavior of at least one other automatic driving device in the surrounding environment, and determine the confidence level corresponding to the possibility of the other automatic driving device performing the possible behavior, and control the driving device 100 based on the determined information. When the driving device 100 is in the automatic driving mode, the driving device 100 can be set to operate without human interaction.

[0073] Driving device 100 may include various subsystems, such as travel system 102, sensor system 104, control system 106, one or more peripheral devices 108 and power supply 110, computer system 112 and user interface 116. Optionally, driving device 100 may include more or less subsystems, and each subsystem may include multiple elements. In addition, each subsystem and element of driving device 100 can be interconnected by wire or wirelessly.

[0074] The travel system 102 may include components that provide powered movement for the driving device 100. In one embodiment, the travel system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels / tires 121. The engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine 118 converts the energy source 119 into mechanical energy.

[0075] Examples of energy source 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 119 may also provide energy to other systems of ride device 100 .

[0076] The transmission 120 can transmit mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 may also include other devices, such as a clutch. Among them, the drive shaft may include one or more shafts that can be coupled to one or more wheels 121.

[0077] The sensor system 104 may include several sensors that sense information about the environment around the driving device 100. For example, the sensor system 104 may include a positioning system 122 (the positioning system may be a global positioning system (GPS) system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU) 124, a radar 126 (or a radar sensor), a laser rangefinder 128, and a camera 130. The sensor system 104 may also include sensors of the internal systems of the monitored driving device 100 (e.g., an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors may be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). Such detection and recognition are key functions for the safe operation of the autonomous driving device 100.

[0078] Positioning system 122 may be used to estimate the geographic location of driving device 100. IMU 124 is used to sense position and orientation changes of driving device 100 based on inertial acceleration. In one embodiment, IMU 124 may be a combination of an accelerometer and a gyroscope.

[0079] The positioning system 122 may also include a receiver, which may receive signals from navigation satellites.

[0080] The radar 126 may utilize radio signals to sense objects within the surrounding environment of the driving device 100. In some embodiments, in addition to sensing objects, the radar 126 may also be used to sense the speed and / or heading of the objects.

[0081] The radar 126 may include an electromagnetic wave transmitting unit and a receiving unit. The radar 126 may be implemented as a pulse radar or a continuous wave radar based on the principle of radio wave transmission. In the continuous wave radar, the radar 126 may be implemented as a frequency modulated continuous wave (FMCW) or a frequency shift keying (FSK) according to the signal waveform.

[0082] The radar 126 can detect an object based on a time of flight (TOF) method or a phase-shift method using electromagnetic waves as a medium, and detect the position of the detected object, the distance to the detected object, and the relative speed. In order to detect an object located in front of, behind, or to the side of the vehicle, the radar 126 can be configured at an appropriate position outside the vehicle. The laser radar 126 can detect an object based on a TOF method or a phase-shift method using laser as a medium, and detect the position of the detected object, the distance to the detected object, and the relative speed.

[0083] Alternatively, in order to detect objects located in front of, behind, or to the sides of the vehicle, the lidar 126 may be configured at a suitable location on the exterior of the vehicle.

[0084] The laser rangefinder 128 may utilize laser light to sense objects in the environment in which the driving device 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0085] The camera 130 may be used to capture multiple images of the surrounding environment of the driving device 100. The camera 130 may be a still camera or a video camera.

[0086] Among them, in the embodiment of the present application, the sensor system 104 can collect the road condition information of the vehicle.

[0087] The control system 106 controls the operation of the driving device 100 and its components. The control system 106 may include various elements, including a steering system 132 , a throttle 134 , a brake unit 136 , a sensor fusion algorithm 138 , a computer vision system 140 , a path control system 142 , and an obstacle avoidance system 144 .

[0088] The steering system 132 is operable to adjust the forward direction of the driving device 100. For example, in one embodiment, it may be a steering wheel system.

[0089] The throttle 134 is used to control the operating speed of the engine 118 and, in turn, the speed of the steering device 100 .

[0090] The brake unit 136 is used to control the driving device 100 to decelerate. The brake unit 136 can use friction to slow down the wheel 121. In other embodiments, the brake unit 136 can convert the kinetic energy of the wheel 121 into electric current. The brake unit 136 can also take other forms to slow down the rotation speed of the wheel 121 to control the speed of the driving device 100.

[0091] The computer vision system 140 can be operated to process and analyze the images captured by the camera 130 in order to identify objects and / or features in the surrounding environment of the driving device 100. The objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 140 can use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 can be used to map the environment, track objects, estimate the speed of objects, and the like.

[0092] The route control system 142 is used to determine the driving route of the driving device 100. In some embodiments, the route control system 142 may combine data from the sensors 138, the positioning system 122, and one or more predetermined maps to determine the driving route for the driving device 100.

[0093] The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise negotiate potential obstacles in the environment of the steering device 100 .

[0094] Among them, in an embodiment of the present application, the computer vision system 140, the route control system 142 and the obstacle avoidance system 144 can be implemented by a neural network (for example, a first neural network model). The first neural network model can determine the vehicle's driving intention, path planning or interaction strategy with the environment (for example, obstacle avoidance strategy, etc.) based on the road condition information collected by the sensor system 104.

[0095] The expert system 145 may be a rule-based algorithm that can determine control signals for the vehicle based on road condition information and the vehicle's driving intention, path planning, or interaction strategy with the environment output by the neural network, so as to control the vehicle (for example, controlling the vehicle's steering system 132, throttle 134, or brake unit 136) to drive safely and comfortably in a traffic environment.

[0096] Of course, in one example, the control system 106 may include additional or alternative components other than those shown and described, or may also reduce some of the components shown above.

[0097] The driving device 100 interacts with external sensors, other autonomous driving devices, other computer systems, or users through the peripheral devices 108. The peripheral devices 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and / or a speaker 152.

[0098] In some embodiments, the peripheral device 108 provides a means for the user of the driving device 100 to interact with the user interface 116. For example, the onboard computer 148 can provide information to the user of the driving device 100. The user interface 116 can also operate the onboard computer 148 to receive the user's input. The onboard computer 148 can be operated by a touch screen. In other cases, the peripheral device 108 can provide a means for the driving device 100 to communicate with other devices located in the car. For example, the microphone 150 can receive audio (e.g., voice commands or other audio input) from the user of the driving device 100. Similarly, the speaker 152 can output audio to the user of the driving device 100.

[0099] The wireless communication system 146 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as code division multiple access (CDMA), EVDO, global system for mobile communications (GSM) / general packet radio service (GPRS), or 4G cellular communication, such as long term evolution (LTE), or 5G cellular communication. The wireless communication system 146 can communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the wireless communication system 146 can communicate directly with the device using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various autonomous driving device communication systems, for example, the wireless communication system 146 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between autonomous driving devices and / or roadside stations.

[0100] The power source 110 can provide power to various components of the driving device 100. In one embodiment, the power source 110 can be a rechargeable lithium ion or lead acid battery. One or more battery packs of such batteries can be configured as a power source to provide power to various components of the driving device 100. In some embodiments, the power source 110 and the energy source 119 can be implemented together, such as in some all-electric vehicles.

[0101] Some or all functions of the driving device 100 are controlled by a computer system 112. The computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer-readable medium such as a memory 114. The computer system 112 may also be a plurality of computing devices that control individual components or subsystems of the driving device 100 in a distributed manner.

[0102] Processor 113 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware based processor. Figure 2 The processor, memory, and other elements of the computer 110 in the same block are functionally illustrated, but it will be appreciated by those of ordinary skill in the art that the processor, computer, or memory may actually include multiple processors, computers, or memories that may or may not be stored in the same physical housing. For example, the memory may be a hard drive or other storage medium located in a housing different from the computer 110. Therefore, references to a processor or computer will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Rather than using a single processor to perform the steps described herein, some components such as the steering assembly and the deceleration assembly may each have their own processor that performs only calculations related to the functions specific to the component.

[0103] In various aspects described herein, the processor may be located remote from the driving device and wirelessly communicate with the driving device. In other aspects, some of the processes described herein are executed on a processor disposed in the driving device and others are executed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0104] In some embodiments, the memory 114 may include instructions 115 (e.g., program logic) that may be executed by the processor 113 to perform various functions of the driving device 100, including those described above. The memory 114 may also include additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the travel system 102, the sensor system 104, the control system 106, and the peripheral device 108.

[0105] In this embodiment of the present application, the processor 113 can implement the functions of the computer vision system 140, the route control system 142, the obstacle avoidance system 144 and the expert system 145 introduced above by executing the instructions 115 in the memory 114.

[0106] In addition to instructions 115, memory 114 may also store data such as road maps, route information, the position, direction, speed, and other such autopilot data of the autopilot, and other information. Such information may be used by the autopilot 100 and computer system 112 during operation of the autopilot 100 in autonomous, semi-autonomous, and / or manual modes.

[0107] The user interface 116 is used to provide information to or receive information from a user of the driving device 100. Optionally, the user interface 116 may include one or more input / output devices within the set of peripherals 108, such as a wireless communication system 146, an onboard computer 148, a microphone 150, and a speaker 152.

[0108] The computer system 112 may control functions of the driving device 100 based on input received from various subsystems (e.g., the travel system 102, the sensor system 104, and the control system 106) and from the user interface 116. For example, the computer system 112 may utilize input from the control system 106 in order to control the steering unit 132 to avoid obstacles detected by the sensor system 104 and the obstacle avoidance system 144. In some embodiments, the computer system 112 may be operable to provide control over many aspects of the driving device 100 and its subsystems.

[0109] Alternatively, one or more of the above components may be installed or associated separately from the driving device 100. For example, the memory 114 may be partially or completely separate from the driving device 100. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0110] Optionally, the above components are only examples. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 2 It should not be understood as limiting the embodiments of the present application.

[0111] An autonomous vehicle traveling on a road, such as the driving device 100 above, can identify objects in its surrounding environment to determine adjustments to the current speed. The objects can be other autonomous driving devices, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and based on the respective characteristics of the object, such as its current speed, acceleration, distance from the autonomous driving device, etc., it can be used to determine the speed to be adjusted by the autonomous vehicle.

[0112] Optionally, the autonomous driving vehicle driving device 100 or a computing device associated with the driving device 100 (such as Figure 2 The computer system 112, computer vision system 140, memory 114) can predict the behavior of the identified object based on the characteristics of the identified object and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all the identified objects can also be considered together to predict the behavior of a single identified object. The driving device 100 can adjust its speed based on the predicted behavior of the identified object. In other words, the autonomous driving car can determine what stable state the autonomous driving device will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the object. In this process, other factors can also be considered to determine the speed of the driving device 100, such as the lateral position of the driving device 100 in the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0113] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device may also provide instructions to modify the steering angle of the driving device 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0114] The driving device 100 may be a car, a truck, a motorcycle, a bus, a ship, an airplane, a helicopter, a lawn mower, an amusement vehicle, an amusement park automatic driving device, construction equipment, a tram, a golf cart, a train, and a cart, etc., and the embodiments of the present application are not particularly limited.

[0115] It should be understood that the steps related to model training and reasoning process in the method provided in the embodiment of the present application involve AI-related operations. Figure 3 The system architecture provided in the embodiments of the present application is introduced in detail.

[0116] Figure 3 This is a schematic diagram of the system architecture provided in the embodiment of the present application. Figure 3As shown, the system architecture 500 includes an execution device 510 , a training device 520 , a database 530 , a client device 540 , a data storage system 550 , and a data collection system 560 .

[0117] The execution device 510 includes a calculation module 511, an I / O interface 512, a preprocessing module 513 and a preprocessing module 514. The calculation module 511 may include a target model / rule 501, and the preprocessing module 513 and the preprocessing module 514 are optional.

[0118] The execution device 510 may be a wheeled mobile device.

[0119] The data acquisition device 560 is used to collect training samples. The training samples can be real vehicle driving data (for example, including road condition information of the vehicle at multiple times and corresponding driving information), or they can be simulated vehicle driving data in a simulator, etc. After collecting the training samples, the data acquisition device 560 stores the training samples in the database 530.

[0120] The training device 520 can train the neural network to be trained (such as the neural network model in the embodiment of the present application (such as including a first neural network model and a second neural network model, etc.)) based on the training samples maintained in the database 530 to obtain the target model / rule 501.

[0121] It should be understood that the training device 520 can perform a pre-training process on the neural network to be trained based on the training samples maintained in the database 530, or fine-tune the model based on the pre-training.

[0122] It should be noted that, in actual applications, the training samples maintained in the database 530 may not all come from the data acquisition device 560, but may also be received from other devices. It should also be noted that the training device 520 may not train the target model / rule 501 entirely based on the training samples maintained in the database 530, but may also obtain training samples from the cloud or other places for model training. The above description should not be used as a limitation on the embodiments of the present application.

[0123] The target model / rule 501 obtained by training the training device 520 can be applied to different systems or devices, such as Figure 3 The execution device 510 shown may be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, an augmented reality (AR) / virtual reality (VR) device, a vehicle terminal, etc., or a server, etc.

[0124] Specifically, the training device 520 may transfer the trained model to the execution device 510 .

[0125] exist Figure 3 In the embodiment, the execution device 510 is configured with an input / output (I / O) interface 512 for data interaction with an external device. A user can input data to the I / O interface 512 through a client device 540 .

[0126] The preprocessing module 513 and the preprocessing module 514 are used to preprocess the input data received by the I / O interface 512. It should be understood that there may be no preprocessing module 513 and the preprocessing module 514 or only one preprocessing module. When there is no preprocessing module 513 and the preprocessing module 514, the computing module 511 may be directly used to process the input data.

[0127] When the execution device 510 preprocesses the input data, or when the computing module 511 of the execution device 510 performs calculations and other related processing, the execution device 510 can call the data, code, etc. in the data storage system 550 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing into the data storage system 550.

[0128] Finally, the I / O interface 512 provides the processing results to the client device 540 and thus to the user.

[0129] exist Figure 3 In the case shown, the user can manually give input data, and the "manually given input data" can be operated through the interface provided by the I / O interface 512. In another case, the client device 540 can automatically send input data to the I / O interface 512. If the client device 540 is required to automatically send input data and needs to obtain the user's authorization, the user can set the corresponding authority in the client device 540. The user can view the results output by the execution device 510 on the client device 540, and the specific presentation form can be a specific method such as display, sound, action, etc. The client device 540 can also be used as a data acquisition terminal to collect the input data of the input I / O interface 512 and the output results of the output I / O interface 512 as shown in the figure as new sample data, and store them in the database 530. Of course, it is also possible not to collect through the client device 540, but the I / O interface 512 directly stores the input data of the input I / O interface 512 and the output results of the output I / O interface 512 as new sample data in the database 530.

[0130] It is worth noting that Figure 3 This is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 3 In the embodiment, the data storage system 550 is an external memory relative to the execution device 510. In other cases, the data storage system 550 may also be placed in the execution device 510. It should be understood that the execution device 510 may be deployed in the client device 540.

[0131] From the inference side of the model:

[0132] In the embodiment of the present application, the computing module 511 of the above-mentioned execution device 520 can obtain the code stored in the data storage system 550 to implement the steps related to the model reasoning process in the embodiment of the present application.

[0133] In an embodiment of the present application, the computing module 511 of the execution device 520 may include a hardware circuit (such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, etc.), or a combination of these hardware circuits. For example, the training device 520 may be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0134] Specifically, the computing module 511 of the execution device 520 can be a hardware system with an execution instruction function, and the steps related to the model reasoning process provided in the embodiment of the present application can be software codes stored in the memory. The computing module 511 of the execution device 520 can obtain the software code from the memory and execute the obtained software code to implement the steps related to the model reasoning process provided in the embodiment of the present application.

[0135] It should be understood that the computing module 511 of the execution device 520 can be a combination of a hardware system that does not have the function of executing instructions and a hardware system that has the function of executing instructions. Some of the steps related to the model reasoning process provided in the embodiments of the present application can also be implemented by the hardware system that does not have the function of executing instructions in the computing module 511 of the execution device 520, which is not limited here.

[0136] From the training side of the model:

[0137] In the embodiment of the present application, the training device 520 can obtain the memory ( Figure 3Not shown in the figure, the code stored in the training device 520 (or deployed separately from the training device 520) can be integrated into the training device 520 to implement the steps related to model training in the embodiments of the present application.

[0138] In an embodiment of the present application, the training device 520 may include a hardware circuit (such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, etc.), or a combination of these hardware circuits. For example, the training device 520 may be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0139] It should be understood that the training device 520 can be a combination of a hardware system that does not have the function of executing instructions and a hardware system that has the function of executing instructions. Some of the steps related to model training provided in the embodiments of the present application can also be implemented by the hardware system that does not have the function of executing instructions in the training device 520, which is not limited here.

[0140] The first neural network model and expert system introduced above can be referred to as the regulation and control module in the vehicle. Figure 4 , Figure 4 The AI ​​model in can be a first neural network model, Figure 4 The vehicle in the figure may specifically be a traveling system in a vehicle.

[0141] The embodiments of the present application are mainly applied to the regulation and control system in autonomous driving. The regulation and control system plays an important role in the autonomous driving system. Figure 4 In the basic framework of autonomous driving shown in the figure, the regulatory control system receives all the information from the perception, positioning, and prediction modules, plans a reasonable trajectory based on the current task and converts it into vehicle control quantity, so as to control the vehicle to drive safely and comfortably in the traffic environment. The AI ​​model of the regulatory control system and the expert system exist at the same time. Due to the unexplainability of the AI ​​model and the difficulty in determining the fuzzy functional boundaries, the AI ​​model of the regulatory control system needs to be backed up by the expert system.

[0142] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.

[0143] (1) Neural Network

[0144] A neural network may be composed of neural units, and a neural unit may refer to an operation unit that takes xs (i.e., input data) and intercept 1 as input, and the output of the operation unit may be:

[0145]

[0146] Where s=1, 2, ...n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neural units.

[0147] (2) Deep Neural Networks

[0148] Deep Neural Network (DNN), also known as multi-layer neural network, can be understood as a neural network with many hidden layers. There is no special metric for "many" here. From the position of different layers of DNN, the neural network inside DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: in, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since DNN has many layers, the coefficient W and the offset vector The definition of these parameters in DNN is as follows: Take coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input.

[0149] In summary, the coefficients from the kth neuron in the L-1th layer to the jth neuron in the Lth layer are defined as

[0150] It should be noted that the input layer does not have a W parameter. In a deep neural network, more hidden layers allow the network to better describe complex situations in the real world. Theoretically, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by many layers of vector W).

[0151] (3) Loss Function

[0152] In the process of training a deep neural network, because we hope that the output of the deep neural network is as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the target value we really want, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the predicted value of the network is high, adjust the weight vector to make it predict a lower value, and keep adjusting until the deep neural network can predict the target value we really want or a value very close to the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0153] (4) Back propagation algorithm

[0154] The error back propagation (BP) algorithm can be used to correct the size of the parameters in the initial model during the training process, so that the error loss of the model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the parameters in the initial model are updated by back propagating the error loss information, so that the error loss converges. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal model parameters, such as the weight matrix.

[0155] The key technologies in the field of autonomous driving include perception, decision-making, planning, and control. Among them, the vehicle's autonomous decision-making ability plays a key role in the intelligence and safety of the entire autonomous driving system. Compared with expert systems, data-driven AI methods have the ability to make human-like decisions in complex scenarios and are the development trend of autonomous driving technology. However, data-driven AI methods are not explainable and need to be used in conjunction with regulatory and control expert systems at present and for a considerable period of time in the future. Therefore, compared with AI applications in the field of perception, AI in the field of regulatory and control must not only have the ability to learn human-like decisions from massive human driving data, but also need to consider adaptation to regulatory and control expert systems.

[0156] Specifically, the AI ​​model in the regulatory control system can predict the vehicle's driving intention, path planning, or interaction strategy with the environment based on road condition information. However, the results output by the AI ​​model may be poor from the perspective of driving safety or comfort. Therefore, a rule-based algorithm is needed to judge the results output by the AI ​​model to determine whether the results output by the AI ​​model can be used, and if it is determined that they can be used, determine the vehicle's control information.

[0157] The current mainstream data-driven methods are mostly open-loop learning methods (such as imitation learning). The idea of ​​this method is relatively direct: learn the end-to-end mapping model from observations to posterior trajectory outputs from massive human driving data. However, this method learning model is difficult to consider the actual closed-loop effect of the output action, resulting in a decrease in the actual application effect.

[0158] In combination with the above description, the embodiments of the present application provide a model training method and a road topology prediction method, which can be applied to the training stage and reasoning stage of the model, respectively, and are described below.

[0159] 1. Training Phase

[0160] In the embodiment of the present application, the training phase is as described above. Figure 3 The training device 520 performs a training operation on the target model 501 using the training samples in the training set. Figure 5 , Figure 5 A flow chart of a training method for a model provided in an embodiment of the present application may specifically include:

[0161] 501. Obtain road condition information of the target vehicle.

[0162] In one possible implementation, when training the AI ​​model in the vehicle's control system (such as the first neural network model in the embodiment of the present application), training samples can be obtained. The training samples may include road condition information of the target vehicle at multiple times and corresponding driving information. For example, the driving information may be driving intentions (such as left turns, right turns, lane keeping, etc.), driving trajectories, and interactive decisions with the environment (such as whether to avoid obstacles, etc.).

[0163] Since driving information is often inaccurate (for example, it is not optimal in terms of safety or comfort), an expert module is needed to provide a safety net and correct the driving information. In existing implementations, during the training of AI models, driving information is often used as a label to update the AI ​​model, and the output of the subsequent expert system is not introduced. Therefore, the labels used to update the first neural network model have a lower confidence level in essence, resulting in lower processing accuracy of the trained model.

[0164] In the embodiment of the present application, the output of the expert system is used to participate in the training of the first neural network model, which is equivalent to closed-loop training of the first neural network model, thereby improving the accuracy of the AI ​​model.

[0165] Next, we first introduce the training samples in the embodiment of this application:

[0166] In a possible implementation, the training sample may include road condition information of the target vehicle at a certain moment.

[0167] The target vehicle may be a physical vehicle, and the road condition information may be derived from road condition information in driving data of a real vehicle.

[0168] For example, the road condition information may be data collected by sensors (such as cameras and radars) on the road and other surrounding environments where the target vehicle is located.

[0169] In addition, the target vehicle may also be a virtual vehicle in the simulator, and the road condition information may come from the road condition information in the simulated driving data of the vehicle in the simulator.

[0170] Among them, road condition information may include status information of surrounding traffic participants, including position, posture, speed, acceleration, etc.; as well as road-related information, including lane speed limit, remaining lane length, virtual and real lines, traffic signals, etc.

[0171] In addition, the state of the target vehicle can also be obtained, and this state can be input into the first neural network model together with the road condition information. The state of the target vehicle can be the vehicle position, speed (such as longitudinal and lateral speeds in vehicle coordinates) and posture (such as vehicle roll, pitch and yaw), acceleration, etc. of the target vehicle.

[0172] 502. According to the road condition information, target information is obtained through a first neural network model, where the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment.

[0173] For example, the target information may include driving intention (such as turning left, turning right, lane keeping, etc.), driving trajectory, and interactive decision-making with the environment (such as whether to avoid obstacles, etc.).

[0174] 503. Update the first neural network model according to the target information through an expert system or a result obtained by the expert system processing the road condition information and the label corresponding to the road condition information.

[0175] In a possible implementation, the road condition information and the labels corresponding to the target information may be input into the expert system to obtain the processing result of the expert system.

[0176] Among them, the label corresponding to the road condition information can be data in the actual driving process (which can be a driving process in a simulation environment or a driving process in an actual physical environment) (for example, it can be a prediction of driving intention, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment), and this data can be used as the true value corresponding to the target information in the training sample.

[0177] Among them, the expert system can be implemented based on rules or based on neural networks.

[0178] In a possible implementation, the feasibility of the label corresponding to the road condition information can be determined according to the processing result, that is, if the target vehicle performs the corresponding driving control according to the label, whether the safety or comfort of the vehicle itself meets the requirements. For example, the feasibility can be represented by 1 or 0, where 1 means feasible and 0 means infeasible. Figure 6 ,If the label is left lane change, and the expert system determines that there is a safety risk for the vehicle after the left lane change, the expert system can output that left lane change is not feasible and control the vehicle to keep the lane.

[0179] In a possible implementation, the processing result may be a feasible score of the label, that is, the degree of satisfaction of the vehicle's own safety or comfort when the target vehicle performs driving control corresponding to the label.

[0180] Among them, the expert system can be a strategy-based algorithm (non-neural network algorithm).

[0181] When processing labels, the expert system can first determine the state s of the regulatory system based on the road condition information. The state s of the regulatory system can be the assignment of some parameters within the expert system. After that, the expert system can obtain the state s' of the regulatory system at the next moment based on the road condition information and the label (for example, taking the label and the road condition information as the input of the expert system, or taking the data obtained by mapping the label and the road condition information as the input of the expert system), and at the same time obtain the feasibility of the label based on the information such as the state s of the regulatory system and the state s' at the next moment.

[0182] It should be understood that the feasibility can be data directly output by the expert system, or the feasibility of the label can be determined based on the output result of the expert system. For example, the expert system can output a corrected label, and the feasibility can be determined based on the difference between the label before and after correction. For example, if the corrected label is no different from the label before correction, the label is considered feasible, and if the corrected label is different from the label before correction, the label is considered infeasible.

[0183] The feasibility obtained by the expert system can be considered as an evaluation of the label. Therefore, the feasibility can be introduced into the training process of the AI ​​model. Specifically, the loss can be determined based on the target information and the corresponding label (since the label may not be accurate, the loss may also be inaccurate. If the AI ​​model is updated based on inaccurate loss, the training accuracy of the model will be poor). Therefore, in the embodiment of the present application, the loss is adjusted by the feasibility, and the update gradient is determined based on the adjusted loss (the adjusted loss is equivalent to introducing the output of the expert system).

[0184] For example, the feasibility indicates whether the label is feasible or not. When the feasibility indicates that the label is feasible, it can be determined that the first neural network model needs to be updated. When the feasibility indicates that the label is not feasible, it can be determined that the first neural network model does not need to be updated, or the absolute size of the update gradient can be reduced accordingly.

[0185] For example, the feasibility may indicate a feasibility score of the label, and the feasibility score may be used as a weight to adjust the loss. When the feasibility score is large, it may be considered that the accuracy of the label is high, and thus the update gradient obtained based on the feasibility score is also large. When the feasibility score is small, it may be considered that the accuracy of the label is low, and thus the update gradient obtained based on the feasibility score is also small.

[0186] Specifically, there is a massive human driving data set D = [x, y] (where x is the network input and y is the label), and D is constructed as a discrete point data set or a time series flow data set as needed. D is connected to the regulatory control system (for example, a rule-based expert system), that is, the regulatory control system state s is set according to x, and the label quantities y and x are used as the regulatory control system inputs to obtain the next state s' of the regulatory control system and the output a of the regulatory control system, and obtain the data tuple [x, y, s, a, s'] containing the regulatory control characteristics. At the same time, an evaluation model R is designed to evaluate the regulatory control response, and the score r = R (s, a (based on x and y), s'). The above operations are performed on the data set D to obtain the score data set Ds = [x, y, r] containing the regulatory control characteristics. When training based on the score data set, the score r will be used as the weight of the corresponding sample to affect the gradient of the model update.

[0187] Reference Figure 7 , taking the target information as lane change decision as an example, for the collected lane change data set D = [x, y], x is the road condition information, y is the lane change information (that is, the label in the embodiment of the present application), that is, left lane change, lane keeping, right lane change. D is injected into the expert system frame by frame, and the state of the expert system is initialized using x to obtain s, and y is input into the expert system as an inspiration command. At this time, the expert system will generate a corresponding output a for each frame of data, and update it to the successor state s', so as to obtain a data tuple [x, y, s, a, s']. Subsequently, a reward model (first reward model) is designed to score the tuple. The reward model can be used in offline learning, specifically for determining the feasibility of the label based on the output of the expert system. Exemplarily, in this embodiment, the reward model can score by whether y is the same as a. If y is the same as a, it proves that the regulatory control expert system can respond to the input of y, and gets 1 point (that is, it indicates that the target information is feasible); if y is different from a, it proves that the regulatory control expert system cannot respond to the input of y, and gets 0 points (that is, it indicates that the target information is not feasible). In this way, the augmented data 1[x, y, r] is obtained.

[0188] The first neural network model is trained based on the augmented data set 1. Compared with the traditional data set whose value includes [input quantity, label quantity], the augmented data set 1 has an additional scoring item. During the training process, the scoring item will participate in the model parameter update process as the weight of this sample. In this embodiment, if the sample is not compatible with the regulation and control, r = 0, this sample will not have any effect on the actual parameter update; if the sample is compatible with the regulation and control, r = 1, the sample will have an effect on the model parameter update.

[0189] In addition, in a possible implementation, the expert system may be a parameterized neural network model (that is, a neural network, which may be referred to as a second neural network model in the embodiment of the present application). The parameterized neural network model may be trained so that it has the same or similar data processing capability as the expert system before the parameters.

[0190] Reference Figure 8 In this embodiment, an offline training model m (that is, the first neural network model) can be used. Under the input x, the expert system receives the output u of the offline training model m, and obtains the corresponding expert system output a, thereby obtaining an augmented data set 2 [x, u, a]. Augmented data set 2 includes the characteristics of the expert system under the offline training model m. Based on augmented data set 2, with (x, u) as input and a as label training, a parameterized model f (that is, the second neural network model) of the expert system can be obtained. At this time, the generalization performance of the parameterized model f under the offline training model m is better.

[0191] For example, assume that the road condition information obtained during the interaction with the regulatory control system is x, and the output of the first neural network model under x is u, and both are connected to the expert system at the same time to obtain the output a of the expert system. After sufficient rounds of interaction, the regulatory control characteristic data set Dp = [x, u, a] can be obtained. Based on the data set Dp, a neural network model a = f (x, u; w) of a rule-based expert system can be trained, where f is the expert system parameterized model (that is, the second neural network model in the embodiment of the present application), and w is the model parameter.

[0192] In the case where the expert system is a non-neural network, the reward value corresponding to the output of the expert system cannot be directly used to determine the update gradient of the first neural network model (the reason is that the expert system is a non-neural network and cannot perform gradient feedback, and the expert system based on the strategy algorithm often has a large number of state machine transitions, random sampling, optimization and other modules. If the characteristics of the expert system are learned through random exploration, it will be very slow, which greatly increases the difficulty of online learning interaction). However, if the expert system is parameterized during the training process, the output of the expert system can be used to determine the reward value, and when the gradient is fed back, it can be passed to the first neural network model. On the one hand, the update gradient corresponding to the first neural network model can be obtained, and the update gradient is obtained based on the output of the expert system implemented based on the neural network model, which is equivalent to introducing the output obtained by the expert system implemented based on the neural network model into the training process of the first neural network model, thereby improving the processing accuracy of the trained model.

[0193] In a possible implementation, the expert system is a second neural network model; the control instructions of the target vehicle can be determined through the second neural network model according to the road condition information and the target information; the target vehicle and the environment around the target vehicle are controlled to interact according to the control instructions to obtain the interaction results, and the first neural network model is updated according to the interaction results. Exemplarily, a reward model (second reward model) can be designed to determine the reward value according to the interaction result, determine the update gradient of the first neural network according to the reward value, and update the first neural network model using the update gradient of the first neural network model.

[0194] Optionally, the second neural network model may also be updated according to the control instruction and the label corresponding to the control instruction, where the label corresponding to the control instruction is obtained by processing the road condition information and the target information through a rule-based expert system.

[0195] Regarding the reward model that determines the reward value based on the interaction result, it can be based on the existing reinforcement learning implementation and will not be elaborated here.

[0196] For example, refer to Fig. 9 The regulation and control system includes an offline training model m (first neural network model) and a parameterized model f (second neural network model) of the regulation and control expert system. This embodiment continues to tune the two models through online interaction. Specifically, m and f are brought into the simulation system. At this time, due to the existence of f, a can be directly explored, and the exploration is more efficient. The reward model (second reward model) is brought into the online exploration to obtain the interaction data [x, (u, a), x', r, d]. Among them, u and a are the output of the first neural network model and the output of the second neural network model under given road condition information x. f can be continuously updated based on the data [u, a]. Among them, a is the exploration data, and m can be updated based on [x, a, x', r, d] using traditional reinforcement learning. x' is the road condition information after the target vehicle interacts with the surrounding environment, r is the score (for example, reward value) obtained by the reward model according to the interaction result after the target vehicle interacts with the surrounding environment, and d is a mark of whether the task execution is completed, such as task execution completion, execution termination, etc. Thus, continuous online optimization of m and f is achieved until convergence.

[0197] It should be noted that, in some embodiments of the present application, there are multiple ways to determine the extent to which the first neural network model should be trained using the updated gradient. The following provides some termination conditions for terminating the training of the first neural network model, including but not limited to:

[0198] (1) The loss function reaches the preset threshold.

[0199] After configuring the loss function, a threshold value (e.g., 0.03) can be set for the loss function in advance. During the iterative training of the first neural network model, after each training, it is determined whether the value of the loss function obtained in the current round of training reaches the threshold value. If not, the training continues. If it reaches the preset threshold value, the training is terminated. Then, the values ​​of the network parameters of the first neural network model determined by the current round of training are used as the network parameter values ​​of the finally trained first neural network model.

[0200] (2) The loss function begins to converge.

[0201] After configuring the loss function, the first neural network model can be iteratively trained. If the difference between the value of the loss function obtained in the current round of training and the value of the loss function obtained in the previous round of training is within a preset range (e.g., 0.01), the loss function is considered to have converged, and the training can be terminated. The values ​​of the network parameters of the first neural network model determined by the current round of training are used as the network parameter values ​​of the finally trained first neural network model.

[0202] (3) The training reaches the preset number of times.

[0203] In this way, the number of iterations for training the first neural network model can be pre-configured (e.g., 1000 times). After the loss function is configured, the first neural network model can be iteratively trained. After each round of training, the values ​​of the network parameters of the first neural network model corresponding to that round will be stored until the number of training iterations reaches a preset number. After that, the first neural network models obtained in each round are verified using test data, and the value of the network parameter with the best performance is selected as the final value of the network parameter of the first neural network model.

[0204] 2. Reasoning Stage

[0205] In the embodiment of the present application, the reasoning stage is as described above. Figure 3 The execution device 510 uses the trained target model 501 to control the vehicle. Specifically, during the actual reasoning, the execution device 510 can obtain the road condition information of the target vehicle and Figure 5 The updated first neural network model obtained in the corresponding embodiment obtains target information, which is the prediction of the driving intention of the target vehicle, the prediction of the driving path, or the prediction of the interactive behavior between the target vehicle and the environment, and then determines the control instructions of the target vehicle through the expert system based on the road condition information and the target information.

[0206] Next, a model training device provided by an embodiment of the present application is introduced from the perspective of a device. The model training device may be the training device 520 mentioned above. Fig. 10A , Fig. 10A A schematic diagram of the structure of a model training device provided in an embodiment of the present application is shown in FIG. Fig. 10A As shown, a model training device 1000 provided in an embodiment of the present application includes:

[0207] The acquisition module 1001 is used to acquire the road condition information of the target vehicle;

[0208] For a detailed introduction to the acquisition module 1001 , reference may be made to the description of step 501 in the above embodiment, which will not be repeated here.

[0209] The processing module 1002 is used to obtain target information through a first neural network model according to the road condition information, where the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment;

[0210] For a detailed introduction to the processing module 1002 , reference may be made to the description of step 502 in the above embodiment, which will not be repeated here.

[0211] The updating module 1003 is used to update the first neural network model according to the target information through the expert system or the result obtained by the expert system processing the road condition information and the label corresponding to the road condition information.

[0212] For a detailed introduction to the update module 1003, reference may be made to the description of step 503 in the above embodiment, which will not be repeated here.

[0213] In a possible implementation, the updating module 1003 is specifically configured to:

[0214] The loss is determined according to the target information and the label corresponding to the road condition information; the loss is adjusted by feasibility to obtain an adjusted loss, where the feasibility is obtained by the result obtained by the expert system processing the road condition information and the label corresponding to the road condition information; the first neural network model is updated according to the adjusted loss.

[0215] In one possible implementation, the feasibility indicates whether the tag is feasible or not.

[0216] In a possible implementation, the feasibility is a feasibility score; the updating module 1003 is specifically configured to:

[0217] The feasibility score is used as a weight to adjust the loss.

[0218] In a possible implementation, the feasibility is related to the following information:

[0219] The safety or comfort of the target vehicle when the target vehicle is controlled according to the tag.

[0220] The expert system is a second neural network model; the updating module 1003 is specifically used for:

[0221] According to the road condition information and the target information, the control instructions of the target vehicle are determined through the second neural network model; the target vehicle is controlled to interact with the environment around the target vehicle according to the control instructions to determine the interaction results; and the first neural network model is updated according to the interaction results.

[0222] In a possible implementation, the update module 1003 is further used to update the second neural network model according to the control instruction and the label corresponding to the control instruction, where the label corresponding to the control instruction is obtained by processing the road condition information and the target information through the rule-based expert system.

[0223] Next, a vehicle control device provided by an embodiment of the present application is introduced from the perspective of a device. The vehicle control device may be the aforementioned execution device 510. Fig. 10B , is a schematic diagram of the structure of a vehicle control device provided in an embodiment of the present application, such as Fig. 10B As shown, a vehicle control device 1010 provided in an embodiment of the present application includes:

[0224] The acquisition module 1011 is used to acquire the road condition information of the target vehicle;

[0225] Processing module 1012, used for Figure 5 The corresponding embodiment obtains the updated first neural network model and obtains target information, where the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment;

[0226] And according to the road condition information and the target information, the control instructions of the target vehicle are determined through the expert system.

[0227] Next, a terminal device 1100 provided in an embodiment of the present application is introduced. Fig.11 , Fig.11A structural diagram of a terminal device provided in an embodiment of the present application, the terminal device 1100 can be specifically manifested as a mobile phone, a tablet, a laptop computer, a smart wearable device, a smart car, an in-vehicle computing platform, an in-vehicle domain controller, an in-vehicle terminal, etc., which is not limited here. Among them, the terminal device 1100 implements the function of the execution device 510. Specifically, the terminal device 1100 includes: a receiver 1101, a transmitter 1102, a processor 1103 and a memory 1104 (wherein the number of processors 1103 in the terminal device 1100 can be one or more), wherein the processor 1103 may include an application processor 11031 and a communication processor 11032. In some embodiments of the present application, the receiver 1101, the transmitter 1102, the processor 1103 and the memory 1104 may be connected via a bus or other means.

[0228] The memory 1104 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1103. A portion of the memory 1104 may also include a non-volatile random access memory (NVRAM). The memory 1104 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0229] The processor 1103 controls the operation of the terminal device. In a specific application, the various components of the terminal device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0230] The method disclosed in the above embodiment of the present application can be applied to the processor 1103, or implemented by the processor 1103. The processor 1103 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware or software instructions in the processor 1103. The above processor 1103 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and a vision processor (vision processing unit, VPU), a tensor processor (tensorprocessing unit, TPU) and other processors suitable for AI computing, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1103 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1104, and the processor 1103 reads the information in the memory 1104, and completes the steps 501 to 503 in the above embodiment in combination with its hardware.

[0231] The receiver 1101 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the terminal device. The transmitter 1102 can be used to output digital or character information through the first interface; the transmitter 1102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1102 can also include a display device such as a display screen.

[0232] The present application embodiment also provides a server, which may be the training device 520 mentioned above, see Fig.12 , Fig.121 is a schematic diagram of a structure of a server provided in an embodiment of the present application. Specifically, the server 1200 is implemented by one or more servers. The server 1200 may have relatively large differences due to different configurations or performances. It may include one or more central processing units (CPU) 1212 (for example, one or more processors) and memory 1232, and one or more storage media 1230 (for example, one or more mass storage devices) storing application programs 1242 or data 1244. Among them, the memory 1232 and the storage medium 1230 can be short-term storage or persistent storage. The program stored in the storage medium 1230 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1212 can be configured to communicate with the storage medium 1230 and execute a series of instruction operations in the storage medium 1230 on the server 1200.

[0233] The server 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input and output interfaces 1258; or, one or more operating systems 1241, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0234] Specifically, the server may perform steps 501 to 503 in the above embodiment.

[0235] Also provided in an embodiment of the present application is a computer program product which, when executed on a computer, enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0236] A computer-readable storage medium is also provided in an embodiment of the present application, which stores a program for signal processing. When the computer-readable storage medium is run on a computer, it enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0237] An embodiment of the present application also provides a vehicle, which includes a sensor, and the above-mentioned execution device 510 or vehicle control device 1010 or terminal device 1100; the sensor is used to collect road condition information of the vehicle.

[0238] The execution device, training device or terminal device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit so that the chip executes the method of the aforementioned embodiment executed by the aforementioned execution device, or so that the chip executes the method of the aforementioned embodiment executed by the aforementioned training device. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0239] For details, please refer to Fig.13 , Fig.13 A schematic diagram of the structure of a chip provided in an embodiment of the present application, the chip can be expressed as a neural network processor NPU1300, NPU 1300 is mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks. The core part of the NPU is the operation circuit 1303, which is controlled by the controller 1304 to extract matrix data from the memory and perform multiplication operations.

[0240] NPU 1300 can achieve Figure 5 The training method of the model provided in the described embodiment.

[0241] More specifically, in some implementations, the operation circuit 1303 in the NPU 1300 includes multiple processing units (Process Engine, PE) inside. In some implementations, the operation circuit 1303 is a two-dimensional systolic array. The operation circuit 1303 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1303 is a general-purpose matrix processor.

[0242] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 1302 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1301 and performs matrix operation with matrix B, and the partial result or final result of the matrix is ​​stored in the accumulator 1308.

[0243] The unified memory 1306 is used to store input data and output data. The weight data is directly transferred to the weight memory 1302 through the direct memory access controller (DMAC) 1305. The input data is also transferred to the unified memory 1306 through the DMAC.

[0244] BIU stands for Bus Interface Unit, i.e., bus interface unit 1310 , which is used for interaction between AXI bus, DMAC and instruction fetch buffer (IFB) 1309 .

[0245] The bus interface unit 1310 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1309 to obtain instructions from the external memory, and is also used for the storage unit access controller 1305 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0246] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1306 or to transfer weight data to the weight memory 1302 or to transfer input data to the input memory 1301.

[0247] The vector calculation unit 1307 includes multiple operation processing units, and further processes the output of the operation circuit 1303 when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as Batch Normalization, pixel-level summation, upsampling of feature planes, etc.

[0248] In some implementations, the vector calculation unit 1307 can store the processed output vector to the unified memory 1306. For example, the vector calculation unit 1307 can apply a linear function; or a nonlinear function to the output of the operation circuit 1303, such as linear interpolation of the feature plane extracted by the convolution layer, and then, for example, a vector of accumulated values ​​to generate an activation value. In some implementations, the vector calculation unit 1307 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1303, for example, for use in a subsequent layer in a neural network.

[0249] An instruction fetch buffer 1309 connected to the controller 1304 is used to store instructions used by the controller 1304;

[0250] Unified memory 1306, input memory 1301, weight memory 1302 and instruction fetch memory 1309 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0251] The processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above programs.

[0252] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0253] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0254] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0255] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A model training method, characterized in that: The method comprises: Obtain the road condition information of the target vehicle; According to the road condition information, target information is obtained through a first neural network model, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment; According to the target information, the first neural network model is updated through an expert system or a result obtained by the expert system processing the road condition information and the labels corresponding to the road condition information.

2. The method according to claim 1, characterized in that The updating of the first neural network model according to the target information by an expert system or a result obtained by the expert system processing the road condition information and the label corresponding to the road condition information includes: Determining a loss according to the target information and the label; adjusting the loss by a feasibility to obtain an adjusted loss, wherein the feasibility is obtained based on the result; The first neural network model is updated according to the adjusted loss.

3. The method according to claim 2, characterized in that The feasibility is a feasibility score; The adjusting the loss by the feasibility comprises: The feasibility score is used as a weight to adjust the loss.

4. The method according to claim 2 or 3, characterized in that: The feasibility is related to the following information: When the target vehicle is controlled to travel according to the target information, the safety or comfort of the target vehicle.

5. The method according to claim 1, characterized in that The expert system is a second neural network model, and the first neural network model is updated according to the target information by the expert system or the result obtained by the expert system processing the road condition information and the label corresponding to the road condition information, including: Determining a control instruction of the target vehicle through the second neural network model according to the road condition information and the target information; Controlling the target vehicle to interact with the environment around the vehicle according to the control instruction, and determining an interaction result; According to the interaction result, the first neural network model is updated.

6. The method according to claim 5, characterized in that The method further comprises: The second neural network model is updated according to the control instruction and the label corresponding to the control instruction, wherein the label corresponding to the control instruction is obtained by processing the road condition information and the target information through an expert system based on a rule.

7. A vehicle control method, characterized in that: The method comprises: Obtain the road condition information of the target vehicle; According to the road condition information, target information is obtained by using the updated first neural network model obtained by any method of claims 1 to 6, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment; According to the road condition information and the target information, a control instruction of the target vehicle is determined through an expert system.

8. A model training device, characterized in that: The device comprises: An acquisition module is used to obtain the road condition information of the target vehicle; A processing module, configured to obtain target information through a first neural network model according to the road condition information, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment; An updating module is used to update the first neural network model according to the target information through an expert system or a result obtained by the expert system processing the road condition information and the labels corresponding to the road condition information.

9. The device according to claim 8, characterized in that The update module is specifically used for: Determining a loss according to the target information and the label; adjusting the loss by a feasibility to obtain an adjusted loss, wherein the feasibility is obtained based on the result; The first neural network model is updated according to the adjusted loss.

10. The device according to claim 9, characterized in that The feasibility is a feasibility score; The update module is specifically used for: The feasibility score is used as a weight to adjust the loss.

11. The device according to claim 9 or 10, characterized in that The feasibility is related to the following information: When the target vehicle is controlled to travel according to the target information, the safety or comfort of the target vehicle.

12. The device according to claim 8, characterized in that The expert system is a second neural network model; the updating module is specifically used for: According to the road condition information and the target information, the control instructions of the target vehicle are determined through the second neural network model; according to the control instructions, the target vehicle and the environment around the target vehicle are controlled to interact and the interaction results are determined; according to the interaction results, the first neural network model is updated.

13. The device according to claim 12, characterized in that The updating module is also used to update the second neural network model according to the control instruction and the label corresponding to the control instruction, wherein the label corresponding to the control instruction is obtained by processing the road condition information and the target information through an expert system based on rules.

14. A vehicle control device, characterized in that: The device comprises: An acquisition module is used to obtain the road condition information of the target vehicle; A processing module, configured to obtain target information according to the road condition information by using the updated first neural network model obtained by any one of the methods described in claims 1 to 6, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment; According to the road condition information and the target information, a control instruction of the target vehicle is determined through an expert system.

15. A computing device, characterized in that: The computing device includes a memory and a processor; the memory stores codes, and the processor is configured to obtain the codes and execute the method according to any one of claims 1 to 7.

16. A vehicle, characterized in that: comprising a sensor, and a vehicle control device as claimed in claim 14; The sensor is used to collect road condition information of the vehicle.

17. A computer storage medium, characterized in that: The computer storage medium stores one or more instructions, which, when executed by one or more computers, enable the one or more computers to implement the method of any one of claims 1 to 7.

18. A computer program product comprising code, characterized in that When the code is executed, it is used to implement the method according to any one of claims 1 to 7.

19. A chip, comprising a processor, characterized in that: The processor is configured to execute the method according to any one of claims 1 to 7.

Images