Model training method, vehicle control method, and related device

Abstract

A model training method, a vehicle control method, and a related device, which can be applied to the field of artificial intelligence (AI). The model training method comprises: acquiring road condition information of a target vehicle; on the basis of the road condition information, obtaining target information by means of a first neural network model, wherein the target information is the driving intention prediction of the target vehicle, the travel route prediction or the interaction behavior prediction of the target vehicle and the environment; and on the basis of the target information, updating the first neural network model by means of an expert system or by means of a result obtained by processing by the expert system the road condition information and tags corresponding to the road condition information. According to the present application, an output of an expert system is used to guide and determine the gradient used for updating an AI model, which is equivalent to performing closed-loop training on the AI model, so that the precision of the AI model is improved.

Description

Model training method, vehicle control method and related device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 11, 2023, with application number 202311018397.2 and application name “Model training method, vehicle control method and related device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence, and in particular to a model training method, a vehicle control method, and related devices. Background Art

[0003] Artificial intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0004] Key technologies in the field of autonomous driving include perception, decision-making, planning, and control. The vehicle's autonomous decision-making capabilities play a crucial role in the intelligence and safety of the entire autonomous driving system. Compared to expert systems, data-driven AI approaches offer human-like decision-making capabilities in complex scenarios and are the trend in the development of autonomous driving technology. However, data-driven AI approaches lack interpretability and will require integration with regulatory and control expert systems for the foreseeable future.

[0005] Currently, mainstream data-driven approaches are mostly open-loop learning methods (such as imitation learning). These methods are relatively straightforward: they learn an end-to-end mapping model from massive amounts of human driving data, from observations to posterior trajectory outputs. However, these learning methods fail to consider the actual closed-loop effects of the model's outputs, resulting in reduced practical application effectiveness.

[0006] Summary of the Invention

[0007] This application provides a model training method, a vehicle control method and related devices, which can perform closed-loop training on the AI ​​model in the vehicle's regulation and control system, thereby improving the accuracy of the AI ​​model.

[0008] In the first aspect, the present application provides a model training method, specifically, obtaining road condition information of a target vehicle; based on the road condition information, obtaining target information through a first neural network model, the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment; based on the target information, the first neural network model is updated by an expert system or the result obtained by processing the road condition information and the labels corresponding to the road condition information through the expert system.

[0009] Since driving information is often inaccurate (for example, it is not optimal in terms of safety or comfort), an expert module is needed to provide a safety net and correct the driving information. In existing implementations, during the training of AI models, driving information is often used as a label to update the AI ​​model, and the output of the subsequent expert system is not introduced. Therefore, the confidence level of the label used to update the first neural network model is actually low, resulting in low processing accuracy of the trained model.

[0010] Among them, the label corresponding to the road condition information can be data in the actual driving process (which can be a driving process in a simulation environment or a driving process in an actual physical environment) (for example, it can be a prediction of driving intention, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment). This data can be used as the true value corresponding to the target information in the training sample.

[0011] In an embodiment of the present application, the expert system or the output of the expert system is utilized to participate in the training of the first neural network model, which is equivalent to closed-loop training of the first neural network model, so that the first neural network model has the characteristics of a regulatory expert system, thereby improving the accuracy of the first neural network model.

[0012] Among them, the expert system can be implemented based on rules or based on neural networks.

[0013] In one possible implementation, when determining the loss for updating the first neural network model, the loss is determined based on the label corresponding to the target information and the road condition information, and the loss is adjusted based on the feasibility of the label. The feasibility of the label can be obtained based on the processing results of the expert system on the road condition information and the label corresponding to the road condition information.

[0014] The feasibility of the label corresponding to the road condition information can be considered as an evaluation of the label. Therefore, the feasibility can be introduced into the training process of the first neural network model. Specifically, the loss can be determined based on the target information and the corresponding label (since the label may not be accurate, the loss may also be inaccurate. If the AI ​​model is updated based on inaccurate loss, the training accuracy of the model will be poor). Therefore, in the embodiment of the present application, the loss is adjusted by the feasibility, and based on the adjusted loss (the adjusted loss is equivalent to introducing the output of the expert system).

[0015] In a possible implementation, the feasibility indicates whether the label corresponding to the road condition information is feasible or not.

[0016] In one possible implementation, the feasibility indicates whether the safety or comfort of the target vehicle itself meets the requirements when the target vehicle performs corresponding driving control according to the label. For example, the feasibility can be represented by 1 or 0, where 1 indicates feasible and 0 indicates infeasible. For example, the feasibility indicates whether the label is feasible or infeasible. When the feasibility indicates that the label is feasible, it can be determined that the AI ​​model needs to be updated. When the feasibility indicates that the label is infeasible, it can be determined that the AI ​​model does not need to be updated, or the absolute size of the update gradient can be reduced accordingly.

[0017] In one possible implementation, the feasibility is a feasibility score; the loss is adjusted by using the feasibility, including: using the feasibility score as a weight to adjust the loss. The feasibility score of the label can be determined based on the processing result of the expert system, that is, the degree of satisfaction of the vehicle's own safety or comfort when the target vehicle performs corresponding driving control according to the label. For example, the feasibility can indicate the feasibility score of the label, and the feasibility score can be used as a weight to adjust the loss. When the feasibility score is large, it can be considered that the accuracy of the label is high, and thus the update gradient obtained based on the feasibility score is also large. When the feasibility score is small, it can be considered that the accuracy of the label is low, and thus the update gradient obtained based on the feasibility score is also small.

[0018] In a possible implementation, the feasibility is related to the following information: the safety or comfort of the target vehicle when driving control of the target vehicle is performed according to the tag.

[0019] In one possible implementation, the expert system is a second neural network model, that is, when training the first neural network model, the expert system is parameterized (the parameterized expert system is the second neural network model); based on the road condition information and the target information, the control instructions of the target vehicle can be determined through the second neural network model; according to the control instructions, the target vehicle is controlled to interact with the environment, the interaction results are determined, and the first neural network model is updated according to the interaction results.

[0020] For example, a reward value can be determined based on the interaction result; based on the reward value, an update gradient of the first neural network model is determined, and the update gradient is used to update the first neural network model.

[0021] In the case where the expert system is not a neural network, the reward value corresponding to the output of the expert system cannot be directly used to determine the update gradient of the first neural network model (the reason is that the expert system is a non-neural network and cannot perform gradient backpropagation, and the expert system based on the policy algorithm often has a large number of state machine transitions, random sampling, optimization and other modules. If the characteristics of the expert system are learned through random exploration, it will be very slow, which greatly increases the difficulty of online learning interaction). However, if the expert system is parameterized during the training process, the output of the expert system can be used to determine the reward value. When the gradient is backpropagated, it can be passed to the first neural network model. On the one hand, the update gradient corresponding to the first neural network model can be obtained, and the update gradient is obtained based on the output of the expert system, which is equivalent to introducing the output obtained by the expert system into the training process of the AI ​​model, thereby improving the processing accuracy of the trained model.

[0022] In a second aspect, the present application provides a vehicle control method, the method comprising: obtaining road condition information of a target vehicle; and obtaining target information based on the road condition information using an updated first neural network model obtained according to the first aspect or any possible implementation method of the first aspect, wherein the target information is a prediction of the target vehicle's driving intention, a prediction of its driving path, or a prediction of the interaction between the target vehicle and its environment.

[0023] According to the road condition information and the target information, a control instruction of the target vehicle is determined through an expert system.

[0024] In a third aspect, the present application provides a model training device, the device comprising:

[0025] An acquisition module is used to obtain the road condition information of the target vehicle;

[0026] a processing module configured to obtain target information based on the road condition information through a first neural network model, the target information being a prediction of the target vehicle's driving intention, a prediction of its driving path, or a prediction of the interaction between the target vehicle and its environment;

[0027] An updating module is used to update the first neural network model according to the target information through an expert system or a result obtained by the expert system processing the road condition information and the label corresponding to the road condition information.

[0028] In one possible implementation, the update module is specifically used to: determine the loss based on the target information and the label; adjust the loss according to the feasibility of the label to obtain an adjusted loss, where the feasibility of the label is obtained based on the processing results of the expert system on the road condition information and the label corresponding to the road condition information; and update the first neural network model based on the adjusted loss.

[0029] In a possible implementation, the feasibility indicates whether the label corresponding to the road condition information is feasible or not.

[0030] In a possible implementation, the feasibility is a feasibility score; the updating module is specifically configured to use the feasibility score as a weight to adjust the loss.

[0031] In a possible implementation, the feasibility is related to the following information: the safety or comfort of the target vehicle when driving control of the target vehicle is performed according to the tag.

[0032] In one possible implementation, the expert system is a second neural network model; the update module is specifically used to: determine the control instructions of the target vehicle through the second neural network model based on the road condition information and the target information; control the target vehicle and the environment around the target vehicle to interact according to the control instructions, and determine the interaction results; and update the first neural network model according to the interaction results.

[0033] In one possible implementation, the update module is further used to update the second neural network model according to the control instruction and the label corresponding to the control instruction, where the label corresponding to the control instruction is obtained by processing the road condition information and the target information through a rule-based expert system.

[0034] In a fourth aspect, the present application provides a vehicle control device, which includes: an acquisition module for acquiring road condition information of a target vehicle; a processing module for obtaining target information based on the road condition information through an updated first neural network model obtained according to the first aspect or any possible implementation method of the first aspect, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment; and based on the road condition information and the target information, a control instruction for the target vehicle is determined through an expert system.

[0035] In a fifth aspect, an embodiment of the present application provides a computing device, which may include a memory, a processor, and a bus system, wherein the memory is used to store programs, and the processor is used to execute the programs in the memory to execute the first aspect or any possible implementation method of the first aspect, as well as the second aspect or any possible implementation method of the second aspect.

[0036] In a sixth aspect, an embodiment of the present application provides a vehicle, which includes a sensor and a vehicle control device as in the fourth aspect; the sensor is used to collect road condition information of the vehicle.

[0037] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, it enables the computer to execute the above-mentioned first aspect or any possible implementation method of the first aspect, as well as the above-mentioned second aspect or any possible implementation method of the second aspect.

[0038] In an eighth aspect, an embodiment of the present application provides a computer program product, comprising code, which, when executed, is used to implement the above-mentioned first aspect or any possible implementation method of the first aspect, as well as the above-mentioned second aspect or any possible implementation method of the second aspect.

[0039] In a ninth aspect, the present application provides a chip system comprising a processor configured to support a computing device in implementing the functions described in the aforementioned aspects, such as transmitting or processing data or information described in the aforementioned methods. In one possible design, the chip system further comprises a memory configured to store program instructions and data necessary for executing or training the device. The chip system may consist solely of a chip or may include a chip and other discrete components. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic diagram of the structure of the artificial intelligence main framework;

[0041] FIG2 is a schematic diagram of an automatic driving device provided in an embodiment of the present application;

[0042] FIG3 is a schematic diagram of a system architecture provided in an embodiment of the present application;

[0043] FIG4 is a schematic diagram of a system architecture provided in an embodiment of the present application;

[0044] FIG5 is a flow chart of a model training method provided in an embodiment of the present application;

[0045] FIG6 is a schematic diagram of a lane-changing behavior scenario of a vehicle in an embodiment of the present application;

[0046] FIG7 is a schematic diagram of determining a reward value according to an embodiment of the present application;

[0047] FIG8 is a schematic diagram of a pre-training method provided in an embodiment of the present application;

[0048] FIG9 is a flow chart of a model training method provided in an embodiment of the present application;

[0049] FIG10A is a schematic structural diagram of a model training device provided in an embodiment of the present application;

[0050] FIG10B is a schematic structural diagram of a vehicle control device provided in an embodiment of the present application;

[0051] FIG11 is a schematic diagram of a terminal device provided in an embodiment of the present application;

[0052] FIG12 is a schematic diagram of a server provided in an embodiment of the present application;

[0053] FIG13 is a schematic diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following describes the embodiments of the present invention in conjunction with the accompanying drawings. The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0055] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0056] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0057] First, let's describe the overall workflow of an AI system. See Figure 1, which shows a schematic diagram of the AI ​​framework. This framework will be explained from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed journey from "data-information-knowledge-wisdom." The "IT value chain," spanning the underlying infrastructure of human intelligence, information (provided and processed by technology), and the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.

[0058] (1) Infrastructure

[0059] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. External communication occurs through sensors; computing power is provided by intelligent chips (CPUs, NPUs, GPUs, ASICs, FPGAs, and other hardware accelerators). The foundational platform includes a distributed computing framework and network-related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.

[0060] (2) Data

[0061] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0062] (3) Data processing

[0063] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0064] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0065] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0066] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0067] (4) General ability

[0068] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0069] (5) Smart products and industry applications

[0070] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0071] This application can be applied to the autonomous driving module of a vehicle.

[0072] In one possible implementation, the vehicle may be an internal combustion engine vehicle using an engine as a power source, a hybrid vehicle using an engine and an electric motor as power sources, an electric vehicle using an electric motor as a power source, and the like.

[0073] In the embodiment of the present application, the vehicle may include a driving device 100 having a driving function.

[0074] Referring to Figure 2, Figure 2 is a functional block diagram of a driving device 100 with an automatic driving function provided by an embodiment of the present application. In one embodiment, the driving device 100 is configured to be in a fully or partially automatic driving mode. For example, the driving device 100 can control itself while in the automatic driving mode, and can determine the current state of the automatic driving device and its surrounding environment through human operation, determine the possible behavior of at least one other automatic driving device in the surrounding environment, and determine the confidence level corresponding to the possibility of the other automatic driving device performing the possible behavior, and control the driving device 100 based on the determined information. When the driving device 100 is in the automatic driving mode, the driving device 100 can be set to operate without human interaction.

[0075] The driving device 100 may include various subsystems, such as a travel system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, and a user interface 116. Optionally, the driving device 100 may include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and component of the driving device 100 may be interconnected by wire or wirelessly.

[0076] The travel system 102 may include components that provide powered motion for the driving device 100. In one embodiment, the travel system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels / tires 121. The engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine 118 converts the energy source 119 into mechanical energy.

[0077] Examples of energy source 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 119 can also provide power to other systems of the driving device 100.

[0078] The transmission 120 can transmit mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 may also include other components, such as a clutch. The drive shaft may include one or more shafts that can be coupled to one or more wheels 121.

[0079] The sensor system 104 may include several sensors that sense information about the environment surrounding the driving device 100. For example, the sensor system 104 may include a positioning system 122 (the positioning system may be a global positioning system (GPS) system, or a BeiDou system or other positioning system), an inertial measurement unit (IMU) 124, a radar 126 (or a radar sensor), a laser rangefinder 128, and a camera 130. The sensor system 104 may also include sensors of the internal systems of the monitored driving device 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). Such detection and recognition are key functions for the safe operation of the autonomous driving device 100.

[0080] The positioning system 122 may be used to estimate the geographic location of the driving device 100. The IMU 124 may be used to sense changes in the position and orientation of the driving device 100 based on inertial acceleration. In one embodiment, the IMU 124 may be a combination of an accelerometer and a gyroscope.

[0081] The positioning system 122 may further include a receiver, which may receive signals from navigation satellites.

[0082] The radar 126 may utilize radio signals to sense objects in the surrounding environment of the driving device 100. In some embodiments, in addition to sensing objects, the radar 126 may also be used to sense the speed and / or heading of the objects.

[0083] Radar 126 may include an electromagnetic wave transmitter and receiver. Based on the principle of radio wave transmission, radar 126 may be implemented as a pulse radar or a continuous wave radar. The continuous wave radar may be implemented as a frequency modulated continuous wave (FMCW) or frequency shift keying (FSK) depending on the signal waveform.

[0084] Radar 126 can detect objects using electromagnetic waves based on a time of flight (TOF) or phase-shift method, and can measure the position, distance, and relative speed of the detected object. To detect objects in front of, behind, or to the side of the vehicle, radar 126 can be positioned appropriately outside the vehicle. LiDAR 126 can detect objects using a TOF or phase-shift method using laser light, and can measure the position, distance, and relative speed of the detected object.

[0085] Alternatively, in order to detect objects located in front of, behind, or to the sides of the vehicle, the lidar 126 may be configured at a suitable location on the exterior of the vehicle.

[0086] The laser rangefinder 128 may utilize laser light to sense objects in the environment in which the driving device 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0087] The camera 130 may be used to capture multiple images of the surrounding environment of the driving device 100. The camera 130 may be a still camera or a video camera.

[0088] Among them, in the embodiment of the present application, the sensing system 104 can collect the road condition information of the vehicle.

[0089] Control system 106 controls the operation of driving device 100 and its components and may include various components, including a steering system 132 , a throttle 134 , a brake unit 136 , a sensor fusion algorithm 138 , a computer vision system 140 , a path control system 142 , and an obstacle avoidance system 144 .

[0090] The steering system 132 is operable to adjust the forward direction of the driving device 100. For example, in one embodiment, it can be a steering wheel system.

[0091] The throttle 134 is used to control the operating speed of the engine 118 and, in turn, the speed of the steering device 100 .

[0092] Braking unit 136 is used to control the deceleration of driving device 100. Braking unit 136 can use friction to slow down wheels 121. In other embodiments, braking unit 136 can convert the kinetic energy of wheels 121 into electric current. Braking unit 136 can also take other forms to slow the rotation speed of wheels 121 and thus control the speed of driving device 100.

[0093] The computer vision system 140 can be operated to process and analyze images captured by the camera 130 to identify objects and / or features in the environment surrounding the driving device 100. The objects and / or features may include traffic signs, road boundaries, and obstacles. The computer vision system 140 can use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 can be used to map the environment, track objects, estimate the speed of objects, and so on.

[0094] Route control system 142 is used to determine a driving route for driving device 100. In some embodiments, route control system 142 may combine data from sensors 138, positioning system 122, and one or more predetermined maps to determine a driving route for driving device 100.

[0095] The obstacle avoidance system 144 is used to identify, assess, and avoid or otherwise negotiate potential obstacles in the environment of the driving device 100 .

[0096] In an embodiment of the present application, the computer vision system 140, the route control system 142, and the obstacle avoidance system 144 can be implemented by a neural network (e.g., a first neural network model). The first neural network model can determine the vehicle's driving intention, path planning, or interaction strategy with the environment (e.g., obstacle avoidance strategy, etc.) based on the road condition information collected by the sensing system 104.

[0097] The expert system 145 can be a rule-based algorithm that can determine the control signal for the vehicle based on road condition information and the vehicle's driving intention, path planning, or interaction strategy with the environment output by the neural network, so as to control the vehicle (for example, control the vehicle's steering system 132, throttle 134, or brake unit 136) to drive safely and comfortably in the traffic environment.

[0098] Of course, in one example, the control system 106 may include additional or alternative components other than those shown and described, or may include fewer than some of the components shown.

[0099] Driving device 100 interacts with external sensors, other autonomous driving devices, other computer systems, or users via peripherals 108. Peripherals 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and / or a speaker 152.

[0100] In some embodiments, peripheral devices 108 provide a means for the user of driving device 100 to interact with user interface 116. For example, onboard computer 148 can provide information to the user of driving device 100. User interface 116 can also operate onboard computer 148 to receive user input. Onboard computer 148 can be operated via a touch screen. In other cases, peripheral devices 108 can provide a means for driving device 100 to communicate with other devices located in the vehicle. For example, microphone 150 can receive audio (e.g., voice commands or other audio input) from the user of driving device 100. Similarly, speaker 152 can output audio to the user of driving device 100.

[0101] The wireless communication system 146 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as code division multiple access (CDMA), EVDO, global system for mobile communications (GSM) / general packet radio service (GPRS), or 4G cellular communication, such as long term evolution (LTE), or 5G cellular communication. The wireless communication system 146 can use WiFi to communicate with a wireless local area network (WLAN). In some embodiments, the wireless communication system 146 can use an infrared link, Bluetooth, or ZigBee to communicate directly with the device. Other wireless protocols, such as various autonomous driving device communication systems, for example, the wireless communication system 146 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between autonomous driving devices and / or roadside stations.

[0102] The power supply 110 can provide power to the various components of the driving device 100. In one embodiment, the power supply 110 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such batteries can be configured as a power source to provide power to the various components of the driving device 100. In some embodiments, the power supply 110 and the energy source 119 can be implemented together, such as in some all-electric vehicles.

[0103] Some or all functions of the driving device 100 are controlled by a computer system 112. The computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer-readable medium such as a memory 114. The computer system 112 may also be a plurality of computing devices that control individual components or subsystems of the driving device 100 in a distributed manner.

[0104] Processor 113 can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although FIG. 2 functionally illustrates a processor, memory, and other elements of the computer 110 in the same block, it will be understood by those skilled in the art that the processor, computer, or memory can actually include multiple processors, computers, or memories that may or may not be stored in the same physical housing. For example, the memory can be a hard drive or other storage medium located in a housing different from the computer 110. Therefore, references to a processor or computer will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Different from using a single processor to perform the steps described herein, each of some components such as a steering assembly and a deceleration assembly can have its own processor that only performs calculations related to the functions of the component.

[0105] In various aspects described herein, the processor may be located remotely from the driving device and in wireless communication with the driving device. In other aspects, some of the processes described herein are performed on a processor disposed within the driving device while others are performed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0106] In some embodiments, the memory 114 may include instructions 115 (e.g., program logic) that may be executed by the processor 113 to perform various functions of the driving device 100, including those described above. The memory 114 may also include additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the travel system 102, the sensor system 104, the control system 106, and the peripheral devices 108.

[0107] In this embodiment of the present application, the processor 113 can implement the functions of the computer vision system 140, route control system 142, obstacle avoidance system 144 and expert system 145 introduced above by executing instructions 115 in the memory 114.

[0108] In addition to instructions 115, memory 114 may also store data such as road maps, route information, the position, direction, speed, and other such autonomous driving device data of the autonomous driving device, and other information. This information may be used by the autonomous driving device 100 and computer system 112 during operation of the autonomous driving device 100 in autonomous, semi-autonomous, and / or manual modes.

[0109] The user interface 116 is used to provide information to or receive information from a user of the driving device 100. Optionally, the user interface 116 may include one or more input / output devices within the set of peripherals 108, such as a wireless communication system 146, an onboard computer 148, a microphone 150, and a speaker 152.

[0110] Computer system 112 may control functions of driving device 100 based on input received from various subsystems (e.g., travel system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 may utilize input from control system 106 to control steering unit 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144. In some embodiments, computer system 112 may be operable to provide control over many aspects of driving device 100 and its subsystems.

[0111] Alternatively, one or more of the above components may be installed or associated separately from the driving device 100. For example, the memory 114 may be partially or completely separate from the driving device 100. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0112] Optionally, the above components are only an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 2 should not be understood as a limitation to the embodiments of the present application.

[0113] An autonomous vehicle traveling on a road, such as driving device 100 above, can identify objects in its surroundings to determine adjustments to its current speed. These objects can be other autonomous driving devices, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and the speed adjustment to be made to the autonomous vehicle can be determined based on its respective characteristics, such as its current speed, acceleration, and distance from the autonomous driving device.

[0114] Optionally, the autonomous vehicle driving device 100 or a computing device associated with the driving device 100 (such as the computer system 112, computer vision system 140, and memory 114 in FIG. 2 ) can predict the behavior of the identified objects based on their characteristics and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so the behavior of all identified objects can be considered together to predict the behavior of a single identified object. The driving device 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what stable state the autonomous driving device will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. Other factors can also be considered in determining the speed of the driving device 100 during this process, such as the lateral position of the driving device 100 on the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0115] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device may also provide instructions to modify the steering angle of the driving device 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0116] The driving device 100 may be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawn mower, recreational vehicle, amusement park automatic driving device, construction equipment, tram, golf cart, train, and cart, etc., and the embodiments of the present application do not impose any particular limitation.

[0117] It should be understood that the steps related to model training and reasoning process in the method provided in the embodiment of the present application involve AI-related operations. The system architecture provided in the embodiment of the present application is introduced in detail below in conjunction with Figure 3.

[0118] FIG3 is a schematic diagram of the system architecture provided by an embodiment of the present application. As shown in FIG3 , the system architecture 500 includes an execution device 510 , a training device 520 , a database 530 , a client device 540 , a data storage system 550 , and a data acquisition system 560 .

[0119] The execution device 510 includes a calculation module 511, an I / O interface 512, a pre-processing module 513, and a post-processing module 514. The calculation module 511 may include the target model / rule 501, and the pre-processing module 513 and the post-processing module 514 are optional.

[0120] The execution device 510 may be a wheeled mobile device.

[0121] Data acquisition device 560 is used to collect training samples. Training samples can be real vehicle driving data (e.g., including road condition information and corresponding driving information at multiple moments) or simulated vehicle driving data in a simulator. After collecting the training samples, data acquisition device 560 stores them in database 530.

[0122] The training device 520 can train the neural network to be trained (such as the neural network model in the embodiment of the present application (such as including the first neural network model and the second neural network model, etc.)) based on the training samples maintained in the database 530 to obtain the target model / rule 501.

[0123] It should be understood that the training device 520 can perform a pre-training process on the neural network to be trained based on the training samples maintained in the database 530, or fine-tune the model based on the pre-training.

[0124] It should be noted that, in actual applications, the training samples maintained in the database 530 may not all be collected by the data acquisition device 560, but may also be received from other devices. It should also be noted that the training device 520 may not train the target model / rule 501 entirely based on the training samples maintained in the database 530, but may also obtain training samples from the cloud or other places for model training. The above description should not be used as a limitation on the embodiments of the present application.

[0125] The target model / rule 501 obtained through training with the training device 520 can be applied to different systems or devices, such as the execution device 510 shown in FIG3 . The execution device 510 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, an augmented reality (AR) / virtual reality (VR) device, an in-vehicle terminal, etc., or a server, etc.

[0126] Specifically, the training device 520 may transfer the trained model to the execution device 510 .

[0127] In FIG. 3 , the execution device 510 is configured with an input / output (I / O) interface 512 for data interaction with an external device. A user can input data to the I / O interface 512 through a client device 540 .

[0128] Preprocessing module 513 and preprocessing module 514 are used to preprocess the input data received by I / O interface 512. It should be understood that preprocessing module 513 and preprocessing module 514 may be absent or only one preprocessing module may be present. If preprocessing module 513 and preprocessing module 514 are absent, computing module 511 may be used directly to process the input data.

[0129] When the execution device 510 preprocesses the input data, or when the computing module 511 of the execution device 510 performs calculations and other related processing, the execution device 510 can call the data, code, etc. in the data storage system 550 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing in the data storage system 550.

[0130] Finally, the I / O interface 512 provides the processed results to the client device 540 and thus to the user.

[0131] In the scenario shown in FIG3 , the user can manually input data, and this "manual input data" can be operated through the interface provided by I / O interface 512. In another scenario, client device 540 can automatically send input data to I / O interface 512. If user authorization is required for client device 540 to automatically send input data, the user can set the corresponding permissions in client device 540. The user can view the results output by execution device 510 on client device 540, and the specific presentation form can be a display, sound, action, or other specific method. Client device 540 can also serve as a data acquisition terminal, collecting input data input into I / O interface 512 and output results output from I / O interface 512 as new sample data, and storing them in database 530. Of course, collection can also be performed without client device 540, and instead the I / O interface 512 directly stores the input data input into I / O interface 512 and output results output from I / O interface 512 as new sample data in database 530.

[0132] It is worth noting that FIG3 is merely a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in FIG3 , the data storage system 550 is an external memory relative to the execution device 510. In other cases, the data storage system 550 can also be placed in the execution device 510. It should be understood that the execution device 510 can be deployed in the client device 540.

[0133] From the inference side of the model:

[0134] In the embodiment of the present application, the computing module 511 of the above-mentioned execution device 510 can obtain the code stored in the data storage system 550 to implement the steps related to the model reasoning process in the embodiment of the present application.

[0135] In an embodiment of the present application, the computing module 511 of the execution device 510 may include a hardware circuit (such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, etc.), or a combination of these hardware circuits. For example, the training device 520 may be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0136] Specifically, the computing module 511 of the execution device 510 can be a hardware system with an execution instruction function, and the steps related to the model reasoning process provided in the embodiment of the present application can be software codes stored in the memory. The computing module 511 of the execution device 510 can obtain the software code from the memory and execute the obtained software code to implement the steps related to the model reasoning process provided in the embodiment of the present application.

[0137] It should be understood that the computing module 511 of the execution device 510 can be a combination of a hardware system that does not have the function of executing instructions and a hardware system that has the function of executing instructions. Some of the steps related to the model reasoning process provided in the embodiment of the present application can also be implemented by the hardware system that does not have the function of executing instructions in the computing module 511 of the execution device 510, which is not limited here.

[0138] From the training side of the model:

[0139] In an embodiment of the present application, the above-mentioned training device 520 can obtain the code stored in the memory (not shown in Figure 3, which can be integrated into the training device 520 or deployed separately from the training device 520) to implement the steps related to model training in the embodiment of the present application.

[0140] In an embodiment of the present application, the training device 520 may include a hardware circuit (such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, etc.), or a combination of these hardware circuits. For example, the training device 520 may be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0141] It should be understood that the training device 520 can be a combination of a hardware system that does not have the function of executing instructions and a hardware system that has the function of executing instructions. Some of the steps related to model training provided in the embodiments of the present application can also be implemented by the hardware system in the training device 520 that does not have the function of executing instructions, which is not limited here.

[0142] The first neural network model and expert system introduced above can be referred to as the regulation and control module in the vehicle. Referring to Figure 4, the AI ​​model in Figure 4 can be the first neural network model, and the vehicle in Figure 4 can specifically be the driving system in the vehicle.

[0143] The embodiments of the present application are mainly applied to the regulation and control system in autonomous driving. The regulation and control system plays an important role in the autonomous driving system. In the basic framework of autonomous driving shown in Figure 4, the regulation and control system receives all information from the perception, positioning, and prediction modules, plans a reasonable trajectory based on the current task, and converts it into vehicle control variables to control the vehicle to drive safely and comfortably in the traffic environment. The AI ​​model of the regulation and control system coexists with the expert system. Due to the unexplainability of the AI ​​model and the difficulty in determining the fuzzy functional boundaries, the AI ​​model of the regulation and control system needs to be backed up by the expert system.

[0144] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.

[0145] (1) Neural Network

[0146] A neural network can be composed of neural units. A neural unit can refer to an operation unit that takes xs (i.e., input data) and intercept 1 as input. The output of the operation unit can be:

[0147] Where s = 1, 2, ... n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal of the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0148] (2) Deep Neural Networks

[0149] Deep Neural Network (DNN), also known as multi-layer neural network, can be understood as a neural network with many hidden layers. There is no special metric for "many" here. Based on the position of different layers in DNN, the neural network inside DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: in, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since there are many DNN layers, the coefficient W and the offset vector The definition of these parameters in DNN is as follows: Take the coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input.

[0150] In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as

[0151] It's important to note that the input layer has no W parameter. In deep neural networks, more hidden layers allow the network to better capture complex real-world situations. Theoretically, a model with more parameters has higher complexity and greater "capacity," meaning it can handle more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrices for all layers of a trained deep neural network (a weight matrix formed by the vectors W across many layers).

[0152] (3) Loss function

[0153] During the training of a deep neural network, because we want the output of the deep neural network to be as close as possible to the desired predicted value, we can compare the current network's predicted value with the desired target value and then update the weight vector of each layer of the neural network based on the difference between the two. (Of course, before the first update, there is usually an initialization process, which pre-configures the parameters for each layer in the deep neural network.) For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value. This adjustment is continued until the deep neural network can predict the desired target value or a value very close to the desired target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value." This is the loss function (or objective function), which is an important equation used to measure the difference between the predicted value and the target value. For example, the loss function output value (loss) indicates a greater difference, so training a deep neural network becomes a process of minimizing this loss.

[0154] (4) Backpropagation algorithm

[0155] The back propagation (BP) algorithm can be used to correct the size of the initial model parameters during training, reducing the model's error loss. Specifically, forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial model, thereby converging the error loss. The BP algorithm is a backward propagation movement driven by error loss, aiming to obtain optimal model parameters, such as the weight matrix.

[0156] Key technologies in the field of autonomous driving include perception, decision-making, planning, and control. The vehicle's autonomous decision-making capabilities play a crucial role in the intelligence and safety of the entire autonomous driving system. Compared to expert systems, data-driven AI approaches offer human-like decision-making capabilities in complex scenarios and represent the trend in the development of autonomous driving technology. However, data-driven AI approaches lack interpretability and will require integration with regulatory and control expert systems for the foreseeable future. Therefore, compared to AI applications in the perception field, AI in the regulatory and control field must not only possess the ability to learn human-like decision-making from massive amounts of human driving data but also consider compatibility with regulatory and control expert systems.

[0157] Specifically, the AI ​​model in the regulation and control system can predict the vehicle's driving intention, path planning, or interaction strategy with the environment based on road condition information. However, the results output by the AI ​​model may be poor from the perspective of driving safety or comfort. Therefore, a rule-based algorithm is needed to judge the results output by the AI ​​model to determine whether the results output by the AI ​​model can be used, and if it is determined that they can be used, determine the vehicle control information.

[0158] Current mainstream data-driven approaches are mostly open-loop learning methods (such as imitation learning). These methods are relatively straightforward: they learn an end-to-end mapping model from massive amounts of human driving data, from observations to posterior trajectory outputs. However, these learning models fail to consider the actual closed-loop effects of the output actions, resulting in reduced practical application effectiveness.

[0159] In combination with the above description, the embodiments of the present application provide a model training method and a road topology prediction method, which can be applied to the training stage and reasoning stage of the model, respectively, and are described below.

[0160] 1. Training Phase

[0161] In the embodiment of the present application, the training phase is the process in which the training device 520 in FIG3 performs a training operation on the target model 501 using the training samples in the training set. Specifically, please refer to FIG5 , which is a flow chart of a model training method provided in the embodiment of the present application, which may specifically include:

[0162] 501. Obtain road condition information of the target vehicle.

[0163] In one possible implementation, when training the AI ​​model in the vehicle's control system (such as the first neural network model in the embodiment of the present application), training samples can be obtained. The training samples may include road condition information of the target vehicle at multiple times and corresponding driving information. For example, the driving information may be driving intention (such as left turn, right turn, lane keeping, etc.), driving trajectory, and interactive decision-making with the environment (such as whether to avoid obstacles, etc.).

[0164] Since driving information is often inaccurate (for example, it is not optimal in terms of safety or comfort), an expert module is needed to provide a safety net and correct the driving information. In existing implementations, during the training of AI models, driving information is often used as a label to update the AI ​​model, and the output of the subsequent expert system is not introduced. Therefore, the confidence level of the label used to update the first neural network model is actually low, resulting in low processing accuracy of the trained model.

[0165] In the embodiment of the present application, the output of the expert system is used to participate in the training of the first neural network model, which is equivalent to closed-loop training of the first neural network model, thereby improving the accuracy of the AI ​​model.

[0166] Next, we first introduce the training samples in the embodiment of this application:

[0167] In a possible implementation, the training sample may include road condition information of the target vehicle at a certain moment.

[0168] The target vehicle may be a physical vehicle, and the road condition information may be derived from road condition information in driving data of a real vehicle.

[0169] For example, the road condition information may be data collected by sensors (such as cameras and radars) on the road and other surrounding environments where the target vehicle is located.

[0170] In addition, the target vehicle may also be a virtual vehicle in the simulator, and the road condition information may be derived from the road condition information in the simulated driving data of the vehicle in the simulator.

[0171] Among them, road condition information may include status information of surrounding traffic participants, including position, posture, speed, acceleration, etc.; as well as road-related information, including lane speed limit, remaining lane length, virtual and real lines, traffic signals, etc.

[0172] In addition, the state of the target vehicle can also be obtained, which can be input into the first neural network model together with the road condition information. The state of the target vehicle can be the vehicle position, speed (such as longitudinal and lateral speeds in vehicle coordinates) and posture (such as vehicle roll, pitch and yaw), acceleration, etc. of the target vehicle.

[0173] 502. According to the road condition information, target information is obtained through a first neural network model, where the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment.

[0174] For example, the target information may include driving intention (such as turning left, turning right, lane keeping, etc.), driving trajectory, and interactive decision-making with the environment (such as whether to avoid obstacles, etc.).

[0175] 503. Update the first neural network model based on the target information through an expert system or a result obtained by the expert system processing the road condition information and the label corresponding to the road condition information.

[0176] In a possible implementation, the road condition information and the labels corresponding to the target information may be input into the expert system to obtain the processing results of the expert system.

[0177] Among them, the label corresponding to the road condition information can be data in the actual driving process (which can be a driving process in a simulation environment or a driving process in an actual physical environment) (for example, it can be a prediction of driving intention, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment). This data can be used as the true value corresponding to the target information in the training sample.

[0178] Among them, the expert system can be implemented based on rules or based on neural networks.

[0179] In one possible implementation, the feasibility of the label corresponding to the road condition information can be determined based on the processing results. Specifically, this means whether the vehicle's safety or comfort requirements would be met if the target vehicle were to perform driving control corresponding to the label. For example, the feasibility can be represented by 1 or 0, with 1 indicating feasibility and 0 indicating infeasibility. For example, referring to Figure 6 , if the label indicates a left lane change and the expert system determines that the vehicle presents a safety risk after making the left lane change, the expert system may output that a left lane change is infeasible and control the vehicle to perform lane keeping.

[0180] In a possible implementation, the processing result may be a feasibility score of the label, that is, the degree of satisfaction of the vehicle's own safety or comfort when the target vehicle performs the driving control corresponding to the label.

[0181] The expert system may be a strategy-based algorithm (a non-neural network algorithm).

[0182] When processing labels, the expert system can first determine the state s of the regulatory system based on the road condition information. The state s of the regulatory system can be the assignment of some parameters within the expert system. Then, the expert system can obtain the next state s' of the regulatory system based on the road condition information and the label (for example, using the label and road condition information as the input of the expert system, or using the data obtained by mapping the label and road condition information as the input of the expert system). At the same time, the feasibility of the label is obtained based on information such as the state s of the regulatory system and the next state s'.

[0183] It should be understood that the feasibility can be data directly output by the expert system, or the feasibility of a label can be determined based on the output of the expert system. For example, the expert system can output a corrected label, and the feasibility can be determined based on the difference between the label before and after the correction. For example, if the corrected label does not differ from the label before the correction, the label is considered feasible; if the corrected label differs from the label before the correction, the label is considered infeasible.

[0184] The feasibility obtained by the expert system can be considered as an evaluation of the label. Therefore, the feasibility can be introduced into the training process of the AI ​​model. Specifically, the loss can be determined based on the target information and the corresponding label (because the label may not be accurate, the loss may also be inaccurate. If the AI ​​model is updated based on inaccurate loss, the training accuracy of the model will be poor). Therefore, in the embodiment of the present application, the loss is adjusted by the feasibility, and the update gradient is determined based on the adjusted loss (the adjusted loss is equivalent to introducing the output of the expert system).

[0185] For example, the feasibility indicates whether the label is feasible or not. When the feasibility indicates that the label is feasible, it can be determined that the first neural network model needs to be updated. When the feasibility indicates that the label is not feasible, it can be determined that the first neural network model does not need to be updated, or the absolute size of the update gradient can be reduced accordingly.

[0186] For example, the feasibility may indicate a feasibility score of the label, and the feasibility score may be used as a weight to adjust the loss. When the feasibility score is large, it may be considered that the label is more accurate, and thus the update gradient obtained based on the feasibility score is also large. When the feasibility score is small, it may be considered that the label is less accurate, and thus the update gradient obtained based on the feasibility score is also small.

[0187] Specifically, consider a massive human driving dataset D = [x, y] (where x is the network input and y is the label). D is constructed as a discrete point dataset or a time-series stream dataset as needed. D is then connected to a regulatory control system (e.g., a rule-based expert system). Specifically, the regulatory control system state s is set based on x, and the labels y and x are used as the regulatory control system inputs. The next state s' and output a of the regulatory control system are obtained, resulting in a data tuple [x, y, s, a, s'] containing the regulatory control characteristics. Simultaneously, an evaluation model R is designed to evaluate the regulatory control response, with a score r = R(s, a (based on x and y), s'). Performing these operations on dataset D yields a score dataset Ds = [x, y, r] containing the regulatory control characteristics. When training on the score dataset, the score r serves as the weight for the corresponding sample, influencing the gradient of the model update.

[0188] Referring to Figure 7, taking the target information as lane change decision as an example, for the collected lane change data set D = [x, y], x is the road condition information, y is the lane change information (that is, the label in the embodiment of the present application), that is, left lane change, lane keeping, and right lane change. D is injected into the expert system frame by frame, and the state of the expert system is initialized using x to obtain s, and y is input into the expert system as an inspiration command. At this time, the expert system will generate a corresponding output a for each frame of data and update it to the successor state s', so as to obtain a data tuple [x, y, s, a, s']. Subsequently, a reward model (first reward model) is designed to score the tuple. The reward model can be used in offline learning, specifically to determine the feasibility of the label based on the output of the expert system. Exemplarily, in this embodiment, the reward model can score by whether y and a are the same. If y is the same as a, it proves that the regulatory expert system can respond to the input of y, and gets a score of 1 (that is, it indicates that the target information is feasible); if y is different from a, it proves that the regulatory expert system cannot respond to the input of y, and gets a score of 0 (that is, it indicates that the target information is not feasible). In this way, the augmented data 1[x, y, r] is obtained.

[0189] The first neural network model is trained based on augmented dataset 1. Compared to traditional datasets, where the value consists of [input quantity, label quantity], augmented dataset 1 includes an additional scoring term. During training, this scoring term serves as the weight for each sample and contributes to the model parameter update process. In this embodiment, if r = 0, when the sample does not match the regulation, then the sample has no impact on the actual parameter update. If r = 1, then the sample does affect the model parameter update.

[0190] In addition, in one possible implementation, the expert system may be a parameterized neural network model (i.e., a neural network, which may be referred to as a second neural network model in the embodiments of the present application). The parameterized neural network model may be trained to have the same or similar data processing capabilities as the pre-parameterized expert system.

[0191] Referring to Figure 8 , in this embodiment, an offline training model m (i.e., the first neural network model) can be used. Given an input x, the expert system receives the output u of the offline training model m and obtains the corresponding expert system output a, resulting in augmented dataset 2 [x, u, a]. Augmented dataset 2 contains the expert system characteristics under the offline training model m. Based on augmented dataset 2, with (x, u) as input and a as the label for training, a parameterized model f (i.e., the second neural network model) of the expert system can be obtained. In this case, parameterized model f has better generalization performance under the offline training model m.

[0192] For example, let x be the road condition information obtained during interaction with the regulatory control system, and u be the output of the first neural network model given x. Both are then connected to the expert system to obtain the expert system output a. After sufficient rounds of interaction, a regulatory characteristic dataset Dp = [x, u, a] is obtained. Based on dataset Dp, a neural network model a = f(x, u; w) of a rule-based expert system can be trained, where f is the expert system parameterized model (i.e., the second neural network model in this embodiment) and w is the model parameter.

[0193] In the case where the expert system is not a neural network, the reward value corresponding to the output of the expert system cannot be directly used to determine the update gradient of the first neural network model (the reason is that the expert system is a non-neural network and cannot perform gradient backpropagation, and the expert system based on the policy algorithm often has a large number of state machine transitions, random sampling, optimization and other modules. If the characteristics of the expert system are learned through random exploration, it will be very slow, which greatly increases the difficulty of online learning interaction). However, if the expert system is parameterized during the training process, the output of the expert system can be used to determine the reward value. When the gradient is backpropagated, it can be passed to the first neural network model. On the one hand, the update gradient corresponding to the first neural network model can be obtained, and the update gradient is obtained based on the output of the expert system implemented based on the neural network model. This is equivalent to introducing the output of the expert system implemented based on the neural network model into the training process of the first neural network model, thereby improving the processing accuracy of the trained model.

[0194] In one possible implementation, the expert system is a second neural network model; based on road condition information and target information, the second neural network model can determine control instructions for the target vehicle; based on the control instructions, the target vehicle is controlled to interact with the environment surrounding the target vehicle, obtaining interaction results, and the first neural network model is updated based on the interaction results. Exemplarily, a reward model (second reward model) can be designed to determine a reward value based on the interaction results, determine an update gradient for the first neural network based on the reward value, and use the update gradient of the first neural network model to update the first neural network model.

[0195] Optionally, the second neural network model may be updated according to the control instruction and the label corresponding to the control instruction, where the label corresponding to the control instruction is obtained by processing the road condition information and the target information through a rule-based expert system.

[0196] Regarding the reward model that determines the reward value based on the interaction result, it can be based on the existing reinforcement learning implementation and will not be elaborated here.

[0197] For example, referring to Figure 9 , the regulatory control system includes an offline training model m (a first neural network model) and a parameterized regulatory control expert system model f (a second neural network model). This embodiment continuously optimizes these two models through online interaction. Specifically, m and f are introduced into the simulation system. Due to the presence of f, exploration of a can be performed directly, making exploration more efficient. The reward model (a second reward model) is introduced into the online exploration to obtain interaction data [x, (u, a), x', r, d]. Here, u and a represent the output of the first and second neural network models for given road condition information x. f can be continuously updated based on the data [u, a]. Where a represents the exploration data, m can be updated using traditional reinforcement learning based on [x, a, x', r, d]. x' represents the road condition information after the target vehicle interacts with the surrounding environment. r represents the score (e.g., reward value) obtained by the reward model based on the interaction between the target vehicle and the surrounding environment. d represents the completion flag for the task, such as task completion or aborted. This enables continuous online optimization of m and f until convergence.

[0198] It should be noted that, in some embodiments of the present application, there are multiple ways to determine the extent to which the first neural network model should be trained using the updated gradient. The following provides some termination conditions for terminating the training of the first neural network model, including but not limited to:

[0199] (1) The loss function reaches the preset threshold.

[0200] After configuring the loss function, a threshold value (e.g., 0.03) can be set for the loss function in advance. During the iterative training of the first neural network model, after each training, it is determined whether the value of the loss function obtained in the current round of training reaches the threshold value. If not, the training is continued. If the preset threshold value is reached, the training is terminated. The values ​​of the network parameters of the first neural network model determined by the current round of training are used as the network parameter values ​​of the finally trained first neural network model.

[0201] (2) The loss function begins to converge.

[0202] After configuring the loss function, the first neural network model can be iteratively trained. If the difference between the value of the loss function obtained in the current round of training and the value of the loss function obtained in the previous round of training is within a preset range (e.g., 0.01), the loss function is considered to have converged, and the training can be terminated. The values ​​of the network parameters of the first neural network model determined by the current round of training are used as the network parameter values ​​of the finally trained first neural network model.

[0203] (3) The training reaches the preset number of times.

[0204] In this way, the number of iterations for training the first neural network model can be pre-configured (e.g., 1000 times). After the loss function is configured, the first neural network model can be iteratively trained. After each round of training, the values ​​of the network parameters of the first neural network model corresponding to that round will be stored until the number of training iterations reaches a preset number. Afterwards, the first neural network models obtained in each round are verified using test data, and the value of the network parameter with the best performance is selected as the final value of the network parameter of the first neural network model.

[0205] 2. Reasoning Stage

[0206] In the embodiment of the present application, the inference stage is the process by which the execution device 510 in FIG3 performs vehicle control using the trained target model 501. Specifically, during actual inference, the execution device 510 can obtain road condition information of the target vehicle and, through the updated first neural network model obtained in the embodiment corresponding to FIG5 , obtain target information, which is a prediction of the target vehicle's driving intention, a prediction of the driving path, or a prediction of the interaction between the target vehicle and the environment. Then, based on the road condition information and the target information, the expert system determines the control instructions for the target vehicle.

[0207] Next, a model training device provided by an embodiment of the present application will be described from the perspective of an apparatus. The model training device may be the aforementioned training device 520. Referring to FIG. 10A , FIG. 10A is a schematic structural diagram of a model training device provided by an embodiment of the present application. As shown in FIG. 10A , a model training device 1000 provided by an embodiment of the present application includes:

[0208] Acquisition module 1001, used to obtain road condition information of the target vehicle;

[0209] For a detailed introduction to the acquisition module 1001 , please refer to the description of step 501 in the above embodiment, which will not be repeated here.

[0210] Processing module 1002 is used to obtain target information based on the road condition information through a first neural network model, where the target information is a prediction of the target vehicle's driving intention, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment;

[0211] For a detailed introduction to the processing module 1002 , reference may be made to the description of step 502 in the above embodiment, which will not be repeated here.

[0212] The updating module 1003 is used to update the first neural network model according to the target information through the expert system or the result obtained by the expert system processing the road condition information and the label corresponding to the road condition information.

[0213] For a detailed introduction to the update module 1003 , please refer to the description of step 503 in the above embodiment, which will not be repeated here.

[0214] In a possible implementation, the update module 1003 is specifically configured to:

[0215] Determine the loss based on the target information and the label corresponding to the road condition information; adjust the loss through feasibility to obtain an adjusted loss, where the feasibility is obtained based on the result obtained by the expert system processing the road condition information and the label corresponding to the road condition information; and update the first neural network model based on the adjusted loss.

[0216] In one possible implementation, the feasibility indicates whether the tag is feasible or not.

[0217] In one possible implementation, the feasibility is a feasibility score; the updating module 1003 is specifically configured to:

[0218] The feasibility score is used as a weight to adjust the loss.

[0219] In a possible implementation, the feasibility is related to the following information:

[0220] The safety or comfort of the target vehicle when driving control of the target vehicle is performed according to the tag.

[0221] The expert system is a second neural network model; the updating module 1003 is specifically used to:

[0222] Based on the road condition information and the target information, the control instructions of the target vehicle are determined through the second neural network model; the target vehicle is controlled to interact with the environment around the target vehicle according to the control instructions to determine the interaction results; and the first neural network model is updated according to the interaction results.

[0223] In one possible implementation, the update module 1003 is further used to update the second neural network model according to the control instruction and the label corresponding to the control instruction, where the label corresponding to the control instruction is obtained by processing the road condition information and the target information through the rule-based expert system.

[0224] Next, a vehicle control device provided in an embodiment of the present application will be described from the perspective of a device. The vehicle control device may be the aforementioned execution device 510. Referring to FIG. 10B , which is a schematic structural diagram of a vehicle control device provided in an embodiment of the present application, as shown in FIG. 10B , a vehicle control device 1010 provided in an embodiment of the present application includes:

[0225] Acquisition module 1011, used to obtain road condition information of the target vehicle;

[0226] Processing module 1012 is configured to obtain target information based on the road condition information by obtaining an updated first neural network model according to the embodiment corresponding to FIG. 5 , where the target information is a prediction of the target vehicle's driving intention, a prediction of the driving path, or a prediction of the interaction between the target vehicle and the environment;

[0227] And according to the road condition information and the target information, the control instructions of the target vehicle are determined through the expert system.

[0228] Next, a terminal device 1100 provided in an embodiment of the present application is introduced. Please refer to Figure 11. Figure 11 is a structural schematic diagram of a terminal device provided in an embodiment of the present application. The terminal device 1100 can be specifically manifested as a mobile phone, a tablet, a laptop computer, a smart wearable device, a smart car, an in-vehicle computing platform, an in-vehicle domain controller, an in-vehicle terminal, etc., which are not limited here. Among them, the terminal device 1100 implements the function of the execution device 510. Specifically, the terminal device 1100 includes: a receiver 1101, a transmitter 1102, a processor 1103 and a memory 1104 (wherein the number of processors 1103 in the terminal device 1100 can be one or more), wherein the processor 1103 may include an application processor 11031 and a communication processor 11032. In some embodiments of the present application, the receiver 1101, the transmitter 1102, the processor 1103 and the memory 1104 may be connected via a bus or other means.

[0229] The memory 1104 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1103. A portion of the memory 1104 may also include non-volatile random access memory (NVRAM). The memory 1104 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0230] Processor 1103 controls the operation of the terminal device. In specific applications, the various components of the terminal device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0231] The method disclosed in the above embodiment of the present application can be applied to the processor 1103, or implemented by the processor 1103. The processor 1103 can be an integrated circuit chip with signal processing capabilities. During the implementation process, the steps of the above method can be completed by the integrated logic circuit of the hardware in the processor 1103 or the instructions in the form of software. The above-mentioned processor 1103 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and a vision processor (vision processing unit, VPU), a tensor processor (tensor processing unit, TPU) and other processors suitable for AI computing, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1103 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of this application can be directly implemented as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1104, and processor 1103 reads the information in memory 1104 and, in conjunction with its hardware, completes steps 501 to 503 in the above embodiment.

[0232] Receiver 1101 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the terminal device. Transmitter 1102 can be used to output digital or character information through the first interface. Transmitter 1102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group. Transmitter 1102 can also include a display device such as a display screen.

[0233] The present application also provides a server, which may be the aforementioned training device 520. See FIG12 , which is a schematic diagram of a server structure provided by an embodiment of the present application. Specifically, server 1200 is implemented by one or more servers. Server 1200 may vary significantly due to configuration or performance differences. It may include one or more central processing units (CPUs) 1212 (e.g., one or more processors), memory 1232, and one or more storage media 1230 (e.g., one or more mass storage devices) storing applications 1242 or data 1244. The memory 1232 and storage media 1230 may be either ephemeral or persistent storage. The program stored in storage media 1230 may include one or more modules (not shown), each of which may include a series of instructions for operating on the server. Furthermore, CPU 1212 may be configured to communicate with storage media 1230, executing the series of instructions stored in storage media 1230 on server 1200.

[0234] The server 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input and output interfaces 1258; or one or more operating systems 1241, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0235] Specifically, the server may perform steps 501 to 503 in the above embodiment.

[0236] An embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0237] A computer-readable storage medium is also provided in an embodiment of the present application, which stores a program for signal processing. When the computer-readable storage medium is run on a computer, it enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0238] An embodiment of the present application also provides a vehicle, which includes a sensor and the above-mentioned execution device 510 or vehicle control device 1010 or terminal device 1100; the sensor is used to collect road condition information of the vehicle.

[0239] The execution device, training device or terminal device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit, so that the chip executes the method of the aforementioned embodiment executed by the aforementioned execution device, or so that the chip executes the method of the aforementioned embodiment executed by the aforementioned training device. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0240] Specifically, see Figure 13, which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip can be represented as a neural network processor NPU 1300. NPU 1300 is mounted on the host CPU (host CPU) as a coprocessor and is assigned tasks by the host CPU. The core of the NPU is the arithmetic circuit 1303, which is controlled by the controller 1304 to extract matrix data from the memory and perform multiplication operations.

[0241] The NPU 1300 can implement the model training method provided in the embodiment described in FIG5 through the mutual cooperation between various internal components.

[0242] More specifically, in some implementations, the arithmetic circuit 1303 in the NPU 1300 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 1303 is a two-dimensional systolic array. The arithmetic circuit 1303 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1303 is a general-purpose matrix processor.

[0243] For example, assume there are input matrix A, weight matrix B, and output matrix C. The computation circuit retrieves the corresponding data of matrix B from weight memory 1302 and caches it on each PE in the computation circuit. The computation circuit then retrieves the data of matrix A from input memory 1301 and performs a matrix operation on it with matrix B. The partial or final matrix result is stored in accumulator 1308.

[0244] Unified memory 1306 is used to store input and output data. Weight data is directly transferred to weight memory 1302 through the Direct Memory Access Controller (DMAC) 1305. Input data is also transferred to unified memory 1306 through the DMAC.

[0245] BIU stands for Bus Interface Unit 1310 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 1309 .

[0246] The bus interface unit 1310 (BIU) is used for the instruction fetch memory 1309 to obtain instructions from the external memory, and is also used for the storage unit access controller 1305 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0247] DMAC is mainly used to move input data in the external memory DDR to the unified memory 1306 or to move weight data to the weight memory 1302 or to move input data to the input memory 1301.

[0248] The vector calculation unit 1307 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit 1303, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0249] In some implementations, the vector calculation unit 1307 can store the processed output vector in the unified memory 1306. For example, the vector calculation unit 1307 can apply a linear function or a nonlinear function to the output of the operation circuit 1303, such as linear interpolation of the feature plane extracted by the convolution layer, or accumulate a vector of values ​​to generate an activation value. In some implementations, the vector calculation unit 1307 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1303, for example, for use in subsequent layers in a neural network.

[0250] An instruction fetch buffer 1309 connected to the controller 1304 is used to store instructions used by the controller 1304;

[0251] Unified memory 1306, input memory 1301, weight memory 1302, and instruction fetch memory 1309 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0252] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above program.

[0253] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0254] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0255] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0256] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A model training method, characterized in that: The method comprises: Obtain the road condition information of the target vehicle; According to the road condition information, target information is obtained through a first neural network model, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment; According to the target information, the first neural network model is updated through an expert system or a result obtained by the expert system processing the road condition information and the labels corresponding to the road condition information.

2. The method according to claim 1, characterized in that The updating of the first neural network model according to the target information by an expert system or a result obtained by the expert system processing the road condition information and the label corresponding to the road condition information includes: Determining a loss according to the target information and the label; adjusting the loss by a feasibility to obtain an adjusted loss, wherein the feasibility is obtained based on the result; The first neural network model is updated according to the adjusted loss.

3. The method according to claim 2, characterized in that The feasibility is a feasibility score; The adjusting the loss by the feasibility comprises: The feasibility score is used as a weight to adjust the loss.

4. The method according to claim 2 or 3, characterized in that: The feasibility is related to the following information: When the target vehicle is controlled to travel according to the target information, the safety or comfort of the target vehicle.

5. The method according to claim 1, characterized in that The expert system is a second neural network model, and the first neural network model is updated according to the target information by the expert system or the result obtained by the expert system processing the road condition information and the label corresponding to the road condition information, including: Determining a control instruction of the target vehicle through the second neural network model according to the road condition information and the target information; Controlling the target vehicle to interact with the environment around the vehicle according to the control instruction, and determining an interaction result; According to the interaction result, the first neural network model is updated.

6. The method according to claim 5, characterized in that The method further comprises: The second neural network model is updated according to the control instruction and the label corresponding to the control instruction, wherein the label corresponding to the control instruction is obtained by processing the road condition information and the target information through an expert system based on a rule.

7. A vehicle control method, characterized in that: The method comprises: Obtain the road condition information of the target vehicle; According to the road condition information, target information is obtained by using the updated first neural network model obtained by any method of claims 1 to 6, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment; According to the road condition information and the target information, a control instruction of the target vehicle is determined through an expert system.

8. A model training device, characterized in that: The device comprises: An acquisition module is used to obtain the road condition information of the target vehicle; A processing module, configured to obtain target information through a first neural network model according to the road condition information, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interactive behavior between the target vehicle and the environment; An updating module is used to update the first neural network model according to the target information through an expert system or a result obtained by the expert system processing the road condition information and the labels corresponding to the road condition information.

9. The device according to claim 8, characterized in that The update module is specifically used for: Determining a loss according to the target information and the label; adjusting the loss by a feasibility to obtain an adjusted loss, wherein the feasibility is obtained based on the result; The first neural network model is updated according to the adjusted loss.

10. The device according to claim 9, characterized in that The feasibility is a feasibility score; The update module is specifically used for: The feasibility score is used as a weight to adjust the loss.

11. The device according to claim 9 or 10, characterized in that The feasibility is related to the following information: When the target vehicle is controlled to travel according to the target information, the safety or comfort of the target vehicle.

12. The device according to claim 8, characterized in that The expert system is a second neural network model; the updating module is specifically used for: According to the road condition information and the target information, the control instructions of the target vehicle are determined through the second neural network model; according to the control instructions, the target vehicle and the environment around the target vehicle are controlled to interact and the interaction results are determined; according to the interaction results, the first neural network model is updated.

13. The device according to claim 12, characterized in that The updating module is also used to update the second neural network model according to the control instruction and the label corresponding to the control instruction, wherein the label corresponding to the control instruction is obtained by processing the road condition information and the target information through an expert system based on rules.

14. A vehicle control device, characterized in that: The device comprises: An acquisition module is used to obtain the road condition information of the target vehicle; A processing module, configured to obtain target information according to the road condition information by using the updated first neural network model obtained by any one of the methods described in claims 1 to 6, wherein the target information is a prediction of the driving intention of the target vehicle, a prediction of the driving path, or a prediction of the interaction behavior between the target vehicle and the environment; According to the road condition information and the target information, a control instruction of the target vehicle is determined through an expert system.

15. A computing device, characterized in that: The computing device includes a memory and a processor; the memory stores codes, and the processor is configured to obtain the codes and execute the method according to any one of claims 1 to 7.

16. A vehicle, characterized in that: comprising a sensor, and a vehicle control device as claimed in claim 14; The sensor is used to collect road condition information of the vehicle.

17. A computer storage medium, characterized in that: The computer storage medium stores one or more instructions, which, when executed by one or more computers, enable the one or more computers to implement the method of any one of claims 1 to 7.

18. A computer program product comprising code, characterized in that When the code is executed, it is used to implement the method according to any one of claims 1 to 7.

19. A chip, comprising a processor, characterized in that: The processor is configured to execute the method according to any one of claims 1 to 7.

Images