ChatMPC-based automatic driving method, apparatus and device, and storage medium

Abstract

The embodiment of the invention provides an automatic driving method and device based on ChatMPC, equipment and a storage medium. The method comprises the following steps: combining an intention extractor with a vehicle driving MPC controller to obtain ChatMPC; a driving operation natural language instruction of a driver is input into ChatMPC and processed by an intention extractor, an update mark for representing a driving intention is output, then an objective function and input of a vehicle driving MPC controller are updated according to the update mark, and the vehicle driving MPC controller solves an optimal control sequence based on the update mark, so that the driving intention of the driver is determined. And extracting a first control value in the solved control sequence as a control input at the current moment, applying the first control value to actual driving of the vehicle, updating the state of the vehicle and calculating a control input at the next moment after each iteration of the vehicle driving MPC controller, and continuously iterating until the task is completed. In this way, the automatic driving strategy can be flexibly adjusted according to the subjective intention of the driver.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and in particular, to an autonomous driving method, device, equipment and storage medium based on ChatMPC. Background Art

[0002] With the rapid development of autonomous driving technology, many studies have been dedicated to improving the decision-making ability and adaptability of autonomous driving systems. Autonomous driving systems usually rely on fixed rules and predetermined strategies for decision-making, which may not be able to adapt to complex and dynamic environments in some cases, especially when facing complex traffic scenarios (such as overtaking, following, emergency obstacle avoidance, etc.). Therefore, how to adjust driving strategies in a more intelligent and flexible way to meet the needs of drivers has become an important direction in autonomous driving research. Summary of the Invention

[0003] In a first aspect, an embodiment of the present invention provides an autonomous driving method based on ChatMPC, the method comprising:

[0004] Construct an intention extractor, wherein the intention extractor includes a sentence BERT and an embedding classifier;

[0005] Input a training set of natural language instructions for driving operations including various contexts, tones, and sentence patterns into the intention extractor to train the embedding classifier therein; in the intention extractor, the sentence BERT performs embedding processing on the natural language instructions for driving operations to obtain its corresponding embedding vector, and then inputs the embedding vector into the embedding classifier, and the embedding classifier outputs an updated tag for representing the driving intention;

[0006] Construct a vehicle kinematic model and, based on this, construct a vehicle driving model predictive control (MPC) controller;

[0007] Combine the intention extractor with the vehicle driving MPC controller to obtain a large language model (ChatMPC);

[0008] The natural language instructions of the driver's driving operations are input into ChatMPC. The Sentence BERT in the intent extractor embeds the natural language instructions of the driving operations to obtain their corresponding embedding vectors. Then, the embedding vectors are input into the embedding classifier, and the embedding classifier outputs an updated tag for representing the driving intent. After that, the objective function and input of the vehicle driving MPC controller are updated according to the updated tag, and the vehicle driving MPC controller solves the optimal control sequence based on this, extracts the first control value in the solved control sequence as the control input at the current moment, and applies it to the actual driving of the vehicle. After each iteration of the vehicle driving MPC controller, the state of the vehicle is updated and the control input for the next moment is calculated, and the iteration continues until the task is completed.

[0009] In some realizable ways of the first aspect, the Sentence BERT includes a tokenizer, a BERT model, and a mean pooling layer.

[0010] In some realizable ways of the first aspect, the Sentence BERT embeds the natural language instructions of the driving operations to obtain their corresponding embedding vectors, including:

[0011] For any natural language instruction of the driving operations, the tokenizer tokenizes the natural language instruction of the driving operations to obtain multiple tokens, and inputs the multiple tokens into the Bert model. The Bert model embeds the multiple tokens and outputs the embedding vectors corresponding to the multiple tokens. Then, the mean pooling layer averages the embedding vectors corresponding to the multiple tokens to obtain a single embedding vector representing the natural language instruction of the driving operations.

[0012] In some realizable ways of the first aspect, a vehicle kinematic model is constructed, and a vehicle driving MPC controller is constructed based on this, including:

[0013] Model the vehicle kinematic models of the host vehicle and the leading vehicle to obtain the vehicle kinematic models of the host vehicle and the leading vehicle;

[0014] Construct a vehicle driving MPC controller according to the vehicle kinematic models of the host vehicle and the leading vehicle.

[0015] In some realizable ways of the first aspect, modeling the vehicle kinematic models of the host vehicle and the leading vehicle to obtain the vehicle kinematic models of the host vehicle and the leading vehicle, including:

[0016] Use the bicycle model to model the vehicle kinematic models of the host vehicle and the leading vehicle respectively to obtain the vehicle kinematic models of the host vehicle and the leading vehicle, which are the control objects of the driving strategy, and initialize the vehicle parameters.

[0017] In some realizable ways of the first aspect, constructing a vehicle driving MPC controller according to the vehicle kinematic models of the host vehicle and the leading vehicle, including:

[0018] According to the vehicle kinematic models of the host vehicle and the leading vehicle, a vehicle driving MPC optimization problem is constructed. Specifically, the constraint conditions for the vehicle driving MPC optimization problem include constraints on the vehicle's speed, acceleration, yaw angle, and the distance between the host vehicle and the leading vehicle. The optimization objective of the vehicle driving MPC optimization problem is set to minimize the objective function composed of the process state cost, the input cost, and the terminal state cost. Then, a vehicle driving MPC controller is constructed based on the vehicle driving MPC optimization problem.

[0019] In a second aspect, an embodiment of the present invention provides an autonomous driving device based on ChatMPC. The device includes:

[0020] A construction module, configured to construct an intention extractor, where the intention extractor includes a sentence BERT and an embedding classifier;

[0021] A training module, configured to input a training set of natural language driving operation instructions including various contexts, tones, and sentence patterns into the intention extractor to train the embedding classifier therein. In the intention extractor, the sentence BERT performs embedding processing on the natural language driving operation instructions to obtain their corresponding embedding vectors, and then the embedding vectors are input into the embedding classifier, and the embedding classifier outputs an updated tag for representing the driving intention;

[0022] The construction module is further configured to construct a vehicle kinematic model and construct a vehicle driving MPC controller based on this;

[0023] A combination module, configured to combine the intention extractor with the vehicle driving MPC controller to obtain ChatMPC;

[0024] A control module, configured to input the natural language driving operation instructions of the driver into ChatMPC. The sentence BERT in the intention extractor performs embedding processing on the natural language driving operation instructions to obtain their corresponding embedding vectors, and then the embedding vectors are input into the embedding classifier, and the embedding classifier outputs an updated tag for representing the driving intention. Then, the objective function and the input of the vehicle driving MPC controller are updated according to the updated tag, and the vehicle driving MPC controller solves the optimal control sequence based on this, extracts the first control value in the solved control sequence as the control input at the current moment, and applies it to the actual vehicle driving. After each iteration of the vehicle driving MPC controller, the state of the vehicle is updated and the control input for the next moment is calculated, and the iteration continues until the task is completed.

[0025] In a third aspect, an embodiment of the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method as described above.

[0026] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method as described above.

[0027] Compared with the prior art, the present invention has at least the following technical effects:

[0028] The method, device, equipment and storage medium for autonomous driving based on ChatMPC proposed by the present invention extract the driving intentions related to the autonomous driving strategy in the natural language instructions of driving operations in different contexts, tones and sentence patterns, and accurately convey the driving intentions to the vehicle driving MPC controller, so as to achieve the purpose of updating the autonomous driving strategy in real time according to the subjective wishes of the driver, and realize a more flexible and operable autonomous driving mode. In this way, the present invention not only improves the degree of dynamic interaction and matching between the autonomous driving strategy and the driver, but also makes the autonomous driving more adaptable to the driver's personal habits and real-time road conditions, thereby enhancing the flexibility and response ability of the autonomous driving system, improving the safety and comfort of driving, and making the driving experience more user-friendly and intelligent.

[0029] It should be understood that the content described in the summary of the invention is not intended to limit the key or important features of the embodiments of the present invention, nor to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] With reference to the accompanying drawings and the following detailed description, the above and other features, advantages and aspects of the embodiments of the present invention will become more obvious. The drawings are used to better understand the present invention and do not constitute a limitation to the present invention. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0031] Figure 1 is a flowchart of a method for autonomous driving based on ChatMPC provided by an embodiment of the present invention;

[0032] Figure 2 is a structural diagram of an intention extractor provided by an embodiment of the present invention;

[0033] Figure 3 is a flowchart of another method for autonomous driving based on ChatMPC provided by an embodiment of the present invention;

[0034] Figure 4 Schematic diagram of parameter update provided by an embodiment of the present invention;

[0035] Figure 5 Structural diagram of an autonomous driving device based on ChatMPC provided by an embodiment of the present invention;

[0036] Figure 6 Structural diagram of an exemplary electronic device capable of implementing the embodiments of the present invention. Detailed implementation manners

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] In addition, the term "and / or" in the present invention is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present invention generally represents an "or" relationship between the associated objects before and after.

[0039] To solve the technical problems in the background art, an embodiment of the present invention provides an autonomous driving method, device, equipment, and storage medium based on ChatMPC. By extracting the driving intention related to the autonomous driving strategy in the natural language instructions of driving operations in different contexts, tones, and sentence patterns, the driving intention is accurately transmitted to the vehicle driving MPC controller, so as to achieve the purpose of updating the autonomous driving strategy in real time according to the subjective will of the driver, and realize a more flexible and more operable autonomous driving mode. In this way, the present invention not only improves the dynamic interaction and matching degree between the autonomous driving strategy and the driver, but also makes the autonomous driving more adaptable to the driver's personal habits and real-time road conditions, thereby enhancing the flexibility and response ability of the autonomous driving system, improving the safety and comfort of driving, and making the driving experience more user-friendly and intelligent.

[0040] The following will specifically describe an autonomous driving method, device, equipment, and storage medium based on ChatMPC provided by an embodiment of the present invention with reference to the accompanying drawings.

[0041] Figure 1 Flowchart of an autonomous driving method based on ChatMPC provided by an embodiment of the present invention, as Figure 1 shown, the autonomous driving method 100 may include:

[0042] S110, construct an intent extractor.

[0043] Among them, the intent extractor can be as Figure 2 shown, including Sentence BERT (i.e., Sentence Bert) and an embedding classifier (i.e., Embedding Classifier). Further, Sentence BERT can include a tokenizer (i.e., Tokenizer), a BERT model (i.e., BERT Model), and a mean pooling layer (i.e., MEAN Pooling).

[0044] S120, input a training set of natural language driving operation instructions including various contexts, tones, and sentence patterns into the intent extractor to train the embedding classifier therein.

[0045] In the intent extractor, for any natural language driving operation instruction, the tokenizer tokenizes the natural language driving operation instruction to obtain multiple tokens, and inputs the multiple tokens into the Bert model. The Bert model performs embedding processing on the multiple tokens and outputs the embedding vectors (i.e., Token Embeddings) corresponding to the multiple tokens. Then, the mean pooling layer performs averaging processing on the embedding vectors corresponding to the multiple tokens to obtain a single embedding vector (i.e., Embedding) e representing the natural language driving operation instruction. Then, the embedding vector is input into the embedding classifier, and the embedding classifier outputs an update marker (i.e., Update Marker) s for representing the driving intent.

[0046] S130, construct a vehicle kinematic model and construct a vehicle driving MPC controller based on this.

[0047] Exemplarily, a bicycle model can be used to model the vehicle kinematic models of the host vehicle and the leading vehicle respectively to obtain the vehicle kinematic models of the host vehicle and the leading vehicle, which are used as the control objects of the driving strategy, and the vehicle parameters are initialized. Then, according to the vehicle kinematic models of the host vehicle and the leading vehicle, a vehicle driving MPC optimization problem is constructed. Specifically, the constraint conditions of the vehicle driving MPC optimization problem include the constraints on the speed, acceleration, and yaw angle of the vehicle and the distance constraint between the host vehicle and the leading vehicle. The optimization objective of the vehicle driving MPC optimization problem is set to minimize the objective function composed of the process state cost, the input cost, and the terminal state cost. Then, a vehicle driving MPC controller is constructed based on the vehicle driving MPC optimization problem.

[0048] S140, combine the intent extractor with the vehicle driving MPC controller to obtain ChatMPC.

[0049] S150. Input the natural language instructions of the driver's driving operations into ChatMPC. The sentence BERT in the intent extractor performs embedding processing on the natural language instructions of the driving operations to obtain its corresponding embedding vector. Then, the embedding vector is input into the embedding classifier, and the embedding classifier outputs an updated tag for representing the driving intent. After that, update the objective function and input of the vehicle driving MPC controller according to the updated tag. And the vehicle driving MPC controller solves the optimal control sequence based on this, extracts the first control value in the solved control sequence as the control input at the current moment, and applies it to the actual driving of the vehicle. After each iteration of the vehicle driving MPC controller, update the state of the vehicle and calculate the control input for the next moment, and continue to iterate until the task is completed.

[0050] For the convenience of further understanding, the above content will be described in detail below in combination with specific embodiments:

[0051] Refer to Figures 3 - 4 , this embodiment provides an autonomous driving method based on ChatMPC. This method combines natural language processing (NLP) with vehicle kinematic modeling. By extracting the driving intent in natural language instructions, it dynamically adjusts the objective function and input of the vehicle driving MPC controller to achieve the optimization of intelligent decision-making and control in autonomous driving. This method not only makes up for the deficiencies in the accuracy and generalization of past driving intent extraction but also significantly improves the flexibility and dynamics of model predictive control.

[0052] First, according to Figure 3 Define the system model:

[0053] P: x(k + 1) = f(x(k), u(k)), z(k) = g(x(k), u(k));

[0054]

[0055] Figure 4 Shows Figure 3 the structure of interpreter A in int which contains an intent extraction function of f up and a parameter update f τ function. Assume that the instruction p contains τ commands, that is, p = {p1, p2,..., p τ}. Update the parameter θ τ according to the instruction p τ . This update rule can be pre-trained for several given types of instructions and is represented by s

[0056] On this basis, the autonomous driving method based on ChatMPC may include the following steps:

[0057] (1) Construct an intent extractor, including a sentence BERT and an embedding classifier. Further, the sentence BERT may include a tokenizer, a BERT model, and a mean pooling layer.

[0058] (2) Input a training set of natural language driving operation instructions including various contexts, tones, and sentence patterns into the intent extractor to train the embedding classifier therein. Each natural language driving operation instruction is processed by the tokenizer to decompose the sentence into several tokens. Then, the tokens are sent into the Bert model for embedding processing, and the model outputs the embedding vectors of each token. Next, the mean pooling layer is used to average the embedding vectors of all tokens to obtain a single embedding vector e representing the entire input instruction. This embedding vector e will be used as a feature to input into the embedding classifier, and the embedding classifier outputs an updated label s according to the input feature vector e. Exemplarily, here s = 1 indicates the intent of overtaking, and s = 0 represents maintaining the current driving strategy. Through continuous optimization, the embedding classifier can accurately identify the driving intent in the instruction, thereby providing a basis for subsequent control decisions.

[0059] (3) Use the bicycle model to respectively model the kinematic models of the host vehicle and the leading vehicle to obtain the kinematic models of the host vehicle and the leading vehicle, which are used as the control objects of the driving strategy, and initialize the vehicle parameters. Assume that the left and right tires of the vehicle have the same steering angle and rotational speed at any time; assume that the movement and steering of the vehicle are front-wheel-only driven, and the sideslip angle is 0; assume that the velocity vectors of the front and rear wheels of the vehicle are consistent with the rotational directions of the front and rear wheels, that is, it can be considered that the lateral velocity of the front and rear wheels in the tire coordinate system is zero, and the wheels have no sideslip, that is, the rear steering angle is 0. Here, x and y are used to represent the coordinates of the vehicle's center of mass, θ represents the yaw angle, V represents the vehicle speed, a represents the acceleration, and δ represents the steering angle. On this basis, set the state variables of the host vehicle as [x1, y1, θ1, V1] T , and the model input is [a1, δ1] T , and initially move in a uniform straight line along the positive X-axis at a speed of 10 m / s. Assume that all inputs of the leading vehicle are known and determined, and also make it move in a uniform straight line along the positive X-axis and always travel in the center of the lane (the single-lane width is 3.75 m). Let: V2 = 10 (m / s), a2 = 0 (m / s 2 ), δ2 = 0 (rad), θ2 = 0 (rad), then the state variables are [x2, 1.875, 0, 10] T , and the model input is [0, 0] T。The vehicle is 5m long, 2m wide, with an initial vehicle distance of 15m. After overtaking, the distance between the host vehicle and the following vehicle is also 15m.

[0060] (4) Based on the vehicle kinematic models of the host vehicle and the leading vehicle, a vehicle driving MPC optimization problem is constructed. In the present invention, the goal of the vehicle driving MPC optimization problem is to ensure that the vehicle can efficiently overtake or follow the vehicle while adhering to traffic rules and driving safety. To achieve this goal, the following constraint conditions are first set: the speed, acceleration, steering angle, and yaw angle of the vehicle are limited within a certain range. Secondly, the minimum safety distance between the host vehicle and the leading vehicle is ensured to prevent collisions during overtaking. According to the vehicle kinematic model, the process state cost, input cost, and terminal state cost are calculated, and these cost functions are combined to construct the objective function of the vehicle driving MPC optimization problem for optimization. The form of the objective function is: l f (x N )=(x N ―x ref ) T Q N (x N ―x ref ), l(x k ,u k )=(x k ―x ref ) T Q k (x k ―x ref )+u k T R k u k , where x ref is the reference value of the state variable, and Q N , Q k are weighting coefficients.

[0061] (5) Based on the vehicle driving MPC optimization problem, a vehicle driving MPC controller is constructed.

[0062] (6) Combine the intention extractor with the vehicle driving MPC controller to obtain ChatMPC.

[0063] (7) Input the driver's natural language driving operation instruction, such as "I want to overtake the vehicle in front", into ChatMPC. The sentence BERT in the intent extractor performs embedding processing on the natural language driving operation instruction to obtain its corresponding embedding vector. Then, the embedding vector is input into the embedding classifier, and the embedding classifier outputs an updated tag for representing the driving intent of "overtaking". After that, the objective function and input of the vehicle driving MPC controller are updated according to the updated tag. At this time, the objective function of the vehicle driving MPC controller will be adjusted to minimize the distance between the current vehicle and the vehicle in front, and consider the vehicle distance to be maintained after overtaking. In this way, the MPC controller continuously optimizes the control strategy according to the overtaking intent and solves the optimal control sequence. In each iteration, the MPC controller updates parameters such as the position, speed, and acceleration of the vehicle according to the current control input and vehicle state until the error is less than the set value or the maximum number of iterations is reached. According to the control sequence solved by the MPC controller, the system outputs the first control value as the control input at the current moment and applies it to actual driving. After each iteration of the controller, the state of the vehicle is updated and the control input for the next moment is calculated, and the iteration continues until the task is completed. Through this real-time control method, the vehicle can intelligently complete the overtaking operation in a complex traffic environment, ensuring driving safety and efficiency.

[0064] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0065] The above is the introduction of the method embodiments. The following further illustrates the solution of the present invention through device embodiments.

[0066] Figure 5 The structure diagram of an autonomous driving device based on ChatMPC provided by an embodiment of the present invention is as Figure 5 shown. The autonomous driving device 500 may include:

[0067] A construction module 510, configured to construct an intent extractor, where the intent extractor includes a sentence BERT and an embedding classifier.

[0068] A training module 520 is configured to input a training set of natural language driving operation instructions including various contexts, tones, and sentence patterns to an intent extractor to train an embedding classifier therein. In the intent extractor, a sentence BERT processes a natural language driving operation instruction to obtain its corresponding embedding vector, and then inputs the embedding vector to the embedding classifier, which outputs an updated tag representing the driving intent.

[0069] A construction module 510 is further configured to construct a vehicle kinematic model and, based on this, construct a vehicle driving MPC controller.

[0070] An integration module 530 is configured to integrate the intent extractor with the vehicle driving MPC controller to obtain ChatMPC.

[0071] A control module 540 is configured to input a natural language driving operation instruction of a driver to ChatMPC. The sentence BERT in the intent extractor processes the natural language driving operation instruction to obtain its corresponding embedding vector, and then inputs the embedding vector to the embedding classifier, which outputs an updated tag representing the driving intent. Then, based on the updated tag, the target function and input of the vehicle driving MPC controller are updated, and the vehicle driving MPC controller solves an optimal control sequence based on this, extracts the first control value in the solved control sequence as the control input at the current moment, and applies it to the actual vehicle driving. After each iteration of the vehicle driving MPC controller, the state of the vehicle is updated and the control input for the next moment is calculated, and the iteration continues until the task is completed.

[0072] It can be understood that Figure 5 each module / unit in the shown autonomous driving device 500 has the function of implementing Figure 1 each step in the shown autonomous driving method 100 and can achieve its corresponding technical effects. For the sake of brevity, they are not described herein again.

[0073] Figure 6 It is a structural diagram of an exemplary electronic device capable of implementing the embodiments of the present invention. The electronic device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 600 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown in the present invention, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed in the present invention.

[0074] As Figure 6As shown, the electronic device 600 may include a computing unit 601, which may perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 may also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0075] Multiple components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disc, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0076] The computing unit 601 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as method 100. For example, in some embodiments, method 100 may be implemented as a computer program product, including a computer program, which is tangibly contained in a computer-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of method 100 described above may be executed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute method 100 in any other appropriate manner (e.g., by means of firmware).

[0077] The various embodiments described above in the present invention can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0078] The program code for implementing the methods of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0079] In the context of the present invention, a computer-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a computer-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0080] It should be noted that the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute method 100 and achieve the corresponding technical effects achieved by the method of the embodiments of the present invention. For the sake of brevity of description, details are not repeated herein.

[0081] In addition, the present invention also provides a computer program product, which includes a computer program that implements method 100 when executed by a processor.

[0082] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present invention can be achieved, and the present invention is not limited herein.

[0083] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An autonomous driving method based on ChatMPC, characterized in that, The method includes: Construct an intent extractor, where the intent extractor includes a sentence BERT and an embedding classifier; Input a training set of natural language instructions for driving operations including various contexts, tones, and sentence patterns into the intent extractor to train the embedding classifier therein; in the intent extractor, the sentence BERT performs embedding processing on the natural language instructions for driving operations to obtain their corresponding embedding vectors, and then inputs the embedding vectors into the embedding classifier, and the embedding classifier outputs an updated tag for representing the driving intent; Construct a vehicle kinematic model and construct a vehicle driving MPC controller based on this; Combine the intent extractor with the vehicle driving MPC controller to obtain ChatMPC; Input the natural language instructions for the driver's driving operations into ChatMPC. The sentence BERT in the intent extractor performs embedding processing on the natural language instructions for driving operations to obtain their corresponding embedding vectors, and then inputs the embedding vectors into the embedding classifier. The embedding classifier outputs an updated tag for representing the driving intent. Then, update the objective function and input of the vehicle driving MPC controller according to the updated tag. And the vehicle driving MPC controller solves the optimal control sequence based on this, extracts the first control value in the solved control sequence as the control input at the current moment, and applies it to the actual vehicle driving. After each iteration of the vehicle driving MPC controller, update the state of the vehicle and calculate the control input at the next moment, and continue to iterate until the task is completed.

2. The method according to claim 1, wherein The sentence BERT includes a tokenizer, a BERT model, and a mean pooling layer.

3. The method according to claim 2, wherein The sentence BERT performs embedding processing on the natural language instructions for driving operations to obtain their corresponding embedding vectors, including: For any natural language instruction for driving operations, the tokenizer tokenizes the natural language instruction for driving operations to obtain multiple tokens, and inputs the multiple tokens into the Bert model. The Bert model performs embedding processing on the multiple tokens and outputs the embedding vectors corresponding to the multiple tokens. Then, use the mean pooling layer to average the embedding vectors corresponding to the multiple tokens to obtain a single embedding vector representing the natural language instruction for driving operations.

4. The method according to claim 1, wherein The constructing the vehicle kinematic model and constructing the vehicle driving MPC controller based on this includes: Model the vehicle kinematic models of the host vehicle and the leading vehicle to obtain the vehicle kinematic models of the host vehicle and the leading vehicle; Construct a vehicle driving MPC controller according to the vehicle kinematic models of the host vehicle and the leading vehicle.

5. The method according to claim 4, characterized in that, The modeling the vehicle kinematic models of the host vehicle and the leading vehicle to obtain the vehicle kinematic models of the host vehicle and the leading vehicle includes: Use the bicycle model to model the vehicle kinematic models of the host vehicle and the leading vehicle respectively to obtain the vehicle kinematic models of the host vehicle and the leading vehicle, which are used as the control objects of the driving strategy, and initialize the vehicle parameters.

6. The method according to claim 4, wherein The constructing the vehicle driving MPC controller according to the vehicle kinematic models of the host vehicle and the leading vehicle includes: According to the vehicle kinematic models of the host vehicle and the leading vehicle, a vehicle driving MPC optimization problem is constructed. Specifically, the constraint conditions of the vehicle driving MPC optimization problem include the constraints on the vehicle's speed, acceleration, yaw angle, and the distance constraint between the host vehicle and the leading vehicle. The optimization objective of the vehicle driving MPC optimization problem is set to minimize the objective function composed of the process state cost, the input cost, and the terminal state cost; then a vehicle driving MPC controller is constructed based on the vehicle driving MPC optimization problem.

7. An autonomous driving device based on ChatMPC, characterized in that, The device includes: A construction module, configured to construct an intention extractor, where the intention extractor includes a sentence BERT and an embedding classifier; A training module, configured to input a training set of natural language instructions for driving operations including various contexts, tones, and sentence patterns into the intention extractor to train the embedding classifier therein; in the intention extractor, the sentence BERT performs embedding processing on the natural language instructions for driving operations to obtain their corresponding embedding vectors, and then inputs the embedding vectors into the embedding classifier, and the embedding classifier outputs an updated tag for representing the driving intention; The construction module is further configured to construct a vehicle kinematic model and construct a vehicle driving MPC controller based on this; A combination module, configured to combine the intention extractor with the vehicle driving MPC controller to obtain ChatMPC; A control module, configured to input the natural language instructions for the driver's driving operations into ChatMPC. The sentence BERT in the intention extractor performs embedding processing on the natural language instructions for driving operations to obtain their corresponding embedding vectors, and then inputs the embedding vectors into the embedding classifier. The embedding classifier outputs an updated tag for representing the driving intention. Then, according to the updated tag, the objective function and the input of the vehicle driving MPC controller are updated, and the vehicle driving MPC controller solves the optimal control sequence based on this, extracts the first control value in the solved control sequence as the control input at the current moment, and applies it to the actual vehicle driving. After each iteration of the vehicle driving MPC controller, the state of the vehicle is updated and the control input for the next moment is calculated, and the iteration continues until the task is completed.

8. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-6.

Images