Vehicle driving decision-making method and device, electronic equipment and storage medium

By converting the vehicle's current driving scenario information into text descriptions and using large language models and driving knowledge base decision-making, the problem of decision accuracy and universality of the autonomous driving system in long-tail scenarios is solved, and the continuous iteration and adaptability of the system is achieved. promote.

CN119058744APending Publication Date: 2024-12-03BEIJING JUEYING INTELLIGENT TECH CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411132127.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The autonomous driving system is difficult to deal with in long-tail scenarios, and the data-driven solution lacks universality and continuous iteration capabilities, resulting in the inability to gain public trust.

Method used

By converting the vehicle's current driving scenario information into text descriptions, using large language models and preset driving knowledge bases to make decisions, improving the accuracy and versatility of driving decisions, and improving the system's adaptability through continuous iterative updates.

Benefits of technology

It improves the decision accuracy and versatility of the autonomous driving system in long-tail scenarios, enhances the ability to predict driving behavior under complex road conditions, and ensures the continuous evolution and iteration of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119058744A_ABST
    Figure CN119058744A_ABST
Patent Text Reader

Abstract

The invention discloses a vehicle driving decision-making method and device, electronic equipment and a computer readable storage medium. The vehicle driving decision-making method comprises the following steps: performing text conversion on scene information for representing a current driving scene of a target vehicle to obtain a scene description text; based on the scene description text, target driving knowledge corresponding to the current driving scene is retrieved in a preset driving knowledge base, and the driving knowledge in the driving knowledge base comprises driving information related to driving decisions in historical driving scenes; and based on the scene description text and the target driving knowledge, a driving behavior of the target vehicle is decided by using a trained large language model, and driving decision information of the target vehicle is obtained. According to the invention, the scene information is converted to the language domain, so that driving knowledge can be added to automatic driving subsequently; the big language model makes a decision on the driving behavior of the target vehicle based on the scene description text and the target driving knowledge, and the accuracy of the vehicle driving decision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a vehicle driving decision-making method, device, electronic device and computer-readable storage medium. Background Art

[0002] Autonomous driving technology has made rapid progress, but it still faces huge challenges in long-tail scenarios. Current advanced autonomous driving systems often face challenges in terms of versatility. Data-driven solutions are constrained by data sets, and the lack of a sufficiently broad training set to cover various long-tail scenarios makes it difficult to deal with long-tail scenarios. In addition, due to its black box nature, data-driven solutions lack the ability to continuously evolve and iterate efficiently when encountering failure scenarios. Therefore, even with the current massive amount of driving data, autonomous driving algorithms still cannot gain public trust. Summary of the invention

[0003] The present application at least provides a vehicle driving decision method, device, electronic device and computer-readable storage medium.

[0004] The first aspect of the present application provides a driving decision method for a vehicle, comprising:

[0005] Performing text conversion on scene information used to represent a current driving scene of the target vehicle to obtain a scene description text;

[0006] Based on the scenario description text, target driving knowledge corresponding to the current driving scenario is retrieved from a preset driving knowledge base, wherein the driving knowledge in the driving knowledge base includes driving information related to driving decisions in historical driving scenarios;

[0007] Based on the scene description text and target driving knowledge, the trained large language model is used to make decisions on the target vehicle's driving behavior and obtain the target vehicle's driving decision information.

[0008] Therefore, in this embodiment, the scene information used to represent the current driving scene of the target vehicle is converted into a scene description text to convert the scene information into a language domain, so as to facilitate the subsequent addition of driving knowledge to the autonomous driving; the large language model makes decisions on the driving behavior of the target vehicle based on the scene description text and the target driving knowledge, and then determines the driving decision information of the target vehicle, thereby improving the accuracy of the vehicle's driving decision; and the large language model has an efficient and continuously evolving iterative capability, which improves the versatility of the vehicle's driving decision method.

[0009] A second aspect of the present application provides a driving decision-making device for a vehicle, the driving decision-making device for the vehicle comprising:

[0010] A conversion module, used for performing text conversion on the scene information used to represent the current driving scene of the target vehicle to obtain a scene description text;

[0011] A retrieval module is used to retrieve target driving knowledge corresponding to the current driving scenario from a preset driving knowledge base based on the scenario description text, wherein the driving knowledge in the driving knowledge base includes driving information related to driving decisions in historical driving scenarios;

[0012] The determination module is used to make decisions on the driving behavior of the target vehicle based on the scene description text and the target driving knowledge using a trained large language model to obtain the driving decision information of the target vehicle.

[0013] A third aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, the processor being used to execute program instructions stored in the memory to implement the steps in the above-mentioned vehicle driving decision method.

[0014] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the steps in the above-mentioned vehicle driving decision-making method when the program instructions are executed by a processor.

[0015] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0018] Figure 1 It is a flowchart of an embodiment of a driving decision-making method for a vehicle provided by the present application;

[0019] Figure 2 It is a schematic diagram of a framework of a specific embodiment of the vehicle driving decision-making method provided by the present application;

[0020] Figure 3 yes Figure 1 A flowchart of a specific implementation of step S2 in the vehicle driving decision method provided;

[0021] Figure 4 It is a schematic diagram of the framework structure of the large language model provided by this application;

[0022] Figure 5 yes Figure 1 A schematic flow chart of an embodiment of step S3 in a driving decision method for a vehicle is provided;

[0023] Figure 6 is a flow chart of another embodiment of the vehicle driving decision-making method provided by the present application;

[0024] Figure 7 yes Figure 6 A flow chart of an implementation of step S42 in the vehicle driving decision-making method process shown;

[0025] Figure 8 It is a schematic diagram of constructing a driving knowledge base provided by this application;

[0026] Fig. 9 It is a schematic diagram of a framework of an embodiment of a driving decision-making device for a vehicle provided by the present application;

[0027] Fig.10 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;

[0028] Fig.11 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0029] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0030] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0031] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0032] Some terms involved in the embodiments of the present application are introduced.

[0033] LLM large language models are artificial intelligence models designed to understand and generate human language. They are trained on large amounts of text data and can perform a large number of tasks related to analyzing text information, such as text summarization, translation, and sentiment analysis.

[0034] Long-tail scenarios mainly refer to a wide variety of scenarios with low probability of occurrence or sudden situations, such as vehicles running red lights, pedestrians crossing the road, intersections with damaged traffic lights, and vehicles illegally parked on the roadside.

[0035] The present embodiment provides a driving decision method for a vehicle, and the driving decision method for the vehicle can be implemented by a server or a terminal alone, or by a server and a terminal in collaboration. In some embodiments, the terminal or the server can implement the driving decision method for the vehicle provided by the embodiment of the present disclosure by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a client that supports virtual scenes, such as a game APP; it can also be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be an application, module or plug-in in any form.

[0036] See also Figure 1 and Figure 2 , Figure 1 It is a flowchart of an embodiment of a driving decision-making method for a vehicle provided by the present application; Figure 2 It is a schematic framework diagram of a specific embodiment of the vehicle driving decision-making method provided in the present application.

[0037] In the disclosed embodiment, a server is used as the execution entity and is applicable to autonomous driving vehicles, and is applicable to autonomous driving vehicles in various road conditions including urban road conditions and complex road conditions. The process of the driving decision method of the vehicle may include the following steps S1 to S3.

[0038] S1: Perform text conversion on scene information used to represent the current driving scene of the target vehicle to obtain a scene description text.

[0039] S2: Based on the scenario description text, target driving knowledge corresponding to the current driving scenario is retrieved from a preset driving knowledge base, wherein the driving knowledge in the driving knowledge base includes driving information related to driving decisions in historical driving scenarios.

[0040] S3: Based on the scene description text and target driving knowledge, the trained large language model is used to make decisions on the target vehicle’s driving behavior and obtain the target vehicle’s driving decision information.

[0041] The vehicle driving decision method provided in this embodiment converts the scene information used to represent the current driving scene of the target vehicle into a scene description text to convert the scene information into a language domain, so as to facilitate the subsequent addition of driving knowledge to the autonomous driving; the large language model makes decisions on the driving behavior of the target vehicle based on the scene description text and the target driving knowledge, and then determines the driving decision information of the target vehicle, thereby improving the accuracy of the vehicle driving decision; and the large language model has an efficient and continuously evolving iterative capability, thereby improving the versatility of the vehicle driving decision method.

[0042] In a disclosed embodiment, the specific implementation method of performing text conversion on the scene information used to represent the current driving scene of the target vehicle in step S1 to obtain the scene description text is as follows.

[0043] Specifically, the execution subject of the driving decision method for executing the vehicle can obtain scene information representing the current driving scene of the target vehicle from the autonomous driving vehicle through an image acquisition device set on the autonomous driving vehicle through a wired connection or a wireless connection. Among them, the scene information used to represent the current driving scene of the target vehicle refers to an image of the environment in which the autonomous driving vehicle is located during driving. The above environment may include but is not limited to one or more of the shape of the driving lane, vehicle distance, obstacles (such as pedestrians, other vehicles and other traffic participants), road signs, sky, etc.

[0044] The scene information used to represent the current driving scene of the target vehicle is converted into text through a pre-trained scenario description module to obtain a scene description text corresponding to the current driving scene.

[0045] In a disclosed embodiment, the scene description text at least includes a natural language obtained by combining one or more of the following: traffic regulations description text, signal light description text, lane line description text, navigation route description text, description text of the target vehicle, and description text of surrounding vehicles. Specifically, the traffic regulations description text includes the direction in which the vehicle is allowed to change lanes when the vehicle is in the current lane. For example, the traffic regulations description text may be that there are three lanes in total, the vehicle is currently in the second lane from the left, the vehicle is allowed to change lanes to the left, and the vehicle is allowed to change lanes to the right. The signal light description text includes the green light on, or the red light on, or the yellow light on. The lane description text may be that there are five lane lines in total, and the distance information light between each adjacent lane line. The navigation route description text includes the location information of the path planned for the vehicle to reach the destination. The description text of the target vehicle may, for example, include text information about things related to the target vehicle, such as the vehicle type, the position information of the target vehicle, and the lane in which the target vehicle is located. The description text of the surrounding vehicles includes one or more of the text information related to the surrounding vehicles, such as the vehicle type, the position information of each vehicle other than the target vehicle in the target scene image, the relative position relationship between each surrounding vehicle and the target vehicle, the lane in which the surrounding vehicles are located, etc.

[0046] Driving decisions are made based on the above-mentioned traffic regulations description text, signal light description text, lane line description text, navigation route description text, target vehicle description text and surrounding vehicle description text, which can improve the prediction accuracy of the target vehicle's driving decision.

[0047] In a disclosed embodiment, the driving knowledge in the driving knowledge base used in step S2 includes driving information related to driving decisions in historical driving scenarios.

[0048] Historical driving scenarios similar to the scenario description text obtained in step S1 can be queried in a preset driving knowledge base, thereby obtaining driving information related to driving decisions in one or more historical driving scenarios similar to the scenario description text as target driving knowledge corresponding to the current driving scenario.

[0049] In some specific sight lines of the present disclosure, the above step S2 retrieves the target driving knowledge corresponding to the current driving scene from the preset driving knowledge base based on the scene description text. Figure 3 The process shown is implemented.

[0050] See also Figure 3 , Figure 3 yes Figure 1 A flowchart of a specific implementation of step S2 in a vehicle driving decision-making method is provided.

[0051] S21: Use a large language model to compress the scene description text to obtain key text description information.

[0052] Specifically, in order to reduce the input length of the scene description text, the key text extraction module can be used to extract the key text of the scene description text corresponding to the target environment image to achieve compression of the scene description text and obtain the key text description information of the scene description text corresponding to the scene information of the current driving scene.

[0053] Specifically, few shot learning and in-context learning can be used to enable the large language model to select key information based on the scene description text and compress it into a concise natural language scene description to obtain the key text description information corresponding to the scene information of the current driving scene.

[0054] In a specific embodiment, a first text description template may be pre-set, and keywords in the scene description text may be extracted and filled into corresponding positions in the first text description template to obtain key text description information.

[0055] In one embodiment, the key text description information corresponding to the scene description text is converted into a vector to be retrieved through a token library.

[0056] S22: Searching in a driving knowledge base based on the key text description information, and obtaining a search result of the key text description information as target driving knowledge.

[0057] The driving knowledge base may contain description information of multiple historical scenes, and the description information may be in various forms such as text, image, video, etc. In an embodiment of the present application, the key text description information may be used to retrieve and match the description information of the historical scenes, and extract the relevant driving information of the matched historical scenes, that is, to obtain the target driving knowledge. Optionally, the relevant driving information of the matched historical scenes may be integrated, or the relevant driving information of the matched historical scenes may be used to perform a secondary search for other relevant driving information in the driving knowledge base, so as to obtain more accurate or richer target driving knowledge.

[0058] In one embodiment, the driving knowledge base includes a driving knowledge graph. The driving knowledge graph can be constructed in the following manner.

[0059] Specifically, the driving knowledge graph includes multiple nodes and connecting edges between nodes; a node represents driving knowledge, and a connecting edge represents that there is a correlation between two connected driving knowledge. The search result is a preset number of driving knowledge. Among them, the driving knowledge can be compressed based on the scene description text corresponding to the historical driving scene, key text description information, driving decision information and evaluation results. Among them, each driving knowledge in the driving knowledge base is a preset vector.

[0060] In a specific embodiment, the driving knowledge corresponding to the historical driving scene is used as a node in the driving knowledge graph; when the actual driving decision in the historical driving scene matches the predicted driving decision of the historical driving scene obtained by reasoning with the large language model, it is determined whether the historical driving scene is a key scene; when the historical driving scene is a key scene, a first connecting edge is constructed between the node of the driving knowledge corresponding to the historical driving scene and the node of the existing driving knowledge in the driving knowledge base referenced when reasoning with the large language model, and the first connecting edge is marked as a related edge.

[0061] In a specific embodiment, when the actual driving decision in the historical driving scene does not match the predicted driving decision of the historical driving scene obtained by reasoning with the large language model, it is determined whether the existing driving knowledge in the driving knowledge base referenced when reasoning with the large language model is erroneous and / or irrelevant to the historical driving scene. If the existing driving knowledge in the driving knowledge base referenced when reasoning with the large language model is erroneous, the existing driving knowledge is updated; if the existing driving knowledge in the driving knowledge base referenced when reasoning with the large language model is irrelevant to the historical driving scene, a second connecting edge is constructed between the node of the driving knowledge corresponding to the historical driving scene and the node of the existing driving knowledge in the driving knowledge base referenced when reasoning with the large language model, and the second connecting edge is marked as a non-relevant edge, thereby improving the accuracy of the driving knowledge graph.

[0062] In one embodiment, a search is performed in the driving knowledge graph based on the key text description information to obtain first driving knowledge related to the key text description information. A graph diffusion search is performed in the driving knowledge graph based on the first driving knowledge to obtain at least one second driving knowledge related to the first driving knowledge, and the first driving knowledge and the second driving knowledge are used as the search results of the key text description information, and the first driving knowledge and the second driving knowledge in the search results of the key text description information are used as target driving knowledge. Graph diffusion search is to continue to search in the driving knowledge graph based on the first driving knowledge as a search term to obtain the second driving knowledge related to the key text description information.

[0063] In a specific embodiment, there is at least one first driving knowledge and at least one second driving knowledge. The similarities between the first driving knowledge, the second driving knowledge and the associated text description information of the target vehicle are calculated. Based on the numerical value of the similarity, each first driving knowledge and each second driving knowledge are sorted. A preset number of driving knowledge with a high ranking is selected as the retrieval result of the key text description information. Among them, the similarity is the Euclidean distance between the vector to be retrieved and the preset vector in the driving knowledge base. The vector to be retrieved is the associated text description information of the target vehicle.

[0064] In this embodiment, by retrieving the key text description information corresponding to the scene information used to represent the current driving scene of the target vehicle, and then using the previous search results as search terms for another search, the driving knowledge selected from the driving knowledge base is made more comprehensive and accurate, which in turn helps to improve the decision-making accuracy and stability of the large language model.

[0065] In this embodiment, by compressing the scene description text, the description text corresponding to the scene information used to represent the current driving scene of the target vehicle is simplified to extract key information in the scene description text. Retrieval based on the key information of the scene description text can improve the accuracy of the retrieval results; accurate retrieval results help to improve the decision accuracy and stability of the large language model.

[0066] In a specific implementation, in step S3, based on the scene description text and the target driving knowledge, a trained large language model is used to make a decision on the driving behavior of the target vehicle. The driving decision information of the target vehicle can be obtained by Figure 5 The process shown is implemented.

[0067] See also Figure 4 , Figure 4 It is a schematic diagram of the framework structure of the large language model provided in this application.

[0068] Specifically, the large language model includes a scene classification module, a lateral decision module, and a longitudinal decision module that are cascaded in sequence; the driving decision information includes lateral decision information and longitudinal decision information. The large language model makes decisions on the scene category, lateral motion decision information, and longitudinal decision information of the target vehicle in the scene description text in sequence through context learning. In Context Learning (ICL) refers to a machine learning method that learns in a specific context. It takes into account the contextual environment of data such as text, as well as the relationship between data and the influence of contextual information. In this method, the learning algorithm uses contextual information to improve the accuracy and effectiveness of decision-making and classification, thereby improving the decision-making performance of the large language model. Among them, the large language model can be GPT-3.

[0069] See also Figure 5 , Figure 5 yes Figure 1 A flowchart of an embodiment of step S3 in a driving decision-making method for a vehicle is provided.

[0070] S31: Processing the scene description text through a scene classification module to determine the scene category of the current driving scene.

[0071] In one embodiment, the scene classification module determines the scene category of the target vehicle based on the search results of the key text description information and the positional relationship between the target vehicle and the traffic light in the key text description information. The scene categories include the vehicle being before the intersection, the vehicle being at the intersection, or the vehicle being after the intersection.

[0072] In a specific embodiment, each scene category is defined in advance, and the definition of each scene category is input into the scene classification model, so that the scene classification model can identify the scene category of the target vehicle.

[0073] S32: Processing the scene category and target driving knowledge through the lateral decision module to obtain lateral motion decision information corresponding to the target vehicle; the lateral motion decision information includes lane selection information.

[0074] Specifically, different scene categories have different decision strategies. A corresponding decision reasoning thinking chain is constructed in advance for each scene category, and the decision reasoning thinking chain corresponding to each scene category is stored in the lateral decision module, so that the lateral decision module can determine the lateral motion decision information corresponding to the target vehicle based on the scene category of the target vehicle combined with the retrieval results of the key text description information.

[0075] In a specific embodiment, the lateral motion decision information corresponding to the target vehicle is determined by a lateral decision module based on the scene category of the target vehicle and the search results of key text description information.

[0076] The lateral motion decision information is one of maintaining the current lane, turning to the right lane, and turning to the left lane.

[0077] S33: Based on the lane selection information and the scene description text, longitudinal decision information is obtained through a longitudinal decision module; the longitudinal decision information includes speed control information.

[0078] In one embodiment, description texts of surrounding vehicles in a target lane are extracted from the scene description text; the target lane is the lane where the target vehicle is located after operating according to the lateral motion decision information.

[0079] Based on the description texts of the vehicles in the target lane and the adjacent lanes of the target vehicle and the driving knowledge base, the longitudinal decision information corresponding to the target vehicle is determined.

[0080] In a specific embodiment, the longitudinal decision information corresponding to the target vehicle is determined based on the description texts of the surrounding vehicles in the target lane and the retrieval results of the scene description texts.

[0081] The longitudinal decision information of the target vehicle is speed control information, and the speed control information may be one of maintaining the current speed, accelerating, decelerating, and stopping.

[0082] In this embodiment, by extracting the description text of the surrounding vehicles in the target lane, the amount of calculation is reduced, which is beneficial for the longitudinal decision module to pay more attention to the relative relationship between the target vehicle and the surrounding vehicles in the target lane, thereby improving the accuracy of the longitudinal decision information.

[0083] In this embodiment, the scene classification module, the lateral decision module and the longitudinal decision module determine the scene category, lateral motion decision information and longitudinal decision information corresponding to the scene description text in turn based on the retrieval results of the key text description information, thereby improving the performance of the large language model in decision reasoning in the vertical field. Since the large language model can more easily understand and combine the driving knowledge in the retrieval results and thus accurately predict the control instructions of the target vehicle, the accuracy and stability of the large language model in predicting vehicle control instructions are improved.

[0084] In one embodiment, description text related to the target lane is extracted from the scene description text; the target lane is the lane where the target vehicle is located after running according to the lane selection information.

[0085] The longitudinal decision information is obtained by processing the description text related to the target lane through the longitudinal decision module.

[0086] In one embodiment, a large language model is used to evaluate the driving decision information to obtain an evaluation result of the driving decision information.

[0087] Specifically, the scene description text corresponding to the target environment image where the target vehicle is located, the scene category and the vehicle control instruction corresponding to the target vehicle are input into the large language model, and the scene classification module, the lateral decision module and the longitudinal decision module in the large language model verify the scene category, lateral motion decision information and longitudinal decision information corresponding to the target vehicle in turn, and after a preset number of verifications, the evaluation result of the vehicle control instruction is obtained based on the verification results of the preset number of times. Among them, the evaluation result includes an evaluation value and an evaluation basis. Among them, the larger the evaluation value, the higher the credibility of the vehicle control instruction. The lower the evaluation value, the lower the credibility of the vehicle control instruction.

[0088] Specifically, if the evaluation value of the vehicle control instruction reaches a preset evaluation value, it is determined that the evaluation result of the vehicle control instruction meets the preset requirement, the credibility of the vehicle control instruction is high, and the target vehicle can be controlled based on the vehicle control instruction.

[0089] In response to the evaluation value being less than a threshold value, it is determined that the vehicle control instruction inference of the large language model decision is unreliable, and the vehicle control instruction and evaluation result of the large language model are associated with the text description scene, and the associated vehicle control instruction and evaluation result, and text description scene are input into the large language model. The large language model re-analyzes the text description scene based on the feedback of the last vehicle control instruction and evaluation result that did not meet the preset requirements, the text description scene, and the driving knowledge base to obtain the vehicle control instruction of the target vehicle.

[0090] In one embodiment, the vehicle driving decision method further includes: Figure 6 Steps shown.

[0091] See also Figure 6 , Figure 6 It is a flow chart of another embodiment of the vehicle driving decision-making method provided in the present application.

[0092] S41: Generate new driving knowledge based on the driving decision information, the evaluation result of the driving decision information and the scenario description text.

[0093] Specifically, the scene description text corresponding to the target environment image where the target vehicle is located, the vehicle control instructions corresponding to the target vehicle, and the evaluation results are input into the large language model for compression to generate new driving knowledge.

[0094] In a specific embodiment, a second text description template needs to be pre-set. The scene description text corresponding to the target environment image where the target vehicle is located, the vehicle control instructions corresponding to the target vehicle, and the keywords in the evaluation results are extracted and filled in the corresponding positions in the second text description template to obtain new driving knowledge. Among them, the new driving knowledge is more abstract than the scene description text, the vehicle control instructions corresponding to the target vehicle, and the evaluation results. Compared with the scene description text, the large language model is easier to understand the driving knowledge in the driving knowledge base.

[0095] S42: Update the new driving knowledge to the driving knowledge base.

[0096] See also Figure 7 and Figure 8 , Figure 7 yes Figure 6 A flow chart of an implementation of step S42 in the vehicle driving decision-making method process shown; Figure 8 It is a schematic diagram of constructing a driving knowledge base provided by this application.

[0097] S421: In response to the vehicle operating based on the vehicle control instruction without collision, a correlation relationship is established between the new driving knowledge and each driving knowledge in the search result corresponding to the scene description text.

[0098] If the target vehicle does not collide with surrounding vehicles in the target lane after operating according to the lateral motion decision information and the longitudinal decision information, a connecting edge is added between the new driving knowledge and the driving knowledge in the first driving knowledge and the second driving knowledge, and then a correlation is established between the new driving knowledge and each driving knowledge in the retrieval results corresponding to the scene description text.

[0099] S422: In response to a collision of the vehicle after the vehicle is operated based on the vehicle control instruction, modifying the nodes and / or connecting edges in the driving knowledge base.

[0100] Specifically, if the target vehicle collides with surrounding vehicles in the target lane after the target vehicle is operated according to the lateral motion decision information and the longitudinal decision information, the driving knowledge in the retrieval results is input into the large language model one by one for inquiry to determine the reason for the error in the driving decision information of the target vehicle. If the reason for the error in the large language model is that a piece of driving knowledge in the retrieval results is incorrect, the driving knowledge can be updated. If the reason for the error in the large language model is that two pieces of driving knowledge with connecting edges are irrelevant, an irrelevant mark is set between the driving knowledge, and the connecting edge between the driving knowledge is removed.

[0101] The above method can be used to correct the driving knowledge in the driving knowledge base and the connection edges between the driving knowledge.

[0102] Therefore, in this embodiment, the control result of the vehicle is updated through the vehicle control instruction, the driving knowledge in the driving knowledge base and the correlation between driving knowledge, and the driving knowledge is supplemented for the large language model to improve the prediction accuracy and stability of the large language model.

[0103] In this embodiment, new driving knowledge is formed by compressing the evaluation result of the driving decision information with the driving decision information of the target vehicle and the scene description text, and is updated to the driving knowledge base, so as to expand the driving knowledge base.

[0104] The vehicle driving decision method provided in this embodiment generates driving knowledge by compressing the description text of historical scenes as well as the decision results and evaluation results, builds a driving knowledge base based on the driving knowledge generated from historical scenes, supplements the driving domain knowledge for the large language model in the pure language domain, and improves the accuracy and stability of the decision reasoning of the large language model.

[0105] In one embodiment, the execution body of the above method includes an online supervision module, an offline supervision module and a decision module.

[0106] The online supervision module can convert the scene information used to represent the current driving scene of the target vehicle into text to obtain a scene description text; and based on the scene description text, retrieve the target driving knowledge corresponding to the current driving scene in the preset driving knowledge base. Then, based on the retrieval result of the scene description text, the decision module is guided to determine the driving behavior of the target vehicle based on the scene description text to make a decision.

[0107] The decision module can make decisions on the driving behavior of the target vehicle based on the scene description text and target driving knowledge using a trained large language model to obtain the driving decision information of the target vehicle.

[0108] The offline supervision module can compress the scene description text corresponding to the historical driving scene, the driving decision information and decision evaluation results of the target vehicle to form driving knowledge, and store the driving knowledge text in the driving knowledge base as the driving knowledge in the next prediction process. Among them, the versatility and conciseness of driving knowledge are better than the scene description text. The offline supervision module can also update the driving knowledge base based on the scene description text corresponding to the scene information of the current driving scene and the driving decision information of the target vehicle. The large language model provided in this embodiment can be an LLM model.

[0109] See also Fig. 9 , Fig. 9 It is a schematic diagram of the framework of an embodiment of a driving decision-making device for a vehicle provided in the present application.

[0110] This embodiment provides a vehicle driving decision-making device 60 , which includes a conversion module 61 , a retrieval module 62 and a determination module 63 .

[0111] The conversion module 61 is used to perform text conversion on the scene information used to represent the current driving scene of the target vehicle to obtain a scene description text.

[0112] The retrieval module 62 is used to retrieve target driving knowledge corresponding to the current driving scenario in a preset driving knowledge base based on the scenario description text, wherein the driving knowledge in the driving knowledge base includes driving information related to driving decisions in historical driving scenarios.

[0113] The determination module 63 is used to make a decision on the driving behavior of the target vehicle based on the scene description text and the target driving knowledge using a trained large language model to obtain the driving decision information of the target vehicle.

[0114] The vehicle driving decision-making device provided in this embodiment converts the target environment image of the target vehicle into a descriptive text to convert the scene image into the language domain, so as to facilitate the subsequent addition of driving knowledge to the autonomous driving; the large language model is combined with the driving knowledge to determine the vehicle control instructions of the target vehicle, which can improve the accuracy of the vehicle control instructions; and the large language model has the iterative capability of efficient and continuous evolution, which improves the versatility of the vehicle driving decision-making method.

[0115] See also Fig.10 , Fig.10 It is a schematic diagram of the framework of an embodiment of the electronic device provided by the present application.

[0116] The electronic device 80 includes a memory 81 and a processor 82 coupled to each other, and the processor 82 is used to execute program instructions stored in the memory 81 to implement the steps in any of the above-mentioned vehicle driving decision-making method embodiments. In a specific implementation scenario, the electronic device 80 may include, but is not limited to: a microcomputer, a server, and in addition, the electronic device 80 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0117] Specifically, the processor 82 is used to control itself and the memory 81 to implement the steps in the driving decision method embodiment of any of the above-mentioned vehicles. The processor 82 can also be called a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 82 can be implemented by an integrated circuit chip.

[0118] In the above scheme, the driving decision method of the vehicle includes:

[0119] Performing text conversion on scene information used to represent a current driving scene of the target vehicle to obtain a scene description text;

[0120] Based on the scenario description text, target driving knowledge corresponding to the current driving scenario is retrieved from a preset driving knowledge base, wherein the driving knowledge in the driving knowledge base includes driving information related to driving decisions in historical driving scenarios;

[0121] Based on the scene description text and target driving knowledge, the trained large language model is used to make decisions on the target vehicle's driving behavior and obtain the target vehicle's driving decision information.

[0122] See also Fig.11 , Fig.11 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application.

[0123] The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor, and the program instructions 901 are used to implement the steps in any of the above-mentioned vehicle driving decision-making method embodiments.

[0124] In the above scheme, the driving decision method of the vehicle includes:

[0125] Performing text conversion on scene information used to represent a current driving scene of the target vehicle to obtain a scene description text;

[0126] Based on the scenario description text, target driving knowledge corresponding to the current driving scenario is retrieved from a preset driving knowledge base, wherein the driving knowledge in the driving knowledge base includes driving information related to driving decisions in historical driving scenarios;

[0127] Based on the scene description text and target driving knowledge, the trained large language model is used to make decisions on the target vehicle's driving behavior and obtain the target vehicle's driving decision information.

[0128] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0129] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0130] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0131] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0132] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A driving decision method for a vehicle, characterized in that: The method comprises: Performing text conversion on scene information used to represent a current driving scene of the target vehicle to obtain a scene description text; Based on the scenario description text, searching a preset driving knowledge base for target driving knowledge corresponding to the current driving scenario, wherein the driving knowledge in the driving knowledge base includes driving information related to driving decisions in historical driving scenarios; Based on the scenario description text and the target driving knowledge, a trained large language model is used to make a decision on the driving behavior of the target vehicle to obtain driving decision information of the target vehicle.

2. The method according to claim 1, characterized in that The step of retrieving target driving knowledge corresponding to the current driving scene from a preset driving knowledge base based on the scene description text includes: Using the large language model to perform text compression on the scene description text to obtain key text description information; A search is performed in the driving knowledge base based on the key text description information, and a search result of the key text description information is obtained as the target driving knowledge.

3. The method according to claim 2, characterized in that The driving knowledge base includes a driving knowledge graph; The searching in the driving knowledge base based on the key text description information to obtain the search result of the key text description information as the target driving knowledge includes: Searching the driving knowledge graph based on the key text description information to obtain first driving knowledge related to the key text description information; Based on the first driving knowledge, a graph diffusion search is performed in the driving knowledge graph to obtain at least one second driving knowledge related to the first driving knowledge, and the first driving knowledge and the second driving knowledge are used as search results of the key text description information.

4. The method according to claim 3, characterized in that The driving knowledge graph is constructed in the following way: The driving knowledge corresponding to the historical driving scene is used as a node in the driving knowledge graph; determining whether the historical driving scene is a critical scene when an actual driving decision in the historical driving scene matches a predicted driving decision of the historical driving scene obtained by inference of the large language model; In the case where the historical driving scene is a key scene, a first connecting edge is constructed between a node of driving knowledge corresponding to the historical driving scene and a node of existing driving knowledge in the driving knowledge base that is referenced when performing the reasoning through the large language model, and the first connecting edge is marked as a related edge.

5. The method according to claim 4, characterized in that The driving knowledge graph is also constructed in the following way: In a case where the actual driving decision in the historical driving scenario does not match the predicted driving decision of the historical driving scenario obtained by reasoning with the large language model, determining whether the existing driving knowledge in the driving knowledge base referenced when performing the reasoning with the large language model is erroneous and / or irrelevant to the historical driving scenario; If an error occurs in the existing driving knowledge in the driving knowledge base referenced when the reasoning is performed through the large language model, the existing driving knowledge is updated; if the existing driving knowledge in the driving knowledge base referenced when the reasoning is performed through the large language model is irrelevant to the historical driving scene, a second connecting edge is constructed between the node of the driving knowledge corresponding to the historical driving scene and the node of the existing driving knowledge in the driving knowledge base referenced when the reasoning is performed through the large language model, and the second connecting edge is marked as a non-relevant edge.

6. The method according to any one of claims 1 to 5, characterized in that The large language model includes a scene classification module, a lateral decision module and a longitudinal decision module; the driving decision information includes lateral decision information and longitudinal decision information; The method of making a decision on the driving behavior of the target vehicle by using a trained large language model based on the scene description text and the target driving knowledge includes: Processing the scene description text by the scene classification module to determine the scene category of the current driving scene; Processing the scene category and the target driving knowledge by the lateral decision module to obtain lateral motion decision information corresponding to the target vehicle; the lateral motion decision information includes lane selection information; Based on the lane selection information and the scene description text, the longitudinal decision information is obtained through the longitudinal decision module; the longitudinal decision information includes speed control information.

7. The method according to claim 6, characterized in that The obtaining the longitudinal decision information by the longitudinal decision module based on the lane selection information and the scene description text includes: Extracting description text related to a target lane from the scene description text; the target lane is the lane where the target vehicle is located after running according to the lane selection information; The longitudinal decision information is obtained by processing the description text related to the target lane and the longitudinal decision module.

8. The method according to claim 1, characterized in that The method further comprises: The large language model is used to evaluate the driving decision information to obtain an evaluation result of the driving decision information.

9. The method according to claim 8, characterized in that The method further comprises: generating new driving knowledge based on the driving decision information, the evaluation result of the driving decision information and the scenario description text; The new driving knowledge is updated to the driving knowledge base.

10. A driving decision-making device for a vehicle, characterized in that: The driving decision device of the vehicle comprises: A conversion module, used for performing text conversion on the scene information used to represent the current driving scene of the target vehicle to obtain a scene description text; A retrieval module, configured to retrieve target driving knowledge corresponding to the current driving scenario from a preset driving knowledge base based on the scenario description text, wherein the driving knowledge in the driving knowledge base includes driving information related to driving decisions in historical driving scenarios; A determination module is used to make a decision on the driving behavior of the target vehicle based on the scenario description text and the target driving knowledge using a trained large language model to obtain driving decision information of the target vehicle.

11. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the steps in the driving decision method for a vehicle as described in any one of claims 1 to 9.

12. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps in the vehicle driving decision-making method according to any one of claims 1 to 9 are implemented.

Citation Information

Cited By

  • Driving decision determination method and device and electronic equipment

    CN120057041A

  • Optimization method for automatic driving control, electronic equipment and vehicle

    CN120972583A

  • Automatic driving-oriented driving behavior semantic library construction method and scene retrieval method and system

    CN122173408A