Cabin scene recommendation method and device, storage medium, program product and vehicle

Abstract

The invention discloses a cockpit scene recommendation method and device, a storage medium, a program product and a vehicle, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the preprocessing of man-machine conversation text data of a voice burying point log of the vehicle, obtaining the natural language description of a user, carrying out the similarity screening in a preset scene knowledge base, and obtaining a voice burying point log of the vehicle; obtaining corresponding scene knowledge; splicing the user natural language description with the scene knowledge to obtain a scene recommendation prompt word; and performing scene action reasoning on the scene recommendation prompt word through the large model agent, and outputting a scene action execution list corresponding to the scene knowledge. According to the method, man-machine conversation text data is converted into user natural language description, corresponding scene knowledge is screened, scene action reasoning is carried out based on the powerful text understanding and generating capacity of a large model agent, and the intelligent scene recommendation service of the cockpit can be automatically realized according to the voice description of a vehicle owner; and the accuracy of cabin scene recommendation is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a cockpit scene recommendation method, device, storage medium, program product and vehicle. Background Art

[0002] With the maturity of technologies such as artificial intelligence and autonomous driving, vehicles are becoming more and more intelligent. However, currently, intelligent active interaction is mostly used for vehicle driving control, such as automatic reversing and active obstacle avoidance, and the interactive operation of the cockpit system still needs to be initiated manually.

[0003] Although the cockpit can be controlled by voice, the traditional in-vehicle voice control system is subject to natural language processing algorithms. It generally recognizes the user's single intention based on the owner's voice signal and outputs the corresponding vehicle control command. It is difficult to flexibly combine vehicle control commands according to the owner's needs and create intelligent cockpit scene recommendation services for the owner. In order to adjust to the most comfortable driving environment, the owner often needs to wake up the car computer and speak the corresponding voice commands multiple times, which is difficult to effectively improve the user experience. Summary of the invention

[0004] The main purpose of this application is to provide a cockpit scene recommendation method, device, storage medium, program product and vehicle, aiming to solve the technical problem that the traditional in-vehicle voice control system is subject to the natural language processing algorithm, and the car owner often needs to wake up the car computer multiple times and speak the corresponding voice commands to adjust to the most comfortable driving environment, which makes it difficult to effectively improve the user experience.

[0005] To achieve the above objectives, the present application proposes a cockpit scene recommendation method, the method comprising:

[0006] Pre-process the human-computer dialogue text data of the vehicle's voice tracking log to obtain the user's natural language description;

[0007] Perform similarity screening on the user's natural language description in a preset scenario knowledge base to obtain scenario knowledge corresponding to the user's natural language description;

[0008] The user's natural language description is combined with the scene knowledge to obtain a scene recommendation prompt word corresponding to the vehicle cabin;

[0009] The scene action reasoning is performed on the scene recommendation prompt words through a preset large model intelligent agent, and a scene action execution list corresponding to the scene knowledge is output.

[0010] In one embodiment, the step of performing similarity screening on the user's natural language description in a preset scenario knowledge base to obtain scenario knowledge corresponding to the user's natural language description includes:

[0011] Performing vector mapping on the user's natural language description through a pre-trained language model to obtain a semantic vector;

[0012] The semantic vector is similarly screened with existing scene semantic vectors in a preset scene knowledge base to obtain scene knowledge corresponding to the semantic vector.

[0013] In one embodiment, the step of performing vector mapping on the user's natural language description through a pre-trained language model to obtain a semantic vector includes:

[0014] Performing word segmentation processing on the user's natural language description to obtain a word segmentation list;

[0015] Randomly initialize the word segmentation list to obtain a corresponding row vector list;

[0016] Inputting the row vector list into a pre-trained language model for vector mapping to obtain a semantic vector list;

[0017] Each vector in the semantic vector list is added together to obtain a semantic vector corresponding to the user's natural language description.

[0018] In one embodiment, the step of similarity screening the semantic vector with existing scene semantic vectors in a preset scene knowledge base to obtain scene knowledge corresponding to the semantic vector includes:

[0019] Calculating similarity between the semantic vector and existing scene semantic vectors in a preset scene knowledge base to obtain similarity scores between the semantic vector and each scene semantic vector;

[0020] Sorting the scene semantic vectors according to the similarity scores to obtain a sorting result;

[0021] The scene semantic vector with the highest similarity score among the scene semantic vectors is selected according to the sorting result, and the scene knowledge corresponding to the scene semantic vector with the highest similarity score is used as the scene knowledge of the semantic vector.

[0022] In one embodiment, the step of performing scene action reasoning on the scene recommendation prompt words through a preset large model agent and outputting a scene action execution list corresponding to the scene knowledge includes:

[0023] Performing model reasoning on the scene recommendation prompt words through a preset large model agent to obtain a scene action recommendation list;

[0024] Writing prompt words of the scene knowledge through a preset relevance framework;

[0025] Through the large model intelligent agent, the scene action recommendation list and the prompt word are screened for relevance, and a scene action execution list corresponding to the scene knowledge is output.

[0026] In one embodiment, the step of screening the scene action recommendation list and the prompt word for relevance through the large model agent and outputting the scene action execution list corresponding to the scene knowledge includes:

[0027] By using the large model agent, each recommended action in the scene action recommendation list is compared with the prompt word for relevance, and a confidence label corresponding to each recommended action is obtained, wherein the confidence label includes a relevant label and an irrelevant label;

[0028] The recommended actions corresponding to the irrelevant tags are deleted from the scenario action recommendation list, and a scenario action execution list corresponding to the scenario knowledge is output.

[0029] In addition, to achieve the above purpose, the present application also proposes a cockpit scene recommendation device, the device comprising:

[0030] The text preprocessing module is used to preprocess the human-computer dialogue text data of the vehicle's voice tracking log to obtain the user's natural language description;

[0031] A similarity screening module is used to perform similarity screening on the user's natural language description in a preset scenario knowledge base to obtain scenario knowledge corresponding to the user's natural language description;

[0032] A prompt word module, used for combining the user's natural language description with the scene knowledge to obtain a scene recommendation prompt word corresponding to the vehicle cockpit;

[0033] The scene action module is used to perform scene action reasoning on the scene recommendation prompt words through a preset large model intelligent agent, and output a scene action execution list corresponding to the scene knowledge.

[0034] In addition, to achieve the above objectives, the present application also proposes a vehicle, which executes the steps of implementing the cockpit scene recommendation method as described above.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the cockpit scene recommendation method described above are implemented.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the cockpit scene recommendation method as described above are implemented.

[0037] One or more technical solutions proposed in this application have at least the following technical effects: This application first pre-processes the human-computer dialogue text data of the vehicle's voice embedding log to obtain the user's natural language description; then the user's natural language description is similarly screened in a preset scene knowledge base to obtain the scene knowledge corresponding to the user's natural language description; then the user's natural language description is spliced ​​with the scene knowledge to obtain the scene recommendation prompt words corresponding to the vehicle cockpit; finally, the scene action reasoning of the scene recommendation prompt words is performed through the preset large model intelligent agent, and the scene action execution list corresponding to the scene knowledge is output. Since this application converts the human-computer dialogue text data into the user's natural language description and screens the corresponding scene knowledge, and relies on the powerful text understanding and generation capabilities of the large model intelligent agent to perform scene action reasoning, it can automatically realize the intelligent cockpit scene recommendation service according to the owner's voice description, avoiding the situation where the traditional vehicle-mounted voice control system is difficult to effectively recommend scenes, and improves the accuracy of the cockpit scene recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 A flowchart of the first embodiment of the cockpit scene recommendation method of this application is provided;

[0041] Figure 2 A schematic diagram of the network structure of the Bert model provided in Example 1 of the present application;

[0042] Figure 3 A flowchart of the second embodiment of the cockpit scene recommendation method of this application is provided;

[0043] Figure 4 A brief flowchart of the main technology provided in Example 2 of this application;

[0044] Figure 5 This is a schematic diagram of the module structure of the cockpit scene recommendation device according to an embodiment of the present application;

[0045] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the cockpit scene recommendation method in the embodiment of the present application.

[0046] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0049] It should be noted that the execution subject of this embodiment can be a computing service device with text data preprocessing, scene screening and scene recommendation functions, such as a personal computer, a car computer, etc., or an electronic device that can realize the above functions, a cockpit scene recommendation device (recommendation device) that executes the cockpit scene recommendation method of this application, etc., and this embodiment does not limit this. The following takes the recommendation device as an example to illustrate this embodiment and the following embodiments.

[0050] Based on this, the present application embodiment provides a cockpit scene recommendation method, referring to Figure 1 , Figure 1 A flowchart of the first embodiment of the cockpit scene recommendation method of the present application is provided.

[0051] In this embodiment, the cockpit scene recommendation method includes steps S10 to S40:

[0052] Step S10: pre-processing the human-computer dialogue text data of the vehicle's voice embedding log to obtain the user's natural language description.

[0053] It should be noted that voice tracking logs are logs that collect and record voice interaction-related data in vehicle-related systems.

[0054] When a user interacts with the vehicle's voice interaction system, the vehicle's system will record various information, such as the voice commands issued by the user, the timestamp of the voice, the results of voice recognition, the system's response to the voice commands, etc., to form a voice tracking log.

[0055] It should be noted that the human-computer dialogue text data is the text information record generated when the user interacts with various smart devices in the vehicle cockpit, and this text information is recorded in the voice embedding log.

[0056] It should be noted that the user's natural language description is a sentence expressed and explained by the user in the language naturally used in daily life.

[0057] Since the data recorded in the voice tracking log is recorded in a standardized programming language or code, there may be special symbols, meaningless characters and other data. Therefore, it is necessary to preprocess the human-computer dialogue text data to obtain the user's natural language description.

[0058] Specifically, the preprocessing process may include: first obtaining the user's natural language query sentence and performing data cleaning on the text content. The data cleaning rules include: first removing underscores, numbers, and special symbols in the human-computer dialogue text data; then deleting stop words, such as deleting modal particles, pause words, and meaningless characters in the human-computer dialogue text data; finally, converting all traditional Chinese characters in the human-computer dialogue text data into simplified Chinese characters, and converting all uppercase English characters in the human-computer dialogue text data into lowercase English characters; finally obtaining the user's natural language description.

[0059] For ease of understanding, an example of constructing a user's natural language description is shown in Table 1.

[0060] Table 1

[0061]

[0062] In this implementation, the recommendation device can obtain the text data of the human-computer dialogue of the user in the car from the voice embedding log, and perform data preprocessing on the human-computer dialogue text data to obtain the preprocessed natural language description of the user.

[0063] Step S20: performing similarity screening on the user's natural language description in a preset scenario knowledge base to obtain scenario knowledge corresponding to the user's natural language description.

[0064] It should be noted that the scene knowledge base is a knowledge repository built based on vector data.

[0065] The scene knowledge base can provide relevant domain background knowledge for the subsequent semantic search of the large model agent, and assist the large model agent in generating answers. With the assistance of the scene knowledge base, the hallucination phenomenon of the large model agent can be more effectively reduced, and the efficiency and accuracy of task completion can be improved.

[0066] It should be noted that the scenario knowledge is the background knowledge stored in the scenario knowledge base corresponding to the user's natural language description.

[0067] Specifically, the similarity screening process may include: first, the user's natural language description may be segmented to obtain multiple segmentation keywords; then, each segmentation keyword is searched one by one in the scene knowledge base to obtain multiple scene knowledge related to each segmentation keyword; then, the multiple scene knowledge is combined with the logic of multiple segmentation keywords to obtain multiple integrated scene knowledge; finally, the knowledge with the highest confidence is selected as the scene knowledge corresponding to the user's natural language description.

[0068] In this implementation, by performing similar screening in a preset scenario knowledge base, relevant scenario knowledge can be quickly assisted in determining, which can more effectively improve the efficiency and accuracy of task completion.

[0069] In a feasible implementation, step S20 of this embodiment may include the steps of: performing vector mapping on the user's natural language description through a pre-trained language model to obtain a semantic vector; performing similarity screening on the semantic vector and the existing scene semantic vectors in a preset scene knowledge base to obtain the scene knowledge corresponding to the semantic vector.

[0070] It should be noted that the pre-trained language model is a model used to capture contextual information in text, learn rich language knowledge and semantics, and perform language processing tasks, such as the Bert model, the GPT model, etc., which is not limited in this embodiment.

[0071] It should be noted that the semantic vector is data that represents the semantic information of the text described by the user's natural language in the form of a vector.

[0072] By converting the words, sentences or texts described in the user's natural language into a set of numerical vectors through a pre-trained language model, it can reflect the semantic features contained in the user's natural language description to a certain extent, such as the similarity and semantic relationship between words, which facilitates analysis and comparison in the scene knowledge base.

[0073] The scene semantic vector is a semantic vector that contains scene knowledge in the scene knowledge base.

[0074] Specifically, in the process of similarity screening, the screening may be performed by calculating the similarity between vectors, or by string matching, which is not limited in this embodiment.

[0075] In this implementation, by mapping the user's natural language description into a semantic vector, the semantic features contained in the user's natural language description can be greatly represented, making it easier to screen in the scenario knowledge base.

[0076] Furthermore, the step of performing vector mapping on the user's natural language description through a pre-trained language model to obtain a semantic vector includes: performing word segmentation on the user's natural language description to obtain a word segmentation list; randomly initializing the word segmentation list to obtain a corresponding row vector list; inputting the row vector list into the pre-trained language model for vector mapping to obtain a semantic vector list; and adding each vector in the semantic vector list to obtain a semantic vector corresponding to the user's natural language description.

[0077] It should be noted that the word segmentation list is a word sequence obtained by segmenting the text corresponding to a user's natural language description according to words, for example, dividing "The weather is really good today" into a word list such as "today", "weather", and "really good".

[0078] The row vector list is a set of row vectors that are formed by converting each word in the word list into multiple row vectors. A row vector is a set of values ​​in a certain order.

[0079] The semantic vector list is a set of semantic vectors obtained by performing vector mapping on row vectors through the pre-trained language model.

[0080] Specifically, after obtaining the user's natural language description, the user's natural language description text can be segmented to form a corresponding segmentation list of the user's natural language description, and a 512-dimensional row vector is randomly initialized for each word in the segmentation list to obtain a row vector list. Then the row vector corresponding to each word is input into the Bert model for training, and the random vector is mapped to the corresponding semantic vector. The Bert model outputs the semantic vector list corresponding to each word in the segmentation list one by one. Finally, all the vectors in the semantic vector list are added together to obtain the semantic vector corresponding to the user's natural language description sentence.

[0081] To understand the process of vector mapping by the Bert model, please refer to Figure 2 , Figure 2 This is a schematic diagram of the network structure of the Bert model provided in Example 1 of the present application. N represents the randomly initialized vector of the Bert model input layer, Trm represents the neuron structure of the Bert model hidden layer, T1, T2 to T N Represents the semantic vector output by the output layer of the Bert model. Through the Bert model, the semantic vector corresponding to each word segmentation can be output one by one.

[0082] For example, for ease of understanding, the user's natural language description is mapped to a vector using the Bert model, and an example of constructing a data set of corresponding semantic vectors is shown in Table 2:

[0083] Table 2

[0084]

[0085] In this implementation, by performing word segmentation processing on the user's natural language description, the classification efficiency of the vector mapping can be further improved.

[0086] Furthermore, the step of performing similarity screening on the semantic vector and the scene semantic vectors already in the preset scene knowledge base to obtain the scene knowledge corresponding to the semantic vector includes: performing similarity calculation on the semantic vector and the scene semantic vectors already in the preset scene knowledge base to obtain the similarity scores between the semantic vector and each scene semantic vector; sorting the scene semantic vectors according to the similarity scores to obtain the sorting results; screening the scene semantic vectors with the highest similarity score according to the sorting results, and taking the scene knowledge corresponding to the scene semantic vector with the highest similarity score as the scene knowledge of the semantic vector.

[0087] It should be noted that the similarity score is a quantitative indicator used to measure the similarity between the semantic vector and the semantic vectors of each scene.

[0088] The similarity calculation may be performed by cosine similarity calculation or Jaccard similarity coefficient calculation, which is not limited in this embodiment.

[0089] By calculating the similarity between the semantic vector and the semantic vectors of each scene and expressing their similarity with a numerical value (i.e., similarity score), the magnitude of the similarity can be intuitively quantified.

[0090] Specifically, after obtaining the semantic vector, the cosine similarity calculation can be performed between the semantic vector corresponding to the user's natural language description and the scene semantic vectors already in the scene knowledge base to obtain the similarity scores between the user's natural language description and all the scenes already in the scene knowledge base. Then, all the scene semantic vectors are sorted based on the similarity scores, and the scene knowledge corresponding to the scene semantic vector with the highest similarity calculation score is selected as the scene knowledge of the semantic vector corresponding to the user's natural language description.

[0091] The specific formula for calculating the cosine similarity between the semantic vector corresponding to the user's natural language description and the scene semantic vector already in the scene knowledge base is as follows:

[0092] cos(θ)=(A·B) / (||A||||B||);

[0093] Among them, A represents the semantic vector corresponding to the user's natural language description, B represents the scene semantic vector already in the scene knowledge base, ||A|| and ||B|| represent the modules of A and B respectively, and A·B represents the dot product of A and B.

[0094] For example, for ease of understanding, the cosine similarity calculation is performed on the semantic vector corresponding to the obtained user natural language description and the existing scene knowledge in the scene knowledge base, and the data construction example sorted according to the similarity score is shown in Table 3:

[0095] Table 3

[0096]

[0097]

[0098] In this implementation, the similarity score can be used to intuitively quantify the similarity between the semantic vector and each scene semantic vector.

[0099] Step S30: combining the user's natural language description with the scene knowledge to obtain scene recommendation prompt words corresponding to the vehicle cockpit.

[0100] It should be noted that the scene recommendation prompt words are prompt words used to guide and prompt users to make mode selections in the vehicle cockpit.

[0101] Exemplarily, for ease of understanding, the template for the scene recommendation prompt may be: "You are an expert in smart cockpit scene recommendation. Please recommend the most suitable car use scene and vehicle control instructions to the car owner based on the car owner's description and known scene knowledge. The known scene information is: {scene information 1}".

[0102] In this embodiment, the pre-processed user natural language description is concatenated with the retrieved scene knowledge to obtain scene recommendation prompts sent to the large model agent. By constructing scene recommendation prompts, it is convenient for users to select corresponding cockpit scenes.

[0103] Step S40: Perform scene action reasoning on the scene recommendation prompt words through a preset large model agent, and output a scene action execution list corresponding to the scene knowledge.

[0104] It should be noted that the large model agent (LLM agent) is an entity with certain intelligent behavior and interaction capabilities built based on a large-scale language model, such as ChatGLM3-6B, GPT-3 and other models, which are not limited in this embodiment.

[0105] Large-model agents are built on deep neural networks containing billions or even tens of billions of parameters and have powerful intelligent processing capabilities. Large-model agents usually use self-supervised learning methods and are trained with large amounts of unlabeled text, images, or video data to learn and understand complex information such as language and vision. They can handle complex natural language tasks such as text generation, question-answering systems, and sentiment analysis.

[0106] By leveraging the powerful language understanding and generation capabilities of the large language model, it can analyze and respond to the input scene recommendation prompts, enable the in-car cabin to interact with the user, and complete various complex intelligent activities.

[0107] It should be noted that the scene action execution list is a list of specific actions that smart devices need to perform in the car cabin scene, such as opening the rear seat touch screen, playing animations, playing voice guidance, locking the child lock, etc.

[0108] Specifically, when performing scene action reasoning, the execution actions that are not very relevant to the scene recommendation prompt word may be first screened out to obtain a scene action execution list; the execution actions may also be sorted to obtain a scene action execution list for user selection. This embodiment does not limit this.

[0109] In this implementation, the scene action execution list helps the user to select a scene.

[0110] In the technical solution provided in this embodiment, the recommendation device can obtain the text data of the human-computer dialogue of the user in the car from the voice embedding log, and perform data preprocessing on the human-computer dialogue text data to obtain the preprocessed user natural language description. Then the user natural language description is similarly screened in the preset scene knowledge base to obtain the scene knowledge corresponding to the user natural language description. Then the obtained preprocessed user natural language description is spliced ​​with the retrieved scene knowledge to obtain the scene recommendation prompt word sent to the large model intelligent body. Finally, the scene action reasoning of the scene recommendation prompt word is performed through the preset large model intelligent body, and the scene action execution list corresponding to the scene knowledge is output. Since this embodiment converts the human-computer dialogue text data into the user natural language description and screens the corresponding scene knowledge, relying on the powerful text understanding and generation capabilities of the large model intelligent body to perform scene action reasoning, it can automatically realize the intelligent cockpit scene recommendation service according to the voice description of the car owner, avoiding the situation that the traditional vehicle voice control system is difficult to effectively recommend scenes, and improving the accuracy of the cockpit scene recommendation.

[0111] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , Figure 3A flowchart of the second embodiment of the cockpit scene recommendation method of the present application is provided.

[0112] In this example, step S40 includes steps S41 to S43:

[0113] Step S41: Perform model reasoning on the scene recommendation prompt words through a preset large model agent to obtain a scene action recommendation list.

[0114] Step S42: Compile prompt words of the scenario knowledge through a preset correlation framework.

[0115] It should be noted that the relevance framework is a structure used for self-reflection or self-examination of scene knowledge, such as a self-reflection framework, etc., and this embodiment does not limit this.

[0116] It should be noted that the prompt words are related to the scene knowledge and are used to guide the large model agent to determine whether the execution actions in the scene action recommendation list are related to the scene knowledge.

[0117] Step S43: Through the large model agent, the scene action recommendation list and the prompt word are screened for relevance, and a scene action execution list corresponding to the scene knowledge is output.

[0118] It should be noted that the relevance screening can be performed by the size of the confidence value; the relevant and irrelevant content can also be distinguished by a classification algorithm, which is not limited in this embodiment.

[0119] In this implementation, the recommendation device can write prompt words based on the self-reflection framework, and explicitly require the large model agent to introspect the correlation between the output results and the scene knowledge in the prompt words, and use the large model agent to determine whether each recommended action in the scene action recommendation list is related to the scene knowledge, thereby improving the performance of the large model in the automotive intelligent cockpit scene recommendation task. If it is not relevant, the action is deleted from the scene action recommendation list, and finally the scene action execution list that is strongly related to the scene knowledge is output. This can reduce the selection complexity of the scene action execution list and improve the user experience.

[0120] In a feasible implementation, step S43 of this example includes the steps of: comparing the relevance of each recommended action in the scene action recommendation list with the prompt word through the large model intelligent agent, obtaining a confidence label corresponding to each recommended action, wherein the confidence label includes a relevant label and an irrelevant label; deleting the recommended action corresponding to the irrelevant label from the scene action recommendation list, and outputting a scene action execution list corresponding to the scene knowledge.

[0121] It should be noted that the confidence label is used to indicate whether each recommended action is strongly related to the scene knowledge. The relevant label is the label that the large model agent judges that the scene recommended action is related to the known scene knowledge; the irrelevant label is the label that the large model agent judges that the scene recommended action is irrelevant to the known scene knowledge. The value range of the confidence label can be set according to the actual situation, and this embodiment does not limit this.

[0122] Exemplarily, for ease of understanding, the specific prompt word template may be: "Please judge whether each recommended action is required by the current recommended scene based on the scene action recommendation list and the known scene knowledge in the prompt words. At the same time, please judge whether the recommended action conflicts with the scene definition. If the scene action is related to the current known scene, output the relevant label; if not, output the irrelevant label."

[0123] For example, a user's natural language description uses a large model agent to generate a list of recommended actions for a scenario, and outputs an example of a scenario action execution list that is strongly related to the scenario knowledge, as shown in Table 4:

[0124] Table 4

[0125]

[0126] The above results show that the results output based on the large model intelligent agent and the scene knowledge base answer the questions asked very correctly, are very similar to the actual scenes and action recommendations, and have outstanding effects, proving the feasibility of the cockpit scene recommendation method based on the large model intelligent agent and the scene knowledge base of this embodiment.

[0127] In this implementation, screening through confidence labels can further improve the accuracy of the large model agent in determining whether the recommended action of the scene is related to the known scene knowledge.

[0128] In the technical solution provided by this embodiment, by filtering the scene action recommendation list, the selection complexity of the scene action execution list can be reduced and the user experience can be improved. By filtering through confidence tags, the accuracy of the large model agent in judging whether the scene recommended action is related to the known scene knowledge can be further improved.

[0129] For example, in order to help understand the implementation process of the cockpit scene recommendation method obtained by combining the above-mentioned embodiment 1 and embodiment 2 in this embodiment, please refer to Figure 4 , Figure 4 A brief flow chart of the main technology provided in Example 2 of this application is as follows:

[0130] Step S1: First, obtain the text data of the user's human-computer dialogue in the car from the car-side voice embedding log, preprocess the text data, and obtain the preprocessed user natural language description.

[0131] Step S2: Use the Bert model to perform vector mapping on the preprocessed user natural language description obtained in step S1 to obtain a semantic vector corresponding to the user natural language description.

[0132] Step S3: Then perform cosine similarity calculation between the semantic vector in step S2 and the existing scene semantic vectors in the scene knowledge base, and select the scene knowledge corresponding to the scene semantic vector with the highest similarity calculation score.

[0133] Step S4, construct a prompt word template, and splice the preprocessed user natural language description obtained in step S1 with the scene knowledge retrieved in step S3 to obtain the scene recommendation prompt words sent to the large model intelligent agent.

[0134] Step S5: Use the large model agent to perform model reasoning based on the scene recommendation prompt words obtained in step S4, and output a scene action recommendation list.

[0135] Step S6, write prompt words based on the self-reflection framework, and use the large model intelligent agent to determine whether each recommended action in the scene action recommendation list is related to the scene knowledge obtained in S3. If not, delete the action from the scene action recommendation list, and finally output a scene action execution list that is strongly related to the scene knowledge.

[0136] Through the above method, on the one hand, the scene knowledge base method is used to assist in constructing prompt words so that the large model agent can output scene recommended actions, which greatly improves the accuracy and intelligence of the large model agent's recommendation of private domain scene actions in the field of automobile smart cockpits, and avoids the large model agent from making wrong recommendations due to knowledge illusion. On the other hand, by writing prompt words through the self-reflection framework, the large model agent can introspect the output results and judge whether each recommended action in the output scene action recommendation list is related to the known scene knowledge, thereby improving the accuracy of scene recommendation. It solves the technical problem that it is difficult to effectively implement intelligent scene recommendation in practical applications.

[0137] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the cockpit scene recommendation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0138] This application also provides a cockpit scene recommendation device, please refer to Figure 5 , Figure 5 Schematic diagram of the module structure of the cockpit scene recommendation device according to the embodiment of the present application; the cockpit scene recommendation device comprises:

[0139] The text preprocessing module 501 is used to preprocess the human-computer dialogue text data of the vehicle's voice embedding log to obtain the user's natural language description;

[0140] A similarity screening module 502 is used to perform similarity screening on the user's natural language description in a preset scenario knowledge base to obtain scenario knowledge corresponding to the user's natural language description;

[0141] A prompt word module 503 is used to combine the user's natural language description with the scene knowledge to obtain a scene recommendation prompt word corresponding to the vehicle cabin;

[0142] The scene action module 504 is used to perform scene action reasoning on the scene recommendation prompt words through a preset large model intelligent agent, and output a scene action execution list corresponding to the scene knowledge.

[0143] In this embodiment, the similarity screening module 502 is also used to perform vector mapping on the user's natural language description through a pre-trained language model to obtain a semantic vector; perform similarity screening on the semantic vector and the existing scene semantic vectors in a preset scene knowledge base to obtain the scene knowledge corresponding to the semantic vector.

[0144] Furthermore, the similarity screening module 502 is also used to perform word segmentation on the user's natural language description to obtain a word segmentation list; randomly initialize the word segmentation list to obtain a corresponding row vector list; input the row vector list into a pre-trained language model for vector mapping to obtain a semantic vector list; add each vector in the semantic vector list to obtain a semantic vector corresponding to the user's natural language description.

[0145] Furthermore, the similarity screening module 502 is also used to calculate the similarity between the semantic vector and the existing scene semantic vectors in a preset scene knowledge base to obtain the similarity scores between the semantic vector and each scene semantic vector; sort the scene semantic vectors according to the similarity scores to obtain the sorting results; screen the scene semantic vectors with the highest similarity score according to the sorting results, and use the scene knowledge corresponding to the scene semantic vector with the highest similarity score as the scene knowledge of the semantic vector.

[0146] Furthermore, the scene action module 504 is also used to perform model inference on the scene recommendation prompt words through a preset large model intelligent agent to obtain a scene action recommendation list; write the prompt words of the scene knowledge through a preset relevance framework; through the large model intelligent agent, perform relevance screening on the scene action recommendation list and the prompt words, and output a scene action execution list corresponding to the scene knowledge.

[0147] Furthermore, the scene action module 504 is also used to compare the relevance of each recommended action in the scene action recommendation list with the prompt word through the large model intelligent agent, and obtain a confidence label corresponding to each recommended action, wherein the confidence label includes a relevant label and an irrelevant label; delete the recommended action corresponding to the irrelevant label from the scene action recommendation list, and output a scene action execution list corresponding to the scene knowledge.

[0148] Other embodiments or specific implementations of the cockpit scene recommendation device of the present application can refer to the above-mentioned method embodiments and will not be described in detail here.

[0149] The cockpit scene recommendation device provided by the present application adopts the cockpit scene recommendation method in the above embodiment, which can solve the technical problem that the traditional vehicle-mounted voice control system is subject to the natural language processing algorithm, and the car owner often needs to wake up the car computer and speak the corresponding voice commands multiple times to adjust to the most comfortable driving environment, which makes it difficult to effectively improve the user experience. Compared with the prior art, the beneficial effects of the cockpit scene recommendation device provided by the present application are the same as the beneficial effects of the cockpit scene recommendation method provided by the above embodiment, and the other technical features of the cockpit scene recommendation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0150] The present application provides a vehicle, wherein the vehicle executes the steps of implementing the above-mentioned cockpit scene recommendation method through a cockpit scene recommendation device (i.e., a vehicle control center device) carried thereon. The cockpit scene recommendation device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the cockpit scene recommendation method in the above-mentioned embodiment 1.

[0151] Reference below Figure 6 , Figure 6 The device structure diagram of the hardware operating environment involved in the cockpit scene recommendation method in the embodiment of the present application shows a schematic diagram of the structure of the cockpit scene recommendation device suitable for implementing the embodiment of the present application. The cockpit scene recommendation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6The cockpit scene recommendation device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0152] like Figure 6 As shown, the cockpit scene recommendation device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the cockpit scene recommendation device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the cockpit scene recommendation device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a cockpit scene recommendation device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0153] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0154] The cockpit scene recommendation device provided by this application adopts the cockpit scene recommendation method in the above embodiment, which can solve the technical problem that the traditional vehicle-mounted voice control system is subject to the natural language processing algorithm, and the owner often needs to wake up the car computer and speak the corresponding voice commands multiple times to adjust to the most comfortable driving environment, which makes it difficult to effectively improve the user experience. Compared with the prior art, the beneficial effects of the cockpit scene recommendation device provided by this application are the same as the beneficial effects of the cockpit scene recommendation method provided by the above embodiment, and the other technical features of the cockpit scene recommendation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0155] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0156] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0157] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the cockpit scene recommendation method in the above-mentioned embodiment.

[0158] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0159] The computer-readable storage medium may be included in the cockpit scene recommendation device; or may exist independently without being assembled into the cockpit scene recommendation device.

[0160] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the cockpit scene recommendation device, the cockpit scene recommendation device enables the following: pre-processing the human-computer dialogue text data of the vehicle's voice embedding log to obtain a user natural language description; performing similarity screening on the user's natural language description in a preset scene knowledge base to obtain scene knowledge corresponding to the user's natural language description; splicing the user's natural language description with the scene knowledge to obtain scene recommendation prompt words corresponding to the vehicle cockpit; performing scene action reasoning on the scene recommendation prompt words through a preset large model intelligent agent, and outputting a scene action execution list corresponding to the scene knowledge.

[0161] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0164] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned cockpit scene recommendation method, and can solve the technical problem that the traditional vehicle-mounted voice control system is subject to the natural language processing algorithm, and the owner often needs to wake up the vehicle computer and speak the corresponding voice commands multiple times to adjust to the most comfortable driving environment, which makes it difficult to effectively improve the user experience. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the cockpit scene recommendation method provided by the above-mentioned embodiment, and will not be repeated here.

[0165] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned cockpit scene recommendation method when executed by a processor.

[0166] The computer program product provided by this application can solve the technical problem that the traditional in-vehicle voice control system is subject to the natural language processing algorithm, and the owner often needs to wake up the car computer and speak the corresponding voice commands multiple times to adjust to the most comfortable driving environment, which makes it difficult to effectively improve the user experience. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the cockpit scene recommendation method provided by the above embodiment, which will not be repeated here.

[0167] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A cockpit scene recommendation method, characterized in that: The method includes: Pre-process the human-computer dialogue text data of the vehicle's voice tracking log to obtain the user's natural language description; Perform similarity screening on the user's natural language description in a preset scenario knowledge base to obtain scenario knowledge corresponding to the user's natural language description; The user's natural language description is combined with the scene knowledge to obtain a scene recommendation prompt word corresponding to the vehicle cabin; The scene action reasoning is performed on the scene recommendation prompt words through a preset large model intelligent agent, and a scene action execution list corresponding to the scene knowledge is output.

2. The method according to claim 1, characterized in that The step of similarity screening the user's natural language description in a preset scenario knowledge base to obtain scenario knowledge corresponding to the user's natural language description includes: Performing vector mapping on the user's natural language description through a pre-trained language model to obtain a semantic vector; The semantic vector is similarly screened with existing scene semantic vectors in a preset scene knowledge base to obtain scene knowledge corresponding to the semantic vector.

3. The method according to claim 2, characterized in that The step of performing vector mapping on the user's natural language description through a pre-trained language model to obtain a semantic vector includes: Performing word segmentation processing on the user's natural language description to obtain a word segmentation list; Randomly initialize the word segmentation list to obtain a corresponding row vector list; Inputting the row vector list into a pre-trained language model for vector mapping to obtain a semantic vector list; Each vector in the semantic vector list is added together to obtain a semantic vector corresponding to the user's natural language description.

4. The method according to claim 2, characterized in that The step of similarity screening the semantic vector with existing scene semantic vectors in a preset scene knowledge base to obtain scene knowledge corresponding to the semantic vector includes: Calculating similarity between the semantic vector and existing scene semantic vectors in a preset scene knowledge base to obtain similarity scores between the semantic vector and each scene semantic vector; Sorting the scene semantic vectors according to the similarity scores to obtain a sorting result; The scene semantic vector with the highest similarity score among the scene semantic vectors is selected according to the sorting result, and the scene knowledge corresponding to the scene semantic vector with the highest similarity score is used as the scene knowledge of the semantic vector.

5. The method according to any one of claims 1 to 4, characterized in that The step of performing scene action reasoning on the scene recommendation prompt words through a preset large model agent and outputting a scene action execution list corresponding to the scene knowledge comprises: Performing model reasoning on the scene recommendation prompt words through a preset large model agent to obtain a scene action recommendation list; Writing prompt words of the scene knowledge through a preset relevance framework; Through the large model intelligent agent, the scene action recommendation list and the prompt word are screened for relevance, and a scene action execution list corresponding to the scene knowledge is output.

6. The method according to claim 5, characterized in that The step of screening the scene action recommendation list and the prompt word for relevance through the large model agent and outputting a scene action execution list corresponding to the scene knowledge includes: By using the large model agent, each recommended action in the scene action recommendation list is compared with the prompt word for relevance, and a confidence label corresponding to each recommended action is obtained, wherein the confidence label includes a relevant label and an irrelevant label; The recommended actions corresponding to the irrelevant tags are deleted from the scenario action recommendation list, and a scenario action execution list corresponding to the scenario knowledge is output.

7. A cockpit scene recommendation device, characterized in that: The device comprises: The text preprocessing module is used to preprocess the human-computer dialogue text data of the vehicle's voice tracking log to obtain the user's natural language description; A similarity screening module is used to perform similarity screening on the user's natural language description in a preset scenario knowledge base to obtain scenario knowledge corresponding to the user's natural language description; A prompt word module, used for combining the user's natural language description with the scene knowledge to obtain a scene recommendation prompt word corresponding to the vehicle cockpit; The scene action module is used to perform scene action reasoning on the scene recommendation prompt words through a preset large model intelligent agent, and output a scene action execution list corresponding to the scene knowledge.

8. A vehicle, characterized in that: The vehicle executes the steps of implementing the cockpit scene recommendation method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the cockpit scene recommendation method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the cockpit scene recommendation method according to any one of claims 1 to 6 are implemented.

Images