Vehicle function execution method, vehicle machine and storage medium

Abstract

The invention provides a vehicle function execution method, a vehicle machine and a storage medium, and relates to the technical field of vehicle voice control. The method comprises the following steps: under the condition that a voice assistant monitors voice data sent by a user after being awakened, performing semantic recognition on the voice data to obtain a preliminary semantic recognition text; and under the condition that the vehicle function control command corresponding to the preliminary semantic recognition text is not found, extracting a plurality of keywords in the voice data. And performing logical reasoning on the plurality of keywords according to a pre-established knowledge graph to obtain a standard semantic text for executing the vehicle function control command. And controlling the vehicle to execute a vehicle function corresponding to the standard semantic text. Therefore, even if the voice data input by the user is relatively complex, the vehicle machine can accurately determine which vehicle function is executed according to the voice data, so that the vehicle function is successfully executed, and the use experience of the user is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle voice control technology, and in particular to a vehicle function execution method, vehicle infotainment system and storage medium. Background Technology

[0002] With the development of technology, intelligent cockpit systems are making great strides towards increasing intelligence. Among them, in-vehicle voice assistants are a convenient intelligent software that enables vehicles to be equipped with voice control functions. That is to say, the vehicle can listen to the user's voice, extract keywords (i.e., text) from the voice data, and perform keyword association functions.

[0003] Currently, when a user issues a voice command to the vehicle to execute a vehicle function, the vehicle can perform the function associated with the voice command. However, when the user's voice input is complex, the vehicle's infotainment system cannot accurately determine which vehicle function to execute based on the voice data. For example, if the voice command is "Turn on the air conditioning in the second row, front right side," the infotainment system cannot understand which air conditioning unit to turn on and may reply to the user with "I haven't learned this yet," causing the task to fail and impacting the user experience. Summary of the Invention

[0004] This application provides a vehicle function execution method, a vehicle-mounted system, and a storage medium to solve the problem in the prior art where, when the voice data input from the vehicle is complex, the vehicle-mounted system cannot accurately determine which vehicle function to execute based on the voice data, resulting in task execution failure and affecting the user experience.

[0005] Firstly, this application provides a vehicle function execution method, applied to a vehicle's infotainment system, which is equipped with a voice assistant. The method provided by this application includes:

[0006] After the voice assistant is activated and listens to the user's voice data, semantic recognition is performed on the voice data to obtain preliminary semantically recognized text.

[0007] In the absence of a vehicle function control command corresponding to the preliminary semantic recognition text, multiple keywords are extracted from the voice data;

[0008] Logical reasoning is performed on multiple keywords based on a pre-established knowledge graph to obtain standard semantic text for executing vehicle function control commands;

[0009] Control the vehicle to perform the vehicle functions corresponding to the standard semantic text.

[0010] In some implementations, logical reasoning is performed on multiple keywords based on a pre-established knowledge graph to obtain standard semantic text for executing vehicle function control commands, including:

[0011] Retrieve logical relationship links associated with multiple keywords from a pre-established knowledge graph;

[0012] Determine the semantic similarity between the preliminary semantically recognized text and multiple logical relationships linked to it;

[0013] Identify the target logical relationship link that has the highest semantic similarity to the text in the initial semantic recognition;

[0014] By linking multiple keywords with the target logical relationship, standard semantic text for executing vehicle function control commands is obtained.

[0015] In some implementations, standard semantic text for executing vehicle function control commands is obtained by linking multiple keywords with target logical relationships, including:

[0016] Identify anchor keywords that the vehicle's infotainment system cannot understand from a pool of keywords;

[0017] Retrieve standard descriptive terms associated with anchor keywords from the knowledge graph;

[0018] Based on standard descriptive vocabulary, keywords other than anchor keywords among multiple keywords, and target logical relationship links, standard semantic text for executing vehicle function control commands is obtained.

[0019] In some implementations, determining the semantic similarity between the initial semantically recognized text and multiple logical relationships includes:

[0020] Convert the initial semantically recognized text into a first text vector;

[0021] Convert each logical relationship link into a second text vector;

[0022] Determine the semantic similarity between the first text vector and each logical relation, where the semantic similarity between the first text vector and each second text vector is: the semantic similarity between the preliminary semantic recognition text and multiple logical relations.

[0023] In some implementations, determining the semantic similarity between the first text vector and each of the second text vectors includes:

[0024] Determine the cosine of the angle between the first text vector and each of the second text vectors;

[0025] Alternatively, determine the Euclidean distance between the first text vector and each of the second text vectors;

[0026] Alternatively, determine the Manhattan distance between the first text vector and each of the second text vectors; where the cosine value, Euclidean distance, and Manhattan distance are all used to characterize semantic similarity.

[0027] In some implementations, logical relationship links associated with multiple keywords are retrieved from a pre-established knowledge graph, including...

[0028] Using a breadth-first search method, logical relationship links associated with multiple keywords are retrieved from a pre-built knowledge graph.

[0029] In some implementations, after controlling the vehicle to perform vehicle functions corresponding to standard semantic text, the method provided in this application further includes:

[0030] The preliminary semantic recognition text is then associated with the vehicle functions corresponding to the standard semantic text.

[0031] Secondly, this application provides a vehicle function execution device applied to a vehicle's infotainment system, which is equipped with a voice assistant. The device provided by this application includes:

[0032] The semantic recognition unit is used to perform semantic recognition on the voice data after the voice assistant is woken up and listens to the voice data issued by the user, so as to obtain preliminary semantic recognition text.

[0033] The keyword extraction unit is used to extract multiple keywords from the voice data when the vehicle function control command corresponding to the preliminary semantic recognition text is not found.

[0034] The logical reasoning unit is used to perform logical reasoning on multiple keywords based on a pre-established knowledge graph to obtain standard semantic text for executing vehicle function control commands.

[0035] The function execution unit is used to control the vehicle to perform vehicle functions corresponding to the standard semantic text.

[0036] Thirdly, this application also provides a vehicle infotainment system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the vehicle infotainment system to perform the method provided in the first aspect of this application.

[0037] Fourthly, this application also provides a storage medium storing a computer program, which, when executed by a processor, causes the computer to perform the method provided in the first aspect of this application.

[0038] Fifthly, this application also provides a computer program product, including a computer program that, when run, causes the vehicle system to perform the method provided in the first aspect of this application.

[0039] This application provides a vehicle function execution method, a vehicle infotainment system, and a storage medium. When the voice assistant is activated and the user's voice data is heard, semantic recognition is performed on the voice data to obtain preliminary semantic recognition text. If no vehicle function control command corresponding to the preliminary semantic recognition text is found, it indicates that the vehicle infotainment system cannot understand the preliminary semantic recognition text normally, and therefore multiple keywords in the voice data are extracted.

[0040] By logically reasoning through multiple keywords using a pre-established knowledge graph, standard semantic text for executing vehicle function control commands is obtained. Understandably, among these keywords are anchor keywords that the vehicle's infotainment system cannot understand. The knowledge graph contains mappings between these anchor keywords and standard keywords that the system can understand. The system can then use this pre-established knowledge graph to logically reason through the keywords to obtain the standard semantic text for executing vehicle function control commands and control the vehicle to perform the corresponding vehicle function. This allows the system to accurately determine which vehicle function to execute even when the user's voice input is complex, thus successfully executing the function and improving the user experience. Furthermore, this method of obtaining standard semantic text eliminates the need for large-scale model training, saving computational resources and manpower costs. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart of a vehicle function execution method provided in an embodiment of this application;

[0043] Figure 2 for Figure 1 The detailed flowchart of S103 in the document;

[0044] Figure 3 A functional block diagram of the vehicle function execution device provided in the embodiments of this application. Detailed Implementation

[0045] Embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0046] The accompanying drawings illustrate various structural schematics according to embodiments of the present disclosure. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0047] In the context of this disclosure, when a layer / element is referred to as being "above" another layer / element, the layer / element may be directly above the other layer / element, or there may be an intermediate layer / element between them. Additionally, if a layer / element is "above" another layer / element in one orientation, then when the orientation is reversed, the layer / element may be "below" the other layer / element.

[0048] The technical solutions of this application and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0049] Please see Figure 1 This application provides a vehicle function execution method, applied to a vehicle's infotainment system, which is equipped with a voice assistant. For example... Figure 1 As shown, the method provided in this application embodiment includes S101-S105, wherein,

[0050] S101: After the voice assistant is woken up and listens to the voice data emitted by the user, perform semantic recognition on the voice data to obtain preliminary semantic recognition text.

[0051] For example, the vehicle's infotainment system is equipped with a voice assistant. Once the voice assistant is activated, it can listen to the sound data of the vehicle's environment. For instance, the infotainment system can activate the voice assistant after hearing the wake-up phrase "Hello, car".

[0052] In this process, a pre-trained semantic recognition model can be used to perform semantic recognition on the speech data to obtain preliminary semantic recognition text. For example, the preliminary semantic recognition text may be, but is not limited to, "the air conditioner in the second row, right front position is on".

[0053] S102: If the vehicle function control command corresponding to the preliminary semantic recognition text is not found, extract multiple keywords from the voice data.

[0054] If no vehicle function control command corresponding to the preliminary semantic recognition text is found, it indicates that the vehicle system cannot understand the preliminary semantic recognition text and further logical reasoning is required. Therefore, for the preliminary semantic recognition text "the air conditioner in the front right position of the second row is turned on", three keywords can be extracted from "the air conditioner in the front right position of the second row is turned on". The three keywords are "front right position of the second row", "air conditioner", and "turn on".

[0055] S103: Based on the pre-established knowledge graph, perform logical reasoning on multiple keywords to obtain standard semantic text for executing vehicle function control commands.

[0056] Understandably, the vehicle's infotainment system can recognize the single keyword "air conditioning" as the device to be controlled, and the single keyword "turn on" as the action to be performed. The system can recognize the single keyword "second row right front position" to represent location information, but it may not be able to understand the specific location represented by "second row right front position" through semantic recognition alone.

[0057] For example, such as Figure 2 As shown, S103 can be specifically implemented as follows:

[0058] S201: Retrieve logical relationship links associated with multiple keywords from a pre-established knowledge graph.

[0059] Specifically, logical relationship links associated with multiple keywords can be retrieved from a pre-built knowledge graph using the Breadth-First Search (BFS) method.

[0060] For example, the logical relationship between multiple keywords is "location", "device", and "action". The retrieved logical relationship links may include, but are not limited to, logical relationship link 1 "second row right front, device, action", logical relationship link 2 "location, device, action", and logical relationship link 3 "back, device, action".

[0061] S202: Determine the semantic similarity between the preliminary semantic recognition text and multiple logical relationships.

[0062] For example, the semantic similarity of the preliminary semantic recognition text "the air conditioner in the front right position of the second row is on" with logical relationship link 1 "front right position of the second row, device, action", logical relationship link 2 "position, device, action", and logical relationship link 3 "back, device, action" can be determined.

[0063] It should be noted that the specific method for determining similarity can be as follows: convert the initial semantically recognized text into a first text vector; and convert each logical relation link into a second text vector. For example, the initial semantically recognized text can be encoded into a first text vector based on a BERT pre-trained language model, and each logical relation link can be converted into a second text vector based on the BERT pre-trained language model. Then, the semantic similarity between the first text vector and each logical relation link is determined. Specifically, the semantic similarity between the first text vector and each second text vector is the semantic similarity between the initial semantically recognized text and multiple logical relation links.

[0064] For example, the semantic similarity between the first text vector and each second text vector can be obtained by determining the cosine of the angle between the first text vector and each second text vector; or by determining the Euclidean distance between the first text vector and each second text vector; or by determining the Manhattan distance between the first text vector and each second text vector.

[0065] Understandably, cosine value, Euclidean distance, and Manhattan distance are all used to represent semantic similarity, and the smaller the cosine value, Euclidean distance, and Manhattan distance, the higher the semantic similarity.

[0066] S203: Determine the target logical relationship link with the highest semantic similarity to the text in the preliminary semantic recognition.

[0067] For example, if the initial semantic recognition text "the air conditioner in the second row, front right position is on" has a semantic similarity of 90% with logical relationship link 1 "second row, front right, equipment, action"; 60% with logical relationship link 2 "position, equipment, action"; and 20% with logical relationship link 2 "back, equipment, action", then the target logical relationship link with the highest similarity is logical relationship link 1 "second row, front right, equipment, action". Logical relationship link 1 represents the logical meaning of controlling the action of the equipment in the second row, front right position.

[0068] S204: Based on the logical relationship between multiple keywords and the target, obtain the standard semantic text used to execute vehicle function control commands.

[0069] For example, S204 can be specifically implemented as follows:

[0070] Step A: Identify anchor keywords that the vehicle's infotainment system cannot understand from a pool of keywords.

[0071] For example, when the three keywords are "second row right front position", "air conditioning", and "on", the anchor keyword that the car system cannot understand is "second row right front position".

[0072] Step B: Retrieve standard descriptive terms associated with anchor keywords from the knowledge graph.

[0073] For example, the anchor keyword "second row right front position" is retrieved from the knowledge graph and associated with the standard descriptive term "passenger seat".

[0074] Step C: Based on the standard descriptive vocabulary, keywords other than anchor keywords among multiple keywords, and target logical relationship links, obtain the standard semantic text used to execute vehicle function control commands.

[0075] For example, in a standard descriptive term "passenger seat", among multiple keywords other than anchor keywords, the keywords include "air conditioning" and "on", and the target logical relationship link is "second row right front, equipment, action", the logical meaning represented by the target logical relationship link is "control the action of the equipment in the second row right front". Since the meaning of passenger seat corresponds to the meaning of second row right front, "second row right front" is replaced with "passenger seat"; the meaning of "air conditioning" corresponds to the meaning of "equipment", so "equipment" is replaced with "air conditioning"; the meaning of "on" corresponds to the meaning of "action", so "action" is replaced with "on". The final standard semantic text for executing vehicle function control commands is "control the air conditioning in the passenger seat to turn on".

[0076] It should be noted that S201-S204 mentioned above can be implemented in a pre-trained large language model.

[0077] S105: Control the vehicle to perform the vehicle functions corresponding to the standard semantic text.

[0078] For example, if the standard semantic text is "control the air conditioning in the passenger seat to turn on", then the text means "control the air conditioning in the passenger seat of the vehicle to turn on".

[0079] Finally, the initial semantically recognized text is associated with the corresponding vehicle functions in the standard semantic text. For example, the initial semantically recognized text "turn on the air conditioning in the front right seat of the second row" can be associated with turning on the air conditioning in the front passenger seat. When the user inputs the voice data "turn on the air conditioning in the front right seat of the second row" again according to their personal habits, they can directly control the air conditioning in the front passenger seat without logical reasoning, saving time and effort.

[0080] In summary, the vehicle function execution method provided in this application embodiment can perform semantic recognition on the voice data emitted by the user after the voice assistant is woken up, and obtain preliminary semantic recognition text; if no vehicle function control command corresponding to the preliminary semantic recognition text is found, it indicates that the vehicle system cannot understand the preliminary semantic recognition text normally, and therefore extracts multiple keywords from the voice data.

[0081] By logically reasoning through multiple keywords using a pre-established knowledge graph, standard semantic text for executing vehicle function control commands is obtained. Understandably, among these keywords are anchor keywords that the vehicle's infotainment system cannot understand. The knowledge graph contains mappings between these anchor keywords and standard keywords that the system can understand. The system can then use this pre-established knowledge graph to logically reason through the keywords to obtain the standard semantic text for executing vehicle function control commands and control the vehicle to perform the corresponding vehicle function. This allows the system to accurately determine which vehicle function to execute even when the user's voice input is complex, thus successfully executing the function and improving the user experience. Furthermore, this method of obtaining standard semantic text eliminates the need for large-scale model training, saving computational resources and manpower costs.

[0082] In addition, such as Figure 3 As shown, this application embodiment provides a vehicle function execution device applied to a vehicle's infotainment system, which is equipped with a voice assistant. It should be noted that the basic principle and technical effects of the vehicle function execution device provided in this application embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this application embodiment can be referred to the corresponding content in the above embodiments. The device provided in this application embodiment includes a semantic recognition unit, a keyword extraction unit, a logical reasoning unit, and a function execution unit, wherein...

[0083] The semantic recognition unit is used to perform semantic recognition on the voice data after the voice assistant is woken up and listens to the voice data issued by the user, so as to obtain preliminary semantic recognition text.

[0084] The keyword extraction unit is used to extract multiple keywords from the voice data when the vehicle function control command corresponding to the preliminary semantic recognition text is not found.

[0085] The logical reasoning unit is used to perform logical reasoning on multiple keywords based on a pre-established knowledge graph to obtain standard semantic text for executing vehicle function control commands.

[0086] The function execution unit is used to control the vehicle to perform vehicle functions corresponding to the standard semantic text.

[0087] In some implementations, the logical reasoning unit is specifically used to retrieve logical relationship links associated with multiple keywords from a pre-established knowledge graph; determine the semantic similarity between the preliminary semantic recognition text and the multiple logical relationship links respectively; determine the target logical relationship link with the highest semantic similarity to the preliminary semantic recognition text; and obtain standard semantic text for executing vehicle function control commands based on the multiple keywords and the target logical relationship link.

[0088] Furthermore, the logical reasoning unit is specifically used to identify anchor keywords that the vehicle system cannot understand from multiple keywords; retrieve standard descriptive terms associated with the anchor keywords from the knowledge graph; and obtain standard semantic text for executing vehicle function control commands based on the standard descriptive terms, keywords other than anchor keywords from multiple keywords, and target logical relationship links.

[0089] In some implementations, the logical reasoning unit is further specifically used to convert the preliminary semantic recognition text into a first text vector; convert each logical relation link into a second text vector; and determine the semantic similarity between the first text vector and each logical relation link, wherein the semantic similarity between the first text vector and each second text vector is: the semantic similarity between the preliminary semantic recognition text and multiple logical relation links.

[0090] In some implementations, the logical reasoning unit is further specifically used to determine the cosine value of the angle between the first text vector and each of the second text vectors; or, to determine the Euclidean distance between the first text vector and each of the second text vectors; or, to determine the Manhattan distance between the first text vector and each of the second text vectors; wherein the cosine value, Euclidean distance, and Manhattan distance are all used to characterize semantic similarity.

[0091] In some implementations, the logical reasoning unit is also specifically used to retrieve logical relationship links associated with multiple keywords from a pre-established knowledge graph using a breadth-first search method.

[0092] In some embodiments, the apparatus provided in this application further includes:

[0093] The preliminary semantic recognition text is then associated with the vehicle functions corresponding to the standard semantic text.

[0094] In addition, this application also provides a vehicle infotainment system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the vehicle infotainment system performs the method provided in the above embodiments of this application.

[0095] In addition, this application embodiment also provides a storage medium storing a computer program, which, when executed by a processor, causes the computer to perform the method provided in the above embodiments of this application.

[0096] In addition, this application also provides a computer program product, including a computer program, which, when run, causes the vehicle system to perform the method provided in the above embodiments of this application.

[0097] The above description does not provide detailed technical specifications regarding the structure of each layer. However, those skilled in the art should understand that layers and regions of desired shapes can be formed using various technical means. Furthermore, to form the same structure, those skilled in the art can also design methods that are not entirely identical to those described above. Additionally, although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be advantageously combined.

[0098] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0099] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for executing vehicle functions, characterized in that, An in-vehicle infotainment system used in vehicles, the infotainment system being equipped with a voice assistant, the method comprising: When the voice assistant is activated and listens to the voice data emitted by the user, semantic recognition is performed on the voice data to obtain preliminary semantic recognition text. If no vehicle function control command corresponding to the preliminary semantic recognition text is found, multiple keywords are extracted from the voice data; Logical reasoning is performed on the multiple keywords based on a pre-established knowledge graph to obtain standard semantic text for executing vehicle function control commands; Control the vehicle to perform the vehicle functions corresponding to the standard semantic text.

2. The method according to claim 1, characterized in that, The step of performing logical reasoning on the multiple keywords based on a pre-established knowledge graph to obtain standard semantic text for executing vehicle function control commands includes: Logical relationship links associated with the multiple keywords are retrieved from the pre-established knowledge graph; Determine the semantic similarity between the preliminary semantically recognized text and each of the multiple logical relationships linked to it; Identify the target logical relationship link that has the highest semantic similarity to the text identified in the preliminary semantic recognition; By linking the multiple keywords with the target logical relationship, standard semantic text for executing vehicle function control commands is obtained.

3. The method according to claim 2, characterized in that, Based on the linking of the multiple keywords with the target logical relationship, standard semantic text for executing vehicle function control commands is obtained, including: From the aforementioned keywords, identify the anchor keywords that the vehicle's infotainment system cannot understand; From the knowledge graph, retrieve standard descriptive terms associated with the anchor keywords; Based on the standard descriptive vocabulary, keywords other than the anchor keyword among multiple keywords, and the target logical relationship link, a standard semantic text for executing vehicle function control commands is obtained.

4. The method according to claim 2, characterized in that, Determining the semantic similarity between the preliminary semantically recognized text and the multiple logical relationships linked to it includes: The preliminary semantically recognized text is converted into a first text vector; Each of the aforementioned logical relationship links is converted into a second text vector; The semantic similarity between the first text vector and each of the logical relationships is determined, wherein the semantic similarity between the first text vector and each of the second text vectors is: the semantic similarity between the preliminary semantic recognition text and the multiple logical relationships.

5. The method according to claim 4, characterized in that, Determining the semantic similarity between the first text vector and each of the second text vectors includes: Determine the cosine of the angle between the first text vector and each of the second text vectors; Alternatively, determine the Euclidean distance between the first text vector and each of the second text vectors; Alternatively, determine the Manhattan distance between the first text vector and each of the second text vectors; wherein the cosine value, the Euclidean distance, and the Manhattan distance are all used to characterize the semantic similarity.

6. The method according to claim 2, characterized in that, The process of retrieving logical relationship links associated with the multiple keywords from a pre-established knowledge graph includes... Using a breadth-first search method, logical relationship links associated with the multiple keywords are retrieved from a pre-established knowledge graph.

7. The method according to any one of claims 1-6, characterized in that, After controlling the vehicle to execute the vehicle function corresponding to the standard semantic text, the method further includes: The preliminary semantic recognition text is associated with the vehicle functions corresponding to the standard semantic text.

8. A vehicle infotainment system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the vehicle system to perform the method as described in any one of claims 1 to 7.

9. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the computer to perform the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is run, it causes the vehicle system to perform the method as described in any one of claims 1 to 7.

Images