A vehicle navigation method and a vehicle navigation device

Abstract

The application provides a vehicle navigation method and a vehicle navigation device, in which a navigation voice instruction of a user is acquired, speech recognition and intention analysis are performed on the navigation voice instruction, all destinations and time sequence dependent relationships between the destinations contained in the navigation voice instruction are recognized, and a directed acyclic graph is generated based on the destinations and the time sequence dependent relationships, so that explicit modeling of the time sequence dependent relationships of multiple destinations is realized, and for a fuzzy attribute POI, a target POI pointed by a fuzzy attribute description in the navigation voice instruction is accurately recognized through an attribute query condition extraction and POI information library retrieval mechanism. In this way, for a voice instruction containing multiple destinations and a fuzzy attribute description, accurate navigation intention recognition is realized, and the accuracy of navigation is effectively improved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more specifically, to a vehicle navigation method and a vehicle navigation device. Background Technology

[0002] With the rapid development of intelligent connected vehicle technology, intelligent cockpit systems have become a standard feature in modern vehicles. Users can issue navigation commands to the intelligent cockpit system via voice interaction. The system converts the user's voice commands into text, and then the vehicle's NLU (Natural Language Understanding) module understands the user's navigation intent to ultimately determine the navigation destination. Currently, users increasingly prefer to express their navigation needs using natural language rather than precise addresses when driving or riding in a vehicle. However, most intelligent cockpit systems in related technologies are designed for single interactive commands, meaning they can only recognize and process a single destination query. When a voice command contains multiple destinations with temporal dependencies and vague attribute descriptions, the intelligent cockpit system easily overlooks the temporal dependencies between destinations and struggles to understand the user's intent, thus failing to provide accurate navigation routes. Summary of the Invention

[0003] The purpose of this application is to provide a vehicle navigation method and a vehicle navigation device, which aims to solve the problem in the related technology that intelligent cockpit systems cannot provide accurate navigation routes when a voice command contains multiple destinations with temporal dependencies and have fuzzy attribute descriptions.

[0004] In a first aspect, this application provides a vehicle navigation method, comprising: acquiring a user's navigation voice command; obtaining multiple candidate destinations and temporal dependencies between the multiple candidate destinations by performing speech recognition and intent analysis on the navigation voice command; when the multiple candidate destinations include fuzzy attribute POIs, extracting attribute query conditions from the fuzzy attribute POIs based on a pre-trained large language model, searching in a POI information database according to the attribute query conditions, obtaining a target POI based on the search results, and replacing the fuzzy attribute POIs with the target POIs; combining the replaced candidate destinations according to the temporal dependencies to generate a directed acyclic graph, and displaying the directed acyclic graph on a navigation interactive interface.

[0005] In the above implementation process, the user's navigation voice commands are acquired. Through speech recognition and intent analysis of these commands, all destinations and their temporal dependencies are identified, generating a directed acyclic graph (DAG). This allows for explicit modeling of the temporal dependencies between multiple destinations. Furthermore, for fuzzy attribute Points of Interest (POIs), attribute query condition extraction and POI information database retrieval mechanisms accurately identify the target POIs pointed to by the fuzzy attribute descriptions in the navigation voice commands. Thus, accurate navigation intent recognition is achieved for voice commands containing multiple destinations and fuzzy attribute descriptions, effectively improving navigation accuracy.

[0006] Furthermore, in some examples, the attribute query conditions include: attribute name, comparison operator, and attribute value; the attribute name represents the structured attribute of the fuzzy attribute POI; the comparison operator represents the comparison relationship; the attribute value represents the target value of the comparison operator; the POI information base is a POI knowledge graph; each POI node in the POI knowledge graph contains a name, coordinates, category, and structured attributes.

[0007] In the above implementation process, attribute queries are performed in the POI knowledge graph based on attribute name, comparison operator and attribute value as attribute query conditions, thereby improving the recall rate of fuzzy attribute POIs.

[0008] Furthermore, in some examples, obtaining the target POI based on the search results includes: sorting the search results and selecting a preset number of POI nodes at the top of the sort as a candidate POI sequence; obtaining the candidate total score of each candidate POI in the candidate POI sequence based on the user's gaze direction, gesture, current vehicle location, and user profile when the user issues the navigation voice command; wherein the candidate total score is used to measure the degree of matching between the candidate POI and the user's navigation intent; and determining the candidate POI with the highest candidate total score as the target POI.

[0009] In the above implementation process, when the confidence of a single voice channel is low due to unclear speech or cabin noise, robust candidate ranking is provided by using modalities such as user gaze, gestures and vehicle position, thereby reducing the false recognition rate.

[0010] Further, in some examples, obtaining the candidate total score for each candidate POI in the candidate POI sequence based on the user's gaze direction, gesture, current vehicle location, and user profile when issuing the navigation voice command includes: for any candidate POI in the candidate POI sequence, calculating the semantic similarity score, alignment score, matching score, and geographic distance score of the candidate POI; wherein, the semantic similarity score is determined based on the semantic similarity between the candidate POI and the attribute query conditions; the alignment score is determined based on the alignment between the candidate POI and the user's gaze direction or gesture; the matching score is determined based on the matching degree between the candidate POI and the user's user profile; the geographic distance score is determined based on the distance between the candidate POI and the current vehicle location; and the weighted combination of the semantic similarity score, the alignment score, the matching score, and the geographic distance score is determined as the candidate total score of the candidate POI.

[0011] In the above implementation process, multimodal data including voice, gaze, gesture, vehicle location, and user profile are used to calculate the total candidate score of candidate POIs, so that robust candidate ranking can still be given in scenarios with unclear voice or noisy cabins, reducing the false recognition rate.

[0012] Furthermore, in some examples, before obtaining the total candidate score of each candidate POI in the candidate POI sequence, the method further includes: filtering the candidate POI sequence based on viewing angle constraints, time window constraints, and access constraints; wherein, the viewing angle constraints are used to filter candidate POIs whose viewing angle does not meet the user's intention; the time window constraints are used to filter candidate POIs that are not in the accessible or optimal time period; and the access constraints are used to filter candidate POIs that the current vehicle cannot reach.

[0013] In the above implementation process, when performing attribute disambiguation on fuzzy attribute POIs, implicit spatiotemporal constraints are introduced, including viewing angle constraints, time window constraints, and access constraints, so that the returned results meet the user's fuzzy navigation intent in terms of spatial perspective, time, and accessibility.

[0014] Furthermore, in some examples, displaying the directed acyclic graph on the navigation interface includes: displaying each node in the directed acyclic graph with a differentiated icon, displaying each directed edge in the directed acyclic graph as a visual path line, and listing the candidate POI sequence next to the node corresponding to the fuzzy attribute POI.

[0015] In the above implementation process, a specific method is provided to render a directed acyclic graph to the navigation interaction interface, thereby improving the user-friendliness of navigation.

[0016] Furthermore, in some examples, before generating a directed acyclic graph by combining the replaced candidate destinations according to the temporal dependency, the method includes: when the multiple candidate destinations include personalized context POIs, parsing and mapping the personalized context POIs to corresponding POI addresses based on user profiles, vehicle current location and current time, and replacing the personalized context POIs with the POI addresses.

[0017] In the above implementation process, the abstract personalized context POI is accurately mapped to the specific navigable POI entity, avoiding navigation errors caused by ambiguity.

[0018] Furthermore, in some examples, after displaying the directed acyclic graph on the navigation interface, the method further includes: upon receiving the user's incremental correction voice command, performing semantic parsing on the incremental correction voice command to determine the target node to be corrected and the type of correction operation; based on the type of correction operation, while retaining the remaining nodes in the directed acyclic graph except for the target node and their temporal dependencies, performing a local rearrangement or local replacement operation on the target node to obtain an updated directed acyclic graph.

[0019] In the above implementation process, incremental local correction based on voice is implemented, which enables users to easily modify the navigation path, thereby improving the friendliness of navigation interaction and user experience.

[0020] Furthermore, in some examples, after displaying the directed acyclic graph on the navigation interface, the method further includes: obtaining real-time navigation progress; and updating the state of the corresponding node in the navigation interface according to the real-time navigation progress.

[0021] In the above implementation process, during navigation, the real-time navigation progress is transmitted back to the navigation interface, and the navigation interface automatically updates the status of the reached nodes, thereby providing users with clear and specific feedback on the trip progress.

[0022] Secondly, this application provides a vehicle navigation device, comprising: an analysis module, configured to acquire a user's navigation voice command, and obtain multiple candidate destinations and temporal dependencies between the multiple candidate destinations by performing speech recognition and intent analysis on the navigation voice command; a query module, configured to, when the multiple candidate destinations contain fuzzy attribute POIs, extract attribute query conditions from the fuzzy attribute POIs based on a pre-trained large language model, perform a retrieval in a POI information database according to the attribute query conditions, obtain a target POI based on the retrieval results, and replace the fuzzy attribute POIs with the target POIs; and a generation module, configured to, according to the temporal dependencies, combine the replaced candidate destinations to generate a directed acyclic graph, and display the directed acyclic graph on a navigation interactive interface.

[0023] Thirdly, this application provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method described in any of the first aspects.

[0024] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in any of the first aspects.

[0025] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method described in any of the first aspects.

[0026] Other features and advantages disclosed in this application will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the above-described technology disclosed in this application.

[0027] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart illustrating a vehicle navigation method provided in this application embodiment; Figure 2 A schematic diagram illustrating the workflow of a closed-loop scheme for fuzzy navigation intent recognition and execution based on multimodal fusion and synchronized map display in an intelligent cockpit, provided in an embodiment of this application; Figure 3 A block diagram of a vehicle navigation device provided in an embodiment of this application; Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0031] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] With the rapid popularization of smart cockpits and in-vehicle voice assistants, users are increasingly inclined to express their navigation needs using natural language rather than precise addresses when driving or riding in a vehicle. Most smart cockpit systems in related technologies are designed for single interactive commands, meaning they can only recognize and process a single destination query. However, when a voice command contains multiple destinations with temporal dependencies and vague attribute descriptions, the smart cockpit system easily overlooks these temporal dependencies and struggles to understand the user's intent, thus failing to provide accurate navigation routes.

[0033] To address the aforementioned issues, this application provides a vehicle navigation solution. By performing voice recognition and intent analysis on the user's navigation voice commands, it identifies all included destinations and the temporal dependencies between them, generating a directed acyclic graph (DAG) to explicitly model the temporal dependencies between multiple destinations. Furthermore, for fuzzy attribute Points of Interest (POIs), it accurately identifies the target POI indicated by the user's attribute description through attribute query condition extraction and POI information database retrieval mechanisms. This improves navigation accuracy.

[0034] The embodiments of this application will be described below: like Figure 1 As shown, Figure 1 This is a flowchart illustrating a vehicle navigation method provided in an embodiment of this application. The method can be applied to an intelligent cockpit system, which may include components such as an in-vehicle infotainment system (including audio, navigation, and communication functions), and a voice assistant. The method includes: Step 101: Obtain the user's navigation voice command, and obtain multiple candidate destinations and the temporal dependency relationship between the multiple candidate destinations by performing voice recognition and intent analysis on the navigation voice command; The navigation voice commands mentioned in this step can be commands issued by the user to the intelligent cockpit system via voice interaction while the vehicle is in motion, containing the user's navigation intent. During vehicle operation, the intelligent cockpit system collects the user's navigation voice commands through an in-vehicle microphone array, then converts them into text using an ASR (Automatic Speech Recognition) engine. Finally, based on a pre-trained large language model, the system performs intent slot filling on the text, thereby obtaining a set of candidate destinations and the temporal dependencies between them.

[0035] In implementation, the parsing process of the large language model can include intent analysis, entity recognition, type labeling, and temporal relation extraction. Taking a user's voice command as "First go to the roundest lake in Shanghai, then go to the tallest building to see the night view, and finally navigate home" as an example, the large language model, through parsing, outputs the following three candidate destinations and temporal dependencies: the first candidate destination is "the roundest lake in Shanghai," with the temporal keyword "first"; the second candidate destination is "the tallest building to see the night view," with the temporal keyword "then"; and the third candidate destination is "home," with the temporal keyword "finally." This lays a good foundation for subsequent explicit modeling of multi-destination temporal dependencies. Furthermore, the large language model can be any type of large language model, such as GPT-3 or Claude; its specific training process can be found in relevant technologies, which will not be elaborated upon in this application.

[0036] Step 102: When the multiple candidate destinations include fuzzy attribute POIs, extract attribute query conditions from the fuzzy attribute POIs based on the pre-trained large language model, search in the POI information database according to the attribute query conditions, obtain the target POI according to the search results, and replace the fuzzy attribute POI with the target POI. The fuzzy attribute POI (Point of Interest) mentioned in this step refers to a location point described by attribute features, such as "the roundest lake" and "the tallest building" mentioned in the previous example. Related technologies heavily rely on manual thesaurus or shallow rules for recalling fuzzy attribute POIs, resulting in limited coverage and a lack of attribute-based parsing for superlative and comparative adjectives like "roundest" and "tallest," which require global ranking. Therefore, this embodiment identifies the expression type of candidate destinations. When a fuzzy attribute POI is identified, the intelligent cockpit system inputs a text fragment related to that fuzzy attribute POI from the navigation voice command into a large language model. The large language model extracts key information from this text fragment as attribute query conditions. Based on these query conditions, the intelligent cockpit system searches the POI information database to obtain several candidate POIs. Then, it selects the one that best matches the user's intent from these candidate POIs as the target POI, replacing the fuzzy attribute POI with the target POI, thereby achieving attribute disambiguation of the fuzzy attribute POI.

[0037] In some embodiments, the attribute query conditions mentioned in this step may include an attribute name, a comparison operator, and an attribute value; the attribute name represents the structured attribute of the fuzzy attribute POI; the comparison operator represents a comparison relationship; the attribute value represents the target value of the comparison operator; the POI information base is a POI knowledge graph; each POI node in the POI knowledge graph includes a name, coordinates, category, and structured attributes. In other words, the large language model identifies the attribute name, comparison operator, and attribute value in the text segment corresponding to fuzzy attribute POIs, and outputs the attribute query conditions in the form of triplets. The attribute name is the structured attribute of the POI to be compared, such as shape roundness, height, popularity, night view index, distance, rating, etc.; the comparison operator is the comparison relationship, such as maximum, minimum, greater than, less than, or equal to; and the attribute value is the target value of the comparison operator. When the comparison operator is an extreme value operator like maximum or minimum, the attribute value is null (i.e., indicating that the extreme value of the attribute is taken in the filtered candidate set). When the comparison operator is a threshold operator like greater than, less than, or equal to, the attribute value takes a specific numerical value or range. For example, when the attribute query condition is (attribute name = rating, comparison operator = greater than, attribute value = 4.5), it means recalling POIs with a rating higher than 4.5. In addition, the attribute query conditions for fuzzy attribute POIs can also include filtering conditions. For example, when the fuzzy attribute... When the POI attribute is "the roundest lake in Shanghai", the attribute query conditions can be (attribute name = shape roundness, comparison operator = maximum, attribute value = null, filter condition = {category = lake, administrative region = Shanghai}). After obtaining the attribute query conditions in the form of attribute key-value pairs, the attribute query can be performed in the POI knowledge graph. The POI knowledge graph can be considered a semantic network specifically designed for POI data, which connects entities such as restaurants, hotels, and attractions on the map, along with their attributes and relationships, in a structured manner. Using the previous example, during the retrieval, we can first filter out all lakes located in Shanghai in the POI knowledge graph using the filter conditions, then perform an attribute query using the attribute name "shape roundness" as the sort key, thus obtaining multiple candidate lakes such as Dishui Lake and Dianshan Lake. Finally, based on the comparison operator and attribute value, Dishui Lake, with the highest shape roundness score, is determined as the target POI. This improves the recall rate of fuzzy attribute POIs.

[0038] In some embodiments, obtaining the target POI based on the search results mentioned in this step may include: sorting the search results and selecting a preset number of POI nodes at the top of the sort as a candidate POI sequence; obtaining the candidate total score of each candidate POI in the candidate POI sequence based on the user's gaze direction, gesture, current vehicle position, and user profile when issuing the navigation voice command; wherein the candidate total score is used to measure the degree of matching between the candidate POI and the user's navigation intent; and determining the candidate POI with the highest candidate total score as the target POI. That is, after obtaining multiple POI nodes that match the attribute query conditions through attribute query, the retrieved multiple POI nodes can be sorted in ascending or descending order based on comparison operators, and then the top K POI nodes are selected as a candidate POI sequence. Then, based on the user's gaze direction, gesture, current vehicle position, and user profile, a candidate total score is calculated for each candidate POI, and the candidate POI with the highest candidate total score is determined as the specific POI corresponding to the fuzzy attribute POI, i.e., the target POI. Thus, when the confidence level of a single voice channel is low due to unclear speech or cabin noise, robust candidate ranking can be provided by leveraging modalities such as gaze, gestures, and vehicle position, thereby reducing the false recognition rate. The preset number K can be set according to the specific needs of the scenario, and this application does not impose any restrictions on it.

[0039] Optionally, obtaining the candidate total score for each candidate POI in the candidate POI sequence based on the user's gaze direction, gestures, current vehicle location, and user profile when issuing the navigation voice command, as mentioned above, may include: calculating a semantic similarity score, alignment score, matching score, and geographic distance score for any candidate POI in the candidate POI sequence; wherein, the semantic similarity score is determined based on the semantic similarity between the candidate POI and the attribute query conditions; the alignment score is determined based on the alignment between the candidate POI and the user's gaze direction or gestures; the matching score is determined based on the matching degree between the candidate POI and the user's user profile; the geographic distance score is determined based on the distance between the candidate POI and the current vehicle location; and the weighted combination of the semantic similarity score, the alignment score, the matching score, and the geographic distance score is determined as the candidate total score for the candidate POI.

[0040] In other words, the first candidate POI sequence Total candidate score for each candidate POI This can be expressed as the following formula:

[0041] in, Indicates candidate POIs and user queries The semantic similarity score can be calculated by the language model on the vehicle's infotainment system. Indicates the candidate POI and the user's gaze / gesture direction. The alignment score is calculated as follows: for example, when the user's gaze is directed at the candidate POI, the alignment score is high, such as 0.8, and when there is no effective gaze, the alignment score is 0. Indicating candidate POIs and user profiles Matching score, this user profile This includes the user's historical access records and preferences; Indicates the candidate POI and the vehicle's current location. The geographical distance score is calculated, with higher values ​​for closer distances. All items are normalized to a range of 0 to 1, and then weighted and summed to obtain the candidate total score. , , and The weights for each item can be learned through small sample learning in the cockpit scenario. This allows for accurate parsing of target locations described by attribute features. Furthermore, when the total candidate score of all candidate POIs in the candidate POI sequence is below a preset threshold, the candidate POI sequence can be displayed in the navigation interface, with the candidate POI with the highest total score highlighted, awaiting user confirmation via voice or key press.

[0042] In some embodiments, before obtaining the total candidate score of each candidate POI in the candidate POI sequence mentioned above, the method may further include: filtering the candidate POI sequence based on viewing angle constraints, time window constraints, and access constraints; wherein, the viewing angle constraints are used to filter candidate POIs whose viewing angle does not meet the user's intention; the time window constraints are used to filter candidate POIs that are not in the accessible or optimal time period; and the access constraints are used to filter candidate POIs that the current vehicle cannot reach. In other words, when disambiguating POIs with fuzzy attributes, implicit spatiotemporal constraints can be introduced. These include viewing angle constraints, time window constraints, and accessibility constraints. Viewing angle constraints restrict the viewing direction or angle of the POI, filtering out candidates whose viewing angle does not meet the user's intent during recall and ranking. For example, when a fuzzy attribute POI includes "watching the sunset," the viewing angle constraint restricts the POI's viewing direction or angle to "westward" or "facing the river." Time window constraints limit the POI's accessibility or optimal experience time range, filtering candidates that are not accessible or in their optimal time period based on the current time. Accessibility constraints restrict the POI's accessibility and traffic conditions, such as restricted / prohibited driving times, one-way streets, and parking availability, excluding candidates that are currently inaccessible or unsuitable for travel. These three types of spatiotemporal constraints work together to recall and re-rank POIs with fuzzy attributes, ensuring that the returned results satisfy the user's fuzzy navigation intent in terms of spatial perspective, time, and accessibility feasibility.

[0043] In practical applications, candidate destinations can be expressed not only as precise POIs (i.e., POI addresses with precise address representations) and fuzzy attribute POIs, but also as personalized contextual POIs. These are POI addresses related to the user's personal characteristics and real-time context, such as "company" or "home." Therefore, in some embodiments, the method may further include: when personalized contextual POIs are included among the multiple candidate destinations, based on the user profile, the vehicle's current location, and the current time, parsing and mapping the personalized contextual POI to the corresponding POI address, and replacing the personalized contextual POI with the POI address. That is, for personalized contextual POIs, the intelligent cockpit system can call the user profile combined with the vehicle's current location and current time to perform contextual disambiguation. For example, when the personalized contextual POI is "company," the user profile is called to find the company address. If the user profile contains several company office addresses, then based on the current vehicle location and current time, the company office address that is closest to the current vehicle location and that the user has visited frequently during the current time period is selected, and "company" is mapped to the selected company office address. In this way, abstract personalized context POIs are accurately mapped to specific navigable POI entities, avoiding navigation errors caused by ambiguity.

[0044] Step 103: Based on the temporal dependency relationship, combine the replaced candidate destinations to generate a directed acyclic graph, and display the directed acyclic graph on the navigation interaction interface.

[0045] In this embodiment, a Directed Acyclic Graph (DAG) is generated based on the temporal dependencies between candidate destinations. The nodes of the DAG are disambiguated Points of Interest (POIs), and the directed edges represent temporal constraints. The DAG is then rendered onto the navigation interface, which can be a smart cockpit HMI (Human Machine Interface) map interface. This visually displays the temporal dependencies between multiple destinations in the form of a DAG, providing users with a complete navigation path and sequence.

[0046] In some embodiments, displaying the directed acyclic graph on the navigation interface mentioned in this step may include: displaying each node in the directed acyclic graph as a differentiated icon, displaying each directed edge in the directed acyclic graph as a visual path line, and listing the candidate POI sequence next to the node corresponding to the fuzzy attribute POI. In other words, when rendering the DAG to the HMI map interface, each node in the DAG can be displayed with a differentiated icon. For example, the node corresponding to an intermediate station can be displayed as a blue pin, while the node corresponding to the destination can be displayed as a flag icon. The directed edges of the DAG are displayed as visual path lines, which can be obtained through path planning based on the addresses of the two nodes connected by the directed edges. Next to the node corresponding to the fuzzy attribute POI, its candidate POI sequence is listed. At this time, the target POI corresponding to the fuzzy attribute POI is highlighted, while other candidate POIs are displayed in a lighter color. When the user selects a lightly displayed candidate POI on the HMI map interface, the intelligent cockpit system upgrades it to highlighted, downgrades the original target POI to a light color, and refreshes the DAG and navigation path lines in real time. This improves the user-friendliness of navigation.

[0047] In some embodiments, after this step, the method may further include: upon receiving the user's incremental correction voice command, performing semantic parsing on the incremental correction voice command to determine the target node to be corrected and the correction operation type; based on the correction operation type, while retaining the remaining nodes in the directed acyclic graph (DAG) except for the target node and their temporal dependencies, performing a local rearrangement or local replacement operation on the target node to obtain an updated DAG. That is, after displaying the DAG on the HMI map interface, the intelligent cockpit system can accept incremental voice correction from the user through a conversational confirmation mechanism. For example, if the second stop displayed in the original DAG was "museum," when the user's incremental correction voice command "change the second stop to science museum" is received, the ASR converts the speech to text, the large language model recognizes the correction intent, and thus determines that the target node to be corrected is the node in the DAG corresponding to "museum," and the correction operation type is replacement. The intelligent cockpit system retrieves the address corresponding to "science museum" in the POI knowledge graph, and replaces the target node with "science museum" while retaining the remaining nodes in the DAG, thereby obtaining an updated DAG. This enables incremental local correction based on voice, allowing users to easily modify navigation routes, thereby improving the friendliness of navigation interaction and user experience.

[0048] In some embodiments, after this step, the method may further include: obtaining real-time navigation progress; and updating the status of the corresponding node in the navigation interface according to the real-time navigation progress. That is, during navigation, the real-time navigation progress is transmitted back to the HMI map interface. When the vehicle actually arrives at a destination, the HMI automatically switches the pin of the arrived node to a completion icon. Through this visual change, clear and explicit feedback on the trip progress is provided to the user.

[0049] In this embodiment, the user's navigation voice commands are acquired. By performing speech recognition and intent analysis on the commands, all destinations and their temporal dependencies are identified, generating a directed acyclic graph (DAG). This enables explicit modeling of the temporal dependencies between multiple destinations. Furthermore, for fuzzy attribute Points of Interest (POIs), attribute query condition extraction and POI information database retrieval mechanisms accurately identify the target POIs pointed to by the fuzzy attribute descriptions in the navigation voice commands. This effectively improves navigation accuracy.

[0050] To provide a more detailed explanation of the solution in this application, a specific embodiment is described below: This embodiment provides a closed-loop solution for fuzzy navigation intent recognition and execution in an intelligent cockpit based on multimodal fusion and synchronized map display. The workflow of this solution is as follows: Figure 2 As shown, it includes: S201, Multimodal data acquisition; Specifically, the microphone array in the smart cockpit collects user voice at a sampling rate of 16 kHz; the in-vehicle driver monitoring camera collects gaze direction at 30 fps; and the multimodal acquisition module simultaneously reads the vehicle's current location, current timestamp, and cloud-based user profile. S202, Intended slot filling; Specifically, the intelligent cockpit system uses ASR to convert user speech into text, and then uses an intent slot filling module to perform intent analysis on the text. Assuming the text is "First go to the roundest lake in Shanghai, then go to the tallest building to see the night view, and finally navigate home," the intent slot filling module uses a pre-trained large language model as the backbone network and outputs the following three candidate destinations and temporal relationship labels: Node 1 is "the roundest lake in Shanghai," with the temporal keyword "first," and the type is a fuzzy attribute POI; Node 2 is "the tallest building to see the night view," with the temporal keyword "then," and the type is a fuzzy attribute POI + implicit spatiotemporal constraints; Node 3 is "home," with the temporal keyword "finally," and the type is a personalized context POI. S203, Fuzzy Attribute POI Recall; Specifically, for node 1, the attribute key-value pair is extracted (attribute name = shape roundness, comparison operator = maximum, attribute value = null, filter condition = {category = lake, administrative region = Shanghai}). Based on this attribute key-value pair, the attribute is queried and sorted in the POI knowledge graph, returning 5 candidate lakes. The first-ranked candidate lake is "Dianshui Lake" with a shape roundness score of 0.96, and the second-ranked candidate lake is "Dianshan Lake" with a shape roundness score of 0.71. This shape roundness score can be calculated offline from the boundary polygon of the POI obtained from the map data source, calculated according to the roundness formula, normalized to the range of 0 to 1, and then pre-stored in the attribute field of the POI knowledge graph. For node 2, extract the attribute key-value pairs (attribute name = height, comparison operator = maximum, attribute value = null, filter condition = {category = high-rise building / observation platform}), and obtain candidate POIs including "Shanghai Tower", "Jin Mao Tower" and "Oriental Pearl Tower" in the POI knowledge graph. Then, overlay the night view index to sort them. At this time, the candidate POI ranked first is "Shanghai Tower Observation Platform". S204, Personalized Context POI Disambiguation; Specifically, for node 3, the home address field in the user profile is called directly for disambiguation; S205, multimodal confidence fusion; Specifically, for candidate POIs in S203, multimodal confidence is calculated based on the user's gaze direction, gestures, the vehicle's current location, and historical preferences. This multimodal confidence is obtained through Bayesian fusion or attention weighting. In implementation, the multimodal confidence of a candidate POI is obtained by weighted fusion of the linguistic semantic similarity score between the candidate POI and the user's query, the alignment score between the candidate POI and the user's gaze direction or gesture direction, the matching score between the candidate POI and the user's preferences, and the geographical distance score between the candidate POI and the vehicle's current location. For example, if a user says "Take me to the nearest convenience store with the most positive reviews" in a noisy car cabin, the ASR output may contain homophonic ambiguity due to background noise. If the user's gaze direction is consistently pointing to the convenience store about 80 meters to the right front of the vehicle, the alignment score will be non-zero. Based on the multimodal confidence, the convenience store to the right front is ultimately selected as the target POI, highlighted in the HMI, and confirmation is initiated. S206, Sequence Dependency Graph Construction; Specifically, the three nodes are used to generate a linear DAG based on time-series keywords: "Drip Lake → Shanghai Tower Observation Deck → Home"; S207, Implicit Spatiotemporal Constraint Reasoning; Specifically, the reasoning for "viewing the night view" at node 2 is based on the time constraint "1 hour to 4 hours after sunset". If the current time is 2 PM, the ETA (Estimation Time of Arrival) verification suggests arriving at node 2 after 5:30 PM. The intelligent cockpit system embeds this constraint as a soft constraint into the path planning. S208 and HMI map are rendered simultaneously; Specifically, the intelligent cockpit system synchronously renders the DAG to the HMI map interface, and the rendering result is as follows: Figure 2 As shown, the HMI map interface displays the currently selected DAG using three path lines: "Vehicle's current location → Dishui Lake (intermediate station, blue pin) → Shanghai Tower Observation Deck (intermediate station, blue pin) → Home (destination, flag icon)". Next to each node, the corresponding candidate POI is listed. The candidate POI is displayed as a light-colored transparent pin. For example, node 1 displays three items: "Dishui Lake", "Dianshan Lake" and "Mingzhu Lake". The currently selected item "Dishui Lake" is highlighted. After displaying the DAG, the system receives visual reverse disambiguation operations from the user on the HMI map interface. These operations include, but are not limited to: clicking on a node's alternative POI to replace the currently selected item; long-pressing and dragging a node to the replacement location via touchscreen; deleting or merging nodes using gestures (pointing / swiping); and fine-tuning attribute query conditions by entering text on the HMI. Continuing with the previous example, when the user clicks on the light-colored transparent pin corresponding to "Dianshan Lake" on the HMI map, the system automatically upgrades it to highlight, downgrades "Dishui Lake" to light color, and instantly refreshes the DAG and navigation path lines. S209, Incremental voice correction; Specifically, when the user issues an incremental correction command via voice, "Change to one that is a little higher," the intelligent cockpit system locates node 2 based on contextual memory, sorts and removes the selected POIs by height, returns a new candidate and replaces node 2 in the DAG, while the other nodes remain unchanged; the HMI map simultaneously highlights and shakes to indicate the update of node 2. Alternatively, when a user issues an incremental correction command via voice, "Add some gas before going to the lake," the intelligent cockpit system inserts a new node "gas station" into the DAG (which can be obtained by performing an attribute query in the POI knowledge graph combined with minimum distance sorting). Subsequently, when the user issues another incremental correction command, "Don't include the last part," the intelligent cockpit system deletes the node corresponding to "home" in the DAG and repairs the temporal edge to "current vehicle location → gas station → Dishui Lake → Shanghai Tower observation deck." S210, Execution confirmation and navigation closed loop; Specifically, when a user issues a confirmation command within a preset time threshold (default 8 seconds), the intelligent cockpit system initiates multi-destination integrated navigation. During navigation, it transmits the current progress back to the HMI map interface in real time, and the HMI automatically switches the pin points of arrived nodes to completion icons. This confirmation command can be triggered by the user via voice or HMI physical buttons. While driving towards node 2, if the user issues an incremental correction command via voice, such as "Don't go to the observation deck, go straight home," the intelligent cockpit system deletes node 2, rearranges the DAG, and adjusts the navigation path in real time without interrupting the current driving.

[0051] This embodiment offers at least the following advantages: First, by using explicit fuzzy attribute POI recall and knowledge graph ranking based on attribute key-value pairs, it accurately handles superlative and descriptive adjective queries, improving the fuzzy attribute POI recall rate. Second, it provides robust candidate ranking even in scenarios with unclear speech or noisy cabins, reducing the false recognition rate. Third, by explicitly constructing a multi-destination sequence DAG, it can identify multiple destinations with temporal constraints in a single command, avoiding multiple rounds of sequential expression and significantly improving interaction efficiency. Fourth, through bidirectional synchronization between the HMI map interface and the DAG, users can intuitively verify and revise their intent understanding results on the map using various confirmation methods. Fifth, it supports issuing intent revision commands again while driving without interrupting the current navigation, improving driving safety and experience. Furthermore, it allows users to make local modifications to any node in the sequence during multi-round interactions without disrupting the rest of the path, effectively improving the user experience.

[0052] Corresponding to the embodiments of the aforementioned methods, this application also provides embodiments of vehicle navigation devices and terminals for their applications: like Figure 3 As shown, Figure 3 This is a block diagram of a vehicle navigation device provided in an embodiment of this application. The device includes: Analysis module 31 is used to acquire the user's navigation voice commands, and to obtain multiple candidate destinations and the temporal dependencies between the multiple candidate destinations by performing speech recognition and intent analysis on the navigation voice commands. The query module 32 is used to extract attribute query conditions from the fuzzy attribute POI based on a pre-trained large language model when the multiple candidate destinations contain fuzzy attribute POIs, perform a retrieval in the POI information database according to the attribute query conditions, obtain the target POI according to the retrieval results, and replace the fuzzy attribute POI with the target POI. The generation module 33 is used to generate a directed acyclic graph by combining the replaced candidate destinations according to the temporal dependency relationship, and to display the directed acyclic graph on the navigation interaction interface.

[0053] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0054] This application also provides an electronic device, please refer to [link to application]. Figure 4 , Figure 4This is a structural block diagram of an electronic device provided in an embodiment of this application. The electronic device may include a processor 410, a communication interface 420, a memory 430, and at least one communication bus 440. The communication bus 440 is used to enable direct communication between these components. In this embodiment, the communication interface 420 of the electronic device is used for signaling or data communication with other node devices. The processor 410 may be an integrated circuit chip with signal processing capabilities.

[0055] The processor 410 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor, or the processor 410 can be any conventional processor.

[0056] The memory 430 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The memory 430 stores computer-readable instructions. When these computer-readable instructions are executed by the processor 410, the electronic device can perform the aforementioned operations. Figure 1 The various steps involved in the method implementation examples.

[0057] Alternatively, the electronic device may also include a storage controller and an input / output unit.

[0058] The memory 430, storage controller, processor 410, peripheral interface, and input / output unit are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses 440. The processor 410 is used to execute executable modules stored in the memory 430, such as software function modules or computer programs included in electronic devices.

[0059] The input / output unit is used to provide users with the ability to create tasks and to set optional start periods or preset execution times for those tasks, thereby enabling user-server interaction. The input / output unit may be, but is not limited to, a mouse and keyboard.

[0060] Understandable. Figure 4 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof.

[0061] This application also provides a storage medium storing instructions. When the instructions are run on a computer, the computer program is executed by a processor to implement the method described in the method embodiment. To avoid repetition, the method will not be described again here.

[0062] This application also provides a computer program product that, when run on a computer, causes the computer to perform the method described in the method embodiment.

[0063] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0064] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0065] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0067] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0068] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A vehicle navigation method, characterized in that, include: The system obtains the user's navigation voice commands, performs speech recognition and intent analysis on the navigation voice commands, and obtains multiple candidate destinations and the temporal dependencies between the multiple candidate destinations. When multiple candidate destinations include fuzzy attribute POIs, attribute query conditions are extracted from the fuzzy attribute POIs based on a pre-trained large language model. The POI information database is searched according to the attribute query conditions. The target POI is obtained based on the search results and the fuzzy attribute POI is replaced with the target POI. Based on the temporal dependencies, the replaced candidate destinations are combined to generate a directed acyclic graph, and the directed acyclic graph is displayed on the navigation interface.

2. The method according to claim 1, characterized in that, in, The attribute query conditions include: attribute name, comparison operator, and attribute value; the attribute name represents the structured attribute of the fuzzy attribute POI; the comparison operator represents the comparison relationship; and the attribute value represents the target value of the comparison operator. The POI information database is a POI knowledge graph; each POI node in the POI knowledge graph includes a name, coordinates, category, and structured attributes.

3. The method according to claim 2, characterized in that, Obtaining the target POI based on the search results includes: The search results are sorted, and a preset number of POI nodes at the top of the sort are selected as candidate POI sequences. Based on the user's gaze direction, gestures, current vehicle location, and user profile when issuing the navigation voice command, the candidate total score of each candidate POI in the candidate POI sequence is obtained; wherein, the candidate total score is used to measure the degree of matching between the candidate POI and the user's navigation intent; The candidate POI with the highest total score is determined as the target POI.

4. The method according to claim 3, characterized in that, The process of obtaining the candidate total score for each candidate POI in the candidate POI sequence based on the user's gaze direction, gestures, current vehicle location, and user profile when the user issues the navigation voice command includes: For any candidate POI in the candidate POI sequence, calculate the semantic similarity score, alignment score, matching score, and geographic distance score of the candidate POI; wherein, the semantic similarity score is determined based on the semantic similarity between the candidate POI and the attribute query conditions; the alignment score is determined based on the alignment between the candidate POI and the user's gaze direction or gesture; the matching score is determined based on the matching degree between the candidate POI and the user's user profile; and the geographic distance score is determined based on the distance between the candidate POI and the vehicle's current location; The weighted combination of the semantic similarity score, the alignment score, the matching score, and the geographical distance score is determined as the candidate total score of the candidate POI.

5. The method according to claim 3, characterized in that, Before obtaining the total candidate score of each candidate POI in the candidate POI sequence, the method further includes: The candidate POI sequence is filtered based on viewing angle constraints, time window constraints, and access constraints. The viewing angle constraints are used to filter candidate POIs whose viewing angle does not meet the user's intention; the time window constraints are used to filter candidate POIs that are not in the accessible or optimal time period; and the access constraints are used to filter candidate POIs that the current vehicle cannot reach.

6. The method according to claim 3, characterized in that, The step of displaying the directed acyclic graph on the navigation interface includes: In the navigation interface, each node in the directed acyclic graph is displayed with a differentiated icon, each directed edge in the directed acyclic graph is displayed as a visual path line, and the candidate POI sequence is listed next to the node corresponding to the fuzzy attribute POI.

7. The method according to claim 1, characterized in that, Before generating a directed acyclic graph by combining the replaced candidate destinations according to the temporal dependency, the method includes: When a personalized context POI is included in the multiple candidate destinations, the personalized context POI is parsed and mapped to the corresponding POI address based on the user profile, the vehicle's current location, and the current time, and the personalized context POI is replaced with the POI address.

8. The method according to claim 1, characterized in that, After displaying the directed acyclic graph on the navigation interface, the method further includes: Upon receiving the user's incremental correction voice command, the incremental correction voice command is semantically parsed to determine the target node to be corrected and the type of correction operation. Based on the type of correction operation, while retaining the remaining nodes in the directed acyclic graph except for the target node and their temporal dependencies, a local rearrangement or local replacement operation is performed on the target node to obtain an updated directed acyclic graph.

9. The method according to claim 1, characterized in that, After displaying the directed acyclic graph on the navigation interface, the method further includes: Get real-time navigation progress; Update the status of the corresponding node in the navigation interface based on the real-time navigation progress.

10. A vehicle navigation device, characterized in that, include: The analysis module is used to acquire the user's navigation voice commands, and to obtain multiple candidate destinations and the temporal dependencies between the multiple candidate destinations by performing speech recognition and intent analysis on the navigation voice commands. The query module is used to extract attribute query conditions from the fuzzy attribute POIs based on a pre-trained large language model when the multiple candidate destinations contain fuzzy attribute POIs, perform a retrieval in the POI information database according to the attribute query conditions, obtain the target POI according to the retrieval results, and replace the fuzzy attribute POI with the target POI. The generation module is used to combine the replaced candidate destinations to generate a directed acyclic graph based on the temporal dependency relationship, and to display the directed acyclic graph on the navigation interaction interface.

Images