Intelligent on-way sensing method and system

Abstract

The invention relates to an intelligent along-the-way sensing method and system, and relates to the technical field of vehicle-mounted intelligent navigation, and the method comprises the following steps: when a sensing system of a vehicle-mounted terminal is in an activated state, receiving a query instruction input by a user; performing intention recognition on the text information to extract a directional word and a target POI type in the query instruction; based on the vehicle orientation data, the angle deviation value of the direction word and a preset angle range, calculating a starting angle and an ending angle of the fan-shaped search area, and generating polygon vertex coordinates of the fan-shaped search area by taking the current position of the vehicle-mounted terminal as a starting point; taking the polygon vertex coordinates of the fan-shaped search area and the target POI type as search parameters, and calling a map API to screen matched POI data in the fan-shaped search area; and sorting the screened POI data to generate a result list. The method and the device have the effect of solving the defect of non-directional perception in POI search.

Description

Technical Field

[0001] This application relates to the technical field of vehicle-mounted intelligent navigation, and in particular to an intelligent route sensing method and system. Background Technology

[0002] As in-vehicle navigation systems upgrade from basic route planning to scenario-based services, POI (Point of Interest) search has become one of the core needs of users during the driving process.

[0003] The current mainstream in-vehicle POI search technology is implemented as follows: based on the GPS positioning of the vehicle terminal, the current location is obtained, a circular search range with a fixed radius is set, POI type keywords such as restaurants and gas stations are extracted from the user's query text, the map API is called to filter the target POI data within the circular range, and a result list is generated and fed back to the user according to the rule of distance from near to far.

[0004] In actual driving, when users initiate POI queries that typically include clear directional directions, such as a gas station ahead or a parking spot to the left rear, existing systems still only use distance as the sole filtering dimension, returning all target POIs within a 360° range of the current location. For example, when a user queries for a service area within 500 meters ahead, the system will still mix in service area data from directions other than the target direction, such as rear and left, resulting in a large amount of redundant data from non-target directions being included in the results. Users need to manually identify directional information, increasing operational costs and reducing user experience.

[0005] Solving this technical problem is a technical challenge that needs to be overcome by those skilled in the art. Summary of the Invention

[0006] To at least partially solve the above-mentioned technical problems, this application provides an intelligent roadside sensing method and system.

[0007] Firstly, the intelligent route sensing method provided in this application adopts the following technical solution.

[0008] When the in-vehicle terminal's perception system is active, it receives a query command input by the user; the query command includes text information and the in-vehicle terminal's current vehicle orientation data. The text information is subjected to intent recognition to extract directional words and target POI types from the query command; Based on the vehicle orientation data, the angle offset value of the direction word, and the preset angle range, the starting angle and ending angle of the fan-shaped search area are calculated, and the polygon vertex coordinates of the fan-shaped search area are generated with the current position of the vehicle terminal as the starting point; the polygon vertex includes the starting point, left boundary point, far left point, far right point, and right boundary point; the polygon vertices are connected in sequence to form a closed fan-shaped search area; Using the polygon vertex coordinates and target POI type of the fan-shaped search area as search parameters, the map API is called to filter matching POI data within the fan-shaped search area; The filtered POI data is sorted to generate a result list containing the Chinese name of the POI, relative direction, distance, and business hours. The result list is then fed back to the user through the display module or voice module of the vehicle terminal.

[0009] By adopting the above technical solution, a directional fan-shaped region is generated by calculating the vehicle orientation data and the directional word angle offset value, eliminating redundant POI data in non-target directions, and solving the defect of no direction perception.

[0010] Secondly, the intelligent roadside sensing system provided in this application adopts the following technical solution.

[0011] In summary, this application has at least the following technical effects: The first processing module is used to: receive a query command input by the user when the perception system of the vehicle terminal is in an active state; the query command includes text information and the current vehicle orientation data of the vehicle terminal. The second processing module is used to: perform intent recognition on the text information and extract directional words and target POI types from the query command; The third processing module is used to: calculate the starting angle and ending angle of the fan-shaped search area based on the vehicle orientation data, the angle offset value of the direction word, and the preset angle range, and generate the polygon vertex coordinates of the fan-shaped search area with the current position of the vehicle terminal as the starting point; the polygon vertex includes the starting point, the left boundary point, the far left point, the far right point, and the right boundary point; and connect the polygon vertices in sequence to form a closed fan-shaped search area. The fourth processing module is used to: use the polygon vertex coordinates and target POI type of the sector search area as search parameters, and call the map API to filter matching POI data within the sector search area; The fifth processing module is used to sort the filtered POI data, generate a result list containing the Chinese name of the POI, relative direction, distance, and business hours, and feed the result list back to the user through the display module or voice module of the vehicle terminal. Attached Figure Description

[0012] Figure 1 This is a flowchart of an intelligent route sensing method according to an embodiment of this application; Figure 2 This is a system block diagram of an intelligent roadside sensing system according to an embodiment of this application; In the diagram, 201 is the first processing module; 202 is the second processing module; 203 is the third processing module; 204 is the fourth processing module; and 205 is the fifth processing module. Detailed Implementation

[0013] The following is in conjunction with the appendix Figure 1-2 The present application will be further described with reference to specific embodiments: This application discloses an intelligent route sensing method, including the following steps: Step 101: When the perception system of the vehicle terminal is activated, receive the query command input by the user; the query command includes text information and the current vehicle orientation data of the vehicle terminal.

[0014] Specifically, the activation methods of the in-vehicle terminal's perception system include one or more of the following: Voice wake-up activation: The user speaks a preset wake-up word, such as "turn on route perception" or "start route search." After the voice recognition module of the in-vehicle terminal recognizes the wake-up word and the recognition confidence is ≥90%, the perception system is activated; Physical button activation: The user presses and holds the OK button on the steering wheel of the in-vehicle terminal for 2 seconds, or clicks the route search button in the "Navigation" interface of the central control screen to activate the perception system; Navigation linkage activation: When the navigation software of the in-vehicle terminal is running and the GPS module detects that the vehicle speed is below 60km / h, the system automatically activates the perception system, supporting the user to directly initiate route POI queries.

[0015] Once the perception system is activated, it acquires the current vehicle orientation data through the GPS module or gyroscope module of the vehicle terminal. The data sampling frequency is 1 time / second, and the average value of 3 consecutive samples is taken as the final vehicle orientation data, without the need for the user to manually input it.

[0016] Step 102: Perform intent recognition on the text information to extract directional words and target POI types from the query instructions.

[0017] Specifically, intent recognition refers to the process of semantically parsing user text information using a pre-defined large text language model; directional words refer to words used in user queries to describe spatial direction, selected from one or more of the following: front, left front, right front, back, left, right, left rear, and right rear; target POI type refers to the category of interest points that the user needs to search for, expressed in Chinese, selected from one or more of the following: restaurant, gas station, shopping mall, hospital, parking lot, service area, charging station, and convenience store.

[0018] Step 103: Based on vehicle orientation data, directional word angle offset values, and preset angle range, calculate the starting and ending angles of the fan-shaped search area and generate polygon vertex coordinates of the fan-shaped search area with the current position of the vehicle terminal as the starting point; the polygon vertices include the starting point, left boundary point, far left point, far right point, and right boundary point; connect the polygon vertices in sequence to form a closed fan-shaped search area.

[0019] Specifically, the angle offset value refers to the angle difference between the predefined direction term and the vehicle's orientation reference. The specific mapping relationship is as follows: "front" is mapped to 0°, "right front" is mapped to 45°, "right" is mapped to 90°, "right rear" is mapped to 135°, "rear" is mapped to 180°, "left rear" is mapped to 225°, "left" is mapped to 270°, and "left front" is mapped to 315°. The preset angle range refers to the angle coverage width of the fan-shaped search area, with a value of 30°-60°.

[0020] Step 104: Using the polygon vertex coordinates and target POI type of the sector search area as search parameters, call the map API to filter matching POI data within the sector search area.

[0021] Specifically, the map API is selected from one of the following: Gaode Map API, Baidu Map API, and Tencent Map API. When calling the map API, in addition to passing the coordinates of the polygon vertices of the fan-shaped search area and the target POI type, the current latitude and longitude data of the vehicle terminal also needs to be passed. The latitude and longitude data must be consistent with the starting latitude and longitude of the generated polygon vertex coordinates to ensure accurate positioning of the search area. If the POI data returned by the map API lacks the "business hours" field, the text big language model is called to generate general business hours reference information by combining the Chinese name of the POI, such as XX Convenience Store, XX Home-Style Restaurant, and address information. The general business hours reference information is added to the corresponding POI entry in the results list and marked with small gray text through the display module to distinguish it from the basic information of the POI, such as name, distance, and direction, to avoid user confusion.

[0022] Step 105: Sort the filtered POI data and generate a result list containing the Chinese name of the POI, relative direction, distance, and business hours. The result list is then fed back to the user through the display module or voice module of the vehicle terminal.

[0023] Specifically, the filtered POI data is sorted, which includes: first, filtering the POI data from multiple dimensions, and then sorting the filtered POI data according to priority; in the generated result list, each POI entry is additionally marked with its specific directional angle relative to the vehicle terminal.

[0024] The filtering criteria include: 1. Business status filtering: Based on the "opentime_today" field of the POI data, compare it with the current system time of the vehicle terminal, and only retain POIs that are currently in business hours; 2. Security attribute filtering: Based on the security label field of the POI data, remove POIs marked as risky locations or temporarily closed; 3. Vehicle type matching filtering: If the target POI type is a parking lot, obtain the current vehicle type data through the vehicle information module of the vehicle terminal, and only retain POIs that support parking for that vehicle type; The filtered POI data are sorted according to the following priorities: First priority: the straight-line distance between the POI and the vehicle terminal, sorted from nearest to farest; the distance is calculated using the Haversine algorithm by the difference between the latitude and longitude of the POI and the current location of the vehicle terminal; Second priority: user ratings of the POI, sorted from highest to lowest; only POIs with user ratings ≥ 4.0 are retained; if the ratings are the same, the number of ratings is compared, and the POI with more ratings is given priority; Third priority: the number of historical queries for the POI, sorted from most to least; the number of historical queries is the statistical value of the query records of the POI on the vehicle terminal in the past 3 months.

[0025] Specifically, firstly, addressing the deficiency of existing technologies in circular areas lacking direction awareness, this application generates directional fan-shaped areas by calculating vehicle orientation data and directional word angle offset values, eliminating redundant POI data from non-target directions and resolving the lack of direction awareness; secondly, it defines the search area using polygon vertex coordinates, preventing POIs from non-target spaces such as oncoming lanes and overpasses from being mixed in, thus solving the problem of uncontrolled search areas; and thirdly, it reduces user operation and judgment time and improves user experience by providing a full information list including the Chinese name of the POI, relative direction, and business hours, along with multimodal feedback.

[0026] As a specific implementation of an intelligent route sensing method, based on vehicle orientation data, angular offset values ​​of direction words, and a preset angle range, the starting and ending angles of a fan-shaped search area are calculated, and the polygon vertex coordinates of the fan-shaped search area are generated with the current position of the on-board terminal as the starting point. This includes: Establish a mapping relationship between directional terms and angle offset values; The starting angle is obtained by adding the angle offset value of the corresponding direction word to the vehicle orientation data and then subtracting half of the preset angle range. The ending angle is obtained by adding the angle offset value of the corresponding direction word to the vehicle orientation data, and then adding half of the preset angle range. Obtain the latitude and longitude of the current location of the vehicle terminal; Calculate the latitude and longitude of the left boundary point based on the starting angle, preset search distance, and the latitude and longitude of the current location; Calculate the latitude and longitude of the right boundary point based on the ending angle, preset search distance, and the latitude and longitude of the current location; Calculate the latitude and longitude of the far left point based on the starting angle, M times the preset search distance, and the latitude and longitude of the current location; The latitude and longitude of the far right point are calculated based on the ending angle, M times the preset search distance, and the current location's latitude and longitude.

[0027] The following example illustrates this: Assuming a user is driving a vehicle due north on a city road (vehicle orientation data 0°), the query is for "restaurant ahead" (direction term "ahead", target POI type "restaurant"), with a preset angle range of 30°, a preset search distance of 2km, and M=1.2.

[0028] Directional word mapping: "forward" corresponds to an angle offset value of 0°; Starting angle = 0° + 0° - 30° / 2 = -15° (i.e., 345°), ending angle = 0° + 0° + 30° / 2 = 15°; Current location latitude and longitude (lat0=39.904200°, lon0=116.407400°); Left boundary point (lat1, lon1): Calculated using the Haversine formula, with an initial angle of 345° and a distance of 2km, resulting in (39.921800°, 116.402000°). Right boundary point (lat2, lon2): End angle 15°, distance 2km, resulting in (39.922000°, 116.435000°); The distal left point (lat3,lon3): with an initial angle of 345° and a distance of 2.4km, we obtain (39.924200°, 116.400500°). The far right point (lat4, lon4): with an ending angle of 15° and a distance of 2.4km, we get (39.924500°, 116.438000°); finally, we connect "starting point → left boundary point → far left point → far right point → right boundary point → starting point" to form a closed fan-shaped area covering 30° north, 2km to the near side and 2.4km to the far side, which matches the search requirement of "restaurant ahead".

[0029] This application solves the problem of inconsistent direction and angle correspondence by using a unified mapping rule to ensure that the angle benchmarks of different directional words are consistent; it uses a formula of vehicle orientation + offset value ± half interval to make the fan-shaped angle range linked with the vehicle orientation in real time, solving the problem of fixed angle range; it uses the Haversine algorithm combined with the difference between near and far distances to generate boundary points, ensuring that the coordinates of each vertex correspond to the fan-shaped outline, solving the problem of blurred region boundaries; and it achieves matching of the fan-shaped search area with user direction requirements and vehicle driving status, improving the efficiency and practicality of POI search in driving scenarios.

[0030] As a specific implementation of an intelligent route sensing method, intent recognition is performed on text information to extract directional words and target POI types from query instructions, including: The text information is input into a preset large text language model and an entity extraction template is loaded. The entity extraction template includes: extracting directional words from the query, extracting the target POI type from the query, and determining whether image assistance is needed. After the large text language model outputs the extraction results, the validity of the results is verified; specifically, this includes: If no direction word is extracted, the first prompt message is played through the voice module and a direction selection button pops up in the display module; if the type of the extracted target POI is unclear, a POI type option list pops up; if the extracted direction word is not within the preset selectable values, the second prompt message is played.

[0031] Specifically, the "pre-defined text language model" refers to a natural language processing model pre-integrated into the vehicle terminal system. It is selected from one of GPT-3.5, Wenxin Yiyan, and Tongyi Qianwen. Its core function is to parse the semantic and entity information of natural language text and adapt to the query characteristics of "short text, colloquial, and contextual" in vehicle scenarios. For example, users often say "find the gas station ahead" instead of "please search for gas stations within 1km ahead of the current location".

[0032] The "Entity Extraction Template" refers to a set of information extraction rules customized for in-vehicle POI search needs, which includes three core tasks: ① extracting directional words in the query; ② extracting the target POI type in the query; ③ determining whether image assistance is needed.

[0033] After the text-based large language model extracts information using entity extraction templates, the output extraction results need to be validated to ensure that the directional words and target POI types meet the search parameter requirements. This is to avoid POI search deviations caused by extraction errors or ambiguity. The specific validation rules and processing methods are as follows: If the text-based large language model fails to extract any directional words from the text information (e.g., the user only enters "help me find a restaurant" or "are there any gas stations nearby?" without mentioning direction), the first prompt message will be played through the voice module of the in-vehicle terminal, and a direction selection button will pop up on the display module. The first prompt message refers to the voice prompt content used to guide the user to supplement the direction, specifically "Your query direction has not been identified. Please select from below or supplement the direction by voice." The direction selection button refers to the visual interactive control displayed on the central control screen. The button options are consistent with the 8 preset directional words in the entity extraction template. Users can select by touch or voice command without having to re-enter the complete query text.

[0034] If the target POI type extracted by the text big language model is ambiguous (e.g., the user enters "looking for food" or "is there a place to refuel up ahead?" without specifying the specific type such as "restaurant" or "gas station"), a POI type option list will pop up in the display module. The "POI type option list" refers to a set of visual options sorted by the high frequency of needs in the in-vehicle scenario. The list includes "restaurant, gas station, shopping mall, hospital, parking lot, service area, charging station, convenience store", and each option is marked with an icon to help users quickly identify it.

[0035] If the directional word extracted by the large text language model is not among the eight preset selectable values ​​in the entity extraction template, a second prompt message will be played through the voice module. The second prompt message is a voice message used to inform the user that the directional word is not supported and to guide the correct input. Specifically, it is "Directional queries such as 'diagonally forward' are not currently supported. Please select or supplement from preset directions such as 'forward, left forward, right forward', etc." At the same time, the preset directional word list will be displayed in the display module for the user to refer to and select.

[0036] This implementation addresses the issue of a single extraction dimension by using templates that include directional terms and POI types, ensuring the acquisition of complete parameters required for POI search. It also resolves search bias issues caused by invalid extraction results through verification rules such as prompts for selection when directional terms are not extracted, pop-up lists for ambiguous types, and corrections guided by non-standard directional terms. This improves the accuracy of text intent recognition.

[0037] As a specific implementation of an intelligent route sensing method, the query command may also include real-scene image data; when receiving a query command input by a user, the method further includes: S401. Determine whether the query command contains real-scene image data; if it does, execute S402. S402. Perform a validity pre-check on the real-scene image data; the validity pre-check includes: detecting the image clarity and identifying whether the image contains resolvable scene features; wherein, if the clarity is less than a preset threshold or the identified image does not contain resolvable scene features, the real-scene image is determined to be invalid; if the real-scene image is determined to be invalid, a third prompt message is played through the voice module. S403. If the real-scene image is determined to be valid, store the URL of the real-scene image data in the cache module and adjust the cache expiration time according to the image file size. S404. Call the preset visual language model and input the URL, text information and geographic environment information of the current location of the vehicle terminal into the visual language model; the visual language model outputs the parsing results; the parsing results include: image scene feature classification and a set of associated POI clues; the image scene feature classification is labeled by main scene and sub-scene; the associated POI clues in the set of associated POI clues include POI type, confidence score and association basis; S406. Filter valid leads with confidence scores greater than the preset confidence score from the set of associated POI leads; if there is a valid lead that is completely consistent with the target POI type and the confidence score of the valid lead is the highest among all valid leads, it is judged as a complete match; if there is no completely consistent lead, but there is an associated lead that contains the target POI type, it is judged as a partial match; if all valid leads are not associated with the target POI type, it is judged as a complete mismatch. S407. Execute differentiated POI search configuration based on the matching judgment result.

[0038] Specifically, the first step is to determine the data type of the query command entered by the user to confirm whether it contains real-scene image data. Real-scene image data refers to the surrounding scene images captured by the user through the vehicle terminal camera, such as roadside building signs, road markings, or the appearance of service areas, which are used to assist the text information in clarifying the search requirements. If the determination result is that it contains the data, then the image validity check is performed.

[0039] The acquired real-scene image data undergoes a validity pre-check. This check involves two core operations performed by the image processing module of the vehicle terminal: first, detecting image sharpness by using an edge detection algorithm to calculate the image edge sharpness value; if this value is less than a preset threshold, the image is considered blurry; second, identifying whether the image contains resolvable scene features. These features must include at least one element that can be used to associate with a Point of Interest (POI), such as building outlines, business signs, or road markings. If these features are absent, the image is considered invalid. If the image sharpness is less than the preset threshold or does not contain resolvable scene features, the real-scene image is deemed invalid. Upon invalidation, a third prompt message is played via the voice module, stating that the image is blurry or does not contain resolvable scene features and requesting a retake. Users are allowed to retry a maximum of one time. This step filters invalid images in advance, reducing wasted computing resources and the probability of parsing failure.

[0040] If the real-scene image is determined to be valid, the URL of the real-scene image data is stored in the cache module. The cache module is a hardware unit in the vehicle terminal used to store temporary data, supporting fast read and write and automatic expiration. At the same time, the cache expiration time is dynamically adjusted according to the image file size: when the image file size is greater than 5MB, because the image contains complex scene details that require longer parsing time, the cache expiration time is set to 120 seconds; when the image file size is less than or equal to 5MB, the image content is relatively simple, and the cache expiration time is set to 80 seconds. Adjusting the cache time can ensure parsing needs while avoiding long-term occupation of storage resources, balancing performance and resource consumption.

[0041] The system invokes a pre-defined visual language model, inputting the URL of the real-world image data, the text information queried by the user, and the geographic environment information of the vehicle terminal's current location. The visual language model is selected from doubao-1.5-vision-pro, GPT-4V, and Wenxin Yige, possessing dual capabilities of image understanding and text association. The geographic environment information is a description of the functional area to which the vehicle terminal's current location belongs, obtained through a map API, such as the core area of ​​a commercial district or the surrounding area of ​​a residential area along the K120 section of a highway, used to assist the model in optimizing the analysis results based on geographic location. The analysis results output by the visual language model include two core components: first, image scene feature classification, which is labeled at two levels: main scene and sub-scene, such as main scene being highway, sub-scene being service area, main scene being urban commercial district, and scene being shopping mall entrance; second, a set of associated POI clues, where each associated POI clue includes POI type, confidence score, and association basis, with POI type being Chinese descriptions such as restaurant or gas station.

[0042] From the set of associated POI clues, clues with a confidence score greater than a preset confidence score of 70 are selected as valid clues; clues with a score lower than this are discarded due to insufficient accuracy. Then, the valid clues are matched against the target POI type extracted from the text information. The matching results are divided into three categories: The first category is a complete match, meaning there is a valid clue that is completely consistent with the target POI type, and this clue has the highest confidence score among all valid clues. For example, if the target POI type is "restaurant," the valid clues include "restaurant" and have the highest confidence score. The second category is a partial match, meaning there is no clue that is completely consistent with the target POI type, but there are related clues that contain the target POI type. For example, if the target POI type is "restaurant," the valid clues include restaurants in service areas or shopping malls. The third category is a complete mismatch, meaning all valid clues are not related to the target POI type. For example, if the target POI type is "gas station," the valid clues only include hospitals and supermarkets.

[0043] If the query command does not contain real-world image data, a preset text language model is invoked to identify whether the text contains scene-dependent descriptions, such as "the building in front of this building" or "this shop on the roadside." If such descriptions are found, it is determined that image assistance is needed. An image upload button with shooting instructions pops up on the display module. The instructions are: "Please take a picture of the target scene within 100 to 400 meters in front of you. It is recommended to take a picture of the front of the building or the signboard of the business." The same instructions are broadcast simultaneously through the voice module. If the user does not upload an image within 10 seconds, it is determined that image assistance is not needed, and a POI search is performed.

[0044] As one implementation of an intelligent route-following perception method, a differentiated POI search configuration is performed based on the matching judgment result, including: If the matching result is a perfect match, then the polygon vertex coordinates of the sector search area, the target POI type, the POI scene label matching parameters, and the POI cue confidence parameters are used as search parameters; the value of the POI scene label matching parameters is the image scene feature classification, and the value of the POI cue confidence parameters is greater than a preset threshold. If the matching result is a partial match, then the polygon vertex coordinates of the sector search area, the target POI type, the POI scene association rule parameters, and the POI current business status parameters are used as search parameters; the POI scene association rule parameters are the values ​​of the sub-scene containing the target POI type; the POI current business status parameters are the values ​​of the current business hours. If the matching result is a complete mismatch, a scene matching comparison interactive pop-up window will appear through the display module of the vehicle terminal.

[0045] Specifically, when the matching result is a perfect match, it means that there is a valid clue that is completely consistent with the target POI type and has the highest confidence. At this time, three types of core parameters need to be used as search conditions for the map API. The first type is the basic spatial parameter, which is the coordinates of the polygon vertices of the fan-shaped search area. These coordinates are calculated from vehicle orientation data, directional word angle offset values, and preset search distances, and are used to limit the physical spatial range of the POI search. The second type is the core content parameter, which is the target POI type. This type is extracted from user text information and is the core requirement of the search. The third type is the multimodal fusion parameter, which includes POI scene label matching parameters and POI clue confidence parameters. The POI scene label matching parameter is a configuration item used to limit the search results to be consistent with the scene feature classification obtained from image parsing. The value is directly taken from the image scene feature classification output by the visual big language model, such as the combination of highway and service area or the combination of city business district and shopping mall entrance, to ensure that the search results come only from the target scene. The POI cue confidence parameter is a configuration item used to filter POIs corresponding to low confidence cues of the model. The value is set to be greater than a preset threshold, which is usually 80, to retain only the associated POIs that the model highly confirms, and to avoid search bias caused by low confidence cues.

[0046] When the matching result is a partial match, meaning there are no completely identical valid clues but there are related clues containing the target POI type, four types of parameters need to be used as search conditions. The basic spatial parameters and core content parameters are the same as for a complete match, with two additional targeted parameters. The first type is the POI scene association rule parameter. This parameter limits the search results to sub-scenes containing the target POI type. The value is set to the sub-scene containing the target POI type. For example, if the target POI type is a restaurant, the parameter is set to "restaurant in a shopping mall" or "restaurant in a service area," ensuring the search results are relevant to the associated scene from the image parsing. The second type is the POI current business status parameter. This parameter filters POIs that are not currently open. The value is set to the current business hours. The judgment is based on the `opentime_today` field in the POI data returned by the parsed map API. This field records the POI's business hours for the day. By comparing this with the current system time of the vehicle terminal, only POIs whose time falls within the business hours are retained, avoiding the recommendation of invalid non-business locations. The vehicle terminal's display module conveys relevant scene information to the user, informing them that relevant scenes have been identified and that the target POI type within that scene will be recommended first. It also displays some matching clues and their corresponding confidence levels to help the user understand the basis for the recommendations.

[0047] When the matching result is a complete mismatch, meaning all valid clues are unrelated to the target POI type, there's no need to directly configure search parameters. Instead, a scene matching comparison interactive pop-up window appears on the vehicle terminal's display module. This pop-up window contains three core areas: the left area displays a list of valid clues obtained from image parsing, with each clue labeled with its POI type and corresponding confidence level; the right area displays the target POI type extracted from the text information, clearly showing the difference between the two parsing results; the bottom area offers three interactive options: use text type as the standard, use the best image clue as the standard, and retake the image. The "use text type as the standard" option searches using the target POI type extracted from the original text; the "use best image clue as the standard" option updates the target POI type to the type with the highest confidence among the valid clues; and the "retake image" option returns to the previous image validity pre-check step, allowing the user to supplement with a clearer scene image.

[0048] As one implementation of an intelligent route perception method, after the display module or voice module of the vehicle terminal feeds back the result list to the user, the method also includes a context-related follow-up questioning step. The steps for handling follow-up questions related to context include: The key data from the first query is stored in the context cache module. The key data includes: the coordinates of the polygon vertices of the sector search area, the target POI type, the sorted POI location data, and the current vehicle orientation data of the vehicle terminal. Receive follow-up questions from the user; Classify the intent of follow-up questions to determine the type of follow-up question: Retrieve key data matching the follow-up question type from the context cache module and adjust the POI search criteria: The map API is called to perform a secondary POI search based on the adjusted search criteria, generate a secondary results list, and provide feedback to the user through the display module or voice module.

[0049] Specifically, key data generated during the initial query is stored in the context cache module. This module is a hardware unit in the vehicle terminal used for short-term storage of core parameters from the initial query. It supports fast read / write operations and automatic expiration cleanup, with a validity period set between 90 and 150 seconds. This covers the time intervals of frequent follow-up queries by users while avoiding long-term occupation of storage resources. The key data includes four types of core information: first, the coordinates of the polygon vertices in the fan-shaped search area; second, the target POI type; third, the sorted POI location data; and fourth, the vehicle's current orientation data.

[0050] Continuously listen to user input and receive follow-up questions initiated by the user. Follow-up questions are supplementary query requests made by the user based on the initial search results. Common forms include asking for details of POI on the left, finding the next gas station ahead, finding the parking lot location of a shopping mall, expanding the scope to find more restaurants, etc. These instructions usually do not contain complete spatial range or POI type information and rely on the context data of the initial query to clarify the needs.

[0051] The received follow-up instructions are classified by intent to determine the specific type of follow-up question. The classification is based on the characteristic expressions contained in the instructions: if the instructions contain directional words such as left, right, front, and back, they are judged as directional follow-up questions, and the core requirement of this type of follow-up question is to obtain POI information in a specific direction; if the instructions contain specific Chinese names of POIs, such as the name of a shopping mall or a restaurant, they are judged as detail follow-up questions, and the core requirement of this type of follow-up question is to obtain related information of a specific POI (such as parking lot, business hours); if the instructions contain words related to spatial range such as further, more, or expanded range, they are judged as range follow-up questions, and the core requirement of this type of follow-up question is to obtain POI information in a larger space.

[0052] The system retrieves key data matching the current follow-up question type from the context cache module and adjusts the POI search conditions based on the data characteristics: For directional follow-up questions, it retrieves the POI location data and vehicle orientation data from the initial query, and recalculates the corresponding directional ...

[0053] The map API is invoked, and the adjusted search criteria are used as input parameters to perform a secondary POI search. The filtering logic for the secondary search is the same as the first search, filtering out POI data that are not the target type or area. After the search is completed, the secondary results are sorted to generate a secondary result list containing the Chinese name of the POI, relative direction, distance, and business hours. Finally, the secondary result list is fed back to the user through the display module or voice module of the in-vehicle terminal. The voice module prioritizes reading the core information of the first three results to meet the user's need for quick information retrieval in driving scenarios.

[0054] Specifically, this implementation uses contextual association design to cache key data to solve the problem of repeated input, so users do not need to supplement complete information when asking follow-up questions; it clarifies the direction of parameter adjustment by classifying intent to ensure that secondary search parameters are adapted to the needs; it achieves the connection between secondary search and primary needs by adjusting the spatial range or POI type in a targeted manner; and it improves the interaction efficiency and user experience in driving scenarios.

[0055] As one implementation of an intelligent path-following perception method, the context-related questioning process further includes: If the critical data in the context cache module has expired when the follow-up question is received, a fourth prompt message will pop up through the display module of the vehicle terminal, along with a button to quickly reuse the last POI type. If the user clicks the "Quickly Reuse Last POI Type" button, the target POI type from the first query will be used. If the user chooses to change the POI type, a POI type selection list will pop up through the display module.

[0056] As one implementation of an intelligent path-following sensing method, the method further includes: Analyze whether the follow-up instruction contains time and space pronouns; If included, the spatiotemporal anchor data at the time of the first query is extracted from the context cache module; the spatiotemporal anchor data includes: the vehicle position, vehicle orientation, coordinates of the vertices of the polygon in the sector search area, and the system timestamp at the time of the first query; The displacement distance is obtained by comparing the current vehicle position with the vehicle position at the time of the first query. If the displacement distance is greater than the preset spatial drift threshold, the original sector area is determined to be invalid; if the displacement distance does not exceed the threshold, the cached sector area is directly reused.

[0057] Specifically, the semantic parsing of user-inputted follow-up commands determines whether they contain temporal or spatial pronouns. Temporal and spatial pronouns are time-related or space-related expressions used by the user in the follow-up command to relate to the initial query scenario. Time-related expressions typically include "just now," "before," or "the time period just now," while space-related expressions typically include "that area," "the range just now," or "the direction before." These expressions themselves do not have a clear meaning and need to be combined with the temporal and spatial scenario of the initial query to be clearly identified. For example, if the user says "the gas station just now," "just now" refers to the time scenario of the initial query, and "gas station" relates to the POI type-related requirement of the initial query. If the parsing result shows the presence of temporal and spatial pronouns, it indicates that the follow-up command is related to the initial query scenario, and the subsequent temporal and spatial anchor point extraction step is required. If the parsing result shows no presence of temporal and spatial pronouns, the standard follow-up processing flow is executed, without additional temporal and spatial association judgment.

[0058] If the follow-up query contains temporal and spatial pronouns, the spatiotemporal anchor data at the time of the initial query is extracted from the context cache module. The spatiotemporal anchor data is a dataset used to fully record the spatiotemporal characteristics of the initial query scenario, adding the vehicle's location and system timestamp at the time of the initial query. The vehicle's location at the time of the initial query is the latitude and longitude data collected by the vehicle terminal via GPS module at the time the initial query was initiated, serving as the benchmark for determining subsequent vehicle displacement. The system timestamp is the system time of the vehicle terminal at the time the initial query command was received, used to assist in determining the timeliness of the initial query scenario and avoid the spatiotemporal scenario becoming invalid after a long period. The spatiotemporal anchor data can prevent misunderstandings of requirements caused by ambiguous scenarios.

[0059] The calculated vehicle displacement distance is compared with a preset spatial drift threshold, which is a critical value for vehicle displacement set based on the characteristics of the vehicle scene. The spatial range covered by the original fan-shaped search area has changed significantly from the surrounding scene of the current vehicle position. Continuing to reuse it may lead to the search results being out of touch with the actual needs. If the vehicle displacement distance is greater than the preset spatial drift threshold, it means that the vehicle has deviated too far from the position of the first query, and the fan-shaped area of ​​the first search can no longer match the current surrounding scene. It is determined that the original fan-shaped area has become invalid and a fan-shaped search area that matches the current vehicle position needs to be recalculated. If the vehicle displacement distance does not exceed the preset spatial drift threshold, it means that the vehicle is still within the reasonable spatial range of the first query scene, and the original fan-shaped area is still valid. The polygon vertex coordinates of the first fan-shaped search area stored in the context cache module can be directly reused without recalculation, reducing the system's computational load.

[0060] As one implementation of an intelligent path-following sensing method, the method further includes: The system loads the current user's personalized profile data from the user account module of the vehicle terminal. The personalized profile data includes: historical POI brand preferences, distribution of frequently visited POI types, sensitivity weights to business status, ratings and distance, and whether the switch to only display frequently used brands is enabled. If personalized profile data exists and personalized mode is enabled, the following processing will be performed on the POI data returned by the map API: if the brand of the POI is in the user's brand preference list, its ranking weight will be increased or a commonly used label will be added; if the user is highly sensitive to the business status, non-business POIs will be filtered. If the user has enabled the "Show only common brands" switch, only POIs matching the brand will be retained; A personalized list of results is generated based on the processed set of POIs.

[0061] This application also provides an intelligent roadside sensing system, including: The first processing module 201 is used to: receive a query command input by the user when the perception system of the vehicle terminal is in an active state; the query command includes text information and the current vehicle orientation data of the vehicle terminal. The second processing module 202 is used to: perform intent recognition on text information to extract directional words and target POI types from query instructions; The third processing module 203 is used to: calculate the starting angle and ending angle of the fan-shaped search area based on vehicle orientation data, angular offset values ​​of directional words, and preset angle range, and generate polygon vertex coordinates of the fan-shaped search area with the current position of the vehicle terminal as the starting point; the polygon vertex includes the starting point, left boundary point, far left point, far right point, and right boundary point; and connect the polygon vertices in sequence to form a closed fan-shaped search area. The fourth processing module 204 is used to: use the polygon vertex coordinates and target POI type of the sector search area as search parameters, and call the map API to filter matching POI data within the sector search area; The fifth processing module 205 is used to: sort the filtered POI data, generate a result list containing the Chinese name of the POI, relative direction, distance and business hours, and feed the result list back to the user through the display module or voice module of the vehicle terminal.

[0062] It should be noted that the above embodiments are only used to illustrate this application and are not intended to limit the technical solutions described in this application. Although this specification has described this application in detail with reference to the above embodiments, those skilled in the art should understand that they can still make modifications or equivalent substitutions to this application. All technical solutions and improvements that do not depart from the spirit and scope of this application should be covered within the scope of the claims of this application.

Claims

1. An intelligent route sensing method, applied to an in-vehicle terminal, characterized in that, include: When the in-vehicle terminal's perception system is active, it receives query commands input by the user. The query command includes text information and the vehicle's current orientation data from the in-vehicle terminal. The text information is subjected to intent recognition to extract directional words and target POI types from the query command; Based on the vehicle orientation data, the angle offset value of the direction word, and the preset angle range, the starting angle and ending angle of the fan-shaped search area are calculated, and the polygon vertex coordinates of the fan-shaped search area are generated with the current position of the vehicle terminal as the starting point; the polygon vertex includes the starting point, left boundary point, far left point, far right point, and right boundary point; the polygon vertices are connected in sequence to form a closed fan-shaped search area; Using the polygon vertex coordinates and target POI type of the fan-shaped search area as search parameters, the map API is called to filter matching POI data within the fan-shaped search area; The filtered POI data is sorted to generate a result list containing the Chinese name of the POI, relative direction, distance, and business hours. The result list is then fed back to the user through the display module or voice module of the vehicle terminal.

2. The intelligent route sensing method according to claim 1, characterized in that, Based on the vehicle orientation data, the angle offset value of the direction term, and the preset angle range, the starting and ending angles of the fan-shaped search area are calculated, and the polygon vertex coordinates of the fan-shaped search area are generated with the current position of the vehicle terminal as the starting point, including: Establish a mapping relationship between directional terms and angle offset values; The starting angle is obtained by adding the angle offset value of the corresponding direction word to the vehicle orientation data and then subtracting half of the preset angle range. The ending angle is obtained by adding the angle offset value of the corresponding direction word to the vehicle orientation data, and then adding half of the preset angle range. Obtain the latitude and longitude of the current location of the vehicle terminal; The latitude and longitude of the left boundary point are calculated based on the starting angle, the preset search distance, and the latitude and longitude of the current position. The latitude and longitude of the right boundary point are calculated based on the ending angle, the preset search distance, and the latitude and longitude of the current position. Calculate the latitude and longitude of the far left point based on the starting angle, M times the preset search distance, and the latitude and longitude of the current location; The latitude and longitude of the far right point are calculated based on the ending angle, M times the preset search distance, and the current location's latitude and longitude.

3. The intelligent route sensing method according to claim 2, characterized in that, The text information is subjected to intent recognition, and directional words and target POI types are extracted from the query command, including: The text information is input into a preset large text language model and an entity extraction template is loaded; the entity extraction template includes: extracting directional words from the query, extracting the target POI type from the query, and determining whether image assistance is needed; After the large text language model outputs the extraction results, the validity of the results is verified; specifically, this includes: If no direction word is extracted, the first prompt message is played through the voice module and a direction selection button pops up in the display module; if the type of the extracted target POI is unclear, a POI type option list pops up; if the extracted direction word is not within the preset selectable values, the second prompt message is played.

4. The intelligent path sensing method according to claim 3, characterized in that, The query instruction may also include real-scene image data; when receiving a query instruction input by a user, the method further includes: S401. Determine whether the query command contains real-scene image data; if it does, execute S402. S402. Perform a validity pre-check on the real-scene image data; the validity pre-check includes: detecting the image clarity and identifying whether the image contains resolvable scene features; wherein, if the clarity is less than a preset threshold or the identified image does not contain resolvable scene features, the real-scene image is determined to be invalid; if the real-scene image is determined to be invalid, a third prompt message is played through the voice module. S403. If the real-scene image is determined to be valid, store the URL of the real-scene image data in the cache module and adjust the cache expiration time according to the image file size. S404. Call the preset visual language model, input the URL, text information of the real-scene image data and the geographical environment information of the current location of the vehicle terminal into the visual language model; the visual language model outputs the parsing result; the parsing result includes: image scene feature classification and a set of associated POI clues; the image scene feature classification is labeled according to the main scene and sub-scenes; the associated POI clues in the set of associated POI clues include POI type, confidence score and association basis; S406. Filter valid leads with confidence scores greater than the preset confidence score from the set of associated POI leads; if there is a valid lead that is completely consistent with the target POI type and the confidence score of the valid lead is the highest among all valid leads, it is judged as a complete match; if there is no completely consistent lead, but there is an associated lead that contains the target POI type, it is judged as a partial match; if all valid leads are not associated with the target POI type, it is judged as a complete mismatch. S407. Execute differentiated POI search configuration based on the matching judgment result.

5. The intelligent route sensing method according to claim 4, characterized in that, Based on the matching results, execute the differentiated POI search configuration, including: If the matching result is a complete match, then the polygon vertex coordinates of the sector search area, the target POI type, the POI scene label matching parameters, and the POI clue confidence parameters are used as search parameters; the value of the POI scene label matching parameters is the image scene feature classification, and the value of the POI clue confidence parameters is greater than a preset threshold. If the matching result is a partial match, then the polygon vertex coordinates of the sector search area, the target POI type, the POI scene association rule parameters, and the POI current business status parameters are used as search parameters; the POI scene association rule parameters are defined as the sub-scene containing the target POI type; the POI current business status parameters are defined as the current business hours. If the matching result is a complete mismatch, a scene matching comparison interactive pop-up window will appear through the display module of the vehicle terminal.

6. The intelligent path sensing method according to claim 5, characterized in that, After the display module or voice module of the vehicle terminal feeds back the result list to the user, the method also includes a context-related follow-up questioning step. The steps for handling follow-up questions related to context include: The key data from the first query is stored in the context cache module; the key data includes: the polygon vertex coordinates of the sector search area, the target POI type, the sorted POI location data, and the current vehicle orientation data of the vehicle terminal. Receive follow-up questions from the user; The intent of the follow-up question is classified to determine the type of follow-up question: Retrieve key data matching the follow-up question type from the context cache module and adjust the POI search criteria: The map API is called to perform a secondary POI search based on the adjusted search criteria, generate a secondary results list, and provide feedback to the user through the display module or voice module.

7. The intelligent path sensing method according to claim 6, characterized in that, The context-dependent follow-up questioning process also includes: If the critical data in the context cache module has expired when the follow-up question is received, a fourth prompt message will pop up through the display module of the vehicle terminal, along with a button to quickly reuse the last POI type. If the user clicks the "Quickly Reuse Last POI Type" button, the target POI type from the first query will be used. If the user chooses to change the POI type, a POI type selection list will pop up through the display module.

8. The intelligent path sensing method according to claim 7, characterized in that, The method further includes: Analyze whether the follow-up question contains time and space pronouns; If included, the spatiotemporal anchor data at the time of the first query is extracted from the context cache module; the spatiotemporal anchor data includes: the vehicle position, vehicle orientation, coordinates of the vertices of the polygon in the sector search area, and system timestamp at the time of the first query; The displacement distance is obtained by comparing the current vehicle position with the vehicle position at the time of the first query. If the displacement distance is greater than the preset spatial drift threshold, the original sector area is determined to be invalid; if the displacement distance does not exceed the threshold, the cached sector area is directly reused.

9. The intelligent path sensing method according to claim 8, characterized in that, The method further includes: The user's personalized profile data is loaded from the user account module of the vehicle terminal; the personalized profile data includes: historical POI brand preferences, distribution of frequently visited POI types, sensitivity weights to business status, rating and distance, and whether the switch to only display frequently used brands is enabled; If personalized profile data exists and personalized mode is enabled, the following processing will be performed on the POI data returned by the map API: if the brand of the POI is in the user's brand preference list, its ranking weight will be increased or a commonly used label will be added; if the user is highly sensitive to the business status, non-business POIs will be filtered. If the user has enabled the "Show only common brands" switch, only POIs matching the brand will be retained; A personalized list of results is generated based on the processed set of POIs.

10. An intelligent route sensing system, characterized in that, include: The first processing module is used to: receive query commands input by the user when the perception system of the vehicle terminal is in an active state; The query command includes text information and the vehicle's current orientation data from the in-vehicle terminal. The second processing module is used to: perform intent recognition on the text information and extract directional words and target POI types from the query command; The third processing module is used to: calculate the starting angle and ending angle of the fan-shaped search area based on the vehicle orientation data, the angle offset value of the direction word, and the preset angle range, and generate the polygon vertex coordinates of the fan-shaped search area with the current position of the vehicle terminal as the starting point; the polygon vertex includes the starting point, the left boundary point, the far left point, the far right point, and the right boundary point; and connect the polygon vertices in sequence to form a closed fan-shaped search area. The fourth processing module is used to: use the polygon vertex coordinates and target POI type of the sector search area as search parameters, and call the map API to filter matching POI data within the sector search area; The fifth processing module is used to sort the filtered POI data, generate a result list containing the Chinese name of the POI, relative direction, distance, and business hours, and feed the result list back to the user through the display module or voice module of the vehicle terminal.

Images