An off-line interest point recommendation method based on multi-source sensors and speech understanding

Abstract

This invention belongs to the field of navigation and location services technology, and discloses an offline point of interest (POI) recommendation method based on multi-source sensors and speech understanding. The method includes the following steps: collecting a user's voice request and performing offline speech recognition to obtain text request information; performing intent understanding and preference extraction based on the obtained text request information; acquiring environmental perception data related to the current location and historical movement trajectory based on multi-source sensors; calculating the similarity between the user's needs and local POI data using cosine similarity, and sorting and filtering to obtain a set of target POIs; finally, outputting the set of target POIs as the recommendation result. This invention can, in an environment without network connection, calculate and comprehensively score candidate POIs based on the user's subjective experience needs expressed through voice, combined with environmental data collected by multi-source sensors and an offline POI database, thereby outputting POI recommendation results that meet the user's personalized preferences.

Description

Technical Field

[0001] This invention relates to the field of navigation and location services technology, and in particular to an offline point of interest recommendation method based on multi-source sensors and voice understanding. Background Technology

[0002] Current Points of Interest (POI) recommendations are mostly based on online services, combining data such as online reviews, images, and weather to recommend restaurants, attractions, and photo spots to users. However, in scenarios such as suburbs, mountainous areas, and road trips, devices often have weak or no signal, rendering online recommendation services unusable. Furthermore, traditional POI recommendations primarily rely on POI categories, geographical location, and a limited number of static attributes, lacking a perception of the real-world environment and failing to meet subjective experience requirements such as walking distance, shade, sparse crowds, and a smooth ride.

[0003] Currently, in-vehicle terminals, cycling terminals, and outdoor devices generally integrate sensors such as GPS, IMU, barometers, and temperature and humidity sensors, enabling real-time sensing of altitude, slope, road vibration, ambient temperature and humidity, and wind speed. If, under offline conditions, this multi-source sensor data could be combined with semantic understanding of the user's natural speech needs, and the experience quality of each point of interest and its surrounding environment could be quantitatively evaluated and recommendations made accordingly, the accuracy and practicality of point-of-interest recommendations would be significantly improved.

[0004] However, existing technologies lack a systematic method for evaluating and recommending POIs in a completely offline scenario, taking into account both multi-source environmental perception and speech semantic understanding. Summary of the Invention

[0005] The purpose of this invention is to provide an offline point of interest recommendation method based on multi-source sensors and speech understanding. This method can calculate and comprehensively score the experience indicators of candidate points of interest in an environment without network connection, based on the user's subjective experience needs expressed by voice, combined with environmental data collected by multi-source sensors and an offline POI database, thereby outputting point of interest recommendation results that meet the user's personalized preferences.

[0006] To achieve the above objectives, this invention provides an offline point of interest recommendation method based on multi-source sensors and speech understanding, comprising the following steps:

[0007] Step S1: Collect the user's voice request about the target point of interest. On the terminal, use the locally deployed FunASR model to perform offline speech recognition on the collected voice request to obtain the text request information.

[0008] Step S2: Based on the large language model and the prompt word technology, perform intent understanding and preference extraction on the acquired text request information;

[0009] Step S3: Acquire environmental perception data related to the current location and historical motion trajectory based on multi-source sensors;

[0010] Step S4: Calculate the similarity between user needs and local POI data using cosine similarity, and sort and filter the candidate points of interest to obtain the target point of interest set;

[0011] Step S5: Use the set of target points of interest as the recommendation result and output it through map icon highlighting, list display or voice broadcast.

[0012] Preferably, in step S2, based on a large language model combined with prompt word technology, intent understanding and preference extraction are performed on the acquired text request information. The specific process is as follows:

[0013] Step S21: Input the obtained text request information into the locally deployed semantic understanding large language model;

[0014] Step S22: Using prompt word technology, based on the local structured POI database, the large language model generates SQL language for relevant data according to the input text information;

[0015] Step S23: Using prompt word technology, the user's interest category intent and a set of experience preference parameters are parsed.

[0016] Preferably, in step S3, environmental perception data related to the current location and historical motion trajectory is acquired based on multi-source sensors. The specific process is as follows:

[0017] Step S31: Read environmental perception data related to the current location and historical movement trajectory from multi-source sensors, including altitude, slope change, road vibration intensity, ambient temperature, humidity, and wind speed; the user judges whether the data matches the actual situation, and if it is valid, the data is stored in the local database as historical experience data;

[0018] Step S32: Based on the current location and preset search range, retrieve several candidate points of interest from the offline map and POI database stored locally on the hardware terminal, and obtain geographic feature information and historical sensing data related to each candidate point of interest.

[0019] Preferably, in step S4, cosine similarity is used to calculate the similarity between user needs and local POI data, and candidate interest points are sorted and filtered to obtain a set of target interest points. The specific process is as follows:

[0020] Step S41: Based on the user preference parameters analyzed from the large model module, the user demand data is vectorized according to the Embedding model;

[0021] Step S42: Based on the Embedding model, the structured data retrieved from the local POI data is vectorized into the database data.

[0022] Step S43: Calculate the similarity value between user requirements and local POI data using cosine similarity;

[0023] Step S44: Based on the similarity score, sort and filter the candidate interest points to obtain the target interest point set.

[0024] Preferably, in step S43, the formula for calculating the similarity value is:

[0025] ;

[0026] Where A represents user preference parameters obtained from large-scale model analysis of user needs; B represents basic data obtained from a local POI database query; the similarity value ranges from [-1, 1], where -1 represents complete dissimilarity and 1 represents complete similarity. For data dimensions; Let A be the value of the user preference parameter A in the i-th dimension; Let be the value of the basic data B of POI in the i-th dimension.

[0027] Preferably, in step S44, the system first queries the database to obtain a set of data objects, and for each data object in the set, calculates the similarity between the user's needs and the local POI data using cosine similarity.

[0028] Then, the calculated similarity value is compared with the user input vector to obtain a numerical value representing the degree of similarity, and this value is assigned to the similarity attribute of the corresponding data object;

[0029] Finally, after calculating the similarity of all objects, the system sorts the entire set in descending order based on the similarity values, placing the data object most similar to the user input at the top of the list, and finally outputting the complete data set sorted from high to low similarity.

[0030] Therefore, the present invention adopts the above-mentioned offline point of interest recommendation method based on multi-source sensors and speech understanding. In an environment without network connection, it can calculate and comprehensively score the experience indicators of candidate points of interest based on the subjective experience needs expressed by the user through speech, combined with environmental data collected by multi-source sensors and an offline POI database, thereby outputting point of interest recommendation results that meet the user's personalized preferences.

[0031] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the technical architecture of an offline point of interest recommendation method based on multi-source sensors and speech understanding according to the present invention.

[0033] Figure 2 This is a schematic diagram of the business architecture of an offline point of interest recommendation method based on multi-source sensors and speech understanding according to the present invention;

[0034] Figure 3 This is a sequence diagram of the design of a large model module for an offline point of interest recommendation method based on multi-source sensors and speech understanding according to the present invention.

[0035] Figure 4 This is a diagram of the perception module design scheme of an offline point of interest recommendation method based on multi-source sensors and speech understanding according to the present invention.

[0036] Figure 5 This is a computational module diagram of an offline point of interest recommendation method based on multi-source sensors and speech understanding according to the present invention. Detailed Implementation

[0037] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] like Figure 1 As shown, the present invention provides an offline point of interest recommendation method based on multi-source sensors and speech understanding, comprising the following steps:

[0039] Step S1: Collect the user's voice request about the target point of interest. On the terminal, use the locally deployed FunASR model to perform offline speech recognition on the collected voice request to obtain the text request information.

[0040] Step S2: Based on the large language model and the prompt word technology, perform intent understanding and preference extraction on the acquired text request information;

[0041] Step S3: Acquire environmental perception data related to the current location and historical motion trajectory based on multi-source sensors;

[0042] Step S4: Calculate the similarity between user needs and local POI data using cosine similarity, and sort and filter the candidate points of interest to obtain the target point of interest set;

[0043] Step S5: Use the set of target points of interest as the recommendation result and output it through map icon highlighting, list display or voice broadcast.

[0044] Example 1

[0045] This invention discloses an offline point-of-interest (POI) recommendation method based on multi-source sensors and speech understanding, applied to a hardware terminal equipped with multi-source sensors and pre-stored offline maps and POI databases. The method includes the following steps:

[0046] Step S1: Voice acquisition and offline recognition.

[0047] Step S11: Collect the user's voice request regarding the target point of interest;

[0048] Step S12: On the terminal, the collected voice requests are processed using the locally deployed FunASR model for offline speech recognition to obtain the text request information.

[0049] Step S2: Based on the large language model and the prompt word technology, perform intent understanding and preference extraction on the obtained text request information.

[0050] Step S21: Input the obtained text request information into the locally deployed semantic understanding large language model.

[0051] Step S22: Using prompt word technology, based on the local structured POI database, the large language model generates SQL language for the relevant data according to the input text information.

[0052] Step S23: Using prompt word technology, the user's interest category intent and a set of experience preference parameters are parsed; among them, the preference parameters include: landscape preference (e.g., beautiful scenery, open view); comfort preference (e.g., cool, moderate temperature, not humid); quiet / crowded preference (e.g., few people, away from noisy roads); safety preference; road condition / smoothness preference, etc.

[0053] Step S3: Acquire environmental perception data related to the current location and historical motion trajectory based on multi-source sensors.

[0054] Step S31: Read environmental perception data related to the current location and historical movement trajectory from multiple sensors such as GPS, IMU, barometer, temperature sensor, and humidity sensor. This includes altitude, slope change, road vibration intensity, ambient temperature, humidity, and wind speed. Users can make subjective judgments based on whether the data matches the actual situation. If the data is valid, it will be stored in the local database as historical experience data.

[0055] Step S32: Based on the current location and preset search range, retrieve multiple candidate points of interest from the offline map and POI database stored locally on the hardware terminal, and obtain geographic feature information and historical sensing data related to each candidate point of interest.

[0056] Step S4: Calculate the similarity between user needs and local POI data using cosine similarity, and sort and filter candidate interest points to obtain a set of target interest points.

[0057] Step S41: Based on the user preference parameters analyzed from the large model module, the user demand data is vectorized using the Embedding model.

[0058] Step S42: Based on the Embedding model, the structured data retrieved from the local POI data is vectorized.

[0059] Step S43: Calculate the similarity between user requirements and local POI data using cosine similarity, as shown below:

[0060] ;

[0061] Where A represents user preference parameters obtained from large-scale model analysis of user needs; B represents basic data obtained from a local POI database query; the similarity value ranges from [-1, 1], where -1 represents complete dissimilarity and 1 represents complete similarity. For data dimensions; Let A be the value of the user preference parameter A in the i-th dimension; Let be the value of the basic data B of POI in the i-th dimension.

[0062] Step S44: Based on the similarity score, sort and filter the candidate interest points to obtain the target interest point set.

[0063] First, the system retrieves a set of data objects from the database. For each data object in the set, the similarity between the user's requirements and the local POI data is calculated using cosine similarity.

[0064] Then, the calculated similarity value is compared with the user input vector to obtain a numerical value representing the degree of similarity, and this value is assigned to the similarity attribute of the corresponding data object.

[0065] Finally, after calculating the similarity of all objects, the system sorts the entire set in descending order based on the similarity scores, placing the data object most similar to the user's input at the top of the list. The final output is the complete data set sorted by similarity from highest to lowest. Step S5: The target interest point set is used as the recommendation result and output through map icon highlighting, list display, or voice announcement.

[0066] Example 2

[0067] This embodiment is based on an offline point-of-interest (POI) recommendation method proposed in this invention, using multi-source sensors and speech understanding. It is applied to a device terminal equipped with multi-source sensors and storing an offline map and POI database, including:

[0068] The system collects users' voice requests about points of interest and performs offline speech recognition on the terminal to obtain text request information.

[0069] The text request information is input into the locally deployed semantic understanding model, which is then parsed to obtain the user's interest category intent and at least one experience preference parameter.

[0070] The system reads environmental perception data from multiple sources and retrieves multiple candidate points of interest related to the stated category intent from offline maps and point of interest databases, and obtains geographic feature information and historical perception data related to each candidate point of interest.

[0071] Based on local POI data and user preference parameters, vectorization is performed. The similarity between user needs and local POI data is calculated using cosine similarity. The similarity values ​​are then sorted and rounded to obtain a set of target interest points. This set of target interest points is then output as the interest point recommendation result.

[0072] For scenarios such as field training, offline environments, and private area data positioning, based on digital equipment such as large models, speech recognition, databases, and sensors, intelligent positioning functions are realized in offline environments to improve operational efficiency.

[0073] Example 3

[0074] like Figure 2 As shown, the offline point of interest recommendation method based on multi-source sensors and speech understanding of the present invention includes a speech module, a large model module, a perception module, a computing module, and a visualization module in practical use.

[0075] Voice module: Based on the user's voice input, the system collects voice information through the sound card of the hardware device and converts the voice input into text information based on the FunASR model deployed offline and locally.

[0076] Large Model Module: Based on the large language model and prompt word technology, this module converts text information into structured user preference parameter information and generates corresponding SQL statements. It uses the Model Context Protocol (MCP) to call a packaged database query tool, retrieving relevant local data based on the SQL, such as... Figure 3 As shown.

[0077] Sensing Module: Based on GPS, IMU, barometric pressure, temperature, and humidity data collected by sensing devices, the user subjectively judges whether the data matches the real environment. If it does, the data can be added to a local database for storage, building a private dataset, such as... Figure 4 As shown.

[0078] The calculation module, based on user preference parameters, current perceived data, and local POI data, uses an embedding model to vectorize the data and employs cosine similarity to calculate the degree of matching with user needs, producing a result dataset, such as... Figure 5 As shown.

[0079] Visualization module: Combines the calculation results into recommendation results and outputs them through map icon highlighting, list display or voice broadcast.

[0080] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] This invention is described with reference to block diagrams of methods and computer program products according to embodiments of the invention. It will be understood that the functions of each block in the block diagram, and combinations of the functions of several blocks in the block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the functions in the blocks. Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the block. Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the block. Figure 1 The steps of the function specified in one or more boxes.

[0084] Therefore, the present invention adopts the above-mentioned offline point of interest recommendation method based on multi-source sensors and speech understanding. In an environment without network connection, it can calculate and comprehensively score the experience indicators of candidate points of interest based on the subjective experience needs expressed by the user through speech, combined with environmental data collected by multi-source sensors and an offline POI database, thereby outputting point of interest recommendation results that meet the user's personalized preferences.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An offline point-of-interest recommendation method based on multi-source sensors and speech understanding, characterized in that, Includes the following steps: Step S1: Collect the user's voice request about the target point of interest. On the terminal, use the locally deployed FunASR model to perform offline speech recognition on the collected voice request to obtain the text request information. Step S2: Based on the large language model and the prompt word technology, perform intent understanding and preference extraction on the acquired text request information; By using cue word technology, the user's interest category intent and a set of experience preference parameters are obtained; Step S3: Acquire environmental perception data related to the current location and historical motion trajectory based on multi-source sensors. The specific process is as follows: Step S31: Read environmental perception data related to the current location and historical movement trajectory from multi-source sensors, including altitude, slope change, road vibration intensity, ambient temperature, humidity, and wind speed; the user judges whether the data matches the actual situation, and if it is valid, the data is stored in the local database as historical experience data; Step S32: Based on the current location and preset search range, retrieve several candidate points of interest related to the category intent from the offline map and POI database stored locally on the hardware terminal, and obtain geographic feature information and historical perception data related to each candidate point of interest. Step S4: Calculate the similarity between user needs and local POI data using cosine similarity, and sort and filter candidate interest points to obtain the target interest point set. The specific process is as follows: Step S41: Based on the user preference parameters analyzed from the large model module, the user preference parameters are vectorized according to the Embedding model; Step S42: Based on the Embedding model, the structured data retrieved from the local POI data is vectorized. Step S43: Calculate the similarity value between user requirements and local POI data using cosine similarity; Step S44: Based on the similarity score, sort and filter the candidate interest points to obtain the target interest point set; Step S5: Use the set of target points of interest as the recommendation result and output it through map icon highlighting, list display or voice broadcast.

2. The offline point of interest recommendation method based on multi-source sensors and speech understanding according to claim 1, characterized in that, In step S2, based on a large language model combined with prompt word technology, intent understanding and preference extraction are performed on the acquired text request information. The specific process is as follows: Step S21: Input the obtained text request information into the locally deployed semantic understanding large language model; Step S22: Using prompt word technology, based on the local structured POI database, the large language model generates SQL language for relevant data according to the input text information; Step S23: Using prompt word technology, the user's interest category intent and a set of experience preference parameters are parsed.

3. The offline point of interest recommendation method based on multi-source sensors and speech understanding according to claim 1, characterized in that, In step S43, the formula for calculating the similarity value is: ； Where A represents user preference parameters obtained from large-scale model analysis of user needs; B represents data obtained from a local POI database query; the similarity value ranges from [-1, 1], where -1 represents complete dissimilarity and 1 represents complete similarity; For data dimensions; Let A be the value of the user preference parameter A in the i-th dimension; Let be the value of POI data B in the i-th dimension.

Images