Map element introduction method and device and electronic equipment

Abstract

The invention relates to the technical field of man-machine interaction, and discloses a map element introduction method and device and electronic device.According to the method, map data corresponding to a target vehicle are obtained, the map data comprise multiple map elements, and a positioning instruction input by a user is responded; performing text similarity matching on relative positions between the target vehicle and map elements according to the positioning instruction, determining a target element from the map elements according to a matching result, generating element introduction information according to the target element, and displaying the element introduction information to a user; through the method, a more practical and more accurate map element introduction method is provided, the intelligence sense and the science and technology sense of the vehicle are improved, and then the user experience sense is improved.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a map element introduction method, device and electronic equipment. Background Art

[0002] In current driving travel scenarios, drivers may be interested in natural scenery, landscapes or landmarks along the way and want to obtain detailed information about these objects. To achieve this goal, the relevant operation method usually includes two main steps: first, locate and confirm the name of the object of interest in the in-vehicle navigation or mobile phone map application; second, based on this name, search for information in search engines or related applications to obtain detailed introduction information and background information.

[0003] However, this traditional operation method has obvious safety hazards and efficiency problems. On the one hand, operating a mobile phone or car computer to search for maps and retrieve information while driving will greatly distract the driver's attention, increase the risk of traffic accidents, and pose a serious threat to driving safety. On the other hand, from the perspective of user experience, this method is relatively inefficient and cumbersome. Users not only need to carefully search and confirm the name of the target object on the map, but also need to transfer this name to the search engine for a secondary search. The whole process is time-consuming and the user experience is not good. In summary, there are still many imperfections in the relevant operation methods, and a new method of introducing map elements is urgently needed to solve at least one of the above problems.

[0004] It should be noted that the above content only provides background technical information related to the present application and does not necessarily constitute prior art. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present application provides a map element introduction method, device and electronic device to provide more accurate and efficient map element introduction information, thereby improving user satisfaction.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0007] According to one aspect of an embodiment of the present application, a map element introduction method is provided, comprising: acquiring map data corresponding to a target vehicle, wherein the map data comprises a plurality of map elements; in response to a positioning instruction input by a user, performing text similarity matching on a relative position between the target vehicle and the map element according to the positioning instruction, and determining a target element from the map elements according to the matching result; generating element introduction information according to the target element, and displaying the element introduction information to a user.

[0008] In one embodiment of the present application, based on the aforementioned scheme, map data corresponding to the target vehicle is obtained, including: obtaining vehicle position information of the target vehicle; determining a target range based on the vehicle position information and a preset range threshold; and generating map data corresponding to the target vehicle according to map elements in the target range.

[0009] In one embodiment of the present application, based on the aforementioned scheme, before obtaining the map data corresponding to the target vehicle, the method also includes: obtaining a current instruction input by a user; classifying the current instruction to obtain an instruction type; if the instruction type is a map-related type, obtaining the map data corresponding to the target vehicle, and generating the positioning instruction based on the current instruction.

[0010] In one embodiment of the present application, based on the aforementioned scheme, the positioning instruction is generated according to the current instruction, including at least one of the following: if the current instruction includes a location keyword, the positioning instruction is generated according to the location keyword; if the current instruction does not include a location keyword, an action instruction is obtained by capturing an image of the user and performing action recognition on the captured user image, and the location keyword is determined according to the action instruction to generate the positioning instruction according to the location keyword.

[0011] In one embodiment of the present application, based on the aforementioned scheme, text similarity matching is performed on the relative position between the target vehicle and the map element according to the positioning instruction, and the target element is determined from the map element according to the matching result, including: determining the relative position between the target vehicle and the map element according to the vehicle position information of the target vehicle and the element position information corresponding to the map element; calculating the text similarity between the positioning instruction and the relative position, and if the text similarity is greater than a preset threshold, determining that the matching result is that the positioning instruction matches the relative position, and determining the map element corresponding to the relative position as the target element.

[0012] In one embodiment of the present application, based on the aforementioned scheme, the method further includes: if there are multiple matching results, displaying interactive information to the user and obtaining a feedback instruction input by the user; matching the feedback instruction with the map element in the matching result to obtain a new matching result; and determining the target element from the map elements in the matching result according to the new matching result.

[0013] In one embodiment of the present application, based on the aforementioned scheme, element introduction information is generated according to the target element, including: obtaining user portrait information, combining the user portrait information and the target element to obtain input information; inputting the input information into a preset prompt word template to obtain a target prompt word; inputting the target prompt word into a preset large language model to obtain the element introduction information.

[0014] In one embodiment of the present application, based on the aforementioned scheme, the method further includes: receiving a storage instruction input by a user, acquiring an element image of the target element; storing the target element and the element image in a preset storage location, generating historical information after a preset time period, and reminding the user based on the historical information.

[0015] According to one aspect of an embodiment of the present application, a map element introduction device is provided, comprising: an acquisition module, used to acquire map data corresponding to a target vehicle, wherein the map data includes a plurality of map elements; a determination module, used to respond to a positioning instruction input by a user, perform text similarity matching on the relative position between the target vehicle and the map element according to the positioning instruction, and determine the target element from the map elements according to the matching result; and an introduction module, used to generate element introduction information according to the target element, and display the element introduction information to the user.

[0016] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements a map element introduction method as described in any one of the above embodiments.

[0017] Beneficial effects of the present application: The present application obtains map data corresponding to a target vehicle, wherein the map data includes a plurality of map elements, responds to a positioning instruction input by a user, performs text similarity matching on the relative position between the target vehicle and the map element according to the positioning instruction, and determines the target element from the map element according to the matching result, thereby obtaining the target element simply and quickly, generating element introduction information according to the target element, and displaying the element introduction information to the user, thereby more accurately and efficiently meeting the user's information acquisition needs during a driving trip, providing more accurate element introduction information, and thereby improving user satisfaction.

[0018] In addition, it can interact with users in multiple ways through voice and / or body movements, and fully utilizes the characteristics of the large language model to provide users with tailored element introduction content, and can also determine more accurate target elements through further interaction with users.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0021] Figure 1 is a schematic diagram of an exemplary system architecture shown in an exemplary embodiment of the present application;

[0022] Figure 2 is a flowchart of a method for introducing map elements according to an exemplary embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a vehicle top view showing a method for introducing map elements according to an exemplary embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of the overall process of a method for introducing map elements shown in an exemplary embodiment of the present application;

[0025] Figure 5 is a schematic diagram of a map data processing flow of a map element introduction method according to an exemplary embodiment of the present application;

[0026] Figure 6 It is a flowchart of a map element introduction method after obtaining a matching result, shown in an exemplary embodiment of the present application;

[0027] Figure 7 is a block diagram of a map element introduction device shown in an exemplary embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0030] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0031] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0032] The first thing that needs to be explained is the map element, that is, POI (Point of Interest). In a geographic information system, a POI can be a house, a shop, a mailbox, a bus stop, etc.

[0033] TTS (Text To Speech) is a technology that can convert files stored in a computer, such as help files or web pages, into natural speech output. TTS can help people with visual impairments read information on a computer, or simply increase the readability of text documents. Current TTS applications include voice-driven email software and voice-sensitive systems. TTS is often used in conjunction with voice recognition programs. TTS synthesis technology is a technology that converts text generated by the computer itself or input from the outside into natural and fluent synthesized spoken output. TTS technology converts text files in real time, and the conversion time can be calculated in seconds. Under the action of its unique intelligent voice controller, the voice output of the text is smooth, making the listener feel natural when listening to the information, without the indifference and awkwardness of machine voice output. TTS speech synthesis technology supports multiple languages, with clear sound quality and coherent and fluent intonation.

[0034] Figure 1 It is a schematic diagram of an exemplary system architecture shown in an exemplary embodiment of the present application.

[0035] Reference Figure 1As shown, the system architecture may include a data acquisition device 101 and a computer device 102. The computer device 102 may be at least one of a vehicle-mounted host, a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. The data acquisition device 101 is used to collect map data corresponding to the target vehicle, wherein the map data includes multiple map elements. In this embodiment, the data acquisition device 101 obtains the above data and provides it to the computer device 102 for processing. Relevant technicians can use the computer device 102 to respond to the positioning instruction input by the user, perform text similarity matching on the relative position between the target vehicle and the map element according to the positioning instruction, determine the target element from the map element according to the matching result, generate element introduction information according to the target element, and display the element introduction information to the user to complete the map element introduction. It should be noted that the data acquisition device 101 and the computer device 102 provided in this embodiment are only an example and should not bring any limitation to the function and scope of use of the embodiment of the present application.

[0036] It should be noted that the map element introduction method provided in the embodiment of the present application is generally executed by the computer device 102 , and accordingly, the map element introduction device is generally set in the computer device 102 .

[0037] Figure 2 is a flowchart of a method for introducing map elements according to an exemplary embodiment of the present application. The method for introducing map elements may be executed by a computing and processing device. The computing and processing device may be Figure 1 The computer device 102 shown in FIG. Figure 2 As shown, the map element introduction method at least includes steps S210 to S230, which are described in detail as follows:

[0038] In step S210, map data corresponding to the target vehicle is obtained.

[0039] In one embodiment of the present application, map data corresponding to the target vehicle is obtained through a vehicle-mounted navigation system such as a GPS (Global Positioning System) navigation system and / or a Beidou satellite navigation system, wherein the map data includes a plurality of map elements.

[0040] In one embodiment of the present application, the process of obtaining map data corresponding to a target vehicle includes the following steps: obtaining vehicle position information of the target vehicle; determining a target range based on the vehicle position information and a preset range threshold; and generating map data corresponding to the target vehicle based on map elements in the target range.

[0041] In this embodiment, the vehicle position information is first obtained from the vehicle sensor of the target vehicle, wherein the vehicle position information may include the vehicle latitude and longitude, the vehicle head direction and the vehicle altitude information. Then, the target range is determined according to the vehicle position information and the preset range, and the map data corresponding to the target vehicle is generated according to the map elements in the target range. Each map element has corresponding map element information, and the map data corresponding to the map elements in the target range is generated according to the map element information corresponding to the map elements in the target range. For example, the preset range is a semicircle with a diameter of n meters directly in front of the vehicle head. Figure 3 As shown, Figure 3 1 is a schematic diagram of a vehicle top view of a map element introduction method shown in an exemplary embodiment of the present application. First, with the front of the vehicle as the center, a semicircle (such as Figure 3 The gray part is shown in the figure), and all map element information within the gray range is obtained. The map element information generally includes the following basic information: name, type, latitude and longitude, area, height difference, and introduction. That is, the map element information generally includes the map element name, map element type, map element latitude and longitude, map element area, map element height difference, and map element introduction. The map element information is shown in Table 1:

[0042] Table 1

[0043]

[0044] It is understandable that this embodiment and Table 1 are only examples, and the basic information contained in the map element information can be increased, reduced or changed according to actual needs, and the corresponding data or content can also be modified according to actual needs, and this application does not limit this. In addition, the preset range in this embodiment can be set according to actual needs, and is not limited to the semicircle illustrated in the above example, and this application does not limit this.

[0045] In one embodiment of the present application, the process before obtaining the map data corresponding to the target vehicle further includes the following steps: obtaining the current instruction input by the user; classifying the current instruction to obtain the instruction type; if the instruction type is a map-related type, obtaining the map data corresponding to the target vehicle, and generating a positioning instruction according to the current instruction; if the instruction type is a type other than the map-related type, not obtaining the map data corresponding to the target vehicle, and not generating a positioning instruction according to the current instruction. The process of classifying the current instruction to obtain the instruction type can be implemented by the following steps: extracting keywords from the current instruction, matching the extracted keywords with preset map keywords, and if the match is successful, determining that the instruction type of the current instruction is a map-related type. For example, the current instruction input by the user is "What is the mountain in front?", "What is the river on the left?", "What is the tall building?", etc., and the current instruction is classified to obtain the instruction type of the map-related type. It can be understood that the current instruction input by the user is only an example, and the present application does not limit this.

[0046] In step S220, in response to the positioning instruction input by the user, text similarity matching is performed on the relative positions of the target vehicle and the map elements according to the positioning instruction, and the target element is determined from the map elements according to the matching result.

[0047] In one embodiment of the present application, generating a positioning instruction based on a current instruction includes at least one of the following: if the current instruction includes a location keyword, generating a positioning instruction based on the location keyword; if the current instruction does not include a location keyword, obtaining an action instruction by capturing an image of the user and performing action recognition on the captured user image, determining the location keyword based on the action instruction, and generating a positioning instruction based on the location keyword.

[0048] In this embodiment, the current instruction input by the user is detected. If it is detected that the current instruction input by the user includes voice input, the voice input part of the current instruction is subjected to voice recognition to obtain text information, and keywords are extracted from the text information. If location keywords are extracted, a positioning instruction is generated according to the location keywords. If it is detected that the current instruction includes action input, wherein the action input includes gesture input and head action input, an image of the user is captured, and the captured user image is input into a preset action recognition model to obtain action information, i.e., action instructions, and keywords are extracted from the action information. If location keywords are extracted, a positioning instruction is generated according to the location keywords. If it is detected that the current instruction input by the user includes voice input and action input, voice recognition is performed on the voice input part of the current instruction to obtain text information, and keywords are extracted from the text information. If the location keywords are not extracted, an image of the user is captured, and the captured user image is input into a preset action recognition model to obtain action information, i.e., action instructions, and keywords are extracted from the action information. If the location keywords are extracted, the text information and the action information are combined to obtain combined information, and a positioning instruction is generated according to the combined information. For example, the voice input part of the current command is subjected to voice recognition, and the obtained text information is "What mountain is that mountain in front?", and the image where the action input part of the current command is located is input into the preset action recognition model, and the obtained action information is "30 degrees to the right", and the combined information obtained based on the text information and the action information is "30 degrees to the right, what mountain is that mountain in front?" It can be understood that this example is for illustrative purposes only and should not bring any limitations to the functions and scope of use of the embodiments of the present application.

[0049] In one embodiment of the present application, a process of performing text similarity matching on the relative position between a target vehicle and a map element according to a positioning instruction, and determining the target element from the map elements according to the matching result includes the following steps: determining the relative position between the target vehicle and each map element according to the vehicle position information of the target vehicle and the element position information corresponding to each map element; calculating the text similarity between the positioning instruction and the relative position, and if the text similarity is greater than a preset threshold, determining that the matching result is that the positioning instruction matches the relative position, and determining the map element corresponding to the relative position as the target element.

[0050] In this embodiment, the relative position and relative distance between the target vehicle and each map element are determined based on the vehicle position information of the target vehicle and the element position information corresponding to each map element, wherein the relative position includes the relative position and the relative distance. Figure 3 Take the schematic diagram shown as an example, with the vehicle centerline as the 0 degree direction ( Figure 3The relative position between the map element and the central axis is calculated (in the direction indicated by the arrow), and the relative position between the map element and the central axis is expressed as an azimuth (such as 45 degrees to the left and 30 degrees to the right). At the same time, the distance between the longitude and latitude of the map element and the longitude and latitude of the vehicle is calculated based on the distance between the two, expressed in meters.

[0051] In one embodiment of the present application, if there are multiple matching results, interactive information is displayed to the user, and feedback instructions input by the user are obtained; the feedback instructions are matched with map elements in the matching results to obtain new matching results; and the target elements are determined from the map elements in the matching results based on the new matching results. In this embodiment, matching the feedback instructions with the map elements in the matching results can also be text similarity matching. Specifically, the feedback instructions are converted into text information. The specific conversion method refers to the corresponding description of speech recognition and action recognition in the aforementioned embodiments, which will not be repeated here. The feedback instructions are matched with the map elements in the matching results for text similarity.

[0052] In this embodiment, the number of matching results is counted. If there are multiple matching results, interactive information is generated based on the map elements in the matching results, and the interactive information is displayed, such as "how many meters away is the river?", "what is the color?", "how high is it approximately?", etc., and a feedback instruction input by the user is received, and the element similarity between the feedback instruction and the map elements in the matching results is calculated. A new matching result is determined based on the element similarity, and the map element with the highest element similarity in the new matching result is determined as the target element. For example, if the feedback instruction input by the user is "the river at 30 meters", then according to the calculated element similarity between the feedback instruction and the map elements in the matching results, the river at 30 meters closest to the target vehicle in the map elements in the matching results is determined as the target element.

[0053] In step S230, element introduction information is generated according to the target element, and the element introduction information is displayed to the user.

[0054] In one embodiment of the present application, the process of generating element introduction information based on the target element includes the following steps: obtaining user portrait information, combining the user portrait information and the target element to obtain input information; inputting the input information into a preset prompt word template to obtain a target prompt word; inputting the target prompt word into a preset large language model to obtain element introduction information.

[0055] In this embodiment, the user portrait information includes age, gender, and place of origin. Assume that the user portrait information is age: 20 years old, gender: male, place of origin: Jilin, and the map element name is Hongen Temple Forest Park. The user portrait information is combined with the map element name of the map element to obtain input information. Assume that the prompt word template is "Now you are a senior outdoor tour guide with more than 20 years of experience. You are introducing the verbal description information related to {map element name} to a {age}-year-old {place of origin}-{gender} man. The introduction information needs to match the identity of the audience, about 200 words." The input information is input into the preset prompt word template, and the target prompt word obtained is "Now you are a senior outdoor tour guide with more than 20 years of experience. You are introducing the verbal description information related to Hongen Temple Forest Park to a 20-year-old man from Jilin. The introduction information needs to match the identity of the audience, about 200 words." The target prompt word is input into the preset large language model to obtain element introduction information. Generating element introduction information through a large language model can provide users with a more realistic and smooth communication experience, and can also greatly improve work efficiency and reduce labor costs.

[0056] In one embodiment of the present application, a storage instruction input by a user is received, an element image of a target element is collected; the target element and the element image are stored in a preset storage location, and historical information is generated after a preset time period, and the user is reminded according to the historical information. By storing the target element and the element image in a preset storage location, it is convenient for the user to query the historical records, and by viewing the element image and / or generating historical information after a preset time period, and reminding the user according to the historical information, the user can feel the emotion and atmosphere at that time again, thereby enhancing the user's emotional connection and improving the user's satisfaction.

[0057] In one embodiment of the present application, the method of displaying element introduction information to the user includes at least one of the following: displaying element introduction information to the user through an interactive interface; generating element introduction voice according to the element introduction information, and playing the element introduction voice through a voice playback device, wherein the voice playback device includes a speaker, a voice assistant, etc.; generating an element introduction video according to the element introduction information, and displaying the element introduction video to the user through an interactive interface. This increases the display methods of element introduction information and improves the user experience. Preferably, an element introduction voice is generated according to the element introduction information, and the element introduction voice is played through a voice playback device.

[0058] In one embodiment of the present application, a vehicle-mounted travel assistant is proposed, which is used to combine map elements with user input to provide users with complete scenic spot map element information, thereby solving the trouble of users having to manually check their smartphones or vehicle computers to understand scenic spot information during travel. It provides users with the satisfaction of "what you say is what you get, what you point is what you get", thereby improving the intelligence of the vehicle and user satisfaction.

[0059] In this embodiment, reference Figure 4 , Figure 4 It is a schematic diagram of the overall flow of the map element introduction method shown in an exemplary embodiment of the present application. The in-vehicle travel assistant includes 7 modules: user intention recognition module, map data processing module, user input processing module, text matching module, matching result processing module, large language model processing module and TTS (Text To Speech) broadcast module. The main function of the user intention recognition module is to determine whether to start the in-vehicle travel assistant according to the current instruction input by the user. The main function of the map data processing module is to convert the map element information of the map element (POI) into a natural language description text, and filter out m candidate points by the vehicle position. The user input processing module mainly processes the current instruction input by the user. The current instruction may be in the form of one or more of voice, gesture, and head movement. The function of this module is mainly to convert the current instruction input by the user into a natural language text. The text matching module matches the natural language text converted from the map element text to the user input to find the n map elements that are most similar to the user input. The matching result processing module is used to process the most similar map element to obtain a unique POI point, namely the target element. The large language model processing module is used to generate a complete language description of the target element and output detailed text description information, namely the element introduction information. The TTS broadcasting module is used to broadcast the element introduction information generated by the large language model to the user.

[0060] In this embodiment, the user intention recognition module is used to understand whether the user's input needs to start the travel assistant. It is essentially a text classification model. The user intention recognition module first converts the current instruction input by the user into text information, and then inputs the text information into the trained text classification model. When the classification model outputs the travel assistant, the travel assistant is started. When classified as other, the travel assistant remains silent. For example, common texts classified as starting the travel assistant are: "What mountain is that mountain in front?", "What river is that river on the left?", "What is that tall building?", etc.

[0061] In this embodiment, reference Figure 5 As shown, Figure 5It is a schematic diagram of the map data processing flow of the map element introduction method shown in an exemplary embodiment of the present application. The map data processing module first obtains the vehicle position information from the vehicle sensor of the target vehicle, including the vehicle longitude and latitude, the front direction and the vehicle altitude information. Then the map element information is obtained, and the map element information may include the map element name, map element type, map element longitude and latitude, map element area, map element height difference, and map element introduction, as shown in Table 1 above. Taking the vehicle center axis of the target vehicle as the 0 degree direction, calculate the relative orientation between the map element and the center axis. At the same time, calculate the relative distance between the two based on the longitude and latitude of the map element and the longitude and latitude of the vehicle. Use a template to combine the map element information with the relative position to generate a natural language description, and the description template is: the orientation is {azimuth}, the distance is {distance} meters, the name is {name}, the range is approximately {range}, the height difference is {height difference}, and {introduction}. For example, the natural language description of Wuyun Lake Park in Table 1 is generated: the orientation is 45 degrees to the right, the distance is 200 meters, the name is Wuyun Lake Park, the scope is about 5,000 square meters, the height difference is 20 meters, and Wuyun Lake Park is an urban oasis that integrates natural landscapes, leisure activities and cultural experiences. The park is named after the Wuyun Lake in the center. The lake water comes from the nearby mountains. The scenery varies in different seasons. In spring, flowers are in full bloom, in summer, trees are shady, in autumn, the ground is golden, and in winter, it is covered with snow. Similarly, the natural language description of each map element in the map data corresponding to the target vehicle is generated according to the above method to form a natural language point set of map elements.

[0062] In this embodiment, there are two main interaction modes when the user uses the travel assistant: pure voice interaction and voice combined with gesture interaction. Taking voice combined with gesture interaction as an example, for the voice input part, after the travel assistant is started, the user input processing module converts the received user voice input into the first text information. For the user gesture part, after the travel assistant is started, the user input processing module will turn on the camera in the car to capture the user gesture. When it is detected that the user has a gesture input, the captured user gesture image will be input into the preset trained motion recognition model. The model will calculate the gesture direction and convert the gesture direction into the relative direction relative to the central axis of the car, expressed as an azimuth (such as 45 degrees to the left and 30 degrees to the right), and obtain gesture information. Convert the gesture information into the second text information. Combine the first text information and the second text information to obtain the combined information, and determine the positioning instruction input by the user according to the combined information. Combine the voice and gesture input by the user, and convert these two parts of the user input into natural language text. The user input processing example is shown in Table 2:

[0063] Table 2

[0064]

[0065] It is understandable that this embodiment and Table 2 are only examples, and the processing information included in the user input processing can be increased, reduced or changed according to actual needs, and the corresponding content can also be modified according to actual needs, and this application does not limit this. In addition, the merged text in this embodiment can be merged according to actual needs, and is not limited to the direct merging illustrated above, and can also be adjusted to a more concise text after merging, and this application does not limit this.

[0066] In this embodiment, the text matching module calculates text similarity using the merged text and the natural language description of the map element, sorts them from large to small by similarity, and selects the top n (eg, 5) POIs whose similarity is greater than a preset threshold.

[0067] In this embodiment, the matching result post-processing process of the matching result processing module is as follows: Figure 6 As shown, Figure 6 This is a flowchart of a map element introduction method after obtaining a matching result, shown in an exemplary embodiment of the present application. When there is only one POI in the matching result, the POI is directly output. When there are more than one matching result, it is necessary to continue processing, and it is necessary to interact with the user twice or multiple times through voice, such as asking the user the following questions: "How many meters away is the river?", "What is the color?", "What is the approximate height?". Then calculate the similarity between the result and the TOP n POI points, and take the most similar POI as the target element.

[0068] In this embodiment, the large language model processing module obtains user portrait information, such as age, gender, and place of origin, and combines the user portrait information with the POI name to obtain the target prompt word. After the target prompt word is input into the large language model, the model will output a detailed description of the POI, namely the element introduction information.

[0069] In this embodiment, the TTS broadcast module sends the output text of the large language model, namely the element introduction information, to the TTS broadcast module, and the TTS broadcast module converts the text information into voice and plays it to the user.

[0070] It should be noted that the vehicle location information, user portrait information and other user data obtained in the embodiments of the present application are obtained with the user, or actively submitted after the user's relevant instructions, or are necessarily uploaded when the user uses the corresponding application through the client, web page, etc. In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of the user's personal information involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and do not violate public order and good customs.

[0071] The present application can combine map element information with vehicle location information to generate natural language text. Map elements are static elements, and vehicle bodies are dynamic elements. By combining the information of the two, the static nature of the map and the dynamic nature of the vehicle body can be better utilized; the user's gesture input can be converted into natural language text. When the user has a gesture, the direction indicated by the user can be converted into an azimuth text relative to the central axis of the car. By adding user gesture information, the matching results are made more accurate; when multiple POI points are matched, secondary interaction with the user through language can better eliminate ambiguity and retain the POI points that the user is most likely to be interested in; the map elements and user input can be converted into natural language text for matching; a complete travel assistant capability is provided to enhance the intelligence of the car and user satisfaction.

[0072] Figure 7 is a block diagram of a map element introduction device shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The implementation environment shown in the figure is specifically configured in the computer device 102. The device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applied.

[0073] like Figure 7 As shown, the exemplary map element introduction device includes: an acquisition module 710, a determination module 720 and an introduction module 730.

[0074] Among them, the acquisition module 710 is used to obtain map data corresponding to the target vehicle, wherein the map data includes multiple map elements; the determination module 720 is used to respond to the positioning instruction input by the user, perform text similarity matching on the relative position between the target vehicle and the map element according to the positioning instruction, and determine the target element from the map elements according to the matching result; the introduction module 730 is used to generate element introduction information according to the target element, and display the element introduction information to the user.

[0075] It should be noted that the map element introduction device provided in the above embodiment and the map element introduction method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In actual application, the map element introduction device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0076] An embodiment of the present application also provides an electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the electronic device implements the map element introduction method provided in the above-mentioned embodiments.

[0077] Figure 8 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 8 The computer system 800 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0078] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 to the random access memory (RAM) 803, such as executing the methods provided in the above-mentioned various embodiments. In the RAM 803, various programs and data required for system operation are also stored. The CPU 801, the ROM 802 and the RAM 803 are connected to each other through the bus 804. The input / output (I / O) interface 805 is also connected to the bus 804.

[0079] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.

[0080] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 809, and / or installed from a removable medium 811. When the computer program is executed by a central processing unit (CPU) 801, various functions defined in the system of the present application are executed.

[0081] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0082] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0084] Another aspect of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the map element introduction method provided in the above embodiments. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently without being assembled into the electronic device.

[0085] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0086] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the map element introduction method provided in each of the above embodiments.

[0087] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0088] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0089] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for introducing map elements, characterized in that: include: Acquire map data corresponding to the target vehicle, wherein the map data includes a plurality of map elements; In response to a positioning instruction input by a user, performing text similarity matching on the relative positions of the target vehicle and the map elements according to the positioning instruction, and determining a target element from the map elements according to the matching result; Generate element introduction information according to the target element, and display the element introduction information to the user.

2. The map element introduction method according to claim 1, characterized in that: Get the map data corresponding to the target vehicle, including: Acquiring vehicle position information of the target vehicle; determining a target range based on the vehicle position information and a preset range threshold; Map data corresponding to the target vehicle is generated according to the map elements in the target range.

3. The map element introduction method according to claim 1, characterized in that: Before acquiring the map data corresponding to the target vehicle, the method further includes: Get the current command entered by the user; Classifying the current instruction to obtain an instruction type; If the instruction type is a map-related type, map data corresponding to the target vehicle is obtained, and the positioning instruction is generated according to the current instruction.

4. The map element introduction method according to claim 3, characterized in that: Generating the positioning instruction according to the current instruction includes at least one of the following: If the current instruction includes a location keyword, generating the positioning instruction according to the location keyword; If the current instruction does not include a location keyword, an action instruction is obtained by capturing an image of the user and performing action recognition on the captured user image, and a location keyword is determined according to the action instruction to generate the positioning instruction according to the location keyword.

5. The map element introduction method according to claim 1, characterized in that: Performing text similarity matching on the relative positions of the target vehicle and the map element according to the positioning instruction, and determining the target element from the map element according to the matching result, including: Determining the relative position between the target vehicle and the map element according to the vehicle position information of the target vehicle and the element position information corresponding to the map element; The text similarity between the positioning instruction and the relative position is calculated. If the text similarity is greater than a preset threshold, a matching result is determined to be that the positioning instruction matches the relative position, and a map element corresponding to the relative position is determined as the target element.

6. The method for introducing map elements according to any one of claims 1 to 5, characterized in that: The method further comprises: If there are multiple matching results, interactive information is displayed to the user, and feedback instructions entered by the user are obtained; Matching the feedback instruction with the map element in the matching result to obtain a new matching result; The target element is determined from the map elements in the matching results according to the new matching result.

7. The method for introducing map elements according to any one of claims 1 to 5, characterized in that: Generate element introduction information according to the target element, including: Acquire user portrait information, and combine the user portrait information with the target element to obtain input information; Inputting the input information into a preset prompt word template to obtain a target prompt word; The target prompt word is input into a preset large language model to obtain the element introduction information.

8. The method for introducing map elements according to any one of claims 1 to 5, characterized in that: The method further comprises: receiving a storage instruction input by a user, and acquiring an element image of the target element; The target element and the element image are stored in a preset storage location, and historical information is generated after a preset time period, and a user is reminded according to the historical information.

9. A map element introduction device, characterized in that: include: An acquisition module, used to acquire map data corresponding to the target vehicle, wherein the map data includes a plurality of map elements; a determination module, configured to respond to a positioning instruction input by a user, perform text similarity matching on the relative positions between the target vehicle and the map elements according to the positioning instruction, and determine the target element from the map elements according to the matching result; The introduction module is used to generate element introduction information according to the target element and display the element introduction information to the user.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the map element introduction method as described in any one of claims 1 to 8.

Images