Vehicle-mounted tour guide interaction method and device, electronic equipment, storage medium and vehicle

Abstract

The invention discloses a vehicle-mounted tour guide interaction method and device, electronic equipment, a storage medium and a vehicle, and relates to the field of voice interaction.The method comprises the steps that vehicle positioning coordinate information is acquired; matching scenic spot POI information based on the vehicle positioning coordinate information; collecting external visual information of the vehicle according to the matched scenic spot POI information; extracting associated information in external visual information of the vehicle according to the scenic spot POI information; generating cue words according to associated information extracted from the external visual information of the vehicle, and outputting the cue words to a human-computer interaction interface; acquiring personnel voice information; analyzing the voice information to generate semantic character information; inputting the semantic character information and the cue word into a GPT model for processing, and generating a TTS broadcast reply; and according to the reply broadcasted by the TTS, continuously performing voice interaction based on the scenic spot POI information and the visual information outside the vehicle. Through the scheme, tourist guide interaction is continuously promoted along with vehicle movement according to vehicle movement and continuous scenic spot conversion, and tourist guide continuity is kept.

Description

Technical Field

[0001] The present application relates to the field of voice interaction, and in particular to an in-vehicle tour guide interaction method, an in-vehicle tour guide interaction device, an electronic device, a storage medium and a vehicle. Background Art

[0002] In most current self-driving travel scenarios, it is difficult or costly to bring a tour guide with you. Traditional use of local scenic spot introduction manuals to learn about scenic spot knowledge cannot be detailed, and traditional electronic manuals or paper manuals have shortcomings such as slow update speed and inability to interact freely, especially for some scenic spots with historical events and related introductions are relatively brief. Some tourists may have higher requirements for some travel information, historical background, celebrity deeds, etc. related to scenic spots, and most products on the market cannot meet their needs.

[0003] With the popularization of AI applications, especially the popularization of large models represented by ChatGPT, the feedback accuracy and real-time capabilities of large models in multimodality, voice, and image (including video) have been greatly improved.

[0004] Therefore, a solution for in-vehicle tour guide interaction is needed. The solution can trigger the tour guide's explanation interaction through the scene conversion of the vehicle, use the vehicle's microphone, camera, and voice, text, image, and audio input capabilities to provide users with a variety of scene modes for real-time tourist attraction feedback, and achieve an immersive experience of human-computer interaction. Summary of the invention

[0005] The purpose of the present invention is to provide a vehicle-mounted tour guide interaction method, a vehicle-mounted tour guide interaction device, an electronic device, a storage medium and a vehicle, which at least solves the technical problem of tour guide interaction, the problem of automatically switching tour guide topics, and the problem of automatically retrieving tour guide related information.

[0006] The present invention provides the following scheme:

[0007] According to one aspect of the present invention, a vehicle tour guide interaction method is provided, the vehicle tour guide interaction method comprising:

[0008] Get vehicle positioning coordinate information;

[0009] Matching scenic spot POI information based on the vehicle positioning coordinate information;

[0010] According to the matching scenic spot POI information, collecting the external visual information of the vehicle;

[0011] Extracting relevant information from the external visual information of the vehicle according to the POI information of the scenic spot;

[0012] Generating prompt words and outputting them to a human-computer interaction interface according to the associated information extracted from the external visual information of the vehicle;

[0013] Collect personnel voice information;

[0014] Analyze speech information and generate semantic text information;

[0015] Inputting the semantic text information and the prompt word into the GPT model for processing to generate a TTS broadcast reply;

[0016] According to the reply of the TTS broadcast, the voice interaction based on the scenic spot POI information and the visual information outside the vehicle is continued.

[0017] Furthermore, matching the scenic spot POI information based on the vehicle positioning coordinate information includes:

[0018] Retrieving a third-party map API based on the vehicle location coordinate information;

[0019] Based on the third-party map API, obtain the scenic spot information associated with POI;

[0020] Stores data matching POI information.

[0021] Furthermore, collecting the external visual information of the vehicle according to the matching scenic spot POI information includes:

[0022] Synchronously upload the vehicle's external visual information to the GPT model for recognition;

[0023] Mark the coordinate information of the third-party map according to the recognition result;

[0024] The POI information of the scenic spot is bundled according to the coordinate information and then the bundled data is stored.

[0025] Furthermore, extracting the associated information from the external visual information of the vehicle according to the scenic spot POI information includes:

[0026] The GPT model outputs prompt words and collects data on voice interaction;

[0027] Determine the matching degree between the vehicle's external visual information and the scenic spot POI information based on the prompt word and voice interaction data;

[0028] Sorting according to the matching degree;

[0029] According to the ranking of the matching degrees, scenic spot keywords are selected for retrieval and output in voice interaction;

[0030] The output in the voice interaction includes generating a new prompt word.

[0031] Furthermore, the sorting according to the matching degree includes:

[0032] After bundling the POI information of the scenic spot according to the coordinate information, the bundled data is stored and a prompt word list is generated;

[0033] The prompt word list is sorted according to the matching degree;

[0034] According to the vehicle positioning coordinate information, the prompt word list and the sorting are refreshed.

[0035] Furthermore, the extracting of associated information from the external visual information of the vehicle includes:

[0036] Setting feature information for judging the matching degree between the vehicle's external visual information and the scenic spot POI information based on the data of the prompt words and voice interaction;

[0037] quantifying the degree of matching according to the feature information;

[0038] According to the quantification of the matching degree, a matching degree threshold is set;

[0039] Determine whether the matching degree between the current vehicle external visual information and the scenic spot POI information is greater than the matching degree threshold;

[0040] If it is less than the matching degree threshold, the current voice interaction is terminated.

[0041] According to two aspects of the present invention, there is provided a vehicle-mounted tour guide interaction device, the vehicle-mounted tour guide interaction device comprising:

[0042] Vehicle positioning module, used to obtain vehicle positioning coordinate information;

[0043] An information matching module, used to match scenic spot POI information based on the vehicle positioning coordinate information;

[0044] A visual acquisition module, used to collect external visual information of the vehicle according to the matching scenic spot POI information;

[0045] An information extraction module, used to extract relevant information from the external visual information of the vehicle according to the POI information of the scenic spot;

[0046] A prompt information module, used to generate prompt words and output them to the human-computer interaction interface according to the associated information extracted from the external visual information of the vehicle;

[0047] Voice collection module, used to collect personnel voice information;

[0048] Speech analysis module, used to analyze speech information and generate semantic text information;

[0049] An interactive response module, used for inputting the semantic text information and the prompt word into the GPT model for processing, and generating a TTS broadcast reply;

[0050] The interactive extension module is used to continue the voice interaction based on the scenic spot POI information and the visual information outside the vehicle according to the reply of the TTS broadcast.

[0051] According to three aspects of the present invention, there is provided an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0052] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the in-vehicle tour guide interaction method.

[0053] According to four aspects of the present invention, a computer-readable storage medium is provided, comprising: a computer program executable by an electronic device is stored therein, and when the computer program runs on the electronic device, the electronic device executes the steps of the in-vehicle tour guide interaction method.

[0054] According to five aspects of the present invention, there is provided a vehicle, comprising:

[0055] An electronic device for implementing the steps of the in-vehicle tour guide interaction method;

[0056] A processor, the processor runs a program, and when the program runs, the steps of the in-vehicle tour guide interaction method are executed from the data output by the electronic device;

[0057] The storage medium is used to store a program, and when the program is running, the steps of the in-vehicle tour guide interaction method are executed for the data output from the electronic device.

[0058] Through the above scheme, the following beneficial technical effects are obtained:

[0059] This application continuously converts the information related to the scenic spots and the tour guide through the moving position of the vehicle, so that the tour guide interaction continues to advance as the vehicle moves, maintaining the continuity of the tour guide interaction.

[0060] This application accurately obtains the retrieval object of the tour guide information by adding visual information, so that the theme content of the tour guide is associated with the user's actual concern.

[0061] This application continuously generates prompt words so that the interactive process of the tour guide interaction revolves around relevant attractions and is constantly updated, enriching the knowledge base retrieval introduced by the tour guide. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1The figure is a flow chart of a vehicle tour guide interaction method provided by one or more embodiments of the present invention.

[0063] Figure 2 It is a structural diagram of a vehicle tour guide interaction device provided by one or more embodiments of the present invention.

[0064] Figure 3 A schematic diagram of a method for identifying scenic spots based on coordinates according to a specific embodiment of the present invention.

[0065] Figure 4 A schematic diagram of a method for converting sightseeing into text based on a specific embodiment of the present invention.

[0066] Figure 5 A schematic diagram of a prompt word generating method according to a specific embodiment of the present invention.

[0067] Figure 6 A schematic diagram of an audio interaction method according to a specific embodiment of the present invention.

[0068] Figure 7 A structural block diagram of an electronic device of a vehicle tour guide interaction method provided by one or more embodiments of the present invention. DETAILED DESCRIPTION

[0069] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] Figure 1 The figure is a flow chart of a vehicle tour guide interaction method provided by one or more embodiments of the present invention.

[0071] like Figure 2 The in-car tour guide interaction method shown includes:

[0072] Step S1, obtaining vehicle positioning coordinate information;

[0073] Step S2, matching scenic spot POI information based on vehicle positioning coordinate information;

[0074] Step S3, collecting the external visual information of the vehicle according to the matching scenic spot POI information;

[0075] Step S4, extracting relevant information from the external visual information of the vehicle according to the scenic spot POI information;

[0076] Step S5, generating prompt words based on the associated information extracted from the external visual information of the vehicle and outputting them to the human-computer interaction interface;

[0077] Step S6, collecting personnel voice information;

[0078] Step S7, parsing the voice information to generate semantic text information;

[0079] Step S8, inputting the semantic text information and the prompt word into the GPT model for processing to generate a TTS broadcast reply;

[0080] Step S9, according to the reply of the TTS broadcast, continue the voice interaction based on the scenic spot POI information and the visual information outside the vehicle.

[0081] Specifically, in this embodiment, based on the navigation of vehicle-mounted GPS, Beidou, etc., the longitude and latitude coordinate information is obtained, and the matching scenic spot POI information is queried on the third-party map according to the longitude and latitude coordinate information. When the current visual information is imported, the image matching the scenic spot POI information is associated. For example, based on the scenic spot POI information, the scenic spot of a certain Buddha statue is learned, and based on the Buddha statue image in the current visual information, the Buddha statue can be used as the currently observable scenic spot object for information extraction. The descriptive vocabulary corresponding to the scenic spot object can be output to the human-computer interaction interface, and after seeing the prompt word, the personnel can match it with the current visual observation to form a voice interaction. The voice information sent by the personnel can be parsed, the semantics therein can be extracted, and it can be input into a large model together with the prompt word, such as processed in the GPT model to generate a TTS file for voice broadcast. According to the reply of TTS broadcast, voice interaction based on scenic spot POI information and visual information outside the vehicle is continuously performed. For example, each voice broadcast is based on the semantic extraction of voice information issued by the current personnel, and is input into the large model together with the prompt words; as the personnel continuously update the voice information to extract new semantics, and input them into the large model together with the prompt words, the TTS file content processed and generated in the GPT model for voice broadcast is constantly refreshed. In addition, as a mobile platform, the vehicle continuously updates the longitude and latitude coordinates, and the scenic spot POI information changes accordingly, which can also cause the prompt words to be improperly updated. As the personnel continuously update the voice information to extract new semantics, and input them into the large model together with the updated prompt words, the TTS file content processed and generated in the GPT model for voice broadcast is constantly refreshed. In addition, the visual camera can be a vehicle-mounted camera or a wearable device camera. As the images collected by the camera are constantly changing, the currently observable scenic spot objects are constantly changing, which in turn causes the prompt words to be improperly updated. As the personnel continuously update the voice information to extract new semantics, and input them into the large model together with the updated prompt words, the TTS file content processed and generated in the GPT model for voice broadcast is constantly refreshed.

[0082] In this embodiment, matching scenic spot POI information includes:

[0083] Retrieve third-party map API based on vehicle location coordinate information;

[0084] Based on the third-party map API, obtain the scenic spot information associated with POI;

[0085] Stores data matching POI information.

[0086] Specifically, in this embodiment, a third-party map API is retrieved based on the vehicle location coordinate information;

[0087] Based on the third-party map API, obtain the scenic spot information associated with the POI; store the data matching the scenic spot POI information.

[0088] Scenic spots include distant scenic spots and near scenic spots. When matching scenic spot POI information, the visual observability is considered and the priority is set. The data of scenic spot POI information is matched according to the visual observability priority.

[0089] When a visual image includes multiple scenic spots, the subject of the semantic interaction can be determined based on the semantics of the user's voice analysis, and then the prompt words can be updated in time to avoid invalid voice interactions that "answer irrelevant questions". Alternatively, the interface can form multiple prompt words based on the current multiple scenic spots for users to choose from, or prompt users to which of the multiple scenes they see are not ordinary scenes but scenes worth understanding.

[0090] In this embodiment, according to the matching scenic spot POI information, collecting the external visual information of the vehicle includes:

[0091] Synchronously upload the vehicle's external visual information to the GPT model for recognition;

[0092] Mark the coordinate information of the third-party map according to the recognition result;

[0093] The POI information of the scenic spot is bundled according to the coordinate information and then the bundled data is stored.

[0094] Specifically, in this embodiment, visually recognized information is identified through a GPT model to identify and extract scenic objects with tour guide value, mark the coordinate information of a third-party map, and then bundle the POI information of the scenic spot according to the coordinate information and store the bundled data for use as historical data or shared data.

[0095] Not every time, the semantic extraction of speech information sent by personnel and inputting it into the big model together with the prompt words to generate the TTS file content of the speech broadcast is successful. In each round of speech interaction, the GPT model recognizes the results of the interaction, such as asking whether to give the desired related information introduction through the prompt words, such as recognizing whether the voice interaction in this round is an effective introduction of scene information through the tone of speech interaction, and further training the scene information retrieval ability and screening range of the big model, so that the matching degree between the retrieved information and the personnel's questions is continuously improved.

[0096] In this embodiment, extracting the associated information from the external visual information of the vehicle according to the scenic spot POI information includes:

[0097] The GPT model outputs prompt words and collects data on voice interaction;

[0098] Determine the matching degree between the vehicle's external visual information and the scenic spot POI information based on the prompt word and voice interaction data;

[0099] Sort by matching degree;

[0100] According to the ranking of matching degree, select the scenic spot keywords for retrieval and output them in voice interaction;

[0101] The output in voice interaction includes generating new prompt words.

[0102] Specifically, in this embodiment, the GPT model outputs prompt words and collects voice interaction data; the matching degree between the vehicle's external visual information and the scenic spot POI information is determined based on the prompt words and voice interaction data. For example, the user directly responds to the situation that the prompt words do not match and the vehicle's external visual information matches the scenic spot POI information through voice, such as indirectly understanding the situation that the prompt words do not match and the vehicle's external visual information matches the scenic spot POI information through the tone of the user's voice.

[0103] After sorting by matching degree, the keywords of scenic spots are selected according to the sorting for retrieval and output in voice interaction. For example, some users are interested in building structures, while others are interested in textual content. After several rounds of interaction, the matching degree of the vehicle's external visual information and the POI information of scenic spots is sorted. Based on the sorting, the tour guide interaction process is advanced in coordination with the user's voice interaction topic. For example, new prompt words are output in voice interaction, and the tour guide interaction process is continued based on the new prompt words.

[0104] In this embodiment, sorting according to the matching degree includes:

[0105] After bundling the POI information of the scenic spot according to the coordinate information, the bundled data is stored and a prompt word list is generated;

[0106] The prompt word list is sorted according to the matching degree;

[0107] Refresh the prompt word list and sort it according to the vehicle positioning coordinate information.

[0108] Specifically, in this embodiment, as multiple users visit the same scenic spot, or the same scenic spot has multiple topics of interest, the bundled data is stored after the scenic spot POI information is bundled according to the coordinate information, and a list of prompt words is generated to provide users with a more open topic selection, wherein the prompt words are arranged in order of matching degree to reduce the workload of users in flipping through, or form prompts to users to quickly expand the topic.

[0109] As the vehicle is constantly moving, the visual information is in an absolute refresh state. The visual information set at each moving position is constantly changing, and the corresponding prompt word list is constantly refreshed. At the same time, the POI information of the corresponding scenic spot is updated as the vehicle is constantly moving, and a new prompt word list and the order of the prompt word list are constantly formed. For example, new prompt words are constantly added to the prompt word list, and prompt words are constantly removed from the list. According to the current POI information of the scenic spot, the new prompt word list is sorted.

[0110] In this embodiment, extracting the associated information from the external visual information of the vehicle includes:

[0111] Setting feature information for judging the matching degree between the vehicle's external visual information and the scenic spot POI information based on the data of the prompt words and voice interaction;

[0112] Quantify the matching degree based on feature information;

[0113] According to the quantification of the matching degree, a matching degree threshold is set;

[0114] Determine whether the matching degree between the current vehicle external visual information and the scenic spot POI information is greater than the matching degree threshold;

[0115] If,is less than the matching threshold, the current voice interaction ends.

[0116] Specifically, feature information is set to determine the matching degree between the vehicle's external visual information and the POI information of the scenic spot according to the data of the prompt word and voice interaction. For example, if the visual information includes multiple suspected scenes, the features in the picture, such as outline, color, texture, etc., are extracted according to the pictures in the picture database. The feature information is quantified, such as by assigning weight values, to form a quantitative matching degree of the feature information. According to the quantification of the matching degree, a matching degree threshold is set, and scene objects with "topic popularity" are selected to form a voice interaction for matching the current vehicle's external visual information with the POI information of the scenic spot.

[0117] Figure 2 It is a structural diagram of a vehicle tour guide interaction device provided by one or more embodiments of the present invention.

[0118] like Figure 2The in-vehicle tour guide interactive device shown includes: a vehicle positioning module, an information matching module, a visual acquisition module, an information extraction module, a prompt information module, a voice acquisition module, a voice analysis module, an interactive response module, and an interactive extension module;

[0119] Vehicle positioning module, used to obtain vehicle positioning coordinate information;

[0120] An information matching module is used to match the POI information of scenic spots based on the vehicle positioning coordinate information;

[0121] The visual acquisition module is used to collect the external visual information of the vehicle according to the matching scenic spot POI information;

[0122] An information extraction module is used to extract relevant information from the vehicle's external visual information based on the POI information of the scenic spot;

[0123] A prompt information module is used to generate prompt words and output them to the human-computer interaction interface based on the associated information extracted from the external visual information of the vehicle;

[0124] Voice collection module, used to collect personnel voice information;

[0125] Speech analysis module, used to analyze speech information and generate semantic text information;

[0126] The interactive response module is used to input semantic text information and prompt words into the GPT model for processing and generate TTS broadcast replies;

[0127] The interactive extension module is used to continue the voice interaction based on the scenic spot POI information and the visual information outside the vehicle according to the response of the TTS broadcast.

[0128] It is worth noting that although the present system only discloses a vehicle positioning module, an information matching module, a visual acquisition module, an information extraction module, a prompt information module, a voice acquisition module, a voice analysis module, an interactive response module, and an interactive extension module, it does not mean that the present device is limited to the above-mentioned basic functional modules. On the contrary, what the present invention wants to express is that, on the basis of the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with the prior art to form an infinite number of embodiments or technical solutions. In other words, the present system is open rather than closed. Just because the present embodiment only discloses individual basic functional modules, it cannot be considered that the scope of protection of the claims of the present invention is limited to the above-mentioned basic functional modules.

[0129] Through the above scheme, the following beneficial technical effects are obtained:

[0130] This application continuously converts the information related to the scenic spots and the tour guide through the moving position of the vehicle, so that the tour guide interaction continues to advance as the vehicle moves, maintaining the continuity of the tour guide interaction.

[0131] This application accurately obtains the retrieval object of the tour guide information by adding visual information, so that the theme content of the tour guide is associated with the user's actual concern.

[0132] This application continuously generates prompt words so that the interactive process of the tour guide interaction revolves around relevant attractions and is constantly updated, enriching the knowledge base retrieval introduced by the tour guide.

[0133] Figure 3 A schematic diagram of a method for identifying scenic spots based on coordinates according to a specific embodiment of the present invention.

[0134] Figure 4 A schematic diagram of a method for converting sightseeing into text based on a specific embodiment of the present invention.

[0135] Figure 5 A schematic diagram of a prompt word generating method according to a specific embodiment of the present invention.

[0136] Figure 6 A schematic diagram of an audio interaction method according to a specific embodiment of the present invention.

[0137] In a specific embodiment, based on GPT-4 and other related models, text interaction is used to preliminarily process the input images / sounds into text content accepted by GPT, and then quickly set up with relevant prompts to set the GPT tour guide "personality".

[0138] In this embodiment, the voice interaction scheme of the on-board tour guide role mainly includes:

[0139] First, confirm the basic POI information of the scenic spot based on GPS coordinate positioning and confirm the basic location near the scenic spot.

[0140] Second, collect the AVM / DVR video stream for image analysis and convert the analysis results into text.

[0141] Third, based on the information generated in the first and second steps, determine the name of the scenic spot and the objects related to the picture, such as setting it as a prompt word for GPT through the API, and completing the selection and input of the prompt word by voice interaction.

[0142] Fourth, the in-vehicle voice recognition recognizes the user's voice as semantics and inputs it into GPT. GPT returns the answer based on the prompt words and user semantics and announces it through TTS.

[0143] In this embodiment, if Figure 3 The method for confirming scenic spots based on coordinates shown confirms basic scenic spot POI information based on GPS coordinate positioning and confirms the basic location near the scenic spot.

[0144] The vehicle operating system application obtains the longitude and latitude data of the GPS sensor through the Android standard API. It sends the coordinate data to the third-party map API, and obtains the POI information of the scenic spot after a series of authentication processes of the third-party API.

[0145] The POI information of the scenic spot includes possible nearby scenic spots, and the above information is saved to the server for storage and backup.

[0146] In this embodiment, if Figure 4 The scenic spot-based visual text conversion method shown collects the analysis results of AVM / DVR video streams for image analysis and converts them into text.

[0147] The AVM (surround view camera) is located on the vehicle's HAD device. The on-board CSC obtains the video stream of the AVM camera through the LVDS channel, and obtains the front and rear view video streams of the DVR camera through the internal Ethernet of the vehicle. The application uses the standard Android Camera API to take pictures through the on-board CSC, which is the Android system on the central control entertainment system, by setting the camera input parameters and output parameters. After the digital photos are stored, the application transmits the pictures to ChatGPT, and obtains the results of GPT feedback through prompts such as "What attraction is this?" After the digital results are stored, they are uploaded to the storage service mentioned in the first step and paired.

[0148] In this embodiment, if Figure 5 The prompt word generation method shown determines the name of the scenic spot and the related objects of the picture according to the information generated in the first step and the second step, such as setting the prompt word to GPT through the API, and completing the selection and input of the prompt word by voice interaction.

[0149] The information pairs generated in the first step and the second step are used as the basis for judgment. According to the information matching degree, all the possibilities in the POI list are sorted to generate the priority of different information in the list, such as the probability percentage of POI-1, the probability percentage of POI-2, and so on.

[0150] Select the most likely POI information and put it into the Prompt template, for example, "I am xxx, the current attraction is "xxx, the specific coordinates are x, y. Please give me some detailed information as a tour guide and have a conversation with me." Set the above combined information as a system prompt through the ChatGPT API.

[0151] In this embodiment, if Figure 6 The audio interaction method shown converts the user's voice into text through the in-vehicle voice recognition and inputs it into GPT, and GPT returns the answer and broadcasts it through TTS.

[0152] After setting up the GPT tour guide "personality", the user uses the system as a tour guide and freely asks questions related to the scenery. The main process is as follows:

[0153] In the first step, after the microphone collects the audio, use the APP on the CSC to record the audio and use the third-party API to identify the content and store it as an item in the context of the chat list.

[0154] In step 2, the entire context is then sent to GPT, which responds based on the content and context of the user's question and stores the new content in the context list.

[0155] Step 3: Play the received reply as TTS to the user through the car amplifier.

[0156] The above three steps are repeated until the user exits and stops asking questions or initiates another round of voice interaction.

[0157] Figure 7 A structural block diagram of an electronic device of a vehicle tour guide interaction method provided by one or more embodiments of the present invention.

[0158] like Figure 7 As shown, the present application provides an electronic device, including: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0159] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of an in-vehicle tour guide interaction method.

[0160] The present application also provides a computer-readable storage medium storing a computer program executable by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of an in-vehicle tour guide interaction method.

[0161] The present application also provides a vehicle, comprising:

[0162] Electronic device, used to implement the steps of the vehicle tour guide interaction method;

[0163] A processor, the processor runs a program, and when the program runs, the steps of the in-vehicle tour guide interaction method are executed from the data output by the electronic device;

[0164] The storage medium is used to store a program, and when the program is running, the steps of the vehicle tour guide interaction method are executed for the data output from the electronic device.

[0165] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0166] The electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory. The operating system can be any one or more computer operating systems that implement electronic device control through a process, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. In addition, in an embodiment of the present invention, the electronic device can be a handheld device such as a smart phone or a tablet computer, or can be an electronic device such as a desktop computer or a portable computer, which is not particularly limited in the embodiment of the present invention.

[0167] The execution subject of the electronic device control in the embodiment of the present invention may be an electronic device, or a functional module in the electronic device that can call and execute a program. The electronic device may obtain the firmware corresponding to the storage medium. The firmware corresponding to the storage medium is provided by the supplier. The firmware corresponding to different storage media may be the same or different, which is not limited here. After the electronic device obtains the firmware corresponding to the storage medium, the firmware corresponding to the storage medium may be written into the storage medium, specifically, the firmware corresponding to the storage medium may be burned into the storage medium. The process of burning the firmware into the storage medium may be implemented using existing technology, which will not be described in detail in the embodiment of the present invention.

[0168] The electronic device may also obtain a reset command corresponding to the storage medium. The reset command corresponding to the storage medium is provided by the supplier. The reset commands corresponding to different storage media may be the same or different, and are not limited here.

[0169] At this time, the storage medium of the electronic device is a storage medium in which the corresponding firmware is written, and the electronic device can respond to the reset command corresponding to the storage medium in the storage medium in which the corresponding firmware is written, so that the electronic device resets the storage medium in which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command can be implemented by the existing technology and will not be described in detail in the embodiments of the present invention.

[0170] For the convenience of description, the above devices are described in terms of functions and are divided into various units and modules. Of course, when implementing the present application, the functions of each unit and module can be implemented in the same or multiple software and / or hardware.

[0171] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined.

[0172] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0173] It can be known from the description of the above implementation modes that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute the methods described in the various implementation modes of the present application or certain parts of the implementation modes.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for interactive tour guide in a vehicle, It is characterized in that The in-vehicle tour guide interaction method comprises: Get vehicle positioning coordinate information; Matching scenic spot POI information based on the vehicle positioning coordinate information; According to the matching scenic spot POI information, collecting the external visual information of the vehicle; Extracting relevant information from the external visual information of the vehicle according to the POI information of the scenic spot; Generating prompt words and outputting them to a human-computer interaction interface according to the associated information extracted from the external visual information of the vehicle; Collect personnel voice information; Analyze speech information and generate semantic text information; Inputting the semantic text information and the prompt word into the GPT model for processing to generate a TTS broadcast reply; According to the reply of the TTS broadcast, the voice interaction based on the scenic spot POI information and the visual information outside the vehicle is continued.

2. The in-vehicle tour guide interaction method according to claim 1, It is characterized in that The matching of scenic spot POI information based on the vehicle positioning coordinate information includes: Retrieving a third-party map API based on the vehicle location coordinate information; Based on the third-party map API, obtain the scenic spot information associated with POI; Stores data matching POI information.

3. The in-vehicle tour guide interaction method according to claim 2, It is characterized in that The collecting of the external visual information of the vehicle according to the matching scenic spot POI information comprises: Synchronously upload the vehicle's external visual information to the GPT model for recognition; Mark the coordinate information of the third-party map according to the recognition result; The POI information of the scenic spot is bundled according to the coordinate information and then the bundled data is stored.

4. The in-vehicle tour guide interaction method according to claim 3, It is characterized in that The extracting of associated information from the external visual information of the vehicle according to the scenic spot POI information includes: The GPT model outputs prompt words and collects data on voice interaction; Determine the matching degree between the vehicle's external visual information and the scenic spot POI information based on the prompt word and voice interaction data; Sorting according to the matching degree; According to the ranking of the matching degrees, scenic spot keywords are selected for retrieval and output in voice interaction; The output in the voice interaction includes generating a new prompt word.

5. The in-vehicle tour guide interaction method according to claim 4, It is characterized in that The sorting according to the matching degree comprises: After bundling the POI information of the scenic spot according to the coordinate information, the bundled data is stored and a prompt word list is generated; The prompt word list is sorted according to the matching degree; According to the vehicle positioning coordinate information, the prompt word list and the sorting are refreshed.

6. The in-vehicle tour guide interaction method according to claim 5, It is characterized in that The extracting of associated information from the vehicle's external visual information includes: Setting feature information for judging the matching degree between the vehicle's external visual information and the scenic spot POI information based on the data of the prompt words and voice interaction; quantifying the degree of matching according to the feature information; According to the quantification of the matching degree, a matching degree threshold is set; Determine whether the matching degree between the current vehicle external visual information and the scenic spot POI information is greater than the matching degree threshold; If it is less than the matching degree threshold, the current voice interaction is terminated.

7. An in-vehicle tour guide interactive device, It is characterized in that The vehicle-mounted tour guide interactive device comprises: Vehicle positioning module, used to obtain vehicle positioning coordinate information; An information matching module, used to match scenic spot POI information based on the vehicle positioning coordinate information; A visual acquisition module, used to collect external visual information of the vehicle according to the matching scenic spot POI information; An information extraction module, used to extract relevant information from the external visual information of the vehicle according to the POI information of the scenic spot; A prompt information module, used to generate prompt words and output them to the human-computer interaction interface according to the associated information extracted from the external visual information of the vehicle; Voice collection module, used to collect personnel voice information; Speech analysis module, used to analyze speech information and generate semantic text information; An interactive response module, used for inputting the semantic text information and the prompt word into the GPT model for processing, and generating a TTS broadcast reply; The interactive extension module is used to continue the voice interaction based on the scenic spot POI information and the visual information outside the vehicle according to the reply of the TTS broadcast.

8. An electronic device, It is characterized in that include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the in-vehicle tour guide interaction method according to any one of claims 1 to 6.

9. A computer-readable storage medium, It is characterized in that include: It stores a computer program executable by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of the in-vehicle tour guide interaction method described in any one of claims 1 to 6.

10. A vehicle, It is characterized in that include: An electronic device for implementing the steps of the in-vehicle tour guide interaction method according to any one of claims 1 to 6; A processor, the processor runs a program, and when the program runs, the data output from the electronic device executes the steps of the in-vehicle tour guide interaction method according to any one of claims 1 to 6; A storage medium for storing a program, wherein when the program is running, the program executes the steps of the in-vehicle tour guide interaction method according to any one of claims 1 to 6 for data output from the electronic device.

Images