Scenic spot user voice interaction method and device and related equipment

Abstract

The invention discloses a scenic spot user voice interaction method and apparatus, and a related device. The method comprises the steps of obtaining a current user coordinate and a current user orientation vector in a scenic spot; inputting the user coordinates and the user orientation vector into a pre-constructed scenic spot map model, and matching to obtain scenery in the current visual field of the user; constructing a view description feature vector of the scenery in the view; pre-training a travel question and answer model; receiving the view description feature vector and the user question as the input of the tourism question and answer model, and generating a personalized answer; according to the scenic spot user voice interaction method, the position and the orientation of the user are obtained through the Bluetooth headset, then the view description feature vector of the user is obtained, the view description feature vector and the question of the user are input into the pre-trained tourism question and answer model, personalized and real-time voice broadcast of the current scenic spot can be obtained, and the user experience is improved. Therefore, the tourism experience is improved.

Description

Technical Field

[0001] The present application relates to the field of tourism, and specifically to a method, device and related equipment for voice interaction with users of scenic spots. Background Art

[0002] In the current tourism scenario, traditional tourist attraction introductions are usually a simple tourist attraction introduction board with limited information and cannot directly answer users' specific questions. Users need to find the answers themselves.

[0003] Although existing travel apps can provide rich text, pictures and videos, they have an obvious shortcoming in the actual travel process: they cannot provide accurate information about the attractions that users are currently visiting in real time. Users often find it difficult to match the general information provided by the app with the actual attractions, which affects the consistency and satisfaction of the travel experience.

[0004] Although tour guide services are a traditional and effective solution to users’ need to obtain real-time knowledge of tourist attractions, they also have some limitations. For example, most mid-level tour guides can only provide limited information about tourist attractions, especially in terms of in-depth explanations in the fields of culture, politics, and history. In addition, tour guides cannot provide exclusive services to each user one-on-one, nor can they meet the personalized needs of multiple users at the same time. Moreover, the cost of hiring a tour guide may be an additional expense for some users.

[0005] Although some more advanced solutions, such as AI-based conversation functions and image recognition technology, can provide information to a certain extent, they often fail to provide users with accurate and timely background knowledge of attractions, and the interaction methods may be cumbersome and not personalized enough, resulting in a poor user experience. Summary of the invention

[0006] This embodiment provides a method, device and related equipment for voice interaction with users at a scenic spot, and explores how to solve the problem that users have difficulty obtaining accurate, timely and personalized introductions to scenic spots during travel.

[0007] In a first aspect, the present invention provides a method for scenic spot user voice interaction, comprising:

[0008] Obtaining the current user coordinates and user orientation vector within the scenic spot;

[0009] Inputting the user coordinates and the user orientation vector into a pre-built scenic spot map model to match the scene within the user's current field of view;

[0010] Constructing a visual field description feature vector of the scene within the visual field;

[0011] Pre-train a tourism question-answering model;

[0012] The visual field description feature vector and the user's question are received as inputs of the tourism question-answering model to generate a personalized answer.

[0013] In some of the embodiments, before obtaining the current user coordinates and user orientation vector in the scenic spot, the method includes:

[0014] Establishing a communication connection between the user's Bluetooth headset and the mobile terminal to form a wireless network; wherein the Bluetooth headset has a built-in electronic compass that can obtain the user's orientation vector and an accelerometer that can correct the tilt angle of the Bluetooth headset;

[0015] After receiving the visual field description feature vector and the user's question as inputs of the tourism question-answering model and generating a personalized answer, the method further includes:

[0016] The personalized answer is transmitted to the Bluetooth headset through the wireless network, and the personalized answer is broadcasted.

[0017] In some of the embodiments, obtaining the current user coordinates and user orientation vector in the scenic spot includes:

[0018] Installing a plurality of RFID readers and writers located in the scenic spot, and an RFID tag located on the Bluetooth headset; wherein the plurality of RFID readers and writers cover the scenic spot area that the user can reach;

[0019] If the RFID tag enters the scenic spot, the RFID reader receives the tag signal sent by the RFID tag;

[0020] The current user coordinates are determined according to the signal strength of the received tag signal and the time when the tag signal reaches the RFID reader.

[0021] In some embodiments, determining the current user coordinates according to the signal strength of the tag signal received and the time when the tag signal reaches the RFID reader includes:

[0022] According to the signal strength of the tag signal received, the first distance d from the RFID reader to the RFID tag is calculated. RSSI,i ;

[0023]

[0024] Among them, d RSSI,i is the first distance between the i-th RFID reader and the RFID tag, P r,i is the signal strength received by the i-th RFID reader, P tis the transmission power of the RFID tag, n is the path loss index, which is between 2 and 4, and C is the correction term for environmental noise and interference;

[0025] According to the time when the tag signal arrives at the RFID reader, calculate the time difference between the tag signal arriving at the two RFID readers; and according to the time difference, calculate the distance difference Δd from the RFID tag to the two RFID readers ij ;

[0026] Δd ij =c·Δt ij

[0027] Where, Δd ij is the distance difference between the RFID tag and the i-th RFID reader and the j-th RFID reader, c is the signal propagation speed, Δt ij is the time difference between the tag signal reaching the i-th RFID reader and the j-th RFID reader;

[0028] According to the first distance d RSSI,i And the distance difference Δd ij , determine the weighted distance d combined,i ;

[0029] d combined,i =α·d RSSI,i +β·Δd ij

[0030] Among them, d combined,i is the weighted distance from the i-th RFID reader to the RFID tag, and α is the first distance d RSSI,i The weight of β is the distance difference Δd ij The weight of

[0031] The weighted distance d combined,i and the location of the RFID reader (x i ,y i ,z i ), substitute into the position equation;

[0032]

[0033] Solve the position equation to obtain the current user coordinate Pos u (x u ,y u ,z u ).

[0034] In some of the embodiments, obtaining the current user coordinates and user orientation vector in the scenic spot further includes:

[0035] Use an electronic compass to measure the vector component of the magnetic field vector on the Bluetooth headset, and use the vector component as the original user orientation vector B of the Bluetooth headset.

[0036] B=(B x ,B y ,B z ),

[0037] Among them, B x ,B y ,B z They are respectively the X-axis vector component, Y-axis vector component and Z-axis vector component of the Bluetooth headset;

[0038] Use the accelerometer to detect the first tilt angle θ of the Bluetooth headset x and the second inclination angle θ y , the first tilt angle θ x and the second inclination angle θ y Substitute the following formula:

[0039] Get the corrected user orientation vector B′;

[0040] Substitute the corrected user orientation vector B′ into the filtering formula to obtain the filtered orientation vector B smooted (t);

[0041] B smooted (t) = α·B′(t) + (1-α)·B smooted (t-1), where α is the smoothing factor and t is the current time step;

[0042] The filtered direction vector B smooted (t) is normalized to obtain the user orientation vector

[0043]

[0044] in, is the filtered heading vector B smooted (t) modulus.

[0045] In some embodiments, the user coordinates and the user orientation vector are input into a pre-built scenic spot map model to match the scene in the user's current field of view, including:

[0046] Based on the user's heading vector and user coordinates Pos u (x u ,y u ,z u) is defined as the user's field of view, and is projected onto the scenic spot map model to calculate the three-dimensional spatial projection area of ​​the user's field of view; the projection calculation formula is:

[0047] ProjectedView=(x u +d x ·l,y u +d y ·l,z u +d z l)

[0048] Among them, ProjectedView is the three-dimensional space projection area of ​​the user's field of view, d x , d y , d z The user's heading vector The coordinate component of , l is the distance from the user coordinate to the boundary of the user's field of view;

[0049] In the scenic spot map model, match the objects in the field of view that are consistent with the field of view of the three-dimensional space projection area.

[0050] In some of the embodiments, in the scenic spot map model, matching the scene in the field of view that is consistent with the field of view of the three-dimensional space projection area includes:

[0051] In the scenic spot map model, the coordinates are (x i ,y i , z i ) i , use the matching condition;

[0052] The matching conditions are:

[0053] x i ∈[x u ,x u +d x ·l]and y i ∈[y u ,y u +d y ·l]and z i ∈[z u ,z u +d z l],

[0054] According to the matching conditions, judge the scene E i Whether it falls within the three-dimensional space projection area of ​​the user's field of view;

[0055] The scene E that meets the matching conditions i It is determined to be a scene within the user's current field of view.

[0056] In some of the embodiments, constructing a field of view description feature vector of the scene in the field of view includes:

[0057] Get the scene coordinates Pos of the scene in the field of view s =(x s ,y s ,z s );

[0058] Calculate the vector difference between the scene coordinates and the user coordinates to obtain the scene vector in the field of view The calculation formula is:

[0059]

[0060] Direct the user towards the vector and the vector of the objects in the field of view Substitute into the deviation calculation formula to obtain the angle deviation AD;

[0061]

[0062] in, is the vector dot product, and is the vector modulus;

[0063] S4-4, obtaining a second distance Di between the scene coordinates and the user coordinates,

[0064]

[0065] Select the scene coordinates Pos s =(x s ,y s ,z s ), user orientation vector and the vector of the objects in the field of view The angle deviation AD, the second distance Di and the preset identifier ID of the scene are used as a feature vector to construct a field of view description feature vector FDF of the scene in the field of view;

[0066] FDF={Pos s ,AD,Di,ID}.

[0067] In some embodiments, pre-training a tourism question-answering model includes:

[0068] Collecting background knowledge texts of the scenic spots as a training set;

[0069] The background knowledge text is input into a large language model and iteratively trained into a tourism question-answering model.

[0070] Compared with the prior art, a method for user voice interaction at a scenic spot of the present invention obtains the user's position and orientation through a Bluetooth headset, and then obtains the user's field of vision description feature vector, and inputs the field of vision description feature vector and the user's question into a pre-trained tourism question-and-answer model, so as to obtain personalized and real-time voice broadcast of the current scenic spot, thereby improving the tourism experience.

[0071] In a second aspect, the present invention provides a scenic spot user voice interaction device, comprising:

[0072] A coordinate and orientation acquisition module, used to acquire the current user coordinates and user orientation vector in the scenic spot;

[0073] A module for acquiring objects in the field of view, used for inputting the user coordinates and the user orientation vector into a pre-built scenic spot map model, and matching and obtaining the objects in the user's current field of view;

[0074] A field of view vector construction module, used to construct a field of view description feature vector of the scene within the field of view;

[0075] Pre-training module, used to pre-train a tourism question-answering model;

[0076] The personalized answer module is used to receive the visual field description feature vector and the user's question as input of the tourism question-answering model and generate a personalized answer.

[0077] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores at least one computer executable instruction, and the processor is configured to run the computer executable instruction, and when the computer executable instruction is run by the processor, a method for voice interaction between a user at a scenic spot described in the first aspect is implemented.

[0078] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for voice interaction between scenic spot users described in the first aspect is implemented.

[0079] Compared with the prior art, the beneficial effects of the scenic spot user voice interaction device, electronic device and storage medium of the present invention are the same as the beneficial effects of the above-mentioned scenic spot user voice interaction method, so they are not repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A schematic diagram of the steps of a method for voice interaction with a scenic spot user in an embodiment of the present invention;

[0081] Figure 2 A structural block diagram of a scenic spot user voice interaction device in an embodiment provided by the present invention;

[0082] Figure 3 A diagram of an electronic device in an embodiment of the present invention; DETAILED DESCRIPTION

[0083] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0084] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0085] refer to Figures 1 to 3 In an embodiment of the present invention, a method for scenic spot user voice interaction is provided. Figure 1 It is a flow chart of a method for scenic spot user voice interaction of the present invention, and the process includes: Step S1, establishing a communication connection between the user's Bluetooth headset and a mobile terminal to form a wireless network; wherein the Bluetooth headset has a built-in electronic compass that can obtain the user's orientation vector and an accelerometer that can correct the tilt angle of the Bluetooth headset.

[0086] Step S1, establishing a communication connection between the user's Bluetooth headset and a mobile terminal to form a wireless network; wherein the Bluetooth headset has a built-in electronic compass that can obtain the user's orientation vector and an accelerometer that can correct the tilt angle of the Bluetooth headset.

[0087] That is to say, first, the Bluetooth headset worn by the user is connected to the mobile phone (mobile terminal) to form a wireless network. This Bluetooth headset is equipped with an electronic compass that can detect the user's current direction, and is also equipped with an accelerometer to correct the tilt angle of the headset. That is, the Bluetooth headset is connected to the mobile phone, and the sensor inside the headset is used to obtain the user's direction and the angle of the headset.

[0088] The process also includes:

[0089] Step S2: obtaining the current user coordinates and user orientation vector in the scenic spot,

[0090] The user coordinates may be acquired by using GPS positioning, Bluetooth beacon, WI-FI access point, RFID tag or other positioning methods.

[0091] Exemplarily, this embodiment uses RFID tags to locate the coordinate position of the Bluetooth headset. Based on this, the step S2 also includes:

[0092] S2-1, installing a plurality of RFID readers and writers located in the scenic spot, and an RFID tag located on the Bluetooth headset; wherein the plurality of RFID readers and writers cover the scenic spot area that the user can reach.

[0093] S2-2, if the RFID tag enters the scenic spot, the RFID reader receives the tag signal sent by the RFID tag.

[0094] S2-3, determining the current user coordinates according to the signal strength of the received tag signal and the time when the tag signal reaches the RFID reader.

[0095] The step S2-3 further includes:

[0096] S2-3-1, calculating a first distance d from the RFID reader to the RFID tag according to the signal strength of the received tag signal RSSI,i ;

[0097]

[0098] Among them, d RSSI,i is the first distance between the i-th RFID reader and the RFID tag, P r,i is the signal strength received by the i-th RFID reader, P t is the transmission power of the RFID tag, n is the path loss exponent, which is between 2 and 4, and C is the correction term for environmental noise and interference.

[0099] S2-3-2, based on the time when the tag signal arrives at the RFID reader, calculate the time difference between the tag signal arriving at the two RFID readers; and based on the time difference, calculate the distance difference Δd from the RFID tag to the two RFID readers ij ;

[0100] Δd ij =c·Δt ij

[0101] Where, Δd ij is the distance difference between the RFID tag and the i-th RFID reader and the j-th RFID reader, c is the signal propagation speed, Δt ij is the time difference between the tag signal reaching the i-th RFID reader and the j-th RFID reader.

[0102] S2-3-3, according to the first distance d RSSI,i And the distance difference Δd ij , determine the weighted distance d combined,i ;

[0103] d combined,i =α·d RSSI,i +β·Δd ij

[0104] Among them, d combined,i is the weighted distance from the i-th RFID reader to the RFID tag, and α is the first distance d RSSI,i The weight of β is the distance difference Δd ij The weight of .

[0105] S2-3-4, the weighted distance d combined,i and the location of the RFID reader (x i ,y i ,z i ), substitute into the position equation;

[0106]

[0107] Solve the position equation to obtain the current user coordinate Pos u (x u ,y u ,z u ).

[0108] In the above implementation, multiple RFID readers are used to receive tag signals sent by RFID tags on Bluetooth headsets, and then the specific location coordinates of the user in the scenic spot are calculated using signal strength and time difference.

[0109] Furthermore, the user orientation vector may be obtained by the following steps:

[0110] S2-4-1, using an electronic compass to measure the vector component of the magnetic field vector on the Bluetooth headset, and using the vector component as the original user orientation vector B of the Bluetooth headset, B = (B x ,B y ,B z );

[0111] Among them, B x ,B y ,B z They are respectively the X-axis vector component, Y-axis vector component and Z-axis vector component of the Bluetooth headset.

[0112] S2-4-2, using an accelerometer to detect a first tilt angle θ of the Bluetooth headset x and the second inclination angle θ y , the first tilt angle θ x and the second inclination angle θ y Substitute the following formula:

[0113] The corrected user orientation vector B′ is obtained.

[0114] S2-4-3, substituting the corrected user orientation vector B′ into the filtering formula to obtain the filtered orientation vector B smooted (t);

[0115] B smooted (t) = α·B′(t) + (1-α)·B smooted (t-1), where α is the smoothing factor and t is the current time step.

[0116] S2-4-4, the filtered direction vector B smooted (t) is normalized to obtain the user orientation vector

[0117]

[0118] in, is the filtered heading vector B smooted (t) modulus.

[0119] The above implementation method finally obtains an accurate and stable user orientation vector through a series of measurement, correction, smoothing and standardization processes, thereby ensuring that the Bluetooth headset can accurately reflect the actual orientation of the user.

[0120] The process also includes:

[0121] Step S3: inputting the user coordinates and the user orientation vector into a pre-built scenic spot map model to match and obtain the scenery in the user's current field of view. Step S3 specifically includes:

[0122] S3-1, based on the user's orientation vector and user coordinates Pos u (x u ,y u ,z u ) is defined as the user's field of view, and is projected onto the scenic spot map model to calculate the three-dimensional spatial projection area of ​​the user's field of view; the projection calculation formula is:

[0123] ProjectedView=(x u +d x ·l,y u +d y ·l,z u +d z l)

[0124] Among them, ProjectedView is the three-dimensional space projection area of ​​the user's field of view, d x , d y , d z The user's heading vector The coordinate component of , l is the distance from the user coordinate to the boundary of the user's field of view.

[0125] S3-2, in the scenic spot map model, matching the sceneries in the field of view that are consistent with the field of view of the three-dimensional space projection area.

[0126] Furthermore, step S3-2 further includes:

[0127] S3-2-1, in the scenic spot map model, and the coordinates are (x i ,y i , z i ) i , using the matching condition,

[0128] The matching conditions are:

[0129] x i ∈[x u ,x u +d x ·l]and y i ∈[y u ,y u +d y ·l]and z i ∈[z u ,z u +d z ·l].

[0130] S3-2-2, judging scene E according to matching conditions i Whether it falls into the three-dimensional space projection area of ​​the user's field of view.

[0131] S3-2-3, the scene E that meets the matching conditions i It is determined to be a scene within the user's current field of view.

[0132] Finally, the orientation vector and coordinate position of the user (Bluetooth headset) are used and then substituted into the three-dimensional map model of the scenic spot, so that the scenery that the user can currently see can be identified. In this embodiment, the scenery can be a scenic spot area or a scenic spot object.

[0133] The process also includes:

[0134] Step S4, constructing a visual field description feature vector of the scene in the visual field. Step S4 specifically comprises:

[0135] S4-1, obtaining the scene coordinates Pos of the scene in the field of view s =(x s ,y s ,z s ).

[0136] S4-2, calculate the vector difference between the scene coordinates and the user coordinates to obtain the scene vector in the field of view The calculation formula is:

[0137]

[0138] S4-3, orient the user towards the vector and the vector of the objects in the field of view Substitute into the deviation calculation formula to obtain the angle deviation AD;

[0139]

[0140] in, is the vector dot product, and is the vector modulus.

[0141] S4-4, obtaining a second distance Di between the scene coordinates and the user coordinates,

[0142]

[0143] S4-5, select the scene coordinates Pos s =(x s ,y s ,z s ), user orientation vector The angle deviation AD of the scene vector →v in the field of view, the second distance Di and the preset identifier ID of the scene are used as feature vectors to construct a field of view description feature vector FDF of the scene in the field of view;

[0144] FDF={Pos s ,AD,Di,ID}.

[0145] That is, the scene coordinates, the user orientation vector, the angle deviation of the scene vector, the distance between the scene and the user, and the identifier (scene name or number) preset by the scenic spot staff for the scene are combined to form a vector with multiple features, namely the field of view description feature vector. Because the field of view description feature vector contains multiple features, it implies the spatial position and relationship of the scene in the user's field of view, and therefore can be used as a feature variable of the pre-trained model.

[0146] The process also includes:

[0147] Step S5: pre-training a tourism question-answering model, specifically:

[0148] S5-1, collecting background knowledge texts of the scenic spots as a training set.

[0149] S5-2, input the background knowledge text into a large language model and iteratively train it into a tourism question-answering model.

[0150] For step S5, which is the earliest working step, the staff collects detailed background knowledge of each area of ​​the scenic spot in advance and constructs a training set for the large language model, or it is called a pre-built knowledge base. The staff collects detailed background knowledge texts of each scene. These texts include historical information, cultural background, architectural features, geographical location, etc., forming a rich training set.

[0151] Before inputting the background knowledge text into the model, the text data needs to be preprocessed, including removing noise, processing missing information, performing text segmentation, and standardization.

[0152] The preprocessed background knowledge text is input into the large language model for iterative training. The model will learn how to extract information from the input background knowledge by continuously adjusting internal parameters. In other words, the tourism question-and-answer model is a personalized model for a specific field. As long as a sufficiently rich training set of data is prepared, the tourism question-and-answer model can be trained. Of course, during the training process, supervised learning methods can be used to allow the model to learn how to generate answers that meet the requirements by providing correct question-and-answer pairs. The model will gradually improve the accuracy and relevance of the answers through repeated training.

[0153] The process also includes:

[0154] Step S6: receiving the visual field description feature vector and the user's question as inputs of the tourism question-answering model, and generating a personalized answer.

[0155] The visual field description feature vector tells the tourism question-answering model the user's current spatial position, orientation, relative position of the scenery and the user, etc., so that the tourism question-answering model can understand the user's current situation.

[0156] When the user inputs the user question into the tourism question and answer model through the Bluetooth headset, the tourism question and answer model receives the vision description feature vector and the user question as feature variables. The tourism question and answer model generates a personalized answer related to the user's current situation and question based on the vision description feature vector and the user question.

[0157] Step S7: Transmit the personalized answer to the Bluetooth headset via the wireless network and broadcast the personalized answer.

[0158] In other words, the personalized answers output by the tourism question-and-answer model are broadcast through the voice broadcast program of the Bluetooth headset, so that users can understand the answers that match the current scenic spot context in real time, providing users with instant and targeted tourism information.

[0159] A method for user voice interaction at a scenic spot in this embodiment obtains the user's position and orientation through a Bluetooth headset, and then obtains the user's field of view description feature vector, and inputs the field of view description feature vector and the user's question into a pre-trained tourism question-and-answer model, so as to obtain personalized and real-time voice broadcast of the current scenic spot, thereby enhancing the tourism experience.

[0160] The embodiment of the present invention also provides a scenic spot user voice interaction device, which is used to implement the above method embodiment, and will not be repeated for what has been described. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0161] like Figure 2 As shown, Figure 2 1 is a structural block diagram of a scenic spot user voice interaction device of the present invention, the device comprising:

[0162] The communication connection module 101 is used to establish a communication connection between the user's Bluetooth headset and the mobile terminal to form a wireless network; wherein the Bluetooth headset is equipped with an electronic compass that can obtain the user's orientation vector and an accelerometer that can correct the tilt angle of the Bluetooth headset;

[0163] A coordinate and orientation acquisition module 102 is used to acquire the current user coordinates and user orientation vector in the scenic spot;

[0164] The module 103 for acquiring objects in the field of view is used to input the user coordinates and the user orientation vector into a pre-built scenic spot map model, and match and obtain the objects in the user's current field of view;

[0165] A field of view vector construction module 104, used to construct a field of view description feature vector of the scene within the field of view;

[0166] A pre-training module 105, used for pre-training a tourism question-answering model;

[0167] A personalized answer module 106, for receiving the visual field description feature vector and the user's question as inputs of the tourism question-answering model, and generating a personalized answer;

[0168] The voice broadcast module 107 is used to transmit the personalized answer to the Bluetooth headset through the wireless network and broadcast the personalized answer.

[0169] In the above-mentioned device, a wireless network is established through the communication connection module 101; the user coordinates and the user orientation vector are obtained through the coordinate and orientation acquisition module 102; the scene in the field of view acquisition module 103 is obtained; the field of view description feature vector is constructed through the field of view vector construction module 104, and the tourism question and answer model is obtained through the pre-training module 105; the personalized answer is generated through the personalized answer module 106; and the personalized answer is broadcasted through the voice broadcast module 107, thereby solving the problem that it is difficult for users to obtain accurate, timely and personalized introductions to scenic spots during the travel process.

[0170] The embodiment of the present invention also provides a device related to the voice interaction of scenic spot users, such as Figure 3 As shown, the related device includes an electronic device provided by an embodiment of the present invention, the electronic device includes a memory 230 and a processor 210, the memory 230 stores at least one computer executable instruction, the processor 210 is configured to run the computer executable instruction, and when the computer executable instruction is run by the processor 210, the above-mentioned method for voice interaction between scenic spots and users is implemented.

[0171] The electronic device may include a processor 210, a communications interface 220, a memory 230, and a communications bus 240, wherein the processor 210, the communications interface 220, and the memory 230 communicate with each other via the communications bus 240. The processor 210 may call the logic instructions in the memory 230 to execute a method for scenic spot user voice interaction disclosed in this embodiment.

[0172] In addition, the logic instructions in the above-mentioned memory 230 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0173] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for scenic spot user voice interaction, characterized in that: include: Obtaining the current user coordinates and user orientation vector within the scenic spot; Inputting the user coordinates and the user orientation vector into a pre-built scenic spot map model to match the scene within the user's current field of view; Constructing a visual field description feature vector of the scene within the visual field; Pre-train a tourism question-answering model; The visual field description feature vector and the user's question are received as inputs of the tourism question-answering model to generate a personalized answer.

2. A method for scenic spot user voice interaction according to claim 1, characterized in that: Before obtaining the current user coordinates and user orientation vector in the scenic spot, the method includes: Establishing a communication connection between the user's Bluetooth headset and the mobile terminal to form a wireless network; wherein the Bluetooth headset has a built-in electronic compass that can obtain the user's orientation vector and an accelerometer that can correct the tilt angle of the Bluetooth headset; After receiving the visual field description feature vector and the user's question as inputs of the tourism question-answering model and generating a personalized answer, the method further includes: The personalized answer is transmitted to the Bluetooth headset through the wireless network, and the personalized answer is broadcasted.

3. A method for scenic spot user voice interaction according to claim 2, characterized in that , obtain the current user coordinates and user orientation vector in the scenic spot, including: Installing a plurality of RFID readers and writers located in the scenic spot, and an RFID tag located on the Bluetooth headset; wherein the plurality of RFID readers and writers cover the scenic spot area that the user can reach; If the RFID tag enters the scenic spot, the RFID reader receives the tag signal sent by the RFID tag; The current user coordinates are determined according to the signal strength of the received tag signal and the time when the tag signal reaches the RFID reader.

4. A method for scenic spot user voice interaction according to claim 3, characterized in that , according to the signal strength of the tag signal received and the time when the tag signal reaches the RFID reader, determining the current user coordinates, including: According to the signal strength of the tag signal received, the first distance d from the RFID reader to the RFID tag is calculated. RSSI,i ; Among them, d RSSI,i is the first distance between the i-th RFID reader and the RFID tag, P r,i is the signal strength received by the i-th RFID reader, P t is the transmission power of the RFID tag, n is the path loss index, which is between 2 and 4, and C is the correction term for environmental noise and interference; According to the time when the tag signal arrives at the RFID reader, calculate the time difference between the tag signal arriving at the two RFID readers; and according to the time difference, calculate the distance difference Δd from the RFID tag to the two RFID readers ij ; Δd ij =c·Δt ij Where, Δd ij is the distance difference between the RFID tag and the i-th RFID reader and the j-th RFID reader, c is the signal propagation speed, Δt ij is the time difference between the tag signal reaching the i-th RFID reader and the j-th RFID reader; According to the first distance d RSSI,i And the distance difference Δd ij , determine the weighted distance d combined,i ; d combined,i =α·d RSSI,i +β·Δd ij Among them, d combined,i is the weighted distance from the i-th RFID reader to the RFID tag, and α is the first distance d RSSI,i The weight of β is the distance difference Δd ij The weight of The weighted distance d combined,i and the location of the RFID reader (x i ,y i ,z i ), substitute into the position equation; Solve the position equation to obtain the current user coordinate Pos u (x u ,y u ,z u ).

5. A method for scenic spot user voice interaction according to claim 4, characterized in that , obtaining the current user coordinates and user orientation vector in the scenic spot, and also including: Use an electronic compass to measure the vector component of the magnetic field vector on the Bluetooth headset, and use the vector component as the original user orientation vector B of the Bluetooth headset. B=(B x ,B y ,B z ), Among them, B x ,B y ,B z They are respectively the X-axis vector component, Y-axis vector component and Z-axis vector component of the Bluetooth headset; Use the accelerometer to detect the first tilt angle θ of the Bluetooth headset x and the second inclination angle θ y , the first tilt angle θ x and the second inclination angle θ y Substitute the following formula: Get the corrected user orientation vector B′; Substitute the corrected user orientation vector B′ into the filtering formula to obtain the filtered orientation vector B smooted (t); B smooted B(t) = α·B'(t) + (1 - α)·B smooted (t - 1), where α is the flat Sliding factor, t is the current time step; The filtered direction vector B smooted (t) is normalized to obtain the user orientation vector in, is the filtered heading vector B smooted (t) modulus.

6. A method for scenic spot user voice interaction according to claim 5, characterized in that , inputting the user coordinates and the user orientation vector into a pre-built scenic spot map model, and matching the scene in the user's current field of view, including: Based on the user's heading vector and user coordinates Pos u (x u ,y u ,z u ) is defined as the user's field of view, and is projected onto the scenic spot map model to calculate the three-dimensional spatial projection area of ​​the user's field of view; the projection calculation formula is: ProjectedView=(x u +d x ·l,y u +d y ·l,z u +d z ·l) Among them, ProjectedView is the three-dimensional space projection area of ​​the user's field of view, d x , d y , d z The user's heading vector The coordinate component of , l is the distance from the user coordinate to the boundary of the user's field of view; In the scenic spot map model, match the objects in the field of view that are consistent with the field of view of the three-dimensional space projection area.

7. A method for scenic spot user voice interaction according to claim 6, characterized in that ,In the scenic spot map model, the objects in the field of view that are consistent with the ,field of the three-dimensional space projection area are matched, including: In the scenic spot map model, the coordinates are (x i ,y i , z i ) i , use the matching condition; The matching conditions are: x i ∈[x u ,x u +d x ·l]and y i ∈[y u ,y u +d y ·l]and z i ∈[z u ,z u +d z ·l], According to the matching conditions, judge the scene E i Whether it falls within the three-dimensional space projection area of ​​the user's field of view; Scenery E that meets the matching conditions i It is determined to be a scene within the user's current field of view.

8. A method for scenic spot user voice interaction according to claim 7, characterized in that , constructing a visual field description feature vector of the scene in the visual field, including: Get the scene coordinates Pos of the scene in the field of view s =(x s ,y s ,z s ); Calculate the vector difference between the scene coordinates and the user coordinates to obtain the scene vector in the field of view The calculation formula is: Direct the user towards the vector and the vector of the objects in the field of view Substitute into the deviation calculation formula to obtain the angle deviation AD; in, is the vector dot product, and is the vector modulus; S4-4, obtaining a second distance Di between the scene coordinates and the user coordinates, Select the scene coordinates Pos s =(x s ,y s ,z s ), user orientation vector and the vector of the objects in the field of view The angle deviation AD, the second distance Di and the preset identifier ID of the scene are used as a feature vector to construct a field of view description feature vector FDF of the scene in the field of view; FDF={Pos s ,AAD,Di,ID}。 9. A method for scenic spot user voice interaction according to claim 8, characterized in that ,Pre-training a tourism question answering model, including: Collecting background knowledge texts of the scenic spots as a training set; The background knowledge text is input into a large language model and iteratively trained into a tourism question-answering model.

10. A scenic spot user voice interaction device, characterized in that: include: A coordinate and orientation acquisition module, used to acquire the current user coordinates and user orientation vector in the scenic spot; A module for acquiring objects in the field of view, used for inputting the user coordinates and the user orientation vector into a pre-built scenic spot map model, and matching and obtaining the objects in the user's current field of view; A field of view vector construction module, used to construct a field of view description feature vector of the scene within the field of view; Pre-training module, used to pre-train a tourism question-answering model; The personalized answer module is used to receive the visual field description feature vector and the user's question as input of the tourism question-answering model and generate a personalized answer.

Images