Virtual assistant interaction method and system for hotel guest room
By identifying and processing multimodal interactive information in hotel rooms, virtual assistants can effectively respond to user needs in a single room, solving the problem that virtual assistants find it difficult to understand multiple different modal information, and achieving improvement of user experience and systematic flexibility.
Patent Information
- Application Number
- CN202411480450.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-10-23
AI Technical Summary
The virtual assistant receives multiple interactive information of different modalities in a single hotel room and is not associated, making it difficult to understand and respond to user needs simultaneously.
By identifying the interactive content input by the user, judging its interaction type, and putting information from different modalities into the task queue, using collaborative correlation information to generate modal conflicting content, dynamically adjusting the dependencies between modalities, and generating a response strategy to process Multimodal information.
It improves the accurate understanding of user intentions by virtual assistants, reduces misunderstandings and resource overload, ensures the smoothness and consistency of user experience, and can flexibly handle task priorities and execution order in complex scenarios.
Smart Images

Figure CN119005994A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of interactive data processing, and in particular to a virtual assistant interaction method and system for hotel guest rooms. Background Art
[0002] With the continuous improvement of information technology and people's living standards, guests have higher requirements for hotel stay experience and management level. In this industry context, traditional hotel information service systems have become increasingly unable to meet the requirements of new hotel management. At the same time, it is difficult to solve problems such as multiple business links, low communication efficiency, inability to meet personalized service needs, and poor room service experience. Therefore, existing hotel services have added virtual assistants to interact with residents in hotel rooms to meet the effective needs of residents in their rooms.
[0003] However, if the virtual assistant receives multiple interactive messages of different modalities in a single hotel room, and these interactive messages are unrelated, it will be difficult for the virtual assistant to understand and respond to these interactive messages at the same time, and it will be unable to meet the needs of the room occupants in a timely manner. Summary of the invention
[0004] The present invention aims to solve the problem that a virtual assistant receives multiple interactive messages of different modes in a single hotel room, and these interactive messages are unrelated, making it difficult for the virtual assistant to understand and respond to these interactive messages at the same time, and provides a virtual assistant interaction method and system for hotel rooms.
[0005] The present invention adopts the following technical means to solve the technical problem: The present invention provides a virtual assistant interaction method for a hotel room, comprising: Based on the pre-unlocked interactive rights of the hotel room, identifying the interactive content input by the user into the hotel room within a preset time period; Determining whether the interactive contents belong to the same interaction type, wherein the interaction type specifically includes image interaction, sound interaction, and text interaction; If not, the interactive content is included in the task queue preset by the virtual assistant, the processing module preset by the hotel room is activated, the interactive content is allocated from the task queue to the processing module, the interactive content is responded to in the hotel room by the processing module, the collaborative association information pre-classified by the interactive content is identified, and the modal conflict content of the interactive content is generated according to the collaborative association information, wherein the collaborative association information specifically includes keyword matching, spatial position matching and task target matching; Determining whether the modal conflict content can be dynamically adjusted; If possible, the input data of the modal conflict content is identified, and the interactive information of different modalities is represented as nodes in a preset graph model based on the input data. The edges between the nodes are represented as dependency relationships between different modalities, and a response strategy for the modal conflict content is generated based on the nodes and the dependency relationships, wherein the input data specifically includes visual images, voice commands, and tactile feedback.
[0006] Further, the step of activating the preset processing module of the hotel room, allocating the interactive content from the task queue to the processing module, and responding to the interactive content in the hotel room through the processing module further includes: Based on the cross-room linkage setting preset by the user through the hotel terminal, presenting the control content of the preset linkage guest room in the hotel guest room; Determining whether the control content can be synchronized to the hotel room; If possible, the virtual assistant will issue a corresponding task request based on the interactive content, identify the pre-recorded content of the linked guest room, generate the task request from the linked guest room based on the pre-recorded content, and coordinate the hotel room and the linked guest room to synchronously execute the task request through the virtual assistant.
[0007] Furthermore, before the step of generating the modal conflict content of the interactive content according to the collaborative association information, it also includes: Based on the modality information pre-generated by the hotel room for the interactive content, obtaining a task request converted from the modality information; Determining whether the task request meets the preset execution content of the hotel room; If not, identify the missing modality of the modal information, generate the content to be completed of the modal information according to the missing modality, obtain the user's response feedback to the content to be completed, reconstruct the response feedback into the modal information according to the interaction type, and convert it again to obtain the task request.
[0008] Furthermore, the step of identifying the input data of the modality conflict content further includes: Based on the preset collected features of the hotel room, obtaining the feature type input by the user into the hotel room through the virtual assistant; Determining whether the feature type exceeds a preset upper limit; If so, the virtual assistant identifies the modal conflict point of the feature type, generates the interceptable paragraph information of the modal conflict content according to the modal conflict point, transfers the interceptable paragraph information to a preset blank paragraph, constructs the interactive content to be uploaded corresponding to the interceptable paragraph information, and synchronizes the interactive content to be uploaded to the virtual assistant.
[0009] Furthermore, the step of determining whether the interactive contents belong to the same interaction type further includes: Identify a corresponding interactive device in the hotel room based on the interactive content; Determining whether the interactive device receives at least two pieces of interactive content within a preset time period; If so, the interaction type of the interactive content is obtained, and the interactive content is classified into preset interaction options corresponding to the virtual assistant, wherein the interaction options specifically include query type interaction, request type interaction and setting type interaction.
[0010] Furthermore, the step of determining whether the modal conflict content can be dynamically adjusted further includes: Acquire the conflict type of the modality conflict content, wherein the conflict type specifically includes space conflict, priority conflict and time conflict; Determining whether the conflict types are diverted to the same interaction path; If not, the influencing device corresponding to the modal conflict content is generated through a virtual assistant, the conflict range of the modal conflict content on the influencing device is identified, and the dynamic adjustment interval of the influencing device is divided based on the conflict range.
[0011] Furthermore, the step of identifying the interactive content input by the user into the hotel room within a preset time period based on the interactive authority of the hotel room pre-unlocking further includes: Identify the total number of users occupying said hotel rooms; Determining whether the total number of users reaches a preset threshold; If so, based on the feature information pre-entered by the user, the priority configuration corresponding to the user in the hotel room is obtained, the interactive content received in the hotel room is collected, and the execution sequence of the interactive content is divided according to the priority configuration.
[0012] The present invention also provides a virtual assistant interactive system for hotel rooms, comprising: An identification unit, configured to identify the interactive content input into the hotel room by the user within a preset time period based on the interactive authority of the hotel room pre-unlocking; A judging unit, used to judge whether the interactive contents belong to the same interaction type, wherein the interaction types specifically include image interaction, sound interaction and text interaction; an execution unit, for, if not, listing the interactive content into a task queue preset by the virtual assistant, activating a processing module preset by the hotel room, allocating the interactive content from the task queue to the processing module, responding to the interactive content in the hotel room through the processing module, identifying collaborative association information pre-classified by the interactive content, and generating modal conflict content of the interactive content according to the collaborative association information, wherein the collaborative association information specifically includes keyword matching, spatial position matching, and task target matching; A second judgment unit, used to judge whether the modal conflict content can be dynamically adjusted; The second execution unit is used to identify the input data of the modal conflict content if possible, represent the interactive information of different modalities as nodes in a preset graph model based on the input data, represent the edges between the nodes as dependency relationships between different modalities, and generate a response strategy for the modal conflict content based on the nodes and the dependency relationships, wherein the input data specifically includes visual images, voice commands, and tactile feedback.
[0013] Furthermore, the execution unit further includes: A presentation subunit, configured to present control contents of preset linked guest rooms in hotel guest rooms based on the cross-room linkage settings preset by the user through the hotel terminal; A judging subunit, used for judging whether the control content can be synchronized to the hotel guest room; An execution subunit is used to, if possible, enable the virtual assistant to issue a corresponding task request based on the interactive content, identify the pre-recorded content of the linked guest room, generate the task request from the linked guest room based on the pre-recorded content, and coordinate the hotel room and the linked guest room to synchronously execute the task request through the virtual assistant.
[0014] Furthermore, it also includes: An acquisition unit, configured to acquire, based on the modality information pre-generated by the hotel room for the interactive content, a task request obtained by converting the modality information; A third judgment unit is used to judge whether the task request meets the preset execution content of the hotel room; The third execution unit is used to, if not, identify the missing modality of the modal information, generate the content to be completed of the modal information according to the missing modality, obtain the user's response feedback to the content to be completed, reconstruct the response feedback into the modal information according to the interaction type, and convert it again to obtain the task request.
[0015] The present invention provides a virtual assistant interaction method and system for hotel rooms, which has the following beneficial effects: The present invention recognizes and classifies different types of interactive content to more accurately understand the user's intentions and reduce the possibility of misunderstanding or misoperation. At the same time, different interactive content is included in the task queue and responded to through a preset processing module, so that the system can effectively allocate resources and avoid resource overload or response delay caused by processing multiple tasks at the same time. Moreover, by identifying and processing modal conflict content, the system can dynamically adjust the conflicts between different modalities to ensure that users have a smooth and consistent experience when interacting with the virtual assistant. When the system identifies a modal conflict in the interactive content, it generates a response strategy by analyzing the dependency relationship between the modalities. The system can maintain a high degree of flexibility in complex multi-tasking scenarios, automatically adjust the priority and execution order of tasks, and avoid task conflicts. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 A flowchart of an embodiment of a virtual assistant interaction method for a hotel room of the present invention; Figure 2 The structure diagram of an embodiment of the virtual assistant interactive system for hotel rooms of the present invention. DETAILED DESCRIPTION
[0017] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The implementation of the objectives, functional features and advantages of the present invention will be further described in conjunction with the embodiments and with reference to the accompanying drawings.
[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] Reference Figure 1 , a virtual assistant interaction method for a hotel room in one embodiment of the present invention, comprising: S1: Based on the interaction rights of the hotel room pre-unlocked, identifying the interaction content input by the user into the hotel room within a preset time period; S2: Determine whether the interactive contents belong to the same interaction type, wherein the interaction type specifically includes image interaction, sound interaction and text interaction; S3: If not, the interactive content is included in the task queue preset by the virtual assistant, the processing module preset by the hotel room is activated, the interactive content is allocated from the task queue to the processing module, the interactive content is responded to in the hotel room by the processing module, the collaborative association information pre-classified by the interactive content is identified, and the modal conflict content of the interactive content is generated according to the collaborative association information, wherein the collaborative association information specifically includes keyword matching, spatial position matching and task target matching; S4: Determine whether the modal conflict content can be dynamically adjusted; S5: If possible, identify the input data of the modal conflict content, represent the interactive information of different modalities as nodes in a preset graph model based on the input data, represent the edges between the nodes as dependency relationships between different modalities, and generate a response strategy for the modal conflict content based on the nodes and the dependency relationships, wherein the input data specifically includes visual images, voice commands, and tactile feedback.
[0020] In this embodiment, after the system confirms that the user has checked into the hotel room, the hotel room will unlock the pre-set interactive permissions, identify the interactive content input by the user into the hotel room within a pre-set time period, and then the system determines whether these interactive contents belong to the same interactive type, which specifically includes image interaction, sound interaction and text interaction, to execute the corresponding steps; for example, when the system determines that the interactive content input by the user into the hotel room belongs to the same interactive type, the system will consider that all inputs are concentrated in the same type, and the system does not need to perform cross-modal analysis or processing. The system will merge the content of the same interactive type into an overall task and process it centrally. If the interactive content has a time sequence, the system can execute these contents in the order of user input, and optimize the feedback method according to the unified interaction type. For example, when the user controls the TV through different methods (such as voice and text), the system will give all feedback in a unified manner through voice or screen display to simplify the user experience, and if the same interaction If the interactive content of the same type contains repeated instructions, the system can identify and reduce redundancy to avoid executing the same operation multiple times. For example, if the user requests to play music twice in a row through voice, the system will only execute it once. For example, when the system determines that the interactive content input by the user into the hotel room does not belong to the same interactive type, the system will believe that there are multiple users in the hotel room who are interacting with the virtual assistant at the same time. The system will include different interactive content in the task queue pre-set by the virtual assistant, activate the processing module pre-set in the hotel room, and allocate these interactive content from the task queue to the processing module. The processing module responds to these interactive content in the hotel room and identifies the collaborative association information that has been pre-classified for these interactive content. The collaborative association information specifically includes keyword matching, spatial position matching and task target matching, and generates modal conflict content of the interactive content according to different collaborative association information. The system can effectively manage the interaction requests of multiple users by including different interactive content in the task queue and allocating them to the processing module. , allowing the system to avoid command confusion and processing conflicts when multiple users initiate interactions at the same time, ensuring that the needs of each user in the hotel room can be handled independently and appropriately. At the same time, by identifying the collaborative correlation information of the interactive content (such as keyword matching, spatial location matching, and task goal matching), the intention and context of different interactive content can be accurately distinguished. This precise identification and classification capability helps the system better understand user needs, thereby providing more relevant and personalized services to different users. When a modal conflict is detected, it can generate conflict content based on the collaborative correlation information and dynamically adjust the processing strategy. This capability allows the system to flexibly handle conflicts when facing multiple interaction types, ensuring interoperability and coordination between different modal inputs, thereby improving the accuracy and consistency of the response; the system then determines whether these modal conflict contents can be dynamically adjusted to execute the corresponding steps;For example, when the system determines that the modal conflict content of the interactive content cannot be dynamically adjusted, the system will consider that there are two or more conflicting interactive contents, and only one of them can be selected for implementation. The system will prioritize the conflicting interactive content, actively feedback the conflict to the user, and request the user to make a selection, including the user's preference settings, the urgency of the interactive content, the time sequence of the user's input, and the importance of the target task of the content. At the same time, the user is prompted with the selected interactive content and the other content that has not been executed is explained through voice prompts, text messages or image displays to ensure that the user understands the system's decisions and processing results, and avoids confusion or dissatisfaction caused by the system's failure to execute a certain content. If the unselected interactive content is equally important to the user, the system will prompt the user after the main task is executed, and suggest whether compensatory execution is needed. For example, when the system determines that the modal conflict content of the interactive content can be dynamically adjusted, the system will consider that the conflicting interactive content can be split and performed simultaneously. The system will identify the input data of the modal conflict content, which specifically includes visual images, voice commands and tactile feedback. The same input data represents the interactive information of different modes as nodes in a pre-set graph model, and the edges between nodes represent the dependencies between different modes. Based on the nodes and dependencies, a response strategy for modal conflict content is generated; the system can flexibly respond to the conflict of multimodal information by splitting the conflicting interactive content for parallel processing, avoiding operation interruption or failure to respond due to modal conflict, dynamically adjusting the processing strategy according to the real-time situation, and better adapting to complex user needs. At the same time, the interactive information of different modes is represented as nodes in the graph model, and processed through the dependencies between nodes, so that the system can allocate and utilize resources more efficiently, reduce the waste of system resources, improve the overall computing efficiency, and be able to process more tasks in the same time period. In addition, the use of the graph model allows the system to be more meticulous when processing complex scenarios. For example, when there are complex dependencies between visual images and voice commands, the system can accurately analyze and process these relationships to ensure that each interactive content can be properly responded to and processed, generate a more reasonable response strategy, and make decisions that meet user expectations in conflict situations, reducing misunderstandings and misoperations. ;
[0021] It should be noted that the input data of the modal conflict content is identified, and the interactive information of different modalities is represented as nodes in a preset graph model according to the input data, and the edges between the nodes are represented as dependency relationships between different modalities. A response strategy for the modal conflict content is generated based on the nodes and the dependency relationships. The specific examples are as follows: Suppose that in a smart room of a high-end hotel, Mr. Zhang, a user, enters the room in the evening and is ready to relax. The room is equipped with a virtual assistant that can interact with the user through multimodal methods such as voice, vision, and touch. At this time, Mr. Zhang wants to make the room more comfortable, so he sends two different interaction requests at the same time: he says "open the curtains so I can see the scenery outside" while pointing to the TV to watch a movie; At this time, the virtual assistant captured Mr. Zhang’s voice command through the microphone array in the room: "Open the curtains and let me see the scenery outside"; at the same time, the camera in the room captured Mr. Zhang’s gesture, and the system recognized that he was pointing his finger in the direction of the TV; at the same time, the system created a "voice node" for Mr. Zhang's voice command, which recorded Mr. Zhang's voice content, voice emotion (because Mr. Zhang's tone was relaxed, indicating that he was in a relaxed state), and related timestamp information; and the system created a "visual node" for Mr. Zhang's gesture, which recorded the direction of the gesture, the direction of the finger, and the object he was pointing to - the TV; the system first recognized the "curtains" and "scenery" mentioned in Mr. Zhang's voice command through keyword analysis and context understanding, which indicated that Mr. Zhang wanted to see the scenery outside and might want the light to change to create a comfortable atmosphere; however, the visual input showed that Mr. Zhang was pointing at the TV, which was not directly related to the content of the voice command (curtains and scenery); therefore , the system creates a dependency edge between the "voice node" and the "visual node", but the weight of the dependency is low because the two nodes point to different targets; the system will analyze that there is a certain conflict because the voice command and the gesture point to different targets; to this end, the system must decide how to coordinate the two requests, and generate a dynamic adjustment strategy to decide whether to respond to the two requests separately: for example, the system gives priority to responding to Mr. Zhang's voice command because the voice expresses a clear demand, so the system sends a command to open the curtains and adjust the lights so that the room light can better present the external scenery. At the same time, the system detects that Mr. Zhang points to the TV, but there is no clear voice command to match it. Therefore, in order to ensure that the user's intention is met, the system will ask Mr. Zhang through the virtual assistant: "Do you want to turn on the TV and play your favorite movie?" After hearing the system's feedback, Mr. Zhang confirmed: "Yes, please play a relaxing movie." The system then executes the command to play the movie, ensuring that all of Mr. Zhang's needs are responded to; The system demonstrated its adaptability to complex user needs by breaking down conflicting modal inputs into independent tasks and processing them separately, so that both of Mr. Zhang's needs were accurately responded to, ensuring the consistency of user experience. At the same time, by asking for confirmation, the system avoided possible misoperations (such as playing TV content directly instead of playing a movie first), which enhanced the user's trust and satisfaction with the system. In addition, the system optimized resource allocation by modeling the dependencies between multi-modal inputs, ensuring that multiple user requests were responded to in the shortest possible time. In summary, this processing flow can simultaneously realize the multi-tasking capabilities of virtual assistants and the intelligent interaction of users, can provide accurate and personalized services in complex scenarios, effectively solve the problem of multimodal information conflict, and significantly improve the quality of user experience when staying in hotel rooms.
[0022] In this embodiment, the processing module preset in the hotel room is activated, the interactive content is allocated to the processing module from the task queue, and the step S3 of responding to the interactive content in the hotel room by the processing module further includes: S31: Based on the cross-room linkage setting preset by the user through the hotel terminal, presenting the control content of the preset linkage guest room in the hotel guest room; S32: Determine whether the control content can be synchronized to the hotel room; S33: If possible, the virtual assistant is enabled to issue a corresponding task request according to the interactive content, identify the pre-recorded content of the linked guest room, generate the task request from the linked guest room according to the pre-recorded content, and coordinate the hotel room and the linked guest room to synchronously execute the task request through the virtual assistant.
[0023] In this embodiment, the system presents the pre-set control contents of the linked guest rooms in the hotel rooms based on the cross-room linkage settings pre-set by the user through the hotel terminal, and then the system determines whether these control contents can be synchronized to the hotel rooms to execute the corresponding steps; for example, when the system determines that the control contents of the linked guest rooms cannot be synchronized to the hotel rooms, the system will consider that the functional contents of the hotel rooms cannot be applied to the linked guest rooms, and the system will display the specific reasons, such as "due to incompatible devices, the contents of the linked guest rooms cannot be synchronized to the current room." , let the user know where the problem lies. After notifying the user, the system will try to automatically reconnect or troubleshoot to see if the problem can be solved to avoid problems caused by temporary network problems or device connection problems. It also recommends that the user change the cross-room linkage settings to exclude the control content that cannot be synchronized in the current guest room, and provide feasible synchronization options based on the actual situation of the current guest room, such as replacing other hotel rooms and linkage rooms that can achieve cross-room linkage; for example, when the system determines that the control content of the linkage guest room can be synchronized to the hotel room, the system will then believe that the functional content of the hotel room can be applied to the linkage guest room for synchronous execution. The system will enable the virtual assistant to issue a corresponding task request based on the interactive content, identify the executable content pre-included in the linkage guest room, generate the same task request from the linkage guest room based on different executable content, and coordinate the hotel room and the linkage guest room to synchronously execute the task request through the virtual assistant; the system Identifying the executable content of the linked guest rooms and automatically generating the same task requests enables the virtual assistant to intelligently coordinate the devices in multiple rooms and automatically execute the user's linkage settings. This not only reduces the user's operating burden, but also demonstrates the efficiency and intelligence of the system, meeting the user's need to use multiple rooms for simultaneous speeches or parties. At the same time, in the multi-room linkage scenario, the system uses a unified task request to keep the operations of each room consistent, avoiding differences in user experience caused by inconsistent settings in different rooms. For example, when users play background music or adjust lighting in multiple rooms simultaneously, the system can ensure that the effects in each room are consistent, improving the user's overall experience, and by synchronously executing task requests, it can quickly respond to user needs in multiple rooms without having to adjust settings room by room. This is especially suitable for scenarios such as business meetings and family gatherings, where users can quickly set up the environment of multiple rooms to meet different needs.
[0024] In this embodiment, before step S3 of generating the modal conflict content of the interactive content according to the collaborative association information, the method further includes: S301: Based on the modal information pre-generated by the hotel room for the interactive content, obtaining a task request converted from the modal information; S302: Determine whether the task request meets the preset execution content of the hotel room; S303: If not, identify the missing modality of the modal information, generate the content to be completed of the modal information according to the missing modality, obtain the user's response feedback to the content to be completed, reconstruct the response feedback into the modal information according to the interaction type, and convert it again to obtain the task request.
[0025] In this embodiment, the system obtains the task requests converted from the modal information based on the modal information pre-generated by the hotel room for the interactive content, and then the system determines whether the task requests meet the pre-set execution content of the hotel room to execute the corresponding steps; for example, when the system determines that the task request obtained by the modal information conversion can meet the pre-set execution content of the hotel room, the system will consider that the modal information processing and task request generation process is effective, and the virtual assistant can correctly understand the user's intention. The system will directly execute these qualified task requests. For example, if the user requires to adjust the room temperature through voice commands, the system will immediately adjust the air conditioning settings to achieve the temperature required by the user, and provide feedback to confirm that the task request has been successfully executed, including through voice prompts, display screen messages or mobile application notifications, such as "The room temperature has been adjusted to 22°C.", and record the results of the task execution for subsequent analysis and optimization. These data can also be used to personalize the user experience, such as automatically adjusting the room settings according to the user's preferences; for example, when the system determines that the task request obtained by the modal information conversion cannot meet the pre-set execution content of the hotel room, the system will consider that the virtual assistant cannot understand the user's intention. Figure 1 shows the system's missing modality, generates the content to be completed of the modal information based on the missing modality, obtains the user's response feedback to the content to be completed, reconstructs the response feedback into the modal information according to the interaction type, and converts it again to obtain the task request. When the system cannot directly understand the user's intention, the robustness of the system can be effectively improved by identifying and completing the missing modal information. Even if the task request converted for the first time does not meet the preset content, the system can adaptively adjust by completing the information to enhance the processing ability of complex interactive scenarios. At the same time, by generating the content to be completed and obtaining the user's feedback, the user can more clearly understand the system's needs and respond. This two-way interaction helps to eliminate misunderstandings and ensure that the final task request is more in line with the user's true intention, thereby improving the user's interaction satisfaction. It also automatically identifies the missing modal information and makes suggestions for completion, reducing the need for users to actively correct or repeat input, making user operations easier and allowing users to interact with the virtual assistant more easily. Through multiple conversions and feedback iterations, the system can more accurately generate task requests that meet user needs. This repeated correction process ensures the accuracy of the final task request and improves the success rate and efficiency of task execution.
[0026] It should be noted that the missing modality of the modality information is identified, the content to be completed of the modality information is generated according to the missing modality, the response feedback of the user to the content to be completed is obtained, the response feedback is reconstructed into the modality information according to the interaction type, and the task request is converted again. The specific examples are as follows: Assume that user Xiao Li is staying in a high-end hotel. He uses a virtual assistant to control various facilities in the room. Since the hotel room is fully functional, Xiao Li wants to adjust the room lighting, temperature and other settings through voice commands. The hotel's virtual assistant can accept multi-modal commands such as voice, touch screen, gestures, and mobile APP to perform tasks; As Xiao Li just entered the hotel room, he found the lights in the room a bit dazzling and wanted to dim the lights. He directly said to the virtual assistant: "Dim the lights a little." However, since there are multiple lighting areas in the room (bedroom, living room, desk, bathroom, etc.), the system cannot determine which area Xiao Li wants to dim based on voice commands alone; First, the system will receive Xiao Li’s voice command: “Dim the lights a little bit.” However, because the voice command does not clearly point out the specific lighting area, the system will recognize this and determine that the voice command lacks the key information of “lighting area”. Although the system understands the need to “dim the lights” through voice recognition technology, because there are multiple lighting areas in the room, the system realizes that it needs to supplement the information of “which area to dim the lights”. Therefore, in order to ensure the accuracy of the operation, the system actively feedbacks information to Xiao Li and prompts him through the voice of the virtual assistant: “Which area do you want to dim the lights in? Is it the bedroom, living room, or desk?” The system analyzes the room layout and the user’s previous preference records, lists several possibilities, and gives the user a voice prompt. The system prompts the user with a voice prompt, and this feedback can also be displayed synchronously through a prompt message on the display screen or a push message on the mobile phone APP, which is convenient for the user to choose; after hearing the system's inquiry, Xiao Li replied by voice: "Dim the lights in the bedroom." At this time, the user supplemented the "area" information missing in the original instruction through voice, and the system received this supplementary information and began to integrate it into the previous instruction; at the same time, if Xiao Li has multiple input methods, such as clicking the "bedroom" icon through the mobile phone APP, this information can also be supplemented; the system integrates the "bedroom" information supplemented by Xiao Li into the original instruction to form a complete modal information: "Dim the lights in the bedroom." , the system updates the task request according to the completed modal information and prioritizes it in the task queue; therefore, the system will associate the task request of "dim the bedroom lights" with the original voice command of "dim the lights" and map this association to a graph model, in which "dim" is an action node and "bedroom lights" is an object node. The system will generate a specific control signal based on the relationship between the two; finally, the system converts the completed modal information into a precise task request: "Dim the bedroom lights to 50% brightness.", the system issues a command to dim the bedroom lights and complete the task; and when executing the task, the system will also consider the user's historical preferences. For example, if the user often dims the bedroom lights to 30% before, the system will prompt the user or automatically apply this preference to further improve the user experience; To sum up, the system successfully understood Xiao Li’s true intention by asking to complete the missing information, thus avoiding erroneous operations caused by incomplete information. Even if there is missing information or unclear instructions in multimodal interaction, the system can still respond flexibly and ultimately perform the task accurately. Through this active interaction method, users do not need to repeat instructions or perform complex operations, which greatly improves user experience and satisfaction. This experience is particularly important in the smart hotel environment. In addition, the system can prioritize tasks and allocate resources based on the completed information, which improves the system's operating efficiency and avoids unnecessary waste of resources.
[0027] In this embodiment, the step S5 of identifying the input data of the modality conflict content further includes: S51: Based on the preset collection features of the hotel room, obtaining the feature type input by the user into the hotel room through the virtual assistant; S52: Determine whether the feature type exceeds a preset upper limit; S53: If so, the modal conflict point of the feature type is identified by the virtual assistant, and the interceptable paragraph information of the modal conflict content is generated according to the modal conflict point, the interceptable paragraph information is transferred to a preset blank paragraph, and the interactive content to be uploaded corresponding to the interceptable paragraph information is constructed, and the interactive content to be uploaded is synchronized to the virtual assistant.
[0028] In this embodiment, the system obtains the types of features input by different users in the hotel room through the virtual assistant based on the pre-set collection features of the hotel room, and then the system determines whether these types of features exceed the pre-set upper limit to execute the corresponding steps; for example, when the system determines that the types of features input by different users through the virtual assistant do not exceed the pre-set upper limit, the system will consider that the user's input behavior is within the range allowed by the system, the number and types of feature types are at a normal level, and the system can process these inputs normally. The system will classify the collected user input features according to the pre-set classification rules and assign them to the corresponding processing modules for processing. At the same time, through multimodal fusion technology, different types of input features are integrated to generate a comprehensive user request description, such as combining voice commands with operations on the touch screen to form a complete task instruction, and after completing the task, the execution result is fed back to the user to confirm whether the task is completed as expected by the user, and the user is replied to by the virtual assistant, providing relevant feedback options, and the user can confirm or make further adjustments; for example, when the system determines that the types of features input by different users through the virtual assistant exceed the pre-set upper limit, the system will consider that the virtual assistant If the system cannot complete the interaction corresponding to these interactive contents, the system will identify the modal conflict points of these feature types through the virtual assistant, generate the interceptable paragraph information of the modal conflict content according to the modal conflict points, transfer the interceptable paragraph information to the pre-set blank paragraph, construct the interactive content to be uploaded corresponding to the interceptable paragraph information, and synchronize the interactive content to be uploaded to the virtual assistant; by identifying the modal conflict points and generating the interceptable paragraph information of the modal conflict content, the system can clarify and decompose complex user requests, help the system clearly understand the user's interactive content, avoid the difficulty in understanding caused by excessive feature types, and make each Paragraphs focus on specific interaction points or request parts, thereby reducing confusion and misunderstanding. At the same time, the interceptable paragraph information of modal conflict content is synchronized to the virtual assistant, enabling the system to gradually process complex interaction requests instead of processing all information at once, thereby improving processing efficiency and accuracy, and clearly presenting segmented request information to users, allowing users to complete the interaction step by step, avoiding confusion caused by too much information entered at one time. Even if the virtual assistant cannot complete the interaction content in one go, the interactive content to be uploaded can be re-uploaded to the virtual assistant through the interceptable paragraph information, thereby realizing secondary interaction between the user and the virtual assistant.
[0029] In this embodiment, the step S2 of determining whether the interactive contents belong to the same interaction type further includes: S21: Identifying a corresponding interactive device in the hotel room based on the interactive content; S22: Determine whether the interactive device receives at least two pieces of interactive content within a preset period of time; S23: If yes, then obtain the interaction type of the interactive content, and classify the interactive content into preset interaction options corresponding to the virtual assistant, wherein the interaction options specifically include query type interaction, request type interaction, and setting type interaction.
[0030] In this embodiment, the system identifies the interactive devices set in the hotel guest room based on the interactive content, and then determines whether these interactive devices have received at least two different interactive contents within a preset time period to execute the corresponding steps; for example, when the system determines that the interactive device has not received two or more interactive contents at the same time, the system will consider that the user's current interactive demand is low, and the device does not need to perform complex multi-tasking. The system will directly execute the tasks related to the interactive content without complex task allocation or conflict management. For example, if the user requires the air-conditioning temperature to be adjusted through voice commands, the system will directly adjust the temperature setting without considering the interference of other commands. While executing the task, the system will continue to monitor whether the device receives new interactive content within the set time period. If new interactive content arrives later, the system will re-judge and perform corresponding processing. Since the interactive demand is relatively small, the system can save resources and reduce unnecessary computing and processing power consumption, thereby improving the overall system operation efficiency; for example, when the system determines that the interactive device receives two or more interactive contents at the same time, the system will consider that the interactive device may need to perform multi-tasking. The system will obtain the interaction type of these interactive contents and will not The system can quickly identify and classify multiple interactive contents to avoid processing delays caused by content confusion. By classifying interactive contents into different interactive options, the system can more effectively allocate resources, handle various tasks, and improve overall processing efficiency. At the same time, through reasonable classification of interaction types, the system can provide users with more accurate and personalized responses. For example, query interactions are prioritized to allow users to get the required information faster, while request interactions can be processed in parallel in the background to increase execution speed. This experience makes users feel that the system is intelligent and responsive. When classifying interactive contents, possible conflicts between different tasks can be identified. For example, if a temperature setting request and a light adjustment request are received at the same time, the system can prioritize them according to the nature of the tasks, or execute non-conflicting tasks at the same time to ensure that each task can be completed in order and avoid task failures caused by conflicts. Because different types of interactions usually have different requirements for system resources, through classification processing, the system can better allocate computing resources and network bandwidth.
[0031] In this embodiment, the step S4 of determining whether the modal conflict content can be dynamically adjusted further includes: S41: Obtaining the conflict type of the modality conflict content, wherein the conflict type specifically includes space conflict, priority conflict and time conflict; S42: Determine whether the conflict types are diverted to the same interaction path; S43: If not, generating the influencing device corresponding to the modal conflict content through a virtual assistant, identifying the conflict range of the modal conflict content on the influencing device, and dividing the dynamic adjustment interval of the influencing device based on the conflict range.
[0032] In this embodiment, the system obtains the conflict type of the modal conflict content, which specifically includes spatial conflict, priority conflict and time conflict, and then determines whether these conflict types are diverted to the same interaction path to execute the corresponding steps; for example, when the system determines that the conflict type of the modal conflict content is diverted to the same interaction path, the system will believe that the interactive content of multiple conflict types ultimately points to the same system resource or device, which will cause resource competition or confusion in the order of task execution. The system will immediately lock the resource and perform conflict detection. For tasks that need to access the same resource at the same time, the system will determine which task to execute first through the task scheduling mechanism, and other tasks will queue up and wait for resource release. At the same time, the priority of the tasks will be adjusted according to preset rules. For example, some tasks may be given a higher priority due to special needs of the user. At this time, the system should give priority to the task and postpone or suspend the task with a lower priority. If the priorities between the tasks are the same, the system can refer to historical records or user habits to determine the execution order, and if the conflict cannot be resolved automatically, a notification can be sent to the user to explain the current conflict situation and solicit the user's opinion. For example, the system can ask the user whether he is willing to execute one task first and then process another task. This approach can not only resolve conflicts, but also enhance the user's sense of control; for example, when the system determines that the conflict type of modal conflict content will not be diverted to the same interaction path, the system will consider that the conflict type of interactive content can be executed separately, and the system will generate the affected devices corresponding to the modal conflict content through the virtual assistant, identify the conflict range of the modal conflict content on the affected devices, and divide the dynamic adjustment interval of the affected devices based on the conflict range; because the conflict type of interactive content is executed separately, the system can handle multiple tasks at the same time without causing task delays or interruptions due to resource conflicts. This strategy enables the virtual assistant to respond to multiple user needs more quickly and enhance the consistency and real-time nature of the user experience. At the same time, by identifying the conflict range of the modal conflict content on the affected devices, the system can accurately divide the dynamic adjustment interval of the device, which can avoid unnecessary device adjustments or interference, and ensure that the device is adjusted only within the required interval, thereby maintaining the stability and efficient operation of the device, and the conflict content is reasonably diverted and processed, avoiding multiple tasks from competing for the same device or resources, reducing the interference or confusion that users may encounter when performing multiple tasks, allowing users to operate more smoothly without being confused or inconvenienced by improper system processing.
[0033] It should be noted that the affected device corresponding to the modal conflict content is generated by a virtual assistant, the conflict range of the modal conflict content on the affected device is identified, and the dynamic adjustment interval of the affected device is divided based on the conflict range. The specific examples are as follows: Assume that there is a smart air conditioner and smart lamp in a hotel room. User A uses voice commands to adjust the room temperature to 22°C. At the same time, user B uses manual operations to dim the lights to 30% brightness. The affected devices are the air conditioner and lamp identified by the system. Since the air conditioner and lamp are affected by different commands, but these commands do not conflict directly, the system finds that the conflict range of these commands is not on the same device, and can divide the dynamic adjustment range. For the air conditioner system, it will make fine adjustments around 22°C to ensure that the temperature change is within the acceptable range for users and take into account energy saving needs. For the lamps, the system will allow fine adjustments based on 30% brightness to ensure that the light is not glaring and meets reading needs. To sum up, in this way, the system ensures the balanced execution of multi-user commands, avoids direct conflicts, and improves the user experience. At the same time, the dynamic adjustment interval setting enables the system to find the best balance between different needs, which not only responds to user needs but also maintains the efficient operation of the equipment. This detailed processing method ensures that in a complex environment, multiple interactive needs can be properly handled, reducing conflicts and optimizing the user experience.
[0034] In this embodiment, based on the interaction authority of the hotel room pre-unlocking, the step S1 of identifying the interaction content input by the user into the hotel room within a preset time period further includes: S11: Identify the total number of users staying in the hotel rooms; S12: Determine whether the total number of users reaches a preset threshold; S13: If yes, based on the feature information pre-entered by the user, obtain the priority configuration corresponding to other users in the hotel room by the user, collect the interactive content received in the hotel room, and divide the execution sequence of the interactive content according to the priority configuration.
[0035] In this embodiment, the system identifies the total number of users staying in hotel rooms, and then determines whether the total number of users reaches a preset threshold to execute corresponding steps; for example, when the system determines that the total number of users staying in hotel rooms does not reach the preset threshold, the system will consider that the number of users in the room is still within the range allowed by the system, and the virtual assistant does not need to perform additional multi-user management or complex interaction feature collection. The system will continue to manage in normal single-user or minority user mode. The virtual assistant will provide personalized services for the current users, such as adjusting the room temperature, playing music, or providing other personalized intelligent services. At the same time, the system will assume that all interactions and requests come from the existing minority of users. The virtual assistant can meet the needs of each user more accurately without complex task allocation or conflict management, and simplifies the user's interaction path, reducing the complex operations that may be caused by multi-user situations. Therefore, the virtual assistant The virtual assistant can quickly respond to and process each user's request without having to perform complex modal conflict processing or task queue management; for example, when the system determines that the total number of users staying in a hotel room has reached a preset threshold, the system will consider that the hotel room's virtual assistant needs to collect different interaction features of multiple users during their stay. The system will identify and confirm the feature information of all staying users through face recognition, voice recognition or other identity authentication methods to ensure that the system can distinguish between different users. At the same time, it will begin to collect the interaction features of each user, including the user's voice command preferences, temperature setting habits, and lighting brightness preferences. According to the interaction features of each user, the system will adjust its response method and optimize its response strategy to adapt to the presence of multiple users. For example, when multiple users issue commands at the same time, the system will decide which user to respond to first, or how to process these commands in parallel, based on the preset priority or frequency of use.
[0036] Reference Figure 2 , is a virtual assistant interactive system for hotel rooms in one embodiment of the present invention, comprising: The identification unit 10 is used to identify the interactive content input into the hotel room by the user within a preset time period based on the interactive authority of the hotel room pre-unlocking; A determination unit 20, configured to determine whether the interactive contents belong to the same interaction type, wherein the interaction type specifically includes image interaction, sound interaction, and text interaction; The execution unit 30 is used for, if not, listing the interactive content into a task queue preset by the virtual assistant, activating a processing module preset by the hotel room, allocating the interactive content from the task queue to the processing module, responding to the interactive content in the hotel room through the processing module, identifying collaborative association information pre-classified by the interactive content, and generating modal conflict content of the interactive content according to the collaborative association information, wherein the collaborative association information specifically includes keyword matching, spatial position matching, and task target matching; A second determination unit 40 is used to determine whether the modality conflict content can be dynamically adjusted; The second execution unit 50 is used to identify the input data of the modal conflict content if possible, represent the interactive information of different modalities as nodes in a preset graph model based on the input data, represent the edges between the nodes as dependency relationships between different modalities, and generate a response strategy for the modal conflict content based on the nodes and the dependency relationships, wherein the input data specifically includes visual images, voice commands and tactile feedback.
[0037] In this embodiment, after the system confirms that the user has checked into the hotel room, the hotel room will unlock the pre-set interactive permissions, identify the interactive content input by the user into the hotel room within a pre-set time period, and then the system determines whether these interactive contents belong to the same interactive type, which specifically includes image interaction, sound interaction and text interaction, to execute the corresponding steps; for example, when the system determines that the interactive content input by the user into the hotel room belongs to the same interactive type, the system will consider that all inputs are concentrated in the same type, and the system does not need to perform cross-modal analysis or processing. The system will merge the content of the same interactive type into an overall task and process it centrally. If the interactive content has a time sequence, the system can execute these contents in the order of user input, and optimize the feedback method according to the unified interaction type. For example, when the user controls the TV through different methods (such as voice and text), the system will give all feedback in a unified manner through voice or screen display to simplify the user experience, and if the same interaction If the interactive content of the same type contains repeated instructions, the system can identify and reduce redundancy to avoid executing the same operation multiple times. For example, if the user requests to play music twice in a row through voice, the system will only execute it once. For example, when the system determines that the interactive content input by the user into the hotel room does not belong to the same interactive type, the system will believe that there are multiple users in the hotel room who are interacting with the virtual assistant at the same time. The system will include different interactive content in the task queue pre-set by the virtual assistant, activate the processing module pre-set in the hotel room, and allocate these interactive content from the task queue to the processing module. The processing module responds to these interactive content in the hotel room and identifies the collaborative association information that has been pre-classified for these interactive content. The collaborative association information specifically includes keyword matching, spatial position matching and task target matching, and generates modal conflict content of the interactive content according to different collaborative association information. The system can effectively manage the interaction requests of multiple users by including different interactive content in the task queue and allocating them to the processing module. , allowing the system to avoid command confusion and processing conflicts when multiple users initiate interactions at the same time, ensuring that the needs of each user in the hotel room can be handled independently and appropriately. At the same time, by identifying the collaborative correlation information of the interactive content (such as keyword matching, spatial location matching, and task goal matching), the intention and context of different interactive content can be accurately distinguished. This precise identification and classification capability helps the system better understand user needs, thereby providing more relevant and personalized services to different users. When a modal conflict is detected, it can generate conflict content based on the collaborative correlation information and dynamically adjust the processing strategy. This capability allows the system to flexibly handle conflicts when facing multiple interaction types, ensuring interoperability and coordination between different modal inputs, thereby improving the accuracy and consistency of the response; the system then determines whether these modal conflict contents can be dynamically adjusted to execute the corresponding steps;For example, when the system determines that the modal conflict content of the interactive content cannot be dynamically adjusted, the system will consider that there are two or more conflicting interactive contents, and only one of them can be selected for implementation. The system will prioritize the conflicting interactive content, actively feedback the conflict to the user, and request the user to make a selection, including the user's preference settings, the urgency of the interactive content, the time sequence of the user's input, and the importance of the target task of the content. At the same time, the user is prompted with the selected interactive content and the other content that has not been executed is explained through voice prompts, text messages or image displays to ensure that the user understands the system's decisions and processing results, and avoids confusion or dissatisfaction caused by the system's failure to execute a certain content. If the unselected interactive content is equally important to the user, the system will prompt the user after the main task is executed, and suggest whether compensatory execution is needed. For example, when the system determines that the modal conflict content of the interactive content can be dynamically adjusted, the system will consider that the conflicting interactive content can be split and performed simultaneously. The system will identify the input data of the modal conflict content, which specifically includes visual images, voice commands and tactile feedback. The same input data represents the interactive information of different modes as nodes in a pre-set graph model, and the edges between nodes represent the dependencies between different modes. Based on the nodes and dependencies, a response strategy for modal conflict content is generated; the system can flexibly respond to the conflict of multimodal information by splitting the conflicting interactive content for parallel processing, avoiding operation interruption or failure to respond due to modal conflict, dynamically adjusting the processing strategy according to the real-time situation, and better adapting to complex user needs. At the same time, the interactive information of different modes is represented as nodes in the graph model, and processed through the dependencies between nodes, so that the system can allocate and utilize resources more efficiently, reduce the waste of system resources, improve the overall computing efficiency, and be able to process more tasks in the same time period. In addition, the use of the graph model allows the system to be more meticulous when processing complex scenarios. For example, when there are complex dependencies between visual images and voice commands, the system can accurately analyze and process these relationships to ensure that each interactive content can be properly responded to and processed, generate a more reasonable response strategy, and make decisions that meet user expectations in conflict situations, reducing misunderstandings and misoperations. ;
[0038] In this embodiment, the execution unit further includes: A presentation subunit, configured to present control contents of preset linked guest rooms in hotel guest rooms based on the cross-room linkage settings preset by the user through the hotel terminal; A judging subunit, used for judging whether the control content can be synchronized to the hotel guest room; An execution subunit is used to, if possible, enable the virtual assistant to issue a corresponding task request based on the interactive content, identify the pre-recorded content of the linked guest room, generate the task request from the linked guest room based on the pre-recorded content, and coordinate the hotel room and the linked guest room to synchronously execute the task request through the virtual assistant.
[0039] In this embodiment, the system presents the pre-set control contents of the linked guest rooms in the hotel rooms based on the cross-room linkage settings pre-set by the user through the hotel terminal, and then the system determines whether these control contents can be synchronized to the hotel rooms to execute the corresponding steps; for example, when the system determines that the control contents of the linked guest rooms cannot be synchronized to the hotel rooms, the system will consider that the functional contents of the hotel rooms cannot be applied to the linked guest rooms, and the system will display the specific reasons, such as "due to incompatible devices, the contents of the linked guest rooms cannot be synchronized to the current room." , let the user know where the problem lies. After notifying the user, the system will try to automatically reconnect or troubleshoot to see if the problem can be solved to avoid problems caused by temporary network problems or device connection problems. It also recommends that the user change the cross-room linkage settings to exclude the control content that cannot be synchronized in the current guest room, and provide feasible synchronization options based on the actual situation of the current guest room, such as replacing other hotel rooms and linkage rooms that can achieve cross-room linkage; for example, when the system determines that the control content of the linkage guest room can be synchronized to the hotel room, the system will then believe that the functional content of the hotel room can be applied to the linkage guest room for synchronous execution. The system will enable the virtual assistant to issue a corresponding task request based on the interactive content, identify the executable content pre-included in the linkage guest room, generate the same task request from the linkage guest room based on different executable content, and coordinate the hotel room and the linkage guest room to synchronously execute the task request through the virtual assistant; the system Identifying the executable content of the linked guest rooms and automatically generating the same task requests enables the virtual assistant to intelligently coordinate the devices in multiple rooms and automatically execute the user's linkage settings. This not only reduces the user's operating burden, but also demonstrates the efficiency and intelligence of the system, meeting the user's need to use multiple rooms for simultaneous speeches or parties. At the same time, in the multi-room linkage scenario, the system uses a unified task request to keep the operations of each room consistent, avoiding differences in user experience caused by inconsistent settings in different rooms. For example, when users play background music or adjust lighting in multiple rooms simultaneously, the system can ensure that the effects in each room are consistent, improving the user's overall experience, and by synchronously executing task requests, it can quickly respond to user needs in multiple rooms without having to adjust settings room by room. This is especially suitable for scenarios such as business meetings and family gatherings, where users can quickly set up the environment of multiple rooms to meet different needs.
[0040] In this embodiment, it also includes: An acquisition unit, configured to acquire, based on the modality information pre-generated by the hotel room for the interactive content, a task request obtained by converting the modality information; A third judgment unit is used to judge whether the task request meets the preset execution content of the hotel room; The third execution unit is used to, if not, identify the missing modality of the modal information, generate the content to be completed of the modal information according to the missing modality, obtain the user's response feedback to the content to be completed, reconstruct the response feedback into the modal information according to the interaction type, and convert it again to obtain the task request.
[0041] In this embodiment, the system obtains the task requests converted from the modal information based on the modal information pre-generated by the hotel room for the interactive content, and then the system determines whether the task requests meet the pre-set execution content of the hotel room to execute the corresponding steps; for example, when the system determines that the task request obtained by the modal information conversion can meet the pre-set execution content of the hotel room, the system will consider that the modal information processing and task request generation process is effective, and the virtual assistant can correctly understand the user's intention. The system will directly execute these qualified task requests. For example, if the user requires to adjust the room temperature through voice commands, the system will immediately adjust the air conditioning settings to achieve the temperature required by the user, and provide feedback to confirm that the task request has been successfully executed, including through voice prompts, display screen messages or mobile application notifications, such as "The room temperature has been adjusted to 22°C.", and record the results of the task execution for subsequent analysis and optimization. These data can also be used to personalize the user experience, such as automatically adjusting the room settings according to the user's preferences; for example, when the system determines that the task request obtained by the modal information conversion cannot meet the pre-set execution content of the hotel room, the system will consider that the virtual assistant cannot understand the user's intention. Figure 1 shows the system's missing modality, generates the content to be completed of the modal information based on the missing modality, obtains the user's response feedback to the content to be completed, reconstructs the response feedback into the modal information according to the interaction type, and converts it again to obtain the task request. When the system cannot directly understand the user's intention, the robustness of the system can be effectively improved by identifying and completing the missing modal information. Even if the task request converted for the first time does not meet the preset content, the system can adaptively adjust by completing the information to enhance the processing ability of complex interactive scenarios. At the same time, by generating the content to be completed and obtaining the user's feedback, the user can more clearly understand the system's needs and respond. This two-way interaction helps to eliminate misunderstandings and ensure that the final task request is more in line with the user's true intention, thereby improving the user's interaction satisfaction. It also automatically identifies the missing modal information and makes suggestions for completion, reducing the need for users to actively correct or repeat input, making user operations easier and allowing users to interact with the virtual assistant more easily. Through multiple conversions and feedback iterations, the system can more accurately generate task requests that meet user needs. This repeated correction process ensures the accuracy of the final task request and improves the success rate and efficiency of task execution.
[0042] In this embodiment, the second execution unit further includes: An acquisition subunit, configured to acquire, based on the preset collection features of the hotel room, the feature type input by the user into the hotel room through the virtual assistant; A second judgment subunit is used to judge whether the feature type exceeds a preset upper limit; The second execution sub-unit is used to identify the modal conflict points of the feature type through the virtual assistant, generate the interceptable paragraph information of the modal conflict content according to the modal conflict points, transfer the interceptable paragraph information to a preset blank paragraph, construct the interactive content to be uploaded corresponding to the interceptable paragraph information, and synchronize the interactive content to be uploaded to the virtual assistant.
[0043] In this embodiment, the system obtains the types of features input by different users in the hotel room through the virtual assistant based on the pre-set collection features of the hotel room, and then the system determines whether these types of features exceed the pre-set upper limit to execute the corresponding steps; for example, when the system determines that the types of features input by different users through the virtual assistant do not exceed the pre-set upper limit, the system will consider that the user's input behavior is within the range allowed by the system, the number and types of feature types are at a normal level, and the system can process these inputs normally. The system will classify the collected user input features according to the pre-set classification rules and assign them to the corresponding processing modules for processing. At the same time, through multimodal fusion technology, different types of input features are integrated to generate a comprehensive user request description, such as combining voice commands with operations on the touch screen to form a complete task instruction, and after completing the task, the execution result is fed back to the user to confirm whether the task is completed as expected by the user, and the user is replied to by the virtual assistant, providing relevant feedback options, and the user can confirm or make further adjustments; for example, when the system determines that the types of features input by different users through the virtual assistant exceed the pre-set upper limit, the system will consider that the virtual assistant If the system cannot complete the interaction corresponding to these interactive contents, the system will identify the modal conflict points of these feature types through the virtual assistant, generate the interceptable paragraph information of the modal conflict content according to the modal conflict points, transfer the interceptable paragraph information to the pre-set blank paragraph, construct the interactive content to be uploaded corresponding to the interceptable paragraph information, and synchronize the interactive content to be uploaded to the virtual assistant; by identifying the modal conflict points and generating the interceptable paragraph information of the modal conflict content, the system can clarify and decompose complex user requests, help the system clearly understand the user's interactive content, avoid the difficulty in understanding caused by excessive feature types, and make each Paragraphs focus on specific interaction points or request parts, thereby reducing confusion and misunderstanding. At the same time, the interceptable paragraph information of modal conflict content is synchronized to the virtual assistant, enabling the system to gradually process complex interaction requests instead of processing all information at once, thereby improving processing efficiency and accuracy, and clearly presenting segmented request information to users, allowing users to complete the interaction step by step, avoiding confusion caused by too much information entered at one time. Even if the virtual assistant cannot complete the interaction content in one go, the interactive content to be uploaded can be re-uploaded to the virtual assistant through the interceptable paragraph information, thereby realizing secondary interaction between the user and the virtual assistant.
[0044] In this embodiment, the judging unit further includes: An identification subunit, configured to identify a corresponding interactive device in the hotel room based on the interactive content; A third judgment subunit is used to judge whether the interactive device receives at least two pieces of interactive content within a preset time period; The third execution subunit is used to obtain the interaction type of the interactive content and classify the interactive content into preset interaction options corresponding to the virtual assistant, wherein the interaction options specifically include query type interaction, request type interaction and setting type interaction.
[0045] In this embodiment, the system identifies the interactive devices set in the hotel guest room based on the interactive content, and then determines whether these interactive devices have received at least two different interactive contents within a preset time period to execute the corresponding steps; for example, when the system determines that the interactive device has not received two or more interactive contents at the same time, the system will consider that the user's current interactive demand is low, and the device does not need to perform complex multi-tasking. The system will directly execute the tasks related to the interactive content without complex task allocation or conflict management. For example, if the user requires the air-conditioning temperature to be adjusted through voice commands, the system will directly adjust the temperature setting without considering the interference of other commands. While executing the task, the system will continue to monitor whether the device receives new interactive content within the set time period. If new interactive content arrives later, the system will re-judge and perform corresponding processing. Since the interactive demand is relatively small, the system can save resources and reduce unnecessary computing and processing power consumption, thereby improving the overall system operation efficiency; for example, when the system determines that the interactive device receives two or more interactive contents at the same time, the system will consider that the interactive device may need to perform multi-tasking. The system will obtain the interaction type of these interactive contents and will not The system can quickly identify and classify multiple interactive contents to avoid processing delays caused by content confusion. By classifying interactive contents into different interactive options, the system can more effectively allocate resources, handle various tasks, and improve overall processing efficiency. At the same time, through reasonable classification of interaction types, the system can provide users with more accurate and personalized responses. For example, query interactions are prioritized to allow users to get the required information faster, while request interactions can be processed in parallel in the background to increase execution speed. This experience makes users feel that the system is intelligent and responsive. When classifying interactive contents, possible conflicts between different tasks can be identified. For example, if a temperature setting request and a light adjustment request are received at the same time, the system can prioritize them according to the nature of the tasks, or execute non-conflicting tasks at the same time to ensure that each task can be completed in order and avoid task failures caused by conflicts. Because different types of interactions usually have different requirements for system resources, through classification processing, the system can better allocate computing resources and network bandwidth.
[0046] In this embodiment, the second judgment unit further includes: A second acquisition subunit is used to acquire the conflict type of the modality conflict content, wherein the conflict type specifically includes space conflict, priority conflict and time conflict; A fourth judgment subunit, used to judge whether the conflict types are diverted to the same interaction path; The fourth execution subunit is used to, if not, generate the influencing device corresponding to the modal conflict content through a virtual assistant, identify the conflict range of the modal conflict content on the influencing device, and divide the dynamic adjustment interval of the influencing device based on the conflict range.
[0047] In this embodiment, the system obtains the conflict type of the modal conflict content, which specifically includes spatial conflict, priority conflict and time conflict, and then determines whether these conflict types are diverted to the same interaction path to execute the corresponding steps; for example, when the system determines that the conflict type of the modal conflict content is diverted to the same interaction path, the system will believe that the interactive content of multiple conflict types ultimately points to the same system resource or device, which will cause resource competition or confusion in the order of task execution. The system will immediately lock the resource and perform conflict detection. For tasks that need to access the same resource at the same time, the system will determine which task to execute first through the task scheduling mechanism, and other tasks will queue up and wait for resource release. At the same time, the priority of the tasks will be adjusted according to preset rules. For example, some tasks may be given a higher priority due to special needs of the user. At this time, the system should give priority to the task and postpone or suspend the task with a lower priority. If the priorities between the tasks are the same, the system can refer to historical records or user habits to determine the execution order, and if the conflict cannot be resolved automatically, a notification can be sent to the user to explain the current conflict situation and solicit the user's opinion. For example, the system can ask the user whether he is willing to execute one task first and then process another task. This approach can not only resolve conflicts, but also enhance the user's sense of control; for example, when the system determines that the conflict type of modal conflict content will not be diverted to the same interaction path, the system will consider that the conflict type of interactive content can be executed separately, and the system will generate the affected devices corresponding to the modal conflict content through the virtual assistant, identify the conflict range of the modal conflict content on the affected devices, and divide the dynamic adjustment interval of the affected devices based on the conflict range; because the conflict type of interactive content is executed separately, the system can handle multiple tasks at the same time without causing task delays or interruptions due to resource conflicts. This strategy enables the virtual assistant to respond to multiple user needs more quickly and enhance the consistency and real-time nature of the user experience. At the same time, by identifying the conflict range of the modal conflict content on the affected devices, the system can accurately divide the dynamic adjustment interval of the device, which can avoid unnecessary device adjustments or interference, and ensure that the device is adjusted only within the required interval, thereby maintaining the stability and efficient operation of the device, and the conflict content is reasonably diverted and processed, avoiding multiple tasks from competing for the same device or resources, reducing the interference or confusion that users may encounter when performing multiple tasks, allowing users to operate more smoothly without being confused or inconvenienced by improper system processing.
[0048] In this embodiment, the identification unit further includes: A second identification subunit is used to identify the total number of users staying in the hotel rooms; A fifth judging subunit, used to judge whether the total number of users reaches a preset threshold; The fifth execution subunit is used to obtain the priority configuration corresponding to other users in the hotel room by the user based on the feature information pre-entered by the user, collect the interactive content received in the hotel room, and divide the execution sequence of the interactive content according to the priority configuration.
[0049] In this embodiment, the system identifies the total number of users staying in hotel rooms, and then determines whether the total number of users reaches a preset threshold to execute corresponding steps; for example, when the system determines that the total number of users staying in hotel rooms does not reach the preset threshold, the system will consider that the number of users in the room is still within the range allowed by the system, and the virtual assistant does not need to perform additional multi-user management or complex interaction feature collection. The system will continue to manage in normal single-user or minority user mode. The virtual assistant will provide personalized services for the current users, such as adjusting the room temperature, playing music, or providing other personalized intelligent services. At the same time, the system will assume that all interactions and requests come from the existing minority of users. The virtual assistant can meet the needs of each user more accurately without complex task allocation or conflict management, and simplifies the user's interaction path, reducing the complex operations that may be caused by multi-user situations. Therefore, the virtual assistant The virtual assistant can quickly respond to and process each user's request without having to perform complex modal conflict processing or task queue management; for example, when the system determines that the total number of users staying in a hotel room has reached a preset threshold, the system will consider that the hotel room's virtual assistant needs to collect different interaction features of multiple users during their stay. The system will identify and confirm the feature information of all staying users through face recognition, voice recognition or other identity authentication methods to ensure that the system can distinguish between different users. At the same time, it will begin to collect the interaction features of each user, including the user's voice command preferences, temperature setting habits, and lighting brightness preferences. According to the interaction features of each user, the system will adjust its response method and optimize its response strategy to adapt to the presence of multiple users. For example, when multiple users issue commands at the same time, the system will decide which user to respond to first, or how to process these commands in parallel, based on the preset priority or frequency of use. Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A virtual assistant interaction method for a hotel room, characterized in that: The following steps are involved: Based on the pre-unlocked interactive rights of the hotel room, identifying the interactive content input by the user into the hotel room within a preset time period; Determining whether the interactive contents belong to the same interaction type, wherein the interaction type specifically includes image interaction, sound interaction, and text interaction; If not, the interactive content is included in the task queue preset by the virtual assistant, the processing module preset by the hotel room is activated, the interactive content is allocated from the task queue to the processing module, the interactive content is responded to in the hotel room by the processing module, the collaborative association information pre-classified by the interactive content is identified, and the modal conflict content of the interactive content is generated according to the collaborative association information, wherein the collaborative association information specifically includes keyword matching, spatial position matching and task target matching; Determining whether the modal conflict content can be dynamically adjusted; If possible, the input data of the modal conflict content is identified, and the interactive information of different modalities is represented as nodes in a preset graph model based on the input data. The edges between the nodes are represented as dependency relationships between different modalities, and a response strategy for the modal conflict content is generated based on the nodes and the dependency relationships, wherein the input data specifically includes visual images, voice commands, and tactile feedback.
2. The virtual assistant interaction method for hotel rooms according to claim 1, characterized in that: The step of activating the preset processing module of the hotel room, allocating the interactive content from the task queue to the processing module, and responding to the interactive content in the hotel room through the processing module also includes: Based on the cross-room linkage setting preset by the user through the hotel terminal, presenting the control content of the preset linkage guest room in the hotel guest room; Determining whether the control content can be synchronized to the hotel room; If possible, the virtual assistant will issue a corresponding task request based on the interactive content, identify the pre-recorded content of the linked guest room, generate the task request from the linked guest room based on the pre-recorded content, and coordinate the hotel room and the linked guest room to synchronously execute the task request through the virtual assistant.
3. The virtual assistant interaction method for hotel rooms according to claim 1, characterized in that: Before the step of generating the modal conflict content of the interactive content according to the collaborative association information, the step further includes: Based on the modality information pre-generated by the hotel room for the interactive content, obtaining a task request converted from the modality information; Determining whether the task request meets the preset execution content of the hotel room; If not, identify the missing modality of the modal information, generate the content to be completed of the modal information according to the missing modality, obtain the user's response feedback to the content to be completed, reconstruct the response feedback into the modal information according to the interaction type, and convert it again to obtain the task request.
4. The virtual assistant interaction method for hotel rooms according to claim 1, characterized in that: The step of identifying the input data of the modal conflict content further includes: Based on the preset collected features of the hotel room, obtaining the feature type input by the user into the hotel room through the virtual assistant; Determining whether the feature type exceeds a preset upper limit; If so, the virtual assistant identifies the modal conflict point of the feature type, generates the interceptable paragraph information of the modal conflict content according to the modal conflict point, transfers the interceptable paragraph information to a preset blank paragraph, constructs the interactive content to be uploaded corresponding to the interceptable paragraph information, and synchronizes the interactive content to be uploaded to the virtual assistant.
5. The virtual assistant interaction method for hotel rooms according to claim 1, characterized in that: The step of determining whether the interactive contents belong to the same interaction type further includes: Identify a corresponding interactive device in the hotel room based on the interactive content; Determining whether the interactive device receives at least two pieces of interactive content within a preset time period; If so, the interaction type of the interactive content is obtained, and the interactive content is classified into preset interaction options corresponding to the virtual assistant, wherein the interaction options specifically include query type interaction, request type interaction and setting type interaction.
6. The virtual assistant interaction method for hotel rooms according to claim 1, characterized in that: The step of determining whether the modal conflict content can be dynamically adjusted further includes: Acquire the conflict type of the modality conflict content, wherein the conflict type specifically includes space conflict, priority conflict and time conflict; Determining whether the conflict types are diverted to the same interaction path; If not, the influencing device corresponding to the modal conflict content is generated through a virtual assistant, the conflict range of the modal conflict content on the influencing device is identified, and the dynamic adjustment interval of the influencing device is divided based on the conflict range.
7. The virtual assistant interaction method for hotel rooms according to claim 1, characterized in that: The step of identifying the interactive content input by the user into the hotel room within a preset time period based on the interactive authority of the hotel room pre-unlocking further includes: Identify the total number of users occupying said hotel rooms; Determining whether the total number of users reaches a preset threshold; If so, based on the feature information pre-entered by the user, the priority configuration corresponding to the user in the hotel room is obtained, the interactive content received in the hotel room is collected, and the execution sequence of the interactive content is divided according to the priority configuration.
8. A virtual assistant interactive system for hotel rooms, characterized in that: include: An identification unit, configured to identify the interactive content input into the hotel room by the user within a preset time period based on the interactive authority of the hotel room pre-unlocking; A judging unit, used to judge whether the interactive contents belong to the same interaction type, wherein the interaction types specifically include image interaction, sound interaction and text interaction; an execution unit, for, if not, listing the interactive content into a task queue preset by the virtual assistant, activating a processing module preset by the hotel room, allocating the interactive content from the task queue to the processing module, responding to the interactive content in the hotel room through the processing module, identifying collaborative association information pre-classified by the interactive content, and generating modal conflict content of the interactive content according to the collaborative association information, wherein the collaborative association information specifically includes keyword matching, spatial position matching, and task target matching; A second judgment unit, used to judge whether the modal conflict content can be dynamically adjusted; The second execution unit is used to identify the input data of the modal conflict content if possible, represent the interactive information of different modalities as nodes in a preset graph model based on the input data, represent the edges between the nodes as dependency relationships between different modalities, and generate a response strategy for the modal conflict content based on the nodes and the dependency relationships, wherein the input data specifically includes visual images, voice commands, and tactile feedback.
9. The virtual assistant interactive system for hotel rooms according to claim 8, characterized in that: The execution unit also includes: A presentation subunit, configured to present control contents of preset linked guest rooms in hotel guest rooms based on the cross-room linkage settings preset by the user through the hotel terminal; A judging subunit, used for judging whether the control content can be synchronized to the hotel guest room; An execution subunit is used to, if possible, enable the virtual assistant to issue a corresponding task request based on the interactive content, identify the pre-recorded content of the linked guest room, generate the task request from the linked guest room based on the pre-recorded content, and coordinate the hotel room and the linked guest room to synchronously execute the task request through the virtual assistant.
10. The virtual assistant interactive system for hotel rooms according to claim 8, characterized in that: Also includes: An acquisition unit, configured to acquire, based on the modality information pre-generated by the hotel room for the interactive content, a task request obtained by converting the modality information; A third judgment unit is used to judge whether the task request meets the preset execution content of the hotel room; The third execution unit is used to, if not, identify the missing modality of the modal information, generate the content to be completed of the modal information according to the missing modality, obtain the user's response feedback to the content to be completed, reconstruct the response feedback into the modal information according to the interaction type, and convert it again to obtain the task request.
Citation Information
Patent Citations
Solution method of controlling multiple terminals by intelligent household core server
CN103888407A
Intelligent home control system based on network technology
CN113341756A
Interactive intelligent fitness mirror surface device
CN115177937A
Information fusion method and related equipment
CN117951637A
Smart home equipment control method and device, computer equipment and storage medium
CN118605195A
Cited By
Linkage method and system of hotel central control sound equipment and light scene
CN119668177A
A linkage method and system for a hotel central control audio and lighting scene
CN119668177B
Energy optimization management system and management method for hotel guest rooms
CN120087562A
An energy optimization management system and management method for hotel rooms
CN120087562B
