A self-service check-in whole-process method for a smart hotel based on human-computer interaction

Abstract

The application discloses a self-service check-in whole-process method for a smart hotel based on human-computer interaction, and relates to the technical field of hotel services.The method comprises the following steps: a user reaches a self-service terminal processing area according to the guidance of a vertical sign, and collects image data of the user; a multi-factor weighted decision model is constructed, an emotional state vector and current context information are input into the multi-factor weighted decision model for fusion decision, a current interaction mode to be triggered is dynamically determined, and the interaction mode is sent to a self-service terminal driving interface to switch to a corresponding interaction mode; according to the determined interaction mode, a display interface of the self-service terminal dynamically loads and renders a corresponding graphical user interface, and the user is guided to check in; after the check-in is completed, the self-service terminal guides the user to select a room card type, and generates a room path dynamic navigation map and sends the room path dynamic navigation map to a mobile terminal of the user.The application generates an emotional state vector and drives dynamic switching of an interaction mode, and realizes intelligent human-computer interaction in a true sense.

Description

Technical Field

[0001] This invention relates to the field of hotel service technology, and in particular to a smart hotel self-service check-in method based on human-computer interaction. Background Technology

[0002] With the deepening of smart hotel construction, self-service technology based on human-computer interaction has been widely applied. Current technologies mainly focus on the integration of self-service terminal equipment, biometric recognition, and mobile payment integration, achieving basic self-service through standardized processes such as touchscreen interaction, document recognition, and room key issuance. In recent years, advancements in computer vision and deep learning technologies have made more intelligent interactions possible, such as using facial recognition for identity verification and attempting to improve the interactive experience through basic expression analysis. Related technologies are gradually evolving from single-function devices to integrated service platforms.

[0003] However, existing technologies have shortcomings in handling emotional cognition and dynamic responses during user interaction. Most current technologies employ pre-set, fixed interface flows, failing to perceive and adapt to changes in the user's emotional state in real time. When users experience confusion or anxiety, rigid interaction flows can actually increase operational burden, leading to decreased efficiency or even operational interruptions. This interaction model, lacking emotional intelligence, struggles to meet the dual demands of personalized experiences and efficient operations in modern hotel services. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a smart hotel self-service check-in method based on human-computer interaction to solve the problems of poor user experience and low operational efficiency caused by the lack of real-time emotional state perception and adaptive interaction capabilities in existing self-service check-in systems.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a method for the entire process of self-service check-in in a smart hotel based on human-computer interaction. The method includes: the user arrives at the smart self-service terminal area according to the guidance of a standing sign, and the user's image data is collected; the user's image data is input into a convolutional neural network to extract facial micro-expression features and generate a confusion feature vector, and simultaneously extracts the user's body language features to generate an anxiety feature vector; the confusion feature vector and the anxiety feature vector are fused into an emotional state vector; a multi-factor weighted decision model is constructed, and the emotional state vector and current context information are input into the multi-factor weighted decision model for fusion decision-making, dynamically determining the interaction mode to be triggered, and sending it to the self-service terminal to drive the interface to switch to the corresponding interaction mode; according to the determined interaction mode, the self-service terminal's display interface dynamically loads and renders the corresponding graphical user interface to guide the user through the check-in process; after check-in is completed, the self-service terminal guides the user to select a room card type and simultaneously generates a dynamic room path navigation map which is sent to the user's mobile terminal.

[0008] As a preferred embodiment of the smart hotel self-service check-in process method based on human-computer interaction described in this invention, the user image data includes facial video stream data, body posture sequence data, and eye tracking data.

[0009] As a preferred embodiment of the intelligent hotel self-service check-in method based on human-computer interaction described in this invention, the specific steps for generating the perplexity feature vector are as follows:

[0010] A three-path convolutional neural network architecture is constructed by initializing three independent feature extraction pathways.

[0011] Facial micro-expression features are obtained by extracting instantaneous facial motion from facial video stream data through the first convolutional neural network pathway.

[0012] The gaze tracking data is extracted by using a second convolutional neural network pathway to obtain gaze behavior features;

[0013] Facial micro-expression features and gaze behavior features are fused at the feature level to form a primary fused feature, which is then processed by a fully connected layer for dimensionality reduction and normalization to generate a perplexity feature vector.

[0014] As a preferred embodiment of the intelligent hotel self-service check-in method based on human-computer interaction described in this invention, the specific steps for fusing the confusion feature vector and the anxiety feature vector into an emotional state vector are as follows:

[0015] The third convolutional neural network pathway is used to extract the motion trajectory of skeletal joints from limb posture sequence data to obtain limb motion features.

[0016] By using a gating attention mechanism, multi-scale temporal feature filtering of limb movement characteristics is performed to generate an anxiety feature vector.

[0017] The confusion feature vector and the anxiety feature vector are fused through cross-modal feature interaction to generate a preliminary emotional state vector.

[0018] The initial sentiment state vector is compressed in dimension and enhanced in features by using a feature distillation network to generate a new sentiment state vector.

[0019] As a preferred embodiment of the intelligent hotel self-service check-in method based on human-computer interaction described in this invention, the context information includes the real-time queue number and the user's historical operation proficiency.

[0020] As a preferred embodiment of the intelligent hotel self-service check-in method based on human-computer interaction described in this invention, the specific steps for constructing the multi-factor weighted decision model are as follows:

[0021] Establish a multi-source data input interface to receive emotional state vectors, real-time queue numbers, users’ historical operation proficiency, and self-service terminal equipment status, forming a set of decision factors;

[0022] The initial weights of each decision factor in the decision factor set are calculated using a dynamic weight allocation mechanism.

[0023] The initial weights of each decision factor are input into the weight adaptive adjustment mechanism to generate dynamic weight parameters that evolve over time.

[0024] By combining dynamic weight parameters with various decision factors to conduct multi-factor collaborative decision-making, interactive mode selection instructions are generated, thus completing the construction of a multi-factor weighted decision-making model.

[0025] As a preferred embodiment of the intelligent hotel self-service check-in method based on human-computer interaction described in this invention, the specific steps for dynamically determining the current interaction mode to be triggered and sending it to the self-service terminal driver interface to switch to the corresponding interaction mode are as follows.

[0026] The emotional state vector and context information are standardized to generate standardized emotional vectors and standardized context vectors.

[0027] Standardized sentiment vectors and standardized context vectors are input into a multi-factor weighted decision model, and the activation probability of each interaction mode is calculated by combining the Softmax function with an adaptive weighting mechanism.

[0028] We apply temporal attention networks to analyze the dynamic evolution trend of activation probabilities of each interaction mode and predict the probability values ​​of each interaction mode at future time steps.

[0029] Based on the probability values ​​of each interaction mode in future time steps, the interaction mode with the highest probability value and a positive growth trend is selected as the current trigger mode, and the trigger command corresponding to the current trigger mode is generated.

[0030] The trigger command corresponding to the current interaction mode is sent to the self-service terminal display interface. The controller identifies the mode code in the trigger command, calls the corresponding interface configuration parameters from the preset interface configuration library according to the mode code, loads them into the renderer, and reorganizes the layout of interface elements and interaction logic through the renderer to drive the self-service terminal display interface to switch to the corresponding interaction mode.

[0031] As a preferred embodiment of the smart hotel self-service check-in process method based on human-computer interaction described in this invention, the interaction modes include guided mode, express mode and standard mode.

[0032] As a preferred embodiment of the intelligent hotel self-service check-in method based on human-computer interaction described in this invention, the specific steps for guiding the user through the check-in process are as follows:

[0033] Based on the determined interaction mode, the corresponding interface template skeleton is indexed from the preset resource library, the interface template skeleton is parsed, and a set of interface elements of the graphical user interface is generated.

[0034] The dynamic rendering priority of each interface element in the set of interface elements is calculated using the visual entropy theory model.

[0035] The rendering pipeline is dynamically scheduled based on rendering priority. During the interface rendering process, the user's gaze focus and operation flow data are monitored in real time. The rendering priority of each interface element is dynamically adjusted and local re-rendering is triggered to guide the user through the check-in process.

[0036] As a preferred embodiment of the intelligent hotel self-service check-in method based on human-computer interaction described in this invention, the step of simultaneously generating a dynamic room path navigation map and sending it to the user's mobile terminal includes the following specific steps.

[0037] After check-in is completed, the self-service terminal activates the room card type selection interface to guide the user to select the room card type and obtains the room location coordinates, self-service terminal coordinates and real-time people density heat map.

[0038] The optimal path is generated by performing multi-objective dynamic weight optimization on room location coordinates, self-service terminal coordinates, and real-time people density heat map through a path navigation optimization algorithm.

[0039] The selected room card type and the optimal route are packaged together to generate an augmented reality navigation data package, which is then sent to the user's mobile terminal.

[0040] The beneficial effects of this invention are as follows: By generating emotional state vectors and driving dynamic switching of interaction modes, truly intelligent human-computer interaction is achieved. Micro-expression and body language features extracted through convolutional neural networks can accurately capture users' emotional states such as confusion and anxiety, forming quantified emotional state vectors. After fusing these emotional state vectors with real-time contextual information through a multi-factor weighted decision model, the most suitable interaction mode for the current user state can be automatically triggered. The adaptive mechanism based on emotional cognition can proactively adapt to users' real-time needs, providing more detailed guidance when users are confused, simplifying process steps when users are anxious, improving operational fluency and interaction efficiency, while reducing user learning costs and creating a natural and efficient end-to-end check-in experience. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the entire process of self-service check-in for smart hotels based on human-computer interaction.

[0043] Figure 2 A flowchart for generating perplexity feature vectors.

[0044] Figure 3 A flowchart for generating emotion state vectors.

[0045] Figure 4 A flowchart for constructing a multi-factor weighted decision model. Detailed Implementation

[0046] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0048] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0049] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for the entire process of self-service check-in in a smart hotel based on human-computer interaction, including the following steps:

[0050] S1. Users arrive at the smart self-service terminal processing area according to the guidance of the standing signs, and user image data is collected.

[0051] S1.1: User image data includes facial video stream data, body posture sequence data, and eye tracking data.

[0052] It should be noted that facial video stream data refers to a sequence of dynamic video images of a user's face captured continuously by a high-definition camera;

[0053] Limb posture sequence data refers to the time-series data of the movement trajectory of the user's whole skeletal joints, collected by a depth camera;

[0054] Eye-tracking data refers to the temporal data of the fixation point position and blink frequency generated by the user's eye movements, recorded by eye-tracking devices.

[0055] S2. Input user image data into a convolutional neural network, extract user facial micro-expression features, generate confusion feature vector, extract user body language features, generate anxiety feature vector, and fuse confusion feature vector and anxiety feature vector into emotional state vector.

[0056] S2.1: Construct a three-path convolutional neural network architecture by initializing three independent feature extraction pathways;

[0057] Specifically, three independent convolutional neural network pathways are initialized, each with its own convolutional layer, pooling layer, and activation layer. The first convolutional neural network pathway is configured to process facial video stream data, the second convolutional neural network pathway is configured to process gaze tracking data, and the third convolutional neural network pathway is configured to process limb pose sequence data. The three-path convolutional neural network architecture shares the same input interface and outputs independent feature vectors.

[0058] S2.2: Extract instantaneous facial motion from facial video stream data through the first convolutional neural network pathway to obtain facial micro-expression features;

[0059] Specifically, the first convolutional neural network path analyzes pixel-level changes between consecutive frames in the facial video stream data, identifies and captures brief and subtle instantaneous motion information of the user's facial muscles (corresponding to facial expression changes that appear and disappear quickly). The first convolutional neural network path extracts feature information representing the user's brief emotional response from the instantaneous motion information, which is the facial micro-expression feature.

[0060] S2.3: Extract gaze point distribution from gaze tracking data through the second convolutional neural network pathway to obtain gaze behavior features;

[0061] Specifically, the second convolutional neural network path analyzes the position sequence of the user's gaze point on the screen in the gaze tracking data, identifies and captures the dwell time and movement path of the gaze point in different areas, and generates spatial and temporal information. The second convolutional neural network path extracts feature information representing the user's visual attention from the spatial and temporal information, which is the gaze behavior feature.

[0062] S2.4: Facial micro-expression features and gaze behavior features are fused at the feature level to form a primary fused feature, and then dimensionality reduction and normalization are performed through a fully connected layer to generate a perplexity feature vector.

[0063] Specifically, facial micro-expression features and gaze behavior features are concatenated along the feature dimension to form a primary fusion feature. The primary fusion feature is input into a fully connected layer, where a linear transformation is performed to reduce its dimensionality, generating a dimensionality-reduced fusion feature. The dimensionality-reduced fusion feature is then normalized to ensure its numerical distribution falls within a specific range. After dimensionality reduction and normalization by the fully connected layer, a perplexity feature vector is generated.

[0064] It should be noted that the specific range refers to scaling the numerical values ​​of the dimensionality-reduced and fused features to a standard numerical range through normalization operations; this can eliminate the dimensional differences between different features, ensure that all features are at the same numerical level, and improve stability and comparability.

[0065] S2.5: Extract the motion trajectory of skeletal joints from the limb posture sequence data through the third convolutional neural network pathway to obtain limb motion features;

[0066] Specifically, the third convolutional neural network pathway focuses on the changes in the coordinates of skeletal joints recorded in the limb posture sequence data, identifies the positional movement of each joint at consecutive time points, generates the motion trajectory of each joint, and extracts feature information representing the overall limb movement pattern of the user from the motion trajectory of each joint, which is the limb movement feature.

[0067] S2.6: Use a gating attention mechanism to perform multi-scale temporal feature filtering on limb movement features to generate an anxiety feature vector;

[0068] Specifically, the gating attention mechanism is applied to limb movement features. It assesses the importance of different time scales by analyzing the intensity, duration and frequency of change of limb movement features, and weights the limb movement features according to the assessed importance to highlight multi-scale temporal information. The gating attention mechanism then filters out a quantitative vector representing the user's anxiety from the multi-scale temporal information, which is the anxiety feature vector.

[0069] S2.7: Perform cross-modal feature interaction fusion between the confusion feature vector and the anxiety feature vector to generate a preliminary emotional state vector;

[0070] Specifically, the confusion feature vector and the anxiety feature vector are concatenated along the feature dimension to form a comprehensive emotional feature vector. This comprehensive emotional feature vector integrates multimodal information from facial expressions, eye contact, and body movements to jointly represent the user's emotional state. Through cross-modal feature combination, a preliminary emotional state vector is generated.

[0071] S2.8: The initial sentiment state vector is compressed in dimension and enhanced in features by using a feature distillation network to generate a new sentiment state vector.

[0072] It should be noted that the pre-training process of the feature distillation network is as follows: Annotated samples from a historical multimodal emotion dataset are used. This dataset includes annotated historical facial video streams, historical gaze tracking data, and historical body posture sequence data with real emotional states. These samples are then forward-propagated through three convolutional neural network pathways: the first pathway analyzes pixel changes between consecutive frames in the historical facial video streams, outputting historical facial micro-expression features. The second pathway analyzes the coordinate sequence and distribution of gaze points in the historical gaze tracking data, outputting historical gaze behavior features. The third pathway analyzes the motion trajectory of skeletal joints in the historical body posture sequence data, outputting historical body posture features. Motion features are constructed by concatenating historical facial micro-expression features with historical gaze behavior features to form a primary fusion feature. This primary fusion feature is then transformed through a fully connected layer to generate a historical confusion feature vector. Historical limb movement features are analyzed through a gated attention mechanism to generate a historical anxiety feature vector. The historical confusion feature vector and the historical anxiety feature vector are fused element-wise to form a preliminary historical emotional state vector sample. The feature distillation network uses this preliminary historical emotional state vector sample as its learning object. The internal parameters of the feature distillation network are adjusted through an optimization algorithm to minimize the difference between the compressed feature vector generated by the feature distillation network and the real emotional state labeled in the sample. This completes the parameter learning process of the feature distillation network, resulting in the trained feature distillation network.

[0073] Specifically, the initial emotional state vector is input into the feature distillation network, which contains a fully connected layer structure. The feature distillation network performs a linear transformation operation on the initial emotional state vector to reduce the feature dimension. At the same time, it performs a non-linear mapping on the linearly transformed features through an activation function to enhance the feature expressive power. After the dimensionality compression and feature enhancement operations of the feature distillation network, the emotional state vector is generated.

[0074] S3. Construct a multi-factor weighted decision model, input the emotional state vector and the current context information into the multi-factor weighted decision model for fusion decision, dynamically determine the interaction mode that should be triggered at the moment, and send it to the self-service terminal to drive the interface to switch to the corresponding interaction mode.

[0075] S3.1: Contextual information includes the real-time queue size and the user's historical operational proficiency.

[0076] It should be noted that the real-time queue number is obtained by using visual sensors installed in the self-service terminal area to count the number of users currently waiting; the user's historical operation proficiency is obtained by querying the user's past operation records of using the self-service terminal, including the average operation time and the number of incorrect operations.

[0077] S3.2: Establish a multi-source data input interface to receive emotional state vectors, real-time queue numbers, users' historical operation proficiency, and self-service terminal equipment status to form a set of decision factors;

[0078] Specifically, the multi-source data input interface connects to four independent data sources, receiving sentiment state vectors, real-time queue numbers, users' historical operational proficiency, and self-service terminal device status. These four data points are used as decision factors, and are aggregated and formatted through the multi-source data input interface to form a set of decision factors.

[0079] S3.3: The initial weights of each decision factor in the decision factor set are calculated using a dynamic weight allocation mechanism. The expression is as follows:

[0080] ;

[0081] In the formula, Indicates the first The initial weights of each decision factor. Indicates the index of each decision factor. Indicates the first The pre-defined importance coefficients of each decision factor. Indicates the first Standardized values ​​of each decision factor This represents the index of the summation loop variable, used to iterate through each decision factor in the set of decision factors. Indicates the index of the summation loop variable Perform a summation operation from 1 to 4. This indicates the index of the summation loop variable in the current summation loop. The pre-defined importance coefficient of the decision factor being referred to. This indicates the index of the summation loop variable in the current summation loop. The standardized value of the decision factor it points to.

[0082] It should be noted that the preset importance coefficient is set based on the business priority of each decision factor in the interaction mode decision. The values ​​are: emotional state vector 0.5, real-time queue number 0.2, user's historical operation proficiency 0.2, and self-service terminal device status 0.1. The basis for the values ​​is that emotional state has the greatest impact on user experience and therefore has the highest weight, while device status stability has the least impact and therefore has the lowest weight.

[0083] S3.4: Input the initial weights of each decision factor into the weight adaptive adjustment mechanism to generate dynamic weight parameters that evolve over time;

[0084] Specifically, by monitoring the duration of the current user session at the self-service terminal and mapping it to a preset decay curve, a time decay factor is obtained. By statistically analyzing the changes in the number of people queuing in a short time window and comparing the difference between the user's current operation and the historical average operation, a real-time impact factor is obtained. The weight adaptive adjustment mechanism receives the initial weights of each decision factor and operates in combination with the time decay factor and the real-time impact factor. The time decay factor gently lowers each decision factor based on the duration of the current session, while the real-time impact factor finely adjusts each decision factor by raising or lowering it based on the magnitude of the value changes. Through the combined effect of the time decay factor and the real-time impact factor, the weight adaptive adjustment mechanism dynamically adjusts the initial weights of each decision factor, generating dynamic weight parameters that evolve over time.

[0085] It should be noted that the preset decay curve refers to the curve used in the weight adaptive adjustment mechanism to describe the mapping relationship between the continuous running time of the self-service terminal equipment and the degree of weight decay.

[0086] S3.5: Perform multi-factor collaborative decision-making with dynamic weight parameters and various decision factors, generate interactive mode selection instructions, and complete the construction of a multi-factor weighted decision-making model.

[0087] Specifically, the dynamic weight parameters are weighted and combined with the corresponding decision factors to obtain the weighted contribution value of each decision factor. The weighted contribution values ​​of all decision factors are accumulated to form a comprehensive decision value. The comprehensive decision value is mapped to a preset interaction mode selection range to generate a mapping result. The interaction mode is determined based on the mapping result. The determined interaction mode is formatted and encapsulated to generate the corresponding interaction mode selection instruction, thus completing the construction of the multi-factor weighted decision model.

[0088] It should be noted that the preset interaction mode selection range refers to the continuous numerical ranges that are pre-divided according to the numerical range of the comprehensive decision value, corresponding to the three specific interaction modes: guided mode, speed mode, and standard mode.

[0089] S3.6: Standardize the emotional state vector and context information to generate standardized emotional vector and standardized context vector;

[0090] Specifically, the Min-Max standardization method is used to transform the values ​​of each dimension in the emotional state vector to between zero and one, generating a standardized emotional vector. The Min-Max standardization method is also used to transform the real-time queuing number and the user's historical operation proficiency values ​​in the context information to between zero and one, generating a standardized context vector. The standardization process ensures that the different dimensions of data in the emotional state vector and context information are converted into a unified numerical range, providing a standardized data foundation for the multi-factor weighted decision model.

[0091] S3.7: Input the standardized sentiment vector and standardized context vector into the multi-factor weighted decision model, and calculate the activation probability of each interaction mode through the Softmax function combined with an adaptive weighting mechanism. The expression is:

[0092] ;

[0093] ;

[0094] In the formula, Indicates the first The activation probability of each interaction mode Indicates the first Weighted decision values ​​for various interaction modes Indicates the interaction mode index. Indicates the first The dynamic weight parameters of each decision factor. Indicates the first The numerical values ​​of each decision factor.

[0095] S3.8: Apply temporal attention network analysis to analyze the dynamic evolution trend of activation probability of each interaction mode and predict the probability value of each interaction mode in future time steps;

[0096] It should be noted that the pre-training process of the temporal attention network is as follows: Training is performed using a historical interaction record dataset, which contains multiple sets of historical interaction mode activation probability sequences arranged chronologically and corresponding annotations of actual interaction mode switching time points. The temporal attention network uses the historical interaction mode activation probability sequences as its learning object. An optimization algorithm is used to adjust the internal parameters of the temporal attention network to minimize the difference between the predicted future time step probabilities generated by the temporal attention network and the actual switching time points annotated in the historical interaction record dataset, thus obtaining the trained temporal attention network.

[0097] Specifically, the temporal attention network analyzes the activation probability sequence of each interaction mode over the most recent time steps. By focusing on the changing patterns and persistence characteristics of the activation probability sequence of each interaction mode, the temporal attention network assesses the relative importance of the probability values ​​at different time points. Based on the identified changing patterns and importance assessment results, the temporal attention network infers the future direction and magnitude of the activation probability of each interaction mode and predicts the probability value of each interaction mode in future time steps.

[0098] S3.9: Based on the probability values ​​of each interaction mode in future time steps, select the interaction mode with the highest probability value and a positive growth trend as the current trigger mode, and generate the trigger command corresponding to the current trigger mode.

[0099] Specifically, the probability values ​​of future time steps corresponding to all interaction modes are compared, the probability value with the largest value is identified, and the changing trend of the probability values ​​of each interaction mode is analyzed. The positive growth trend is confirmed by calculating the slope value of the probability change trend. The interaction mode that simultaneously satisfies the condition of having the largest probability value and a positive slope of the probability change trend is selected as the current trigger mode. Based on the type of the current trigger mode, a structured data packet containing mode code and timestamp information is generated. The structured data packet is the trigger instruction corresponding to the current trigger mode.

[0100] The expression for calculating the slope of the probability change trend is:

[0101] ;

[0102] In the formula, The slope value represents the trend of probability change. Indicates the current moment. Indicates the current time The probability value of the interaction mode at that time. Indicates the previous moment The probability value of the interaction mode at that time. It represents a fixed time interval between two adjacent moments.

[0103] S3.10: Send the trigger command corresponding to the current interaction mode to the self-service terminal display interface. The controller identifies the mode code in the trigger command, calls the corresponding interface configuration parameters from the preset interface configuration library according to the mode code, loads them into the renderer, and reorganizes the layout of interface elements and interaction logic through the renderer to drive the self-service terminal display interface to switch to the corresponding interaction mode.

[0104] Specifically, the trigger command corresponding to the current interaction mode is transmitted to the controller of the self-service terminal display interface through the data transmission channel inside the self-service terminal. The controller parses the mode encoding information contained in the trigger command, retrieves the corresponding interface layout parameters and interaction logic configuration from the preset interface configuration library according to the mode code, and loads them into the renderer of the self-service terminal display interface. The renderer organizes the interface elements and operation flow according to the corresponding interface layout parameters and interaction logic configuration, thereby completing the switching process of the self-service terminal display interface from the current mode to the target interaction mode.

[0105] It should be noted that the pre-built interface configuration library is created by the interface design tool in advance, and the interface layout and interaction logic of the guided mode, the fast mode and the standard mode are coded into configuration files and stored in the local storage of the self-service terminal.

[0106] The controller refers to the hardware in the self-service terminal display interface that is responsible for receiving trigger commands, parsing mode encoding information, and calling configuration parameters;

[0107] Mode encoding refers to the string identifier contained in the trigger command that is used to uniquely identify the interaction mode (such as boot mode, speed mode, and standard mode);

[0108] Interface configuration parameters refer to the interface layout parameters (such as element position and size) and interaction logic configurations (such as operation flow rules) preset in the interface configuration library.

[0109] The renderer refers to the mechanism in the self-service terminal display interface that dynamically organizes interface elements and generates visual rendering based on interface configuration parameters.

[0110] Interface element layout and interaction logic refer to the arrangement structure of visual elements such as buttons and text in the interface, as well as the operation flow and response rules when users interact with the interface.

[0111] S3.11: Interaction modes include Guided Mode, Speed ​​Mode, and Standard Mode.

[0112] It should be noted that the guided mode refers to the interactive interface of the self-service terminal that provides users with detailed operation instructions and additional prompts; the guided mode is aimed at users who are highly confused, unfamiliar with the operation of the self-service terminal, and who need additional help and guidance.

[0113] The express mode refers to the self-service terminal simplifying the operation process for users and retaining only the necessary steps in the interactive interface; the express mode is aimed at users who are highly anxious, familiar with the operation of self-service terminals, and pursue efficient processing.

[0114] The standard mode refers to the interactive interface provided by the self-service terminal for users with a regular operation interface and standard processing procedures; the standard mode is designed for users who are emotionally stable, have basic self-service operation capabilities, and do not require special assistance.

[0115] S4. Based on the determined interaction mode, the self-service terminal's display interface dynamically loads and renders the corresponding graphical user interface to guide the user through the check-in process.

[0116] S4.1: Based on the determined interaction mode, index the corresponding interface template skeleton from the preset resource library, parse the interface template skeleton, and generate a set of interface elements for the graphical user interface.

[0117] It should be noted that the pre-built resource library is a structured data file containing the interface template skeleton files of the boot mode, speed mode and standard mode, which are then stored in the local storage of the self-service terminal.

[0118] Specifically, based on the determined interaction mode, the corresponding interface template skeleton file is searched in the pre-built resource library using pattern matching. The interface template skeleton uses a structured data format to describe the interface layout and component attributes. The parsing process includes reading the hierarchical structure data, component type definitions, and attribute configuration information in the interface template skeleton file. After parsing, a set of interface elements for the graphical user interface is generated. The set of interface elements includes visual components such as buttons, text labels, and input boxes, as well as corresponding position, size, style, and other attribute parameters.

[0119] S4.2: Calculate the dynamic rendering priority of each interface element in the interface element set using the visual entropy theory model. The expression is:

[0120] ;

[0121] ;

[0122] In the formula, Indicates the first The visual entropy value of each interface element. Indicates the visual feature dimension index. Indicates the total number of visual feature dimensions. Indicates the first The interface element in the first The relative proportion of each visual feature dimension Indicates a base-2 pair Take the logarithm. Indicates the first The dynamic rendering priority of each UI element.

[0123] It should be noted that using a base-2 logarithm is the standard mathematical convention for measuring information entropy in information theory, ensuring that the calculated visual entropy is consistent with the dimensions and physical meaning of entropy values ​​in information theory.

[0124] S4.3: Dynamically schedule the rendering pipeline based on rendering priority. During the interface rendering process, monitor the user's gaze focus and operation flow data in real time, dynamically adjust the rendering priority of each interface element and trigger local re-rendering to guide the user through the check-in process.

[0125] Specifically, the rendering pipeline organizes the drawing order of interface elements according to their rendering priority, prioritizing the drawing of high-priority elements. During the interface rendering process, the pipeline continuously acquires the coordinates of the user's gaze focus through eye-tracking devices and continuously acquires user operation flow data through the touch sensors of the self-service terminal. When it detects that the user's gaze lingers on a certain interface element or that the user's operation flow generates an interaction event in the same interface element area, the rendering priority of the interface element is automatically increased, and an updated rendering priority is generated. The rendering pipeline then triggers a local re-rendering operation based on the updated rendering priority, updating only the interface areas where the rendering priority has changed, thus guiding the user to complete the check-in process.

[0126] S5. After check-in is completed, the self-service terminal guides the user to select the room card type and generates a dynamic navigation map of the room route, which is then sent to the user's mobile terminal.

[0127] S5.1: After check-in is completed, the self-service terminal activates the room card type selection interface to guide the user to select the room card type and obtains the room location coordinates, self-service terminal coordinates and real-time people density heat map.

[0128] Specifically, after check-in is completed, the self-service terminal automatically activates the room card type selection interface, which includes physical room cards and virtual electronic room cards. At the same time, the self-service terminal obtains the room location coordinates of the currently assigned room from the pre-set hotel management database, obtains its own coordinates through the built-in locator, and obtains real-time heat map data of the flow of people in public areas from the hotel security monitoring platform.

[0129] It should be noted that the pre-installed hotel management database is a centralized data storage system in which the smart hotel management party pre-enters room information, user reservations, and real-time room status data, deploys it on the hotel server, and authorizes self-service terminals to access it.

[0130] S5.2: The optimal path is generated by performing multi-objective dynamic weight optimization on room location coordinates, self-service terminal coordinates, and real-time people density heat map through a path navigation optimization algorithm.

[0131] Specifically, the route navigation optimization algorithm considers the dual requirements of minimizing path length and avoiding densely populated areas. It also takes into account the geometric distance between the room location coordinates and the self-service terminal coordinates, as well as the population density values ​​of each area in the real-time population density heatmap. With the dual optimization objectives of reducing the total travel distance and avoiding high-density areas, it obtains a route that achieves the best balance between path length and congestion by assigning dynamic weights to the shortest distance and smoothness of passage. The route with the best balance is the optimal path from the self-service terminal location to the room location.

[0132] S5.3: Package the selected room card type with the optimal path to generate an augmented reality navigation data package, and send it to the user's mobile terminal.

[0133] Specifically, the selected room card type and the optimal route are encapsulated and combined to form a structured data set. The structured data set is then encoded according to the augmented reality navigation format to generate an augmented reality navigation data package containing room card type information and route navigation information. The augmented reality navigation data package is transmitted to the user's mobile terminal through the wireless communication link established by the self-service terminal.

[0134] This embodiment also provides a computer device applicable to the whole process method of smart hotel self-service check-in based on human-computer interaction, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the whole process method of smart hotel self-service check-in based on human-computer interaction as proposed in the above embodiment.

[0135] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0136] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the method for realizing the entire process of smart hotel self-service check-in based on human-computer interaction as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0137] In summary, this invention achieves truly intelligent human-computer interaction by generating emotional state vectors and driving dynamic switching of interaction modes. Micro-expression and body language features extracted through convolutional neural networks can accurately capture users' emotional states such as confusion and anxiety, forming quantified emotional state vectors. These emotional state vectors, fused with real-time contextual information through a multi-factor weighted decision model, can automatically trigger the interaction mode most suitable for the current user state. The adaptive mechanism based on emotional cognition proactively adapts to users' real-time needs, providing more detailed guidance when users are confused and simplifying process steps when users are anxious, improving operational fluency and interaction efficiency, while reducing user learning costs and creating a natural and efficient end-to-end check-in experience.

[0138] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A self-service check-in process for a smart hotel based on human-computer interaction, characterized in that: The method comprises the steps of: The user reaches the self-service terminal area according to the guidance of the vertical sign, and collects user image data; The user image data is input into a convolutional neural network to extract user facial micro-expression features and generate a confusion degree feature vector, and user body language features are extracted to generate an anxious degree feature vector, and the confusion degree feature vector and the anxious degree feature vector are fused into an emotional state vector; A multi-factor weighted decision model is constructed, the emotional state vector and the current context information are input into the multi-factor weighted decision model for fusion decision, the current interaction mode to be triggered is dynamically determined, and the self-service terminal driving interface is switched to the corresponding interaction mode, wherein the context information includes real-time queue number and user historical operation proficiency; the multi-factor weighted decision model is constructed in the following specific steps, A multi-source data input interface is established to receive the emotional state vector, real-time queue number, user historical operation proficiency and self-service terminal device state, and form a decision factor set; The initial weights of the decision factors in the decision factor set are calculated through a dynamic weight distribution mechanism; The initial weights of the decision factors are input into a weight self-adaptive adjustment mechanism to generate dynamic weight parameters evolving over time; The dynamic weight parameters and the decision factors are subjected to multi-factor collaborative decision to generate an interaction mode selection instruction, and the multi-factor weighted decision model is constructed; According to the determined interaction mode, the display interface of the self-service terminal dynamically loads and renders the corresponding graphical user interface to guide the user to check in; After completing the check-in, the self-service terminal guides the user to select a room card type and generates a room path dynamic navigation map to send to the user's mobile terminal.

2. The human-computer interaction-based self-service check-in process method for a smart hotel according to claim 1, characterized in that: The user image data includes facial video stream data, body posture sequence data and gaze tracking data.

3. The human-computer interaction-based self-service check-in process method for a smart hotel according to claim 1, characterized in that: The confusion degree feature vector is generated in the following specific steps, A three-path convolutional neural network architecture is constructed by initializing three independent feature extraction paths; Facial micro-expression features are obtained by extracting facial instantaneous motion of the facial video stream data through a first convolutional neural network path; Gaze behavior features are obtained by extracting gaze point distribution of the gaze tracking data through a second convolutional neural network path; The facial micro-expression features and the gaze behavior features are fused at the feature level to form primary fusion features, which are subjected to dimension reduction and normalization processing through a fully connected layer to generate a confusion degree feature vector.

4. The human-computer interaction-based self-service check-in process method for a smart hotel according to claim 3, characterized in that: The confusion degree feature vector and the anxious degree feature vector are fused into an emotional state vector in the following specific steps, Limb motion features are obtained by extracting limb joint motion trajectories of the body posture sequence data through a third convolutional neural network path; The limb motion features are subjected to multi-scale time sequence feature screening through a gated attention mechanism to generate an anxious degree feature vector; The confusion degree feature vector and the anxious degree feature vector are subjected to cross-modal feature interaction fusion to generate a preliminary emotional state vector; The preliminary emotional state vector is subjected to dimension compression and feature enhancement through a feature distillation network to generate an emotional state vector.

5. The human-computer interaction-based self-service check-in process method for a smart hotel according to claim 1, characterized in that: The current interaction mode to be triggered is dynamically determined, and the self-service terminal driving interface is switched to the corresponding interaction mode in the following specific steps, The emotional state vector and the context information are standardized to generate a standardized emotional vector and a standardized context vector; The standardized emotional vector and the standardized context vector are input into a multi-factor weighted decision model, and the activation probability of each interaction mode is calculated by a Softmax function combined with an adaptive weight mechanism; The dynamic evolution trend of the activation probability of each interaction mode is analyzed by applying a time series attention network to predict the probability value of each interaction mode at a future time step; According to the probability value of each interaction mode at a future time step, the interaction mode with the highest probability value and a positive growth trend is selected as the current trigger mode, and a trigger instruction corresponding to the current trigger mode is generated; The trigger instruction corresponding to the current interaction mode is sent to the self-service terminal display interface, the mode code in the trigger instruction is recognized by the controller, the corresponding interface configuration parameters are called from the pre-configured interface configuration library according to the mode code and loaded into the renderer, the interface element layout and interaction logic are reorganized by the renderer, and the self-service terminal display interface is switched to the corresponding interaction mode.

6. The human-computer interaction-based self-service check-in process method for a smart hotel according to claim 1, characterized in that: The interaction mode includes a guide mode, an express mode and a standard mode.

7. The human-computer interaction-based self-service check-in process method for a smart hotel according to claim 1, characterized in that: The guide user to check in, the specific steps are as follows, According to the determined interaction mode, the corresponding interface template skeleton is indexed from the pre-configured resource library, the interface template skeleton is parsed, and a set of interface elements of a graphical user interface is generated; The dynamic rendering priority of each interface element in the set of interface elements is calculated by a visual entropy theory model; According to the rendering priority, the rendering pipeline is dynamically scheduled, and in the interface rendering process, the user's visual focus and operation flow data are monitored in real time, the rendering priority of each interface element is dynamically adjusted and local re-rendering is triggered, and the user is guided to check in.

8. The human-computer interaction-based self-service check-in process method for a smart hotel according to claim 1, characterized in that: The room path dynamic navigation map is generated at the same time and sent to the user's mobile terminal, and the specific steps are as follows, After completing the check-in, the self-service terminal activates the room card type selection interface to guide the user to select the room card type, and obtains the room position coordinates, the self-service terminal coordinates and the real-time crowd density heat map; The room position coordinates, the self-service terminal coordinates and the real-time crowd density heat map are optimized by a path navigation optimization algorithm to generate an optimal path; The selected room card type and the optimal path are packaged to generate an augmented reality navigation data packet, which is sent to the user's mobile terminal.

Images