Human-computer interaction method and device for smart home

Abstract

The present invention discloses a human-computer interaction method and device for smart home, and relates to the field of data processing technology. The method includes: establishing a local LAN of the smart home, and creating a local user database in the local LAN; reconstructing the connection strength of the local LAN based on collaborative call association and predicted collaborative call association. When receiving a user's trigger instruction at any time node, activating the backtracking window, calling the user activity data and environmental data with the backtracking window, using the local server to perform situational perception and emotion recognition of the user activity data and environmental data, and establishing a perception recognition fusion result; activating the local LAN based on the perception recognition fusion result, executing the joint call of the smart home, and generating a response interaction of the smart home. The technical problem that the smart home lacks response to the personalized needs of users and has a slow response speed in the prior art is solved, and the technical effect of improving the response speed of the smart home and enhancing the user experience is achieved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a human-computer interaction method and device for smart homes. Background Art

[0002] In the field of smart home, with the increase in the types and number of devices, how to achieve efficient human-computer interaction has become a technical difficulty. Current smart home systems often rely on a single device or application to execute user commands, lacking the ability to deeply learn users' daily behaviors and coordinate calls between multiple devices. At the same time, due to the personalized, diversified and real-time characteristics of user needs, existing technologies are difficult to identify user intentions in real time and respond quickly, especially in complex scene switching and dynamic scene recognition. In addition, the processing of user behavior data by existing smart home systems is mostly centered on individual devices, and fails to effectively integrate users' historical data and environmental information, resulting in a poor interactive experience. Summary of the invention

[0003] The present application provides a human-computer interaction method and device for a smart home, which solves the technical problem in the prior art that the smart home lacks response to the personalized needs of users and has a slow response speed.

[0004] In view of the above problems, the present application provides a human-computer interaction method and device for smart home.

[0005] In a first aspect of the present application, a human-computer interaction method for a smart home is provided, the method comprising:

[0006] A local area network of a smart home is established, and a local user database is created in the local area network, wherein the local user database is constructed by collecting historical behavior data of users; when a trigger instruction from a user is received at any time node, a backtracking window is activated, user activity data and environmental data are called through the backtracking window, and a local server is used to perform situational perception and emotion recognition of the user activity data and environmental data, and a perception and recognition fusion result is established, wherein the backtracking window refers to a time range for determining the required user activity data and environmental data; based on the perception and recognition fusion result, the local area network is activated, a joint call of the smart home is executed, and a response interaction of the smart home is generated.

[0007] Furthermore, based on the trigger instruction, the user's real-time activity recognition is performed to establish a real-time activity recognition result; the real-time activity recognition result is used to configure the window length of the backtracking window, and the backtracking window is activated based on the configuration result, and the user activity data and environmental data are called through the backtracking window; the context perception network of the local server is called, and the context perception network is used to perform context perception prediction based on the user activity data and environmental data to establish a perception result; the emotion recognition network of the local server is called, and the emotion recognition network is used to perform emotion recognition prediction based on the user activity data and environmental data to establish a recognition result; the perception result and the recognition result are weightedly authenticated and fused to establish a perception and recognition fusion result.

[0008] Furthermore, the feature extraction layer is used to extract basic data from the user activity data and environmental data to establish a basic feature set, wherein the basic data includes location data, temperature and humidity data, light data, and noise data; the feature extraction layer is used to extract activity repeatability features from the user activity data to establish activity repeatability features, and activity trend evaluation of the user activity data is performed to establish activity trend features; the activity repeatability features, activity trend features, and basic feature sets are sent to the perception layer for situational perception prediction to establish perception results.

[0009] Furthermore, a collaborative call association of the local area network is established with the local user data, and the local user data is encrypted and sent to a trusted third party to obtain a trusted third party receipt. The trusted third party receipt includes a predicted collaborative call association, and the connection strength of the local area network is reconstructed based on the collaborative call association and the predicted collaborative call association.

[0010] Furthermore, the activity repeatability features, activity trend features, and basic feature sets are integrated into the input feature matrix, the prediction function of the perception layer is activated to perform scenario label prediction, and a prediction confidence score is established; the prediction confidence score is used for score screening to establish the perception result.

[0011] Further, sentiment fluctuation analysis is performed as follows:

[0012] ;in, Represents the emotional fluctuation results, N is the total number of emotional data points in the lookback window, Represents the emotional data value collected at the i-th time point, is the average value of sentiment data;

[0013] Perform sentiment change rate analysis as follows:

[0014] ; Among them, Emotion Rate represents the rate of change of emotion, j represents the time point, Represents the emotional data value collected at the j+1th time point, Represent the emotional data value collected at the jth time point; perform emotion recognition prediction based on the emotion fluctuation results and emotion change rate, and establish the recognition result.

[0015] Furthermore, the perception recognition fusion result is confidence verified; if the perception recognition fusion result is within the confidence interval, a combined linkage selection is established based on the perception recognition fusion result; the combined linkage selection is used as a display result, the local area network is activated to execute the display, and the combined linkage selection is screened according to user feedback to generate a response interaction of the smart home.

[0016] Furthermore, a data classification for stored data is established, wherein the data classification includes direct data classification and analytical data classification; direct data classification is divided into basic data and privacy data, and a first encryption level is created with the basic data, and a second encryption level is created with the privacy data; analytical data classification is divided into direct feature analysis data and indirect feature reasoning data, and a third encryption level is created with the direct feature analysis data, and a fourth encryption level is created with the indirect feature reasoning data; hierarchical encryption management of data within the local area network is performed according to the first encryption level, the second encryption level, the third encryption level, and the fourth encryption level.

[0017] The second aspect of the present application provides a human-computer interaction device for a smart home, the device comprising:

[0018] A database construction module, the database construction module is used to establish a local area network of the smart home and create a local user database in the local area network, and the local user database is constructed by collecting historical behavior data of users; a perception and recognition module, the perception and recognition module is used to activate the backtracking window when receiving a trigger instruction from the user at any time node, call the user activity data and environmental data with the backtracking window, use the local server to perform situational perception and emotion recognition of the user activity data and environmental data, and establish a perception and recognition fusion result, wherein the backtracking window refers to the time range for determining the required user activity data and environmental data; a response interaction module, the response interaction module is used to activate the local area network based on the perception and recognition fusion result, execute the joint call of the smart home, and generate a response interaction of the smart home.

[0019] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0020] First, a local area network of a smart home is established, and a local user database is created in the local area network. The local user database is constructed by collecting historical behavior data of users. Then, the collaborative call association of the local area network is established with the local user data, and the local user data is encrypted and sent to the trusted third party to obtain a trusted third-party receipt. The trusted third-party receipt includes a predicted collaborative call association, and the connection strength of the local area network is reconstructed based on the collaborative call association and the predicted collaborative call association. When a user's trigger instruction is received at any time node, the backtracking window is activated, and the user activity data and environmental data are called with the backtracking window. The local server is used to perform situational perception and emotion recognition of the user activity data and environmental data, and establish a perception recognition fusion result. Finally, the local area network is activated based on the perception recognition fusion result, and the joint call of the smart home is executed to generate the response interaction of the smart home. The technical problem that the smart home lacks response to the personalized needs of users and has a slow response speed in the prior art is solved, and the technical effect of improving the response speed of the smart home and enhancing the user experience is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic flow chart of a human-computer interaction method for a smart home provided in an embodiment of the present application.

[0023] Figure 2 A schematic diagram of the structure of a human-computer interaction device for a smart home provided in an embodiment of the present application.

[0024] Explanation of the reference numerals: database construction module 11, perception and recognition module 12, response and interaction module 13. DETAILED DESCRIPTION

[0025] The present application solves the technical problem in the prior art that smart homes lack response to user's personalized needs and have a slow response speed by providing a human-computer interaction method and device for smart homes.

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0027] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or are inherent to these processes, methods, products or devices.

[0028] Embodiment 1, as Figure 1 As shown, the present application provides a human-computer interaction method for smart home, wherein the method includes:

[0029] A local area network of a smart home is established, and a local user database is created in the local area network. The local user database is constructed by collecting historical behavior data of users.

[0030] Build a local area network in the smart home environment to connect and manage various devices in the smart home (such as smart bulbs, smart sockets, smart door locks, cameras, temperature and humidity sensors, etc.). Then, create a local user database in the local area network to store and analyze user behavior data. The local user database is built by continuously collecting users' daily behavior information, operating habits and preference data, thereby accumulating interaction records between users and smart home devices. These historical behavior data can support the automated decision-making of smart homes and provide a data basis for subsequent situational analysis and emotion recognition.

[0031] A collaborative call association of the local area network is established with the local user data, and the local user data is encrypted and sent to a trusted third party to obtain a trusted third party receipt. The trusted third party receipt includes a predicted collaborative call association, and the connection strength of the local area network is reconstructed based on the collaborative call association and the predicted collaborative call association.

[0032] In the local area network of the smart home, the established local user database data is used to build a collaborative call association relationship between multiple devices. The collaborative call association is determined by analyzing the user's behavior pattern and device linkage requirements in daily operations, so that the devices in the local area network can work together efficiently according to user habits. To ensure the security of user data, a secure encryption algorithm (such as AES, RSA, etc.) is used to encrypt the local user data, and then the encrypted data is sent to a trusted third party (such as a cloud service provider, data analysis company, etc.); after receiving the encrypted data, the trusted third party will generate a predicted collaborative call association and return the receipt containing the predicted collaborative call association to the smart home system; after receiving the receipt, the smart home system will combine the predicted collaborative call association with the existing collaborative call association to re-optimize the device connection strength in the local area network. This optimization can dynamically adjust the connection strength between devices according to changes in user behavior and device linkage requirements, ensuring that the devices in the local area network can work together more efficiently and stably in different scenarios, thereby improving the overall response speed and user experience of the smart home system.

[0033] Furthermore, the method further comprises:

[0034] Establish data classification for stored data, wherein the data classification includes direct data classification and analytical data classification; divide the direct data classification into basic data and privacy data, and create a first encryption level with the basic data, and create a second encryption level with the privacy data; divide the analytical data classification into direct feature analysis data and indirect feature reasoning data, and create a third encryption level with the direct feature analysis data, and create a fourth encryption level with the indirect feature reasoning data; perform hierarchical encryption management of data within the local area network according to the first encryption level, the second encryption level, the third encryption level, and the fourth encryption level.

[0035] Preferably, a data classification structure is established in the local area network, and the data is divided into direct data classification and analytical data classification. The direct data classification includes the user's original operation records and other unprocessed direct data (such as user operation records, device usage records, etc.), and the analytical data classification includes the in-depth analysis results of the user's behavior patterns and operation habits (such as user behavior patterns); the direct data classification is further divided into basic data and private data, the basic data involves the user's non-sensitive operation information, which is used for ordinary system responses and operation records (such as basic operation records of equipment, timestamps, etc.), and the private data contains the user's sensitive information or personal preferences; the first encryption level is set for the basic data, and a weaker encryption algorithm or a lower key length is used for encryption; the private data is set The second encryption level uses a stronger encryption algorithm or a higher key length for encryption; the analysis data is further divided into direct feature analysis data and indirect feature inference data. Direct feature analysis data is feature data extracted directly from direct data (such as user preferences, device usage frequency, etc.), and indirect feature inference data refers to data obtained by reasoning and analyzing direct data through machine learning algorithms (such as user behavior patterns, interoperability between devices, etc.); the third encryption level is set for direct feature analysis data; the fourth encryption level is set for indirect feature inference data; in the local area network, hierarchical encryption management is performed on data of different categories according to the set first encryption level, second encryption level, third encryption level, and fourth encryption level. Data with low encryption levels can be used for daily calls and basic functions, while data with high encryption levels must undergo strict permission verification when called.

[0036] When a user's trigger instruction is received at any time node, the backtracking window is activated, and the user activity data and environmental data are called through the backtracking window. The local server is used to perform situational perception and emotion recognition of the user activity data and environmental data, and establish a perception and recognition fusion result.

[0037] The control center or terminal device of the smart home system (such as smartphone applications, voice assistants, etc.) receives the user's trigger instructions at any time point. The trigger instructions can be voice commands, gesture controls, touch operations, etc.; according to the received trigger instructions, a lookback window is activated. The lookback window is a time range used to determine the time period of user activity data and environmental data that need to be called. The length of the lookback window can be set according to actual needs, such as data in the past few minutes, hours or days.

[0038] After the lookback window is started, the user activity data and environmental data within the specified time period will be called, including the user's operation behavior, location information, habitual patterns, and current environmental parameters (such as temperature, humidity, light, etc.). The local server is used to perform situational awareness and emotion recognition analysis on the user activity data and environmental data; situational awareness is an analysis of the current situation, including the user's possible needs, the relationship between devices, and the environmental status; emotion recognition infers the user's emotional state (such as relaxation, tension, urgency, etc.) by analyzing the user's operation speed, frequency and other behavioral characteristics; the situational awareness results are integrated with the emotion recognition results to generate a perception recognition fusion result, which fully reflects the user's current needs, emotional state, and the need to adapt to the environment.

[0039] Furthermore, when a trigger instruction from a user is received at any time node, the backtracking window is activated, the user activity data and environmental data are called through the backtracking window, the local server is used to perform situational awareness and emotion recognition of the user activity data and environmental data, and a perception and recognition fusion result is established, which also includes:

[0040] Based on the trigger instruction, the user's real-time activity recognition is performed to establish the real-time activity recognition result; the window length of the backtracking window is configured using the real-time activity recognition result, and the backtracking window is activated based on the configuration result, and the user activity data and environmental data are called through the backtracking window; the context perception network of the local server is called to perform context perception prediction based on the user activity data and environmental data through the context perception network to establish the perception result; the emotion recognition network of the local server is called to perform emotion recognition prediction based on the user activity data and environmental data through the emotion recognition network to establish the recognition result; the perception result and the recognition result are weightedly authenticated and fused to establish the perception and recognition fusion result.

[0041] Specifically, when receiving the user's trigger instruction, the real-time activity recognition result is generated by analyzing the user's current behavior in real time; based on the real-time activity recognition result, the window length of the backtracking window is intelligently configured to ensure that the user activity data and environmental data called by the backtracking window are of appropriate range, neither redundant nor omitted; after the backtracking window is activated, the context perception network of the local server is called, and the context perception network performs context perception prediction based on the user activity data and environmental data in the backtracking window to generate context perception results, so as to more comprehensively understand the user's current situation and needs; at the same time, the emotion recognition network of the local server is called, and emotion recognition prediction is performed based on the user activity data and environmental data to generate emotion recognition results, and personalized information is added to the response of the smart home by identifying the user's current emotions and psychological state; the context perception results and emotion recognition results are weightedly authenticated and fused, and the information of both context and emotion is comprehensively considered to generate perception recognition fusion results.

[0042] Furthermore, the context-aware network performs context-aware prediction based on user activity data and environmental data to establish perception results, which also includes:

[0043] The feature extraction layer is used to extract basic data from the user activity data and environmental data to establish a basic feature set, wherein the basic data includes location data, temperature and humidity data, light data, and noise data; the feature extraction layer is used to extract activity repeatability features from the user activity data to establish activity repeatability features, and activity trend evaluation of the user activity data is performed to establish activity trend features; the activity repeatability features, activity trend features, and basic feature sets are sent to the perception layer for scenario perception prediction to establish perception results.

[0044] In the context-aware network, multi-layer feature extraction and analysis are used to deepen the understanding of user activities and environment, so as to generate more accurate context-aware results. Specifically, through the feature extraction layer, basic data is extracted from user activity data and environmental data to generate a basic feature set. The basic data includes user location data, temperature and humidity data, light data, and noise data; the feature extraction layer further analyzes the activity repetitiveness characteristics of user activity data and identifies the repetitive behavior of users in daily operations; based on user activity data, the feature extraction layer evaluates activity trends and identifies the changing trends of user behaviors, such as the increase or decrease in activity frequency and intensity, thereby generating activity trend features to identify the user's possible operation tendencies in different time periods; the extracted activity repetitiveness features, activity trend features, and basic feature sets are summarized and sent to the perception layer; the perception layer uses algorithms (such as machine learning, deep learning, etc.) to perform context-aware predictions on the received features. By analyzing the correlation and changing trends between these features, the perception layer can predict the user's current situation and possible needs, and generate the final perception results.

[0045] Furthermore, the method further comprises:

[0046] The activity repeatability features, activity trend features, and basic feature sets are integrated into the input feature matrix, and the prediction function of the perception layer is activated to perform scenario label prediction and establish a prediction confidence score. The prediction confidence score is used for score screening to establish the perception result.

[0047] Preferably, the activity repeatability features, activity trend features and basic feature sets are integrated into an input feature matrix, which summarizes the multi-dimensional data of the user and provides comprehensive input information for the scenario prediction of the perception layer; the prediction function of the perception layer (such as machine learning models, deep learning networks, etc.) is activated, and the input feature matrix is ​​used to predict scenario labels, predict possible scenario categories (such as rest, work, entertainment, etc.), and generate a prediction confidence score for each scenario label, which reflects the prediction accuracy of each scenario label; score screening is performed based on the prediction confidence score; scenario labels with higher confidence scores are selected as the final perception results.

[0048] Furthermore, the emotion recognition network is used to predict emotion recognition based on user activity data and environmental data, and the recognition results are established, which also includes:

[0049] Perform sentiment analysis as follows: ;in, Represents the emotional fluctuation results, N is the total number of emotional data points in the lookback window, Represents the emotional data value collected at the i-th time point, is the average value of the sentiment data; perform sentiment change rate analysis as follows: ; Among them, Emotion Rate represents the rate of change of emotion, j represents the time point, Represents the emotional data value collected at the j+1th time point, Represent the emotional data value collected at the jth time point; perform emotion recognition prediction based on the emotion fluctuation results and emotion change rate, and establish the recognition result.

[0050] Specifically, the sentiment fluctuation result is obtained by calculating the fluctuation degree of sentiment data within the lookback window. The specific formula is as follows: ;in, Characterizes the emotional fluctuation results, representing the degree of emotional fluctuation of users in a specific time period. N is the total number of emotional data points in the lookback window. Represents the emotional data value collected at the i-th time point, is the average value of the sentiment data; further, by calculating the change rate of the sentiment data, the sentiment change rate is obtained, and the formula is as follows: ; Among them, Emotion Rate represents the rate of change of emotion, reflecting the fluctuation frequency of user emotion, j represents the time point, Represents the emotional data value collected at the j+1th time point, Represent the emotional data value collected at the jth time point; use the emotional fluctuation results and the emotional change rate as input, integrate them into the emotion recognition network, use the algorithms in the emotion recognition network (such as machine learning, deep learning, etc.) combined with user activity data and environmental data to perform emotion recognition prediction and establish recognition results.

[0051] Based on the perception and recognition fusion results, the local area network is activated, the joint call of the smart home is executed, and the response interaction of the smart home is generated.

[0052] According to the perception recognition fusion results, relevant smart home devices in the local area network are activated; the functions of multiple devices are integrated in the local area network to perform joint calls. For example, according to the user's emotional state and current situation, lighting equipment, temperature control equipment and audio equipment can be called at the same time for coordinated control and adjustment. Joint calls can achieve the collaborative work of multiple devices to ensure that the response is more in line with the user's current needs. The response interaction generated by the joint call provides users with a set of operations that match the current situation. For example, when the user is in a relaxed state, the lights may be dimmed, the volume may be lowered, and the temperature may be adjusted to make the environment more comfortable; when the user is nervous or in urgent need of help, a quick response mode may be triggered to provide support to the user.

[0053] Furthermore, the method further comprises:

[0054] Perform confidence verification on the perception and recognition fusion result; if the perception and recognition fusion result is within the confidence interval, establish a combined linkage selection based on the perception and recognition fusion result; use the combined linkage selection as the display result, activate the local area network to execute the display, and generate a smart home response interaction after screening the combined linkage selection based on user feedback.

[0055] Preferably, the generated perception recognition fusion result is confidence verified to determine whether the credibility of the result is within a preset confidence interval to avoid false triggering of irrelevant or unsuitable devices for user needs; if the verification result shows that the perception recognition fusion result is within the confidence interval, a combined linkage selection is generated based on the result, and the selection scheme includes the joint operation of multiple devices to ensure that the response can cover the user's multi-faceted needs, for example, the light brightness, temperature and volume may be adjusted at the same time to meet the current situation and the user's emotional state. The combined linkage selection scheme is activated and displayed to the user in the local area network as a display result, for example, the user is notified of possible situational response schemes through a smart home application or voice prompt. The combined linkage selection is screened according to user feedback (including direct acceptance, fine-tuning or complete change suggestions) to determine the final response interaction scheme.

[0056] In summary, the embodiments of the present application have at least the following technical effects:

[0057] First, a local area network of a smart home is established, and a local user database is created in the local area network. The local user database is constructed by collecting historical behavior data of users. Then, the collaborative call association of the local area network is established with the local user data, and the local user data is encrypted and sent to the trusted third party to obtain a trusted third-party receipt. The trusted third-party receipt includes a predicted collaborative call association, and the connection strength of the local area network is reconstructed based on the collaborative call association and the predicted collaborative call association. When a user's trigger instruction is received at any time node, the backtracking window is activated, and the user activity data and environmental data are called with the backtracking window. The local server is used to perform situational perception and emotion recognition of the user activity data and environmental data, and establish a perception recognition fusion result. Finally, the local area network is activated based on the perception recognition fusion result, and the joint call of the smart home is executed to generate the response interaction of the smart home. The technical problem that the smart home lacks response to the personalized needs of users and has a slow response speed in the prior art is solved, and the technical effect of improving the response speed of the smart home and enhancing the user experience is achieved.

[0058] Embodiment 2, based on the same inventive concept as the human-computer interaction method for smart home in the above embodiment, Figure 2 As shown, the present application provides a human-computer interaction device for smart home, wherein the device includes:

[0059] A database construction module 11, the database construction module 11 is used to establish a local area network of the smart home and create a local user database in the local area network, and the local user database is constructed by collecting historical behavior data of the user; a perception recognition module 12, the perception recognition module 12 is used to activate the backtracking window when receiving a trigger instruction from the user at any time node, call the user activity data and environmental data with the backtracking window, use the local server to perform situational perception and emotion recognition of the user activity data and environmental data, and establish a perception recognition fusion result, wherein the backtracking window refers to the time range for determining the required user activity data and environmental data; a response interaction module 13, the response interaction module 13 is used to activate the local area network based on the perception recognition fusion result, execute the joint call of the smart home, and generate a response interaction of the smart home.

[0060] Furthermore, the perception and recognition module 12 is used to perform the following method:

[0061] Based on the trigger instruction, the user's real-time activity recognition is performed to establish the real-time activity recognition result; the window length of the backtracking window is configured using the real-time activity recognition result, and the backtracking window is activated based on the configuration result, and the user activity data and environmental data are called through the backtracking window; the context perception network of the local server is called to perform context perception prediction based on the user activity data and environmental data through the context perception network to establish the perception result; the emotion recognition network of the local server is called to perform emotion recognition prediction based on the user activity data and environmental data through the emotion recognition network to establish the recognition result; the perception result and the recognition result are weightedly authenticated and fused to establish the perception and recognition fusion result.

[0062] Furthermore, the perception and recognition module 12 is used to perform the following method:

[0063] The feature extraction layer is used to extract basic data from the user activity data and environmental data to establish a basic feature set, wherein the basic data includes location data, temperature and humidity data, light data, and noise data; the feature extraction layer is used to extract activity repeatability features from the user activity data to establish activity repeatability features, and activity trend evaluation of the user activity data is performed to establish activity trend features; the activity repeatability features, activity trend features, and basic feature sets are sent to the perception layer for scenario perception prediction to establish perception results.

[0064] Furthermore, the perception and recognition module 12 is used to perform the following method:

[0065] The activity repeatability features, activity trend features, and basic feature sets are integrated into the input feature matrix, and the prediction function of the perception layer is activated to perform scenario label prediction and establish a prediction confidence score. The prediction confidence score is used for score screening to establish the perception result.

[0066] Furthermore, the perception and recognition module 12 is used to perform the following method:

[0067] Perform sentiment analysis as follows: ;in, Represents the emotional fluctuation results, N is the total number of emotional data points in the lookback window, Represents the emotional data value collected at the i-th time point, is the average value of the sentiment data; perform sentiment change rate analysis as follows: ; Among them, Emotion Rate represents the rate of change of emotion, j represents the time point, Represents the emotional data value collected at the j+1th time point, Represent the emotional data value collected at the jth time point; perform emotion recognition prediction based on the emotion fluctuation results and emotion change rate, and establish the recognition result.

[0068] Furthermore, the perception and recognition module 12 is used to perform the following method:

[0069] Perform confidence verification on the perception and recognition fusion result; if the perception and recognition fusion result is within the confidence interval, establish a combined linkage selection based on the perception and recognition fusion result; use the combined linkage selection as the display result, activate the local area network to execute the display, and generate a smart home response interaction after screening the combined linkage selection based on user feedback.

[0070] Furthermore, the response interaction module 13 is used to execute the following method:

[0071] A collaborative call association of the local area network is established with the local user data, and the local user data is encrypted and sent to a trusted third party to obtain a trusted third party receipt. The trusted third party receipt includes a predicted collaborative call association, and the connection strength of the local area network is reconstructed based on the collaborative call association and the predicted collaborative call association.

[0072] Furthermore, the response interaction module 13 is used to execute the following method:

[0073] Establish data classification for stored data, wherein the data classification includes direct data classification and analytical data classification; divide the direct data classification into basic data and privacy data, and create a first encryption level with the basic data, and create a second encryption level with the privacy data; divide the analytical data classification into direct feature analysis data and indirect feature reasoning data, and create a third encryption level with the direct feature analysis data, and create a fourth encryption level with the indirect feature reasoning data; perform hierarchical encryption management of data within the local area network according to the first encryption level, the second encryption level, the third encryption level, and the fourth encryption level.

[0074] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0075] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0076] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A human-computer interaction method for smart home, characterized in that: The method comprises: Establishing a local area network of the smart home, and creating a local user database in the local area network, wherein the local user database is constructed by collecting historical behavior data of users; When a trigger instruction from a user is received at any time node, a lookback window is activated, user activity data and environmental data are called through the lookback window, and the local server is used to perform situational awareness and emotion recognition of the user activity data and environmental data, and establish a perception and recognition fusion result, wherein the lookback window refers to a time range for determining the required user activity data and environmental data; Based on the perception recognition fusion results, the local area network is activated, the joint call of the smart home is executed, and the response interaction of the smart home is generated; When receiving a user's trigger instruction at any time node, the backtracking window is activated, the user activity data and environmental data are called by the backtracking window, the local server is used to perform situational perception and emotion recognition of the user activity data and environmental data, and the perception recognition fusion result is established, including: Based on the trigger instruction, the user's real-time activity recognition is performed to establish the real-time activity recognition result; Use the real-time activity recognition results to configure the window length of the backtracking window, and activate the backtracking window based on the configuration results to call user activity data and environment data through the backtracking window; Calling the context-aware network of the local server to perform context-aware prediction based on user activity data and environmental data, and establish a perception result; Calling the emotion recognition network of the local server, using the emotion recognition network to perform emotion recognition prediction based on user activity data and environmental data, and establishing a recognition result; Perform weighted authentication fusion on the perception results and recognition results to establish the perception and recognition fusion results; The method of performing situational awareness prediction based on user activity data and environmental data using a situational awareness network to establish a perception result includes: Using the feature extraction layer to extract basic data from the user activity data and environmental data to establish a basic feature set, the basic data includes location data, temperature and humidity data, light data and noise data; Using the feature extraction layer to extract activity repeatability features of user activity data, establish activity repeatability features, perform activity trend evaluation of user activity data, and establish activity trend features; Send activity repetitive features, activity trend features and basic feature sets to the perception layer for situational perception prediction and establish perception results; The emotion recognition network is used to perform emotion recognition prediction based on user activity data and environmental data to establish a recognition result, including: Perform sentiment analysis as follows: ； in, Represents the emotional fluctuation results, N is the total number of emotional data points in the lookback window, Represents the emotional data value collected at the i-th time point, is the average value of sentiment data; Perform sentiment change rate analysis as follows: ； Among them, Emotion Rate represents the rate of change of emotion, j represents the time point, Represents the emotional data value collected at the j+1th time point, Represents the emotional data value collected at the jth time point; Emotion recognition prediction is performed based on emotion fluctuation results and emotion change rate to establish recognition results.

2. The human-computer interaction method for smart home according to claim 1, characterized in that: The method further comprises: A collaborative call association of the local area network is established with the local user data, and the local user data is encrypted and sent to a trusted third party to obtain a trusted third party receipt, wherein the trusted third party receipt includes a predicted collaborative call association, and the connection strength of the local area network is reconstructed based on the collaborative call association and the predicted collaborative call association.

3. The human-computer interaction method for smart home according to claim 1, characterized in that: The method further comprises: The activity repetitive features, activity trend features and basic feature sets are integrated into the input feature matrix, the prediction function of the perception layer is activated to predict the scenario label, and the prediction confidence score is established; Use prediction confidence scores to perform scoring screening and establish perception results.

4. The human-computer interaction method for smart home according to claim 1, characterized in that: The method further comprises: Conduct confidence verification on the perception and recognition fusion results; If the perception recognition fusion result is within the confidence interval, a combined linkage selection is established based on the perception recognition fusion result; The combined linkage selection is used as the display result, the local area network is activated to execute the display, and the combined linkage selection is filtered according to user feedback to generate a response interaction of the smart home.

5. The human-computer interaction method for smart home according to claim 1, characterized in that: The method further comprises: Establishing data classification for stored data, wherein the data classification includes direct data classification and analytical data classification; Classify direct data into basic data and private data, and create a first encryption level with basic data and a second encryption level with private data; Classify the analysis data into direct feature analysis data and indirect feature inference data, and create a third encryption level with the direct feature analysis data and a fourth encryption level with the indirect feature inference data; Hierarchical encryption management of data within the local area network is performed according to the first encryption level, the second encryption level, the third encryption level and the fourth encryption level.

6. A human-computer interaction device for smart home, characterized in that: For implementing the human-computer interaction method for smart home according to any one of claims 1 to 5, the device comprises: A database construction module, the database construction module is used to establish a local area network of the smart home and create a local user database in the local area network, the local user database is constructed by collecting historical behavior data of users; A perception and recognition module, which is used to activate the backtracking window when receiving a user's trigger instruction at any time node, call the user activity data and environmental data with the backtracking window, use the local server to perform situational perception and emotion recognition of the user activity data and environmental data, and establish a perception and recognition fusion result, wherein the backtracking window refers to the time range used to determine the required user activity data and environmental data; a response interaction module, which is used to activate the local area network based on the perception and recognition fusion result, execute the joint call of the smart home, and generate a response interaction of the smart home.

Images