Method and system for constructing linkage control scene of smart home

Abstract

The invention discloses a linkage control scene building method and system for a smart home, and relates to the technical field of smart home control, and the method comprises the steps: predicting a motion track of a user through a user behavior prediction model, and outputting a user behavior prediction result. And setting a scene template, and selecting the most conforming scene configuration from the scene template in combination with the user behavior prediction result and the environmental parameters to obtain a recommended scene configuration scheme. And designing a graphical user interface, viewing the recommended scene configuration scheme by the user through the graphical user interface, performing fine adjustment according to personal preference, and confirming a final scene configuration scheme. And according to the final scene configuration scheme, sending an instruction to the target home equipment, and outputting a home equipment state execution report. According to the invention, the scene configuration scheme is optimized by using the genetic algorithm, the balance between the energy consumption and the user comfort is considered, and the recommended scene configuration scheme is ensured to be economical and environment-friendly.

Description

Technical Field

[0001] The present invention relates to the field of smart home control technology, and in particular to a method and system for building a linkage control scene of a smart home. Background Art

[0002] With the integration of IoT and AI technologies, the linkage control technology of smart homes has gradually evolved from single device automation to multiple scenarios. In the early days, many scenarios of smart home linkage control adopted static control strategies based on rule engines, used user behavior data for modeling and environmental perception, and used time series analysis algorithms to predict user behavior trajectories to optimize scenario configuration parameters.

[0003] There are still many problems with the existing linkage control scene construction methods for smart homes. Conventional linkage control scene configuration methods often adopt a single-objective optimization strategy and lack the ability to solve multiple objectives, which leads to the inability to achieve a dynamic balance between user preferences and energy-saving needs. The user interaction interface is mostly limited to the adjustment of preset options, lacks the intention understanding and multi-round dialogue capabilities of natural language processing, and the real-time feedback efficiency of user personalized needs is low. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for building a linkage control scene for a smart home, which solves the problems of insufficient dynamic optimization capability of multi-target scene configuration and poor efficiency of personalized adaptation of the interactive interface in the existing method for building a linkage control scene for a smart home.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a method for building a linkage control scene of a smart home, which includes collecting environmental parameters, selecting a deep learning model, and training it into a user behavior prediction model; Use the user behavior prediction model to predict the user's activity trajectory and output the user behavior prediction results; Set the scenario template, combine the user behavior prediction results and environmental parameters, select the most suitable scenario configuration from the scenario template, and obtain the recommended scenario configuration solution; Design a graphical user interface, through which users can view recommended scenario configuration solutions, make fine adjustments based on their personal preferences, and confirm the final scenario configuration solution; According to the final scenario configuration plan, send instructions to the target home device and output the home device status execution report.

[0007] As a preferred solution of the method for building a linkage control scene of a smart home described in the present invention, the environmental parameters include temperature and humidity data, light intensity, air quality, noise level, door and window status, and power consumption.

[0008] As a preferred solution of the method for building a linkage control scene of a smart home described in the present invention, a deep learning model is selected and trained as a user behavior prediction model, including the following steps: Collect user behavior data and user historical behavior data, use the user historical behavior data as a training set, and pre-process the user behavior data and environmental parameters; Select LSTM as the deep learning model and use the training set to train LSTM as a user behavior prediction model.

[0009] As a preferred solution of the method for building a linkage control scene of a smart home according to the present invention, the user's activity trajectory is predicted using a user behavior prediction model, and the user behavior prediction result is output, including the following steps: The preprocessed user behavior data and environmental parameters are combined into a user comprehensive feature vector and input into the user behavior prediction model to obtain a preliminary prediction result; The preliminary prediction results are logically verified and adjusted to obtain the final user behavior prediction results.

[0010] As a preferred solution of the linkage control scene construction method of the smart home described in the present invention, wherein: setting a scene template, combining the user behavior prediction results and environmental parameters, selecting the most suitable scene configuration from the scene template, and obtaining a recommended scene configuration solution, includes the following steps: By setting the scene template and reading the latest environmental parameters of the user's home, a preliminary scene template is determined, and a preliminary scene configuration is selected according to the preliminary scene template to obtain a preliminary scene configuration plan; A genetic algorithm is used to perform selection, cross-breeding, and mutation operations on the preliminary scenario configuration schemes and iterate them to output a recommended scenario configuration scheme.

[0011] As a preferred solution of the method for building a linkage control scene of a smart home according to the present invention, a graphical user interface is designed, and a user views a recommended scene configuration scheme through the graphical user interface, and makes fine adjustments according to personal preferences, and confirms the final scene configuration scheme, including the following steps: Conduct user surveys, determine the core requirements of the graphical user interface based on the survey results, draw sketches of the graphical user interface, and convert the sketches into interactive prototypes; Invite users to conduct participatory testing on the interactive prototype, adjust the interactive prototype, integrate the voice recognition engine on the adjusted interactive prototype and perform voice command parsing, and increase the context management capability of the adjusted interactive prototype to form a complete graphical user interface; The user views the recommended scenario configuration plan through the graphical user interface and confirms the final scenario configuration plan.

[0012] As a preferred solution of the method for building a linkage control scene of a smart home described in the present invention, the final scene configuration plan is converted into a smart home device operation instruction list and sent to the corresponding target home device. When the target home device receives the operation instruction, the status display of the target home device is updated and a home device status execution report is output.

[0013] In a second aspect, the present invention provides a linkage control scene building system for a smart home, including a model training module, collecting environmental parameters, selecting a deep learning model, and training it into a user behavior prediction model; The result prediction module uses the user behavior prediction model to predict the user's activity trajectory and output the user behavior prediction result; The output solution module sets the scenario template, combines the user behavior prediction results and environmental parameters, selects the most suitable scenario configuration from the scenario template, and obtains the recommended scenario configuration solution; The scheme adjustment module designs a graphical user interface through which users can view the recommended scenario configuration schemes, make fine adjustments based on their personal preferences, and confirm the final scenario configuration scheme; The report generation module sends instructions to the target home devices according to the final scenario configuration plan and outputs the home device status execution report.

[0014] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for building a linkage control scene of a smart home as described in the first aspect of the present invention is implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the method for building a linkage control scene for a smart home as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are as follows: setting a scene template, combining the user behavior prediction results and environmental parameters, selecting the most suitable scene configuration from the scene template, and obtaining a recommended scene configuration plan. According to the final scene configuration plan, send instructions to the target home device, and output a home device status execution report. The function of automatically adapting the smart home linkage control scene to user needs is realized, and the scene configuration plan is optimized by using a genetic algorithm, taking into account the balance between energy consumption and user comfort, ensuring that the recommended scene configuration plan is both economical and environmentally friendly. It not only improves the intelligence level of smart home devices, but also effectively reduces energy consumption, achieving the effect of improving the comfort of resident users while reducing energy consumption. The graphical user interface integrates natural language processing technology, which greatly simplifies the user's operation process and meets the user's personalized needs through participatory testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 This is a flow chart of the method for building a smart home linkage control scene in Example 1.

[0019] Figure 2 A system structure diagram is built for the smart home linkage control scenario in Example 1.

[0020] Figure 3 This is a flow chart of the genetic algorithm optimization scenario configuration solution in Example 1.

[0021] Figure 4 This is a schematic diagram of the graphical user interface interaction prototype in Example 1. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0025] Example 1, reference Figure 1~Figure 4 , which is the first embodiment of the present invention, and provides a method for building a linkage control scene of a smart home, comprising the following steps: S1. Collect environmental parameters, select a deep learning model, and train it into a user behavior prediction model.

[0026] The following steps are included: S1.1. First, user behavior data and environmental parameters need to be collected. Specifically, user behavior data includes device usage records (such as operation records for adjusting air conditioning temperature), timestamps (i.e., the specific time point when the behavior event occurred), and voice command records (control instructions and execution status issued by users through voice assistants, such as turning on the toilet light). Environmental parameters include temperature and humidity data, light intensity, air quality, noise level, door and window status, and power consumption. User behavior data and environmental parameters are uploaded to the cloud server through the Zigbee wireless communication protocol. The cloud server removes noise from user behavior data and environmental parameters by using a sliding window filtering algorithm. For example, within a 5-minute time window, the light intensity reading is [800, 795, 805, 10000, 810] lux, where 10000 lux is an outlier (i.e., an extremely high reading). The sliding window filter calculates the average value within these five minutes, i.e., (800+795+805+810)÷4=802.5, and replaces 10000 lux with the average value of 802.5 lux.

[0027] Linear interpolation is then used to fill in missing values ​​in user behavior data and environmental parameters. For example, if the temperature at 14:00 and 14:02 is 22°C and 23°C respectively, but there is no temperature reading at 14:01, the temperature at 14:01 can be calculated as (22+23)÷2=22.5°C using linear interpolation.

[0028] S1.2. Collect the user's historical behavior data and use it as a training set. The user's historical behavior data includes wake-up time, sleep time, return home time, leaving home time, past device usage records, preference settings (i.e., user environmental preference settings for different scenarios, such as light brightness, background music type, etc.) and special event records (such as changes in operating habits when guests visit, adjustments to behavior patterns on holidays, etc.). Then divide the preprocessed user behavior data and environmental parameters according to a fixed time window (such as a week), and select basic features (wake-up time, return home time), statistical features (standard deviation of wake-up time, average return home time), periodic features (such as day of the week, whether it is a holiday, etc.) and environmental parameter features (such as temperature, humidity, whether to return home earlier in cold weather, etc.) from each time window as core feature points; S1.3. Based on the core feature points, select LSTM (Long Short-Term Memory Network) as the core architecture and use the training set to train it as a user behavior prediction model. Specifically, first set a set of initial hyperparameters for LSTM, including learning rate (0.001), batch size (64) and number of iterations (100), use Xavier initialization method to initialize all weights in LSTM, and set the bias item in LSTM to 0, and then use the training set to train LSTM. The specific training process is: extract a part of the data from the training set according to the set batch size as the current batch. For example, if there are 1,000 records in total, the first batch contains records 1 to 64, the second batch contains records 65 to 128, and so on. Input the data of the current batch into LSTM, first enter the network through the input layer, and then pass through each layer of LSTM units in turn, calculate the output value of each layer one by one, and when the data of the current batch reaches the last fully connected layer of LSTM, convert the output of the LSTM layer into the final behavior prediction result. For example, for the task of predicting the user's wake-up time, the output is a specific wake-up time point or a probability distribution. The behavior prediction result is compared with the actual label (that is, the real user historical behavior data), and the mean square error is used to calculate the loss function value between the behavior prediction result and the actual label. According to the loss function value, the chain rule is used to calculate the gradient of the loss function with respect to each layer of LSTM weights (that is, back propagation), and then the stochastic gradient descent SGD algorithm is used to update all weights in LSTM (that is, gradually improve the prediction ability of LSTM). For each batch of data, the output behavior prediction result, the calculation of the loss function value, and the back propagation are repeated until all batches of data are processed or the preset number of iterations are reached, forming a user behavior prediction model.

[0029] S2. Use the user behavior prediction model to predict the user's activity trajectory and output the user behavior prediction results.

[0030] The following steps are included: S2.1. Combine the pre-processed user behavior data and environmental parameters into a user comprehensive feature vector (unify the current ambient temperature into Celsius units, the light intensity into lux, convert the timestamp into the time difference from the start of the calculation at a fixed time point, use standardization and convert the unified user behavior data and environmental parameters into a normal distribution with a mean of 0 and a standard deviation of 1 to form a user comprehensive feature vector). This comprehensive vector contains multiple feature dimensions, such as timestamp, user current location, current ambient temperature, light intensity, etc. For example, assuming the current time is 15:43, the user's current location is in the study, the indoor temperature is 24°C, and the light intensity is 500lux, then the user comprehensive feature vector is expressed as [15:43, study, 24, 500].

[0031] S2.2. Input the user comprehensive feature vector into the user behavior prediction model. The user behavior prediction model will calculate the output value of each layer through each layer of LSTM units, and finally obtain a preliminary prediction result of user behavior in the future (this process involves forward propagation of a multi-layer neural network, and the output of each layer is used as the input of the next layer until the last layer outputs a preliminary prediction result).

[0032] S2.3. Based on the preliminary prediction results of user behavior in the future (this preliminary prediction result can be understood as one or more numerical values, representing the probability of occurrence of different behavioral events or specific numerical predictions. For example, the user behavior prediction model may output that the probability of the user going home in 15 minutes is 85%, or it is expected that the user will enter the living room at 20:11), further determine the specific behavior that the user may perform next and the user's operation needs. For example, if it is predicted that the user will go home and enter the living room in 15 minutes, it can be inferred that the user will turn on the lights in the living room and adjust the air-conditioning temperature to a comfortable range to a large extent. In addition, more detailed predictions can be made in combination with the current environmental parameters. For example, if the current indoor temperature is low, it is recommended to set the air-conditioning temperature to 22°C. If the current light is dim, it is recommended to adjust the light brightness to a higher mode.

[0033] S2.4. Next, we need to perform logical verification on the preliminary prediction results to ensure the rationality of the prediction of the user behavior prediction model. For example, if the user behavior prediction model predicts that the user will go home at 3 a.m., but the user's historical behavior data shows that the user has never gone home during this time period, the prediction may be considered unreasonable.

[0034] S2.5. Adjust unreasonable prediction results. Adjustments are made based on user historical behavior data, environmental parameters, and common sense. Adjustments are made based on user historical behavior data. Suppose the user behavior prediction model predicts that the user will frequently go in and out of the kitchen late at night, while the user's historical behavior data shows that the user usually only enters the kitchen in the morning and dinner time. In this specific case, the predicted frequency of entering and leaving the kitchen is adjusted from once every 5 minutes to once every morning and dinner time, or if the user is asleep during this late night time period, the predicted time period can be adjusted to a more reasonable time, such as activities before and after getting up in the morning.

[0035] S2.6. Sometimes, the prediction results may seem unreasonable, but in fact, it may be because the current environmental parameters affect the user's habits. For example, suppose the user behavior model predicts that the user will frequently adjust the air-conditioning temperature in cold weather, but the user's historical behavior data shows that the user hardly adjusts the temperature frequently. This can solve the problem of deviation in the prediction results by appropriately increasing the predicted adjustment frequency (that is, considering that the current outdoor temperature is low, the user may adjust the indoor temperature more frequently than usual to maintain comfort) or the current indoor temperature is close to the user's comfort range (such as 24°C), and the specific operation requirements can also be adjusted, such as suggesting that the user only slightly adjust the temperature (such as from 24°C to 22.5°C) instead of a large adjustment.

[0036] S2.7. When the deviation in the prediction results is not related to the user's historical behavior data, it is necessary to adjust the prediction results through common sense and preset rules. For example, if the user behavior prediction model predicts that the user will stay in a specific area of ​​the home (such as a storage room) for a long time, but common sense shows that a specific area is usually not a place where users stay for a long time, when this specific situation occurs, the prediction results can be adjusted by shortening the user's stay time (adjusting the predicted stay time from a longer time (such as 30 minutes) to a shorter time (such as 5 minutes) to match the actual situation) or correcting the user's behavior type (if the user behavior prediction model predicts that the user will read in the storage room, but common sense shows that the storage room is not a suitable place for reading. Generally, most of the scenes of reading are in the living room, bedroom and study, so the user's behavior type can be adjusted to a more reasonable option, such as folding clothes or sorting out sundries).

[0037] S2.8. Summarize the prediction results based on the user's historical behavior data, environmental parameters, and common sense to form the final user behavior prediction result (for example, the initial prediction result is that the user enters and exits the kitchen once every 5 minutes, which is adjusted to the user entering the kitchen once every morning and dinner time).

[0038] S3. Set the scenario template, combine the user behavior prediction results and environmental parameters, select the most suitable scenario configuration from the scenario template, and obtain the recommended scenario configuration solution.

[0039] The following steps are included: S3.1. First, you need to set up scene templates (based on the user's specific needs and living habits). Scene templates include the user's home scene (activated when the user is about to arrive home, including turning on the living room lights, adjusting the air conditioning temperature to a comfortable range, playing background music, etc.), the sleep scene (activated before the user is ready to go to bed, including dimming the bedroom lights, closing the curtains, adjusting the air conditioning to the user's desired sleeping temperature, and starting the night mode, such as silencing mobile phone notifications) and the leaving home scene (activated when the user leaves home to ensure safety and energy saving, including turning off all unnecessary electrical appliances, closing the curtains, etc.), and each scene contains a series of specific device operation instructions to achieve automatic control. For example, the home scene includes the following operation instructions: turn on the living room lights (brightness set to 60%, color warm yellow), adjust the air conditioning temperature to 22°C, start the air purifier, and play classical background music through the smart speaker.

[0040] S3.2. Use sensors to read the latest environmental parameters from the user's home. For example, the temperature in the living room is 18°C, the humidity is 50%, and the light intensity is 300lux, and the temperature in the bedroom is 20°C and the humidity is 55%. Evaluate the overall state of the current environment in the user's home based on the latest environmental parameters. Specifically, for example, if the current indoor temperature is low (18°C) and the outdoor temperature is lower, this means that the indoor temperature needs to be increased to ensure comfort. At the same time, if the current light is weak (300lux), the light brightness needs to be increased. After the evaluation is completed, compare the latest environmental parameters with the historical environmental parameters (i.e., the common indoor temperature in the same time period in the past, such as 22°C) to identify whether the current environmental state deviates from the normal range. For example, if the indoor temperature should usually be around 22°C in the same time period, but the current indoor temperature is only 18°C, it means that the current environmental state deviates from the normal range and the indoor temperature needs to be adjusted to 22°C.

[0041] S3.3. According to the user's behavior prediction results and comparison results, analyze the specific needs of the user's required scene and determine a preliminary scene template, and select the preliminary scene configuration based on the preliminary scene template. Specifically, if the user's behavior prediction results show that the user will go home in 15 minutes and is most likely to enter the living room first, it can be inferred that the user may want to feel a warm and comfortable atmosphere as soon as he enters the door. Therefore, it is necessary to prepare the scene template of the home scene in advance (because the home scene includes functions such as raising the indoor temperature and turning on the lights). If there are multiple scene templates that can meet the current needs of the user's required scene, they are sorted according to priority. For example, if the current indoor temperature is low and the user is about to go home, the home scene is given priority over the sleep scene or the leaving home scene.

[0042] S3.4. While meeting the current needs of the user's required scenarios, other goals also need to be considered, such as the balance between energy saving (equipment energy efficiency) and user comfort. Genetic algorithms are needed to optimize the preliminary recommended scenario configuration scheme. Specifically, it is first necessary to define an objective function based on user comfort and equipment energy efficiency (and define the weights of user comfort and equipment energy efficiency based on user needs), and use a genetic algorithm to select an appropriate population size (such as 100 individuals) to generate a set of initial solutions (each solution represents a possible recommended scenario configuration scheme, that is, the initial population. For example, a solution may mean setting the living room temperature to 21°C, the light brightness to 60%, and the background music volume to 30%). For each individual in the population (that is, the initial solution), the fitness score of each individual is calculated according to the objective function (user comfort and equipment energy efficiency are multiplied by their respective weights and added together to obtain the result). According to the fitness score of each individual, a selection probability is assigned to each individual using the roulette wheel selection method (if there are five individuals, that is, individual A, individual B, individual C, Individual D and individual E have fitness scores of 69.96, 57.44, 63.39, 76.68, and 91.52, respectively, so the total fitness score is 358.99. Use the fitness score of each individual to divide the total fitness score, so that the selection probability of each individual is obtained, such as the selection probability of individual D is 0.2135), set the fitness score threshold (this fitness score threshold needs to be set according to the specific scenario required by the user, here the top 30% of individuals are selected as the specific numerical range of the fitness score threshold), and the individuals exceeding the fitness score threshold are regarded as individuals with higher fitness scores and enter the next generation population (the five individuals are arranged in order from large to small, and the arrangement order is individual E, individual D, individual A, individual C, and individual B. Since the top 30% of individuals are selected, individuals E and D are individuals with relatively higher fitness scores).

[0043] S3.5. In the next generation population (i.e., the parent population), determine all the parameters contained in each individual (for example, an individual may include multiple parameters such as temperature setting, light brightness, background music volume, and each parameter can be regarded as part of the genetic gene). At the same time, in order to ensure the diversity of solutions, randomly select one or more intersections. Assuming that the parent solution consists of five parameters, the second and fourth parameters can be selected as intersections. For example, for the parameter list [temperature, light brightness, background music volume, air purifier status, curtain status], you can choose to cross between light brightness and air purifier status. Disconnect the two selected parent solutions at the intersection and exchange some information. For example, the parameters of parent A are [21°C, 60%, 30%, on, off], and the parameters of parent B are [23°C, 50%, 40%, off, on]. If you choose to crossover between light brightness and air purifier status, the new parent solutions generated are [21°C, 50%, 40%, on, off] and [23°C, 60%, 30%, off, on].

[0044] The next step is to randomly select one or several solutions from the generated new parent solutions for mutation. The selection of mutation is completely random, but it can also be selected according to certain strategies, such as giving priority to solutions with lower fitness scores for mutation to increase the chance of finding a better solution. Then set the mutation probability (5%) and select the appropriate mutation method according to the specific application scenario. Common mutation methods include fine-tuning the value, replacing it with another value, and randomly resetting it. For example, in temperature control, the temperature setting of the selected solution can be fine-tuned, such as changing 21°C to 20.8°C, or completely randomly resetting the temperature setting to a new value (such as 19°C). At the same time, when selecting the mutation method, in order to improve the effectiveness of the mutation, a local search can be performed during the mutation. For example, if the current temperature is set to 21°C, a local search can be performed by randomly adjusting the temperature value in the range of [-1°C, +1°C].

[0045] Finally, the selected solution is mutated according to the selected mutation method. Specifically, for example, the solution of the parent generation A after the crossover operation is [21°C, 50%, 40%, on, off]. After mutation, the temperature setting is changed to 20.8°C, while other parameters remain unchanged, and the mutated solution is [20.8°C, 50%, 40%, on, off] (the purpose of mutation is to introduce enough diversity to avoid premature convergence to the local optimal solution, but not to mutate too much and lose the information of the high-quality solution. For example, the temperature variation range should be controlled within a reasonable range (such as ±1°C), rather than large fluctuations (such as ±10°C)).

[0046] S3.6. Repeat the selection, cross-breeding and mutation operations and iterate until each individual reaches convergence, and the individual with the highest fitness score is obtained (the individual with the highest fitness score is used as the final recommended scenario configuration plan. For example, after multiple iterations, it is found that the individual with the highest fitness score sets the temperature to 21°C, the light brightness to 65%, and the background music volume to 35%, achieving the best balance between user comfort and energy saving).

[0047] S4. Design a graphical user interface, through which users can view recommended scenario configuration solutions, make fine adjustments based on personal preferences, and confirm the final scenario configuration solution.

[0048] The following steps are included: S4.1. First, conduct user surveys to understand the target users' usage habits, preferences, and pain points for smart home devices. Information can be collected through questionnaires, user interviews, etc. For example, ask users about the most common problems they encounter when controlling smart home devices (such as users find that smart bulbs or thermostats occasionally disconnect, making it impossible to control them through mobile phones), or what features users would like to see (for example, many users hope to be able to control home devices using natural language commands, such as Alexa (the name of the voice assistant), to brighten the living room lights).

[0049] S4.2. Based on the survey results, clarify the core requirements that the graphical user interface (GUI) needs to meet (such as easy-to-operate device control, intuitive scene display, personalized interface settings, etc.) and start drawing a sketch of the GUI. The sketch needs to cover all major functional areas, such as the device control panel, scene configuration options, and the display area for recommended scene configuration solutions, and the function and layout of each area should be reflected in the sketch. For example, the device control panel can adopt a card-style design, with each card displaying the status of a device and basic control buttons (such as switches and brightness adjustments). After the sketch design is completed, use Figma's prototyping tool to convert the sketch into an interactive prototype. The prototype should simulate the appearance and interactive experience of the final product to the greatest extent possible. For example, when the user clicks on the light card, a small window containing brightness adjustment and color selector should pop up. When the user drags the slider to adjust the light brightness, the light icon on the GUI interface should also be updated in real time to reflect the current setting operation.

[0050] S4.3. Invite some users to conduct participatory testing on the interactive prototype. Let users try to use the interactive prototype to complete a series of tasks (such as turning on the lights in the living room and adjusting the air conditioning temperature). During the user test, observe the user's operation process, record the difficulties they encounter during use, and form the prototype usability test results. According to the prototype usability test results, adjust and optimize the prototype. Specifically, for example, when it is found that users often cannot find a certain function button, consider rearranging the position of the function button or integrating related functions into a more conspicuous place (assuming that in the usability test, it is found that users often have difficulty finding the slider for adjusting the light brightness, and the slider was originally placed at the bottom of each room device card without obvious identification. The light brightness slider can be moved to the top of the device card and right below the light bulb icon so that users can see it at a glance. In addition, a brightness icon such as a sun icon is added next to the light brightness slider to more intuitively prompt users to the function of the light brightness slider. If there are other light-related control options (such as color selectors), they can also be placed in the same area as the light brightness slider to form a light control area, making it easier for users to find and operate all light-related settings).

[0051] S4.4. Add convenient functions for adding and deleting smart home devices to the adjusted prototype. Users only need to scan the smart home QR code or enter the smart home ID according to the prompts to quickly add new devices to the adjusted prototype. Similarly, for smart home devices that are no longer needed, users can directly remove them from the adjusted prototype.

[0052] S4.5. Select Amazon Transcribe as the integrated speech recognition engine and integrate it into the backend service of the adjusted prototype. Then create a vocabulary containing all the user's voice commands and corresponding operations (for example, turn on the kitchen light, increase the bedroom temperature by 1°C, play a certain song by a certain singer, etc.), and use natural language processing technology (NLP) to understand the intention of the user's voice command, recognize the voice into text, and convert it into specific operation instructions. For example, when the user says to turn up the kitchen light, the adjusted prototype should not only recognize the two words "living room" and "light", but also understand the meaning of "brighten" and convert it into an operation to increase the brightness.

[0053] S4.6. On the basis that the adjusted prototype realizes voice command parsing, the adjusted prototype needs to have the context management capability to support multiple rounds of dialogue. It can be understood as letting the adjusted prototype remember the previous conversation with the user and continue the conversation. For example, if the user first asks which room the lights are on, and then says to turn off the lights in this room, the adjusted prototype needs to know that the lights in this room refer to the lights in the room mentioned before. When the user's initial command is not clear enough, the adjusted prototype can obtain further information by asking questions. For example, if the user says to turn up the temperature, the adjusted prototype will ask which room you want to turn up the temperature.

[0054] S4.7. The user views the recommended scene configuration scheme in the designed GUI and makes fine adjustments according to personal preferences (for example, if the user does not like the recommended jazz-type background music, he or she can directly select a music list of his or her favorite on the GUI interface, such as rock background music). When the user completes the adjustment, all adjustment operations performed by the user will be immediately reflected in the smart home device operation instruction list of the GUI. The GUI will pop up a confirmation window, listing all the operations to be performed and waiting for the user to confirm. After the user confirms, the final scene configuration scheme is formed.

[0055] S5. According to the final scenario configuration plan, send instructions to the target home device and output the home device status execution report.

[0056] The following steps are included: S5.1. Convert the final scenario configuration plan into a detailed list of smart home device operation instructions. For example, for the home scene, the generated instructions may be device ID: living room smart light, operation type: turn on, brightness 70%, color warm white; device ID: living room air conditioner, operation type: adjust temperature, parameter value: 23°C.

[0057] S5.2. Send the list of smart home device operation instructions to the corresponding smart home device through the Z-Wave wireless communication protocol. When the smart home device receives the instruction, it will immediately perform the corresponding operation. For example, after the living room smart light is successfully turned on, a confirmation message will be returned, indicating that the indicator light has been turned on according to the specified parameters (brightness 70%, color warm white). According to the corresponding operation performed by the smart home device, the smart home device status display is updated in real time. For example, the status of the living room smart light is updated on the GUI to be turned on, and the brightness and color of the current light setting are displayed. If a smart home device fails to successfully execute the instruction (such as the status of the smart home device is not updated), the error detection mechanism should be activated to try to resend the instruction. For example, if the temperature adjustment instruction sent to the air conditioner fails for the first time, the GUI will try to send it again after a certain time interval (such as five minutes). When the instruction cannot be successfully executed after multiple retries, the GUI will push notifications or voice assistants on the mobile phone App to remind the user that there is a problem with the device and provide solution suggestions. For example, if your living room air conditioner fails to adjust the temperature as set, please check whether the smart home device is working properly.

[0058] S5.3. Integrate the sending result and execution result of each GUI command into JSON format to form a home device status execution report.

[0059] This embodiment also provides a linkage control scene building system for a smart home, including: a model training module, collecting environmental parameters, selecting a deep learning model, and training it into a user behavior prediction model; The result prediction module uses the user behavior prediction model to predict the user's activity trajectory and output the user behavior prediction result; The output solution module sets the scenario template, combines the user behavior prediction results and environmental parameters, selects the most suitable scenario configuration from the scenario template, and obtains the recommended scenario configuration solution; The scheme adjustment module designs a graphical user interface through which users can view the recommended scenario configuration schemes, make fine adjustments based on their personal preferences, and confirm the final scenario configuration scheme; The report generation module sends instructions to the target home devices according to the final scenario configuration plan and outputs the home device status execution report.

[0060] This embodiment also provides a computer device, which is suitable for the method of building a linkage control scene for a smart home, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the method of building a linkage control scene for a smart home proposed in the above embodiment.

[0061] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0062] The present embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for building a linkage control scene of a smart home proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0063] In summary, the present invention obtains a recommended scene configuration scheme by: setting a scene template, combining the user behavior prediction results and environmental parameters, selecting the most suitable scene configuration from the scene template. According to the final scene configuration scheme, send instructions to the target home device, and output the home device status execution report. The function of automatically adapting the smart home linkage control scene to user needs is realized, and the scene configuration scheme is optimized by genetic algorithm, which takes into account the balance between energy consumption and user comfort, ensuring that the recommended scene configuration scheme is both economical and environmentally friendly. It not only improves the intelligence level of smart home devices, but also effectively reduces energy consumption, achieving the effect of improving the comfort of resident users while reducing energy consumption. The graphical user interface integrates natural language processing technology, which greatly simplifies the user's operation process and meets the user's personalized needs through participatory testing.

[0064] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for building a linkage control scene of a smart home, characterized in that: include, Collect environmental parameters, select a deep learning model, and train it into a user behavior prediction model; Use the user behavior prediction model to predict the user's activity trajectory and output the user behavior prediction results; Set the scenario template, combine the user behavior prediction results and environmental parameters, select the most suitable scenario configuration from the scenario template, and obtain the recommended scenario configuration solution; Design a graphical user interface, through which users can view recommended scenario configuration solutions, make fine adjustments based on their personal preferences, and confirm the final scenario configuration solution; According to the final scenario configuration plan, send instructions to the target home device and output the home device status execution report.

2. The method for building a linkage control scene of a smart home as claimed in claim 1, characterized in that: The environmental parameters include temperature and humidity data, light intensity, air quality, noise level, door and window status, and power consumption.

3. The method for building a linkage control scene of a smart home as claimed in claim 2, characterized in that: Select a deep learning model and train it as a user behavior prediction model, including the following steps: Collect user behavior data and user historical behavior data, use the user historical behavior data as a training set, and pre-process the user behavior data and environmental parameters; Select LSTM as the deep learning model and use the training set to train LSTM as a user behavior prediction model.

4. The method for building a linkage control scene of a smart home as claimed in claim 3, characterized in that: The user behavior prediction model is used to predict the user's activity trajectory and output the user behavior prediction result, including the following steps: The preprocessed user behavior data and environmental parameters are combined into a user comprehensive feature vector and input into the user behavior prediction model to obtain a preliminary prediction result; The preliminary prediction results are logically verified and adjusted to obtain the final user behavior prediction results.

5. The method for building a linkage control scene of a smart home as claimed in claim 4, characterized in that: Set the scenario template, combine the user behavior prediction results and environmental parameters, select the most suitable scenario configuration from the scenario template, and get the recommended scenario configuration solution, including the following steps: By setting the scene template and reading the latest environmental parameters of the user's home, a preliminary scene template is determined, and a preliminary scene configuration is selected according to the preliminary scene template to obtain a preliminary scene configuration plan; A genetic algorithm is used to perform selection, cross-breeding, and mutation operations on the preliminary scenario configuration schemes and iterate them to output a recommended scenario configuration scheme.

6. The method for building a linkage control scene of a smart home as claimed in claim 5, characterized in that: Design a graphical user interface, through which users can view recommended scenario configuration solutions, make fine adjustments based on personal preferences, and confirm the final scenario configuration solution, including the following steps: Conduct user surveys, determine the core requirements of the graphical user interface based on the survey results, draw sketches of the graphical user interface, and convert the sketches into interactive prototypes; Invite users to conduct participatory testing on the interactive prototype, adjust the interactive prototype, integrate the voice recognition engine on the adjusted interactive prototype and perform voice command parsing, and increase the context management capability of the adjusted interactive prototype to form a complete graphical user interface; The user views the recommended scenario configuration plan through the graphical user interface and confirms the final scenario configuration plan.

7. The method for building a linkage control scene of a smart home as claimed in claim 6, characterized in that: The final scenario configuration plan is converted into a list of smart home device operation instructions and sent to the corresponding target home device. When the target home device receives the operation instruction, the status display of the target home device is updated and a home device status execution report is output.

8. A system for building a linkage control scene of a smart home, based on the method for building a linkage control scene of a smart home according to any one of claims 1 to 7, characterized in that: include, Model training module collects environmental parameters, selects deep learning models, and trains them into user behavior prediction models; The result prediction module uses the user behavior prediction model to predict the user's activity trajectory and output the user behavior prediction result; The output solution module sets the scenario template, combines the user behavior prediction results and environmental parameters, selects the most suitable scenario configuration from the scenario template, and obtains the recommended scenario configuration solution; The scheme adjustment module designs a graphical user interface through which users can view the recommended scenario configuration schemes, make fine adjustments based on their personal preferences, and confirm the final scenario configuration scheme; The report generation module sends instructions to the target home devices according to the final scenario configuration plan and outputs the home device status execution report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for building a linkage control scene of a smart home as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for building a linkage control scene of a smart home as described in any one of claims 1 to 7 are implemented.

Images