Intelligent home environment regulation and control method and device based on multi-modal behavior perception

Abstract

The invention relates to the technical field of smart home, and discloses a smart home environment regulation and control method and device based on multi-modal behavior perception, and the method and device can understand the home environment parameter preference of a user in different scenes through a home habit reference value. The current physiological state, behavior mode and environment condition of the user can be comprehensively captured by collecting multi-mode sensing data. And calculating a multi-modal data deviation value to quantify the difference between the current home environment and the user habits, and positioning environment factors needing to be adjusted. The multi-modal data deviation value is subjected to weighted fusion calculation according to the weight matrix driven by the current health state of the user, and the comfort degree deviation index is generated, so that the recognition accuracy of user requirements is improved, and personalized and dynamic comfort degree evaluation is realized. An intelligent regulation and control strategy is determined according to the comfort degree deviation index, precise regulation and control of the home environment in the fields of medical science and technology and home-based care for the aged are achieved, and the accuracy and adaptability of home environment parameter regulation and control are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a method and device for intelligent control of home environment based on multimodal behavior perception. Background Technology

[0002] With the development of smart home technology, smart home systems applied in medical technology and elderly care can provide certain conveniences, but they still have shortcomings in age-friendly design, especially in behavior sensing and environmental regulation. The behavioral patterns and needs of the elderly differ significantly from those of younger people; therefore, a system is needed that can proactively sense the behavior of the elderly and adjust the environment accordingly.

[0003] The core challenge in current age-friendly smart home environment control lies in the inability of traditional systems to dynamically adapt to the personalized needs of elderly people as their health status and behavioral habits change. Existing technologies mainly rely on preset threshold rules or a single data source (such as temperature sensors or wearable devices) for passive responses, which has significant limitations.

[0004] On the one hand, static threshold control (such as setting the air conditioner temperature to 26°C) ignores the fact that the sensitivity of the elderly to environmental parameters such as temperature, humidity and light fluctuates with their physical condition, leading to comfort problems such as feeling too cold in summer or insufficient heating in winter.

[0005] On the other hand, most solutions sever the correlation between physiological indicators (such as heart rate and body surface temperature), behavioral patterns (such as activity trajectory and sedentary time) and environmental parameters (such as indoor PM2.5 and illuminance). For example, one patent adjusts the air conditioner temperature based solely on the heart rate data of the wearable device without taking into account the impact of environmental humidity on the body's sensation, which may cause respiratory discomfort or dry skin.

[0006] In addition, existing interaction methods rely heavily on voice or APP operation, which is extremely unfriendly to elderly people with cognitive impairment or low digital literacy. When sudden health abnormalities occur (such as needing to turn on the lights and call the alarm after a fall), users often delay rescue due to operational difficulties.

[0007] More importantly, traditional architectures lack the ability to learn from long-term behavioral habits and cannot adaptively adjust strategies: for example, elderly people prefer 500 lux of illumination for morning reading, but the system still uses historical fixed values ​​and cannot recognize the changing trend that it needs to be gradually increased to 800 lux as their vision deteriorates.

[0008] These shortcomings collectively lead to problems such as a fragmented user experience, increased security risks, and energy waste in the application of smart home systems in the field of home-based elderly care. Therefore, how to accurately identify users' needs for the home environment in order to improve the accuracy of home environment parameter control has become an urgent technical problem to be solved. Summary of the Invention

[0009] This invention provides a method, device, computer equipment, and medium for intelligent control of home environment based on multimodal behavior perception, so as to improve the accuracy of anti-fraud identification of medical claim requests and reduce fraud risk.

[0010] Firstly, a method for intelligent control of home environment based on multimodal behavior perception is provided, including:

[0011] Obtain baseline values ​​of the target user's home habits, and collect multimodal perception data of the target user in the current home environment;

[0012] Calculate the difference between the target user's multimodal perception data in the current home environment and its corresponding home habit baseline value to obtain the multimodal data deviation value;

[0013] Based on the weight matrix corresponding to the current health status of the target user, the multimodal data deviation values ​​are weighted and fused to generate the comfort deviation index of the target user in the current home environment.

[0014] Based on the deviation level corresponding to the comfort deviation index, an intelligent control strategy for the target user in the current home environment is determined.

[0015] Secondly, a smart home environment control device based on multimodal behavior perception is provided, comprising:

[0016] The data acquisition module is used to acquire baseline values ​​of the target user's home habits and to collect multimodal perception data of the target user in the current home environment;

[0017] The deviation calculation module is used to calculate the difference between the multimodal perception data of the target user in the current home environment and its corresponding home habit benchmark value, and obtain the multimodal data deviation value.

[0018] The deviation index generation module is used to perform weighted fusion calculation on the deviation values ​​of the multimodal data based on the weight matrix corresponding to the current health status of the target user, and generate the comfort deviation index of the target user in the current home environment.

[0019] The control strategy determination module is used to determine the intelligent control strategy for the target user in the current home environment based on the deviation level corresponding to the comfort deviation index.

[0020] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-mentioned intelligent control method for home environment based on multimodal behavior perception.

[0021] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described intelligent control method for home environment based on multimodal behavior perception.

[0022] The aforementioned solution, implemented using a multimodal behavior perception-based intelligent home environment control method, device, computer equipment, and storage medium, utilizes home habit benchmarks to understand user preferences for home environment parameters in different scenarios. Collecting multimodal perception data comprehensively captures the user's current physiological state, behavioral patterns, and environmental conditions. Calculating multimodal data deviation values ​​quantifies the difference between the current home environment and user habits, enabling more precise identification of environmental factors requiring adjustment, thereby improving the accuracy of user need recognition. Weighted fusion calculation of multimodal data deviation values ​​based on a weight matrix driven by the user's current health status generates a comfort deviation index, which not only improves the accuracy of user need recognition but also achieves personalized and dynamic comfort assessment. Intelligent control strategies are determined based on the comfort deviation index, enabling precise control of the home environment and effectively improving the accuracy and adaptability of home environment parameter control. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an application environment for a home environment intelligent control method based on multimodal behavior perception, according to an embodiment of the present invention.

[0025] Figure 2 A flowchart illustrating the first embodiment of the intelligent home environment control method based on multimodal behavior perception provided by this invention.

[0026] Figure 3 This is a schematic diagram of a smart home environment control device based on multimodal behavior perception in one embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0028] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] The intelligent home environment control method based on multimodal behavior perception provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can obtain the target user's home habit baseline value and collect the target user's multimodal perception data in the current home environment; calculate the difference between the target user's multimodal perception data in the current home environment and its corresponding home habit baseline value to obtain a multimodal data deviation value; based on the weight matrix corresponding to the target user's current health status, perform weighted fusion calculation on the multimodal data deviation value to generate the target user's comfort deviation index in the current home environment; and determine the intelligent control strategy for the target user in the current home environment based on the deviation level corresponding to the comfort deviation index.

[0031] This invention addresses the technical problems of fragmented user experience, increased security risks, and energy waste in smart home systems for elderly care, leading to low accuracy in home environment parameter control. By using home habit benchmarks, the invention aims to understand user preferences for home environment parameters in different scenarios. Collecting multimodal perception data comprehensively captures the user's current physiological state, behavioral patterns, and environmental conditions. Calculating multimodal data deviation values ​​quantifies the difference between the current home environment and user habits, enabling more precise identification of environmental factors requiring adjustment, thereby improving the accuracy of user need recognition. A weighted fusion calculation of multimodal data deviation values ​​based on a weight matrix driven by the user's current health status generates a comfort deviation index. This not only improves the accuracy of user need recognition but also enables personalized and dynamic comfort assessment. Based on the comfort deviation index, an intelligent control strategy is determined, achieving precise control of the home environment and effectively improving the accuracy and adaptability of home environment parameter control.

[0032] The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.

[0033] Please see Figure 2 As shown, Figure 2A flowchart illustrating the first embodiment of the intelligent home environment control method based on multimodal behavior perception provided by this invention includes the following steps:

[0034] S101: Obtain the baseline value of the target user's home habits, and collect the multimodal perception data of the target user in the current home environment;

[0035] In one embodiment, a user profile containing multi-dimensional attributes such as basic user information, behavioral characteristics, and environmental preferences can be constructed by mining the historical home habit data of target users. A user habit baseline can be established based on the user profile, and the benchmark values ​​of home habits of target users under different home scenarios and different health conditions can be analyzed.

[0036] In one embodiment, the home habit benchmark values ​​include physiological parameter benchmark values ​​and environmental parameter benchmark values ​​of the target user in different home environments.

[0037] In one embodiment, obtaining the baseline value of the target user's home habits includes: obtaining the target user's historical home habit data; based on the historical home habit data, filtering at least one subset of home habit data corresponding to a home environment and a subset of physiological parameters corresponding to at least one health state of the target user; calculating the mean value of environmental parameters in the subset of home habit data to obtain the baseline value of environmental parameters corresponding to each home environment; and calculating the mean value of physiological parameters in the subset of physiological parameters corresponding to the health state to obtain at least one baseline value of physiological parameters corresponding to the health state.

[0038] Specifically, historical home habit data of target users can be collected from multiple channels, including various sensors in the home environment (including physiological parameter sensors, environmental parameter sensors, etc.), wearable devices, and user manual operation records (such as operation logs of adjusting environmental parameters through a mobile app). This data covers information such as the user's physiological state, behavioral habits, and preferred settings of environmental parameters at different times and in different home environment scenarios.

[0039] The collected historical home habits data is stored in a distributed storage system (such as Hive). Hive can efficiently store and manage massive amounts of structured data and supports SQL queries, facilitating subsequent data mining and analysis. During storage, the data is categorized and labeled to enable quick filtering and querying of subsets of data based on specific conditions.

[0040] In one specific embodiment, the home environment can be divided into multiple different scenarios based on different functional areas (such as living room, bedroom, kitchen, bathroom, etc.) and user behavior patterns at different times (such as waking up, daytime activities, nighttime sleep, etc.). For example, the scenario of preferred temperature in the morning can be defined as the environmental parameter data corresponding to the user's activities in the bedroom or living room from 6:00 to 9:00 every morning.

[0041] Based on predefined home environment scenarios, a subset of home habit data corresponding to each scenario is filtered from historical home habit data. This filtering process can be implemented by writing SQL queries, such as using the WHERE clause in Hive to specify scenario and health status conditions, thereby obtaining the desired subset of data.

[0042] For each home environment scenario, the mean value of the environmental parameter data in the corresponding subset of home habit data is calculated to obtain the baseline value of the environmental parameters in that scenario.

[0043] For example, when calculating the preferred morning temperature, records between 6 a.m. and 9 a.m. can be filtered out. Then, the ambient temperature data set by the user within this time period is used to calculate the approximate 90th percentile value to avoid the influence of extreme values. Finally, the 90th percentile value of the user's temperature data between 6 a.m. and 9 a.m. is used to calculate the average value of the 90th percentile value of the temperature data, which is then used as the benchmark value for the user's preferred morning temperature.

[0044] Simultaneously, based on the user's different health states (such as normal state, cold state, abnormal heart rate state, etc.), corresponding subsets of physiological parameters are selected from historical physiological parameter data. For the physiological parameter data in the subset corresponding to each health state, their mean is calculated to obtain the baseline value of the physiological parameters corresponding to that health state.

[0045] For example, to calculate a user's baseline resting heart rate under normal conditions, heart rate data of users in a normal healthy state and in a state of rest and activity can be filtered out, and the average heart rate value of each user can be calculated as their baseline resting heart rate under normal conditions.

[0046] In addition to the mean, statistical indicators such as the median and mode can be calculated as needed to supplement the benchmark value. For example, when the data has extreme values ​​or a skewed distribution, the median may better reflect the actual preferences of users. At the same time, analyzing the distribution of environmental parameter data and understanding the degree of dispersion and trends of change provides a basis for further optimizing environmental control strategies.

[0047] Considering that users' physiological states may gradually change over time, the baseline values ​​of physiological parameters are recalculated and updated regularly to ensure that they accurately reflect the user's current physiological characteristics. For example, the baseline values ​​are recalculated monthly or quarterly based on new physiological parameter data, and the trends of the baseline values ​​are compared and analyzed to provide a reference for users' health management.

[0048] In one embodiment, the multimodal perception data includes the target user's real-time physiological parameters, the target user's behavioral perception parameters, and the real-time environmental parameters of the current home environment.

[0049] The real-time physiological parameters of the target user may include one or more parameters such as heart rate, body temperature, and respiratory rate.

[0050] The behavioral perception parameters of the target user may include one or more parameters such as the target user's activity trajectory, body posture, and daily routine.

[0051] The real-time environmental parameters of the current home environment may include one or more parameters such as ambient temperature, ambient humidity, light intensity, and regional characteristics (such as indoor, outdoor, living room, bathroom, etc.) of the target user's current location.

[0052] In one embodiment, the step of collecting the target user's multimodal perception data in the current home environment includes: collecting the target user's real-time physiological parameters through a wearable device worn on the target user; collecting the real-time environmental parameters of the current home environment based on environmental sensors; and identifying the target user's posture based on a posture recognition device to obtain the target user's behavioral perception parameters.

[0053] In one specific embodiment, target users can be equipped with medical-grade wearable devices, such as smart bracelets, smartwatches, or smart chest straps. These devices have high-precision, high-sampling-rate physiological signal sensors and connect to gateway devices in the home environment via wireless communication technologies such as Bluetooth.

[0054] For example, a photoplethysmography (PPG) sensor can be used to measure heart rate. Its principle is based on the light absorption characteristics of blood. By emitting light of a specific wavelength onto the skin surface and detecting changes in the intensity of the reflected light, fluctuations in blood volume are sensed, thereby calculating the heart rate value. Alternatively, a thermistor or other temperature sensor can be attached to the user's skin to measure body surface temperature. The collected temperature data can then be processed using a body surface temperature compensation algorithm to eliminate interference from ambient temperature changes and improve measurement accuracy.

[0055] For example, wearable devices collect users' physiological parameters in real time according to a set sampling frequency (e.g., heart rate sampled once per second, body temperature sampled once per minute). The collected data first undergoes preliminary filtering on the device to remove high-frequency noise interference, and then the data is transmitted to a gateway device via Bluetooth. The gateway device performs further preprocessing operations on the received data, such as parsing, format conversion, and timestamp marking, to generate the target user's real-time physiological parameters.

[0056] In one specific embodiment, a multi-point sensor cluster can be deployed in a home environment to achieve comprehensive real-time monitoring of environmental parameters in different areas. For example, temperature and humidity sensors, light sensors, and air quality sensors can be installed in key activity areas such as the living room, bedroom, kitchen, and bathroom. The sensors are connected to the gateway device via wired (e.g., RS485 bus) or wireless (e.g., Bluetooth, Wi-Fi) communication methods.

[0057] Generally, since sensors at different locations may be affected by local environmental factors and thus produce measurement errors, a multi-point weighted fusion algorithm is needed to process the collected real-time environmental parameter data.

[0058] For example, for temperature data, a weighted average is calculated based on the location weight of each sensor in the room (e.g., sensors near windows are more affected by the outside temperature and have a relatively lower weight; sensors near the center of the room have a higher weight) and the accuracy weight of historical measurement data, to obtain a more accurate representative value of indoor temperature.

[0059] In one specific embodiment, attitude recognition devices such as millimeter-wave radar and infrared arrays can be deployed in a home environment. Millimeter-wave radar utilizes the reflection characteristics of high-frequency electromagnetic waves to penetrate non-metallic obstacles (such as walls, furniture, etc.) to sense minute human movements. Its working principle is to emit millimeter-wave signals and receive reflected signals, analyzing the motion state and attitude of the target object based on changes in the frequency, phase, and amplitude of the reflected signals. Infrared arrays, on the other hand, are based on the characteristic of objects emitting infrared radiation, capturing the intensity distribution and changes of human infrared radiation to achieve human attitude recognition.

[0060] Signal processing and pattern recognition algorithms are used to analyze the acquired radar and infrared signals. First, preprocessing operations such as filtering, denoising, and feature extraction are performed on the signals to extract key feature parameters of human movement, such as speed, acceleration, and joint angle changes. Then, these feature parameters are matched against a pre-established behavioral pattern database. Machine learning algorithms (such as support vector machines and deep neural networks) are used to identify and classify user behavior, such as determining whether the user is sitting, lying down, walking, or has fallen, and encoding this behavior into specific behavioral perception parameters (e.g., the behavioral intent of "radar detecting an elderly person walking towards the window" can be encoded as "INTENT_WINDOW_OPEN").

[0061] S102: Calculate the difference between the target user's multimodal perception data in the current home environment and its corresponding home habit baseline value to obtain the multimodal data deviation value;

[0062] Before calculating the multimodal data deviation value, the multimodal sensing data (including real-time physiological parameters, behavioral sensing parameters, and real-time environmental parameters) obtained from the sensing layer and the home habit baseline data obtained from the analysis layer are first standardized and cleaned. This ensures that all data units, timestamp formats, etc., are consistent, and obvious outliers and erroneous data are removed to improve the accuracy of the deviation value calculation.

[0063] In one embodiment, real-time sensing data is matched and aligned with corresponding home habit baseline data based on user identifier (user_id) and timestamp information. Since the sensing data is collected in real time, while the baseline values ​​are calculated based on historical data, there may be temporal inconsistencies. Therefore, an interpolation algorithm (such as linear interpolation or moving average interpolation) is needed to extend the baseline values ​​in the time dimension, making them correspond to the real-time sensing data in time, in order to perform subsequent deviation calculations.

[0064] In one embodiment, the difference between the multimodal sensing data of the same modality and its corresponding home habit benchmark value is calculated. For example, the difference between real-time ambient temperature and ambient temperature benchmark value, and the difference between real-time heart rate and resting heart rate benchmark value are calculated.

[0065] For example, assuming the multimodal sensing data includes real-time ambient temperature, real-time heart rate parameters, and real-time ambient illuminance parameters, then the corresponding ambient temperature baseline value, resting heart rate baseline value, and illuminance baseline value need to be obtained. Multimodal data deviation values ​​include ambient temperature deviation values, heart rate deviation values, and illuminance deviation values. Multimodal data deviation values ​​can be expressed as:

[0066]

[0067] Among them, T realtimeT represents the real-time ambient temperature parameter. benchmark This represents the ambient temperature baseline value in the corresponding home living habits benchmark, where ΔT is the temperature deviation value; HR realtime This represents the real-time heart rate parameter, HR. benchmark This represents the baseline heart rate value; the heart rate deviation value ΔHR is the difference between the two. realtime L represents the real-time ambient illuminance parameter. benchmark This represents the ambient illuminance reference value, and the illuminance deviation value ΔL is the difference between the two.

[0068] In one embodiment, considering the different sensitivities of different users to temperature changes, as well as temperature differences in different seasons and regions, a temperature sensitivity coefficient (k) can be introduced for the temperature deviation value. T The temperature deviation is adjusted, and the adjusted temperature deviation is ΔT′=k T ×ΔT. The temperature sensitivity coefficient can be determined comprehensively based on user feedback, regional climate data, and seasonal factors. For example, the temperature sensitivity coefficient can be appropriately increased in cold regions or during winter.

[0069] Regarding heart rate deviation, heart rate is influenced not only by the user's own physiological habits but also by short-term emotional fluctuations and exercise status. To more accurately reflect the impact of heart rate deviation on comfort, it can be corrected by incorporating behavioral perception parameters (such as whether the user is exercising or experiencing stress). For example, when the user is detected to be exercising, the allowable deviation range of the heart rate baseline can be appropriately increased, or the weight of heart rate deviation in the overall deviation calculation can be reduced.

[0070] Regarding illuminance deviation, different activities have significantly different requirements for ambient illuminance; for example, the illuminance requirements for reading, watching television, and resting are all different. Therefore, when calculating illuminance deviation, the baseline illuminance value can be adjusted by incorporating the user's current behavioral perception parameters (such as whether they are reading, watching television, or resting). For instance, when it is detected that a user is reading, the baseline illuminance value for the reading scenario should be used first for deviation calculation, and the allowable illuminance deviation range can be appropriately narrowed to ensure good visual comfort for the user while reading.

[0071] Understandably, for other types of multimodal data bias values, such as ambient humidity and user respiration rate, appropriate adjustments can also be made based on the data type and characteristics to adapt to the needs of actual application scenarios.

[0072] S103: Based on the weight matrix corresponding to the current health status of the target user, perform weighted fusion calculation on the deviation values ​​of the multimodal data to generate the comfort deviation index of the target user in the current home environment;

[0073] In one embodiment, to address the issue of different dimensions and large differences in numerical ranges among various parameters, the deviation values ​​of different parameters in the multimodal data deviation values ​​are normalized. For example, temperature deviation may vary within a few degrees Celsius, while heart rate deviation may vary within tens of beats per minute. Directly weighting and summing these deviation values ​​with different dimensions would result in the calculation results being dominated by parameters with large numerical ranges, failing to accurately reflect the comprehensive impact of each parameter on comfort.

[0074] By normalizing the parameters, the deviation values ​​are mapped to the same scale range (such as -1 to 1 or 0 to 1), making them comparable and additive in the comprehensive deviation calculation, thus more reasonably assessing the overall comfort deviation of the home environment.

[0075] Generally, the preferences for home environment parameters and physiological characteristics of the same user differ under different health conditions. For example, heart rate and respiratory rate may differ in a normal state and when experiencing a cold, and the user's needs for temperature and humidity may also differ. For instance, a user in a normal state may prefer lower indoor humidity, while a user with a cold may prefer higher indoor temperature. Therefore, to more accurately regulate home environment parameters, a user health status-driven approach can be adopted to calculate a comfort deviation index, thereby more accurately assessing the user's needs for regulating home environment parameters.

[0076] In one embodiment, a comprehensive health status assessment system can be established, which utilizes physiological parameters (such as heart rate, body temperature, blood oxygen saturation, etc.) collected by wearable devices, user behavioral data (such as activity level, sleep quality, etc.), and health information manually entered by users (such as self-reported symptoms, medication status, etc.) to perform real-time assessment and classification of the target user's current health status through machine learning algorithms (such as decision trees, neural networks, etc.).

[0077] For example, indicators such as heart rate variability and body temperature fluctuations can be used to determine whether a user has a cold; characteristics such as whether the heart rate deviates from the baseline value for a long period of time or whether there are abnormal heart rhythms can be used to determine whether a user has an abnormal heart rate state. For instance, if a user's body temperature exceeds 37.2℃ for one hour and the amplitude of heart rate fluctuations increases, the user is determined to have a cold; if the heart rate deviates from the baseline value by more than 10 bpm for a long period of time at rest, it is determined to be an abnormal heart rate state.

[0078] Furthermore, based on the current health status of the target user, a weight matrix corresponding to the current health status is matched to determine the weight coefficients corresponding to each of the multimodal deviation values; based on the weight coefficients corresponding to each of the multimodal deviation values, the multimodal deviation values ​​are weighted and summed to obtain the comfort deviation index.

[0079] In one embodiment, based on the health status assessment results, a weight matrix corresponding to the current health status is matched from a predefined dynamic weight matrix. Based on the matched weight matrix, the absolute value of each multimodal deviation value is multiplied by its corresponding weight coefficient, and the results are then summed to obtain the Comfort Deviation Index (CDI). The calculation formula is as follows:

[0080] CDI = f(ΔT, ΔHR, ΔL, Health)

[0081] Wherein, ΔL is the temperature deviation value; ΔHR is the heart rate deviation value; ΔL is the illumination deviation value; and Health is the health status factor, used to dynamically adjust the weighting coefficient of the multimodal deviation value coupling.

[0082] Specifically, the weight matrix is ​​stored in the system database and indexed by health status to quickly retrieve and obtain the corresponding parameter values ​​such as temperature weight α, heart rate weight β, and light intensity weight γ. For example, when a user is assessed as having a cold, the matched weight matrix is ​​α = 0.7, β = 0.2, and γ = 0.1; when the user is in an abnormal heart rate state, the matched weight matrix is ​​α = 0.3, β = 0.6, and γ = 0.1.

[0083] In the real-time data stream, a scrolling window with a certain time period (e.g., 5 seconds or 10 seconds) can be used to acquire the multimodal deviation values ​​for each user within the window. These values ​​include temperature deviation (the difference between real-time temperature and preferred temperature), heart rate deviation (the difference between real-time heart rate and resting heart rate), and illuminance deviation (the difference between real-time illuminance and preferred illuminance). The multimodal deviation values ​​are then coupled and calculated in real-time to generate the Comfort Deviation Index (CDI).

[0084] CDI=α×ΔT+β×ΔHR+γ×ΔL+…

[0085] Here, CDI stands for Comfort Deviation Index, used to quantitatively assess the overall deviation between the current home environment and the user's habitual baseline. α, β, and γ represent the weighting coefficients of different parameters such as temperature, heart rate, and illuminance in the comprehensive deviation calculation, and α + β + γ + ... = 1. The weighting coefficients can be dynamically adjusted based on the importance of the parameters to user comfort and the needs of the current scenario. ΔT, ΔHR, and ΔL are the deviation values ​​of each parameter such as temperature, heart rate, and illuminance obtained from the above calculations.

[0086] Understandably, the calculation of the comfort deviation index can further consider other environmental parameters (such as humidity and noise) and the user's current health status based on the actual application scenario. For example, when a user has a cold, the weight of temperature and humidity parameters in the comprehensive deviation calculation can be increased to better reflect the need for a warm and humid environment during a cold; or when excessive indoor noise is detected, a noise deviation term can be introduced and given an appropriate weight to participate in the comprehensive deviation calculation, thereby comprehensively reflecting the comfort deviation of the home environment.

[0087] S104: Based on the deviation level corresponding to the comfort deviation index, determine the intelligent control strategy for the target user in the current home environment.

[0088] In one embodiment, the comfort deviation index is graded and evaluated based on a preset grading threshold to determine the deviation level corresponding to the comfort deviation index; and an intelligent control strategy corresponding to the deviation level is retrieved from a predefined strategy library.

[0089] Specifically, in the real-time data processing system, the calculated CDI value is continuously monitored. Conditional judgment logic is set in the stream computing engine (such as Doris) to determine the deviation level of the current CDI in real time based on preset grading thresholds. Based on the determined CDI deviation level, the corresponding intelligent control strategy is retrieved from a predefined strategy library and adapted and optimized in conjunction with specific home environment parameters and user habits to generate precise control instructions for the current home environment.

[0090] For example, suppose the Comfort Deviation Index (CDI) is divided into three deviation levels: CDI<4, 4≤CDI<8, and CDI≥8.

[0091] When the CDI value is less than 4, it indicates that the current home environment is close to the user's comfort needs, and there is no need to make significant environmental adjustments. Control all smart devices to maintain the current operating status and ensure that environmental parameters (such as temperature, humidity, illuminance, etc.) remain stable at the current level.

[0092] When the CDI value reaches or exceeds 8, it indicates a significant deviation between the current home environment and the user's comfort needs, which may have a substantial impact on the user's health or comfort. An emergency linkage strategy should be activated immediately, rapidly adjusting environmental parameters to a comfortable range while triggering a safety warning mechanism. For example, if the temperature is too high and the user's heart rate rises abnormally, the air conditioner will quickly lower the temperature and simultaneously send an alarm message to the user's emergency contact or medical service provider.

[0093] When the CDI value is greater than or equal to 4 and less than 8, the specific parameter increment or decrement for each adjustment is calculated based on the CDI value and the deviation of the current values ​​of each environmental parameter from the baseline value. According to the calculated adjustment parameters, the corresponding intelligent devices are controlled to make small, gradual adjustments. For example, if the temperature needs to be lowered, the air conditioner will gradually lower the set temperature while adjusting the fan speed as needed; if the illuminance needs to be increased, the lights will be gradually brightened to avoid sudden changes in light that may cause discomfort to the user.

[0094] During the adjustment process, control the step size of each adjustment (e.g., temperature adjustment should not exceed 1℃ each time, illuminance adjustment should not exceed 100 lux each time) and the time interval between two adjustments (e.g., adjust once every 5 minutes) to prevent environmental parameters from changing too quickly and causing discomfort to users, while avoiding the potential damage caused by frequent start-ups and shutdowns of the equipment.

[0095] Furthermore, when the deviation level is greater than the first deviation level range, based on the intelligent control strategy, the control object and the target control index of the control object are determined; based on the index difference between the current index of the control object and the target control index, the single-step adjustment range of the control object is determined; the control object is gradually adjusted according to the single-step adjustment range until the control object is adjusted from the current index to the target control index.

[0096] In one embodiment, the first deviation level range can be a deviation level that makes the environmental parameters closer to the comfort requirements of the target user, such as the case where CDI < 4 as described above. In this case, the environmental parameters can be kept unchanged, and there is no need to adjust the current environmental parameters.

[0097] In one embodiment, based on the intelligent control strategy and the current home environment status, specific environmental parameters that need to be adjusted are identified as control objects, such as temperature, humidity, and illuminance.

[0098] At the same time, the priorities of the control targets can be sorted based on the user's health status and current activity. For example, if a user is in an abnormal heart rate state and has a high CDI value, temperature and humidity regulation may be given higher priority because they have a greater impact on the user's physical comfort and health.

[0099] In one embodiment, a target control index is set for each controllable object based on the user's home habits baseline and the current comfort deviation. The target control index refers to the ideal value that the controllable object is expected to be adjusted to so that the user's comfort is optimal. For example, if the user's preferred temperature is 26°C, and the current temperature deviation is +2°C (i.e., the actual temperature is 28°C), then the target control index may be set to 26°C.

[0100] For each controlled object, calculate the difference between its current index and the target control index. For example, if the current temperature is 28℃ and the target temperature is 26℃, the temperature index difference is +2℃; if the current illuminance is 600 lux and the target illuminance is 500 lux, the illuminance index difference is +100 lux.

[0101] Based on preset adjustment rate limits (e.g., temperature adjustment not exceeding 1°C per step, illuminance adjustment not exceeding 100 lux per step) and the magnitude of the index difference, the single-step adjustment range for each controlled object is determined. Based on the determined single-step adjustment range, specific adjustment commands are generated and sent to the corresponding smart devices for execution via the smart home control system.

[0102] The adjustment rate limit is preset based on factors such as user comfort and equipment performance to ensure a smooth adjustment process without causing user discomfort. For example, if the temperature index difference is +2℃, the adjustment increment is 1℃ each time, requiring two adjustments to reach the target temperature; if the illuminance index difference is +150 lux, the adjustment increment is 100 lux each time. The first adjustment reduces the illuminance to 500 lux (a reduction of 100 lux), and the second adjustment further fine-tunes the result based on the actual situation.

[0103] For example, if it is necessary to adjust the ambient temperature of the target user's current environment, this can be achieved through a pre-set gradual temperature adjustment protection mechanism function. This function receives the target temperature and the current temperature as input parameters. First, it calculates the temperature difference δ between the target and current temperatures. Assuming a single-step adjustment increment of 1℃, if δ is greater than 0, the single-step adjustment increment is 1.0; if δ is less than 0, the increment is -1.0, to prevent vascular stress responses caused by excessive temperature changes. Next, it checks if the absolute value of δ is less than or equal to 1℃: if yes, it directly returns to the target temperature; otherwise, it adjusts the current temperature according to the step size of the single-step adjustment, gradually approaching the target temperature and returning the adjusted temperature value. This mechanism ensures a smooth transition in temperature changes by limiting the single-step adjustment increment, thus providing protection.

[0104] In one embodiment, the controlled object is gradually adjusted according to the single-step adjustment range.

[0105] Each time an adjustment is made, multimodal perception data of the target user in the current home environment is collected; based on the multimodal perception data, the comfort deviation index of the target user after each adjustment of the control object is calculated; if the current index of the control object is not adjusted to the target control index, and the deviation level corresponding to the comfort deviation index is less than the first deviation level range, the index adjustment operation of the control object is stopped.

[0106] After each adjustment, sensors monitor the changes in the current index of the regulated object and the user's physiological and behavioral feedback in real time to evaluate the adjustment effect. For example, monitoring whether the temperature has accurately dropped to the target value, whether the user's physiological parameters such as heart rate and body temperature have improved, and whether the user expresses satisfaction with the environmental changes through actions or voice. Based on the monitoring results, subsequent adjustment strategies can be dynamically adjusted, such as appropriately speeding up or slowing down the adjustment and optimizing the target regulation index.

[0107] The system continuously adjusts the parameters step-by-step according to the set adjustment range and time interval (e.g., every 5 minutes) until the current index of the controlled object reaches or approaches the target control index. Throughout the adjustment process, real-time monitoring of the user's status and environmental parameters is maintained. If necessary, the adjustment can be paused or the adjustment strategy adjusted to adapt to changes in the user's real-time needs. For example, if the user manually adjusts the device settings during the adjustment process, the system will reassess and adjust the subsequent adjustment plan.

[0108] After each adjustment of the current index of the controlled object, the comfort deviation index of the target user under the current environmental parameters can be recalculated based on the collected multimodal perception data. The comfort deviation index is then evaluated. If the current index of the controlled object has not yet been adjusted to the target control index, but the deviation level corresponding to the comfort deviation index is already less than the first deviation level range, then the current environmental parameters can be considered to be the comfort environmental parameters of the target user. In this case, the index adjustment of the controlled object can be stopped, or the adjustment can continue to the target control index according to the user's instructions.

[0109] By supplementing and improving the technical steps of determining the control object, target control index, single-step adjustment range, and gradual adjustment execution based on the intelligent control strategy, it is possible to achieve precise and gradual adjustment of the home environment, ensuring that users feel a gradual increase in comfort during the adjustment process, while avoiding discomfort or health risks caused by rapid changes in environmental parameters.

[0110] As can be seen, the above solution, by using home habit benchmarks, can understand users' preferences for home environment parameters in different scenarios; collecting multimodal perception data can comprehensively capture users' current physiological state, behavioral patterns, and environmental conditions, laying the foundation for improving the accuracy of home environment parameter regulation and solving the passive response problem caused by traditional systems relying on a single data source. Calculating multimodal data deviation values ​​can quantify the difference between the current home environment and user habits, clarifying in which aspects environmental parameters deviate from the user's comfort needs, and more accurately locating environmental factors that need adjustment, thereby improving the accuracy of identifying user needs. Weighted fusion calculation of multimodal data deviation values ​​based on a weight matrix driven by the user's current health status allows for flexible adjustment of the focus of environmental parameter regulation according to the user's real-time health status, generating a comfort deviation index that comprehensively reflects user comfort. This not only improves the accuracy of identifying user needs but also achieves personalized and dynamic comfort assessment, solving the problem that traditional systems' static threshold control cannot adapt to fluctuations in users' physical condition. The intelligent control strategy determined by the comfort deviation index can transform the abstract deviation index into specific control measures. It can adjust the intensity of the adjustment according to the severity of the deviation, rationally allocate resources, and achieve precise control of the home environment, effectively improving the accuracy and adaptability of home environment parameter control.

[0111] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0112] In one embodiment, a smart home environment control device based on multimodal behavior perception is provided, which corresponds one-to-one with the smart home environment control method based on multimodal behavior perception in the above embodiments. For example... Figure 3 As shown, the intelligent home environment control device based on multimodal behavior perception includes: a data acquisition module 201, a deviation calculation module 202, a deviation index generation module 203, and a control strategy determination module 204. Detailed descriptions of each functional module are as follows:

[0113] The data acquisition module 201 is used to acquire the baseline value of the target user's home habits and to collect the multimodal perception data of the target user in the current home environment;

[0114] The deviation calculation module 202 is used to calculate the difference between the multimodal perception data of the target user in the current home environment and its corresponding home habit benchmark value, and obtain the multimodal data deviation value.

[0115] The deviation index generation module 203 is used to perform weighted fusion calculation on the deviation values ​​of the multimodal data based on the weight matrix corresponding to the current health status of the target user, and generate the comfort deviation index of the target user in the current home environment.

[0116] The control strategy determination module 204 is used to determine the intelligent control strategy for the target user in the current home environment based on the deviation level corresponding to the comfort deviation index.

[0117] In one embodiment, the multimodal sensing data includes the target user's real-time physiological parameters, the target user's behavioral sensing parameters, and the real-time environmental parameters of the current home environment;

[0118] The data acquisition module 201 includes:

[0119] A real-time physiological parameter acquisition unit is used to acquire the real-time physiological parameters of the target user through a wearable device worn on the target user.

[0120] The real-time environmental parameter acquisition unit is used to acquire real-time environmental parameters of the current home environment based on environmental sensors.

[0121] The behavior perception parameter acquisition unit is used to identify the posture of the target user based on the posture recognition device and obtain the behavior perception parameters of the target user.

[0122] In one embodiment, the home habit benchmark values ​​include physiological parameter benchmark values ​​and environmental parameter benchmark values ​​of the target user in different home environments;

[0123] The data acquisition module 201 further includes:

[0124] The historical data acquisition unit is used to acquire historical home habits data of the target users;

[0125] The data subset filtering unit is used to filter at least one subset of home habit data corresponding to the home environment and at least one subset of physiological parameters corresponding to the health status of the target user based on the historical home habit data.

[0126] An environmental parameter baseline value calculation unit is used to calculate the mean value of environmental parameters in the subset of home habit data, and obtain the environmental parameter baseline value corresponding to each home environment;

[0127] The physiological parameter baseline value calculation unit is used to calculate the mean value of physiological parameters in the subset of physiological parameters corresponding to the health state, and obtain at least one physiological parameter baseline value corresponding to the health state.

[0128] In one embodiment, the deviation index generation module 203 includes:

[0129] The weight coefficient determination unit is used to determine the weight coefficients corresponding to each of the multimodal deviation values ​​by matching the weight matrix corresponding to the current health status of the target user.

[0130] The deviation value weighted calculation unit is used to perform a weighted summation of each of the multimodal deviation values ​​based on the weight coefficients corresponding to each of the multimodal deviation values ​​to obtain the comfort deviation index.

[0131] In one embodiment, the home environment intelligent control device based on multimodal behavior perception further includes an intelligent control module, comprising:

[0132] The control object determination unit is used to determine the control object and the target control index of the control object based on the intelligent control strategy when the deviation level is greater than the first deviation level range.

[0133] A single-step adjustment amplitude determination unit is used to determine the single-step adjustment amplitude of the controlled object based on the index difference between the current index of the controlled object and the target controlled index;

[0134] The index adjustment unit is used to gradually adjust the controlled object according to the single-step adjustment range until the controlled object is adjusted from the current index to the target control index.

[0135] In one embodiment, the intelligent control module further includes:

[0136] The data acquisition unit is used to gradually adjust the control object according to the single-step adjustment range, and to collect multimodal perception data of the target user in the current home environment once for each adjustment;

[0137] The deviation index stepwise calculation unit is used to calculate the comfort deviation index of the target user after each adjustment of the control object based on the multimodal perception data.

[0138] The deviation level progressive evaluation unit is used to stop the index adjustment operation of the controlled object if the current index of the controlled object is not adjusted to the target control index and the deviation level corresponding to the comfort deviation index is less than the first deviation level range.

[0139] In one embodiment, the regulation strategy determination module 204 includes:

[0140] The deviation level determination unit is used to perform a graded evaluation of the comfort deviation index based on a preset grading threshold, and determine the deviation level corresponding to the comfort deviation index.

[0141] The strategy retrieval unit is used to retrieve the intelligent control strategy corresponding to the deviation level from a predefined strategy library.

[0142] This invention provides a smart home environment control device based on multimodal behavior perception. By using home habit benchmarks, it can understand users' preferences for home environment parameters in different scenarios. Collecting multimodal perception data comprehensively captures the user's current physiological state, behavioral patterns, and environmental conditions, laying the foundation for improving the accuracy of home environment parameter control and solving the passive response problem caused by traditional systems relying on a single data source. Calculating multimodal data deviation values ​​quantifies the difference between the current home environment and user habits, clarifying in which aspects environmental parameters deviate from the user's comfort needs, and more accurately locating environmental factors that need adjustment, thereby improving the accuracy of identifying user needs. Weighted fusion calculation of multimodal data deviation values ​​based on a weight matrix driven by the user's current health status allows for flexible adjustment of the control focus of environmental parameters according to the user's real-time health condition, generating a comfort deviation index that comprehensively reflects user comfort. This not only improves the accuracy of identifying user needs but also achieves personalized and dynamic comfort assessment, solving the problem that traditional systems' static threshold control cannot adapt to fluctuations in the user's physical condition. The intelligent control strategy determined by the comfort deviation index can transform the abstract deviation index into specific control measures. It can adjust the intensity of the adjustment according to the severity of the deviation, rationally allocate resources, and achieve precise control of the home environment, effectively improving the accuracy and adaptability of home environment parameter control.

[0143] Specific limitations regarding the intelligent home environment control device based on multimodal behavior perception can be found in the limitations of the intelligent home environment control method based on multimodal behavior perception mentioned above, and will not be repeated here. Each module in the aforementioned intelligent home environment control device based on multimodal behavior perception can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0144] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores operations, computer programs, and the database. The internal memory provides an environment for the operation of the operations and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a server-side method for intelligent control of a home environment based on multimodal behavior perception.

[0145] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input device connected via a bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores operating systems and computer programs. The internal memory provides an environment for the operation of the operating systems and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a smart home environment control method based on multimodal behavior perception.

[0146] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0147] Obtain baseline values ​​of the target user's home habits, and collect multimodal perception data of the target user in the current home environment;

[0148] Calculate the difference between the target user's multimodal perception data in the current home environment and its corresponding home habit baseline value to obtain the multimodal data deviation value;

[0149] Based on the weight matrix corresponding to the current health status of the target user, the multimodal data deviation values ​​are weighted and fused to generate the comfort deviation index of the target user in the current home environment.

[0150] Based on the deviation level corresponding to the comfort deviation index, an intelligent control strategy for the target user in the current home environment is determined.

[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0152] Obtain baseline values ​​of the target user's home habits, and collect multimodal perception data of the target user in the current home environment;

[0153] Calculate the difference between the target user's multimodal perception data in the current home environment and its corresponding home habit baseline value to obtain the multimodal data deviation value;

[0154] Based on the weight matrix corresponding to the current health status of the target user, the multimodal data deviation values ​​are weighted and fused to generate the comfort deviation index of the target user in the current home environment.

[0155] Based on the deviation level corresponding to the comfort deviation index, an intelligent control strategy for the target user in the current home environment is determined.

[0156] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0158] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0159] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for intelligent control of home environment based on multimodal behavior perception, characterized in that, The method includes: Obtain baseline values ​​of the target user's home habits, and collect multimodal perception data of the target user in the current home environment; Calculate the difference between the target user's multimodal perception data in the current home environment and its corresponding home habit baseline value to obtain the multimodal data deviation value; Based on the weight matrix corresponding to the current health status of the target user, the multimodal data deviation values ​​are weighted and fused to generate the comfort deviation index of the target user in the current home environment. Based on the deviation level corresponding to the comfort deviation index, an intelligent control strategy for the target user in the current home environment is determined.

2. The intelligent home environment control method based on multimodal behavior perception according to claim 1, characterized in that, The multimodal perception data includes the target user's real-time physiological parameters, the target user's behavioral perception parameters, and the real-time environmental parameters of the current home environment. The collection of multimodal perception data of the target user in the current home environment includes: Real-time physiological parameters of the target user are collected through wearable devices worn on the target user. Based on environmental sensors, real-time environmental parameters of the current home environment are collected; Based on the posture recognition device, the posture of the target user is identified, and the behavioral perception parameters of the target user are obtained.

3. The intelligent home environment control method based on multimodal behavior perception according to claim 1, characterized in that, The home habit benchmark values ​​include the physiological parameter benchmark values ​​and environmental parameter benchmark values ​​of the target users in different home environments; The acquisition of the target user's home habit baseline values ​​includes: Obtain historical home living habits data of target users; Based on the historical home habit data, at least one subset of home habit data corresponding to the home environment and at least one subset of physiological parameters corresponding to the health status of the target user are selected. Calculate the mean values ​​of environmental parameters in the subset of home habit data to obtain the baseline values ​​of environmental parameters corresponding to each home environment; Calculate the mean value of the physiological parameters in the subset of physiological parameters corresponding to the health state to obtain at least one baseline value of the physiological parameters corresponding to the health state.

4. The intelligent home environment control method based on multimodal behavior perception according to claim 1, characterized in that, The weight matrix corresponding to the current health status of the target user is used to perform weighted fusion calculation on the multimodal data deviation values ​​to generate a comfort deviation index for the target user in the current home environment, including: Based on the current health status of the target user, a weight matrix corresponding to the current health status is matched, and the weight coefficients corresponding to each of the multimodal deviation values ​​are determined. Based on the weighting coefficients corresponding to each of the multimodal deviation values, the multimodal deviation values ​​are weighted and summed to obtain the comfort deviation index.

5. The intelligent home environment control method based on multimodal behavior perception according to claim 1, characterized in that, After determining the intelligent control strategy for the target user in the current home environment based on the deviation level corresponding to the comfort deviation index, the method further includes: When the deviation level is greater than the first deviation level range, the control object and the target control index of the control object are determined based on the intelligent control strategy. Based on the index difference between the current index of the controlled object and the target controlled index, the single-step adjustment range of the controlled object is determined; The control object is gradually adjusted according to the single-step adjustment range until the control object is adjusted from the current index to the target control index.

6. The intelligent home environment control method based on multimodal behavior perception according to claim 5, characterized in that, After determining the single-step adjustment range of the controlled object based on the index difference between the current index of the controlled object and the target controlled index, the method further includes: The control object is gradually adjusted according to the single-step adjustment range. Each time the adjustment is performed, multimodal perception data of the target user in the current home environment is collected. Based on the multimodal perception data, calculate the comfort deviation index of the target user after each adjustment of the control object; If the current index of the controlled object is not adjusted to the target controlled index, and the deviation level corresponding to the comfort deviation index is less than the first deviation level range, the index adjustment operation of the controlled object shall be stopped.

7. The intelligent home environment control method based on multimodal behavior perception according to claim 1, characterized in that, The intelligent control strategy for determining the target user's home environment based on the deviation level corresponding to the comfort deviation index includes: Based on a preset grading threshold, the comfort deviation index is graded and evaluated to determine the deviation level corresponding to the comfort deviation index. Retrieve the intelligent control strategy corresponding to the deviation level from the predefined strategy library.

8. A smart home environment control device based on multimodal behavior perception, characterized in that, The intelligent home environment control device based on multimodal behavior perception includes: The data acquisition module is used to acquire baseline values ​​of the target user's home habits and to collect multimodal perception data of the target user in the current home environment; The deviation calculation module is used to calculate the difference between the multimodal perception data of the target user in the current home environment and its corresponding home habit benchmark value, and obtain the multimodal data deviation value. The deviation index generation module is used to perform weighted fusion calculation on the deviation values ​​of the multimodal data based on the weight matrix corresponding to the current health status of the target user, and generate the comfort deviation index of the target user in the current home environment. The control strategy determination module is used to determine the intelligent control strategy for the target user in the current home environment based on the deviation level corresponding to the comfort deviation index.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the intelligent home environment control method based on multimodal behavior perception as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the intelligent home environment control method based on multimodal behavior perception as described in any one of claims 1 to 7.

Images