Environment regulation and control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an environment regulation and control method and device, electronic equipment and a storage medium. The method comprises the steps that environment data collected in a target room and physiological data of a user are acquired; performing alignment processing on the environment data and the physiological data; the aligned environmental data and physiological data are input into a target cross-modal model, first environmental parameter adjustment values corresponding to the environmental data and the physiological data are output, and the target cross-modal model predicts the environmental parameter adjustment values by fusing the aligned environmental data and physiological data; and performing environment adjustment on the target room by adopting the first environment parameter adjustment value. Through the embodiment of the invention, the environmental data and the physiological data are aligned, and the target cross-modal model is adopted to realize fusion prediction of the first environmental parameter adjustment value, so that the room environment is accurately regulated and controlled.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an environment regulation method and device, electronic equipment and storage medium. BACKGROUND

[0002] The traditional intelligent bedroom system has the problems of data island and shallow decision-making limitation. The environment sensors (temperature and humidity, CO2) and physiological sensors (heart rate, respiration) adopt independent control logic, and lack cross-modal correlation analysis capability. The existing regulation strategies are mostly based on threshold judgment (such as starting cooling when the temperature is greater than 26℃), but the human thermal comfort is affected by multiple factors such as body mass index, daily exercise, sleep stage, etc. in a nonlinear manner, so it is difficult for a simple rule engine to achieve precise regulation. At the same time, some intelligent regulation adopts direct stacking of environment parameters and physiological signals into the model, resulting in feature redundancy and dimension disaster. SUMMARY

[0003] In view of the above problems, an environment regulation method and device, electronic equipment and storage medium are provided to overcome the above problems or at least partially solve the above problems, comprising:

[0004] An environment regulation method, the method comprising:

[0005] acquiring environment data and physiological data of a user collected in a target room;

[0006] aligning the environment data and the physiological data;

[0007] inputting the aligned environment data and physiological data into a target cross-modal model to output a first environment parameter adjustment value corresponding to the environment data and the physiological data, the target cross-modal model predicting the environment parameter adjustment value by fusing the aligned environment data and physiological data;

[0008] regulating the environment of the target room by using the first environment parameter adjustment value.

[0009] Optionally, further comprising:

[0010] inputting the aligned environment data and physiological data into a target cross-modal model to output first health risk prediction data corresponding to the environment data and the physiological data.

[0011] Optionally, further comprising:

[0012] controlling one or more home devices in the target room according to the health risk prediction data.

[0013] Optionally, the controlling one or more home devices in the target room according to the health risk prediction data comprises:

[0014] when the heart rate variability in the health risk prediction data is less than a preset variability threshold, controlling a mattress of the target room to locally heat to a preset temperature, and controlling a sound device of the target room to play alpha wave music.

[0015] Optionally, the controlling one or more home devices in the target room according to the health risk prediction data comprises:

[0016] when the heart rate in the health risk prediction data is greater than a preset heart rate and the blood oxygen is less than a preset value, turning on an oxygen injection device and notifying an associated user.

[0017] Optionally, the aligning the environmental data and the physiological data comprises:

[0018] acquiring timestamp data of the environmental data and the physiological data;

[0019] aligning the environmental data and the physiological data according to the timestamp data.

[0020] Optionally, the method further comprises:

[0021] acquiring sample data and an initial cross-modal model for model training;

[0022] inputting the sample data into the initial cross-modal model to output a second environmental parameter adjustment value and second health risk prediction data;

[0023] determining a loss function based on the second environmental parameter adjustment value and the second health risk prediction data;

[0024] updating model parameters of the initial cross-modal model according to the loss function.

[0025] Optionally, the determining a loss function based on the second environmental parameter adjustment value and the second health risk prediction data comprises:

[0026] acquiring user feedback data;

[0027] determining a loss function based on the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data.

[0028] 9. The method of claim 8, wherein the determining a loss function based on the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data comprises:

[0029] acquiring weight data corresponding to the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data;

[0030] The loss function is obtained by weighted summation of the user feedback data, the second environment parameter adjustment value and the second health risk prediction data according to the weight data.

[0031] An environment regulation device, the device comprising:

[0032] A data acquisition module is configured to acquire environment data and physiological data of a user collected in a target room.

[0033] A data alignment module is configured to align the environment data and the physiological data.

[0034] An environment parameter adjustment determination module is configured to input the aligned environment data and physiological data into a target cross-modal model to output a first environment parameter adjustment value corresponding to the environment data and the physiological data, the target cross-modal model being configured to predict the environment parameter adjustment value by fusing the aligned environment data and physiological data.

[0035] An environment adjustment module is configured to adjust the environment of the target room by using the first environment parameter adjustment value.

[0036] An electronic device comprises a processor, a memory, and a computer program stored on the memory and capable of running on the processor, the computer program being executed by the processor to implement the environment regulation method described above.

[0037] A computer readable storage medium stores a computer program, the computer program being executed by a processor to implement the environment regulation method described above.

[0038] The embodiments of the present application have the following advantages:

[0039] The embodiments of the present application acquire environment data and physiological data of a user collected in a target room, align the environment data and the physiological data, input the aligned environment data and physiological data into a target cross-modal model to output a first environment parameter adjustment value corresponding to the environment data and the physiological data, the target cross-modal model being configured to predict the environment parameter adjustment value by fusing the aligned environment data and physiological data, and adjust the environment of the target room by using the first environment parameter adjustment value, thereby aligning the environment data and the physiological data and using the target cross-modal model to implement fusion prediction of the first environment parameter adjustment value for precise regulation of the environment of the room. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the present application, the drawings needed to be used in the description of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0041] Figure 1 is a step flow chart of an environment regulation method provided by an embodiment of the present application;

[0042] Figure 2 is a step flow chart of another environment regulation method provided by an embodiment of the present application;

[0043] Figure 3 is a step flow chart of another environment regulation method provided by an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of an environment regulation architecture provided by an embodiment of the present application;

[0045] Figure 5 is a structural schematic diagram of an environment regulation device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the scope of protection of the present application.

[0047] Referring to Figure 1 , a step flow chart of an environment regulation method provided by an embodiment of the present application is shown, which can specifically include the following steps:

[0048] In step S101, environment data and physiological data of a user collected in a target room are acquired.

[0049] In actual application, a data collection device can be arranged in a specific space, and then the data in the space can be collected through the data collection device. Specifically, the data collection device can include a first data collection device for collecting environment data and a second data collection device for collecting physiological data.

[0050] In the embodiment of the present application, the collection space is a target room, and the target room can be a bedroom in which a user is present. The scenario to which the embodiment of the present application is directed can be a room environment control scenario when the user rests in the bedroom.

[0051] The environmental data can include any one or more of temperature, humidity, CO2, PM2.5, air flow rate and the like; the physiological data can include any one or more of heart rate, blood oxygen, respiration, turning action, body surface temperature distribution and the like.

[0052] In an embodiment of the present application, the home host can use a laser radar (air flow rate), CO2 concentration, PM2.5, ultrasonic humidity meter, pressure test and the like first data acquisition device to obtain the corresponding data type (specifically, the data type corresponding to the environmental data) in the target room, and then convert the obtained time series data into frequency domain features (such as 12-hour sliding mean of CO2 concentration) and volatility rate features (such as acceleration of temperature change), and further realize the construction of environmental data feature engineering.

[0053] In another embodiment of the present application, the home host can use a mattress piezoelectric sensor (for detecting the turning action of the user), infrared thermal imaging (for measuring the body surface temperature distribution), wearable device (for measuring heart rate, blood oxygen) and the like second data acquisition device to collect the corresponding data type, and perform optical flow denoising on the captured data to generate a 3D posture sequence, and realize the construction of physiological data feature engineering. In 3D posture estimation or motion capture, optical flow denoising can be used to smooth the original captured time series data (such as joint sequence), and thus reduce jitter or noise.

[0054] In step S102, the environmental data and the physiological data are aligned.

[0055] After obtaining the environmental data and the physiological data, the environmental data and the physiological data and the retrograde alignment data can be aligned, so that the environmental data and the physiological data are in the same period, so as to better realize the fusion of the environmental data and the physiological data. For unaligned environmental data and physiological data, it is difficult to establish the association between different structure data, and it is also difficult to perform fusion analysis on the two types of heterogeneous data of environmental data and physiological data.

[0056] In an embodiment of the present application, the alignment of the environmental data and the physiological data includes: obtaining the timestamp data of the environmental data and the physiological data; and aligning the environmental data and the physiological data according to the timestamp data.

[0057] In actual application, the home host can use the timestamp to realize the alignment of the heterogeneous data (i.e. the environmental data and the physiological data), specifically, the timestamp of each environmental data and physiological data can be further determined after obtaining the environmental data and the physiological data, which can be collected by the first data acquisition device or the second data acquisition device.

[0058] After determining the time stamp of each environmental data and physiological data, the time stamps of both can be matched and aligned in reverse, specifically, the first time stamp of the environmental data can be taken as a reference time stamp, and the time stamp in the second time stamp of the physiological data that is closest to each first time stamp can be determined, and in this way, the environmental data and the physiological data can be sequentially aligned.

[0059] In an embodiment of the present application, the first time stamp of the environmental data can be taken as a reference time stamp, and the second time stamp at the same time point as the first time stamp can be searched in the physiological data, when the second time stamp at the same time as the first time stamp is not found, then the first time stamp position can be interpolated according to the collected physiological data, and the interpolated value is the physiological data aligned with the physiological data at the time point of the first time stamp.

[0060] In an embodiment of the present application, before data collection, all sensor data can be calibrated by a high-precision RTC clock (error <1ms) to ensure time stamp synchronization. Specifically, the user's turning over, snoring and other preset events can be taken as anchor points, and event-driven alignment can be ensured by aligning environmental parameter changes (such as adjusting the air speed within 0.5 seconds after turning over).

[0061] In step S103, the aligned environmental data and physiological data are input into a target cross-modal model, and a first environmental parameter adjustment value corresponding to the environmental data and the physiological data is output, the target cross-modal model predicts the environmental parameter adjustment value by fusing the aligned environmental data and physiological data.

[0062] In an embodiment of the present application, a target cross-modal model can be pre-trained, which can realize prediction of environmental parameter adjustment value based on input multi-modal data, specifically, the fused aligned environmental data and physiological data can be input, and then the environmental parameter adjustment value is output.

[0063] The environmental parameter adjustment value can be a parameter value suitable for the target user in the current scene, in an embodiment of the present application, the training process of the target cross-modal model can include the following sub-steps:

[0064] In sub-step S11, sample data used for model training and an initial cross-modal model are obtained.

[0065] In actual application, related sample data in the scene in the embodiment of the present application can be obtained, which actually records the most suitable environmental parameter adjustment value corresponding to different physiological data and environmental data.

[0066] In the embodiment of the present application, the initial cross-modal model can select any network model, for example, the initial cross-modal model can be a Transformer model. The Transformer model is a deep learning architecture based on self-attention mechanism (Self-Attention), and the goal of the Transformer model is to capture long-range dependencies between elements in a sequence without relying on the locality restrictions of recurrent neural networks (RNN) or convolutional neural networks (CNN). In the embodiment of the present application, the Transformer model is mainly used to capture long-range dependencies between physiological data and environmental data.

[0067] In sub-step S12, the sample data is input into the initial cross-modal model to output a second environmental parameter adjustment value and second health risk prediction data.

[0068] After obtaining the sample data and the initial cross-modal model, the sample data can be input into the initial cross-modal model to obtain the output second environmental parameter adjustment value and second health risk prediction data.

[0069] In sub-step S13, a loss function is determined based on the second environmental parameter adjustment value and the second health risk prediction data.

[0070] In actual application, the second environmental parameter adjustment value and the second health risk prediction data can be used to construct a loss function. By constructing the loss function, the model parameters can be adjusted through feedback, and the model can be iteratively updated to achieve the expected effect.

[0071] In sub-step S14, the model parameters of the initial cross-modal model are updated according to the loss function.

[0072] After obtaining the loss function, the second environmental parameter adjustment value and the second health risk prediction data predicted by the model can be used as the target to adjust the model parameters, and the cross-modal model can be iteratively updated.

[0073] According to the above steps, multiple iterations are performed. When a preset iteration termination condition is triggered during the iteration process, the corresponding target cross-modal model is output.

[0074] In an embodiment of the present application, the cross-modal model can include an input module and a cross-attention module, wherein the input module can be further divided into an environmental branch and a physiological branch. In the environmental branch, a CNN-LSTM (Convolutional Neural Network-Long Short-Term Memory) hybrid model can be used to extract spatial-temporal features (such as temperature gradient map and CO2 concentration trend). In the physiological branch, a Graph Neural Network (GNN) can be used to model the correlation between body parts (such as the correlation between left arm heart rate and right arm blood pressure).

[0075] In an embodiment of the present application, the cross-modal model can further include a cross-attention mechanism module, in which Query-Value Attention is used to calculate the contribution weight of environmental features to physiological abnormalities (such as a weight coefficient of 0.7 for high humidity on breathing difficulty). Dynamic Weight Allocation can be used to dynamically adjust the modal weight according to the sleep stage (deep sleep period focuses on environmental stability, and REM period focuses on action comfort).

[0076] In an embodiment of the present application, multi-modal sensors can be used to obtain temperature, humidity, CO2, PM2.5, pressure, and other multi-source data sets, and environmental modal feature vectors can be obtained through spatio-temporal feature extraction. Non-contact sensors can be used to obtain breathing and heartbeat parameters, and piezoelectric film sensors can be used to obtain 3D posture modeling and GNN to build skeletal joints. Cross-modal fusion modeling is performed, and based on the bidirectional cross-attention mechanism and dynamic weight allocation, a Transformer-based attention calculation model is used to quantify the dynamic influence of various environmental parameters on physiological indicators. When the system detects that a certain indicator exceeds the attention weight, the corresponding purification strategy is triggered. At the same time, a reverse attention path is established to map abnormal physiological signals to environmental control requirements, and human information is used as an important parameter.

[0077] In an embodiment of the present application, the purification strategy can be adjusted according to different scenarios, for example:

[0078] Example 1: Scenario: CNN-LSTM detects that CO2 concentration is continuously >1200ppm (slope 0.8%↑ / min) PM2.5 sudden peak >75μg / m 3 (kitchen oil fume penetration)

[0079] Cross attention mechanism: CO2 weight on respiratory disorder 0.6 (REM stage physiological weight ratio 65%) PM2.5 weight on nasal mucosa irritation 0.9 Purification strategy adopts hierarchical air purification: (1) CO2 priority: start fresh air system (50% air volume) + sleep mode mute; (2) PM2.5 emergency response: close bedroom doors and windows (intelligent magnetic sealing) purifier turbine mode (30min) + negative ion deodorization; (3) Reverse path: when the piezoelectric film detects the frequency of cough vibration > 5Hz, automatically upgrade the purification level.

[0080] Example 2: Conflict scenario: environmental branch requires temperature rise (low temperature alarm) physiological branch detects sudden rise in palm skin conductance (overheating sweating signal) Dynamic weight distribution adopts zoned microenvironment construction: (1) Torso: carbon fiber heating film maintains 32℃ (core temperature protection in deep sleep stage); (2) Limb ends: local cooling with gel cooling pads (palm / sole -2℃); (3) Real-time verification of turning frequency through mattress pressure map, dynamically balancing thermal comfort.

[0081] In an embodiment of the present application, the loss function is determined based on the second environmental parameter adjustment value and the second health risk prediction data, comprising: obtaining user feedback data; determining the loss function based on the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data.

[0082] In actual application, the home host can also obtain user feedback data for this adjustment, and then can determine the loss function according to the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data. The feedback data can be a user satisfaction score for sleep quality.

[0083] Specifically, the weight data corresponding to the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data can be obtained first; then the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data can be weighted and summed according to the weight data to obtain the loss function.

[0084] That is, the loss function = weight data 1*user feedback data + weight data 2*second environmental parameter adjustment value + weight data 3*second health risk prediction data, wherein weight data 1 + weight data 2 + weight data 3 = 1, weight data 1 represents the weight data corresponding to the user feedback data, weight data 2 represents the weight data corresponding to the second environmental parameter adjustment value, and weight data 3 represents the weight data corresponding to the second health risk prediction data.

[0085] In an embodiment of the present application, multi-objective joint optimization can be performed by setting a loss function, wherein the loss function is designed as:

[0086] Loss = a * Regulation_Loss (environmental parameters) + b * Health_Loss (abnormality detection) + g * Comfort_Loss (user satisfaction score)

[0087] In an example, a = 0.4, b = 0.3, g = 0.3, wherein the weights can be dynamically adjusted according to different scenarios.

[0088] In an embodiment of the present application, the driving factors for dynamically adjusting the weights can include but are not limited to the following categories:

[0089] a) Sleep stage: sleep staging (deep sleep, REM, light sleep, wakefulness) through electroencephalogram (EEG), electrooculogram (EOG), etc. Different stages have different focuses on the three goals.

[0090] b) Physiological abnormality level: the associated abnormality score of each part of the body (0-1) output by the GNN model, the weight is adjusted according to the score classification.

[0091] c) User feedback: including morning APP satisfaction score and number of real-time motion interruptions at night.

[0092] In an embodiment of the present application, the dynamic adjustment strategy of the weights of the loss function is as follows:

[0093] a) Real-time adjustment of sleep stage: adjust the weights according to the current sleep stage probability distribution.

[0094] b) Physiological abnormality level adjustment: when a medium-high risk abnormality is detected, the b weight (health risk intervention) is increased, and the g (comfort) weight is reduced.

[0095] c) User feedback adjustment: according to the user satisfaction history, the g weight is adjusted for a long time through reinforcement learning.

[0096] The specific weight calculation process is as follows:

[0097] Let the weights at time t be a_t, b_t, g_t, and satisfy a_t + b_t + g_t = 1.

[0098] Adjustment process: a_t = f1 (current sleep stage, basic a, other factors) b_t = f2

[0099] (current sleep stage, physiological abnormality level, basic b, other factors) g_t = f3 (current sleep stage, user feedback, basic g, other factors)

[0100] The specific adjustment formula can be referred to as follows:

[0101] Sleep stage adjustment: According to the probability vector P = [p_deep, p REM, p light, p awake] output by the sleep stage classifier, and the weight coefficient vector corresponding to each stage (set by expert experience): α_t = (c_α_deep * p_deep + c_α REM * p REM + c_α_light * p_light + c_α_awake * p_awake) * α_base, calculate β_t and γ_t in the same way, and then normalize.

[0102] Physiological abnormality level adjustment: When the physiological abnormality score S_abnormal exceeds the threshold, emergency adjustment is performed: if S_abnormal > 0.6: β_t = β_t + 0.15, γ_t = γ_t - 0.10 (while ensuring that the weight is non-negative and the sum is 1, it needs to be re-normalized) if 0.3 <= S_abnormal <= 0.6: β_t = β_t + 0.07, γ_t = γ_t - 0.05.

[0103] User feedback adjustment: Based on the user satisfaction score (1-5 points), calculate the long-term average score avg_score. When the average score is less than 4 points, increase the γ weight: γ_t = γ_base + k * (1 / (1 + exp(-(5-avg_score))) # where k is the adjustment coefficient, such as 0.1 4.

[0104] The constraint condition is used to prevent the weight from mutating and causing the device to frequently disturb sleep. The weight change rate limit sets the maximum change rate per minute (such as 0.05).

[0105] Assuming that in deep sleep, the system focuses on environmental stability (higher α). At this time, if physiological abnormalities (such as respiratory abnormalities) are detected, β is increased and γ is decreased. At the same time, if the user recently feedbacks poor comfort, γ is appropriately increased, but is limited by health risks, and the increase of γ is limited. Through the above mechanism, the system can dynamically adjust the weights of the three targets according to the real-time situation, and realize multi-objective optimization.

[0106] In an embodiment of the present application, the sample data used for training can also be enhanced, for example:

[0107] (1) Apply disturbance to environmental parameters (such as simulate air conditioner failure to cause temperature to rise suddenly), improve model robustness.

[0108] (2) Use a generative adversarial network (GAN) to synthesize extreme scenario data (such as physiological signals when the user has a heart attack).

[0109] Step S104, adjusting the environment of the target room using the first environmental parameter adjustment value.

[0110] After obtaining the first environmental parameter adjustment value, the control instruction of the corresponding home device in the target room can be determined based on the first environmental parameter adjustment value, and then the home host can send the control instruction to each home device, and each home device executes the control instruction to adjust the target room to the expected value.

[0111] In the embodiment of the present application, personalized temperature adjustment can also be performed, and the specific manner is as follows:

[0112] When the user portrait is: body type: BMI 28kg / m 2 (need more air volume to dissipate heat). Health history: chronic rhinitis (sensitive to cold air).

[0113] Model adaptation: transfer learning fine-tuning - optimize the temperature-comfort mapping function using user historical data (10 nights of sleep records). Dynamic regulation - automatically adjust the night temperature from 22℃ to 24℃ (3℃ drop in apparent temperature), and turn on the air purification mode (HEPA filter).

[0114] User feedback: sleep quality score.

[0115] The above scheme is mainly based on the obtained air pressure, air circulation and user sleep quality in the bedroom, and the intelligent bedroom dynamic regulation system and method based on multi-modal deep learning fusion are used to open the devices and windows in the bedroom, etc., to ensure that the user's bedroom sleep environment is better. For example: if the air pressure in the bedroom is too high, the air circulation is poor, and the user's breathing is not smooth, and the heart rate is also unstable, then according to different user groups (old people, children, young people, etc.), the data obtained based on the above system will open the windows in the user's home for ventilation, and also open the air conditioner and air purifier during sleep to realize linkage, and provide intelligent and thoughtful service for the user's good sleep environment.

[0116] In the embodiment of the present application, the environmental data and the physiological data of the user collected in the target room can be obtained; the environmental data and the physiological data are aligned; the aligned environmental data and physiological data are input into a target cross-modal model, and a first environmental parameter adjustment value corresponding to the environmental data and the physiological data is output, the target cross-modal model predicts the environmental parameter adjustment value by fusing the aligned environmental data and physiological data; the target room is adjusted by using the first environmental parameter adjustment value, which realizes the alignment of the environmental data and the physiological data, and realizes the fusion prediction of the first environmental parameter adjustment value by using the target cross-modal model, to accurately regulate the room environment.

[0117] Referring to Figure 2 , a step flowchart of another environment regulation method provided by an embodiment of the present application is shown, which can specifically include the following steps:

[0118] Step S201, obtaining environmental data collected in a target room and physiological data of a user;

[0119] The data collection device can include a first data collection device for collecting environmental data and a second data collection device for collecting physiological data.

[0120] In the embodiments of the present application, the collected space is a target room, and the target room can be a bedroom where a user is present. The scenario targeted by the embodiments of the present application can be a room environment control scenario when the user rests in the bedroom.

[0121] The environmental data can include any one or more of temperature, humidity, CO2, PM2.5, air flow rate, etc. The physiological data can include any one or more of heart rate, blood oxygen, respiration, turning action, body surface temperature distribution, etc.

[0122] In an embodiment of the present application, the home host can use a laser radar (air flow rate), CO2 concentration, PM2.5, ultrasonic humidity meter, pressure test, etc. first data collection device to obtain corresponding data types (specifically, data types corresponding to environmental data) in the target room, and then convert the obtained time series data into frequency domain features (such as 12-hour sliding mean value of CO2 concentration) and volatility features (such as acceleration of temperature change), and further realize the construction of environmental data feature engineering.

[0123] In another embodiment of the present application, the home host can use a mattress piezoelectric sensor (for detecting the turning action of the user), an infrared thermal imaging (for measuring the body surface temperature distribution), a wearable device (for measuring the heart rate and blood oxygen), etc. second data collection device to collect corresponding data types, and perform optical flow denoising on the captured data to generate a 3D pose sequence, and realize the construction of physiological data feature engineering. In 3D pose estimation or motion capture, optical flow denoising can be used to smooth the original captured time series data (such as joint sequence), and thus reduce jitter or noise.

[0124] Step S202, aligning the environmental data and the physiological data;

[0125] After obtaining the environmental data and the physiological data, the environmental data and the physiological data and the reverse alignment data can be aligned, so that the environmental data and the physiological data are in the same period, so as to better realize the fusion of the environmental data and the physiological data. For unaligned environmental data and physiological data, it is difficult to establish the association between different structural data, and it is also difficult to perform fusion analysis on the two types of heterogeneous data of environmental data and physiological data.

[0126] In step S203, the aligned environmental data and physiological data are input into a target cross-modal model, and a first environmental parameter adjustment value corresponding to the environmental data and the physiological data is output, the target cross-modal model predicting the environmental parameter adjustment value by fusing the aligned environmental data and physiological data.

[0127] In the embodiment of the present application, a target cross-modal model can be pre-trained, which can predict the environmental parameter adjustment value based on the input multi-modal data. Specifically, the aligned environmental data and physiological data can be input, and then the environmental parameter adjustment value is output.

[0128] In step S204, the target room is adjusted in the environment by using the first environmental parameter adjustment value.

[0129] In step S205, the aligned environmental data and physiological data are input into a target cross-modal model, and a first health risk prediction data corresponding to the environmental data and the physiological data is output.

[0130] In actual application, the target cross-modal model can also analyze the input environmental data and physiological data to determine the corresponding first health risk prediction data.

[0131] The first health risk prediction data can be applied to analyze the health risk of the user, so as to prevent the risk in advance and ensure the safety of the user.

[0132] In the embodiment of the present application, the environmental data and physiological data collected in the target room can be obtained, the environmental data and the physiological data are aligned, the aligned environmental data and physiological data are input into a target cross-modal model, and a first environmental parameter adjustment value corresponding to the environmental data and the physiological data is output, the target cross-modal model predicting the environmental parameter adjustment value and the first health risk prediction data by fusing the aligned environmental data and physiological data, and the target room is adjusted in the environment by using the first environmental parameter adjustment value, which realizes the alignment of the environmental data and the physiological data, and the fusion prediction of the first environmental parameter adjustment value and the first health risk prediction data by using the target cross-modal model, so as to accurately control the room environment.

[0133] Referring to Figure 3 FIG. 3 shows a step flowchart of another environmental control method provided by an embodiment of the present application, which can specifically include the following steps:

[0134] In step S301, environmental data and physiological data of a user collected in a target room are obtained.

[0135] The space can be used to collect data in the space, specifically, the data collection device can include a first data collection device for collecting environmental data and a second data collection device for collecting physiological data.

[0136] In the embodiment of the application, the space for which data is collected is a target room, and the target room can be a bedroom in which a person is present. The scenario to which the embodiment of the application is directed can be a room environment control scenario when a user is resting in a bedroom.

[0137] The environmental data can include any one or more of temperature, humidity, CO2, PM2.5, air flow rate, etc. The physiological data can include any one or more of heart rate, blood oxygen, respiration, turning-over action, body surface temperature distribution, etc.

[0138] In an embodiment of the application, the home host can use a laser radar (air flow rate), CO2 concentration, PM2.5, ultrasonic humidity meter, pressure test, etc. first data collection device to obtain corresponding data types (specifically, data types corresponding to environmental data) in the target room, and then convert the obtained time series data into frequency domain features (such as a 12-hour sliding mean of CO2 concentration) and volatility features (such as the acceleration of temperature change), thereby realizing the construction of environmental data feature engineering.

[0139] In another embodiment of the application, the home host can use a mattress piezoelectric sensor (for detecting the turning-over action of a user), infrared thermal imaging (for measuring body surface temperature distribution), wearable devices (for measuring heart rate and blood oxygen), etc. second data collection device to collect corresponding data types, and perform optical flow denoising on the captured data obtained, to generate a 3D posture sequence, thereby realizing the construction of physiological data feature engineering. In 3D posture estimation or motion capture, optical flow denoising can be used to smooth the original captured time series data (such as a joint sequence), thereby reducing jitter or noise.

[0140] In step S302, the environmental data and the physiological data are aligned.

[0141] After obtaining the environmental data and the physiological data, the environmental data and the physiological data and the retrograde alignment data can be aligned, so that the environmental data and the physiological data are in the same period, to facilitate better fusion of the environmental data and the physiological data. For unaligned environmental data and physiological data, it is difficult to establish a correlation between different structural data, and it is also difficult to perform fusion analysis on the two types of heterogeneous data, i.e., the environmental data and the physiological data.

[0142] Step S303, input the aligned environmental data and physiological data into a target cross-modal model, output a first environmental parameter adjustment value corresponding to the environmental data and the physiological data, and the target cross-modal model predicts the environmental parameter adjustment value by fusing the aligned environmental data and physiological data.

[0143] In an embodiment of the present application, a target cross-modal model can be pre-trained, which can predict the environmental parameter adjustment value based on the input multi-modal data. Specifically, the aligned environmental data and physiological data can be inputted, and then the environmental parameter adjustment value is outputted.

[0144] Step S304, using the first environmental parameter adjustment value to adjust the environment of the target room.

[0145] Step S305, input the aligned environmental data and physiological data into a target cross-modal model, output a first health risk prediction data corresponding to the environmental data and the physiological data.

[0146] Step S306, controlling one or more home devices in the target room according to the first health risk prediction data.

[0147] After determining the first health risk prediction data, one or more home devices in the target room can be controlled according to the first health risk prediction data to cope with the health risk and provide a safe sleep environment for the user.

[0148] In an embodiment of the present application, the controlling one or more home devices in the target room according to the health risk prediction data comprises: when the heart rate variability in the health risk prediction data is less than a preset variability threshold, controlling the mattress in the target room to locally heat to a preset temperature, and controlling the sound equipment in the target room to play alpha wave music.

[0149] Alpha wave music is a special type of music or sound frequency designed to guide or enhance the alpha brain waves of the listener's brain through specific sound wave frequencies (usually 8-13 Hz), helping to relax, reduce stress, improve focus or promote meditation.

[0150] For example, based on the Binary Cross-Entropy loss function, 12 kinds of sleep disorders (such as insomnia, sleep apnea) are classified when HRV < 50 ms, the mattress is locally heated to 38℃ to promote blood circulation, and alpha wave music (frequency 8-13 Hz) is played.

[0151] In an embodiment of the present application, the control of the one or more home devices in the target room according to the health risk prediction data comprises: when the heart rate in the health risk prediction data is greater than a preset heart rate and the blood oxygen is less than a preset value, turning on an oxygen injection device and notifying the associated user.

[0152] For example, when the heart rate is > 140 bpm and the blood oxygen is < 90%, the oxygen injection device is automatically turned on (flow rate 3L / min), and the emergency contact phone is dialed.

[0153] The oxygen injection device is a device that concentrates and injects oxygen through high-pressure or high-speed airflow, which can be used in medical, industrial or scientific research fields to achieve rapid oxygen supply, cutting, combustion enhancement or other specific functions.

[0154] In an embodiment of the present application, the environment data and the physiological data collected in the target room are obtained; the environment data and the physiological data are aligned; the aligned environment data and physiological data are input into a target cross-modal model to output first environment parameter adjustment values corresponding to the environment data and the physiological data, the target cross-modal model predicts environment parameter adjustment values and first health risk prediction data by fusing the aligned environment data and physiological data; the target room is adjusted using the first environment parameter adjustment values, and one or more home devices in the target room are controlled according to the health risk prediction data, which realizes the alignment of environment data and physiological data, and realizes the fusion of the target cross-modal model to predict the first environment parameter adjustment value and the first health risk prediction data, to accurately control the room environment and ensure user safety.

[0155] Referring to Figure 4 , an environment regulation architecture schematic diagram in an embodiment of the present application is shown, which can specifically include a multi-modal data acquisition layer, a data preprocessing module, an edge computing and fusion layer, a cross-modal Transformer architecture, an intelligent decision and execution layer, and a user feedback and model updating layer.

[0156] The multi-modal data acquisition layer can include an environment sensor and a physiological sensor; the environment sensor is used to collect environment data of a room where a user sleeps, and the physiological sensor is used to collect physiological data of the user.

[0157] The data preprocessing module can perform denoising processing on the collected environment data and physiological data to reduce data noise interference and improve the accuracy of data prediction, wherein the denoising method can include any one or more of wavelet denoising and optical flow denoising.

[0158] The edge computing and fusion layer is mainly used for spatio-temporal alignment and feature extraction between physiological data and environmental data, so as to facilitate subsequent analysis of the fusion of the physiological data and the environmental data.

[0159] The cross-modal Transformer architecture is a module for actually performing prediction, and can include cross-modal attention and dynamic weight distribution. Through the architecture, corresponding environmental reference adjustment values and health risk prediction values can be output based on the input physiological data and environmental data.

[0160] The intelligent decision and execution layer controls one or more furniture devices in the room to perform regulation and control and countermeasures for health intervention based on the environmental reference adjustment values and the health risk prediction values output by the cross-modal Transformer architecture, so as to provide a safe and healthy environment for the user during sleep.

[0161] The user feedback and model updating layer can further optimize the model by combining the feedback of the user on the actual regulation and control, so as to provide personalized services that are more suitable for the user.

[0162] It should be noted that, for the method embodiment, in order to simply describe, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the order of the described actions, because according to the embodiments of the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the present application.

[0163] Referring to Figure 5 , a structural schematic diagram of an environment regulation device provided by an embodiment of the present application is shown, which can specifically include the following modules:

[0164] The data acquisition module 501 is configured to acquire environmental data and physiological data of a user collected in a target room.

[0165] The data alignment module 502 is configured to perform alignment processing on the environmental data and the physiological data.

[0166] The environmental parameter adjustment determination module 503 is configured to input the aligned environmental data and physiological data into a target cross-modal model, and output first environmental parameter adjustment values corresponding to the environmental data and the physiological data, wherein the target cross-modal model predicts the environmental parameter adjustment values by fusing the aligned environmental data and physiological data.

[0167] The environmental adjustment module 504 is configured to perform environmental adjustment on the target room by using the first environmental parameter adjustment values.

[0168] In an embodiment of the present application, the device can further include:

[0169] a health risk prediction data output module, configured to input the aligned environmental data and physiological data into a target cross-modal model, and output first health risk prediction data corresponding to the environmental data and the physiological data.

[0170] In an embodiment of the present application, the device can further include:

[0171] a home device control module, configured to control one or more home devices in the target room according to the health risk prediction data.

[0172] In an embodiment of the present application, the home device control module can include:

[0173] a first control submodule, configured to control a mattress in the target room to locally heat to a preset temperature and control a sound device in the target room to play alpha wave music when the heart rate variability in the health risk prediction data is less than a preset variability threshold.

[0174] In an embodiment of the present application, the home device control module can include:

[0175] a second control submodule, configured to turn on an oxygen injection device and notify an associated user when the heart rate in the health risk prediction data is greater than a preset heart rate and the blood oxygen is less than a preset value.

[0176] In an embodiment of the present application, the data alignment module 502 can include:

[0177] a timestamp data acquisition submodule, configured to acquire timestamp data of the environmental data and the physiological data;

[0178] an alignment processing submodule, configured to perform alignment processing on the environmental data and the physiological data according to the timestamp data.

[0179] In an embodiment of the present application, the device can further include:

[0180] a sample data acquisition module, configured to acquire sample data used for model training and an initial cross-modal model;

[0181] a training module, configured to input the sample data into the initial cross-modal model, and output a second environmental parameter adjustment value and second health risk prediction data;

[0182] a loss function determination module, configured to determine a loss function based on the second environmental parameter adjustment value and the second health risk prediction data;

[0183] a model parameter update module, configured to update model parameters of the initial cross-modal model according to the loss function.

[0184] In an embodiment of the present application, the loss function determination module can comprise:

[0185] a feedback data acquisition submodule, configured to acquire user feedback data;

[0186] a loss function determination submodule, configured to determine a loss function based on the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data.

[0187] In an embodiment of the present application, the loss function determination submodule can comprise:

[0188] a weight data acquisition unit, configured to acquire weight data corresponding to the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data;

[0189] a loss function determination unit, configured to perform weighted summation on the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data according to the weight data, to obtain a loss function.

[0190] In an embodiment of the present application, by acquiring environmental data and physiological data of a user collected in a target room, the environmental data and the physiological data are aligned, and the aligned environmental data and physiological data are input into a target cross-modal model to output a first environmental parameter adjustment value corresponding to the environmental data and the physiological data, the target cross-modal model predicting the environmental parameter adjustment value by fusing the aligned environmental data and physiological data; the target room is adjusted in the environment by using the first environmental parameter adjustment value, thereby realizing alignment of the environmental data and the physiological data, and realizing fusion prediction of the first environmental parameter adjustment value by using the target cross-modal model to accurately control the room environment.

[0191] An embodiment of the present application further provides an electronic device, which can comprise a processor, a memory, and a computer program stored on the memory and capable of running on the processor, and the computer program is executed by the processor to implement the above-mentioned environment control method.

[0192] An embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the above-mentioned environment control method.

[0193] For the device embodiment, it is basically similar to the method embodiment, so the description is relatively simple, and the related parts refer to the part of the method embodiment.

[0194] The various embodiments described in this specification are intended to be illustrative only. Each embodiment was chosen for illustration only, and not as a limitation of the scope of the disclosure. Numerous alternatives not specifically set forth herein will be apparent in view of the teachings herein.

[0195] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In a software embodiment, various embodiments of the present application can be implemented by computer software programs or

[0196] Embodiments of the present application are described herein with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0197] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing terminal devices to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal devices to cause a series of operational steps to be performed on the computer or other programmable terminal devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal devices provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0199] While preferred embodiments of the application have been described, those skilled in the art will appreciate that other modifications than those specifically described can be made within the scope of the application. Accordingly, the appended claims are intended to embrace all such alternatives as well as the embodiments specifically described.

[0200] Finally, it should be noted that, in the description above, relative terms such as first and second, etc. are used merely to distinguish one entity or action from another, without necessarily implying any actual relationship or order between such entities or actions. Also, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0201] The above provides an environment regulation method and device, electronic equipment and storage medium, and the principle and implementation of the present application are described by applying specific examples in the description. The above description of the embodiments is only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, the specific implementation and application range can be changed according to the idea of the present application. In conclusion, the content of the description should not be understood as a limitation of the present application.

Claims

1. An environmental control method, characterized in that, The method includes: Acquire environmental data and user physiological data within the target room; The environmental data and the physiological data are aligned. Aligned environmental and physiological data are input into a target cross-modal model, and the first environmental parameter adjustment value corresponding to the environmental and physiological data is output. The target cross-modal model predicts the environmental parameter adjustment value by fusing the aligned environmental and physiological data. The target room is environmentally adjusted using the first environmental parameter adjustment value.

2. The method according to claim 1, characterized in that, Also includes: The aligned environmental and physiological data are input into the target cross-modal model, and the first health risk prediction data corresponding to the environmental and physiological data is output.

3. The method according to claim 1, characterized in that, Also includes: Control one or more home appliances in the target room based on the health risk prediction data.

4. The method according to claim 3, characterized in that, The method of controlling one or more home appliances in the target room based on the health risk prediction data includes: When the heart rate variability in the health risk prediction data is less than a preset variability threshold, the mattress in the target room is locally heated to a preset temperature, and the audio equipment in the target room is controlled to play alpha wave music.

5. The method according to claim 3, characterized in that, The method of controlling one or more home appliances in the target room based on the health risk prediction data includes: When the heart rate in the health risk prediction data is greater than the preset heart rate and the blood oxygen is less than the preset value, the oxygen injection device is activated and the associated user is notified.

6. The method according to claim 1, characterized in that, The process of aligning the environmental data and the physiological data includes: Obtain the timestamp data of the environmental data and the physiological data; The environmental data and the physiological data are aligned based on the timestamp data.

7. The method according to claim 1, characterized in that, Also includes: Obtain sample data and an initial cross-modal model for model training; The sample data is input into the initial cross-modal model, and the second environmental parameter adjustment value and the second health risk prediction data are output. The loss function is determined based on the second environmental parameter adjustment value and the second health risk prediction data; The model parameters of the initial cross-modal model are updated according to the loss function.

8. The method according to claim 7, characterized in that, The step of determining the loss function based on the second environmental parameter adjustment value and the second health risk prediction data includes: Obtain user feedback data; The loss function is determined based on the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data.

9. The method according to claim 8, characterized in that, The step of determining the loss function based on user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data includes: Obtain the weighted data corresponding to the user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data; The user feedback data, the second environmental parameter adjustment value, and the second health risk prediction data are weighted and summed according to the weighted data to obtain the loss function.

10. An environmental control device, characterized in that, The device includes: The data acquisition module is used to acquire environmental data and user physiological data collected in the target room; A data alignment module is used to align the environmental data and the physiological data. An environmental parameter adjustment determination module is used to input aligned environmental data and physiological data into a target cross-modal model and output a first environmental parameter adjustment value corresponding to the environmental data and the physiological data. The target cross-modal model predicts the environmental parameter adjustment value by fusing the aligned environmental data and physiological data. An environment adjustment module is used to adjust the environment of the target room using the first environmental parameter adjustment value.

11. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the environmental control method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the environmental control method as described in any one of claims 1 to 9.

Images