A home environment self-adaptive adjustment method and system based on multi-modal physiological perception and end-cloud cooperation

Abstract

The application relates to a kind of based on multi-modal physiological perception and end cloud collaborative home environment adaptive adjustment method and system, the method comprises: the micro-motion signal of user in target area is collected by non-contact sensing device, and the physiological characteristic index of user is extracted accordingly;Real-time monitoring user's explicit intervention behavior to home environment equipment, and generating corresponding explicit feedback label;According to the stationary deviation index, implicit comfort feedback label is generated, and according to the confidence of explicit feedback label, the fusion weight of implicit comfort feedback label is dynamically adjusted, and target training sample is generated;Based on target training sample, the parameter of local environment parameter mapping model is updated using online incremental learning strategy;According to the output control instruction of updated model, the home environment equipment is adjusted, and the physiological characteristic index change in the preset time period after adjustment is continuously monitored to form verification closed loop.The application can realize the intelligent adjustment of home environment comfort.

Description

Technical Field

[0001] This invention belongs to the field of smart home and artificial intelligence technology, and in particular relates to a method and system for adaptive adjustment of the home environment based on multimodal physiological perception and edge-cloud collaboration. Background Technology

[0002] Existing smart home environmental control systems (such as smart air conditioners) primarily rely on manual user settings or simple temperature sensor feedback. This approach suffers from latency and cannot perceive the user's actual physiological thermal comfort. While some high-end devices incorporate infrared human body sensors, they can only determine whether someone is present, not whether the user is "cold" or "hot," nor can they automatically fine-tune environmental parameters based on the user's sleep depth or physiological stress. Furthermore, existing cloud-based control solutions often pose privacy risks. Summary of the Invention

[0003] The purpose of this invention is to provide a home environment adaptive adjustment method and system based on multimodal physiological perception and edge-cloud collaboration. It extracts the user's breathing stability as implicit feedback through non-contact devices such as millimeter-wave radar, and combines this with explicit feedback such as user voice / keystrokes. Online incremental learning is used to continuously optimize the local environment control model. Furthermore, negative sample mining and federated learning mechanisms are introduced to achieve intelligent adjustment that "understands you better the more you use it," while protecting privacy.

[0004] This invention provides a home environment adaptive adjustment method based on multimodal physiological perception and edge-cloud collaboration, comprising the following steps: Step 1, Physiological Feature Extraction: The system collects micro-motion signals of users within a target area using non-contact sensing devices, and extracts physiological characteristic indicators of users based on these micro-motion signals; the physiological characteristic indicators include at least respiratory rate and its stability deviation relative to the user's resting baseline value. Step 2, Explicit Feedback Capture: Real-time monitoring of users' explicit intervention behaviors towards home environment devices; when explicit intervention is detected, corresponding explicit feedback tags are generated. Step 3, Dynamic Weighted Fusion: Implicit comfort feedback labels are generated based on the stability deviation index, and the fusion weight between the explicit feedback labels and the implicit comfort feedback labels is dynamically adjusted based on the confidence level of the explicit feedback labels to generate target training samples. Step 4, Local Adaptive Update: Based on the target training samples, the parameters of the local environment parameter mapping model are updated using an online incremental learning strategy. Step 5, Closed-loop adjustment and verification: Based on the updated model, output control commands to adjust the home environment equipment, and continuously monitor changes in physiological characteristic indicators within a preset time period after adjustment to form a verification closed loop.

[0005] Furthermore, the non-contact sensing device is a millimeter-wave radar or an ultra-wideband radar.

[0006] Furthermore, the step of extracting the user's physiological characteristic indicators in step 1 includes: Bandpass filtering and spectrum analysis are performed on the phase signal of the radar echo. After removing large-amplitude body motion interference, the standard deviation of the respiratory rate within the current time window is calculated, and the ratio of this standard deviation to the baseline standard deviation of the user's historical resting state is calculated as the stability deviation index.

[0007] Furthermore, step 2 also includes: Reverse intervention monitoring mechanism: If, after the adjustment described in step 5, an explicit intervention opposite to the adjustment direction is detected within the preset regret monitoring time window, the adjustment operation in step 5 is marked as a negative sample, and a punitive correction is applied to the parameter weights that caused the adjustment operation during the next model update.

[0008] Furthermore, the method also includes a multi-user conflict handling strategy: when the non-contact sensing device detects multiple users in the target area, the stability deviation index of each user is calculated separately; if the environmental needs of different users conflict, the adjustment direction corresponding to the user with the highest stability deviation index is responded to first, or the user command that issues explicit intervention is responded to first.

[0009] Furthermore, the online incremental learning strategy in step 4 employs a gradient descent-based parameter optimization algorithm, which includes at least one of stochastic gradient descent, momentum optimization, or adaptive moment estimation, and does not require uploading original user data.

[0010] Furthermore, the method also includes an end-to-end cloud collaboration step: The gradient parameters updated on the local model are anonymized and encrypted before being uploaded to the cloud server; Receive global model parameters aggregated by the cloud server based on the federated learning algorithm; The local model is updated using the global model parameters to incorporate cross-device environmental tuning experience.

[0011] The present invention also provides a home environment regulation system, comprising: The sensing module is used to collect the micro-motion signals of users within the target area using a non-contact sensing method; The interaction module is used to monitor users' explicit intervention behaviors towards home environment devices in real time; A processing module, configured to execute the described method; An execution module, communicatively connected to the home environment device, for performing environmental parameter adjustment.

[0012] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the home environment adaptive adjustment method based on multi-modal physiological perception and end-cloud collaboration.

[0013] With the above solution, through the home environment adaptive adjustment method and system based on multi-modal physiological perception and end-cloud collaboration, the breathing smoothness of the user is extracted as an implicit feedback through non-contact devices such as millimeter-wave radars, combined with explicit feedback such as the user's voice / keystrokes, and the local environment control model is continuously optimized using online incremental learning, and a negative sample mining and federated learning mechanism is introduced, which can achieve intelligent adjustment of the home environment comfort under the premise of protecting privacy.

[0014] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly and implement it according to the content of the specification, the following takes the preferred embodiments of the present invention and describes them in detail with the accompanying drawings as follows. Description of the Drawings

[0015] Figure 1 is a flowchart of the home environment adaptive adjustment method based on multi-modal physiological perception and end-cloud collaboration of the present invention; Figure 2 is a system architecture block diagram provided by an embodiment of the present invention; Figure 3 is a flowchart of the closed-loop adaptive adjustment method provided by an embodiment of the present invention. Detailed Embodiments

[0016] The following combines the drawings and embodiments to further describe the detailed embodiments of the present invention in detail. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0017] As shown in Figure 1 This embodiment provides a home environment adaptive adjustment method based on multi-modal physiological perception and end-cloud collaboration, including the following steps: Step S1, physiological feature extraction: Collect the micro-motion signals of the user in the target area through a non-contact sensing device, and extract the physiological feature indicators of the user based on the micro-motion signals; the physiological feature indicators at least include the breathing frequency and the smoothness deviation index relative to the user's resting reference value; Step S2, explicit feedback capture: Real-time monitoring of users' explicit intervention behaviors towards home environment devices; when explicit intervention is detected, corresponding explicit feedback tags are generated. Step S3, Dynamic Weighted Fusion: Implicit comfort feedback labels are generated based on the stability deviation index, and the fusion weight between the explicit feedback labels and the implicit comfort feedback labels is dynamically adjusted based on the confidence level of the explicit feedback labels to generate target training samples. Step S4, Local Adaptive Update: Based on the target training samples, the parameters of the local environment parameter mapping model are updated using an online incremental learning strategy. Step S5, Closed-loop adjustment and verification: Based on the updated model, output control commands to adjust the home environment equipment, and continuously monitor changes in physiological characteristic indicators within a preset time period after adjustment to form a verification closed loop.

[0018] In this embodiment, the non-contact sensing device is a millimeter-wave radar or an ultra-wideband (UWB) radar.

[0019] In this embodiment, the step of extracting the user's physiological characteristic indicators in step S1 includes: Bandpass filtering and spectrum analysis are performed on the phase signal of the radar echo. After removing large-amplitude body motion interference, the standard deviation of the respiratory rate within the current time window is calculated, and the ratio of this standard deviation to the baseline standard deviation of the user's historical resting state is calculated as the stability deviation index.

[0020] In this embodiment, step S2 further includes: Reverse intervention monitoring mechanism: If, after the adjustment described in step S5, an explicit intervention opposite to the adjustment direction is detected within a preset regret monitoring time window, the adjustment operation in step S5 is marked as a negative sample, and a punitive correction is applied to the parameter weights that caused the adjustment operation during the next model update.

[0021] In this embodiment, the method further includes a multi-user conflict handling strategy: when the non-contact sensing device detects multiple users in the target area, the stability deviation index of each user is calculated respectively; if the environmental needs of different users conflict, the adjustment direction corresponding to the user with the highest stability deviation index is responded to first, or the user command that issues explicit intervention is responded to first.

[0022] In this embodiment, the online incremental learning strategy in step S4 adopts a gradient descent-based parameter optimization algorithm, which includes at least one of stochastic gradient descent (SGD), momentum optimization, or adaptive moment estimation (Adam), and does not require uploading original user data.

[0023] In this embodiment, the method further includes an end-to-cloud collaboration step: The gradient parameters updated on the local model are anonymized and encrypted before being uploaded to the cloud server; Receive global model parameters aggregated by the cloud server based on the federated learning algorithm; The local model is updated using the global model parameters to incorporate cross-device environmental tuning experience.

[0024] The present invention also provides a home environment regulation system, comprising: The sensing module is used to collect the micro-motion signals of users within the target area using a non-contact sensing method; The interaction module is used to monitor users' explicit intervention behaviors towards home environment devices in real time; The processing module is configured to execute the method described above; The execution module communicates with home environment devices to perform environmental parameter adjustments.

[0025] Figure 2 The data flow of the system in one embodiment is illustrated. Data from millimeter-wave radar and environmental sensors is fed into the edge computing layer, where it undergoes physiological feature extraction and feedback analysis to drive local model updates. The model outputs control commands to the execution layer devices, while also supporting optional cloud-based federated learning interaction.

[0026] Figure 3 The core process of logical judgment in the adjustment method is demonstrated. It highlights the error correction mechanism that generates negative samples when "reverse intervention" is detected, as well as the processing logic when explicit and implicit feedback coexist, ultimately forming a closed loop of "perception-decision-execution-verification".

[0027] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned home environment adaptive adjustment method based on multimodal physiological perception and edge-cloud collaboration.

[0028] Example 1: Multimodal Perception and Feature Extraction The system uses a 60GHz FMCW millimeter-wave radar as the main sensing unit. The radar extracts the target's range, velocity, and angle information by transmitting frequency-modulated continuous waves and receiving the echoes.

[0029] Breath extraction: This involves analyzing the phase signal of a static human target. Bandpass filtering is used to extract the breathing waveform.

[0030] Stability calculation: Set a sliding time window (e.g., 30 seconds) and calculate the coefficient of variation (CV) of respiratory rate within the window. If the CV value is significantly higher than the user's sleep baseline, it is judged as an "uncomfortable / restless" state (implicit negative feedback); if it is close to the baseline, it is judged as a "comfortable" state (implicit positive feedback).

[0031] Example 2: Multi-user conflicts and weight allocation In a living room scenario, if two users are detected: User A is reading (breathing steadily), and User B is sleeping but breathing rapidly and tossing and turning frequently (indicating heat discomfort).

[0032] Conflict handling: The system recognizes that user B has a high degree of physiological deviation, so it assigns user B a higher adjustment weight and automatically lowers the air conditioning temperature or turns on the fresh air.

[0033] Explicit priority: If user A feels cold and manually increases the temperature (explicit intervention), the system immediately responds to user A's instruction and records the conflict, learning in subsequent models that "user A has the dominant role in this time period / scenario".

[0034] Example 3: Negative Sample Mining Based on Reverse Intervention (Error Correction Mechanism) Assuming the system determines that the user has entered sleep mode, it automatically adjusts the room temperature from... Down to .

[0035] Regret detection: If the user wakes up within 5 minutes of adjusting the temperature and manually adjusts it back... .

[0036] Model correction: The system detects this "reverse intervention" and marks the previous "cooling action" as a negative sample. In the next gradient update, the penalty term for this erroneous action in the loss function is increased, thereby preventing the system from incorrectly cooling again under similar conditions.

[0037] Example 4: End-to-Cloud Collaboration Local devices only upload gradient information updated in the model, rather than the original radar data or user recordings. The cloud server aggregates the gradients from each household using a federated averaging algorithm (FedAvg) to generate a more general environmental control model, which is then distributed to local devices to solve the "cold start" problem for new devices.

[0038] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for adaptive adjustment of the home environment based on multimodal physiological perception and edge-cloud collaboration, characterized in that, Includes the following steps: Step 1, Physiological Feature Extraction: The system collects micro-motion signals of users within a target area using non-contact sensing devices, and extracts physiological characteristic indicators of users based on these micro-motion signals; the physiological characteristic indicators include at least respiratory rate and its stability deviation relative to the user's resting baseline value. Step 2, Explicit Feedback Capture: Real-time monitoring of users' explicit intervention behaviors towards home environment devices; when explicit intervention is detected, corresponding explicit feedback tags are generated. Step 3, Dynamic Weighted Fusion: Implicit comfort feedback labels are generated based on the stability deviation index, and the fusion weight between the explicit feedback labels and the implicit comfort feedback labels is dynamically adjusted based on the confidence level of the explicit feedback labels to generate target training samples. Step 4, Local Adaptive Update: Based on the target training samples, the parameters of the local environment parameter mapping model are updated using an online incremental learning strategy. Step 5, Closed-loop adjustment and verification: Based on the updated model, output control commands to adjust the home environment equipment, and continuously monitor changes in physiological characteristic indicators within a preset time period after adjustment to form a verification closed loop.

2. The adaptive adjustment method for home environment based on multimodal physiological perception and edge-cloud collaboration as described in claim 1, characterized in that, The non-contact sensing device is a millimeter-wave radar or an ultra-wideband radar.

3. The adaptive adjustment method for home environment based on multimodal physiological perception and edge-cloud collaboration according to claim 2, characterized in that, Step 1, which involves extracting the user's physiological characteristic indicators, includes: Bandpass filtering and spectrum analysis are performed on the phase signal of the radar echo. After removing large-amplitude body motion interference, the standard deviation of the respiratory rate within the current time window is calculated, and the ratio of this standard deviation to the baseline standard deviation of the user's historical resting state is calculated as the stability deviation index.

4. The adaptive adjustment method for home environment based on multimodal physiological perception and edge-cloud collaboration as described in claim 1, characterized in that, Step 2 also includes: Reverse intervention monitoring mechanism: If, after the adjustment described in step 5, an explicit intervention opposite to the adjustment direction is detected within the preset regret monitoring time window, the adjustment operation in step 5 is marked as a negative sample, and a punitive correction is applied to the parameter weights that caused the adjustment operation during the next model update.

5. The adaptive adjustment method for home environment based on multimodal physiological perception and edge-cloud collaboration as described in claim 1, characterized in that, The method also includes a multi-user conflict handling strategy: when the non-contact sensing device detects multiple users in the target area, the stability deviation index of each user is calculated separately; if the environmental needs of different users conflict, the adjustment direction corresponding to the user with the highest stability deviation index is responded to first, or the user command that issues explicit intervention is responded to first.

6. The adaptive adjustment method for home environment based on multimodal physiological perception and edge-cloud collaboration as described in claim 1, characterized in that, The online incremental learning strategy in step 4 employs a gradient descent-based parameter optimization algorithm, which includes at least one of stochastic gradient descent, momentum optimization, or adaptive moment estimation, and does not require uploading original user data.

7. The adaptive adjustment method for home environment based on multimodal physiological perception and edge-cloud collaboration as described in claim 6, characterized in that, The method also includes an end-to-end cloud collaboration step: The gradient parameters updated on the local model are anonymized and encrypted before being uploaded to the cloud server; Receive global model parameters aggregated by the cloud server based on the federated learning algorithm; The local model is updated using the global model parameters to incorporate cross-device environmental tuning experience.

8. A home environment regulation system, characterized in that, include: The sensing module is used to collect the micro-motion signals of users within the target area using a non-contact sensing method; The interaction module is used to monitor users' explicit intervention behaviors towards home environment devices in real time; The processing module is configured to perform the method described in any one of claims 1 to 7; The execution module communicates with home environment devices to perform environmental parameter adjustments.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the adaptive adjustment method for home environment based on multimodal physiological perception and edge-cloud collaboration as described in any one of claims 1 to 7.

Images