Scene-Adaptive Recommendation Methods for Smart Homes Based on Multimodal Data

Abstract

This invention discloses a scene-adaptive recommendation method for smart homes based on multimodal data, belonging to the field of multimodal data processing technology. The method first classifies smart home scenarios based on multimodal perception data to accurately identify the user's current living scenario. Simultaneously, it analyzes the device linkage status based on the classification results to determine whether device operation matches scenario requirements. Then, based on the linkage analysis results, it identifies scene patterns to generate preliminary recommendations and determines whether to update the user's dynamic weight spatial relationship model. Finally, it quantifies the recommendation effect based on the weight adjustment judgment results and optimizes the recommendation strategy through bidirectional feedback to improve adaptability to user needs. This achieves a closed-loop process encompassing multimodal data classification, device linkage analysis, pattern recognition recommendation, dynamic weight updates, and effect quantification feedback in smart home scenarios, effectively solving the problem of low intelligence in scene recommendations in smart homes under dynamically changing user behavior scenarios.

Description

Technical Field

[0001] This invention relates to the field of multimodal data processing technology, and in particular to a scene-adaptive recommendation method for smart homes based on multimodal data. Background Technology

[0002] With the widespread adoption of smart home devices and the upgrading of user needs, single-modal data is no longer sufficient to meet the requirements for accurate scene recommendations. Currently, smart home systems are gradually transforming from passive control to proactive service. Multimodal data (such as visual, audio, environmental sensing, and device status data) has become the core basis for understanding user behavior and scene characteristics, driving scene-adaptive recommendation technology to become a key direction for industry development. Devices involved include smart air conditioners, smart lighting, smart curtains, smart water heaters, and smart kitchen appliances.

[0003] Existing technologies primarily achieve scene recommendation through three steps: The first step is multimodal data acquisition, utilizing devices such as cameras, microphones, temperature and humidity sensors, and smart sockets to collect real-time data on user activity images, voice commands, environmental parameters, and device on / off status, establishing a multi-source data input channel; the second step is data preprocessing and fusion, processing the raw data through filtering and noise reduction, format standardization, and then using feature splicing or attention mechanisms to map different modal data to a unified feature space, eliminating data heterogeneity; the third step is scene recognition and recommendation, based on machine learning models (such as support vector machines and decision trees) to classify the fused features into scenes, and finally, according to a pre-set scene-device policy mapping table, pushing the corresponding device control policy.

[0004] For example, Chinese Invention Patent CN117349531B discloses a user information recommendation method and system based on smart homes, which includes: performing user matching processing on multiple smart home users based on the corresponding smart home device operation data to determine user matching relationships among the multiple smart home users; identifying smart home users to be recommended information among the multiple smart home users; identifying each matched smart home user corresponding to the smart home user to be recommended information among the multiple smart home users based on the user matching relationships; and performing user information recommendation operations on the smart home user to be recommended based on the user behavior feature information corresponding to the smart home user to be recommended and the user behavior feature information corresponding to each matched smart home user, thereby determining the target recommendation data for the smart home user to be recommended.

[0005] However, in smart home scenarios where multiple users share the same space and user behavior dynamically changes with life stages (such as adjusting sleep schedules with seasonal changes or changing activity habits due to the addition of new family members), existing machine learning models are often static. Their parameters and structure are basically fixed after training, relying solely on initial training data to learn scenario features. This results in a lag in learning and adapting to dynamic changes in user behavior. For example, if a user changes their sleep schedule from 11:00 PM to 12:30 AM due to a work change, the model will still recommend starting sleep mode at 11:00 PM based on the original schedule due to the lag. Furthermore, the interaction logic between multimodal data and users is not fully considered during multimodal data fusion. For instance, the voice command for relaxation given by a user in the evening is not linked to information such as the ambient temperature of 26°C and the brightness of the living room lights at 50% for analysis. This makes scene recognition prone to errors due to insufficient utilization of data association, resulting in a low level of intelligence in scene recommendations in smart homes and a poor match between scene recommendations and actual user needs. Summary of the Invention

[0006] To address the technical problems in existing technologies, this invention provides a method for rapid identification of power plant equipment anomalies based on cloud-edge collaboration. The technical solution is as follows:

[0007] Step 1: Based on multimodal perception data across all smart home scenarios, scene classification is performed to accurately identify the specific user's living scenario within the smart home ecosystem. Simultaneously, scene linkage status analysis is conducted to assess whether the operational status of each smart home device matches the identified living scenario requirements. The multimodal perception data reflects the status information related to the specified user's behavior, environmental conditions, and device operation within the smart home scenario. Step 2: Based on the results of the scene linkage status analysis, scene pattern recognition is performed to generate preliminary recommendation information. Simultaneously, scene weight adjustment is determined to assess whether the dynamic weight spatial relationship model assigned to the specified user should be adaptively updated. The dynamic weight spatial relationship model quantifies the impact of different scene features on the specified user's scenario classification. Step 3: Based on the results of the scene weight adjustment determination, the recommendation effect is quantified to assess the matching degree between the recommended scenario and the specified user's scenario requirements. Two-way feedback of the recommendation strategy is also implemented to improve the adaptability of the recommended scenario to the specified user's needs, ultimately generating the final recommendation information.

[0008] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0009] 1. This invention, within a smart home scenario, combines dynamic multimodal sensing data with scene-device linkage analysis to proactively capture dynamic data changes caused by seasonal changes, adjustments in daily routines, and changes in family structure. Based on this data, scene classification accurately identifies the user's current living scenario. Simultaneously, it analyzes the device linkage status based on the classification results to determine whether device operation matches scenario requirements, solving the problems of lag in static model adaptation and insufficient data correlation. Scene pattern recognition is performed based on the scene linkage status analysis results to generate preliminary recommendations. Simultaneously, scene weight adjustment determines whether to update the user's dynamic weight spatial relationship model. This process responds promptly to dynamic data changes, avoiding recommendation bias caused by fixed static model parameters, further optimizing the accuracy of scene recommendations. The recommendation effect is quantified based on the weight adjustment determination results, clarifying the degree of matching between recommendations and user needs. Then, through bidirectional feedback of the recommendation strategy, adaptability is optimized, significantly improving the intelligence level and demand matching degree of smart home scene recommendations.

[0010] 2. To address the shortcomings of traditional models, such as insufficient multimodal data association and large scene recognition errors, this invention designs a two-level optimization mechanism: The first level is dynamic feature invocation. Standardized multimodal perception data is input into a pre-trained scene classification model, and combined with a historical scene feature library, a candidate category set that meets the user's current needs is output, avoiding the lag of static models relying solely on initial data. The second level is scene-specific confidence correction. Independent deviation-correction coefficient lookup tables are constructed for different scenes such as sleep and cooking. First, the matching degree of each candidate category is taken and input into the confidence mapping table to calculate the deviation value from the initial matching degree of the preset scene. Then, the scene-specific confidence correction lookup table in the historical library is consulted, and the correction coefficient is multiplied by the initial assignment to obtain the final confidence value. This process, through dynamic invocation of historical features and scene-specific correction, fully associates the multimodal data interaction logic, reduces recognition errors, solves the problems of lag in static model adaptation and insufficient data association, and improves the intelligence of scene recommendation and the degree of matching with needs.

[0011] 3. By converting standardized multimodal perception data into feature vectors and extracting initial feature vectors for each preset scenario from a historical scene feature library, a dynamic weight matrix is ​​constructed to highlight the classification contribution of each scenario and avoid the problem of static model parameter fixation. An error function is also constructed with the goal of minimizing weighted error to quantify the difference between the current data and the preset scenarios, fully relating the multimodal data interaction logic. Finally, the partial derivative of the error function is calculated and normalized to obtain the scene feature matching degree. The entire process dynamically adapts to changes in user behavior, reduces recognition errors caused by insufficient data association, and improves the intelligence level and demand matching degree of scene recommendation.

[0012] 4. A multi-level effective scenario judgment and device linkage closed-loop verification process: In the scenario judgment stage, valid categories with high confidence are first screened. Single categories are directly confirmed, while multiple categories are determined based on matching degree. If there is no high-confidence category, it is classified as a potential category and stored in the association layer; otherwise, it is classified as pending manual verification. This process accurately locates the user's current scenario, reducing the scenario confusion error of the static model. Scenario linkage status analysis monitors whether the device operation matches the preset requirements. If they do not match, an adjustment command is generated and pushed for execution. After adjustment, re-verification is performed. If they still do not match, an abnormal prompt is sent and the fault is investigated; otherwise, a normal message is sent. The entire process dynamically adapts to changes in user behavior, fully integrates multi-modal data logic, and improves the intelligence of scenario recommendation and the degree of matching with requirements.

[0013] 5. By constructing a closed-loop process encompassing recommendation performance quantification, anomaly detection, and two-way feedback: In the quantification stage, the recommendation performance index is calculated using user confirmation rate as the core indicator. If the index is met, continuous monitoring continues; if it is not met, the process proceeds to the anomaly detection stage, which combines the recommendation performance index value with the corresponding rate of change for a two-dimensional assessment. In the two-way feedback stage, the quantification results are input into the mapping set query scheme: if the index is met, high-quality strategies are stored for reference; if the index is not met but there are no anomalies, feature dimensions are optimized; if the index is not met and there are anomalies, the parameters of the scenario mode library are updated. The entire process dynamically tracks performance and optimizes strategies, fully adapting to changes in user behavior, reducing data association errors, and improving the intelligence and demand matching of recommendations. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart of a scene-adaptive recommendation method for smart homes based on multimodal data provided in an embodiment of the present invention;

[0016] Figure 2 A flowchart for scene classification and confidence assignment provided in embodiments of the present invention;

[0017] Figure 3 A flowchart of scene pattern recognition provided in an embodiment of the present invention;

[0018] Figure 4 A flowchart for quantifying the recommendation effect provided in the embodiments of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0020] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] This invention provides a scene-adaptive recommendation method for smart homes based on multimodal data, such as... Figure 1 The flowchart shown is for a scene-adaptive recommendation method in smart homes based on multimodal data. The processing flow of this method may include:

[0023] Step 1: Based on multimodal perception data across all smart home scenarios, scene classification is performed to accurately identify the living scenarios of specific users within the smart home ecosystem. Simultaneously, scene linkage status analysis is conducted based on the scene classification results to assess whether the operational status of each smart home device matches the identified living scenario requirements. Multimodal perception data reflects the status information related to specified user behavior, environmental conditions, and device operation within the smart home scenario. Specifically, multimodal perception data includes environmental parameter data, user behavior data, device status data, and time-related data corresponding to different seasons (spring, summer, autumn, and winter).

[0024] Step two involves scene pattern recognition based on the results of scene linkage state analysis to generate preliminary recommendation information. Simultaneously, scene weight adjustment is determined to decide whether to adaptively update the dynamic weight spatial relationship model assigned to a specific user. The dynamic weight spatial relationship model is used to quantify the influence of different scene features (such as temperature features in a sleep scene and lighting features in a movie-watching scene) on the scene classification of a specific user. This enables real-time adaptation to dynamic changes in user behavior and personalized weight adjustment, breaking the limitations of static model parameter fixation. This avoids recommendation lag and bias caused by factors such as changes in user work and rest schedules and lifestyle habits, significantly improving the accuracy of scene recognition and the fit of preliminary recommendations.

[0025] Step three involves quantifying the recommendation effect based on the results of scene weight adjustment to quantify the degree of matching between the recommended scene and the needs of the specified user scene. At the same time, the recommendation strategy is subjected to two-way feedback based on the results of the recommendation effect quantification to improve the adaptability of the recommended scene to the needs of the specified user, thereby generating the final recommendation information. This realizes the closed-loop iteration and continuous optimization of the recommendation strategy, effectively improving the intelligence level of smart home scene recommendations and user satisfaction.

[0026] Specifically, environmental parameter data includes: real-time indoor temperature (28-35℃), humidity (60%-80%), and light intensity (800-1200 lux) during hot summer months; and indoor temperature (12-18℃) and PM2.5 concentration (often rising to 50-100 μg / m³ due to heating) during cold winter months. 3 ), Indoor and outdoor temperature difference (15-25℃); User behavior data: For example, in spring, users often turn on balcony ventilation and plant care between 19:00-21:00, and in autumn, users often read in the living room and turn on the humidifier between 20:00-22:00, corresponding to the user location trajectory captured by the camera and the voice commands such as turning on the humidifier collected by the microphone; Device status data: For example, in summer sleep scenarios, the air conditioner is mostly in cooling mode (set temperature 24-26℃) and the fan is on low speed, in winter bathing scenarios, the bathroom heater is turned on 10 minutes in advance (set temperature 30-32℃) and the water heater maintains a constant temperature of 50-55℃; Time-related data: For example, in winter, the commuting preparation scenario is concentrated between 7:00-8:00 (corresponding to the morning rush hour), and in summer, this scenario is earlier at 6:30-7:30 (due to the longer day and shorter night), with seasonal labels and special holiday markings (such as the extended time for the family reunion scenario during the Spring Festival).

[0027] It's important to understand that the dynamic weight space relationship model is assigned individually to each specific user, ensuring the model adapts to the user's personalized needs. Its construction is user-centric. First, it extracts the user's historical interaction data (such as scene usage frequency, recommendation confirmation rate, and manual adjustment records) from a historical scene feature library. Then, it combines this data with scene-related features from real-time multimodal perception data (environmental parameters in different seasons, user behavior, and device status), and optimizes the initial values ​​of the feature weights using a gradient descent algorithm. For example, if a user frequently adjusts the temperature in a sleep scene, the temperature feature weight is increased; if they frequently manually switch the lighting in a movie-watching scene, the lighting feature weight is increased. Ultimately, this forms a dynamic weight matrix specific to that user, i.e., the dynamic weight space relationship model.

[0028] The input to the dynamic weighted spatial relationship model is usually a scene feature vector transformed from real-time multimodal perception data, such as a sleep scene [temperature 24℃, light intensity 10 lux, time 23:30]. The output is the dynamic weight value corresponding to each scene feature, such as temperature weight 0.4, light intensity weight 0.3, and time weight 0.3. These weight values ​​are directly used to determine the feature importance when classifying scenes, ensuring that the classification results are consistent with the user's real-time behavior changes.

[0029] In a specific embodiment, taking the example of a user changing their sleep schedule from 11:00 PM to 12:30 AM during the summer: Using multimodal perception data (environmental parameter 26℃, user issuing a rest command at 11:50 PM, bedroom light on, time-related data 00:20), the sleep scenario is accurately identified. Analysis reveals that the bedroom light being on and the air conditioner set to 28℃ do not meet the sleep scenario requirements. Based on this, an initial recommendation is generated to turn off the bedroom light and adjust the air conditioner to 25℃. Simultaneously, because the user has repeatedly delayed going to sleep recently, it is determined that their dynamic weight spatial relationship model needs to be updated, increasing the weight of time features and voice command features. Through user confirmation of the recommendation, the recommendation effect is quantified, and the strategy is stored in a high-quality sample library. Subsequent similar scenarios will directly call the optimized model to generate recommendations.

[0030] like Figure 2 The flowchart shown illustrates the scene classification and confidence assignment process. It includes first classifying the scene and then assigning confidence scores. Specifically, the process involves: inputting standardized multimodal sensing data into a pre-trained scene classification model; performing classification calculations using a historical scene feature library; and outputting a candidate category set corresponding to the current scene. This candidate category set represents the set of scene categories within the entire smart home scenario corresponding to a specified user that meet the user's current scenario needs. Based on the obtained scene feature matching degree, a confidence score is assigned to each candidate category in the candidate category set to quantify the degree of fit between the current multimodal sensing data and each preset scene category. Simultaneously, an effective scene category determination is performed to identify the corresponding living scenario in the current home environment, avoiding recommendation bias caused by multiple scene confusion. The scene feature matching degree is used to quantify the degree of fit between the current multimodal sensing data and each preset scenario within the entire smart home scenario at the feature level, intuitively reflecting the closeness of the match. A higher matching degree indicates that the current multimodal data better meets the feature requirements of the corresponding preset scenario.

[0031] The specific process for obtaining scene feature matching degree is as follows: Standardized multimodal perception data is transformed into feature vector X, allowing various types of data to participate in calculations in a unified vector form. Standard feature vectors Y1, Y2, Y3...Y4 for each preset scene (such as sleep, cooking, watching movies, etc.) are extracted from the historical scene feature library. N-1 Y NThese are vector representations of typical features for each scenario. At the same time, a dynamic weight matrix W is constructed based on the historical matching accuracy, which assigns different weights to different feature dimensions. Features that have made greater contributions to scenario matching in the past have higher weights, reflecting the differences in the importance of features.

[0032] To minimize the weighted error between the current feature vector and the standard feature vector, an error function is constructed. (i=1,2,3,...,N), in this error function, i is the number of the life scene, and N is the total number of life scenes. It is a vector (XY) i The transpose of X is used to perform operations with the original vector and the dynamic weight matrix W, ultimately yielding a scalar error value. This value is used to measure the difference between the current feature vector X and the standard feature vector Y of a predetermined scenario. i The difference when X and Y i The closer they are, (XY) i The smaller the magnitude of the vector, the smaller the value of the error function E after weighted calculation by the dynamic weight matrix W, which means that the current data has a higher initial matching degree with the corresponding scenario.

[0033] Take the partial derivative of the error function E with respect to the eigenvector X The sensitivity of each feature dimension to the error is analyzed by partial derivative analysis. If the absolute value of the partial derivative of a certain dimension is small, it indicates that the feature has a high degree of fit with the standard scenario. Then, combined with the error function value and the partial derivative sensitivity, the error is calculated by normalization (setting [0, ...). The interval error is mapped to the interval matching degree of [100, 0], and the scene feature matching degree of each candidate category is obtained, where the dynamic weight matrix is ​​a symmetric matrix.

[0034] Assigning confidence scores to each candidate category in the candidate category set based on the acquired scene feature matching scores includes: acquiring the scene feature matching scores corresponding to each candidate category, inputting the scene feature matching scores of each candidate category into a set confidence score mapping table, calculating the deviation between the matching scores of each candidate category and the initial matching scores of the preset scene. The initial matching scores of the preset scene are obtained from historical scene data and the matching scores between the preset scene and ideal standard features (such as the ideal combination of lighting, temperature, and other features in a sleep scene). By statistically analyzing a large number of historical normal feature matching cases under this scene, the average score is calculated. All are obtained; the deviation values ​​of each candidate category are input into the preset confidence correction lookup table in the historical scene feature library to map the confidence correction coefficients corresponding to each deviation value, and the corresponding confidence correction coefficients are processed with the initial confidence values ​​of each candidate category to obtain the final confidence of each candidate category. The preset confidence correction lookup table represents a scene-specific correction lookup table with the candidate category deviation value as the index and the confidence correction coefficient as the corresponding result. The scene-specific correction lookup table corresponds one-to-one with different smart home life scene categories (sleep, cooking, watching movies, reading, etc.).

[0035] Specifically, pre-trained scene classification models typically refer to models built on deep learning frameworks (such as CNN, LSTM, or Transformer) that can automatically extract multimodal data features and predict scene categories, covering core smart home scene categories such as sleep, cooking, and watching movies. The construction and training process is as follows: First, collect massive amounts of multimodal data (including different seasonal environments, user behavior, and device status data), labeling the corresponding scene categories as the training set; then design the model structure (e.g., using CNN to process environmental / device data, LSTM to process temporal behavior data, and fusing multimodal features through an attention mechanism); finally, iteratively train using the training set, optimizing parameters with the cross-entropy loss function, adjusting model hyperparameters using the validation set, and completing pre-training once the model's scene classification accuracy on the test set reaches the target. Subsequent fine-tuning can be done using historical user data to adapt to personalized needs.

[0036] First, collect basic characteristics of the entire smart home scenario (such as typical environmental parameters and device status ranges for each scenario); then, for each user, continuously collect their scenario interaction data (such as the scenario category confirmed each time, manually adjusted parameters, and usage time), and extract the user's scenario characteristic preferences (such as user A's preferred temperature for sleep scenarios being 23-25℃ and time being 23:30-7:00); finally, store the data according to the structure of user ID, scenario category, feature benchmark, and update time to form a historical scenario feature library exclusive to each user, and regularly update the feature benchmark based on new user interaction data to ensure data timeliness.

[0037] In this embodiment, feature weights are allocated by combining a dynamic weight matrix with historical matching accuracy, making features that contribute significantly to scene classification more prominent and avoiding interference from irrelevant features. A weighted error function is constructed and combined with partial derivative analysis to accurately quantify the differences between data and scenes, and also to identify the feature dimension fit. Normalization processing transforms the error into an intuitive matching degree, improving the accuracy of the judgment. This effectively solves the problems of insufficient multimodal data association and coarse scene matching in existing technologies, reducing multi-scene confusion.

[0038] Furthermore, the effective scene category is determined as follows: the candidate category corresponding to the final confidence level of the candidate category set is not less than the maximum value of the reference confidence level interval (i.e., 85%), and is recorded as the effective scene category, that is, the candidate category with high confidence. If there is only one effective scene category, the candidate category is directly confirmed as the current scene classification result. If there are multiple effective scene categories, the corresponding effective scene categories are counted to obtain the effective scene set, and the effective scene category corresponding to the highest matching degree in the effective scene set is taken as the final scene classification result.

[0039] It's important to understand that the category of the valid scenario with the highest matching degree in the set of valid scenarios is taken as the final scenario classification result. There are no cases with the same matching degree because it is designed to meet the actual needs of users. If there were cases with the same matching degree, users would be confused about which scenario to choose first. In the process of calculating the corresponding matching degree, rules such as multi-dimensional weighting and prioritizing the matching of high-frequency user scenarios are used to ensure a unique highest matching degree, helping users directly lock in the optimal result and avoid the dilemma of choosing.

[0040] If there is no candidate category in the candidate category set with a final confidence level not less than the maximum value of the reference confidence level interval, then the candidate categories are determined by the association layer based on the corresponding confidence level. The specific process is as follows: Candidate categories in the candidate category set with a final confidence level within the reference confidence level interval (usually set to 60%-85%) are recorded as potential scenario categories, i.e., candidate categories with medium confidence, and the corresponding candidate categories are stored in the potential association layer candidate set to avoid missing potential user demand scenarios due to a single confidence level threshold; Candidate categories in the candidate category set with a final confidence level less than the minimum value of the reference confidence level interval (i.e., 60%) are recorded as irrelevant scenario categories, i.e., candidate categories with low confidence, and the corresponding candidate categories are stored in the candidate set to be manually verified to avoid invalid scenarios interfering with the classification results.

[0041] The specific values ​​mentioned above (60%, 85%) were set by analyzing the correlation between confidence and accuracy in historical scene classification data to ensure that the initial thresholds can cover the judgment needs of most effective scenes. In practical applications, these values ​​can be fine-tuned based on user group characteristics (e.g., elderly users have higher requirements for scene stability) and home scene types (e.g., kitchen scenes require more accurate judgment) to adapt to the classification accuracy requirements of different scenes and improve the flexibility and applicability of scene judgment.

[0042] In this embodiment, scene classification is filtered through an association layer. When there is no high-confidence category, the medium-confidence category is assigned to the potential association layer to avoid missing potential user needs (such as leisure scenarios temporarily switched by the user). At the same time, the low-confidence category is assigned to the layer awaiting manual verification to eliminate invalid interference. This hierarchical judgment logic ensures accurate classification of regular scenarios and provides a reasonable processing path for special scenarios, reducing classification bias caused by rigid thresholds and providing a more practical scenario foundation for subsequent device linkage analysis and recommendations.

[0043] Further, scene linkage status analysis is performed. The specific process is as follows: If the monitored current device operating status (such as the brightness of the master bedroom lights and the air conditioning temperature in a sleep scene) does not meet the corresponding preset scene requirements (the sleep scene requires light brightness ≤10 lux and air conditioning temperature 22-24℃), a device linkage adjustment operation is performed. Otherwise, a scene linkage normal confirmation message is sent to the user's APP. Performing a device linkage adjustment operation means generating a targeted adjustment command, such as adjusting the master bedroom air conditioning temperature from 26℃ to 23℃. The generated targeted adjustment command is pushed to the corresponding home device through the smart home gateway for execution until the corresponding device status is detected to meet the corresponding preset scene requirements. After performing the device linkage adjustment operation, the scene linkage status verification is re-executed. If the device operating status still does not meet the corresponding preset scene requirements, a scene linkage abnormality prompt is sent to the user's APP, and a fault diagnosis process is triggered, such as checking the communication connection of home devices. Otherwise, the scene linkage is determined to be up to standard and the adjustment parameters are recorded.

[0044] In this embodiment, the proactive service capabilities of smart homes are enhanced by correcting the discrepancy between device status and scenario requirements. A secondary verification step is added after the adjustment. If the device still fails to meet the standards, a fault check is triggered and an anomaly notification is pushed, which can promptly detect problems such as device communication failures and avoid scenario experience failure due to device abnormalities. After meeting the standards, the adjustment parameters are recorded, which can provide a reference for device linkage in similar scenarios in the future, optimize the accuracy of the adjustment strategy, and at the same time, the information or anomaly notification is confirmed through the APP to ensure that users are timely aware of the scenario status.

[0045] like Figure 3The scene pattern recognition flowchart shown below is designed as follows: First, multimodal perception data is acquired, and a matching reference table is generated by combining it with a historical scene feature library to quantify the matching degree of each scene pattern. Through two-level judgments—whether the matching ratio is not less than the reference ratio and whether only one pattern is satisfied—the scene is accurately identified: if a single pattern is satisfied, it is directly judged and a recommendation is generated; if multiple patterns are satisfied, key features are manually compared to determine the final result; if no pattern is satisfied, the data is marked for verification and stored in the database, providing a basis for subsequent matching after supplementing data. This ensures that scene recognition is both automated and capable of handling complex and ambiguous situations.

[0046] Further understanding is needed regarding the specific process of scene pattern recognition: First, acquire the current multimodal sensing data and combine it with a historical scene feature library to generate a scene matching lookup table. This table visually distinguishes the matching status between the current multimodal sensing data and the features of each pattern. The historical scene feature library represents a database storing all preset scenes under the full smart home scenario. If the proportion of feature matching counts for a certain scene pattern in the scene matching lookup table is not less than a reference proportion (usually set to 75%), then that scene pattern is determined as the current matching scene. Based on this pattern, preliminary recommendation information is generated. For example, if the sleep mode match meets the criteria, it is recommended to turn off the main light and turn on the air conditioner in sleep mode. If the proportion of feature matching in two or more scene modes to the total number of features in that mode is not less than the reference proportion, then feedback on the feature matching status of each mode will be provided to prompt the preset personnel to further compare the matching status of each mode. For example, the key feature of the cooking mode is that the range hood is on, and the key feature of the movie-watching mode is that the TV is on, and this will be used as the final matching result. If the proportion of feature matching in all scene modes to the total number of features in that mode is less than the reference proportion, then the current multimodal perception data will be marked as data to be verified, and scene recommendations will not be generated for the time being. Instead, it will be stored in the historical scene feature library and re-analyzed after user behavior data (such as records of manual device operation) is added later.

[0047] The process of determining scene weight adjustments includes: After scene pattern recognition and recommendation are completed, the response rate of a specified user to the initial scene recommendation is collected. The response rate represents the proportion of the number of times the specified user performs recommendation operations within a preset period (e.g., 24 hours) after the recommendation is generated, out of the total number of recommendations. If the response rate is greater than or equal to the threshold response rate (usually set to 60%), there is no need to adaptively update the dynamic weight space relationship model for the specified user. If the response rate is less than the threshold response rate, it is determined that the dynamic weight space relationship model for the specified user needs to be adaptively updated, specifically as follows:

[0048] Obtain the difference between the defined response rate and the target response rate, and input it into the preset response rate difference-weight update frequency mapping set in the historical scene feature library to match and obtain the corresponding weight update frequency. During the weight update frequency update process, monitor the update resource occupancy rate of the dynamic weight spatial relationship model for the specified user in real time. Within the corresponding allowable range (usually set to ≤30%), improve the operational stability of the dynamic weight spatial relationship model to ensure that updates are performed during off-peak hours, thus not affecting the operational stability of the model, while also meeting the response needs of the specified user. Otherwise, pause the update and restart the update process once the resource occupancy rate is detected to be within the corresponding allowable range. If the monitored resource occupancy rate is still outside the allowable range during the monitoring period, an intervention warning will be issued to prompt designated personnel to intervene manually. A new scenario recommendation scheme will be generated based on the adaptively updated dynamic weight spatial relationship model. The response rate of the recommendation scheme before and after the update will be compared. If the increase in response rate compared to before the update is not less than the reference increase (usually set at 10%), the adaptive update is confirmed to be effective. Otherwise, a model rollback mechanism will be triggered to restore the model to the version before the update and add it to the historical scenario feature library to improve the mapping relationship between the response rate difference and the weight update strategy, providing a more accurate reference for subsequent weight adjustment judgments for similar users.

[0049] In this embodiment, the aforementioned values ​​are typically set by analyzing historical scene recognition and user response data, combined with the model's operational stability requirements. For example, a reference ratio of 75% corresponds to high scene recognition accuracy, while a defined response rate of 60% matches general user acceptance. In practical applications, these values ​​can be fine-tuned based on user habits (e.g., relaxing the defined response rate for elderly users) and scene complexity (e.g., increasing the reference ratio for kitchen scenes) to balance recognition accuracy and user experience. This example accurately identifies scenes by comparing feature matching ratios and key features, avoiding multi-mode confusion. The data storage mechanism for verification can also accumulate data to optimize subsequent matching. The user response rate is used as the core to determine whether to update the dynamic weighted spatial relationship model, and the update timing is controlled by resource utilization. This ensures that the model meets user needs while guaranteeing stable operation, achieving a coordinated adaptation between scene recognition and model optimization.

[0050] like Figure 4 The flowchart illustrating the recommendation performance quantification process is designed as follows: First, determine if the recommendation performance index meets the target. If it does, continue monitoring. If it doesn't, further determine if there are any abnormal conditions. If an anomaly is found, increase the monitoring frequency and conduct a second assessment. If no anomaly is found, check if the rate of change exceeds a threshold. If it does, continue monitoring; otherwise, supplementary data collection and feedback are provided. After the second assessment, if there are no anomalies and the rate of change meets the requirements, maintain the current strategy. Otherwise, issue an alert for recommendation performance anomalies. This allows for dynamic, hierarchical monitoring and strategy adjustment of recommendation performance, ensuring stable and relevant recommendation results.

[0051] Further understanding is needed regarding the quantification of recommendation effectiveness. Specifically, based on the scenario recommendation confirmation rate of a specified user within a preset monitoring period and the reference scenario recommendation confirmation rate, a recommendation effectiveness index is obtained to reflect the degree of matching between the current recommendation strategy and the actual needs of the specified user during the scenario recommendation process. That is, the ratio of the obtained scenario recommendation confirmation rate to the reference scenario recommendation confirmation rate. If the recommendation effectiveness index is not less than the reference recommendation effectiveness index, the recommendation effectiveness is determined to be up to standard, and the scenario recommendation process continues to be monitored. Otherwise, it is determined whether there are abnormal conditions for recommendation effectiveness. Abnormal conditions for recommendation effectiveness indicate that within the preset monitoring period, the proportion of time corresponding to the recommendation effectiveness index being less than the set recommendation effectiveness index exceeds the duration warning value.

[0052] If no abnormal conditions exist for the recommendation effect, the rate of change of the recommendation effect index within the preset monitoring period is obtained. If the obtained rate of change of the recommendation effect index is not greater than the defined rate of change (usually set to 5% / hour), the recommendation process is continuously monitored. The rate of change of the recommendation effect index represents the ratio of the difference between the recommendation effect index at the beginning of the preset monitoring period and the recommendation effect index at the end of the period to the duration of the preset monitoring period. Otherwise, supplementary data collection and feedback are performed based on the obtained rate of change of the recommendation effect index to locate the cause of the corresponding change in the recommendation effect index. If abnormal conditions exist for the recommendation effect, the frequency of scenario recommendation monitoring is increased based on the deviation of the rate of change (e.g., from hourly monitoring to monitoring every 30 minutes). After adjustment, a second judgment is made: if no abnormal conditions exist for the recommendation effect and the obtained rate of change of the recommendation effect index is not greater than the defined rate of change, the current recommendation strategy is maintained; otherwise, an abnormal recommendation effect warning is issued to provide feedback on the abnormal recommendation effect, and intervention is carried out to preset personnel or designated users through system pop-ups, SMS notifications, etc.

[0053] The recommendation strategy involves two-way feedback based on the quantified results of the recommendation effect. The specific steps are as follows:

[0054] Obtain the quantified recommendation effect result and input it into the preset quantified effect result-strategy adjustment direction mapping set in the historical scene feature library. Query the corresponding adjustment scheme. The quantified recommendation effect result includes the recommendation effect index and the rate of change of the recommendation effect index. The process of querying the corresponding adjustment scheme is as follows:

[0055] If the quantification results show that the recommendation effect meets the target, the recommendation effect and corresponding data in the current recommendation strategy are added to the corresponding mapping set as a high-quality strategy sample library for reference in subsequent recommendation strategies for similar users. At the same time, based on the current high-quality strategy sample library, the final recommendation information that meets the current needs of the specified user is generated and output, such as recommending an air conditioning temperature of 23-25℃ and a light brightness of 10% according to the current sleep mode. If the quantification results show that the recommendation effect does not meet the target but there are no abnormalities, feature dimension optimization feedback is performed to enhance the adaptability of the dynamic weight space relationship model to the user's real-time needs, such as increasing the proportion of manual intervention features in the dynamic weight model based on the user's recent operation logs. After the feature dimension optimization is completed, the final recommendation information is generated and output based on the updated dynamic weight space relationship model.

[0056] If the quantitative results of the recommendation effect show that the recommendation effect is not up to standard and there are anomalies, the feature scene pattern library will be updated to correct the parameters in the scene pattern library that are out of touch with user needs. For example, the temperature parameter threshold of the sleep mode in the scene pattern library will be adjusted from 22-24℃ to the user's actual preference of 23-25℃. After the feature scene pattern library is updated, the final recommendation information will be generated and output based on the corrected scene patterns. The generated final recommendation information will be pushed to the corresponding designated user's APP for visualization. The APP will present the recommendation content in a clear card format (e.g., the current recommended sleep mode: air conditioner 24℃, master bedroom light 10%), and provide confirmation. The app offers three interactive options: Confirm Execution, Custom Adjustment, and Rejection. If the user selects Confirm Execution, the app directly sends a command to the smart home gateway, triggering the device to execute the recommended solution. If the user selects Custom Adjustment (e.g., changing the air conditioner temperature to 25℃), the adjustment parameters are recorded and updated synchronously to the user's historical interaction database, providing data for subsequent model optimization. If the user selects Rejection, a simple questionnaire pops up (e.g., reasons for rejection: inappropriate recommendation timing / parameters do not meet requirements). After collecting the corresponding user feedback, the recommendation is temporarily suspended. At the same time, the rejection record and feedback reasons are added to the data source for quantifying the recommendation effect, providing a reference for the next round of two-way strategy feedback.

[0057] The establishment of a pre-defined mapping set of quantified performance results and strategy adjustment directions involves first collecting historical recommendation data, extracting quantified performance indicators (such as recommendation confirmation rate and user satisfaction) and corresponding strategy adjustment schemes (such as optimizing feature dimensions and updating the scenario pattern library), and analyzing the correlation between the two. Then, the data is divided into intervals according to performance level, and a fixed adjustment direction is matched for each interval (such as updating the scenario pattern library for poor intervals), forming an initial mapping set. Subsequently, the interval boundaries and adjustment directions are continuously corrected based on new recommendation data to ensure that the mapping logic conforms to actual needs.

[0058] Similarly, other mapping sets in the embodiments of the present invention, such as the confidence mapping table and the preset response rate difference-weight update frequency mapping set, are all based on historical scenario data and user interaction records. They are established by statistical feature association patterns, dividing indicator intervals, matching corresponding outputs (such as the confidence level corresponding to the matching degree interval), and dynamically iteratively optimizing to ensure the accuracy and adaptability of the mapping relationship.

[0059] In this embodiment, by using indices and anomaly condition judgments, the deviation between the recommendation strategy and user needs is accurately located, avoiding optimization lag caused by fuzzy evaluation. Two-way feedback adjusts the strategy tiered according to the results: strategies that meet the criteria are stored in a high-quality sample library for future reference; when no anomalies are found, the model feature dimensions are optimized; when anomalies are found, the scene pattern library is updated, ensuring targeted adjustments and avoiding blind optimization. The APP's visual display and multi-interactive options design not only allow users to clearly understand the recommended content but also collect real-time needs through confirmation, adjustment, and rejection feedback. This supplementary data feeds back into the historical library, further refining the strategy. In case of anomalies, the monitoring frequency and early warning mechanism are increased, enabling rapid response to strategy deviations and reducing user experience degradation. This overall closed-loop system continuously improves the adaptability of the recommendation strategy to user needs, avoids ineffective recommendations and strategy rigidity, and significantly enhances the intelligence and humanization of smart home scenario recommendations.

[0060] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0061] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0062] In various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0063] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0064] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A scene-adaptive recommendation method for smart homes based on multimodal data, characterized in that, Includes the following steps: Step 1: Based on multimodal perception data of the entire smart home scenario, scene classification is performed to achieve accurate identification of the life scenario of the specified user in the entire smart home scenario. At the same time, scene linkage status analysis is performed to evaluate whether the operating status of each smart home device in the current life scenario matches the needs of the identified life scenario. The multimodal perception data is used to reflect the status information related to the behavior of the specified user, the environmental status and the operation of the device in the smart home scenario. Step 2: Based on the results of the scene linkage state analysis, scene pattern recognition is performed to generate preliminary recommendation information. At the same time, scene weight adjustment is determined to determine whether to adaptively update the dynamic weight spatial relationship model assigned to the specified user. The dynamic weight spatial relationship model is used to quantify the degree of influence of different scene features on the scene classification of the specified user. Step 3: Based on the results of the scene weight adjustment judgment, the recommendation effect is quantified to quantify the degree of matching between the recommendation scene and the needs of the specified user scene. At the same time, two-way feedback of the recommendation strategy is carried out to improve the adaptability of the recommendation scene to the needs of the specified user, and then the final recommendation information is generated. The quantification of recommendation effectiveness is specifically as follows: Based on the scenario recommendation confirmation rate of a specified user within a preset monitoring period and the reference scenario recommendation confirmation rate, obtain the recommendation effect index, which reflects the degree of matching between the current recommendation strategy and the actual needs of the specified user during the scenario recommendation process; If the recommendation effect index is not less than the reference recommendation effect index, the recommendation effect is determined to be up to standard, and the scene recommendation process continues to be monitored. Otherwise, it is determined whether there are abnormal conditions for recommendation effect. Abnormal conditions for recommendation effect mean that within the preset monitoring period, the recommendation effect index is less than the percentage of time corresponding to the set recommendation effect index is greater than the time warning value. If there are no abnormal conditions for the recommendation effect, the rate of change of the recommendation effect index within the preset monitoring period is obtained. If the rate of change of the obtained recommendation effect index is not greater than the defined rate of change, the recommendation process is continuously monitored. Otherwise, based on the value of the obtained rate of change of the recommendation effect index, supplementary data collection and feedback are carried out to locate the cause of the corresponding change in the recommendation effect index. If there are abnormal conditions in the recommendation effect, the frequency of scene recommendation monitoring will be increased based on the deviation of the rate of change. After the adjustment is completed, a second judgment will be made. The second determination is as follows: if there are no abnormal conditions for the recommendation effect and the rate of change of the obtained recommendation effect index is not greater than the defined rate of change, then the current recommendation strategy is maintained; otherwise, an abnormal recommendation effect warning is issued to provide feedback on the abnormal recommendation effect. The specific steps for implementing the two-way feedback of the recommendation strategy are as follows: Obtain the recommendation effect quantification result and input it into the preset effect quantification result-strategy adjustment direction mapping set in the historical scene feature library to query the corresponding adjustment scheme. The recommendation effect quantification result includes the recommendation effect index and the rate of change of the recommendation effect index. The process of querying the corresponding adjustment scheme is as follows: If the quantitative results of the recommendation effect show that the recommendation effect meets the standard, the recommendation effect and corresponding data in the current recommendation strategy will be added to the corresponding mapping set as a high-quality strategy sample library for reference in subsequent recommendation strategies for similar users. At the same time, based on the current high-quality strategy sample library, the final recommendation information that meets the current needs of the specified user will be generated and output. If the quantitative results of the recommendation effect show that the recommendation effect does not meet the standard but there are no abnormalities, then feature dimension optimization feedback will be carried out to enhance the adaptability of the dynamic weight space relationship model to the real-time needs of users. After the feature dimensions are optimized, the final recommendation information is generated and output based on the updated dynamic weight space relationship model. If the quantitative results of the recommendation effect show that the recommendation effect is not up to standard and there are abnormalities, the feature scene pattern library will be updated and feedback will be provided to correct the parameters in the scene pattern library that are out of touch with user needs. Once the feature scene pattern library is updated, the final recommendation information will be generated and output based on the corrected scene patterns. The generated final recommendation information will be pushed to the corresponding designated user's APP for visualization.

2. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 1, characterized in that, The process of classifying scenarios based on multimodal perception data across all smart home scenarios includes: The standardized multimodal perception data is input into the pre-trained scene classification model, and the classification calculation is performed in combination with the historical scene feature library. The candidate category set corresponding to the current scene is output. The candidate category set represents the set of scene categories that meet the user's current scene needs in the full smart home scene of the specified user. The confidence level is assigned to each candidate category in the candidate category set based on the obtained scene feature matching degree, so as to quantify the degree of fit between the current multimodal perception data and each preset scene category. At the same time, the effective scene category is determined to identify the living scene corresponding to the current home environment. The scene feature matching degree is used to quantify the degree of fit between the current multimodal perception data and each preset scene in the smart home full scene at the feature level.

3. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 2, characterized in that, The scene feature matching degree is obtained through the following method: The standardized multimodal perception data is transformed into feature vectors, and the initial feature vectors corresponding to each preset scenario are extracted from the historical scene feature library simultaneously. A dynamic weight matrix is ​​constructed to highlight the contribution of each life scenario in scenario classification. At the same time, with the goal of minimizing the weighted error between the transformed feature vector and the standard feature vector, an error function is constructed to quantify the degree of difference between the current multimodal data and each preset scenario. The error function constructed based on the transformed feature vector is subjected to partial derivative processing, and the result of partial derivative processing is normalized to obtain the scene feature matching degree used to quantify the degree of fit between the current multimodal perception data and each preset scene.

4. The scene adaptive recommendation method for smart homes based on multimodal data as described in claim 3, characterized in that, Assigning confidence scores to each candidate category in the candidate category set based on the acquired scene feature matching degree includes: Obtain the scene feature matching degree corresponding to each candidate category, and input the scene feature matching degree of each candidate category into the set confidence mapping table, and calculate the deviation value between the matching degree of each candidate category and the initial matching degree of the preset scene; The deviation values ​​of each candidate category are input into a preset confidence correction lookup table in the historical scene feature library to map the confidence correction coefficients corresponding to each deviation value. The corresponding confidence correction coefficients are then processed with the initial confidence values ​​of each candidate category to obtain the final confidence of each candidate category. The confidence correction lookup table represents a scene-specific correction lookup table with the candidate category deviation value as the index and the confidence correction coefficient as the corresponding result. The scene-specific correction lookup table corresponds one-to-one with different smart home living scene categories.

5. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 4, characterized in that, The determination of the effective scene category is as follows: Obtain the candidate category whose final confidence score is not less than the maximum value of the reference confidence score interval, and record it as the effective scene category; If there is only one valid scene category, then that candidate category is directly confirmed as the current scene classification result; If there are multiple valid scene categories, the corresponding valid scene categories will be counted to obtain a set of valid scenes, and the valid scene category with the highest matching degree in the set of valid scenes will be used as the final scene classification result. If there is no candidate category in the candidate category set with a final confidence level not less than the maximum value of the reference confidence level interval, then the candidate categories are determined by the association layer based on the corresponding confidence level. The specific process is as follows: Candidate categories whose final confidence level is within the reference confidence level range are recorded as potential scenario categories, and the corresponding candidate categories are stored in the candidate set of the potential association layer to avoid missing potential user demand scenarios due to a single confidence level threshold. Candidate categories whose final confidence level is less than the minimum value of the reference confidence level interval are recorded as irrelevant scene categories, and the corresponding candidate categories are stored in the candidate set to be manually verified to avoid invalid scenes interfering with the classification results.

6. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 5, characterized in that, The specific process for performing scene linkage state analysis is as follows: If the monitored current device operating status does not meet the corresponding preset scenario requirements, the device linkage adjustment operation will be performed; otherwise, a scenario linkage normal confirmation message will be sent to the user's APP. The device linkage adjustment operation refers to generating a targeted adjustment command, which is then pushed to the corresponding home device through the smart home gateway for execution until the device status is detected to meet the corresponding preset scenario requirements. After performing device linkage adjustment operations, the scene linkage status verification is re-executed. If the device operating status still does not meet the corresponding preset scene requirements, a scene linkage exception prompt is sent to the user's APP; otherwise, the scene linkage is determined to meet the standards and the adjustment parameters are recorded.

7. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 1, characterized in that, The specific process for scene pattern recognition is as follows: Acquire current multimodal sensing data and combine it with a historical scene feature library to generate a scene matching comparison table for visually distinguishing the matching situation between the current multimodal sensing data and the features of each mode. The historical scene feature library represents a database that stores various preset scenes under the full smart home scenario. If the proportion of the number of feature matches for a certain scene pattern in the scene matching lookup table to the total number of features for that pattern is not less than the reference proportion, then that scene pattern is determined as the current matching scene, and preliminary recommendation information is generated based on that pattern. If the proportion of feature matching in two or more scene modes to the total number of features in that mode is not less than the reference proportion, then feedback on the feature matching status of each mode will be provided to prompt the pre-selected personnel to further compare the matching status of each mode, and this will be used as the final matching result. If the proportion of feature matching numbers for all scene modes to the total number of features for that mode is less than the reference proportion, the current multimodal perception data will be marked as data to be verified, and scene recommendations will not be generated for the time being; it will only be stored in the historical scene feature library.

8. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 7, characterized in that, The specific process of determining scene weight adjustment includes: After scene pattern recognition and recommendation are completed, the response rate of a specified user to the initial scene recommendation is collected. The response rate represents the proportion of the number of times the specified user performs the recommendation operation within a preset period after the recommendation is generated to the total number of recommendations. If the response rate is greater than or equal to the defined response rate, there is no need to adaptively update the dynamic weight space relationship model for the specified user. If the response rate is less than the defined response rate, it is determined that the dynamic weight space relationship model for the specified user needs to be adaptively updated, specifically as follows: Obtain the difference between the defined response rate and the response rate, and input it into the preset response rate difference-weight update frequency mapping set in the historical scene feature library to match and obtain the corresponding weight update frequency; During the update process of the weight update frequency, the update resource occupancy rate of the dynamic weight space relationship model for the specified user is monitored in real time. If it is within the corresponding allowable range, the operation stability of the dynamic weight space relationship model is improved. Otherwise, the update is paused and the update process is restarted when the resource occupancy rate is detected to be within the corresponding allowable range. If the monitored resource occupancy rate is still not within the corresponding allowable range during the preset monitoring period, an intervention warning feedback will be issued to prompt the preset personnel to intervene manually. The scene recommendation scheme is regenerated based on the adaptively updated dynamic weight spatial relationship model. The response rate of the recommendation scheme before and after the update is compared. If the improvement in response rate compared with that before the update is not less than the reference improvement, the adaptive update is confirmed to be effective. Otherwise, the model rollback mechanism is triggered to restore the model to the previous version and add it to the historical scene feature library to improve the mapping relationship between the response rate difference and the weight update strategy.

Images