Scene adaptive recommendation method in smart home based on multi-modal data

Abstract

The invention discloses a scene adaptive recommendation method in smart home based on multi-modal data, and relates to the technical field of multi-modal data processing. According to the method, firstly, scene classification is carried out based on multi-mode sensing data for a smart home scene, a current life scene of a user is accurately recognized, meanwhile, an equipment linkage state is analyzed in combination with a classification result, whether equipment operation is matched with a scene requirement or not is judged, and then a scene mode is recognized according to a linkage analysis result to generate preliminary recommendation; whether the dynamic weight space relation model of the user is updated or not is judged, finally, the recommendation effect is quantified according to the weight adjustment judgment result, the recommendation strategy is optimized through two-way feedback, the adaptability to the user requirement is improved, and the user experience is improved. A full-process closed loop of multi-modal data classification, equipment linkage analysis, mode recognition recommendation, weight dynamic updating and effect quantitative feedback in a smart home scene is realized, and the problem that the intelligent degree of scene recommendation in smart home is not high in a user behavior dynamic change scene is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal data processing, and in particular to a scene adaptive recommendation method in a smart home based on multi-modal data. BACKGROUND

[0002] With the popularization of smart home devices and the upgrading of user needs, single modal data has been difficult to meet the needs of accurate scene recommendation. Currently, smart home systems are gradually transforming from passive control to active service, and multi-modal data (such as vision, audio, environmental sensing, device state data, etc.) has become the core basis for understanding user behavior and scene characteristics, and has promoted scene adaptive recommendation technology to become a key direction for industry development, including smart air conditioners, smart lighting, smart curtains, smart water heaters, smart kitchen appliances, etc.

[0003] The existing technology mainly realizes scene recommendation through three steps: the first step is multi-modal data collection, using cameras, microphones, temperature and humidity sensors, smart sockets, etc. to collect user activity images, voice commands, environmental parameters, and device switch states, etc. data in real time, establishing a multi-source data input channel; the second step is data preprocessing and fusion, filtering and denoising the original data, and then using feature splicing or attention mechanism to map different modal data to a unified feature space, eliminating data heterogeneity; the third step is scene recognition and recommendation, based on machine learning models (such as support vector machines, decision trees), the fused features are classified into scenes, and finally according to the preset scene-device strategy mapping table, the corresponding device control strategy is pushed.

[0004] For example, the user information recommendation method and system based on smart home disclosed in the Chinese patent with publication number CN117349531B includes: based on the corresponding smart home device operation data, multiple smart home users are processed for user matching to determine user matching relationships among the multiple smart home users; among the multiple smart home users, a to-be-processed smart home user for information recommendation is determined; among the multiple smart home users, based on the user matching relationship, each matching smart home user corresponding to the to-be-processed smart home user is determined; based on the user behavior characteristic information corresponding to the to-be-processed smart home user and the user behavior characteristic information corresponding to each matching smart home user, a user information recommendation operation is performed on the to-be-processed smart home user to determine target recommendation data for the to-be-processed smart home user.

[0005] However, in the scenarios of smart home multi-user sharing and dynamic changes of user behavior with life stages (such as adjusting the work and rest schedule with the change of seasons, and changing the activity habits with the addition of new family members), the machine learning model in the existing method is often static, and its parameters and structure are basically fixed after training, which only relies on the initial training data to learn the scene characteristics, so that the learning and adaptation to the dynamic changes of user behavior lag, such as the user changing from going to bed at 23:00 to going to bed at 00:30 due to job changes, at this time, due to the lag, the model will still recommend to start the sleep mode at 23:00 according to the original work and rest schedule. And when fusing multi-modal data, the interaction logic between the multi-modal data and the user is not fully considered, for example, the voice instruction of the user to relax in the evening is not bound and analyzed with the information of the ambient temperature of 26℃ and the living room light brightness of 50% at that time, which makes it easy to have errors in scene recognition due to insufficient utilization of data association, and there is a problem that the intelligent degree of scene recommendation in the smart home is not high, resulting in a low matching degree between the scene recommendation and the actual needs of the user. SUMMARY

[0006] In order to solve the technical problems in the prior art, the embodiments of the present application provide a power plant equipment abnormal rapid identification method based on cloud edge collaboration. The technical scheme is as follows: Step one, scene classification based on multi-modal perception data in the whole scene of smart home, to realize accurate identification of the life scene of the specified user in the whole scene of smart home, and scene linkage state analysis is carried out to evaluate whether the running state of each smart home device in the current life scene matches the demand of the identified life scene, the multi-modal perception data is used to reflect the state information related to the behavior of the specified user, the environmental state and the device running in the smart home scene; Step two, scene mode recognition based on the result of scene linkage state analysis, to generate preliminary recommendation information, and scene weight adjustment determination is carried out to determine whether to adaptively update the dynamic weight space relationship model allocated to the specified user, the dynamic weight space relationship model is used to quantify the influence degree of different scene characteristics on the scene classification of the specified user; Step three, recommendation effect quantification based on the result of scene weight adjustment determination, to quantify the matching degree between the recommended scene and the scene demand of the specified user, and bidirectional feedback of the recommendation strategy is carried out to improve the adaptability of the recommended scene to the demand of the specified user, and finally the final recommendation information is generated.

[0007] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects: 1. The application actively captures the data dynamic changes caused by seasonal changes, work and rest adjustments, and family structure changes in the smart home scene by combining dynamic multi-modal perception data and scene-device linkage analysis, classifies the scene based on these data, accurately identifies the current life scene of the user, analyzes the device linkage state combined with the classification result, judges whether the device operation matches the scene demand, solves the problems of static model adaptation lag and insufficient data correlation; based on the scene linkage state analysis result, scene mode recognition is carried out to generate preliminary recommendations, and whether to update the dynamic weight space relationship model of the user is judged through scene weight adjustment judgment, this process can respond to data dynamic changes in time, avoid recommendation deviation caused by static model parameter solidification, and further optimize the accuracy of scene recommendation; according to the weight adjustment judgment result, the matching degree of recommendation and user demand is quantified, and the adaptability is optimized through bidirectional feedback of the recommendation strategy, which significantly improves the intelligent level and demand matching degree of smart home scene recommendation.

[0008] 2. In view of the defects of insufficient multi-modal data correlation and large scene recognition error of traditional models, the application designs a two-level optimization mechanism: the first level is dynamic feature calling, the standardized multi-modal perception data is input into the pre-trained scene classification model, the historical scene feature library is calculated, and the candidate category set meeting the current demand of the user is output, avoiding the lag of static model relying on initial data only; the second level is scene-specific confidence correction, an independent bias value-correction coefficient lookup table is constructed for different scenes such as sleep and cooking: first, the matching degree of each candidate category is input into the confidence mapping table to calculate the bias value of the preset scene initial matching degree, and then the scene-specific confidence correction lookup table in the historical library is used to multiply the initial assignment by the correction coefficient to obtain the final confidence. This process dynamically calls historical features and scene-specific corrections, fully correlates multi-modal data interaction logic, reduces recognition error, solves the problems of static model adaptation lag and insufficient data correlation, and improves the intelligence and demand matching degree of scene recommendation.

[0009] 3. By converting the standardized multi-modal perception data into a feature vector, and extracting the initial feature vector of each preset scene from the historical scene feature library; then a dynamic weight matrix is constructed to highlight the contribution of each scene classification, avoiding the problem of static model parameter solidification, and an error function is constructed with the goal of minimizing the weighted error to quantify the difference between the current data and the preset scene, fully correlating multi-modal data interaction logic; finally, the partial derivative of the error function is taken and normalized to obtain the scene feature matching degree. The whole process dynamically adapts to user behavior changes, reduces recognition errors caused by insufficient data correlation, and improves the intelligence level and demand matching degree of scene recommendation.

[0010] 4. The process of multi-level effective scene judgment and device closed-loop verification: in the scene judgment link, first filter the effective categories of confidence, directly confirm single category, and determine the result according to the matching degree for multiple categories; if there is no high confidence category, it is classified as a potential category and stored in the correlation layer, otherwise it is classified as to be manually checked, this process accurately locates the current scene of the user, reduces the confusion error of the static model scene. Scene linkage state analysis monitors whether the device operation matches the preset requirements, if not, generates adjustment instructions for execution, re-verifies after adjustment, still does not match, sends an abnormal prompt and troubleshoots the fault, otherwise sends normal information. The whole process dynamically adapts to the change of user behavior, fully associates the multi-modal data logic, and improves the intelligence of scene recommendation and the matching degree of demand.

[0011] 5. The whole process of closed loop of recommendation effect quantization, abnormality judgment and bidirectional feedback is constructed: in the quantization link, the recommendation effect index is calculated with the user confirmation rate as the core index, if it meets the standard, it will be continuously monitored, if it does not meet the standard, it will enter the abnormality judgment link, that is, the bidimensional judgment is carried out combining the recommendation effect index value and the corresponding change rate; in the strategy bidirectional feedback link, the quantization result is input into the mapping set to check the scheme: if it meets the standard, it will be stored as a high-quality strategy for reference, if it does not meet the standard and is not abnormal, it will optimize the feature dimension, if it does not meet the standard and is abnormal, it will update the scene mode library parameters. The whole process dynamically tracks the effect and optimizes the strategy, fully adapts to the change of user behavior, reduces the data association error, and improves the intelligence of recommendation and the matching degree of demand. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating any inventive labor.

[0013] Figure 1 The flowchart of the scene self-adaptive recommendation method in the intelligent home based on multi-modal data provided by the embodiment of the present application; Figure 2 The flowchart of scene classification and confidence assignment provided by the embodiment of the present application; Figure 3 The scene mode recognition flowchart provided by the embodiment of the present application; Figure 4 The recommendation effect quantization flowchart provided by the embodiment of the present application. DETAILED DESCRIPTION

[0014] The technical solutions in the present application will be described below in combination with the drawings.

[0015] In the embodiments of the present application, the words such as "example", "for example" are used to represent an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0016] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, the following will be described in detail in combination with the drawings and specific embodiments.

[0017] The embodiments of the present application provide a scene adaptive recommendation method in smart home based on multi-modal data, as shown in Figure 1 The flowchart of the scene adaptive recommendation method in smart home based on multi-modal data is shown in the figure, and the processing flow of the method can include: Step one, based on multi-modal perception data in the whole scene of smart home, scene classification is carried out to realize accurate identification of the life scene of the specified user in the whole scene of smart home, and scene linkage state analysis is carried out based on the result of scene classification to evaluate whether the running state of each smart home device in the current life scene matches the demand of the identified life scene. Multi-modal perception data is used to reflect the state information related to the behavior of the specified user, the state of the environment and the running of the device in the smart home scene. The multi-modal perception data specifically includes corresponding environmental parameter data, user behavior data, device state data and time correlation data in different seasons (spring, summer, autumn and winter).

[0018] Step two, based on the result of scene linkage state analysis, scene mode recognition is carried out to generate preliminary recommendation information, and scene weight adjustment determination is carried out to determine whether to adaptively update the dynamic weight space relationship model allocated to the specified user. The dynamic weight space relationship model is used to quantify the influence degree of different scene characteristics (such as temperature characteristics in sleep scene, light characteristics in movie watching scene) on the scene classification of the specified user, realizes real-time adaptation and personalized weight adjustment of user behavior dynamic change, breaks the limitation of static model parameter solidification, thereby avoiding the lag and deviation of recommendation caused by factors such as user work and rest adjustment and change of living habits, and significantly improving the accuracy of scene recognition and the fitting degree of preliminary recommendation.

[0019] In step three, the recommendation effect quantification is performed based on the result of the scene weight adjustment determination, to quantify the matching degree between the recommended scene and the specified user scene demand, and meanwhile, the bidirectional feedback of the recommendation strategy is performed according to the result of the recommendation effect quantification, to improve the adaptability of the recommended scene to the specified user demand, and then generate the final recommendation information, realize the closed-loop iteration and continuous optimization of the recommendation strategy, and effectively improve the intelligent level and user satisfaction of the smart home scene recommendation.

[0020] Specifically, the environmental parameter data includes, for example, the indoor real-time temperature (28-35℃) and humidity (60%-80%) and light intensity (800-1200 lux) in summer, the indoor temperature (12-18℃) and PM2.5 concentration (commonly increased to 50-100 μg / m 3 ) and indoor-outdoor temperature difference (15-25℃) in winter; the user behavior data includes, for example, the user's behavior of opening the balcony ventilation and plant maintenance at 19:00-21:00 in spring, and the user's behavior of reading in the living room and opening the humidifier at 20:00-22:00 in autumn, corresponding to the user's position trajectory captured by the camera and the voice command of opening the humidifier collected by the microphone; the device state data includes, for example, the air conditioner in the cooling mode (set temperature 24-26℃) and the fan in the low gear in the sleep scene in summer, and the bathroom heater turned on 10 minutes in advance (temperature set to 30-32℃) and the water heater maintained at 50-55℃ in the bathing scene in winter; the time correlation data includes, for example, the commuting preparation scene concentrated in 7:00-8:00 (corresponding to the morning peak) in winter, and the scene advanced to 6:30-7:30 (due to the short day) in summer, with the seasonal identifier and special holiday marker (such as the extended family reunion scene in winter during the Spring Festival).

[0021] It should be understood that the dynamic weight space relationship model is allocated to each specified user individually, to ensure that the model adapts to the user's personalized demand. The construction takes the user as the core, first extracts the user's historical interaction data (such as scene usage frequency, recommendation confirmation rate, and manual adjustment record) from the historical scene feature library, then combines the scene correlation features in the real-time multi-modal perception data (different seasonal environmental parameters, user behavior, and device state), and optimizes the initial feature weight value through the gradient descent algorithm. For example, if the user frequently adjusts the temperature of the sleep scene, the temperature feature weight is increased, and if the user often manually switches the light of the viewing scene, the light feature weight is increased, and finally a dynamic weight matrix exclusive to the user is formed, i.e., the dynamic weight space relationship model.

[0022] The input to the dynamic weighted spatial relationship model is usually a scene feature vector transformed from real-time multimodal perception data, such as a sleep scene [temperature 24℃, light intensity 10 lux, time 23:30]. The output is the dynamic weight value corresponding to each scene feature, such as temperature weight 0.4, light intensity weight 0.3, and time weight 0.3. These weight values ​​are directly used to determine the feature importance when classifying scenes, ensuring that the classification results are consistent with the user's real-time behavior changes.

[0023] In a specific embodiment, taking the example of a user changing their sleep schedule from 11:00 PM to 12:30 AM during the summer: Using multimodal perception data (environmental parameter 26℃, user issuing a rest command at 11:50 PM, bedroom light on, time-related data 00:20), the sleep scenario is accurately identified. Analysis reveals that the bedroom light being on and the air conditioner set to 28℃ do not meet the sleep scenario requirements. Based on this, an initial recommendation is generated to turn off the bedroom light and adjust the air conditioner to 25℃. Simultaneously, because the user has repeatedly delayed going to sleep recently, it is determined that their dynamic weight spatial relationship model needs to be updated, increasing the weight of time features and voice command features. Through user confirmation of the recommendation, the recommendation effect is quantified, and the strategy is stored in a high-quality sample library. Subsequent similar scenarios will directly call the optimized model to generate recommendations.

[0024] like Figure 2 The flowchart shown illustrates the scene classification and confidence assignment process. It includes first classifying the scene and then assigning confidence scores. Specifically, the process involves: inputting standardized multimodal sensing data into a pre-trained scene classification model; performing classification calculations using a historical scene feature library; and outputting a candidate category set corresponding to the current scene. This candidate category set represents the set of scene categories within the entire smart home scenario corresponding to a specified user that meet the user's current scenario needs. Based on the obtained scene feature matching degree, a confidence score is assigned to each candidate category in the candidate category set to quantify the degree of fit between the current multimodal sensing data and each preset scene category. Simultaneously, an effective scene category determination is performed to identify the corresponding living scenario in the current home environment, avoiding recommendation bias caused by multiple scene confusion. The scene feature matching degree is used to quantify the degree of fit between the current multimodal sensing data and each preset scenario within the entire smart home scenario at the feature level, intuitively reflecting the closeness of the match. A higher matching degree indicates that the current multimodal data better meets the feature requirements of the corresponding preset scenario.

[0025] The specific process for obtaining scene feature matching degree is as follows: Standardized multimodal perception data is transformed into feature vector X, allowing various types of data to participate in calculations in a unified vector form. Standard feature vectors Y1, Y2, Y3...Y4 for each preset scene (such as sleep, cooking, watching movies, etc.) are extracted from the historical scene feature library. N-1 Y NThese are the vector representations of the typical characteristics of each scene, and a dynamic weight matrix W is constructed based on the historical matching accuracy, which assigns different weights to different feature dimensions. The features that contribute more to the scene matching in history have higher weights, reflecting the importance differences of the features.

[0026] The error function is constructed by minimizing the weighted error between the current feature vector and the standard feature vector (i=1, 2, 3..., N), in this error function, i is the number of life scenes, N is the total number of life scenes, is the transpose of the vector (X-Y i ), through the operation of this transpose and the original vector, dynamic weight matrix W, finally a scalar form error value can be obtained, which is used to measure the difference between the current feature vector X and the standard feature vector Y i of a certain preset scene, when X and Y i are closer, the length of the (X-Y i ) vector is smaller, after weighted calculation by dynamic weight matrix W, the value of error function E is smaller, which means the initial matching degree of the current data and the corresponding scene is higher.

[0027] The partial derivative of the error function E with respect to the feature vector X is calculated , through the partial derivative analysis of the influence sensitivity of each feature dimension on the error, if the absolute value of the partial derivative of a certain dimension is small, it means that the feature is highly consistent with the standard scene; combined with the error function value and the partial derivative sensitivity, the scene feature matching degree of each candidate category is obtained through normalization calculation (mapping the [0, ] interval error to the [100, 0] interval matching degree), and the dynamic weight matrix is a symmetric matrix.

[0028] According to the acquired scene feature matching degree, a confidence is assigned to each candidate category in the candidate category set, including: acquiring the scene feature matching degree corresponding to each candidate category respectively, and inputting the scene feature matching degree of each candidate category into the set confidence mapping table respectively, calculating the deviation value of the matching degree of each candidate category and the initial matching degree of the preset scene, the initial matching degree of the preset scene is the matching degree between the preset scene and the ideal standard feature (such as the ideal light, temperature and other feature combinations of the sleep scene) in the acquired historical scene data, which is obtained by averaging a large number of historical feature matching conditions under the normal scene; the deviation values of each candidate category are input into the preset confidence correction lookup table in the historical scene feature library, and the corresponding confidence correction coefficients are mapped to obtain the corresponding confidence correction coefficients, and the corresponding confidence correction coefficients are processed with the initial confidence value of each candidate category to obtain the final confidence of each candidate category. The preset confidence correction lookup table represents a scene-specific correction lookup table with candidate category deviation value as index and confidence correction coefficient as corresponding result, and the scene-specific correction lookup table corresponds to the intelligent home different life scene categories (sleep, cooking, viewing, reading, etc.) one by one.

[0029] Specifically, the pre-trained scene classification model generally refers to a model based on a deep learning framework (such as CNN, LSTM or Transformer) that can automatically extract multi-modal data features and realize scene category prediction, covering sleep, cooking, viewing and other core scene categories of intelligent home. Its construction and training process is as follows: first, collect a large amount of multi-modal data (including different seasonal environment, user behavior and device state data), and label the corresponding scene categories as a training set; then design the model structure (such as using CNN to process environmental / device data, using LSTM to process time-series behavior data, and fusing multi-modal features through attention mechanism); finally, use the training set to iteratively train, optimize the parameters with cross-entropy loss function, adjust the model hyperparameters with the validation set, and complete the pre-training when the scene classification accuracy of the model on the test set meets the standard. Subsequent fine-tuning can be combined with user historical data to adapt to individual needs.

[0030] First, collect the basic features of the whole scene of the intelligent home (such as typical environmental parameters and device state ranges of each scene); then for each user, continuously collect their scene interaction data (such as the scene category confirmed each time, the parameters adjusted manually, and the usage time), and extract the scene feature preferences of the user (such as user A's sleep scene preference temperature 23-25℃, time 23:30-7:00); finally, store the data according to the structure of user ID, scene category, feature benchmark and update time, form a historical scene feature library exclusive to each user, and regularly update the feature benchmark according to the new interaction data of the user to ensure the timeliness of the data.

[0031] In the embodiment, the dynamic weight matrix is combined with the historical matching accuracy to assign feature weights, so that the features that contribute greatly to scene classification are more prominent, and irrelevant features are avoided. A weighted error function is constructed and combined with partial derivative analysis to accurately quantify the differences between data and scenes, and the feature dimension fit degree can also be identified. Normalization processing converts the error into intuitive matching degree, improving the accuracy of judgment. It effectively solves the problem of insufficient correlation of multi-modal data and rough scene matching in the prior art, and reduces multi-scene confusion.

[0032] Further, effective scene category determination is performed, specifically: obtaining the candidate class corresponding to the final confidence not less than the maximum value of the reference confidence interval (i.e. 85%) in the candidate class set, and recording it as an effective scene class, i.e. a candidate class with high confidence; if there is only one effective scene class, the candidate class is directly confirmed as the current scene classification result; if there are multiple effective scene classes, the corresponding effective scene classes are counted to obtain an effective scene set, and the effective scene class corresponding to the highest matching degree in the effective scene set is taken as the final scene classification result.

[0033] It should be understood that taking the effective scene class corresponding to the highest matching degree in the effective scene set as the final scene classification result does not exist the same matching degree, the reason is to meet the actual use demand of the user, if the same matching degree appears, the user will face the confusion of which scene to choose first, in the process of calculating the corresponding matching degree, through multi-dimensional weighting, priority matching user high-frequency scene and other rules, ensure the unique highest matching degree, help users directly lock the optimal result, avoid the selection dilemma.

[0034] If there is no candidate class in the candidate class set with a final confidence not less than the maximum value of the reference confidence interval, the candidate classes are associated according to the corresponding confidence for the associated layer determination, specifically: the candidate classes in the candidate class set with a final confidence within the reference confidence interval (usually set to 60%-85%) are recorded as potential scene classes, i.e. candidate classes with medium confidence, and the corresponding candidate classes are stored in the potential associated layer candidate set to avoid missing user potential demand scenes due to a single confidence threshold; the candidate classes in the candidate class set with a final confidence less than the minimum value of the reference confidence interval (i.e. 60%) are recorded as irrelevant scene classes, i.e. candidate classes with low confidence, and the corresponding candidate classes are stored in the manual checking candidate set to avoid invalid scene interference with the classification result.

[0035] The specific numerical values (60%, 85%) are set by analyzing the correlation between the confidence and the determination accuracy in the historical scene classification data, to ensure that the initial threshold can cover most of the effective scene determination requirements. In actual application, the classification accuracy requirements in different scenes can be adjusted according to the characteristics of the user group (such as the higher demand for scene stability of the elderly users) and the type of home scene (such as the need for more accurate determination in the kitchen scene), to improve the flexibility and applicability of scene determination.

[0036] In this embodiment, the scene classification is screened through the association layer determination. When there is no high-confidence category, the medium-confidence category is classified into the potential association layer to avoid missing the user's potential demand scene (such as the user's temporarily switched leisure scene). At the same time, the low-confidence category is classified into the manual checking layer to exclude invalid interference. This layered determination logic not only ensures the accurate classification of regular scenes, but also provides a reasonable processing path for special scenes, reduces the classification deviation caused by rigid threshold, and provides a more practical demand-based scene basis for subsequent device linkage analysis and recommendation.

[0037] Further, the scene linkage state analysis is performed. The specific process is as follows: if the current device running state monitored (such as the light brightness of the master bedroom in the sleep scene and the air conditioner temperature) does not meet the corresponding preset scene demand (the light brightness of the sleep scene should be ≤10 lux and the air conditioner temperature should be 22-24℃), the device linkage adjustment operation is performed, otherwise, the scene linkage normal confirmation information is sent to the user APP; the device linkage adjustment operation means that the targeted adjustment instruction is generated, such as adjusting the air conditioner temperature of the master bedroom from 26℃ to 23℃, and the generated targeted adjustment instruction is pushed to the corresponding home device through the smart home gateway for execution until the corresponding device state meets the corresponding preset scene demand; after the device linkage adjustment operation, the scene linkage state verification is performed again, if the device running state and the corresponding preset scene demand still do not meet, the scene linkage abnormal prompt is sent to the user APP, and the fault diagnosis process is triggered at the same time, such as checking the communication connection of the home device, otherwise, it is determined that the scene linkage meets the standard and the adjustment parameters are recorded.

[0038] In this embodiment, by correcting the deviation between the device state and the scene demand, the active service ability of the smart home is improved; after adjustment, a secondary verification link is added, if the device still does not meet the standard, the fault diagnosis is triggered and the abnormal prompt is pushed, which can timely find the problems such as device communication failure, to avoid the invalidation of scene experience caused by device abnormality; after meeting the standard, the adjustment parameters are recorded, which can provide reference for the device linkage of the same type of scene in the future, optimize the adjustment strategy accuracy, and at the same time, through the APP synchronous confirmation information or abnormal prompt, the timely understanding of the scene state by the user is ensured.

[0039] For example, Figure 3The scene mode recognition flowchart shown is designed to first obtain multi-modal perception data, generate a matching table in combination with a historical scene feature library, and quantify the matching degree of each scene mode. Through two layers of judgment on whether there is a matching ratio not less than a reference ratio and whether only one mode meets the requirements, the scene is accurately identified: if a single mode meets the requirements, the recommendation is directly determined and generated; if multiple modes meet the requirements, the final result is determined by manually comparing key features; and if no mode meets the requirements, the data to be verified is marked and stored in the library, providing a basis for subsequent matching after supplementing data, ensuring that scene recognition is both automated and capable of dealing with complex and unclear situations.

[0040] It is further understood that scene mode recognition is performed as follows: current multi-modal perception data is obtained, and a scene matching table for visually distinguishing the matching between the current multi-modal perception data and the features of each mode is generated in combination with a historical scene feature library, which represents a database storing each preset scene in a smart home full scene; if the proportion of the number of feature matches of a scene mode in the total number of features of the mode in the scene matching table is not less than a reference ratio (usually set to 75%), the scene mode is determined as the current matching scene, and preliminary recommendation information is generated based on the mode, such as recommending closing the main light and turning on the air conditioner sleep mode if the sleep mode matches the requirements; if the proportion of the number of feature matches of two or more scene modes in the total number of features of the mode is not less than the reference ratio, the feature matching situation of each mode is fed back to prompt the preset personnel to further compare the matching situation of each mode, such as the key feature of the cooking mode being the opening of the exhaust hood and the key feature of the viewing mode being the opening of the television, which is used as the final matching result; and if the proportion of the number of feature matches of all scene modes in the total number of features of the mode is less than the reference ratio, the current multi-modal perception data is marked as to-be-verified data, no scene recommendation is generated, and the data is only stored in the historical scene feature library, waiting for subsequent supplement of user behavior data (such as manual operation device records) for re-matching analysis.

[0041] In the scene weight adjustment determination, the specific process includes: after the completion of this scene mode recognition and recommendation, the response rate of a specified user to the preliminary scene recommendation is collected, the response rate representing the proportion of the number of recommended operations performed by the specified user within a preset period (such as 24 hours) after the generation of the recommendation to the total number of recommendations; if the response rate is greater than or equal to a defined response rate (usually set to 60%), the dynamic weight space relationship model of the specified user does not need to be adaptively updated; if the response rate is less than the defined response rate, it is determined that the dynamic weight space relationship model of the specified user needs to be adaptively updated, specifically as follows: Obtain the difference between the defined response rate and the target response rate, and input it into the preset response rate difference-weight update frequency mapping set in the historical scene feature library to match and obtain the corresponding weight update frequency. During the weight update frequency update process, monitor the update resource occupancy rate of the dynamic weight spatial relationship model for the specified user in real time. Within the corresponding allowable range (usually set to ≤30%), improve the operational stability of the dynamic weight spatial relationship model to ensure that updates are performed during off-peak hours, thus not affecting the operational stability of the model, while also meeting the response needs of the specified user. Otherwise, pause the update and restart the update process once the resource occupancy rate is detected to be within the corresponding allowable range. If the monitored resource occupancy rate is still outside the allowable range during the monitoring period, an intervention warning will be issued to prompt designated personnel to intervene manually. A new scenario recommendation scheme will be generated based on the adaptively updated dynamic weight spatial relationship model. The response rate of the recommendation scheme before and after the update will be compared. If the increase in response rate compared to before the update is not less than the reference increase (usually set at 10%), the adaptive update is confirmed to be effective. Otherwise, a model rollback mechanism will be triggered to restore the model to the version before the update and add it to the historical scenario feature library to improve the mapping relationship between the response rate difference and the weight update strategy, providing a more accurate reference for subsequent weight adjustment judgments for similar users.

[0042] In this embodiment, the aforementioned values ​​are typically set by analyzing historical scene recognition and user response data, combined with the model's operational stability requirements. For example, a reference ratio of 75% corresponds to high scene recognition accuracy, while a defined response rate of 60% matches general user acceptance. In practical applications, these values ​​can be fine-tuned based on user habits (e.g., relaxing the defined response rate for elderly users) and scene complexity (e.g., increasing the reference ratio for kitchen scenes) to balance recognition accuracy and user experience. This example accurately identifies scenes by comparing feature matching ratios and key features, avoiding multi-mode confusion. The data storage mechanism for verification can also accumulate data to optimize subsequent matching. The user response rate is used as the core to determine whether to update the dynamic weighted spatial relationship model, and the update timing is controlled by resource utilization. This ensures that the model meets user needs while guaranteeing stable operation, achieving a coordinated adaptation between scene recognition and model optimization.

[0043] like Figure 4 The flowchart illustrating the recommendation performance quantification process is designed as follows: First, determine if the recommendation performance index meets the target. If it does, continue monitoring. If it doesn't, further determine if there are any abnormal conditions. If an anomaly is found, increase the monitoring frequency and conduct a second assessment. If no anomaly is found, check if the rate of change exceeds a threshold. If it does, continue monitoring; otherwise, supplementary data collection and feedback are provided. After the second assessment, if there are no anomalies and the rate of change meets the requirements, maintain the current strategy. Otherwise, issue an alert for recommendation performance anomalies. This allows for dynamic, hierarchical monitoring and strategy adjustment of recommendation performance, ensuring stable and relevant recommendation results.

[0044] It is further understood that the recommendation effect quantification is performed, specifically: based on the scene recommendation confirmation rate of the specified user in the preset monitoring period and the reference scene recommendation confirmation rate, the recommendation effect index reflecting the matching degree of the current recommendation strategy and the actual demand of the specified user in the scene recommendation process is obtained, that is, the ratio of the obtained scene recommendation confirmation rate and the reference scene recommendation confirmation rate; if the recommendation effect index is not less than the reference recommendation effect index, it is determined that the recommendation effect meets the standard, and the scene recommendation process is continuously monitored, otherwise, it is determined whether there is a recommendation effect abnormal condition, and the recommendation effect abnormal condition indicates that in the preset monitoring period, the recommendation effect index is less than the set recommendation effect index corresponding to the proportion of time length greater than the time length warning value.

[0045] If there is no recommendation effect abnormal condition, the recommendation effect index change rate in the preset monitoring period is obtained, if the obtained recommendation effect index change rate is not greater than the defined change rate (usually set to 5% / hour), the recommendation process is continuously monitored, the recommendation effect index change rate represents the difference between the recommendation effect index at the beginning of the preset monitoring period and the recommendation effect index at the end of the period, and the ratio of the corresponding difference to the time length of the preset monitoring period is calculated. Otherwise, based on the value of the obtained recommendation effect index change rate, supplementary feedback is collected to locate the cause of the corresponding recommendation effect index change; if there is a recommendation effect abnormal condition, the scene recommendation monitoring frequency is increased based on the change rate deviation (such as from monitoring every hour to monitoring every 30 minutes), after the adjustment is completed, secondary determination is performed, specifically: if there is no recommendation effect abnormal condition and the obtained recommendation effect index change rate is not greater than the defined change rate, the current recommendation strategy is maintained, otherwise, the recommendation effect abnormality warning is performed to perform the recommendation effect abnormality feedback, and the intervention is performed to the preset personnel or the person himself through the system pop-up window, short message notification and other ways.

[0046] Among them, the bidirectional feedback of the recommendation strategy is performed according to the result of the recommendation effect quantification, and the specific steps are: The recommendation effect quantification result is obtained, and input into the preset effect quantification result-strategy adjustment direction mapping set in the historical scene feature library to query the corresponding adjustment scheme, and the recommendation effect quantification result includes the recommendation effect index and the recommendation effect index change rate; the corresponding adjustment scheme is queried, and the process is: If the recommended effect quantification result shows that the recommended effect meets the standard, the recommended effect in the current recommended strategy and the corresponding data are supplemented to the corresponding mapping set as a high-quality strategy sample library for reference for subsequent similar user recommended strategies. Meanwhile, the final recommended information conforming to the current needs of the specified user is generated and output based on the current high-quality strategy sample library, such as recommending 23-25℃ air conditioner temperature and 10% light brightness according to the existing sleep mode. If the recommended effect quantification result shows that the recommended effect does not meet the standard but is not abnormal, feature dimension optimization feedback is performed to enhance the adaptability of the dynamic weight space relationship model to the real-time needs of the user, such as increasing the proportion of manual intervention features in the dynamic weight model based on the user's recent operation log. After the feature dimension optimization is completed, the final recommended information is generated and output based on the updated dynamic weight space relationship model.

[0047] If the recommended effect quantification result shows that the recommended effect does not meet the standard and is abnormal, feature scene mode library updating feedback is performed to correct the parameters in the scene mode library that are inconsistent with the user's needs, such as adjusting the temperature parameter threshold of the sleep mode in the scene mode library from 22-24℃ to the user's actual preference of 23-25℃. After the feature scene mode library updating is completed, the final recommended information is generated and output based on the corrected scene mode. The generated final recommended information is pushed to the corresponding specified user APP for visual display. The APP presents the recommended content in the form of clear cards (such as the current recommended sleep mode: air conditioner 24℃, master bedroom light 10%) and provides three interactive options: confirmation execution, custom adjustment, and rejection. If the user selects confirmation execution, the APP directly sends instructions to the smart home gateway to trigger device linkage to execute the recommended scheme. If the user selects custom adjustment (such as changing the air conditioner temperature to 25℃), the adjusted parameters are recorded and updated to the user's historical interaction database to provide data for subsequent model optimization. If the user selects rejection, a simple questionnaire is popped up (such as the reason for rejection: inappropriate timing / recommendation parameters do not meet the needs of the user), and the recommendation is not executed after collecting the corresponding user feedback. At the same time, the rejection record and feedback reason are supplemented to the data source of the recommended effect quantification to provide a reference for the next round of strategy bidirectional feedback.

[0048] The establishment of the preset effect quantification result-strategy adjustment direction mapping set first collects historical recommendation data, extracts effect quantification indicators (such as recommendation confirmation rate, user satisfaction) and corresponding strategy adjustment schemes (such as optimizing feature dimensions, updating scene mode library), and analyzes the association rules between the two. Then, the intervals are divided according to the effect level, and fixed adjustment directions are matched for each interval (such as updating the scene mode library for the poor interval), forming an initial mapping set. Subsequently, the interval boundaries and adjustment directions are continuously corrected combined with new recommendation data to ensure that the mapping logic conforms to the actual needs.

[0049] Similarly, other mapping sets in the embodiments of the present application, such as the confidence mapping table and the preset response rate difference-weight update frequency mapping set, are all based on historical scene data and user interaction records, and are established through statistical feature association rules, division of index intervals, matching of corresponding outputs (such as matching of confidence intervals), and dynamic iteration optimization, to ensure the accuracy and adaptability of the mapping relationship.

[0050] In the present embodiment, by means of exponential and abnormal condition determination, the deviation of the recommended strategy from the user demand is accurately located, and optimization lag caused by fuzzy evaluation is avoided; the bidirectional feedback is adjusted in layers according to the effect result, the strategies that meet the standard are stored in the high-quality sample library for subsequent reference, the optimization model feature dimension is updated when the strategies that do not meet the standard are not abnormal, and the scene mode library is updated when there is an abnormality, so as to ensure that the adjustment direction is highly targeted and blind optimization is avoided; the APP terminal visual display and multi-interaction option design not only enable the user to clearly know the recommended content, but also collect real-time demand through confirmation, adjustment and rejection feedback, and the supplemented data is fed back to the historical library to further improve the strategy; the monitoring frequency and early warning mechanism are improved when there is an abnormality, which can quickly respond to the deviation of the strategy and reduce the damage to the user experience. The whole forms a closed loop, continuously improves the adaptability of the recommended strategy to the user demand, avoids invalid recommendation and strategy ossification, and significantly enhances the intelligence and humanization of the smart home scene recommendation.

[0051] The above embodiments can be implemented in whole or in part by software, hardware (such as a circuit), firmware, or any combination thereof. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0052] It should be understood that the term "and / or" in this text only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In addition, the character " / " in this text generally represents an "or" relationship between the associated objects before and after it, but it can also represent an "and / or" relationship, which can be understood in the context before and after it.

[0053] In various embodiments of the present application, the size of the sequence number of each process described above does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0054] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0055] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A scene-adaptive recommendation method for smart homes based on multimodal data, characterized in that, Includes the following steps: Step 1: Based on multimodal perception data of the entire smart home scenario, scene classification is performed to achieve accurate identification of the life scenario of the specified user in the entire smart home scenario. At the same time, scene linkage status analysis is performed to evaluate whether the operating status of each smart home device in the current life scenario matches the needs of the identified life scenario. The multimodal perception data is used to reflect the status information related to the behavior of the specified user, the environmental status and the operation of the device in the smart home scenario. Step 2: Based on the results of the scene linkage state analysis, scene pattern recognition is performed to generate preliminary recommendation information. At the same time, scene weight adjustment is determined to determine whether to adaptively update the dynamic weight spatial relationship model assigned to the specified user. The dynamic weight spatial relationship model is used to quantify the degree of influence of different scene features on the scene classification of the specified user. Step 3: Based on the results of the scene weight adjustment judgment, the recommendation effect is quantified to quantify the degree of matching between the recommended scene and the needs of the specified user scene. At the same time, two-way feedback of the recommendation strategy is carried out to improve the adaptability of the recommended scene to the needs of the specified user, and then the final recommendation information is generated.

2. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 1, characterized in that, The process of classifying scenarios based on multimodal perception data across all smart home scenarios includes: The standardized multimodal perception data is input into the pre-trained scene classification model, and the classification calculation is performed in combination with the historical scene feature library. The candidate category set corresponding to the current scene is output. The candidate category set represents the set of scene categories that meet the user's current scene needs in the smart home full scene corresponding to the specified user. The confidence level is assigned to each candidate category in the candidate category set based on the obtained scene feature matching degree, so as to quantify the degree of fit between the current multimodal perception data and each preset scene category. At the same time, the effective scene category is determined to identify the living scene corresponding to the current home environment. The scene feature matching degree is used to quantify the degree of fit between the current multimodal perception data and each preset scene in the smart home full scene at the feature level.

3. The scene adaptive recommendation method for smart homes based on multimodal data as described in claim 2, characterized in that, The scene feature matching degree is obtained through the following method: The standardized multimodal perception data is transformed into feature vectors, and the initial feature vectors corresponding to each preset scenario are extracted from the historical scene feature library simultaneously. A dynamic weight matrix is ​​constructed to highlight the contribution of each life scenario in scenario classification. At the same time, with the goal of minimizing the weighted error between the transformed feature vector and the standard feature vector, an error function is constructed to quantify the degree of difference between the current multimodal data and each preset scenario. The error function constructed based on the transformed feature vector is subjected to partial derivative processing, and the result of partial derivative processing is normalized to obtain the scene feature matching degree used to quantify the degree of fit between the current multimodal perception data and each preset scene.

4. The scene adaptive recommendation method for smart homes based on multimodal data as described in claim 3, characterized in that, Assigning confidence scores to each candidate category in the candidate category set based on the acquired scene feature matching degree includes: Obtain the scene feature matching degree corresponding to each candidate category, and input the scene feature matching degree of each candidate category into the set confidence mapping table, and calculate the deviation value between the matching degree of each candidate category and the initial matching degree of the preset scene; The deviation values ​​of each candidate category are input into a preset confidence correction lookup table in the historical scene feature library to map the confidence correction coefficients corresponding to each deviation value. The corresponding confidence correction coefficients are then processed with the initial confidence values ​​of each candidate category to obtain the final confidence of each candidate category. The confidence correction lookup table represents a scene-specific correction lookup table with the candidate category deviation value as the index and the confidence correction coefficient as the corresponding result. The scene-specific correction lookup table corresponds one-to-one with different smart home living scene categories.

5. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 4, characterized in that, The determination of the effective scene category is as follows: Obtain the candidate category whose final confidence score is not less than the maximum value of the reference confidence score interval, and record it as the effective scene category; If there is only one valid scene category, then that candidate category is directly confirmed as the current scene classification result; If there are multiple valid scene categories, the corresponding valid scene categories will be counted to obtain a set of valid scenes, and the valid scene category with the highest matching degree in the set of valid scenes will be used as the final scene classification result. If there is no candidate category in the candidate category set with a final confidence level not less than the maximum value of the reference confidence level interval, then the candidate categories are determined by the association layer based on the corresponding confidence level. The specific process is as follows: Candidate categories whose final confidence level is within the reference confidence level range are recorded as potential scenario categories, and the corresponding candidate categories are stored in the candidate set of the potential association layer to avoid missing potential user demand scenarios due to a single confidence level threshold. Candidate categories whose final confidence level is less than the minimum value of the reference confidence level interval are recorded as irrelevant scene categories, and the corresponding candidate categories are stored in the candidate set to be manually verified to avoid invalid scenes interfering with the classification results.

6. The scene adaptive recommendation method for smart homes based on multimodal data as described in claim 5, characterized in that, The specific process for performing scene linkage state analysis is as follows: If the monitored current device operating status does not meet the corresponding preset scenario requirements, the device linkage adjustment operation will be performed; otherwise, a scenario linkage normal confirmation message will be sent to the user's APP. The device linkage adjustment operation refers to generating a targeted adjustment command, which is then pushed to the corresponding home device through the smart home gateway for execution until the device status is detected to meet the corresponding preset scenario requirements. After performing device linkage adjustment operations, the scene linkage status verification is re-executed. If the device operating status still does not meet the corresponding preset scene requirements, a scene linkage exception prompt is sent to the user's APP; otherwise, the scene linkage is determined to meet the standards and the adjustment parameters are recorded.

7. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 1, characterized in that, The specific process for scene pattern recognition is as follows: Acquire current multimodal sensing data and combine it with a historical scene feature library to generate a scene matching comparison table for visually distinguishing the matching situation between the current multimodal sensing data and the features of each mode. The historical scene feature library represents a database that stores various preset scenes under the full smart home scenario. If the proportion of the number of feature matches for a certain scene pattern in the scene matching lookup table to the total number of features for that pattern is not less than the reference proportion, then that scene pattern is determined as the current matching scene, and preliminary recommendation information is generated based on that pattern. If the proportion of feature matching in two or more scene modes to the total number of features in that mode is not less than the reference proportion, then feedback on the feature matching status of each mode will be provided to prompt the pre-selected personnel to further compare the matching status of each mode, and this will be used as the final matching result. If the proportion of feature matching numbers for all scene modes to the total number of features for that mode is less than the reference proportion, the current multimodal perception data will be marked as data to be verified, and scene recommendations will not be generated for the time being; it will only be stored in the historical scene feature library.

8. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 7, characterized in that, The specific process of determining scene weight adjustment includes: After scene pattern recognition and recommendation are completed, the response rate of a specified user to the initial scene recommendation is collected. The response rate represents the proportion of the number of times the specified user performs the recommendation operation within a preset period after the recommendation is generated to the total number of recommendations. If the response rate is greater than or equal to the defined response rate, there is no need to adaptively update the dynamic weight space relationship model for the specified user. If the response rate is less than the defined response rate, it is determined that the dynamic weight space relationship model for the specified user needs to be adaptively updated, specifically as follows: Obtain the difference between the defined response rate and the response rate, and input it into the preset response rate difference-weight update frequency mapping set in the historical scene feature library to match and obtain the corresponding weight update frequency; During the update process of the weight update frequency, the update resource occupancy rate of the dynamic weight space relationship model for the specified user is monitored in real time. If it is within the corresponding allowable range, the operation stability of the dynamic weight space relationship model is improved. Otherwise, the update is paused and the update process is restarted when the resource occupancy rate is detected to be within the corresponding allowable range. If the monitored resource occupancy rate is still not within the corresponding allowable range during the preset monitoring period, an intervention warning feedback will be issued to prompt the preset personnel to intervene manually. The scene recommendation scheme is regenerated based on the adaptively updated dynamic weight spatial relationship model. The response rate of the recommendation scheme before and after the update is compared. If the improvement in response rate compared with that before the update is not less than the reference improvement, the adaptive update is confirmed to be effective. Otherwise, the model rollback mechanism is triggered to restore the model to the previous version and add it to the historical scene feature library to improve the mapping relationship between the response rate difference and the weight update strategy.

9. The scene-adaptive recommendation method for smart homes based on multimodal data as described in claim 1, characterized in that, The quantification of recommendation effectiveness is specifically as follows: Based on the scenario recommendation confirmation rate of a specified user within a preset monitoring period and the reference scenario recommendation confirmation rate, obtain the recommendation effect index, which reflects the degree of matching between the current recommendation strategy and the actual needs of the specified user during the scenario recommendation process; If the recommendation effect index is not less than the reference recommendation effect index, the recommendation effect is determined to be up to standard, and the scene recommendation process continues to be monitored. Otherwise, it is determined whether there are abnormal conditions for recommendation effect. Abnormal conditions for recommendation effect mean that within the preset monitoring period, the recommendation effect index is less than the percentage of time corresponding to the set recommendation effect index is greater than the time warning value. If there are no abnormal conditions for the recommendation effect, the rate of change of the recommendation effect index within the preset monitoring period is obtained. If the rate of change of the obtained recommendation effect index is not greater than the defined rate of change, the recommendation process is continuously monitored. Otherwise, based on the value of the obtained rate of change of the recommendation effect index, supplementary data collection and feedback are carried out to locate the cause of the corresponding change in the recommendation effect index. If there are abnormal conditions in the recommendation effect, the frequency of scene recommendation monitoring will be increased based on the deviation of the rate of change. After the adjustment is completed, a second judgment will be made. The second determination is as follows: if there are no abnormal conditions for the recommendation effect and the rate of change of the obtained recommendation effect index is not greater than the defined rate of change, then the current recommendation strategy is maintained; otherwise, an abnormal recommendation effect warning is issued to provide feedback on the abnormal recommendation effect.

10. The scene adaptive recommendation method for smart homes based on multimodal data as described in claim 9, characterized in that, The specific steps for implementing the two-way feedback of the recommendation strategy are as follows: Obtain the recommendation effect quantification result and input it into the preset effect quantification result-strategy adjustment direction mapping set in the historical scene feature library to query the corresponding adjustment scheme. The recommendation effect quantification result includes the recommendation effect index and the rate of change of the recommendation effect index. The process of querying the corresponding adjustment scheme is as follows: If the quantitative results of the recommendation effect show that the recommendation effect meets the standard, the recommendation effect and corresponding data in the current recommendation strategy will be added to the corresponding mapping set as a high-quality strategy sample library for reference in subsequent recommendation strategies for similar users. At the same time, based on the current high-quality strategy sample library, the final recommendation information that meets the current needs of the specified user will be generated and output. If the quantitative results of the recommendation effect show that the recommendation effect does not meet the standard but there are no abnormalities, then feature dimension optimization feedback will be carried out to enhance the adaptability of the dynamic weight space relationship model to the real-time needs of users. After the feature dimensions are optimized, the final recommendation information is generated and output based on the updated dynamic weight space relationship model. If the quantitative results of the recommendation effect show that the recommendation effect is not up to standard and there are abnormalities, the feature scene pattern library will be updated and feedback will be provided to correct the parameters in the scene pattern library that are out of touch with user needs. Once the feature scene pattern library is updated, the final recommendation information will be generated and output based on the corrected scene patterns. The generated final recommendation information will be pushed to the corresponding designated user's APP for visualization.

Images