Smart home-oriented privacy protection type multi-source data fusion processing method and device

Abstract

This application provides a privacy-preserving multi-source data fusion processing method and apparatus for smart homes, belonging to the field of data fusion technology. It involves performing behavioral semantic analysis on a home scene data sequence to extract behavioral feature parameters that correlate user behavior patterns with device operating states, and generating a privacy sensitivity coefficient based on these parameters. Based on the privacy sensitivity coefficient and the home scene data sequence, a privacy-constrained feature representation is generated in the feature representation layer. A cross-device scene semantic association model is established based on the privacy-constrained feature representation, and the collaborative relationship between changes in home device states and user behavior patterns is semantically encoded to obtain scene association features. Privacy-driven multi-source data fusion is then performed based on the privacy sensitivity coefficient and the scene association features to generate a fused data representation result. Using the scheme of this application, effective data fusion under user privacy protection conditions can be achieved during the multi-source data fusion process in smart homes.

Description

Technical Field

[0001] This application relates to the field of data fusion technology, and more specifically, to a privacy-preserving multi-source data fusion processing method and apparatus for smart homes. Background Technology

[0002] With the rapid development of IoT technology and the widespread adoption of smart home devices, modern home environments are equipped with a wide variety of smart terminal devices, including environmental sensors, smart appliances, security equipment, and voice interaction devices. These devices continuously generate massive amounts of multi-source heterogeneous data, collectively forming a comprehensive digital mapping of the home environment. By fusing and processing this multi-source data, we can deeply mine user behavior patterns, device operating rules, and scene context information, thereby providing smart home systems with more precise automated control, personalized services, and anomaly detection capabilities.

[0003] However, in existing technologies, the multi-source data fusion processing of smart homes faces a prominent contradiction between privacy protection and data availability. Traditional multi-source data fusion methods typically aggregate and centrally process raw data collected from various terminal devices. While this approach can preserve the original information of the data to the greatest extent, it also exposes sensitive privacy data from users' daily lives to potential leakage risks. Another type of existing technology uses a uniform desensitization or anonymization method to protect the privacy of all data. Although this method reduces the risk of privacy leakage to some extent, it often fails to differentiate the privacy sensitivity of different data, resulting in insufficient protection of highly sensitive data and excessive protection of less sensitive data, leading to poor privacy protection or a serious decrease in data availability. In addition, existing fusion methods often focus on simple aggregation at the data level, making it difficult for the fusion results to truly reflect the complete semantic information of the home scene, affecting the accurate understanding of user intent and scene status by upper-layer applications. Therefore, how to achieve effective data fusion under the condition of user privacy protection in the process of multi-source data fusion in smart homes, so as to reduce the analytical bias caused by privacy protection on data utilization, has become a challenge for the industry. Summary of the Invention

[0004] This application provides a privacy-preserving multi-source data fusion processing method and apparatus for smart homes, which can effectively fuse data under the condition of protecting user privacy during the multi-source data fusion process in smart homes.

[0005] In a first aspect, this application provides a privacy-preserving multi-source data fusion processing method for smart homes, comprising the following steps: Acquire multi-source raw data generated by various terminal devices in the smart home environment, and construct a unified home scene data sequence based on device identifiers, timestamps, and home scene identifiers; Behavioral semantic parsing is performed on the home scene data sequence to extract behavioral feature parameters that represent the correlation between user behavior patterns and device operating status, and a privacy sensitivity coefficient is generated based on the extracted behavioral feature parameters to describe the degree of data privacy sensitivity. Based on the privacy sensitivity coefficient, differentiated feature perturbation and structured desensitization processing are applied to data of different privacy levels in the family scene data sequence, thereby generating privacy-constrained feature representations in the feature representation layer; Based on the privacy constraint feature representation, a cross-device scene semantic association model is established, and then the collaborative relationship between changes in home device status and user behavior patterns is semantically encoded to obtain scene association features. Privacy-driven multi-source data fusion is performed based on the privacy sensitivity coefficient and the scene association features to generate a fused data representation that balances privacy protection and data availability.

[0006] In conjunction with the first aspect, one possible implementation method for constructing a unified home scene data sequence based on device identifier, timestamp, and home scene identifier specifically includes: Extract the device identifier, original timestamp, and home scene identifier corresponding to each data point from the acquired multi-source raw data; The extracted device identifier is anonymized and mapped, and the original timestamp is time-synchronized and calibrated. Based on the anonymized device identifier, the calibrated timestamp, and the home scene identifier, the multi-source raw data is reorganized into a home scene data sequence with a unified format and time order.

[0007] In conjunction with the first aspect, in one possible implementation, the behavioral semantic parsing of the home scene data sequence to extract behavioral feature parameters representing the correlation between user behavior patterns and device operating states specifically includes: The home scene data sequence is divided into time windows, and the temporal triggering relationship between user behavior events and device status change events is identified within each time window; The frequency and confidence of the aforementioned time-series triggering relationships within multiple time windows are statistically analyzed. Based on the obtained statistical results, behavioral feature parameters are extracted to characterize the relationship between user behavior patterns and device operating status.

[0008] In conjunction with the first aspect, in one possible implementation, generating a privacy sensitivity coefficient to describe the degree of data privacy sensitivity based on extracted behavioral feature parameters specifically includes: Determine the basic privacy sensitivity weight corresponding to each extracted behavioral feature parameter; The initial sensitivity of the scene is calculated by weighting and fusing the basic privacy sensitivity weights of multiple behavioral feature parameters and combining them with the actual values ​​of each behavioral feature parameter within the current time window. The initial sensitivity of the scenario is dynamically adjusted based on the time period attribute of the current time window and the scenario type corresponding to the family scenario identifier, generating a privacy sensitivity coefficient to describe the degree of data privacy sensitivity.

[0009] In conjunction with the first aspect, in one possible implementation, differentiated feature perturbation and structured desensitization processing are applied to data of different privacy levels in the family scene data sequence based on the privacy sensitivity coefficient, thereby generating privacy-constrained feature representations in the feature representation layer. Specifically, this includes: Based on the privacy sensitivity coefficient, each data record in the family scene data sequence is classified into privacy levels, and corresponding differentiated privacy protection strategies are matched for data of different privacy levels. According to the matching differentiated privacy protection strategy, feature perturbation or structured desensitization processing is applied to the data records of the corresponding privacy level to generate processed desensitized data units; The desensitized data units are reorganized according to their original temporal order, and then dimensional reduction and feature compression are performed to generate privacy-constrained feature representations in the feature representation layer.

[0010] In conjunction with the first aspect, in one possible implementation, establishing a cross-device scene semantic association model based on the privacy-constrained feature representation specifically includes: Using the privacy constraint feature representation as input, a network structure is constructed to capture the long-distance dependency relationship between different devices and between devices and user behavior; The scene semantic association model is trained using a self-supervised learning method, enabling it to learn the inherent synergistic relationship between changes in the state of home devices and user behavior patterns, thus obtaining the trained scene semantic association model.

[0011] In conjunction with the first aspect, one possible implementation involves semantically encoding the collaborative relationship between changes in home device status and user behavior patterns to obtain scene-related features, specifically including: Obtain the trained scene semantic association model and input the privacy constraint feature representation generated in real time into the scene semantic association model; Through forward inference computation of the scene semantic association model, the input privacy-constrained feature representation is mapped to a latent space rich in scene semantic information to obtain the initial encoded features; The initial encoded features are post-processed and dimensionally standardized to generate scene association features that characterize the collaborative relationship between changes in the state of home devices and user behavior patterns in the current scenario.

[0012] In conjunction with the first aspect, in one possible implementation, privacy-driven multi-source data fusion is performed based on the privacy sensitivity coefficient and the scene-related features to generate a fused data representation that balances privacy protection and data usability. Specifically, this includes: The contribution weight of the scene association features in the fusion process is determined based on the privacy sensitivity coefficient, and then the scene association features from multiple sources are weighted and fused. The fused feature vectors are post-processed and used to enhance their usability, resulting in a fused data representation that balances privacy protection and data usability.

[0013] In conjunction with the first aspect, in one possible implementation, the multi-source raw data includes environmental sensor data, device operating status data, and user interaction behavior data.

[0014] Secondly, this application provides a privacy-preserving multi-source data fusion processing device for smart homes, comprising: The acquisition module is used to acquire multi-source raw data generated by various terminal devices in the smart home environment, and construct a unified home scene data sequence based on device identifiers, timestamps, and home scene identifiers; The processing module is used to perform behavioral semantic parsing on the home scene data sequence, extract behavioral feature parameters that represent the relationship between user behavior patterns and device operating status, and generate a privacy sensitivity coefficient to describe the degree of data privacy sensitivity based on the extracted behavioral feature parameters. The processing module is also used to perform differentiated feature perturbation and structured desensitization processing on data of different privacy levels in the family scene data sequence according to the privacy sensitivity coefficient, thereby generating privacy-constrained feature representations in the feature representation layer; The processing module is also used to establish a cross-device scene semantic association model based on the privacy constraint feature representation, and then semantically encode the collaborative relationship between changes in the state of home devices and user behavior patterns to obtain scene association features; The execution module is used to perform privacy-driven multi-source data fusion based on the privacy sensitivity coefficient and the scene association features, thereby generating a fused data representation result that balances privacy protection and data availability.

[0015] The technical solution provided in this application has the following beneficial effects: This application's solution, firstly, identifies the correlation between user behavior patterns and device operating status by performing behavioral semantic analysis on home scene data sequences, and extracts behavioral feature parameters to further generate privacy sensitivity coefficients. This identifies the sensitivity level of privacy information in the data, thus providing a basis for privacy level identification in subsequent data processing, establishing a foundation for privacy identification in the early stages of data processing, and improving the targeted nature of privacy protection during subsequent data fusion. Secondly, based on the privacy sensitivity coefficients, differentiated feature perturbation and structured desensitization processing are implemented on data of different privacy levels, so that highly sensitive information is more strongly protected, while low-sensitivity information maintains a high degree of data integrity. This forms a privacy-constrained feature representation at the feature representation layer, which can reduce the loss of effective data information while protecting privacy information, providing a usable data foundation for subsequent semantic analysis and data fusion. Then, in the privacy-constrained feature representation... Based on this, a cross-device scenario semantic association model is established. Through semantic encoding analysis, the collaborative relationship between changes in home device status and user behavior patterns is obtained, yielding scenario association features. This model can still extract key association information between multiple devices under privacy-controlled conditions, thereby improving the smart home system's understanding of home scenario states. Finally, based on the privacy sensitivity coefficient and scenario association features, privacy-driven multi-source data fusion processing is performed. Privacy parameters are used to adjust the fusion strategy, ensuring feature integration while protecting sensitive information. This generates a fused data representation that balances privacy protection and data usability, enabling effective data utilization in smart home scenario analysis. In summary, this scheme can achieve effective data fusion under user privacy protection conditions during the multi-source data fusion process in smart homes, reducing analytical biases caused by privacy protection in data utilization. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is an exemplary flowchart of a privacy-preserving multi-source data fusion processing method for smart homes, as shown in some embodiments of this application. Figure 2 This is an exemplary flowchart illustrating the implementation of behavioral semantic parsing according to some embodiments of this application; Figure 3 This is an exemplary flowchart illustrating the generation of privacy constraint feature representations according to some embodiments of this application; Figure 4This is a schematic diagram of the structure of a privacy-preserving multi-source data fusion processing device for smart homes, as shown in some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device that implements a privacy-preserving multi-source data fusion processing method for smart homes, according to some embodiments of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] refer to Figure 1 The figure is an exemplary flowchart of a privacy-preserving multi-source data fusion processing method for smart homes, according to some embodiments of this application. This privacy-preserving multi-source data fusion processing method for smart homes mainly includes the following steps: In step 101, multi-source raw data generated by various terminal devices in the smart home environment are obtained, and a unified home scene data sequence is constructed based on device identifiers, timestamps, and home scene identifiers.

[0020] In practical implementation, acquiring multi-source raw data generated by various terminal devices in a smart home environment can be achieved in the following way: A smart home gateway with multi-protocol access capability is deployed within the home network as a unified data acquisition node. This gateway listens to and receives various types of data reported from different terminal devices in real time through built-in wireless communication modules. Specifically, environmental sensor data is acquired by collecting numerical data output from devices such as temperature and humidity sensors, air quality detectors, light sensors, and human infrared detectors through periodic polling by the gateway or proactive reporting by the devices. Device operating status data is acquired by subscribing to smart device status change topics or receiving device heartbeat packets to collect device-level operating information such as the power on / off status of smart TVs, the operating mode and set temperature of smart air conditioners, the on / off status and unlocking method of smart door locks, the brightness and color temperature of smart lights, and the current power and cumulative energy consumption of various home appliances. User interaction behavior data is acquired by capturing the control command stream generated when the user interacts with the device, specifically including data from the local... The system parses voice commands issued by users through smart speakers in the local network, records device control operations remotely triggered by users via mobile apps from the home automation system, and collects human-computer interaction events such as user gestures, touch clicks, and facial recognition triggers from smart panels or sensors. During data collection, the gateway adds a receiving timestamp to each received raw data packet and parses its unique identifier from the source device. Simultaneously, it performs integrity checks and outlier filtering, discarding garbled data caused by transmission errors or data points that clearly exceed reasonable physical limits. Finally, the three types of data that pass verification are temporarily stored in a local cache queue according to their original format, forming an unprocessed initial data set consisting of device identifiers, timestamps, and raw data values. This serves as multi-source raw data generated by various terminal devices in the smart home environment, including environmental sensor data, device operating status data, and user interaction behavior data. Other methods can also be used in other embodiments, which are not specifically limited here.

[0021] It should be noted that the multi-source raw data in this application refers to the underlying data records that retain objective values ​​of the operation of various terminal devices and environmental changes in the smart home environment, as well as the original traces of user interaction.

[0022] In some embodiments, constructing a unified home scene data sequence based on device identifier, timestamp, and home scene identifier can be achieved through the following steps: Extract the device identifier, original timestamp, and home scene identifier corresponding to each data point from the acquired multi-source raw data; The extracted device identifier is anonymized and mapped, and the original timestamp is time-synchronized and calibrated. Based on the anonymized device identifier, the calibrated timestamp, and the home scene identifier, the multi-source raw data is reorganized into a home scene data sequence with a unified format and time order.

[0023] In specific implementation, extracting the device identifier, original timestamp, and associated home scene identifier for each data entry from the acquired multi-source raw data can be achieved in the following way: First, read the unprocessed initial data set consisting of the device identifier, timestamp, and original data value temporarily stored in the local cache queue. For each data record, parse the device identifier used to uniquely identify the source device from the protocol field in its data packet header. The device identifier includes the device's media access control address, product serial number, or short address automatically assigned by the gateway. Simultaneously, read the original timestamp recorded and reported by the device itself from the data packet. This original timestamp is the local time when the device event occurred. Then, according to the pre-configured home topology mapping table, associate and match the device identifier with the home scene identifier. The home topology mapping table records the room area to which each device identifier belongs, such as bedroom, living room, kitchen, and the associated home network identifier, thereby determining the home scene identifier corresponding to each data record. Finally, temporarily bind the extracted device identifier, original timestamp, and home scene identifier with the original data value to form a data tuple to be processed containing complete context information.

[0024] In specific implementation, the anonymization mapping of the extracted device identifier and the time synchronization calibration of the original timestamp can be achieved in the following way: First, for the extracted device identifier, a preset identifier anonymization mapping table or hash function is called to convert the real device identifier into an anonymized device identifier unique within the home scenario. For example, a key-based hash message authentication code algorithm is used to calculate the original device address, generating a fixed-length meaningless string as the anonymized device identifier, and ensuring that the same real device is mapped to the same anonymous identifier within the same processing cycle; simultaneously, for the extracted original timestamp, based on the home network... Using the system clock of the gateway as the reference time source, the gateway itself is periodically calibrated using the Network Time Protocol to ensure the accuracy of the reference time. Then, the time deviation between the original timestamp of the device and the timestamp received by the gateway is calculated. For data with a deviation within a preset threshold range, the timestamp received by the gateway is used as the calibrated timestamp. For data with a deviation exceeding the threshold, linear compensation correction is performed according to the historical clock drift pattern of the device. Finally, a unified calibrated timestamp based on the gateway time is generated for each data record. After the above processing is completed, the anonymized device identifier, the calibrated timestamp, and the original data value are recombined to form a desensitized data unit that can be used in subsequent steps.

[0025] In practical implementation, the reorganization of multi-source raw data into a family scene data sequence with a unified format and time order, based on the anonymized device identifier, the calibrated timestamp, and the family scene identifier, can be achieved in the following way: First, the anonymized data units composed of the anonymized device identifier, the calibrated timestamp, the family scene identifier, and the original data values ​​are used as basic data elements. Grouping is then performed according to the family scene identifier, grouping data belonging to the same family scene (e.g., the same residence or apartment) into the same processing partition. Then, within each processing partition, all data units are globally sorted using the calibrated timestamp as the primary key, forming a data stream arranged strictly in ascending chronological order. Next, a unified serialized data structure is defined. The structure is organized in the form of a four-tuple, which sequentially includes a home scene identifier, a calibrated timestamp, an anonymized device identifier, and a data value vector after format normalization. The data value vector is uniformly converted into double-precision floating-point numbers or standardized text encoding format to ensure the comparability of the same type of data reported by different devices. Finally, the sorted data units are written to a distributed message queue or time series database for persistent storage according to the four-tuple format, thereby generating a home scene data sequence that is cross-device and cross-protocol but has a unified timeline and a unified identifier system. This sequence completely records the state trajectory of each anonymous device over time in a specific home scene. Other methods can also be used in other embodiments, which are not limited here.

[0026] It should be noted that, in this application, the device identifier refers to the identity code used to uniquely distinguish different terminal devices in a smart home environment; the original timestamp in this application refers to the moment of event occurrence recorded by the terminal device itself and reported along with the data, which is used to mark the actual time point of the device status change or user interaction behavior corresponding to each data entry; the home scene identifier in this application refers to the classification code indicating the specific home area or spatial environment to which each data record belongs, which is used to aggregate scattered multi-source data according to the spatial dimension, clarify the physical context of data generation, and provide a spatial constraint basis for realizing cross-device data correlation analysis within the same scene; the home scene data sequence in this application refers to the standardized time-series data structure used to convert heterogeneous and discrete raw data into a unified time axis and a unified identifier system.

[0027] In step 102, behavioral semantic parsing is performed on the home scene data sequence to extract behavioral feature parameters that characterize the relationship between user behavior patterns and device operating status, and a privacy sensitivity coefficient is generated based on the extracted behavioral feature parameters to describe the degree of data privacy sensitivity.

[0028] In some embodiments, reference Figure 2As shown, this diagram is an exemplary flowchart for implementing behavioral semantic parsing in some embodiments of this application. In this embodiment, behavioral semantic parsing of the home scene data sequence to extract behavioral feature parameters representing the correlation between user behavior patterns and device operating states can be achieved through the following steps: In step 1021, the home scene data sequence is divided into time windows, and the temporal triggering relationship between user behavior events and device status change events is identified within each time window; In step 1022, the frequency and confidence of the occurrence of the time-series triggering relationship within multiple time windows are statistically analyzed; In step 1023, behavioral feature parameters are extracted based on the obtained statistical results to characterize the relationship between user behavior patterns and device operating status.

[0029] In specific implementation, the home scene data sequence is divided into time windows. Identifying the timing relationship between user behavior events and device status change events within each time window can be achieved as follows: First, a continuous data stream is read from the home scene data sequence. Based on a preset time window length, such as 30 seconds, 5 minutes, or a dynamically adjusted sliding window size according to the specific scenario, the continuous data sequence is divided into multiple equal-length or overlapping time window segments. Each time window contains calibrated timing data reported by all anonymized devices within that time period. Then, within each time window, the data records are divided into two main categories: user behavior events and device status change events. This includes events with clear user intent, such as voice command initiation, mobile application control, and gesture sensing triggers, parsed from user interaction behavior data. Device state change events include records reflecting changes in device state, such as numerical jumps, on / off switching, and mode conversions detected from environmental sensor data and device operating status data. Next, a time-series-based pattern matching method is used to identify event pairs with a fixed temporal order within the same time window or across consecutive windows. For example, if a user behavior event is accompanied by a state change of a specific device within a short time threshold after the event, the event pair is marked as a candidate temporal trigger relationship, and the type of trigger relationship is recorded, including the behavior event type, target device type, and response delay time.

[0030] In specific implementation, the frequency and confidence of the time-series triggering relationships within multiple time windows can be achieved as follows: First, the complete time range covered by the home scene data sequence is taken as a statistical period, and the time-series triggering relationship records identified in all time windows within this period are traversed; for each type of time-series triggering relationship, such as the relationship type "voice command to turn on the TV", the total number of event pairs appearing in all time windows is counted as the frequency of occurrence of the relationship; simultaneously, the confidence index of the relationship is calculated, specifically using a conditional probability calculation method based on user behavior events, that is, counting a certain user behavior... The number of times a specific device state change event occurs within a preset response time threshold after an event occurs is divided by the total number of times the user behavior event occurs throughout the entire statistical period. The resulting ratio is used as the confidence level of the time-series triggering relationship, which characterizes the reliability of the device response triggered by the user behavior. In addition, for cases where the same user behavior may trigger multiple device responses, the independent confidence levels between the behavior and each device response are calculated separately, and the temporal order relationship between each response is recorded. Finally, all the frequency and confidence data obtained from the statistics are aggregated according to the home scene identifier and time window information to form a statistical result set across the time dimension.

[0031] In specific implementation, the behavioral feature parameters used to characterize the correlation between user behavior patterns and device operating status, based on the obtained statistical results, can be implemented in the following way: First, select time-series triggering relationships with a confidence level higher than a preset threshold (e.g., higher than 80%) from the obtained statistical result set as strong correlations, and use their occurrence frequency and average response delay as basic behavioral feature parameters; then, for user behavior patterns under each home scenario identifier, extract a series of quantitative parameters, specifically including the total frequency of various behaviors initiated by users per unit time, the average number of devices triggering device responses for various behaviors, the average time interval between behaviors and responses, and the transition probability matrix between different behavioral events, used to describe... This paper describes the behavioral sequence patterns of users performing multiple operations continuously within a scenario. Simultaneously, it extracts parameters from the perspective of device operating status, including the device's state maintenance time triggered by user behavior, the device's state change patterns under different behavior triggers, and the frequency of collaborative work indirectly formed between devices through user behavior. Finally, all extracted quantitative parameters are structured and encapsulated according to home scenario identifiers, time window information, and parameter types to form a multi-dimensional behavioral feature parameter. This parameter fully characterizes the complex patterns of how user behavior drives device state changes and how devices are associated through user behavior within the home scenario. Other methods can also be used in other embodiments, and are not limited here.

[0032] It should be noted that the temporal triggering relationship in this application refers to a fixed temporal sequence between the occurrence of user behavior events and specified device state change events within the same time window or across consecutive time windows, which is used to identify the causal or accompanying pattern between user operations and device responses; the occurrence frequency in this application refers to quantifying the universality and stability of the association between user behavior and device response, reflecting the degree of repetition of the behavior pattern in the user's daily life; the confidence level in this application refers to measuring the reliability and certainty of user behavior triggering device responses, used to distinguish between accidental associations and stable behavioral habits; the behavioral characteristic parameters in this application refer to numerical representations used to characterize how user behavior drives device state changes and how devices are associated through user behavior.

[0033] In some embodiments, generating a privacy sensitivity coefficient to describe the degree of data privacy sensitivity based on extracted behavioral feature parameters can be achieved through the following steps: Determine the basic privacy sensitivity weight corresponding to each extracted behavioral feature parameter; The initial sensitivity of the scene is calculated by weighting and fusing the basic privacy sensitivity weights of multiple behavioral feature parameters and combining them with the actual values ​​of each behavioral feature parameter within the current time window. The initial sensitivity of the scenario is dynamically adjusted based on the time period attribute of the current time window and the scenario type corresponding to the family scenario identifier, generating a privacy sensitivity coefficient to describe the degree of data privacy sensitivity.

[0034] In specific implementation, determining the basic privacy sensitivity weight corresponding to each extracted behavioral feature parameter can be achieved in the following way: First, a privacy sensitivity mapping rule base is constructed. This rule base predefines the correspondence between different types of behavioral feature parameters and basic privacy sensitivity weights. The correspondence is set based on general understanding in the field of privacy protection. For example, behavioral feature parameters directly related to user biometrics, such as voiceprint features of voice commands and face recognition trigger events, are given higher basic weights; behavioral feature parameters related to user daily life patterns, such as sleep time, wake-up time, and patterns of leaving and returning home, are given medium basic weights; and behavioral feature parameters related to device operating status are given higher basic weights. Behavioral feature parameters related to environmental changes, such as temperature setting changes and light switching frequency, are assigned lower base weights. Then, for each extracted behavioral feature parameter, its parameter type is identified, and a base privacy sensitivity weight value matching that type is searched from the mapping rule base. For new behavioral feature parameters not explicitly listed in the rule base, a calculation method based on the similarity between the parameter and known sensitive attributes is adopted. For example, by calculating the cosine similarity between the parameter and the labeled sensitive parameters in the feature space, the base weight of the known parameter with the highest similarity is used as the initial weight of the new parameter, thereby ensuring that each behavioral feature parameter is assigned a quantified base privacy sensitivity weight.

[0035] In specific implementation, the initial sensitivity of the scene is calculated by weighting and fusing the basic privacy sensitivity weights of multiple behavioral feature parameters and combining them with the actual values ​​of each behavioral feature parameter within the current time window. This can be achieved in the following way: First, read all the behavioral feature parameters extracted within the current processing time window and their corresponding basic privacy sensitivity weights, and simultaneously obtain the actual value of each behavioral feature parameter within the current time window. The actual value includes the frequency of occurrence, intensity level, or specific numerical value of the parameter. Then, using a weighted summation method, multiply the actual value of each behavioral feature parameter by its corresponding basic privacy sensitivity weight to obtain the sensitivity of the parameter to the current scene. Sensitivity contribution value; then, the sensitivity contribution values ​​of all behavioral feature parameters are summed to obtain a comprehensive sum; then, this sum is divided by the total number of behavioral feature parameters in the current time window or normalized, for example, by using the min-max normalization method to map the sum to the range of 0 to 1, to obtain a dimensionless initial sensitivity of the scene, which reflects the baseline of privacy sensitivity calculated based on the actual user behavior patterns within the current time window; as a preferred embodiment, for behavioral feature parameters with special importance or authority, additional adjustment coefficients can be assigned to them during the weighted fusion process to highlight their dominant role in privacy assessment.

[0036] In specific implementation, the initial sensitivity of the scene is dynamically adjusted based on the time period attribute of the current time window and the scene type corresponding to the family scene identifier. The privacy sensitivity coefficient used to describe the degree of data privacy sensitivity can be generated in the following way: First, obtain the start time of the current processing time window and determine the time period attribute based on this time. The time period attribute includes predefined time period types such as late night, weekday daytime, weekend, or holiday periods. Different time period types correspond to different privacy sensitivity adjustment factors. For example, late night usually has higher privacy requirements, so it corresponds to an adjustment factor greater than 1. Simultaneously, based on the family scene identifier associated with the currently processed data, identify the scene type corresponding to the identifier. The scene types include highly private areas such as bedrooms and bathrooms, moderately private areas such as living rooms and dining rooms, and areas with high privacy such as bedrooms and bathrooms. For low-privacy areas such as tables and storage rooms, each scene type corresponds to a preset privacy sensitivity adjustment factor. Then, the time period attribute adjustment factor is combined with the scene type adjustment factor, for example, by multiplication or weighted averaging, to obtain a comprehensive dynamic adjustment coefficient. Next, the calculated initial scene sensitivity is multiplied by the dynamic adjustment coefficient to obtain the final privacy sensitivity coefficient after context adjustment. Finally, the final privacy sensitivity coefficient is limited to ensure that its value falls within a preset effective range, such as discrete integer values ​​between 0 and 1 or between 1 and 10, thereby generating a privacy sensitivity coefficient that reflects both the privacy attributes of the behavior itself and incorporates temporal and spatial context information. This coefficient will serve as the core control parameter for differentiated privacy protection and fusion processing. Other methods can also be used in other embodiments, which are not limited here.

[0037] It should be noted that the basic privacy sensitivity weight in this application refers to a fixed sensitivity benchmark value pre-set for different types of behavioral feature parameters, which is used to quantify the basic privacy importance of each type of behavioral feature parameter due to its inherent attributes (such as whether it involves biometrics or reflects life patterns); the initial sensitivity of the scenario in this application refers to the original privacy risk level formed based on real user behavior activities in the current specified time segment without context adjustment; and the privacy sensitivity coefficient in this application refers to a comprehensive privacy measurement index that considers both the inherent privacy attributes of the behavior itself and incorporates temporal and spatial context information.

[0038] In step 103, based on the privacy sensitivity coefficient, differentiated feature perturbation and structured desensitization processing are applied to data of different privacy levels in the family scene data sequence, thereby generating privacy-constrained feature representations in the feature representation layer.

[0039] In some embodiments, reference Figure 3As shown, this figure is an exemplary flowchart of generating privacy-constrained feature representations in some embodiments of this application. In this embodiment, based on the privacy sensitivity coefficient, differentiated feature perturbation and structured desensitization processing are applied to data of different privacy levels in the family scene data sequence, thereby generating privacy-constrained feature representations in the feature representation layer. This can be achieved through the following steps: Based on the privacy sensitivity coefficient, each data record in the family scene data sequence is classified into privacy levels, and corresponding differentiated privacy protection strategies are matched for data of different privacy levels. According to the matching differentiated privacy protection strategy, feature perturbation or structured desensitization processing is applied to the data records of the corresponding privacy level to generate processed desensitized data units; The desensitized data units are reorganized according to their original temporal order, and then dimensional reduction and feature compression are performed to generate privacy-constrained feature representations in the feature representation layer.

[0040] In specific implementation, the privacy level classification of each data record in the family scene data sequence based on the privacy sensitivity coefficient, and the matching of corresponding differentiated privacy protection strategies for data of different privacy levels, can be achieved in the following way: First, each data record is read from the family scene data sequence, and the generated privacy sensitivity coefficient corresponding to the time window to which the data record belongs is obtained; then, the privacy sensitivity coefficient is compared with multiple preset threshold intervals, including low sensitivity interval, medium sensitivity interval, and high sensitivity interval. For example, the first threshold is set to 0.3 and the second threshold is set to 0.7. When the privacy sensitivity coefficient is less than or equal to 0.3, it is determined to be of low sensitivity level; when the privacy sensitivity coefficient is greater than 0.3, it is determined to be of high sensitivity level. Furthermore, a privacy sensitivity coefficient less than 0.7 is classified as medium sensitivity, while a privacy sensitivity coefficient greater than or equal to 0.7 is classified as high sensitivity. Next, based on the determined privacy level, a corresponding differentiated privacy protection strategy is matched for each data record from a pre-configured privacy protection strategy library. This library contains multiple processing methods for different privacy levels; for example, a lightweight noise addition strategy is matched for low-sensitivity data, an identifier generalization and time precision reduction strategy is matched for medium-sensitivity data, and a strong perturbation or homomorphic encryption strategy is matched for high-sensitivity data. Finally, each data record and its matched privacy protection strategy are associated and bound to form a list of pending tasks containing the original data, privacy level, and processing strategy.

[0041] In specific implementation, according to the matched differentiated privacy protection strategy, feature perturbation or structured desensitization processing is applied to data records of corresponding privacy levels. The generated desensitized data units can be implemented in the following way: First, traverse each data record in the task list to be processed and perform the corresponding processing operation according to its matched privacy protection strategy type; for low-sensitivity data matched with a lightweight noise addition strategy, a Laplace noise addition mechanism is used to generate random noise values ​​that conform to the Laplace distribution and superimpose them on the original numerical data. For example, noise is added to environmental sensing data such as temperature and humidity, while keeping the device identifier and timestamp unchanged; for medium-sensitivity data matched with an identifier generalization and time precision reduction strategy, the original anonymized device identifier is further replaced with a coarser-grained device type identifier. For example, the specific identifier of "bedroom temperature sensor" is replaced with "temperature sensor", and the precision of the calibrated timestamp is reduced from the second level to the minute level or half-hour level. For example, "2024-03-15 22:35:12" is generalized to "2024-03-15". 22:30-23:00”; For highly sensitive data matching strong perturbation or homomorphic encryption strategies, a text perturbation method based on locality-sensitive hashing is used to semantically obfuscate the user's voice command text, or a homomorphic encryption algorithm is used to encrypt the user's behavioral feature parameters to generate ciphertext data, ensuring that the original sensitive content is not directly exposed; After processing, the processing results of each data record are encapsulated in a unified data format to form a desensitized data unit containing the processed data value, a rough version of the original timestamp, and a device type identifier.

[0042] In specific implementation, the desensitized data units are reorganized according to their original temporal order, and dimensionality reduction and feature compression are performed to generate privacy-constrained feature representations in the feature representation layer. This can be achieved in the following way: First, all desensitized data units generated in the previous step are reordered according to the timestamp order of their original data records in the family scene data sequence to restore their temporal structure and ensure that the processed data still maintains the original temporal order. Then, for the sorted desensitized data sequence, dimensionality reduction methods such as principal component analysis or autoencoder networks are used to map the high-dimensional desensitized data to a low-dimensional feature space. Specifically, multiple consecutive desensitized data units are combined into a fixed-length input. The input vector is used to calculate a compact low-dimensional feature vector through a trained dimensionality reduction model. Then, the dimensionality-reduced feature vector is bound to the corresponding privacy sensitivity coefficient and the home scene identifier to form a complete data package containing the feature vector, privacy metadata, and scene information. The privacy metadata includes the privacy level of the feature vector and the type of desensitization strategy used. Finally, the generated data package is output as a privacy-constrained feature representation. This representation retains the key statistical characteristics and correlation patterns in the original data that can be used for subsequent scene analysis at the feature level, while effectively blocking the possibility of directly deducing the original sensitive information at the numerical or structural level. Other methods can also be used in other embodiments, and are not limited here.

[0043] It should be noted that the differentiated privacy protection strategy in this application refers to a set of pre-configured processing rules for data of different privacy levels to guide specific de-identification operations. It can match the corresponding protection strength and processing method for data with different levels of sensitivity, ensuring that high privacy data receives stronger protection while low privacy data retains more usability. The de-identified data unit in this application refers to an intermediate data carrier containing processed data values ​​and corresponding coarse contextual information generated after performing feature perturbation or structured de-identification processing on the original data record according to the matched differentiated privacy protection strategy. The privacy-constrained feature representation in this application refers to a compact data expression form that is subject to privacy constraints at the feature level. It is used to retain the key statistical characteristics and correlation patterns in the original data that can be used for subsequent scenario analysis, while effectively blocking the possibility of directly deducing the original sensitive information at the numerical or structural level.

[0044] In step 104, a cross-device scene semantic association model is established based on the privacy constraint feature representation, and then the collaborative relationship between changes in home device status and user behavior patterns is semantically encoded to obtain scene association features.

[0045] In some embodiments, establishing a cross-device scene semantic association model based on the privacy constraint feature representation can be achieved through the following steps: Using the privacy constraint features as input, a network structure is constructed to capture a scene semantic association model for capturing long-distance dependencies between different devices and between devices and user behavior; The scene semantic association model is trained using a self-supervised learning method, enabling it to learn the inherent synergistic relationship between changes in the state of home devices and user behavior patterns, thus obtaining the trained scene semantic association model.

[0046] In specific implementation, the network structure for constructing a scene semantic association model that captures long-distance dependencies between different devices and between devices and user behavior, using the privacy constraint feature representation as input, can be implemented in the following way: First, a deep neural network based on the Transformer architecture is selected as the basic framework of the scene semantic association model. This architecture can effectively capture the dependencies between any two positions in the input sequence through a self-attention mechanism. Then, the input layer of the model is constructed, the dimension of which matches the dimension of the privacy constraint feature representation, to receive a temporal input sequence composed of privacy constraint feature vectors from multiple consecutive time windows. Next, a position encoding layer is embedded after the input layer. Since the self-attention mechanism itself does not have temporal awareness, a learnable position encoding vector needs to be added to each input position to preserve the temporal order information of the privacy constraint feature representation. Subsequently, a core processing module consisting of multiple stacked Transformer encoder layers is constructed, each encoder layer... Internally, the model comprises a multi-head self-attention sublayer and a feedforward neural network sublayer, with residual connections and layer normalization operations added after each sublayer. The multi-head self-attention sublayer uses multiple parallel attention heads to calculate attention weights between different device features and between device features and user behavior features, thereby capturing cross-device collaborative relationships and long-distance dependencies. As a preferred embodiment, a graph attention mechanism can be introduced into the Transformer encoder layer, treating each device type in the home as a node in a graph, and modeling the physical proximity or logical association between devices as edges. The graph attention layer aggregates the feature information of neighboring nodes, thereby more accurately characterizing the collaborative working patterns between devices. Finally, the output layer of the model is designed as a feature representation layer with the same dimension as the input, used to output the feature vector after context enhancement, while retaining the output interface of the intermediate hidden layers for subsequent extraction of semantic features at different levels of abstraction. After completing the above network structure design, the initial network structure of the scene semantic association model to be trained is obtained.

[0047] In specific implementation, a self-supervised learning approach is used to train the scene semantic association model, enabling it to learn the inherent synergistic relationship between changes in home device status and user behavior patterns. The trained scene semantic association model can be implemented as follows: First, a large amount of continuous time window data is extracted from historically stored privacy-constrained feature representations as a training sample set. These training samples only contain the desensitized output feature data and do not involve any manually labeled information. Then, a mask modeling pre-training task is designed as the goal of self-supervised learning. Specifically, this involves randomly masking feature values ​​at certain time steps or device dimensions in the input sequence, replacing the masked feature values ​​with special mask labels. Next, the masked input sequence is fed into the constructed scene semantic association model for forward computation. The model predicts the original feature values ​​at the masked locations based on the unmasked context features. The reconstruction error between the predicted value and the actual masked feature value is used as the loss function; for example, for continuous features, a loss function is used. Mean squared error loss is used, and cross-entropy loss is used for discrete features. The Adam optimizer is used to iteratively update the model parameters. The gradient of the loss function with respect to the parameters of each layer is calculated through the backpropagation algorithm, and the weights are adjusted. After each training epoch, the reconstruction accuracy of the model is evaluated using a validation set. An early stopping mechanism is set up to stop training when the validation set loss no longer decreases for several consecutive epochs to prevent overfitting. After training, the encoder part of the model has learned how to infer missing information based on context. This means that the model has formed a deep understanding of the changing patterns of device states and user behavior patterns. It can capture which device states usually change simultaneously, which user behaviors trigger which device responses, and what kind of collaborative relationships exist between different devices. Finally, the trained model parameters are solidified to obtain the trained scene semantic association model. This model has the ability to map the input privacy-constrained feature representation to a feature space rich in scene semantic information. Other methods can also be used in other embodiments, which are not limited here.

[0048] It should be noted that the network structure in this application refers to a deep neural network framework used to capture long-distance dependencies between different devices and between devices and user behavior; the scene semantic association model in this application refers to a neural network model that is trained through self-supervised learning and can map the privacy-constrained feature representation of the input to a feature space rich in scene semantic information. It is used to learn and solidify the inherent synergistic relationship between changes in the state of home devices and user behavior patterns.

[0049] In some embodiments, semantically encoding the collaborative relationship between changes in home device status and user behavior patterns to obtain scene-related features can be achieved through the following steps: Obtain the trained scene semantic association model and input the privacy constraint feature representation generated in real time into the scene semantic association model; Through forward inference computation of the scene semantic association model, the input privacy-constrained feature representation is mapped to a latent space rich in scene semantic information to obtain the initial encoded features; The initial encoded features are post-processed and dimensionally standardized to generate scene association features that characterize the collaborative relationship between changes in the state of home devices and user behavior patterns in the current scenario.

[0050] In specific implementation, acquiring the trained scene semantic association model and inputting the real-time generated privacy constraint feature representation into the scene semantic association model can be achieved in the following way: First, load the parameter file of the trained scene semantic association model from the model repository, restore the model to memory and set it to inference mode to ensure that the model does not update the weight parameters during operation; simultaneously, read the privacy constraint feature representation of the current time window output from the real-time data stream. This privacy constraint feature representation has been desensitized and dimensionally reduced, and contains feature vectors from multiple consecutive time steps; then, perform format validation on the input privacy constraint feature representation to ensure that its dimension matches the expected dimension of the scene semantic association model's input layer. If there is a dimension inconsistency, perform necessary padding or truncation operations; next, organize the validated privacy constraint feature representation into the batch format required by the model, for example, stacking the feature vectors of a single time window or multiple consecutive time windows into a three-dimensional tensor, with the dimensions being the batch size, time step length, and feature dimension, respectively; finally, send the organized input data into the input layer of the scene semantic association model to trigger the model's forward computation process.

[0051] In specific implementation, the privacy-constrained feature representations of the input are mapped to a latent space rich in scene semantic information through forward inference computation of the scene semantic association model. The initial encoded features can be obtained in the following way: First, the input data flows through the embedding layer and position encoding layer of the scene semantic association model, adding position information to the feature vector at each time step; then, the data passes through multiple stacked Transformer encoder layers in sequence. Within each encoder layer, a multi-head self-attention mechanism calculates the attention weight between any two time steps in the input sequence, enabling the model to capture the dependencies between changes in the state of distant devices, and also calculate the correlation strength between device features and user behavior features; after self-attention processing... The feature vectors are then subjected to nonlinear transformation through a feedforward neural network, and the stability of the training is maintained through residual connections and layer normalization. As the data propagates forward layer by layer, the original input features are gradually abstracted and fused, and the collaborative relationship between devices and behaviors is encoded into higher-level representations. Finally, the output is extracted from the last encoder layer of the model or from a specified intermediate hidden layer. This output is a vector sequence with the same length as the input sequence but whose feature dimensions may change, or a fixed-length vector obtained by pooling the entire input sequence. This extracted vector is used as the initial encoded feature, which already contains deep semantic knowledge about device state change patterns and user behavior patterns learned by the model from a large amount of unlabeled data.

[0052] In specific implementation, the initial encoded features are post-processed and their dimensions standardized to generate scene-related features that characterize the collaborative relationship between changes in the state of home devices and user behavior patterns in the current scenario. This can be achieved in the following way: First, the extracted initial encoded features are normalized using the L2 norm, dividing each feature vector by its L2 norm to ensure that all feature vectors have a uniform scale, eliminating the influence of differences in modulus between different samples, and facilitating subsequent comparison and fusion operations. Then, if the initial encoded features are in the form of a variable-length sequence, they are further compressed according to the needs of the upper-layer application, for example, by using average pooling or max pooling to aggregate the feature sequence of the entire time step into a fixed-length global feature vector. Next, the normalized and pooled feature vectors are checked for dimensionality to ensure that they conform to the pre-defined dimensions. The scene association feature dimension is set. If the dimension is too high, an additional linear projection layer is used for dimensionality reduction; if the dimension is too low, zero-padding or repeated expansion is performed. Subsequently, the processed feature vector is associated and encapsulated with the corresponding home scene identifier, time window information, and generated privacy sensitivity coefficient to form a complete data package containing the feature vector itself and its context metadata. Finally, this data package is output as a scene association feature to shared memory or a message queue. This feature, in a compact numerical form, fully describes the collaborative pattern between the state changes of various devices in the home and user behavior within the current time window. For example, it can implicitly express high-level scene semantics such as "the user is watching a movie in the living room" or "the user is preparing dinner in the kitchen." Other methods can also be used in other embodiments, which are not limited here.

[0053] It should be noted that the latent space rich in scene semantic information in this application refers to a high-dimensional abstract representation space used to characterize the complex collaborative relationship between device status and user behavior; the initial encoded features in this application refer to the intermediate representation vector extracted from the model's encoding layer, which contains deep semantic knowledge about the patterns of device status changes and user behavior learned by the model from a large amount of unlabeled data; the scene-related features in this application refer to the final feature vector generated after post-processing and dimensional standardization of the initial encoded features, which is associated with and encapsulated with home scene identifiers and time window information, and is used to completely characterize the collaborative pattern between changes in home device status and user behavior within the current time window in a compact numerical form.

[0054] In step 105, privacy-driven multi-source data fusion is performed based on the privacy sensitivity coefficient and the scene association features to generate a fused data representation result that balances privacy protection and data availability.

[0055] In some embodiments, the following steps can be used to generate a fused data representation that balances privacy protection and data availability by performing privacy-driven multi-source data fusion based on the privacy sensitivity coefficient and the scene association features: The contribution weight of the scene association features in the fusion process is determined based on the privacy sensitivity coefficient, and then the scene association features from multiple sources are weighted and fused. The fused feature vectors are post-processed and used to enhance their usability, resulting in a fused data representation that balances privacy protection and data usability.

[0056] In specific implementation, the contribution weight of the scene association features in the fusion process is determined based on the privacy sensitivity coefficient, and then the weighted fusion of multi-source scene association features can be achieved in the following way: First, all scene association features generated in the current processing cycle and their corresponding privacy sensitivity coefficients are read from shared memory or data bus. The scene association features may come from multiple different time windows or multiple different family sub-regions, and each scene association feature is associated with a corresponding calculated privacy sensitivity coefficient. Then, for each scene association feature, its contribution weight in the fusion process is determined according to the magnitude of its privacy sensitivity coefficient. Specifically, a mapping rule that is inversely proportional to the privacy sensitivity coefficient and the fusion weight is adopted, that is, the higher the privacy sensitivity coefficient, the more sensitive the privacy information contained in the feature, and the lower the weight should be assigned during fusion to reduce sensitivity. To mitigate the risk of information leakage, a lower privacy sensitivity coefficient indicates a lower privacy risk for the feature, allowing it to be assigned a higher weight to retain more data availability. Next, the determined weights are normalized to ensure the sum of the weights of all features participating in the fusion is 1. Then, a weighted average fusion method is used, multiplying each scenario-related feature vector by its corresponding normalized weight and summing the results to obtain a weighted aggregated feature vector. As a preferred embodiment, for features with extremely high privacy sensitivity coefficients requiring strict protection, their weights can be set to zero during the weighted fusion process, effectively excluding them from the fusion process. Alternatively, a dynamic weighting method based on an attention mechanism can be used, with the privacy sensitivity coefficient as one of the inputs to the attention scoring function, calculating the final fusion weight together with the feature content. After weighted fusion, a preliminary fused feature vector is obtained.

[0057] In practice, post-processing and usability enhancement operations are performed on the fused feature vector to generate a fused data representation that balances privacy protection and data usability. This can be achieved as follows: First, the initial fused feature vector is normalized using the L2 norm, dividing the vector by its magnitude to ensure the final output feature vector has a unit length, eliminating inconsistencies caused by differences in feature scale between different fusion batches. Then, based on the specific needs of the upper-layer application, the normalized feature vector is dimension-adapted. For example, if it needs to be input into a specific machine learning model, a linear transformation layer maps the feature vector to the target dimension space. Next, the feature vector is clipped to ensure all feature values ​​fall within a reasonable range, such as between -1 and 1 or between 0 and 1, to avoid outliers interfering with subsequent applications. Finally, the processed feature vector is compared with the data used in the fusion process. Metadata is associated and encapsulated, including the number of scene-related features involved in the fusion, the average privacy sensitivity coefficient used during fusion, the fusion timestamp, and the corresponding home scene identifier. In a preferred embodiment, an optional privacy budget consumption indicator field can be added to the feature vector to indicate the degree of privacy loss in the differential privacy sense of the fusion result. Finally, the encapsulated complete data packet is output as the final fusion data representation result to the upper-layer application interface or storage system. Mathematically, this result guarantees that the difficulty of inferring any single highly sensitive feature from the result is proportional to the privacy sensitivity coefficient. Simultaneously, weighted fusion preserves common information from multiple scenes, providing high-quality input for upper-layer applications in smart homes, such as scene recognition, anomaly detection, and personalized services, while protecting user privacy and maintaining sufficient data availability. Other methods can also be used in other embodiments, which are not limited here.

[0058] Furthermore, in another aspect of this application, in some embodiments, this application provides a privacy-preserving multi-source data fusion processing device for smart homes, with reference to... Figure 4 The figure is a schematic diagram of the structure of a privacy-preserving multi-source data fusion processing device for smart homes, according to some embodiments of this application. The privacy-preserving multi-source data fusion processing device 400 for smart homes includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below: The acquisition module 401 in this application is mainly used to acquire multi-source raw data generated by various terminal devices in the smart home environment, and to construct a unified home scene data sequence based on device identifier, timestamp and home scene identifier; Processing module 402, in this application, is mainly used to perform behavioral semantic analysis on the family scene data sequence, extract behavioral feature parameters that represent the relationship between user behavior patterns and device operating status, and generate a privacy sensitivity coefficient to describe the degree of data privacy sensitivity based on the extracted behavioral feature parameters. The processing module 402 described in this application is further configured to perform differentiated feature perturbation and structured desensitization processing on data of different privacy levels in the family scene data sequence according to the privacy sensitivity coefficient, thereby generating privacy-constrained feature representations in the feature representation layer; The processing module 402 described in this application is also used to establish a cross-device scene semantic association model based on the privacy constraint feature representation, and then semantically encode the collaborative relationship between changes in home device status and user behavior patterns to obtain scene association features; The execution module 403 in this application is mainly used to perform privacy-driven multi-source data fusion based on the privacy sensitivity coefficient and the scene association features, thereby generating a fused data representation result that takes into account both privacy protection and data availability.

[0059] The foregoing detailed examples of a privacy-preserving multi-source data fusion processing method and apparatus for smart homes provided in this application. It is understood that, to achieve the aforementioned functions, the apparatus includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed through hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0060] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described privacy-preserving multi-source data fusion processing method for smart homes.

[0061] In some embodiments, reference Figure 5 The dashed lines in the figure indicate that the unit or module is optional. This figure is a schematic diagram of the structure of a computer device implementing the privacy-preserving multi-source data fusion processing method for smart homes according to this application. The privacy-preserving multi-source data fusion processing method for smart homes in the above embodiments can be achieved through… Figure 5The computer device 500 shown is used to implement this, and the computer device 500 includes at least one processor 501, a memory 502 and at least one communication unit 505. The computer device 500 may be a terminal device, a server or a chip.

[0062] The processor 501 can be a general-purpose processor or a special-purpose processor. For example, the processor 501 can be a central processing unit (CPU). The CPU can be used to control the computer device 500, execute software programs, and process data from the software programs. The computer device 500 may also include a communication unit 505 for inputting (receiving) and outputting (transmitting) signals.

[0063] For example, computer device 500 may be a chip, communication unit 505 may be the input and / or output circuit of the chip, or communication unit 505 may be the communication interface of the chip, and the chip may be a component of terminal device, network device or other device.

[0064] For example, computer device 500 may be a terminal device or a server, and communication unit 505 may be a transceiver of the terminal device or the server, or communication unit 505 may be a transceiver circuit of the terminal device or the server.

[0065] The computer device 500 may include one or more memories 502 storing a program 504. The program 504 can be executed by a processor 501 to generate instructions 503, causing the processor 501 to perform the methods described in the above method embodiments according to the instructions 503. Optionally, the memory 502 may also store data (such as a target audit model). Optionally, the processor 501 may also read data stored in the memory 502, which may be stored at the same storage address as the program 504, or the data may be stored at a different storage address than the program 504.

[0066] The processor 501 and memory 502 can be configured separately or integrated together, for example, integrated on the system-on-chip (SOC) of the terminal device.

[0067] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 501. The processor 501 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gate, transistor logic devices, or discrete hardware components.

[0068] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the above-described privacy-preserving multi-source data fusion processing method for smart homes.

[0070] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0071] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A privacy-preserving multi-source data fusion processing method for smart homes, characterized in that, Includes the following steps: Acquire multi-source raw data generated by various terminal devices in the smart home environment, and construct a unified home scene data sequence based on device identifiers, timestamps, and home scene identifiers; Behavioral semantic parsing is performed on the home scene data sequence to extract behavioral feature parameters that represent the correlation between user behavior patterns and device operating status, and a privacy sensitivity coefficient is generated based on the extracted behavioral feature parameters to describe the degree of data privacy sensitivity. Based on the privacy sensitivity coefficient, differentiated feature perturbation and structured desensitization processing are applied to data of different privacy levels in the family scene data sequence, thereby generating privacy-constrained feature representations in the feature representation layer; Based on the privacy constraint feature representation, a cross-device scene semantic association model is established, and then the collaborative relationship between changes in home device status and user behavior patterns is semantically encoded to obtain scene association features. Privacy-driven multi-source data fusion is performed based on the privacy sensitivity coefficient and the scene association features to generate a fused data representation that balances privacy protection and data availability.

2. The method as described in claim 1, characterized in that, Constructing a unified family scene data sequence based on device identifiers, timestamps, and family scene identifiers specifically includes: Extract the device identifier, original timestamp, and home scene identifier corresponding to each data point from the acquired multi-source raw data; The extracted device identifier is anonymized and mapped, and the original timestamp is time-synchronized and calibrated. Based on the anonymized device identifier, the calibrated timestamp, and the home scene identifier, the multi-source raw data is reorganized into a home scene data sequence with a unified format and time order.

3. The method as described in claim 1, characterized in that, The behavioral semantic analysis of the home scene data sequence, and the extraction of behavioral feature parameters that represent the correlation between user behavior patterns and device operating status, specifically include: The home scene data sequence is divided into time windows, and the temporal triggering relationship between user behavior events and device status change events is identified within each time window; The frequency and confidence of the aforementioned time-series triggering relationships within multiple time windows are statistically analyzed. Based on the obtained statistical results, behavioral feature parameters are extracted to characterize the relationship between user behavior patterns and device operating status.

4. The method as described in claim 1, characterized in that, The privacy sensitivity coefficient, generated based on the extracted behavioral feature parameters to describe the degree of data privacy sensitivity, specifically includes: Determine the basic privacy sensitivity weight corresponding to each extracted behavioral feature parameter; The initial sensitivity of the scene is calculated by weighting and fusing the basic privacy sensitivity weights of multiple behavioral feature parameters and combining them with the actual values ​​of each behavioral feature parameter within the current time window. The initial sensitivity of the scenario is dynamically adjusted based on the time period attribute of the current time window and the scenario type corresponding to the family scenario identifier, generating a privacy sensitivity coefficient to describe the degree of data privacy sensitivity.

5. The method as described in claim 1, characterized in that, Based on the privacy sensitivity coefficient, differentiated feature perturbation and structured desensitization processing are applied to data of different privacy levels in the family scene data sequence, thereby generating privacy-constrained feature representations in the feature representation layer. Specifically, this includes: Based on the privacy sensitivity coefficient, each data record in the family scene data sequence is classified into privacy levels, and corresponding differentiated privacy protection strategies are matched for data of different privacy levels. According to the matching differentiated privacy protection strategy, feature perturbation or structured desensitization processing is applied to the data records of the corresponding privacy level to generate processed desensitized data units; The desensitized data units are reorganized according to their original temporal order, and then dimensional reduction and feature compression are performed to generate privacy-constrained feature representations in the feature representation layer.

6. The method as described in claim 1, characterized in that, Establishing a cross-device scene semantic association model based on the aforementioned privacy constraint feature representation specifically includes: Using the privacy constraint feature representation as input, a network structure is constructed to capture the long-distance dependency relationship between different devices and between devices and user behavior; The scene semantic association model is trained using a self-supervised learning method, enabling it to learn the inherent synergistic relationship between changes in the state of home devices and user behavior patterns, thus obtaining the trained scene semantic association model.

7. The method as described in claim 1, characterized in that, Semantic encoding of the collaborative relationship between changes in home device status and user behavior patterns yields scene-related features, specifically including: Obtain the trained scene semantic association model and input the privacy constraint feature representation generated in real time into the scene semantic association model; Through forward inference computation of the scene semantic association model, the input privacy-constrained feature representation is mapped to a latent space rich in scene semantic information to obtain the initial encoded features; The initial encoded features are post-processed and dimensionally standardized to generate scene association features that characterize the collaborative relationship between changes in the state of home devices and user behavior patterns in the current scenario.

8. The method as described in claim 1, characterized in that, Based on the privacy sensitivity coefficient and the scene association features, privacy-driven multi-source data fusion is performed to generate a fused data representation that balances privacy protection and data usability. Specifically, this includes: The contribution weight of the scene association features in the fusion process is determined based on the privacy sensitivity coefficient, and then the scene association features from multiple sources are weighted and fused. The fused feature vectors are post-processed and used to enhance their usability, resulting in a fused data representation that balances privacy protection and data usability.

9. The method as described in claim 1, characterized in that, The multi-source raw data includes environmental sensor data, equipment operating status data, and user interaction behavior data.

10. A privacy-preserving multi-source data fusion processing device for smart homes, characterized in that, include: The acquisition module is used to acquire multi-source raw data generated by various terminal devices in the smart home environment, and construct a unified home scene data sequence based on device identifiers, timestamps, and home scene identifiers; The processing module is used to perform behavioral semantic parsing on the home scene data sequence, extract behavioral feature parameters that represent the relationship between user behavior patterns and device operating status, and generate a privacy sensitivity coefficient to describe the degree of data privacy sensitivity based on the extracted behavioral feature parameters. The processing module is also used to perform differentiated feature perturbation and structured desensitization processing on data of different privacy levels in the family scene data sequence according to the privacy sensitivity coefficient, thereby generating privacy-constrained feature representations in the feature representation layer; The processing module is also used to establish a cross-device scene semantic association model based on the privacy constraint feature representation, and then semantically encode the collaborative relationship between changes in the state of home devices and user behavior patterns to obtain scene association features; The execution module is used to perform privacy-driven multi-source data fusion based on the privacy sensitivity coefficient and the scene association features, thereby generating a fused data representation result that balances privacy protection and data availability.

Images