Method and apparatus for identifying user behavior based on smart home device

Abstract

The application relates to a method and device for identifying user behavior based on smart home equipment, wherein the method comprises the following steps: acquiring original event data generated by the smart home equipment within a preset time window; performing semantic analysis on the original event data to generate a fact-level semantic representation corresponding to the preset time window; constructing a behavior semantic prototype library; performing semantic matching on the fact-level semantic representation and semantic prototypes of each behavior category in the behavior semantic prototype library, and calculating the comprehensive semantic similarity between the fact-level semantic representation and each behavior category; and determining a target behavior category corresponding to the preset time window and outputting the target behavior category according to the comprehensive semantic similarity. Through the application, the problems that in the prior art, based on smart home equipment, user behavior identification is difficult to form a unified semantic representation that can be directly used for behavior identification, and the accuracy of the identification result is low, are solved.

Description

Technical Field

[0001] This application relates to the field of smart home devices, and in particular to a method and apparatus for identifying user behavior based on smart home devices. Background Technology

[0002] Currently, existing technologies for user behavior recognition based on smart home devices rely on data from door sensors, infrared sensors, temperature and humidity sensors, switch control records, and appliance operation logs to determine daily user behavior. A common approach is to first organize this data in a fixed format, then extract features such as time, location, and device status, and finally feed it into a classification model to identify behaviors such as cooking, sleeping, going out, and washing. However, existing smart home systems do not handle sensor events and device logs uniformly, making it difficult to form a unified semantic representation of multi-source heterogeneous data that can be directly used for behavior recognition. Furthermore, most existing behavior recognition solutions treat behavior as simple labels, resulting in inaccurate recognition results and hindering reuse across devices and home scenarios. Summary of the Invention

[0003] This application provides a method and apparatus for identifying user behavior based on smart home devices, in order to solve the problems in the prior art where it is difficult to form a unified semantic representation that can be directly used for behavior identification and the accuracy of the identification results is low.

[0004] Firstly, this application provides a method for identifying user behavior based on smart home devices, comprising: acquiring raw event data generated by smart home devices within a preset time window; performing semantic parsing on the raw event data to generate a fact-level semantic representation corresponding to the preset time window; wherein, the fact-level semantic representation refers to mapping the multi-source heterogeneous raw event data generated by the smart home devices within the preset time window to a unified semantic expression framework; constructing a behavior semantic prototype library, wherein the behavior semantic prototype library includes multiple preset behavior categories and their corresponding semantic prototypes; the semantic prototype is a set of feature vectors constructed in a preset semantic space; performing semantic matching between the fact-level semantic representation and the semantic prototypes of each behavior category in the behavior semantic prototype library, and calculating the comprehensive semantic similarity between the fact-level semantic representation and each behavior category; determining and outputting the target behavior category corresponding to the preset time window based on the comprehensive semantic similarity.

[0005] Secondly, this application provides an apparatus for identifying user behavior based on smart home devices, comprising: an acquisition module for acquiring raw event data generated by smart home devices within a preset time window; a first processing module for performing semantic parsing on the raw event data to generate a fact-level semantic representation corresponding to the preset time window; wherein the fact-level semantic representation refers to mapping the multi-source heterogeneous raw event data generated by the smart home devices within the preset time window to a unified semantic expression framework; a construction module for constructing a behavior semantic prototype library, wherein the behavior semantic prototype library includes multiple preset behavior categories and their corresponding semantic prototypes; the semantic prototype is a set of feature vectors constructed in a preset semantic space; a second processing module for semantically matching the fact-level semantic representation with the semantic prototypes of each behavior category in the behavior semantic prototype library, and calculating the comprehensive semantic similarity between the fact-level semantic representation and each behavior category; and a third processing module for determining and outputting the target behavior category corresponding to the preset time window based on the comprehensive semantic similarity.

[0006] Thirdly, this application provides an electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the processor is configured to execute the method for identifying user behavior based on a smart home device as described in the first aspect of this application.

[0007] Fourthly, this application also provides a computer storage medium storing computer-executable instructions, which are used to execute the method for identifying user behavior based on smart home devices described in the first aspect of this application.

[0008] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application first acquires the raw event data generated by smart home devices within a preset time window, then constructs a behavioral semantic prototype library and performs semantic parsing on the raw event data to generate a fact-level semantic representation corresponding to the preset time window. Then, the fact-level semantic representation is semantically matched with the semantic prototypes of each behavior category in the behavioral semantic prototype library, and the comprehensive semantic similarity between the fact-level semantic representation and each behavior category is calculated. Finally, based on the comprehensive semantic similarity, the target behavior category corresponding to the preset time window is determined and output. It can be seen that in this application, semantic parsing can uniformly convert raw data from both sensor event streams and device operation logs into fact-level semantic representations, and all of them can be mapped to the same expression framework. This reduces the impact of differences in different device protocols and data formats on behavior recognition, while enhancing the interpretability of the behavior recognition process and providing clear semantic basis for subsequent behavior judgment. Furthermore, by semantic matching and calculating comprehensive semantic similarity, the system's understanding of the meaning of behavior can be improved, making the recognition results more accurate and stable. At the same time, the model's dependence on specific device configurations can be reduced, the system's generalization ability in different home and device environments can be improved, and the application value in zero-shot or weakly supervised scenarios can be enhanced. Attached Figure Description The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0010] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0011] Figure 1 A flowchart illustrating a method for recognizing user behavior based on smart home devices, provided in this application embodiment; Figure 2 This application provides one of the optional flowcharts for a method of recognizing user behavior based on smart home devices. Figure 3 This application provides a second optional flowchart of a method for identifying user behavior based on smart home devices. Figure 4 This application provides a third optional flowchart of a method for identifying user behavior based on smart home devices; Figure 5 This application provides a fourth optional flowchart of a method for identifying user behavior based on smart home devices; Figure 6 A flowchart illustrating a semantically enhanced behavior recognition method for multi-source heterogeneous data in smart homes, provided in this application embodiment; Figure 7 The apparatus for recognizing user behavior based on smart home devices provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0014] To address the problems of low accuracy in existing technologies for user behavior recognition based on smart home devices, which struggle to generate a unified semantic representation directly applicable to behavior identification, this application provides a method for user behavior recognition based on smart home devices, such as... Figure 1 As shown, the steps of this method include: Step 101: Obtain raw event data generated by smart home devices within a preset time window; In this embodiment, the preset time window refers to a continuous time period pre-defined by the system to capture all event data generated within that time period. The length of the time window can be flexibly set according to the actual scenario, for example, it can be set to 30 seconds, 1 minute, or 2 minutes. The reason for setting a time window instead of using single-point events is that a single event (such as turning on the light) does not provide enough information to support accurate behavior judgment, while multiple event sequences within a continuous time period (such as bedroom activity → turning on the light → walking to the bathroom) can constitute meaningful behavior segments, providing sufficient contextual information for subsequent behavior recognition.

[0015] Furthermore, raw event data refers to data directly reported or recorded by smart home devices without any semantic processing. Examples include door magnetic sensors reporting statuses such as "door open," infrared human body sensors reporting triggers, smart switches recording button presses, and air conditioners recording a temperature set to 26°C. This data comes from different types of devices, and its format, field meanings, and time precision are inconsistent, exhibiting multi-source heterogeneous characteristics.

[0016] Step 102: Perform semantic parsing on the raw event data to generate a fact-level semantic representation corresponding to a preset time window; wherein, the fact-level semantic representation refers to mapping the multi-source heterogeneous raw event data generated by smart home devices within the preset time window to a unified semantic expression framework; In this embodiment, semantic parsing refers to the process of understanding and translating raw event data. Instead of directly using raw data (such as PIR-03 trigger) as the final input, the meaning of the event is first identified: which device generated the event, what operation or state change was involved, and when and where it occurred. Through semantic parsing, the raw data is endowed with human-readable semantic information, laying the foundation for subsequent semantic matching. Multi-source heterogeneity refers to the differences in structure, format, field meaning, and time precision of data generated by different devices. For example, an infrared sensor reports a trigger / non-trigger binary state, while a smart switch records an on / off action log, and an air conditioner log records the temperature setting value. Their data structures are completely different and cannot be directly processed together. This application eliminates the impact of differences in device protocols and data formats on behavior recognition by uniformly mapping data from different sources to the same semantic expression framework.

[0017] Furthermore, in this application and embodiments, fact-level semantic representation refers to mapping multi-source heterogeneous raw event data generated by smart home devices within a preset time window to a unified semantic expression framework. This unified semantic expression framework includes at least six semantic elements: subject (actor), action (operation performed), object (target of the action), state (state before and after the change), time (time of occurrence), and space (location of occurrence). These six semantic elements fully cover all the key information needed to describe an event. After semantic parsing, any raw event data is filled into this framework, and regardless of whether the data comes from sensors, switches, or appliance logs, it ultimately presents a consistent semantic structure.

[0018] Step 103: Construct a behavioral semantic prototype library, which includes multiple preset behavioral categories and their corresponding semantic prototypes; the semantic prototype is a set of feature vectors constructed in a preset semantic space. In this embodiment, the behavioral semantic prototype library refers to a pre-built database that stores standard semantic profiles of all identifiable behavioral categories. The behavioral semantic prototype library serves as a reference for subsequent matching processes, comparing the currently observed factual semantic representation with the behavioral prototypes in the library to determine the category to which the current behavior belongs. A behavioral category refers to the type of user behavior that the system can identify, such as getting up, cooking, sleeping, going out, washing up, and getting up at night. Each behavioral category has a corresponding semantic prototype in the prototype library.

[0019] A semantic prototype is a set of feature vectors constructed within a pre-defined semantic space. This pre-defined semantic space is a high-dimensional vector space where texts or data with similar semantic meanings are mapped to similar positions. This space is obtained through pre-training, enabling computers to understand and compare the semantic relationships between different expressions. For example, "turn on the lighting equipment" and "turn on the light," although different expressions, are mapped to similar vector positions in the semantic space. A feature vector is the result of mathematically representing semantic information. After encoding, a piece of text becomes a vector composed of several numerical values ​​(e.g., [0.23, -0.45, 0.67, ...]). Computers cannot directly understand the semantics of the sentence "turn on the lighting," but they can determine the semantic similarity between different texts by calculating the mathematical distance between vectors.

[0020] Step 104: Perform semantic matching between the fact-level semantic representation and the semantic prototype of each behavior category in the behavior semantic prototype library, and calculate the comprehensive semantic similarity between the fact-level semantic representation and each behavior category. In this embodiment, semantic matching refers to the process of comparing the currently observed fact-level semantic representation with the semantic prototypes of each behavior category in the prototype library one by one. This matching is not a simple keyword comparison (such as checking whether it contains the word "turn on the light"), but is based on similarity calculation in the semantic vector space, which can understand that turning on the lighting and turning on the light are similar in meaning, thus possessing the ability to recognize synonymous expressions.

[0021] Comprehensive semantic similarity is a numerical matching score used to quantify the degree of matching between the current observation segment and a certain behavior category. The higher the score, the closer the current observation is to that behavior category. Comprehensive semantic similarity is a comprehensive evaluation result that integrates multiple dimensions, considering both whether the factual details match (such as specific event sequences, time, space, and other features) and whether the behavioral intentions are consistent (such as macro-level behavioral purposes, device functions, and roles), thereby achieving a comprehensive understanding of the behavior.

[0022] Step 105: Determine and output the target behavior category corresponding to the preset time window based on the comprehensive semantic similarity.

[0023] The target behavior category refers to the user behavior result ultimately determined by the system, which is the behavior category most likely to correspond to the current observation window after matching and comparing with various behavior prototypes. The output refers to providing the recognition results to the upper-layer application or user. The output content includes at least the behavior category name (such as waking up), and may further include interpretable evidence (such as key matching events: bedside lamp turned on, bathroom door magnetic sensor triggered) to support subsequent automated scene linkage or manual review.

[0024] Through steps 101 to 105 above, raw event data generated by smart home devices within a preset time window is first acquired. Then, a behavioral semantic prototype library is constructed, and semantic parsing is performed on the raw event data to generate a fact-level semantic representation corresponding to the preset time window. Next, the fact-level semantic representation is semantically matched with the semantic prototypes of each behavior category in the behavioral semantic prototype library, and the comprehensive semantic similarity between the fact-level semantic representation and each behavior category is calculated. Finally, based on the comprehensive semantic similarity, the target behavior category corresponding to the preset time window is determined and output. It can be seen that in this application, semantic parsing can uniformly convert raw data from both sensor event streams and device operation logs into fact-level semantic representations, and all can be mapped to the same expression framework. This reduces the impact of differences in device protocols and data formats on behavior recognition, while enhancing the interpretability of the behavior recognition process and providing clear semantic basis for subsequent behavior judgment. Furthermore, by semantic matching and calculating comprehensive semantic similarity, the system's understanding of the meaning of behavior can be improved, making the recognition results more accurate and stable. At the same time, the model's dependence on specific device configurations can be reduced, the system's generalization ability in different home and device environments can be improved, and the application value in zero-shot or weakly supervised scenarios can be enhanced.

[0025] In optional embodiments of this application, such as Figure 2 As shown, the method of performing semantic parsing on the original event data and generating a fact-level semantic representation corresponding to a preset time window, involved in step 101 above, can further include: Step 11: Identify the event type of each raw event data within the preset time window; In this embodiment, the event type refers to the business category corresponding to the original event data, used to identify the nature of the device event. Events generated by different smart home devices have different event types, such as sensor-triggered events, device status change events, device operation log events, and scene linkage events. The original data for different event types have different data structures and field meanings. The system needs to first identify the event type of each original event data so that the corresponding parsing strategy can be used for field extraction. Event type identification can be based on fields such as device identifier, event code, and message type carried in the original event data. For example, data from a human infrared sensor can be identified as a sensor-triggered event, data from a smart switch can be identified as a device status change event, and data from appliance operation logs can be identified as a device operation log event.

[0026] Step 12: Invoke the corresponding parsing strategy according to the event type to extract the subject field, action field, object field, status field, time field, and space field from each raw event data; In this embodiment, the parsing strategy refers to the pre-defined field extraction rules for different event types. Since the original data for different event types differ in data structure, field naming, and numerical encoding, a corresponding parsing strategy needs to be configured for each event type to correctly extract the fields required for subsequent slot filling from the original data.

[0027] The action field identifies the executor of the behavior. In a smart home scenario, the executor is typically the user, but can also be a home control system (such as device operations triggered by automation rules). The action field identifies the type of operation performed. It describes the core operation of the behavior, such as triggering, turning on, turning off, opening, or adjusting. The object field identifies the target of the action. For example, for the event of turning on a bedside lamp, the object is the lighting device; for the event of triggering a human infrared sensor, the object is the human presence sensor. The state field identifies the state change of the object. It describes the state changes before and after the event, such as changing from off to on, from not triggered to triggered, or the temperature changing from 26℃ to 24℃. The time field identifies the moment the event occurred, recording the specific time, such as "06:40:35". The space field identifies the location of the event, recording the spatial location of the event, such as bedroom, living room, kitchen, or bathroom. The process of extracting the above six fields from the raw event data is essentially translating the raw data reported by the device into field information with clear semantic meaning, providing a data foundation for subsequent filling into the unified semantic slot structure.

[0028] Step 13: Fill the extracted fields into the preset unified semantic slot structure to generate a structured semantic representation corresponding to the original event data; wherein, the unified semantic slot structure includes at least subject slot, action slot, object slot, state slot, time slot and space slot; In this embodiment, the unified semantic slot structure refers to a standardized data structure predefined by the system to carry the core semantic information of an event. This structure includes at least six slots: subject slot, action slot, object slot, state slot, time slot, and space slot, corresponding to the six fields mentioned above. Therefore, filling refers to filling the extracted subject, action, object, state, time, and space fields into the corresponding subject, action, object, state, time, and space slots in the unified semantic slot structure. After filling, a structured semantic representation corresponding to the original event data is generated. A structured semantic representation is a standardized data structure organized using slot-value pairs. Each structured semantic representation corresponds to one piece of original event data and consists of a set of slots and corresponding field values. The structured semantic representation has a fixed format and unified field definitions, ensuring that regardless of the device from which the original data originates, it presents the same structural form after filling.

[0029] In a specific example, the generated structured semantic representation is as follows: {Main Slot: User, Action Slot: Activity Occurrence, Object Slot: Human Presence Sensor, State Slot: Trigger, Time Slot: 06:40:35, Space Slot: Bedroom} As can be seen, through the above-described filling process, each piece of raw event data within the preset time window is converted into a structured semantic representation in a unified format, thus achieving uniformity at the data structure level for raw data from different sources and in different formats. Simultaneously, the structured semantic representation is presented in a slot-value format, facilitating subsequent temporal combination and the extraction and verification of semantic elements in specific dimensions.

[0030] Step 14: Combine the structured semantic representations generated within the preset time window in chronological order to generate fact-level semantic representations.

[0031] In this embodiment, combining in chronological order refers to sorting all structured semantic representations generated within the same preset time window according to the occurrence time recorded in their respective time fields, and then combining them into a complete data sequence. The combination order is usually arranged in ascending order from earliest to latest. Fact-level semantic representation refers to a complete sequence composed of all structured semantic representations within the same preset time window arranged in chronological order. This sequence fully records all events that occur within the preset time window and their temporal relationships, providing complete temporal context information for subsequent semantic matching.

[0032] As can be seen, based on steps 11 to 14 above, differentiated processing of raw event data from different sources and in different formats is achieved through event type identification and the invocation of corresponding parsing strategies. Each event type has its own independent data structure and encoding method. By configuring the corresponding parsing strategy, the meaning of each type of raw data can be correctly understood and key fields can be extracted, ensuring the accuracy of subsequent semantic representation. In addition, by filling the extracted fields into a unified semantic slot structure to generate structured semantic representations, the heterogeneous raw data from multiple sources is converted into a unified slot-value format. Regardless of whether the raw data comes from a door magnetic sensor, a human infrared sensor, or an appliance log, after filling, it presents a unified structure containing six slots: subject, action, object, state, time, and space, eliminating the impact of differences in different device protocols and data formats on subsequent processing. By combining multiple structured semantic representations within the same time window in chronological order, the temporal relationship between events is preserved. The factual semantic representation is a time-ordered sequence of events, fully recording the temporal context of user activities within a preset time window. This provides sufficient temporal contextual information for subsequent semantic matching, enabling behavior recognition to be based on event sequences rather than isolated events, thereby improving recognition accuracy. The structured semantic representation is stored in a slot-value format, facilitating the extraction and verification of semantic elements in specific dimensions, while also enhancing the interpretability of the behavior recognition process.

[0033] In an optional implementation of this application, the method of combining the structured semantic representations generated within a preset time window in chronological order to generate a fact-level semantic representation, as involved in step 14 above, may further include: Step 21: Convert each structured semantic representation generated within the preset time window into a fact-level text description in natural language form; In this embodiment, natural language form refers to declarative sentences that conform to human everyday language expression habits, rather than structured field-value formats. Natural language form of fact-level text description involves organizing the values ​​of each slot in a structured semantic representation into a complete declarative sentence or declarative fragment according to the grammatical rules of natural language. Fact-level text description refers to a text description expressed in natural language form generated after semantic parsing of a single raw event data point. Each fact-level text description corresponds to one raw event data point, fully describing the subject, action, object, state, time, and spatial information of the event.

[0034] Through step 21 above, the structured slot data, which is easily processed by machines, is transformed into natural language text that is easily understood by humans and rich in semantic relationships. Natural language text retains all the information from the structured data and is presented in the form of continuous text, making it easier to encode using a pre-trained semantic encoder. Furthermore, the factual text description in natural language form can be directly output to the user as interpretable evidence, enhancing the comprehensibility of the system's recognition results. For example, in a specific example, a structured semantic representation ({Subject slot: User, Action slot: Activity Occurred, Object slot: Human Presence Sensor, State slot: Trigger, Time slot: 06:40:35, Space slot: Bedroom}) can be converted into a factual text description in natural language form: "The user was active in the bedroom at 06:40:35."

[0035] Step 22: Combine the fact-level text descriptions generated within the same preset time window in chronological order to generate a fact-level semantic representation.

[0036] In this embodiment, "combination" refers to sorting multiple fact-level text descriptions generated within the same preset time window according to the occurrence time indicated by their respective time information, and then merging them into a complete data sequence. The chronological order refers to the order in which the events corresponding to each fact-level text description occur, typically arranged in ascending order from earliest to latest.

[0037] The purpose of combining factual text descriptions in chronological order is to generate a complete event sequence as input for subsequent semantic matching. This sequence, presented as natural language text, can be directly input into a pre-trained semantic encoder (such as the subsequent first and second semantic encoders) for encoding, generating embedding vectors. Simultaneously, the text sequence preserves the temporal relationships between events, enabling the encoder to capture the contextual associations between events during the encoding process.

[0038] In this specific example, the three generated factual text descriptions are as follows: Text description 1: "The user was active in the bedroom at 06:40:35".

[0039] Text description 2: "The user turned on the lighting equipment at the bedside in the bedroom at 06:41:02".

[0040] Text description 3: "The user opened the bathroom door at 06:43:01".

[0041] The three factual text descriptions above are combined into a complete sequence according to the time sequence (06:40:35 → 06:41:02 → 06:43:01). This sequence is the factual semantic representation corresponding to the preset time window: "The user is active in the bedroom at 06:40:35. The user turns on the lighting equipment by the bedside in the bedroom at 06:41:02. The user opens the bathroom door at 06:43:01." This factual semantic representation, in the form of a natural language text sequence, completely records the complete event chain within the preset time window from "activity in the bedroom" to "turning on the lighting equipment" to "opening the bathroom door."

[0042] As can be seen, in this embodiment, by converting structured semantic representation into factual text descriptions in natural language form, event information is presented in a text format that conforms to human language habits. This allows it to be directly output as interpretable evidence to users or upper-layer applications, enhancing the understandability and interpretability of the system's recognition results. Furthermore, combining factual text descriptions within the same time window into a complete sequence in chronological order preserves the temporal relationships and contextual information between events. Compared to processing each event independently, the combined text sequence provides the encoder with a complete temporal context, enabling the encoder to capture the correlation information between events, thereby improving the accuracy of subsequent semantic matching.

[0043] In optional embodiments of this application, such as Figure 3 As shown, the method of semantically matching the fact-level semantic representation with the semantic prototypes of each behavior category in the behavior semantic prototype library and calculating the comprehensive semantic similarity between the fact-level semantic representation and each behavior category in step 104 above can further include: Step 31: The first semantic encoder is used to encode the combined sequence of fact-level text descriptions in the fact-level semantic representation to generate a fact embedding vector; wherein, the first semantic encoder is used to extract the specific value features of the fact-level semantic representation in the six-dimensional semantic elements; the six-dimensional semantic elements include subject, action, object, state, time and space; In this embodiment, the first semantic encoder refers to a neural network model that performs semantic encoding on the input text and outputs a fixed-dimensional vector representation. The first semantic encoder is used to extract specific value features of the factual semantic representation in six dimensions of semantic elements, that is, to extract specific numerical features from the six dimensions of subject, action, object, state, time, and space. The first semantic encoder focuses on capturing factual details of event sequences, paying attention to specific factual information such as what events occurred, what devices were involved, and when and where they occurred.

[0044] Furthermore, the fact embedding vector refers to the vector representation output by the first semantic encoder after encoding the combined sequence of fact-level text descriptions in the fact-level semantic representation. This vector is a numerical representation focusing on the detail dimension of facts within a predefined semantic space, containing specific value information for each event in the input sequence across six dimensions: subject, action, object, state, time, and space. Based on this, specific value features refer to concrete and clearly identifiable information extracted from the fact-level semantic representation, such as who the subject is, what the action is, which device the object is, what changes occurred in the state, the exact time, and which room the space is. The first semantic encoder focuses on and extracts this specific information during the encoding process, encoding it into the fact embedding vector.

[0045] Step 32: The second semantic encoder is used to encode the combined sequence of fact-level text descriptions in the fact-level semantic representation to generate behavior embedding vectors; wherein, the second semantic encoder is used to extract the intention tendency features, functional role tendency features and spatiotemporal tendency features included in the fact-level semantic representation; the second semantic encoder is a different encoder from the first semantic encoder; In this embodiment, the second semantic encoder refers to a neural network model that performs semantic encoding on the input text and outputs a fixed-dimensional vector representation. It is a different encoder from the first semantic encoder, and the two can be based on different training objectives and network structures (e.g., different pre-trained models or different network layer configurations) to represent the same input data from different levels. The second semantic encoder is used to extract the intentional tendency features, functional role tendency features, and spatiotemporal tendency features contained in the factual-level semantic representation. The second semantic encoder focuses on capturing the high-level semantic tendency embodied in the event sequence, paying attention to macro-level information such as what the event sequence is generally about, what functional devices are used, and the overall time and space context.

[0046] Furthermore, intent tendency features refer to the tendency of user behavior purpose or intent implied in the factual semantic representation. For example, an event sequence including enhanced bedroom activity → turning on the lights → entering the bathroom generally tends to indicate an intent to begin morning activities, rather than an intent to get up at night or prepare to go out. Further, functional role tendency features refer to the tendency of device functional roles involved in the factual semantic representation. Device functional roles refer to the general functional category description obtained after abstracting the specific brand information and / or specific model information of smart home devices at the functional layer, such as lighting devices, cooking devices, cleaning devices, and environmental control devices, rather than specific brands and models such as brand A bedside lamps or brand B air conditioners. Functional role tendency features reflect which functional categories of devices are activated in the event sequence. Based on this, the behavior embedding vector refers to the vector representation output by the second semantic encoder after encoding the factual semantic representation. This vector is a numerical representation focused on the behavioral intent dimension in a predefined semantic space, containing high-level semantic information such as intent tendency, functional role tendency, and spatiotemporal tendency reflected in the input sequence.

[0047] Step 33: For each behavior category, extract the second feature vector and the fourth feature vector from the semantic prototype of the behavior category and concatenate them into a fact prototype vector, and extract the first feature vector and the third feature vector and concatenate them into a behavior prototype vector; wherein, the feature vector set corresponding to the semantic prototype includes a first feature vector for representing the behavioral intention, a second feature vector for representing the typical action pattern, a third feature vector for representing the device's functional role, and a fourth feature vector for representing the spatiotemporal context. The first feature vector characterizes the behavioral intent, reflecting the purposefulness of the behavior. For example, the intent of getting up is to begin morning activities or end a sleep state. The second feature vector characterizes the typical action pattern, reflecting the typical event sequence of the behavior. For example, the typical action pattern of getting up is increased human activity in the bedroom → continuous triggering in the bedside area → subsequently leaving the bedroom. The third feature vector characterizes the functional role of the devices involved, reflecting the functional categories of the devices involved in the behavior. For example, the functional roles of devices involved in getting up include turning on the lights and opening the curtains. The fourth feature vector characterizes the spatiotemporal context, reflecting the temporal and spatial characteristics of the behavior. For example, the spatiotemporal context of getting up is the early morning time period and the path from the bedroom to the bathroom.

[0048] The fact prototype vector is the vector obtained by extracting the second and fourth feature vectors from the semantic prototype and concatenating them. The second feature vector (typical action pattern) and the fourth feature vector (spatiotemporal context) focus on describing the external manifestation of this type of behavior at the factual level—how, when, and where the event occurs. These two vectors are concatenated to form the fact prototype vector, which is used to calculate similarity with the fact embedding vector generated by the first semantic encoder. This measures the degree of matching between the observation window and a certain behavior category at the factual detail level. Concatenation refers to joining the two independent feature vectors end-to-end to merge them into a longer vector.

[0049] The behavior prototype vector is a vector obtained by extracting the first and third feature vectors from the semantic prototype and concatenating them. The first feature vector (behavioral intent) and the third feature vector (device function role) focus on describing the intrinsic attributes of this type of behavior at the high-level semantic level—what the purpose of the behavior is and what functional device is used. These two vectors are concatenated to form the behavior prototype vector, which is used to calculate similarity with the behavior embedding vector generated by the second semantic encoder to measure the degree of matching between the observation window and a certain behavior category at the behavioral intent level.

[0050] In a specific example, taking the semantic prototype of the "getting up" behavior category, the system's semantic prototype contains four feature vectors: a first feature vector (behavioral intention: starting morning activities / ending sleep), a second feature vector (typical action pattern: increased bedroom activity → turning on lights → leaving the bedroom), a third feature vector (device function role: lighting device, curtain device), and a fourth feature vector (spatiotemporal context: early morning, from bedroom to bathroom). For the "getting up" behavior category, the second and fourth feature vectors are extracted and concatenated to form the fact prototype vector, and the first and third feature vectors are extracted and concatenated to form the behavior prototype vector.

[0051] Step 34: Calculate the factual semantic similarity between the fact embedding vector and the fact prototype vector of each behavior category, and calculate the behavioral semantic similarity between the behavior embedding vector and the behavior prototype vector of each behavior category; Factual semantic similarity is a measure of similarity between fact embedding vectors and fact prototype vectors. It quantifies the degree of matching between the current observation window and a certain behavior category at the level of factual details. Factual semantic similarity can be calculated using cosine similarity, which measures the similarity by calculating the cosine of the angle between two vectors. The value ranges from -1 to 1, with values ​​closer to 1 indicating greater similarity. Similarly, behavioral semantic similarity is a measure of similarity between behavior embedding vectors and behavior prototype vectors. It quantifies the degree of matching between the current observation window and a certain behavior category at the level of behavioral intent. Behavioral semantic similarity can also be calculated using cosine similarity. These two similarities evaluate the matching degree between the observation window and the behavior category from different dimensions: factual semantic similarity focuses on whether the event details match, while behavioral semantic similarity focuses on whether the overall intent is consistent. They complement each other and together constitute a comprehensive evaluation of the behavior category.

[0052] Step 35: Perform a weighted fusion of factual semantic similarity and behavioral semantic similarity to obtain a comprehensive semantic similarity.

[0053] Weighted fusion refers to the weighted summation of factual semantic similarity and behavioral semantic similarity according to their respective weight coefficients, merging them into a comprehensive score, namely the comprehensive semantic similarity. The comprehensive semantic similarity is a single matching score obtained after fusing factual semantic similarity and behavioral semantic similarity, used to comprehensively quantify the overall matching degree between the current observation window and a certain behavioral category. The comprehensive semantic similarity considers both the matching at the level of factual details (whether the event sequence matches) and the matching at the level of behavioral intent (whether the macro-intention is consistent), thus achieving a more comprehensive evaluation of the behavioral category.

[0054] As can be seen, in this embodiment, the comprehensive semantic similarity is a single matching score obtained by fusing factual semantic similarity and behavioral semantic similarity, used to comprehensively quantify the overall matching degree between the current observation window and a certain behavioral category. The comprehensive semantic similarity considers both the matching at the level of factual details (whether the event sequence matches) and the matching at the level of behavioral intent (whether the macro-intention is consistent), thereby achieving a more comprehensive evaluation of the behavioral category.

[0055] As can be seen, in this embodiment, by employing two different semantic encoders to generate fact embedding vectors and behavior embedding vectors respectively, the representation of the same fact-level semantic representation from different levels is achieved. The first encoder focuses on the specific value features of six dimensions: subject, action, object, state, time, and space, capturing the factual details of the event sequence; the second encoder focuses on intention tendency, functional role tendency, and spatiotemporal tendency, capturing the high-level semantic information of the event sequence. The two extract features from different levels, complementing each other and providing a richer information foundation for subsequent similarity calculation. In addition, by extracting and concatenating feature vectors of different dimensions from the semantic prototype, fact prototype vectors and behavior prototype vectors are constructed, achieving a precise correspondence with the output vectors of the two encoders. The fact prototype vector is formed by concatenating a second feature vector representing a typical action pattern and a fourth feature vector representing the spatiotemporal context, corresponding to the fact embedding vector of the first encoder, and jointly focusing on matching at the factual detail level; the behavior prototype vector is formed by concatenating a first feature vector representing behavioral intention and a third feature vector representing the device's functional role, corresponding to the behavior embedding vector of the second encoder, and jointly focusing on matching at the behavioral intention level. Furthermore, by separately calculating and weighting the factual semantic similarity and behavioral semantic similarity, a comprehensive evaluation of matching results across different dimensions is achieved. The system considers both "whether the event details match" and "whether the overall intent is consistent," with the two complementing and verifying each other. When the factual details are clear, the system emphasizes detail matching; when the factual details are sparse, the system emphasizes intent inference, enhancing the system's adaptability and robustness.

[0056] In optional embodiments of this application, the method of weighted fusion of factual semantic similarity and behavioral semantic similarity to obtain a comprehensive semantic similarity involved in step 35 above may further include: Step 41: Configure a first weight coefficient for factual semantic similarity and a second weight coefficient for behavioral semantic similarity; wherein, both the first weight coefficient and the second weight coefficient are non-negative real numbers, and their sum is 1; In this application and embodiments, the first weight coefficient refers to the weight value assigned to factual semantic similarity, used to control the influence of factual semantic similarity in the calculation of comprehensive semantic similarity. The second weight coefficient refers to the weight value assigned to behavioral semantic similarity, used to control the influence of behavioral semantic similarity in the calculation of comprehensive semantic similarity. The specific value of the weight coefficient can be configured according to the actual application scenario. When the sensor data quality is high and the event details are relatively complete, the first weight coefficient can be set higher (e.g., 0.7) to make the system focus more on matching at the level of factual details; when the sensor data is relatively sparse and the event details are not complete, the second weight coefficient can be set higher (e.g., 0.7) to make the system focus more on matching at the level of behavioral intent, and to compensate for the lack of factual details through high-level semantic inclination. The weight coefficient can be determined by parameter tuning through the validation set, or it can be adaptively adjusted according to the time period and / or spatial region of the current preset time window. For example, when sensor data is relatively sparse at night, the second weight coefficient can be appropriately increased; when sensor data is relatively abundant during the day, the first weight coefficient can be appropriately increased.

[0057] Step 42: Calculate the first product of the first weight coefficient and the factual semantic similarity, and calculate the second product of the second weight coefficient and the behavioral semantic similarity; Step 43: The sum of the first product and the second product is used as the comprehensive semantic similarity.

[0058] In a specific example, taking a first weight coefficient w1 = 0.6 and a second weight coefficient w2 = 0.4 as an example, for the matching result of a certain behavior category, if the factual semantic similarity is 0.92 and the behavior semantic similarity is 0.88, then: the first product = 0.6 × 0.92 = 0.552; the second product = 0.4 × 0.88 = 0.352. The above calculation results show that under this weight configuration, the contribution of factual semantic similarity to the overall semantic similarity is 0.552, and the contribution of behavior semantic similarity to the overall semantic similarity is 0.352. Therefore, the overall semantic similarity between this observation window and this behavior category is 0.552 + 0.352 = 0.904.

[0059] If the weights are adjusted to w1 = 0.3 and w2 = 0.7 (with a greater emphasis on behavioral intent), then: S_total (Comprehensive Semantic Similarity) = 0.3 × 0.92 + 0.7 × 0.88 = 0.276 + 0.616 = 0.892; If the weights are adjusted to w1 = 0.8 and w2 = 0.2 (with a greater emphasis on factual details), then: S_total = 0.8 × 0.92 + 0.2 × 0.88 = 0.736 + 0.176 = 0.912; The above comparison shows that the overall semantic similarity will change under different weight configurations, reflecting the system's emphasis on different dimensions.

[0060] As can be seen, in this embodiment, by configuring weight coefficients for factual semantic similarity and behavioral semantic similarity respectively, flexible adjustment of matching scores for different dimensions is achieved. Furthermore, the weight ratio of the two dimensions can be dynamically adjusted according to the needs of actual application scenarios, emphasizing factual matching when factual details are clear and emphasizing intent inference when factual details are sparse, thereby adapting to different data conditions and application requirements.

[0061] In optional embodiments of this application, such as Figure 4 As shown, the method of determining and outputting the target behavior category corresponding to the preset time window based on the comprehensive semantic similarity involved in step 105 above can further include: Step 51: Select the maximum value from the comprehensive semantic similarity of all behavior categories, and take the behavior category corresponding to the maximum value as the candidate target behavior category; In this embodiment, the maximum value refers to the highest numerical value among the comprehensive semantic similarity scores of all behavior categories. For example, if the system supports four behavior categories—getting up, cooking, sleeping, and going out—and the calculated comprehensive semantic similarity scores are 0.87, 0.34, 0.21, and 0.15 respectively, then the maximum value is 0.87. The candidate target behavior category refers to the behavior category corresponding to the maximum value, i.e., the behavior category with the highest score in semantic matching in the current observation window. This category, as a candidate for the final identification result, still needs to be verified by subsequent threshold conditions before it can be finally confirmed.

[0062] In this specific example, the comprehensive semantic similarity of each behavior category is as follows: 0.87 for "getting up," 0.62 for "getting up at night," and 0.34 for "preparing to go out." The maximum value is 0.87, corresponding to the behavior category "getting up," so "getting up" is selected as the candidate target behavior category. The purpose of this selection step is to filter out the one with the highest matching degree from all possible behavior candidates as the object of subsequent verification, avoiding unnecessary processing overhead by verifying all candidates one by one.

[0063] Step 52: Determine whether the maximum value is greater than or equal to a preset first threshold, and whether the difference between the maximum value and the second largest value is greater than or equal to a preset second threshold. The first threshold is a pre-set confidence threshold value used to determine whether the comprehensive semantic similarity of candidate target behavior categories reaches a sufficiently credible level. The first threshold is usually set to a relatively high value (e.g., 0.75), meaning that only when the matching score of a candidate behavior category exceeds this threshold is the recognition result considered to have sufficient confidence to be output. If the highest score does not reach the first threshold, it indicates that the information in the current observation window is insufficient to determine any behavior category with enough confidence. In this case, even if there is a relatively highest score, the recognition result should not be output hastily.

[0064] The second-highest value refers to the score that is second only to the highest value among all the comprehensive semantic similarity scores of all behavior categories. The behavior category corresponding to the second-highest value is the competing category that is closest to the candidate target behavior category, i.e., the other behavior category most likely to be confused during the judgment. The second threshold is a pre-set discrimination threshold value used to determine whether the candidate target behavior category has sufficient discrimination against other behavior categories. The second threshold is usually set to a small positive number (e.g., 0.15), which means that only when the comprehensive semantic similarity of the candidate target behavior category is significantly higher than that of all other behavior categories is the recognition result considered clear and without the possibility of confusion. If the difference between the highest score and the second-highest score does not reach the second threshold, it indicates hesitation between the candidate target behavior category and the closest competing category. In this case, hastily outputting the result may lead to misjudgment.

[0065] Step 53: If all are yes, then determine the candidate target behavior category as the target behavior category corresponding to the preset time window and output it; otherwise, output the pending confirmation status and wait for the judgment result of the next time window.

[0066] In a specific example, if the current time window only observes two events: activity in the bedroom and the lighting being turned on, the score for getting up is 0.72, and the score for getting up at night is 0.68. Let the first threshold T1 = 0.75. The maximum value 0.72 < 0.75, so condition one is not met. Output a pending confirmation status and wait for data from the next time window. In the next time window, if the bathroom door sensor is triggered, the events from the two windows are merged and re-evaluated. The score for getting up increases to 0.87, and the score for getting up at night decreases to 0.62, satisfying the double threshold condition. At this point, "Getting up" is output. Alternatively, if the current window only observes activity in the bedroom and the bedroom light being turned on, the score for getting up is 0.80, and the score for getting up at night is 0.72. The maximum value 0.80 ≥ 0.75 (condition one is met), but the difference 0.08 < 0.15 (condition two is not met). Output a pending confirmation status and wait for subsequent events to distinguish whether it is getting up or getting up at night.

[0067] As can be seen, in this embodiment, by setting a first threshold (confidence threshold), the recognition result is only output when the matching score of the candidate behavior category reaches sufficient confidence. When the comprehensive semantic similarity of all behavior categories is lower than the first threshold, it indicates that the information in the current observation window is insufficient to support any reliable behavior judgment. In this case, a pending confirmation status is output instead of forcibly outputting the category with the highest score, effectively avoiding misjudgment under insufficient information and improving recognition reliability and user experience. In addition, by setting a second threshold (discrimination threshold), the recognition result is only output when there is sufficient discrimination between the candidate target behavior category and other behavior categories. When the difference between the highest score and the second highest score is less than the second threshold, it indicates hesitation or confusion between multiple behavior categories. In this case, a pending confirmation status is output instead of forcibly choosing one of the two, effectively avoiding misjudgment under ambiguous conditions, especially avoiding confusion between similar behaviors such as getting up and getting up at night, thus improving the accuracy of recognition.

[0068] In optional embodiments of this application, such as Figure 5 As shown, the method of constructing the behavioral semantic prototype library involved in step 103 above can further include: Step 61: For each behavior category, construct at least one semantic prototype description text; wherein, the semantic prototype description text includes a description of the behavior intention of the behavior category, a description of typical action patterns, a description of the device function role, and a description of the spatiotemporal context. In this embodiment, semantic prototype description text refers to free text written in natural language that semantically describes a certain behavior category. Each semantic prototype description text is a complete characterization of that type of behavior, describing its typical characteristics from multiple dimensions. Specifically, the behavioral intent description refers to a textual description of the purpose or motivation for the behavior category, answering the question "Why does the user do this?" For example, the behavioral intent of getting up can be described as starting morning activities or ending a sleep state. The typical action pattern description refers to a textual description of the typical event sequence or action flow in the behavior category, answering the question "How does the user usually do this?" The typical action pattern description focuses on the order and logical relationship of events. For example, the typical action pattern of getting up can be described as increased human activity in the bedroom, continuous triggering in the bedside area, and then leaving the bedroom. The device function role description refers to a textual description of the device function categories involved in the behavior category, answering the question "What functional devices are used to do this?" The device function role refers to a general functional category description obtained after abstracting the specific brand information and / or specific model information of smart home devices at the functional layer, such as lighting equipment, cooking equipment, cleaning equipment, and environmental control equipment. For example, the functional role of the equipment used to get up can be described as turning on the lights or opening the curtains. Spatiotemporal context description refers to the text that describes the time and space environment in which this behavior occurs, answering the question, "When and where does this usually happen?" For example, the spatiotemporal context of getting up can be described as the early morning time period or the path from the bedroom to the bathroom.

[0069] Taking the "getting up" behavior category as an example, the following two semantic prototype description texts can be constructed: Prototype description text A: "Behavioral intent is to begin morning activities; typical action pattern is increased human activity in the bedroom, continuous triggering in the bedside area, followed by leaving the bedroom; device function role is turning on the lighting device and opening the curtain device; spatiotemporal context is the early morning time period and the path from the bedroom to the bathroom." Prototype description text B: "The behavioral intention is to end the sleep state; the typical action pattern is the end of the bedroom's static state and the occurrence of continuous movement events; the device function role is that of a lighting device or an environmental control device switching from night mode to day mode; the spatiotemporal context is weekdays from 06:00 to 08:30 or a relatively later time period on weekends." The two descriptive texts above portray the same behavior category (waking up) from different perspectives. Prototype descriptive text A focuses more on the outward manifestations of the behavior (increased activity → turning on the lights → leaving the bedroom), while prototype descriptive text B focuses more on the inner meaning of the behavior (end of sleep → mode switching). By constructing multiple descriptive texts, the system can cover the differentiated habits of different residents (such as early risers vs. late risers, different patterns in winter and summer, etc.), thereby improving the coverage and robustness of semantic prototypes.

[0070] Step 62: Map each semantic prototype description text to a preset multi-dimensional semantic vector space to generate a corresponding initial feature vector set, wherein the initial feature vector set includes behavioral intention sub-vectors, typical action pattern sub-vectors, device function role sub-vectors, and spatiotemporal context sub-vectors. In this embodiment, the pre-defined multidimensional semantic vector space refers to a high-dimensional vector space in which texts or data with similar semantic meanings are mapped to similar positions in the space. This space is obtained through pre-training (e.g., through a language model pre-trained on a large-scale corpus), enabling computers to understand and compare the semantic relationships between different texts. For example, although "turn on the lighting equipment" and "turn on the light" are different expressions, they will be mapped to similar vector positions in the semantic space. Mapping refers to inputting the semantic prototype description text in natural language form into a semantic encoding model (such as a pre-trained language model based on the Transformer architecture), and the model converts the variable-length text sequence into a fixed-length numerical vector. This vector is a mathematical representation of the text's semantic information in the pre-defined semantic space.

[0071] Furthermore, the initial feature vector set refers to the complete vector set generated after mapping a semantic prototype description text. Since the semantic prototype description text simultaneously contains descriptions of four dimensions: behavioral intent, typical action pattern, device function / role, and spatiotemporal context, the initial feature vector set generated after mapping correspondingly includes four sub-vectors: behavioral intent sub-vector, typical action pattern sub-vector, device function / role sub-vector, and spatiotemporal context sub-vector. Specifically, the behavioral intent sub-vector is the sub-vector in the initial feature vector set used to represent the behavioral intent dimension, generated by mapping the behavioral intent description portion of the semantic prototype description text. The typical action pattern sub-vector is the sub-vector in the initial feature vector set used to represent the typical action pattern dimension, generated by mapping the typical action pattern description portion of the semantic prototype description text. The device function / role sub-vector is the sub-vector in the initial feature vector set used to represent the device function / role dimension, generated by mapping the device function / role description portion of the semantic prototype description text. The spatiotemporal context sub-vector is the sub-vector in the initial feature vector set used to represent the spatiotemporal context dimension, generated by mapping the spatiotemporal context description portion of the semantic prototype description text. Each sub-vector is a numerical vector of a fixed dimension, such as 256 or 512 dimensions. The four sub-vectors together constitute the initial feature vector set corresponding to a semantic prototype describing the text.

[0072] To address this, taking the aforementioned text A describing the prototype of waking up as an example, it is mapped to a preset multi-dimensional semantic vector space to generate a corresponding initial feature vector set V_A. This initial feature vector set V_A includes: behavioral intention sub-vector (corresponding to starting morning activities), typical action pattern sub-vector (corresponding to increased human activity in the bedroom, continuous triggering in the bedside area, and subsequently leaving the bedroom), device function role sub-vector (corresponding to turning on lighting devices and opening curtain devices), and spatiotemporal context sub-vector (corresponding to the morning time period and the path from the bedroom to the bathroom).

[0073] Step 63: For multiple initial feature vector sets corresponding to multiple semantic prototype description texts under the same behavior category, perform fusion processing according to four dimensions: behavior intention, typical action mode, device function role, and spatiotemporal context, to generate the first feature vector of the behavior category in the behavior intention dimension, the second feature vector in the typical action mode dimension, the third feature vector in the device function role dimension, and the fourth feature vector in the spatiotemporal context dimension. Fusion processing refers to the process of merging sub-vectors belonging to the same dimension from multiple initial feature vector sets corresponding to multiple semantic prototype description texts under the same behavior category into a single vector. The purpose of fusion processing is to integrate the different aspects of multiple description texts depicting the same behavior category into a unified standard profile. Based on this, separate fusion processing means fusing each of the four dimensions independently without interference. Specifically: the behavioral intent sub-vectors from all initial feature vector sets are merged into the first feature vector; the typical action pattern sub-vectors from all initial feature vector sets are merged into the second feature vector; the device function / role sub-vectors from all initial feature vector sets are merged into the third feature vector; and the spatiotemporal context sub-vectors from all initial feature vector sets are merged into the fourth feature vector.

[0074] The fusion process can be implemented using various mathematical operations, such as arithmetic mean (calculating the average of the elements at corresponding positions of each sub-vector), weighted average (calculating the weighted average after assigning different weights to each sub-vector), and dimensionality reduction after concatenation (concatenating the sub-vectors and then reducing their dimensionality through a fully connected layer). When using the arithmetic mean method, the feature vectors obtained after fusion have the same dimensionality as the sub-vectors before fusion.

[0075] Specifically, if the behavior category generates two semantic prototype description texts, then two initial feature vector sets, V_A and V_B, are generated. These are then fused along four dimensions, calculated as follows: First feature vector (behavioral intention dimension) = Fusion (behavioral intention sub-vector of V_A, behavioral intention sub-vector of V_B); Second feature vector (typical action pattern dimension) = Fusion (typical action pattern sub-vector of V_A, typical action pattern sub-vector of V_B); The third feature vector (device function role dimension) = fusion (device function role sub-vector of V_A, device function role sub-vector of V_B); Fourth feature vector (spatiotemporal context dimension) = fusion (spatiotemporal context sub-vector of V_A, spatiotemporal context sub-vector of V_B).

[0076] Through the above fusion process, information from multiple descriptive texts under the same behavior category can be integrated into a unified standard profile. This profile combines the different perspectives of each descriptive text on the behavior category, and has stronger representativeness, coverage and robustness.

[0077] Step 64: The feature vector set consisting of the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector is used as the semantic prototype of the behavior category.

[0078] As can be seen, in this embodiment, by constructing at least one semantic prototype description text for each behavior category, the traditional behavior label is expanded to include a semantic description encompassing four dimensions: behavioral intent, typical action pattern, device functional role, and spatiotemporal context. This makes the behavior definition more comprehensive and accurate, overcoming the problem in existing solutions where behavior labels only reflect the behavior name and cannot reflect deep semantic information. Furthermore, by mapping the semantic prototype description text to a preset multi-dimensional semantic vector space, natural language descriptions are converted into mathematical feature vectors, enabling computers to understand and compare the semantic relationships between different texts, providing a quantifiable computational basis for subsequent semantic matching calculations. Simultaneously, since the feature vectors are constructed in the semantic space, they possess the ability to recognize synonymous expressions—although "turning on the lighting device" and "turning on the light" are different expressions, they are mapped to similar positions in the semantic space, allowing subsequent semantic matching to recognize different expressions. Furthermore, by fusing multiple initial feature vector sets corresponding to various semantic prototype descriptions of the same behavior category, the different descriptions of the behavior category from different perspectives are integrated into a unified standard profile. This not only improves the coverage of the semantic prototype (capable of covering the differentiated habits of different households) but also enhances its robustness (the bias of a single description is averaged out during the fusion process). Simultaneously, multiple semantic descriptions can be constructed and fused within the same behavior category, allowing the system to flexibly adapt to the differences in user habits across different households; for example, both early risers and late risers can be covered by the same semantic prototype.

[0079] The following description, in conjunction with specific embodiments of this application, further explains this application. This specific embodiment provides a semantically enhanced behavior recognition method and system for multi-source heterogeneous data in smart homes. The system is deployed in a home, and the access objects include, but are not limited to, door magnetic sensors, human infrared sensors, temperature and humidity sensors, power switch controllers, smart appliance control logs, and scene linkage logs. To ensure the stability of subsequent recognition results, the system first performs unified preprocessing: mapping the timestamps reported by different devices to a unified time zone and unified precision, removing duplicate reports and abnormal null values, and standardizing the event source identifier, device identifier, action code, and status code to form an event stream to be parsed. Based on this, combined with... Figure 6 The method of this specific implementation includes the following stages and steps: In the fact-level semantic modeling phase, the system invokes the event-aware semantic parsing framework and executes the corresponding parsing strategy based on the event type: for sensor-triggered events, at least the triggering subject, triggering action, affected object, state change, and spatiotemporal information are extracted; for device operation logs, at least the operating subject, control action, device object, target state, and execution time and location are extracted. The system writes the above information into a unified semantic slot structure to form a fact-level semantic representation. To balance interpretability and transferability, this fact-level semantic representation can be simultaneously saved as a structured slot record and a fact-level text description.

[0080] Example as follows: Example 1 (Sensor Event): The original event is "2026-03-31 06:42:10, bedroom human infrared sensor PIR-03 triggered". The parsed slots can be: Subject = User, Action = Activity Occurred, Object = Human Body Exists Sensor Event, State = Triggered, Time = 06:42:10, Space = Bedroom. The corresponding fact-level text can be "User was active in the bedroom at 06:42". Example 2 (Device Status Change Log): The original event is "2026-03-31 06:43:02, Device Light-Bed-01, Status off->on, Source APP Automation Rule". The parsed slots can be: Subject = Home Control System, Action = On, Object = Lighting Device, Status = Changed from Off to On, Time = 06:43:02, Space = Bedside in Bedroom. The corresponding fact-level text can be "The system turned on the lighting device at the bedside in the bedroom at 06:43".

[0081] In the behavioral semantic prototype construction phase, the system establishes a behavioral semantic prototype for each behavioral category. Unlike traditional methods that only retain behavioral label names, this invention supplements each behavioral category with four types of semantic information: behavioral intent, typical action pattern, device functional role, and spatiotemporal context. Specifically, it maps specific device names to functional roles, such as "lighting equipment," "cooking equipment," "cleaning equipment," and "environmental control equipment," thereby reducing the system's dependence on specific hardware models and naming conventions. For the same behavioral category, multiple semantic descriptions can be constructed and fused to form a prototype semantic representation for that category, improving coverage and robustness.

[0082] Example as follows: The "Get Up" tag can be constructed as a prototype description as follows: Prototype Description A: The behavioral intention is to begin morning activities; the typical action pattern is increased human activity in the bedroom, continuous triggering in the bedside area, and then leaving the bedroom; the device function role is turning on the lighting device and opening the curtain device; the spatiotemporal context is the early morning time period and the path from the bedroom to the bathroom.

[0083] Prototype Description B: The behavioral intention is to end the sleep state; the typical action pattern is the end of the static state in the bedroom and the occurrence of continuous movement events; the device function role is a lighting device or environmental control device switching from night mode to day mode; the spatiotemporal context is weekdays from 06:00 to 08:30 or a relatively later time period on weekends.

[0084] The "Cooking" tag can be used to construct prototype descriptions as follows: behavioral intent is to prepare meals; typical action patterns include entering the kitchen, continuous activity on the countertop, and triggering in the food storage area; equipment function roles include cooking equipment, lighting equipment, and ventilation equipment being activated sequentially or partially; and the spatiotemporal context is the high-frequency period of breakfast and dinner. The system encodes multiple prototype descriptions under the same tag and merges them into the prototype vector center of that tag to cover different residents' habits.

[0085] During the behavior recognition phase, the system maps the real-time observed fact-level semantic representations to a unified semantic space with the behavior semantic prototype library and performs matching calculations. The matching process includes the following steps: Step S1: Construct the current observation window. Combine factual semantic events within a recent period into observation segments, such as "06:42 Bedroom human infrared trigger - 06:43 Bedside lamp turned on - 06:45 Bathroom door magnetic trigger".

[0086] Step S2: Calculate semantic similarity. Calculate the semantic similarity between the observed segment and each candidate behavior prototype to obtain the basic similarity score.

[0087] factual semantic similarity :

[0088] Where W represents the currently constructed observation window, which is a set of fact-level events (e.g., {event A, event B, event C}). c: represents the candidate behavior category, such as c = get up, or c = cook. It is the embedding vector that maps the observation window W to the fact semantic space. It encodes the specific details of all events within the window (who, where, what did they do). It is the prototype vector of behavior category c in the fact semantic space, which encodes the typical pattern (typical event flow) of this type of behavior at the fact level.

[0089] behavioral semantic similarity :

[0090] in, It is an embedding vector that maps the observation window W to the semantic space of behavioral intentions. It encodes the high-level intentions reflected by the combination of events within the window (such as whether to end sleep or prepare to go out). It is the prototype vector of behavior category c in the behavior intention semantic space, which encodes the intention, functional role and spatiotemporal context of this type of behavior.

[0091] Step S3: Calculate the final matching score. The basic similarity and each consistency score are weighted and summed according to preset weights to obtain the final score for each candidate behavior, and then sorted.

[0092] By comprehensively utilizing information from different semantic levels, a weighted fusion method is used to integrate the similarity between the two perspectives, and a final behavior matching score is defined. for:

[0093] in And satisfy: , These are weight parameters, used to adjust the importance of semantic concept matching and factual consistency constraints in different scenarios. They are optimized through validation sets or adaptively set based on behavior categories. Ultimately, the behavior recognition result is obtained through the principle of maximizing semantic matching:

[0094] Step S4: Output the recognition result. If the maximum value is greater than the first recognition threshold T1 and the difference between the maximum value and the second maximum value is greater than the second discrimination threshold T2, then the highest score behavior is directly output; otherwise, "behavior to be confirmed" is output, and subsequent events are accumulated and re-determined.

[0095] Taking "getting up" as an example, the current window observes enhanced activity in the bedroom, the lighting being turned on, and the user leaving the bedroom and entering the bathroom. The system calculates a similarity of 0.86 with the "getting up" prototype, a device function consistency of 0.90, a time period consistency of 0.88, and a spatial path consistency of 0.84. After weighted fusion, the final score is 0.87. The score with the "getting up at night" prototype is 0.71, and the score with the "preparing to go out" prototype is 0.63. Since the "getting up" score is the highest and exceeds the threshold, and the difference with the second-highest score meets the condition, "getting up" is output.

[0096] In addition, the system will also output interpretable evidence, including key matching events: bedside lamp turned on, bathroom door magnetic trigger, key functional role: lighting equipment, and key time period: 06:30-07:00, to support subsequent automated linkage or manual review.

[0097] As can be seen, this embodiment no longer directly splices together various raw data. Instead, it first performs semantic parsing according to event type, converting structured or semi-structured data into a unified fact-level semantic text or semantic slot structure. This way, regardless of whether the data comes from door magnets, switches, environmental sensors, or device logs, it can be mapped to the same expression framework. This improves the unified processing capability of multi-source data, reduces the impact of differences in device protocols and data formats on behavior recognition, and enhances the interpretability of the behavior recognition process, providing clear semantic basis for subsequent behavior judgment. Furthermore, in this embodiment, each behavior label is expanded into a natural language semantic prototype. The semantic prototype not only describes the behavior name but also includes the behavior intent, typical action pattern, device functional role, and spatiotemporal context information. Then, the currently observed fact-level semantics is semantically matched with each behavior semantic prototype to output the final recognition result. This improves the system's understanding of the meaning of behavior, making the recognition result more accurate and stable. Simultaneously, it reduces the model's dependence on specific device configurations, enhances the system's generalization ability in different home and device environments, and strengthens its application value in zero-shot or weakly supervised scenarios.

[0098] Corresponding to the above Figure 1 This application also provides a device for recognizing user behavior based on smart home devices, such as... Figure 7 As shown, the device includes: The acquisition module 702 is used to acquire raw event data generated by smart home devices within a preset time window; The first processing module 704 is used to perform semantic parsing on the raw event data and generate a fact-level semantic representation corresponding to a preset time window; wherein, the fact-level semantic representation refers to mapping the multi-source heterogeneous raw event data generated by smart home devices within the preset time window to a unified semantic expression framework. Module 706 is used to build a behavior semantic prototype library, which includes multiple preset behavior categories and their corresponding semantic prototypes; the semantic prototype is a set of feature vectors constructed in a preset semantic space. The second processing module 708 is used to perform semantic matching between the fact-level semantic representation and the semantic prototype of each behavior category in the behavior semantic prototype library, and to calculate the comprehensive semantic similarity between the fact-level semantic representation and each behavior category. The third processing module 710 is used to determine and output the target behavior category corresponding to the preset time window based on the comprehensive semantic similarity.

[0099] In an optional embodiment of this application, the first processing module in this application includes: The identification unit is used to identify the event type of each raw event data within a preset time window; The first processing unit is used to call the corresponding parsing strategy according to the event type in order to extract the subject field, action field, object field, status field, time field and space field from each raw event data; The second processing unit is used to fill the extracted fields into a preset unified semantic slot structure to generate a structured semantic representation corresponding to the original event data; wherein, the unified semantic slot structure includes at least subject slots, action slots, object slots, state slots, time slots and space slots; The third processing unit is used to combine the structured semantic representations generated within a preset time window in chronological order to generate fact-level semantic representations.

[0100] In an optional embodiment of this application, the third processing unit in this application is used to perform the following steps: Each structured semantic representation generated within a preset time window is converted into a fact-level text description in natural language form. The fact-level text descriptions generated within the same preset time window are combined in chronological order to generate a fact-level semantic representation.

[0101] In an optional embodiment of this application, the second processing module includes: The fourth processing unit is used to encode the combined sequence of fact-level text descriptions in the fact-level semantic representation using the first semantic encoder to generate a fact embedding vector; wherein, the first semantic encoder is used to extract the specific value features of the fact-level semantic representation in the six-dimensional semantic elements; the six-dimensional semantic elements include subject, action, object, state, time and space; The fifth processing unit is used to encode the combined sequence of fact-level text descriptions in the fact-level semantic representation using a second semantic encoder to generate a behavior embedding vector; wherein, the second semantic encoder is used to extract the intention tendency features, functional role tendency features and spatiotemporal tendency features included in the fact-level semantic representation; the second semantic encoder is a different encoder from the first semantic encoder; The sixth processing unit is used to extract the second feature vector and the fourth feature vector from the semantic prototype of each behavior category and concatenate them into a fact prototype vector, and to extract the first feature vector and the third feature vector and concatenate them into a behavior prototype vector; wherein, the feature vector set corresponding to the semantic prototype includes a first feature vector for representing the behavioral intention, a second feature vector for representing the typical action pattern, a third feature vector for representing the device's functional role, and a fourth feature vector for representing the spatiotemporal context; The seventh processing unit is used to calculate the fact semantic similarity between the fact embedding vector and the fact prototype vector of each behavior category, and to calculate the behavior semantic similarity between the behavior embedding vector and the behavior prototype vector of each behavior category. The eighth processing unit is used to perform weighted fusion of factual semantic similarity and behavioral semantic similarity to obtain a comprehensive semantic similarity.

[0102] In an optional embodiment of this application, the eighth processing unit in this application embodiment is used to perform the following steps: Assign a first weight coefficient to factual semantic similarity and a second weight coefficient to behavioral semantic similarity; wherein the first weight coefficient and the second weight coefficient are both non-negative real numbers, and their sum is 1; Calculate the first product of the first weight coefficient and the factual semantic similarity, and calculate the second product of the second weight coefficient and the behavioral semantic similarity; The sum of the first and second products is used as the comprehensive semantic similarity.

[0103] In an optional embodiment of this application, the third processing module includes: The ninth processing unit is used to select the maximum value from the comprehensive semantic similarity of all behavior categories, and take the behavior category corresponding to the maximum value as the candidate target behavior category; The judgment unit is used to determine whether the maximum value is greater than or equal to a preset first threshold, and whether the difference between the maximum value and the second largest value is greater than or equal to a preset second threshold. The tenth processing unit is used to determine the candidate target behavior category as the target behavior category corresponding to the preset time window and output it if all are yes; otherwise, it outputs the pending confirmation status and waits for the judgment result of the next time window.

[0104] In optional embodiments of this application, the construction module includes: The building unit is used to construct at least one semantic prototype description text for each behavior category; wherein, the semantic prototype description text includes a description of the behavior intention, a description of typical action patterns, a description of the device function role, and a description of the spatiotemporal context of the behavior category; The eleventh processing unit is used to map each semantic prototype description text to a preset multi-dimensional semantic vector space and generate a corresponding initial feature vector set, wherein the initial feature vector set includes behavioral intention sub-vectors, typical action pattern sub-vectors, device function role sub-vectors and spatiotemporal context sub-vectors. The twelfth processing unit is used to fuse multiple initial feature vector sets corresponding to multiple semantic prototype description texts under the same behavior category according to four dimensions: behavior intention, typical action mode, device function role and spatiotemporal context, to generate the first feature vector of the behavior category in the behavior intention dimension, the second feature vector in the typical action mode dimension, the third feature vector in the device function role dimension and the fourth feature vector in the spatiotemporal context dimension. The thirteenth processing unit is used as the semantic prototype of the behavior category by the feature vector set consisting of the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector.

[0105] like Figure 8 As shown in the figure, this application provides an electronic device, including a processor 811, a communication interface 812, a memory 813, and a communication bus 814, wherein the processor 811, the communication interface 812, and the memory 813 communicate with each other through the communication bus 814. Memory 813 is used to store computer programs; In one embodiment of this application, when the processor 811 executes the program stored in the memory 813, it implements the method for identifying user behavior based on smart home devices provided in any of the aforementioned method embodiments. Its function is similar and will not be described again here.

[0106] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method for recognizing user behavior based on smart home devices as provided in any of the foregoing method embodiments.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0109] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0110] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for recognizing user behavior based on smart home devices, characterized in that, include: Acquire raw event data generated by smart home devices within a preset time window; Semantic parsing is performed on the raw event data to generate a fact-level semantic representation corresponding to the preset time window; wherein, the fact-level semantic representation refers to mapping the multi-source heterogeneous raw event data generated by the smart home devices within the preset time window to a unified semantic expression framework; Construct a behavioral semantic prototype library, wherein the behavioral semantic prototype library includes multiple preset behavioral categories and their corresponding semantic prototypes; the semantic prototype is a set of feature vectors constructed in a preset semantic space; The fact-level semantic representation is semantically matched with the semantic prototypes of each behavior category in the behavior semantic prototype library, and the comprehensive semantic similarity between the fact-level semantic representation and each behavior category is calculated. Based on the comprehensive semantic similarity, the target behavior category corresponding to the preset time window is determined and output.

2. The method according to claim 1, characterized in that, Performing semantic parsing on the original event data to generate a fact-level semantic representation corresponding to the preset time window includes: Identify the event type of each raw event data within the preset time window; The corresponding parsing strategy is invoked according to the event type to extract the subject field, action field, object field, state field, time field, and space field from each raw event data; The extracted fields are filled into a preset unified semantic slot structure to generate a structured semantic representation corresponding to the original event data; wherein, the unified semantic slot structure includes at least subject slots, action slots, object slots, state slots, time slots and space slots; The structured semantic representations generated within the preset time window are combined in chronological order to generate the fact-level semantic representation.

3. The method according to claim 2, characterized in that, The structured semantic representations generated within the preset time window are combined in chronological order to generate the fact-level semantic representation, including: Each of the structured semantic representations generated within the preset time window is converted into a fact-level text description in natural language form; The fact-level text descriptions generated within the same preset time window are combined in chronological order to generate the fact-level semantic representation.

4. The method according to claim 1, characterized in that, The fact-level semantic representation is semantically matched with the semantic prototypes of each behavior category in the behavior semantic prototype library, and the comprehensive semantic similarity between the fact-level semantic representation and each behavior category is calculated, including: A first semantic encoder is used to encode the combined sequence of fact-level text descriptions in the fact-level semantic representation to generate a fact embedding vector; wherein, the first semantic encoder is used to extract the specific value features of the fact-level semantic representation in six-dimensional semantic elements; the six-dimensional semantic elements include subject, action, object, state, time and space; A second semantic encoder is used to encode the combined sequence of fact-level text descriptions in the fact-level semantic representation to generate a behavior embedding vector; wherein, the second semantic encoder is used to extract the intention tendency features, functional role tendency features, and spatiotemporal tendency features included in the fact-level semantic representation; the second semantic encoder is a different encoder from the first semantic encoder; For each behavior category, a second feature vector and a fourth feature vector are extracted from the semantic prototype of that behavior category and concatenated to form a fact prototype vector, and a first feature vector and a third feature vector are extracted and concatenated to form a behavior prototype vector; wherein, the feature vector set corresponding to the semantic prototype includes a first feature vector for representing behavioral intent, a second feature vector for representing typical action patterns, a third feature vector for representing the functional role of the device, and a fourth feature vector for representing the spatiotemporal context. Calculate the fact semantic similarity between the fact embedding vector and the fact prototype vector of each behavior category, and calculate the behavior semantic similarity between the behavior embedding vector and the behavior prototype vector of each behavior category; The factual semantic similarity and the behavioral semantic similarity are weighted and fused to obtain the comprehensive semantic similarity.

5. The method according to claim 4, characterized in that, The comprehensive semantic similarity is obtained by weighted fusion of the factual semantic similarity and the behavioral semantic similarity, including: A first weight coefficient is configured for the factual semantic similarity, and a second weight coefficient is configured for the behavioral semantic similarity; wherein the first weight coefficient and the second weight coefficient are both non-negative real numbers, and their sum is 1; Calculate the first product of the first weight coefficient and the factual semantic similarity, and calculate the second product of the second weight coefficient and the behavioral semantic similarity; The sum of the first product and the second product is taken as the comprehensive semantic similarity.

6. The method according to claim 1, characterized in that, Based on the comprehensive semantic similarity, the target behavior category corresponding to the preset time window is determined and output, including: Select the maximum value from the comprehensive semantic similarity of all behavior categories, and use the behavior category corresponding to the maximum value as the candidate target behavior category; Determine whether the maximum value is greater than or equal to a preset first threshold, and whether the difference between the maximum value and the second largest value is greater than or equal to a preset second threshold; If all are yes, then the candidate target behavior category is determined as the target behavior category corresponding to the preset time window and output; Otherwise, output a pending confirmation status and wait for the judgment result in the next time window.

7. The method according to claim 1, characterized in that, Construct a behavioral semantic prototype library, including: For each behavior category, at least one semantic prototype description text is constructed; wherein, the semantic prototype description text includes a description of the behavior intention, a description of typical action patterns, a description of the device's functional role, and a description of the spatiotemporal context of the behavior category; Each semantic prototype description text is mapped to a preset multi-dimensional semantic vector space to generate a corresponding initial feature vector set, wherein the initial feature vector set includes behavioral intention sub-vectors, typical action pattern sub-vectors, device function role sub-vectors, and spatiotemporal context sub-vectors. For multiple initial feature vector sets corresponding to multiple semantic prototype description texts under the same behavior category, the fusion processing is performed according to four dimensions: behavior intention, typical action mode, device function role and spatiotemporal context, to generate the first feature vector of the behavior category in the behavior intention dimension, the second feature vector in the typical action mode dimension, the third feature vector in the device function role dimension and the fourth feature vector in the spatiotemporal context dimension. The feature vector set consisting of the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector serves as the semantic prototype of the behavior category.

8. A device for recognizing user behavior based on smart home devices, characterized in that, include: The acquisition module is used to acquire raw event data generated by smart home devices within a preset time window; The first processing module is used to perform semantic parsing on the raw event data and generate a fact-level semantic representation corresponding to the preset time window; wherein, the fact-level semantic representation refers to mapping the multi-source heterogeneous raw event data generated by the smart home device within the preset time window to a unified semantic expression framework; A construction module is used to build a behavioral semantic prototype library, wherein the behavioral semantic prototype library includes multiple preset behavioral categories and their corresponding semantic prototypes; the semantic prototype is a set of feature vectors constructed in a preset semantic space; The second processing module is used to perform semantic matching between the fact-level semantic representation and the semantic prototypes of each behavior category in the behavior semantic prototype library, and to calculate the comprehensive semantic similarity between the fact-level semantic representation and each behavior category. The third processing module is used to determine and output the target behavior category corresponding to the preset time window based on the comprehensive semantic similarity.

9. An electronic device, characterized in that, include: The system includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor, when executing the computer programs, implements the method for recognizing user behavior based on any one of claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for recognizing user behavior based on smart home devices as described in any one of claims 1-7.

Images