Skill generation method, control device and control system for body-equipped agent

Abstract

The invention is suitable for the technical field of smart home control, and provides a skill generation method, control equipment and a control system for an intelligent agent with a body. The method comprises the following steps: acquiring a first behavior sequence formed by interaction events of a user and smart home equipment in a first time period; performing multi-dimensional mining on the first behavior sequence, and mining a plurality of behavior modes; for any behavior mode, generating a corresponding skill package based on the behavior mode; and when a trigger source is detected, the intelligent agent controls the corresponding smart home equipment to execute corresponding actions according to the generated skill packages. According to the invention, behavior modes can be mined from multiple dimensions, complex behaviors can be understood, and self-adaptive smart home personalized services can be realized.

Description

Technical Field

[0001] This application belongs to the field of smart home control technology, and in particular relates to a skill generation method, control device and control system for embodied intelligent agents. Background Technology

[0002] Currently, personalized services for smart home systems primarily rely on two technical solutions: "manual rule configuration" and "automation based on simple statistics." Manual rule configuration involves users manually setting scheduled tasks or linked scenarios (e.g., "turn on the living room light at 7 PM every day") via an app or voice assistant, and the system executes the fixed action according to the preset rules. Automation based on simple statistics involves the system collecting device usage frequency data and recommending possible automation rules to the user (e.g., "You often turn on the living room light at night; do you want to create an automation?"). After user confirmation, a fixed trigger-action rule is generated. This method typically only counts frequency in a single dimension and cannot understand complex behaviors. Summary of the Invention

[0003] This application provides a skill generation method, control device, and control system for embodied intelligent agents, which can mine behavioral patterns from multiple dimensions, understand complex behaviors, and realize adaptive personalized smart home services.

[0004] In a first aspect, embodiments of this application provide a skill generation method for embodied intelligent agents, including: Obtain the first behavioral sequence formed by the various interaction events between the user and smart home devices within the first time period; The first behavior sequence is mined from multiple dimensions to uncover multiple behavior patterns. These multiple behavior patterns include at least two of the following: periodic pattern, first sequence pattern, first association pattern, and scenario pattern. The periodic pattern represents a user behavior pattern with a time regularity. The first sequence pattern represents the temporal relationship between different device actions, where the device action refers to the action performed by the smart home device. The first association pattern represents the association between the action execution condition and the device action. The scenario pattern represents the user's behavior scenario. For any of the aforementioned behavior patterns, a corresponding skill set is generated based on the behavior pattern; Upon detecting a trigger source, the embodied intelligent agent controls the corresponding smart home device to perform the corresponding action based on the generated skill packages.

[0005] In this embodiment, by acquiring the first behavior sequence formed by various interaction events between the user and smart home devices within a first time period, and by performing multi-dimensional mining on the first behavior sequence, it is possible to automatically mine high-order behavior patterns that include multiple device actions, action execution conditions, and user behavior scenarios that have time regularity and temporal relationship. This allows for the understanding of the user's complex behavior. Based on these high-order behavior patterns, skill packages adapted to the user's lifestyle can be automatically generated. When a trigger source is detected, the embodied intelligent agent controls the corresponding smart home devices to perform corresponding actions according to the generated skill packages, enabling the embodied intelligent agent to achieve adaptive personalized smart home services.

[0006] In some embodiments of the first aspect, the periodic pattern includes periodic sequence patterns and / or periodic association patterns; the mining method of the periodic sequence pattern includes: For the second behavior sequence of each day in the first behavior sequence, the second behavior sequence is grouped according to different second time periods and date types to obtain each first group; Extract frequently occurring first action subsequences within each of the first groups, and determine the first action subsequences as the second sequence pattern; If the second sequence pattern appears stably within N consecutive observation periods, then the second sequence pattern is determined as the periodic sequence pattern, where N is an integer greater than 1; The methods for mining the periodic association patterns include: The first behavior sequence is organized into transactions according to a fixed time window, and a time tag is attached to each transaction. The time tag includes the time period corresponding to the transaction and the date type. Transactions with the same time tag are grouped into the same group to obtain each second group; The association between the action execution conditions and the device actions is mined from each of the second groups, and the association between the action execution conditions and the device actions mined from each of the second groups is determined as a second association pattern; If the support of the second correlation pattern remains stable over M observation periods and is concentrated in a certain time period, then the second correlation pattern is determined as the periodic correlation pattern, where M is an integer greater than 1.

[0007] In some embodiments of the first aspect, the method for mining the first association pattern includes: When the action execution condition is another device action, for the third time period in the first behavior sequence, the linkage relationship between different device actions is mined to obtain the inter-device association pattern; When the action execution condition is an environmental context condition, for the fourth time period in the first behavior sequence, the relationship between the environmental context condition and the device action is mined to obtain the context and device association pattern; When the action execution condition includes both another device state and the environmental context condition, for the fifth time period in the first behavior sequence, the correlation between the other device state and the environmental context condition and the device action is mined to obtain a first hybrid mode. The other device state refers to the operating state of another smart home device. When the action execution conditions simultaneously include the other device action and the environmental context conditions, for the sixth time period in the first behavior sequence, the correlation between the other device action and the environmental context conditions and the device action is mined to obtain the second hybrid mode; At least one of the device association mode, the context and device association mode, the first hybrid mode, and the second hybrid mode is the first association mode.

[0008] In some embodiments of the first aspect, the method for mining the first sequence pattern includes: For the seventh time period in the first behavior sequence, frequently occurring second action subsequences are extracted, the average time interval between each device action in the second action subsequence is recorded, and the second action subsequence is determined as the first sequence pattern.

[0009] In some embodiments of the first aspect, the method for mining the scene pattern includes: Divide the daily second behavior sequence in the first behavior sequence into multiple behavior segments; The multiple behavioral segments are vectorized to obtain multiple vectors; Clustering is performed on multiple vectors to obtain the behavioral scenario, and the behavioral scenario is determined as the scenario pattern.

[0010] In some embodiments of the first aspect, generating a corresponding skill set based on any of the said behavioral patterns includes: The behavior pattern is input into the first model to perform intent recognition on the behavior pattern and generate a natural language description, which is used to explain the user intent represented by the behavior pattern. The skill set is generated based on the behavioral patterns and the natural language descriptions.

[0011] In some embodiments of the first aspect, after generating the skill pack, the method further includes: Based on the natural language description, a semantic description for the skill package is generated; The skill pack is pushed to the user, along with the semantic description; In response to the confirmation command for the skill pack, the skill pack is marked as activated. Alternatively, in response to a denial instruction for the skill pack, the skill pack may be marked as pending or obsolete.

[0012] In some embodiments of the first aspect, upon detecting a trigger source, the embodied agent controls the corresponding smart home device to perform corresponding actions based on the generated skill packs, including: Upon detecting a trigger source, the embodied agent determines a target skill pack that matches the trigger source from among the generated skill packs; The embodied intelligent agent triggers the target skill package to control the corresponding smart home device to perform the corresponding action.

[0013] In some embodiments of the first aspect, prior to detecting the trigger source, the method further includes: When the embodied agent is activated, the embodied agent scans the skill library to read the metadata in each skill package and constructs a skill index table based on the metadata. The skill library includes at least one of the skill packages. The embodied intelligent agent detects the existence of a trigger source based on the skill index table and obtains the detection result; When a trigger source is detected, the embodied agent determines a target skill pack matching the trigger source from the generated skill packs, including: If the detection result indicates the presence of a trigger source, then in response to the trigger source, the embodied agent retrieves the skill library via vector and determines the target skill pack that matches the trigger source.

[0014] In some embodiments of the first aspect, the skill index table includes at least one of time-triggered conditions, sequence-triggered conditions, state-triggered conditions, and natural language descriptions for each skill package, wherein the natural language descriptions are used to interpret the user intent represented by the corresponding behavioral pattern; the embodied agent detects the existence of a trigger source based on the skill index table and obtains a detection result, including: The embodied intelligent agent periodically detects whether the first current time matches any of the time triggering conditions in the skill index table. If the first current time matches any of the time triggering conditions, it is determined that there is a detection result of a trigger source. And / or, when subscribing to device state change events, the embodied agent detects whether the device state change event matches any of the sequence trigger conditions or any of the state trigger conditions in the skill index table. If the device state change event matches any of the sequence trigger conditions or any of the state trigger conditions, then a detection result indicating the existence of a trigger source is determined. And / or, in response to a user instruction, the embodied agent detects whether the user instruction matches any of the natural language descriptions in the skill index table; if the user instruction matches the natural language description, then a detection result indicating the existence of a trigger source is determined.

[0015] In some embodiments of the first aspect, when there are multiple target skill packs, the embodied agent triggers the target skill packs to control the corresponding smart home devices to perform corresponding actions, including: The embodied intelligent agent detects whether there is a skill conflict between at least two of the multiple target skill packages, and determines the conflict relationship between the at least two target skill packages if there is a skill conflict between them. The skill conflict includes at least one of resource conflict, logical conflict and intent conflict. The resource conflict refers to at least two target skill packages using the same smart home device and the device actions are mutually exclusive. The logical conflict refers to at least two target skill packages having semantically contradictory execution results. The intent conflict refers to at least two target skill packages reflecting contradictory user intents. The embodied intelligent agent generates prompt words based on the task objective, the candidate skill list, the conflict relationship, the current environmental context, and the user's recent feedback. The candidate skill list includes the skill names, skill descriptions, triggering conditions, execution plans, and triggering priorities of multiple target skill packages. The user's recent feedback represents the user's acceptance or rejection of multiple target skill packages within an eighth time period. The end time of the eighth time period is earlier than the second current time. The embodied intelligent agent inputs the prompt words into the second large model to perform skill intent understanding on multiple target skill packs through the second large model, and obtains a decision result. The decision result includes the set of skill packs to be executed and the execution order of each skill pack in the skill pack set. The embodied intelligent agent parses the execution plan of each skill package in the skill package set into skill execution instructions, and controls the corresponding smart home device to execute the corresponding skill execution instructions according to the execution order.

[0016] In some embodiments of the first aspect, the ruling result further includes decision reasons, including reasons for selecting the skill pack set and reasons for executing each skill pack in the skill pack set in the execution order; after obtaining the ruling result, it further includes: The embodied intelligent agent presents the rationale for the decision to the user.

[0017] In some embodiments of the first aspect, after the embodied agent triggers the target skill pack, the method further includes: Record user behavior after the target skill pack is triggered; If the user behavior is a corrective behavior, and the corrective behavior for the target skill pack occurs frequently, then the target skill pack is deleted. The corrective behavior refers to the behavior used to correct the device action corresponding to the target skill pack.

[0018] In some embodiments of the first aspect, after generating the corresponding skill pack, the method further includes: The system continuously monitors recent user behavior. If, based on the user's recent behavior, it detects that the behavior pattern corresponding to a certain skill pack is no longer significant, then the skill pack is marked as expired, and the trigger priority of the skill pack is reduced. Alternatively, the system may ask the user whether to disable a certain skill pack. The user's recent behavior refers to the user's interaction events with the smart home device during the ninth time period, and the end time of the ninth time period is earlier than the third current time.

[0019] In some embodiments of the first aspect, obtaining a first behavioral sequence formed by various interaction events between the user and smart home devices within a first time period includes: The interactive events are cleaned and aligned to obtain the processed interactive events. The processed interactive events are sorted by time to obtain the structured first behavior sequence.

[0020] Secondly, embodiments of this application provide a skill generation apparatus for an embodied intelligent agent, comprising: The sequence acquisition module is used to acquire the first behavioral sequence formed by the various interaction events between the user and smart home devices within the first time period; The pattern mining module is used to perform multi-dimensional mining on the first behavior sequence to mine multiple behavior patterns. The multiple behavior patterns include at least two of the following: periodic pattern, first sequence pattern, first association pattern, and scenario pattern. The periodic pattern represents a user behavior pattern with a time regularity. The first sequence pattern represents the temporal relationship between different device actions. The device action refers to the action performed by the smart home device. The first association pattern represents the association between the action execution condition and the device action. The scenario pattern represents the user's behavior scenario. The skill generation module is used to generate a corresponding skill pack based on any of the aforementioned behavior patterns. An embodied intelligent agent is used to control the corresponding smart home device to perform corresponding actions based on the generated skill packages when a trigger source is detected.

[0021] Thirdly, embodiments of this application provide a control device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the control device implements the skill generation method as described in any one of the first aspects above.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a computer, implements the skill generation method as described in any one of the first aspects above.

[0023] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when run, causes the skill generation method as described in any one of the first aspects above to be executed.

[0024] In a sixth aspect, embodiments of this application provide a smart home control system, including a control device and at least one smart home device, wherein the control device performs the skill generation method as described in any one of the first aspects above.

[0025] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of an application scenario of the smart home control system provided in this application embodiment; Figure 2 This is an overall architecture diagram of the smart home control system provided in the embodiments of this application; Figure 3 This is a first flowchart illustrating the skill generation method for embodied intelligent agents provided in an embodiment of this application; Figure 4 This is a second flowchart illustrating the skill generation method for embodied intelligent agents provided in an embodiment of this application; Figure 5 This is a third flowchart illustrating the skill generation method for embodied intelligent agents provided in an embodiment of this application; Figure 6 This is a fourth flowchart illustrating the skill generation method for embodied intelligent agents provided in this application embodiment; Figure 7 This is a fifth flowchart illustrating the skill generation method for embodied intelligent agents provided in this application embodiment; Figure 8 This is a sixth flowchart illustrating the skill generation method for embodied intelligent agents provided in this application embodiment; Figure 9 This is a seventh flowchart illustrating the skill generation method for embodied intelligent agents provided in this application embodiment; Figure 10 This is a schematic diagram of the structure of the skill generation device for embodied intelligent agents provided in the embodiments of this application; Figure 11 This is a schematic diagram of the structure of the control device provided in the embodiments of this application. Detailed Implementation

[0028] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0029] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0030] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0031] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0033] "Automation based on simple statistics" solutions typically collect data on the on / off times of smart home devices. When a certain action of a smart home device occurs more frequently than a threshold within a specific time period, a suggestion is pushed to the user. After user confirmation, a fixed automation rule is generated. However, the above solutions have the following drawbacks: 1. The single dimension of habit mining makes it impossible to understand complex behaviors: Based only on the time frequency statistics of a single device and a single action, it is impossible to capture the sequential patterns in user behavior (such as the movie-watching process of "turning on the lights → turning on the TV → drawing the curtains"), conditional associations (such as turning on the air conditioner when the room temperature is below 18°C ​​and it is morning) and complex habits of multi-device collaboration.

[0034] 2. The generated automation rules are rigid and lack dynamic adaptability: The generated automation rules are static "IF-THEN" triggers and cannot adaptively adjust according to short-term changes in user behavior or environmental context. For example, a user is used to turning off the lights at 22:00 every night, but if they work overtime and come home late one day, the system will still turn off the lights at 22:00, causing a conflict in user experience.

[0035] 3. Lack of semantic understanding and interpretability: The system cannot understand the intent behind user behavior (e.g., "dimming the lights" may be for "watching a movie" or "helping to sleep"), and cannot explain to the user why a certain automated rule is recommended.

[0036] 4. Poor skill reuse and scalability: The generated automated rules are usually stored in the rule engine in a hard-coded manner, which cannot be dynamically called and combined by the intelligent assistant in a standardized and modular way, thus limiting the system's flexibility and scalability in complex tasks (such as "I'm going out" triggering a series of related actions).

[0037] Based on this, the embodiments of this application introduce a multi-dimensional behavior pattern mining and semantic-driven dynamic skill package generation mechanism, enabling the smart home control system to actively observe, understand and adapt to the user's living habits, automatically generate modular exclusive skill packages, and realize adaptive personalized services in daily interactions. This solves the problems of poor personalized experience, weak dynamic adaptability, insufficient interpretability, and poor skill reuse and scalability caused by the single dimension of habit mining, rigid rules, lack of semantic understanding, and hard-coded storage in smart home control systems.

[0038] This application's embodiments construct a closed-loop adaptive system of "behavior perception - pattern mining - skill encapsulation - dynamic invocation", specifically embodied in: 1. Multi-dimensional user behavior pattern mining engine: It is not limited to the time frequency statistics of a single device, but combines time patterns, temporal relationships, correlations and behavioral scenarios to automatically discover high-level behavior patterns with multiple devices, multiple actions and conditions from the interaction events between users and smart home devices.

[0039] 2. Semantic-driven automatic generation of executable skills: The discovered behavioral patterns are semantically summarized and intent-understood through a large model, and automatically packaged into independent skill packages (including metadata, triggering conditions, execution plans, etc.) according to a standardized Agent Skills structure, so that user habits are transformed from "data" into "modules that can be called by embodied intelligent agents".

[0040] 3. Dynamic skill invocation mechanism based on metadata preloading and on-demand loading of execution plans: The embodied agent only preloads lightweight metadata (name, description, trigger type) of all custom skill packages at runtime. When the trigger condition is met or the user command is relevant, the complete skill content is dynamically loaded and executed, realizing the unity of efficient use of context and unlimited expansion of the skill library.

[0041] 4. User habit evolution tracking and adaptive skill pack updates: The system continuously monitors user behavior and the execution effect of generated skill packs. When a significant change in user habits is detected, the system automatically triggers skill pack updates or abandonment to ensure that the generated skill packs always match the user's current lifestyle.

[0042] It should be understood that the control device in this application may also be referred to as a home hub device, used to connect and control various smart home devices.

[0043] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0044] Please see Figure 1 , Figure 1This illustration shows an application scenario diagram of the smart home control system provided in an embodiment of this application. The smart home control system may include a control device 101 and a smart home device 102, etc. The control device 101 and the smart home device 102 can communicate with each other via wired or wireless means.

[0045] The control device 101 carries the core logic of this application and is responsible for learning and decision-making. For example, it continuously learns user habits, discovers behavioral patterns, generates and manages a skill library, listens for triggering conditions, and makes decisions at appropriate times, issuing control commands to specific smart home devices 102, etc.

[0046] It should be understood that this application does not limit the specific type of control device 101, which can be selected according to actual needs. For example, control device 101 can be at least one of devices such as smart speaker, smart home gateway, home server, high-performance router, and mobile phone.

[0047] In some embodiments, an embodied intelligent agent may run on the control device 101, which is the core program on the control device 101.

[0048] Among them, an embodied intelligent agent can refer to an intelligent agent equipped with artificial intelligence and possessing the ability to perceive and interact with the outside world. It can interact with the environment in real time through perception and interaction, and can perform various tasks in virtual or real physical environments.

[0049] In this application, the embodied intelligent agent is mainly responsible for understanding user instructions and coordinating device control. For details, please refer to the detailed description of the role of the embodied intelligent agent in the following method embodiments.

[0050] The number of smart home devices 102 can be one or more (i.e., at least two). Smart home devices 102 serve as the carrier of user behavior habits and are also the objects of the final control actions executed by the smart home control system. For example, users can turn smart home devices 102 on, off, and adjust them via an app, voice, or manual operation, thereby achieving interaction with the smart home devices 102.

[0051] It should be understood that this application does not limit the specific type of smart home device 102, which can be selected according to actual needs. For example, smart home device 102 can be a living room light, bedroom light, television, bedroom air conditioner, thermometer and hygrometer, smart curtain motor (such as a curtain robot or electric curtain), coffee machine, robot vacuum cleaner, companion robot, etc. Figure 1 Three smart home devices were shown: a living room light, a television, and a bedroom air conditioner.

[0052] like Figure 2The diagram shows the overall architecture of the smart home control system provided in this embodiment. It illustrates multiple processes, including data acquisition and preprocessing, behavior pattern mining, semantic-driven automatic skill generation, dynamic invocation mechanism of embodied intelligent agents, and skill evolution and adaptive updates. By continuously learning user interaction behavior, the aforementioned smart home control system automatically generates and dynamically invokes personalized skill packages, achieving adaptive control of the home environment.

[0053] Data acquisition and preprocessing may include: collecting all interaction events between the user and the smart home device 102 within a certain time period, and cleaning and aligning them to form a structured sequence of behaviors.

[0054] Behavioral pattern mining can include: running offline or near-line, periodically (e.g., daily) mining of accumulated behavioral sequences in multiple dimensions to identify users' potential habits.

[0055] Semantic-driven automatic skill generation can include: once the discovered behavioral patterns meet the preset confidence and support thresholds, the skill generation phase begins.

[0056] The skill generation phase can include multiple processes such as pattern semanticization, skill encapsulation, user confirmation, and activation. Pattern semanticization can include inputting the mined behavioral patterns (e.g., a sequence of actions) into a large model, requiring the model to generate a natural language description explaining the user intent that the behavioral pattern may represent. Skill encapsulation can include converting these abstract behavioral patterns into standardized skill packages that can be dynamically invoked by embodied agents after identification. Its core lies in automatically generating metadata, triggering conditions, and execution plans for the skill packages based on different pattern types, and storing them in a skill library according to an agreed-upon directory structure. User confirmation and activation: Newly generated skill package suggestions can be pushed to users via an app or voice, along with a semantic description (e.g., "I've noticed you often adjust the lights when watching TV at night; would you like me to automatically perform this operation for you?"). Once the user confirms, the skill package is marked "activated"; otherwise, it is marked "pending" or "discarded."

[0057] The embodied agent's dynamic invocation mechanism can include multiple processes such as skill library index loading, multi-source trigger listening, and dynamic skill loading and execution. The skill library index includes: upon startup, the embodied agent scans the skill library (including system-preset skill packages and user-defined skill packages), reads the metadata (skill name, skill description, trigger type, etc.) from each skill package, constructs a lightweight skill index table, and keeps it resident in memory. Multi-source trigger listening includes: the embodied agent can continuously listen for three types of trigger sources: time-based triggers, event-based triggers, and user command triggers. Dynamic skill loading and execution: when the trigger conditions for a skill package are met, the embodied agent can dynamically load the complete content (execution plan) of that skill package from the skill library, parse the execution steps, and call the corresponding device control interface to execute it. Logs can be recorded during execution for skill effect evaluation and evolution.

[0058] Skill evolution and adaptive updates can include multiple processes such as performance evaluation, habit drift detection, and new skill iteration generation. Performance evaluation can include recording user behavior after each skill pack is triggered. For example, after a skill automatically turns on the living room light, does the user immediately turn it off manually? If "user correction" events occur frequently, such as more than twice, the skill is deleted. Habit drift detection can include continuously monitoring recent user behavior. If a behavioral pattern corresponding to a generated skill pack is no longer significant (e.g., the user hasn't watched TV at night for the past two weeks), the skill pack is marked as "expired," its trigger priority is reduced, or the user is asked to confirm whether to deactivate it. New skill iteration generation can include continuously running the above behavioral pattern mining process as new interaction events accumulate, constantly generating new skill packs or updating the parameters of existing skill packs (e.g., time windows, action subsequences, etc.).

[0059] In this embodiment, the control device 101 acquires a first behavior sequence formed by various interaction events between the user and the smart home device 102 within a first time period, and performs multi-dimensional mining on the first behavior sequence. This allows it to automatically mine high-order behavior patterns that include multiple device actions, action execution conditions, and user behavior scenarios that have time regularity and temporal relationships. This enables the control device to understand the user's complex behavior. Based on these high-order behavior patterns, it can automatically generate skill packages that adapt to the user's lifestyle. When a trigger source is detected, the embodied intelligent agent controls the corresponding smart home device 102 to perform corresponding actions according to the generated skill packages. This enables the embodied intelligent agent to provide adaptive personalized smart home services.

[0060] Please see Figure 3 , Figure 3 This illustration shows a flowchart of a skill generation method for embodied intelligent agents provided in an embodiment of this application. This skill generation method can be applied to, for example... Figure 1The control device 101 in the smart home control system shown. As an example, and not a limitation, the skill generation method includes the following steps: Step 301: Obtain the first behavioral sequence formed by the interaction events between the user and the smart home device within the first time period.

[0061] It should be understood that the number of users in step 301 can be one or more, depending on the actual number of users in the smart home scenario. Similarly, the number of smart home devices in step 301 can be one or more, depending on the actual number of smart home devices in the smart home scenario.

[0062] In this embodiment, when there are multiple users, for any user, the first behavioral sequence formed by the interaction events between the user and the smart home device in the first time period is obtained. Based on this, a personalized skill package is generated for the user. When a trigger source is detected, the embodied intelligent agent can combine the current user identification information (such as identifying the user's identity through voice commands or determining the main user of the current activity through sensors) to call the skill package corresponding to the user, thereby realizing a truly personalized service.

[0063] Optionally, the first time period can be set according to actual needs or experience. This application does not limit the specific length of the first time period. For example, the first time period can be the past month or the past 7 days, etc.

[0064] As an example, and not a limitation, the smart home scenario is user A's home, which also includes users B, C, and D. Control devices will link and control smart home devices in user A's home, such as living room lights, bedroom lights, television, bedroom air conditioner, thermometer / hygrometer, smart curtain motor, coffee maker, and robot vacuum cleaner. In this smart home scenario, the system can obtain the first behavioral sequence of all interactions between user A and all smart home devices over the past month, generating a personalized skill pack for user A; the same applies to user B, user C, and user D.

[0065] Optionally, the data types included in the interaction events can be set according to actual needs, and this application does not limit this. For example, interaction events may include data types such as timestamps, user IDs (if there are multiple users), device IDs, device actions (e.g., "on", "off", "adjust to 30%", etc.), device status values, and environmental context (e.g., indoor temperature, light intensity, whether at home, etc.). Device status values ​​can refer to the specific quantitative state or mode of a smart home device after performing a certain action.

[0066] Step 302: Perform multi-dimensional mining on the first behavior sequence to uncover multiple behavior patterns.

[0067] The various behavioral patterns can include at least two of the following: periodic patterns, first sequence patterns, first association patterns, and scenario patterns. Periodic patterns represent user behavior patterns with time regularity; first sequence patterns represent the temporal relationship between different device actions (device actions refer to actions performed by smart home devices); first association patterns represent the association between action execution conditions and device actions; and scenario patterns represent user behavior scenarios.

[0068] In this embodiment, by performing multi-dimensional mining on each user's first behavioral sequence, the system can autonomously learn the user's complex and ever-changing lifestyle habits, automatically discover high-level behavioral patterns involving multiple devices, multiple actions, and conditions, and dynamically generate personalized skill packages. This transforms the smart home control system from a "passive execution tool" into a "proactive butler who understands the user," greatly enhancing the user's living experience. Furthermore, by integrating multi-dimensional mining of periodicity, sequence, association, and behavioral scenarios, the discovered habits are no longer limited to simple timing but cover real-life scenarios such as multi-step processes and conditional triggers, significantly improving the accuracy and richness of habit mining and making skill generation more practical.

[0069] Step 303: For any given behavior pattern, generate the corresponding skill pack based on the behavior pattern.

[0070] In this embodiment, by introducing a time dimension and mining patterns with stable time regularity (i.e., periodic patterns) in user behavior, it is possible to generate time-aware skill packages. This allows for more precise control of smart home devices that aligns with users' actual lifestyles, solving the problems of rigid rules, inability to distinguish between weekdays and weekends, or neglect of changes in habits over time in existing technologies. For example, a skill package could be generated that says "Automatically turn on the bedroom light and start the coffee machine at 7:30 am on weekdays," instead of simply "Execute at 7:30 am every day."

[0071] In this embodiment, by mining the first sequence pattern, a fixed sequential order between device actions can be discovered, generating a multi-step linked skill package, thus solving the problem that existing technologies cannot capture complex multi-step behavior patterns. For example, the movie-watching preparation process of "turning on the living room lights → turning on the TV → dimming the living room lights" can be mined and a multi-step linked skill package can be generated.

[0072] In this embodiment, by mining the first association pattern, it is possible to discover the statistical regularity of "under what conditions users tend to perform what actions", generate intelligent condition-triggered skill packages, and solve the problem that existing technologies cannot understand the conditional logic and contextual dependencies behind behavior.

[0073] In this embodiment, by mining scene patterns, typical scenarios composed of multiple behaviors that users frequently engage in can be identified. For example, the "activity flow after returning home" can be identified (which may include turning on the living room lights, turning on the air conditioner, playing music, etc.). Based on this, a composite skill package for the overall scenario can be generated. When the event of a user "returning home" is detected, all related operations in that scenario can be executed with one click or automatically, providing a "one-stop" intelligent service, rather than a fragmented response to individual habits.

[0074] Step 304: When a trigger source is detected, the embodied intelligent agent controls the corresponding smart home devices to perform corresponding actions based on the generated skill packs.

[0075] The trigger source can refer to a condition or signal that can activate a generated skill pack.

[0076] In this embodiment, the embodied intelligent agent automatically and accurately invokes and executes the corresponding personalized skill package when a trigger source is detected, driving smart home devices to complete a series of actions. It can seamlessly and in real time transform user habits mined offline or near-line into proactive services that fit the user's current situation, thereby changing the traditional interaction mode of smart home control systems. Without requiring users to manually configure or find automation rules, it can proactively understand and meet their needs, realizing a truly sustainable and evolving personalized home experience.

[0077] In this embodiment, by acquiring the first behavior sequence formed by various interaction events between the user and smart home devices within a first time period, and by performing multi-dimensional mining on the first behavior sequence, it is possible to automatically mine high-order behavior patterns that include multiple device actions, action execution conditions, and user behavior scenarios that have time regularity and temporal relationship. This allows for the understanding of the user's complex behavior. Based on these high-order behavior patterns, skill packages adapted to the user's lifestyle can be automatically generated. When a trigger source is detected, the embodied intelligent agent controls the corresponding smart home devices to perform corresponding actions according to the generated skill packages, enabling the embodied intelligent agent to achieve adaptive personalized smart home services.

[0078] In some embodiments of this application, the acquisition of the first behavioral sequence formed by the various interaction events between the user and the smart home device within a first time period includes: Each interactive event is cleaned and aligned to obtain the processed interactive events. The processed interactive events are sorted by time to obtain a structured first behavior sequence.

[0079] Cleaning interactive events can refer to identifying and processing noise, errors, invalid or redundant information in interactive events. For example, deleting duplicate reported events caused by network jitter (such as receiving multiple identical "lights are on" status messages consecutively); correcting obviously erroneous timestamps (such as future times or times that are too far in the past).

[0080] Aligning interactive events can refer to unifying data from different sources into a consistent benchmark and framework in terms of time and logic. For example, synchronizing the timestamps of different smart home devices to a unified system time can eliminate errors caused by clock asynchrony between different smart home devices; mapping the same state reported by different smart home devices (such as "lights on", "lights on", "ON") to a unified enumeration value (such as "on").

[0081] Sort the processed interactive events by time. This means arranging the processed interactive events according to their timestamps (i.e., the timestamps included in the interactive event) from earliest to latest. Based on this, a coherent sequence of behaviors (i.e., the first behavior sequence) can be formed in chronological order. For example, the first behavior sequence may include multiple processed interactive events such as [2025-03-02 21:03:15, user1, living room light, on, brightness 80%, at home] and [2025-03-02 21:05:10, user1, TV, on, volume 30%, at home].

[0082] In this embodiment, cleaning and aligning each interaction event can remove noise, correct errors, and unify the format, ensuring that the subsequent mining of real and effective user behavior patterns, rather than pseudo-patterns caused by data errors. Furthermore, sorting by time can transform the scattered processed interaction events into a time sequence with a clear chronological relationship, thereby providing a standardized input format for subsequent behavior pattern mining and improving system processing efficiency.

[0083] In some embodiments of this application, periodic patterns include periodic sequence patterns and / or periodic association patterns. By mining periodic sequence patterns and / or periodic association patterns, the system not only grasps the user's single-point periodic events, but also understands the complex behavioral processes composed of multiple steps and the time-stable association habits, providing structured input for the subsequent generation of multi-step linkage skills with time awareness.

[0084] In some embodiments of this application, such as Figure 4 As shown, the method for mining periodic sequence patterns may include steps 401 to 403.

[0085] Step 401: For the daily second-row sequence in the first-row sequence, group the second-row sequence according to different second time periods and date types to obtain each first group.

[0086] It should be understood that the number of first groups is determined by the number of different second time periods and the number of date types. For any first group, all interactive events within that first group belong to the same second time period and date type.

[0087] As an example, and not a limitation, the second time period can include three time periods: morning, afternoon, and evening; or it can include two time periods: morning and afternoon. The date type can include weekdays and weekends. After grouping the second row of each day in the first row sequence according to these three time periods and two date types (weekdays and weekends), six different first groups are obtained: weekday-morning, weekday-afternoon, weekday-evening, weekend-morning, weekend-afternoon, and weekend-evening.

[0088] Step 402: Extract frequently occurring first action subsequences within each first group and determine the first action subsequences as the second sequence pattern.

[0089] In some embodiments, for each first group, a sequence pattern mining algorithm such as PrefixSpan or CM-SPAM can be used to extract the frequently occurring first action subsequence within that first group. It should be understood that other sequence pattern mining algorithms can also be used, and this application does not limit the specific type of sequence pattern mining algorithm.

[0090] In other embodiments, for each first group, the occurrence count or frequency of a certain action subsequence can be counted. If the occurrence count reaches a first threshold or the frequency reaches a first frequency threshold, then the action subsequence can be determined to be a frequently occurring first action subsequence within the first group. Optionally, the first threshold or the first frequency threshold can be set according to actual needs or empirical values. This application does not limit the specific value of the first threshold or the first frequency threshold.

[0091] It should be understood that the second sequence pattern is a sequence pattern within the same time period of the cycle. The first sequence pattern is a sequence pattern within any time period.

[0092] Step 403: If the second sequence pattern appears stably within N consecutive observation periods, then the second sequence pattern is determined to be a periodic sequence pattern, where N is an integer greater than 1.

[0093] Optionally, N can be set according to actual needs or empirical values; this application does not limit the specific value of N. For example, N can be 4.

[0094] The observation period can refer to a statistical event window of fixed duration used to detect whether a certain behavioral pattern (such as a second sequence pattern or a second association pattern) repeats; it is the basic time unit for determining periodicity. Optionally, the length of the observation period can be set according to actual needs or empirical values; this application does not impose any limitations on this. For example, one week can be used as an observation period.

[0095] In some embodiments, the number of days the second sequence pattern appears within N observation periods can be counted, and the percentage of days appearing can be calculated. If the percentage of days appearing is greater than or equal to a percentage threshold, it is determined that the second sequence pattern has appeared stably within N consecutive observation periods. If the percentage of days appearing is less than the percentage threshold, it is determined that the second sequence pattern has not appeared stably within N consecutive observation periods. Optionally, the percentage threshold can be set according to actual needs or empirical values. This application does not limit the specific value of the percentage threshold.

[0096] For example, if the percentage threshold is 70%, and the action subsequence that frequently occurs on weekday mornings is [turn on the bedroom light, open the living room curtains, start the coffee machine], and its percentage of occurrence days in each week for four consecutive weeks is greater than or equal to 70%, then this action subsequence can be identified as a periodic sequence pattern of "weekday morning wake-up routine".

[0097] In this embodiment, by grouping according to different second time periods and date types, frequently occurring action subsequences are mined within each group, and periodic stability tests are performed on the action subsequences. This allows for the precise selection of multi-step operation processes that recur stably in specific life scenarios from the interaction time. This ensures that the skill packages automatically generated by the system correspond to the user's real, stable, long-term life habits, rather than temporary or accidental behavioral fragments. Ultimately, this lays a reliable foundation for the smart home control system to achieve high-quality personalized adaptive services that "automatically execute the correct process at the right time".

[0098] In some embodiments of this application, such as Figure 5 As shown, the method for mining periodic association patterns may include steps 501 to 504.

[0099] Step 501: Organize the first row sequence into transactions according to a fixed time window, and attach a time label to each transaction.

[0100] The timestamp includes the time period and date type corresponding to the transaction.

[0101] Optionally, a fixed time window can be set based on actual needs or experience. For example, a fixed time window can be a day or every time period.

[0102] The aforementioned transaction can refer to a collection of user interaction events with smart home devices that occur within a fixed time window. For example, all interaction events that occur on the day "March 2, 2025" (such as turning on the lights, turning off the air conditioner, turning on the TV, etc.) together constitute a transaction.

[0103] Step 502: Group transactions with the same timestamp into the same group to obtain each second group.

[0104] As an example, and not a limitation, the time period corresponding to a transaction can be one of three time periods: morning, afternoon, and evening. The date type corresponding to the transaction can include weekdays and weekends. After grouping transactions with the same time label into the same group, six different second groups can be obtained. These six different second groups are: weekday-morning, weekday-afternoon, weekday-evening, weekend-morning, weekend-afternoon, and weekend-evening.

[0105] Step 503: Extract the correlation between action execution conditions and equipment actions from each second group, and determine the correlation between action execution conditions and equipment actions extracted from each second group as the second correlation pattern.

[0106] It should be understood that the action execution conditions in step 503 can be the same as those in the first association pattern mining. Specifically, the action execution conditions in step 503 may include at least one of the following: another device action, environmental context conditions, simultaneously including another device state and environmental context conditions, simultaneously including another device action and the environmental context conditions. Based on this, for each second group, the method for mining the second association pattern from that second group can refer to the mining method for the first association pattern described below. For example, association pattern mining algorithms such as Apriori or FP-Growth can be used to mine the second association pattern from each second group. It should be understood that other association pattern mining algorithms (such as Hidden Markov Models, Recurrent Neural Networks, Transformers, etc.) can also be used, and this application does not limit the specific type of association pattern mining algorithm.

[0107] It should be understood that the second correlation pattern is a correlation pattern within the same time period of the cycle. The first correlation pattern is a correlation pattern within any time period.

[0108] Step 504: If the support of the second correlation pattern remains stable over M consecutive observation periods and is concentrated in a certain time period, then the second correlation pattern is determined as a periodic correlation pattern, where M is an integer greater than 1.

[0109] Optionally, M can be set according to actual needs or empirical values. This application does not limit the specific value of M. For example, M is 4.

[0110] The support of the second association pattern within an observation period can be defined as the ratio of the number of transactions containing the second association pattern to the total number of transactions within that observation period.

[0111] In some embodiments, it can be detected whether the difference between the maximum and minimum support values ​​calculated for the second association pattern over M consecutive observation periods is less than a fluctuation threshold. If it is less than the fluctuation threshold, the support of the second association pattern over the M consecutive observation periods is determined to be stable. If it is greater than or equal to the fluctuation value, the support of the second association pattern over the M consecutive observation periods is determined to be unstable. Optionally, the fluctuation threshold can be set according to actual needs or empirical values. This application does not limit the specific value of the fluctuation threshold. For example, the fluctuation threshold is 5%.

[0112] It should be understood that, in addition to judging whether the support of the second correlation pattern remains stable over M consecutive observation periods by using the fluctuation threshold, other methods can also be used to judge whether the support of the second correlation pattern remains stable over M consecutive observation periods (for example, judging whether there is a downward or upward trend in the support over M consecutive observation periods; if there is a downward or upward trend, it is determined that the support of the second correlation pattern has not remained stable over M consecutive observation periods; if there is neither a downward trend nor an upward value, it is determined that the support of the second correlation pattern has remained stable over M consecutive observation periods). This application does not limit this.

[0113] In some embodiments, the transactions occurring in the first row sequence or the most recent multiple observation periods can be statistically analyzed for the second association pattern, and the time tag to which each transaction belongs can be recorded. The occurrence frequency of the second association pattern in different time periods can be calculated. If the maximum occurrence frequency in different time periods is greater than or equal to a frequency threshold, it can be determined that the second association pattern is concentrated in the time period with the maximum occurrence frequency; if the maximum occurrence frequency in different time periods is less than the frequency threshold, it can be determined that the second association pattern is not concentrated in a certain time period. Optionally, the frequency threshold can be set according to actual needs or empirical values. This application does not limit the specific value of the frequency threshold. For example, the frequency threshold is 60%.

[0114] For example, if the pattern {turn on the TV} → {dim the living room lights} is extracted from a certain second group and appears consistently on weekday evenings, it can be identified as a periodic association pattern of "preparing to watch a movie on weekday evenings"; if the pattern {bedroom temperature <18℃} → {turn on the air conditioner to heat} is extracted from a certain second group and appears consistently in the morning, it can be identified as a periodic association pattern of "cold morning habits".

[0115] In this embodiment, by organizing the first behavior sequence into transactions according to a fixed time window, attaching time tags and grouping them, and then mining the correlation between action execution conditions and device actions within the group, and performing cross-cycle support stability testing on the correlation, the accurate identification and solidification of user triggering habits that depend on specific environmental conditions or device states and have time regularity are achieved. This ensures that the final generated automated skill package is not based on accidental temporary correlations, but on long-term stable and reliable decision-making logic that closely matches specific life scenarios, thereby significantly improving the accuracy and practicality of condition-triggered intelligent services.

[0116] In some embodiments of this application, the method for mining the first association pattern may include: When the action execution condition is another device action, for the third time period in the first action sequence, the linkage relationship between different device actions is explored to obtain the inter-device association pattern; When the action execution condition is the environmental context condition, for the fourth time period in the first action sequence, the relationship between the environmental context condition and the device action is mined to obtain the context and device association pattern. When the action execution condition includes both the state of another device and the environmental context condition, for the fifth time period in the first action sequence, the relationship between the state of another device and the environmental context condition and the device action is mined to obtain the first hybrid mode. The state of another device refers to the operating state of another smart home device. When the action execution conditions simultaneously include another device action and environmental context conditions, for the sixth time period in the first behavior sequence, the other device action and environmental context conditions are mined, and the relationship between them and the device action is obtained to obtain the second hybrid mode. The first association mode is at least one of the following: inter-device association mode, context and device association mode, first hybrid mode and second hybrid mode.

[0117] Optionally, a third, fourth, fifth, and sixth time period can be set according to actual needs or experience. This application does not limit the specific length of the third, fourth, fifth, and sixth time periods.

[0118] It should be noted that the third, fourth, fifth, and sixth time periods mentioned above may be the same or different, and this application does not impose any restrictions on this.

[0119] Environmental context includes, but is not limited to, time period, temperature, light intensity, humidity, and whether the user is present.

[0120] In this embodiment, association pattern mining algorithms such as Apriori or FP-Growth can be used to mine inter-device association patterns, context-device association patterns, first mixed patterns, and second mixed patterns from the first row sequence. During mining, the first row sequence can be organized into transactions according to the third, fourth, fifth, and sixth time periods, and each transaction can be labeled with device status and environmental context. Then, the association rule mining algorithm is run to filter out patterns that meet at least one of the above four definitions from the corresponding transactions. It should be understood that other association pattern mining algorithms (such as Hidden Markov Models, Recurrent Neural Networks, Transformers, etc.) can also be used. This application does not limit the specific type of association pattern mining algorithm.

[0121] The device association pattern can take the form of {Device A Action} → {Device B Action}, reflecting the user's habit of operating one device followed by operating another. For example, {Turn on the TV} → {Dim the living room lights} (85% confidence level) indicates that the user is accustomed to dimming the lights when watching TV.

[0122] The context-device association pattern can take the form of {environmental context conditions} → {device actions}, reflecting the user's operating habits under specific environmental conditions. For example, {bedroom temperature < 18℃, time period = morning} → {turn on air conditioner for heating} (92% confidence level) indicates that the user habitually turns on the air conditioner on cold mornings.

[0123] The first hybrid mode can take the form of {another device state, environmental context conditions} → {device action}, reflecting the user's interactive operation habits under specific device states and environmental context conditions. For example, {time period = evening, TV is on} → {dimming living room lights} (confidence level 88%) indicates that the user is accustomed to dimming the lights when watching TV at night.

[0124] The second hybrid mode can take the form of {another device action, environmental context conditions} → {device action}, reflecting the user's interactive operation habits under specific device actions and environmental context conditions. For example, {time period = evening, turn on TV} → {dim the living room lights} (confidence level 88%) indicates that the user is accustomed to dimming the lights when watching TV at night.

[0125] In this embodiment, by mining association patterns at multiple levels such as device action linkage, environmental context conditions, device status and environmental context conditions, and device action and environmental context conditions, it is possible to comprehensively capture various "IF-THEN" decision-making habits of users in different scenarios. This provides a complete and structured rule foundation for generating accurate, flexible, and scenario-deeply understanding condition-triggered intelligent skill packages, solving the limitation of traditional methods that can only capture a single type of association.

[0126] In some embodiments of this application, the mining method for the first sequence pattern may include: For the seventh time period in the first action sequence, extract the frequently occurring second action subsequence, record the average time interval between each device action in the second action subsequence, and determine the second action subsequence as the first sequence pattern.

[0127] Optionally, a seventh time period can be set according to actual needs or experience. This application does not limit the specific length of the seventh time period.

[0128] In some embodiments, sequence pattern mining algorithms such as PrefixSpan or CM-SPAM can be used to extract frequently occurring action subsequences in the seventh time period of the first line sequence. For example, if the sequence "turn on the living room light → turn on the TV → dim the living room light" occurs frequently, it can be identified as a "movie preparation" sequence pattern.

[0129] It should be understood that other sequence pattern mining algorithms may also be used, and this application does not limit the specific type of sequence pattern mining algorithm.

[0130] In other embodiments, for the seventh time period in the first action sequence, the number of occurrences or frequency of a certain action subsequence can be counted. If the number of occurrences reaches a second threshold or the frequency of occurrence reaches a second frequency threshold, then the action subsequence can be determined to be a frequently occurring second action subsequence. Optionally, the second threshold or the second frequency threshold can be set according to actual needs or empirical values. This application does not limit the specific value of the second threshold or the second frequency threshold.

[0131] In this embodiment, recording the average time interval between each device action in the second action subsequence can be used to refine the triggering conditions of subsequent skill packs. For any second action subsequence, the time interval between each two adjacent device actions at each occurrence of the second action subsequence can be counted, and the average time interval between these two device actions can be calculated for all occurrences of the second action subsequence. This average value is the average time interval between these two device actions in the second action subsequence.

[0132] In this embodiment, by extracting frequently occurring action subsequences within the seventh time period and accurately recording the average time interval between actions of each device within them, stable and coherent multi-step operation processes can be automatically identified from the user's historical behavior, and their execution rhythm can be quantified. This provides a key basis for generating a refined skill package that can not only restore the correct action sequence but also simulate the real operation sequence, enabling subsequent automated services to seamlessly integrate into the user's original lifestyle rhythm and significantly improve the realism of automated execution of complex processes and user experience.

[0133] In some embodiments of this application, the methods for mining scene patterns include: Divide the daily second-behavior sequence in the first-behavior sequence into multiple behavior segments; Multiple behavioral segments are vectorized to obtain multiple vectors; Clustering multiple vectors yields behavioral scenarios, which are then defined as scenario patterns.

[0134] In this embodiment, the aforementioned multiple behavioral segments are divided by certain specific events. For example, the user's daily behavioral sequence is divided into multiple behavioral segments such as "going home," "leaving home," and "going to sleep" using events such as "returning home," "leaving home," and "going to sleep" as boundaries.

[0135] In this embodiment, after vectorizing multiple behavioral segments, DBSCAN or K-Means clustering algorithms can be used to discover typical behavioral scenarios. Examples include behavioral scenarios such as "activity flow after returning home on a weekday" and "leisure flow on a weekend morning."

[0136] In this embodiment, by dividing daily behavior sequences into meaningful segments, vectorizing and clustering them, the system can automatically abstract and summarize recurring typical life scenarios composed of multiple behaviors from users' scattered daily operations. This enables the system to go beyond understanding single actions or simple rules, grasp users' overall behavior patterns and life rhythms at a higher level, and provide key basis for generating composite intelligent skill packages for complex scenarios that can be triggered with one click. Ultimately, this achieves scenario-based intelligent services that are closer to users' real life needs.

[0137] In some embodiments of this application, after identifying multiple behavioral patterns, for any given behavioral pattern, it can be determined whether the confidence level of the behavioral pattern is greater than or equal to a confidence threshold, and whether the support level of the behavioral pattern is greater than or equal to a support threshold. If the confidence level and support level of the behavioral pattern are both greater than or equal to the confidence threshold, then the skill pack generation stage begins (i.e., the corresponding skill pack is generated). If the confidence level or support level of the behavioral pattern is less than the confidence threshold or less than the support threshold, then the skill pack generation stage does not begin (i.e., the corresponding skill pack is not generated). Optionally, the confidence threshold and support level can be set according to actual needs or empirical values. This application does not limit the range of values ​​for the confidence threshold and support threshold. For example, the confidence threshold is 70%, and the support threshold is 5%.

[0138] In some embodiments of this application, such as Figure 6 As shown, for any behavior pattern, generating a corresponding skill pack based on the behavior pattern may include steps 601 to 602.

[0139] Step 601: Input the behavior pattern into the first big model to perform intent recognition on the behavior pattern through the first big model and generate a natural language description. The natural language description is used to explain the user intent represented by the behavior pattern.

[0140] The primary model can be any large model capable of text generation and understanding, or it can be a combination of multiple specialized models (e.g., one for pattern semantics and another for skill pack code generation). This application does not limit the specific type of the primary model. For example, the primary model can be a Large Language Model (LLM), a Generative Pre-trained Transformer (GPT) series, etc.

[0141] For example, for the behavior pattern "turn on the living room light → turn on the TV → dim the living room light", the first major model can output: "Users are accustomed to turning on the lights first and then dimming them when watching TV at night to create a viewing atmosphere."

[0142] Step 602: Generate a skill pack based on behavioral patterns and natural language descriptions.

[0143] In this embodiment, behavioral patterns and natural language descriptions can be converted into standardized skill packages that can be dynamically invoked by embodied intelligent agents, facilitating reuse and combination. Specifically, for different behavioral patterns, metadata and execution plans for skill packages can be automatically generated based on the pattern type and natural language description of the behavioral pattern, and stored in the skill library according to an agreed directory structure.

[0144] The metadata of a skill pack can include skill name, skill description (i.e., natural language description), author, version, type, trigger conditions, etc. For example, skill name: Prepare for watching TV at night; skill description: Automatically adjust the living room lights and TV when preparing to watch TV at night according to user habits; author: automatically generated (based on user habits); version: 1.0; type: habit; trigger conditions: type: sequence trigger; conditions: device: living room light, action: turn on, time window: [19:00, 23:00].

[0145] An execution plan defines the sequence of actions to be performed when a skill pack is triggered, and may include conditional statements. For example, an execution plan may include the following steps: if the living room light status is 'off', then turn on the living room light to 50% and wait 5 seconds; if the TV status is 'off', then turn on the TV, wait 10 seconds, and then dim the living room light to 20%.

[0146] Optionally, the skill pack may also include a history log to record when the skill pack is triggered and executed, for subsequent evolutionary analysis.

[0147] It should be noted that this application does not impose any restrictions on the file structure of the skill package. For example, the file structure of the skill package is not limited to YAML format; it can also use any structured storage format such as JSON, XML, or database records. The execution plan is also not limited to parsing YAML; it can be directly stored as a lightweight script (such as Lua or Python) and executed by a sandbox environment to support more complex logic.

[0148] In this embodiment, by introducing a large language model to perform semantic intent recognition and natural language description generation on the mined behavioral patterns, the generation and invocation of skill packages can be injected with key "interpretability" and "semantic understanding" capabilities. This not only enables the system to understand the user's true intent behind complex behavioral patterns (e.g., distinguishing whether "dimming the lights" is for "watching a movie" or "helping to sleep"), but also generates standardized skill descriptions that can be understood and confirmed by the user and used for semantic matching by the embodied intelligent agent. This significantly improves the accuracy of skill package generation and the precision of the embodied intelligent agent in invoking skill packages in complex scenarios.

[0149] In some embodiments of this application, such as Figure 7 As shown, after generating the skill pack, steps 701 to 704 may also be included.

[0150] Step 701: Generate a semantic description for the skill pack based on natural language description.

[0151] Step 702: Push the skill pack to the user, along with a semantic description.

[0152] Step 703: In response to the confirmation instruction for the skill pack, mark the skill pack as activated.

[0153] Step 704: In response to the denial instruction for the skill pack, mark the skill pack as pending or discarded.

[0154] Optionally, the natural language description can be directly identified as a semantic description, or the natural language description can be adaptively modified before being identified as a semantic description.

[0155] In this embodiment, newly generated skill package suggestions can be pushed to users via an app or voice, along with a semantic description. After viewing the semantic description, users can send confirmation or denial commands to the control device. This allows for dynamic management of skill packages (activation, pending, or abandonment) based on user confirmation or denial feedback. This ensures that the skill packages ultimately deployed align with the user's current intentions and habits, effectively avoiding the risk that automatically generated skill packages may not meet user expectations or may be outdated. This enhances the acceptability, security, and user experience of personalized services.

[0156] For example, a skill pack might be described as: "I've noticed you often adjust the lights when watching TV at night. Would you like me to automate this for you?", with two options: confirm and deny. If the user clicks the confirm option, the skill pack is marked as active; if the user clicks the deny option, the skill pack is marked as pending or abandoned.

[0157] In some embodiments, the user confirmation process can be replaced by implicit learning: the control device can automatically activate low-risk skill sets (such as lighting adjustment) but keep monitoring, and automatically roll back and reduce the trigger priority once the user corrects the behavior, without requiring explicit user confirmation, thus achieving completely seamless personalized adaptation.

[0158] In some embodiments of this application, after generating the corresponding skill pack, the method further includes: Continuously monitor recent user behavior. If the behavior pattern corresponding to a certain skill pack is no longer significant based on recent user behavior, mark the skill pack as expired and reduce the trigger priority of the skill pack, or ask the user to confirm whether to deactivate the skill pack. Recent user behavior refers to the user's interaction events with smart home devices in the ninth time period. The end time of the ninth time period is earlier than the current time of the third time period.

[0159] The third current time can refer to the current system clock time obtained when the behavior pattern corresponding to a certain skill pack is no longer significant.

[0160] Trigger priority is a numerical value assigned to each generated skill pack. Its main function is to provide a decision-making basis for the embodied agent when multiple skill packs meet the trigger conditions at the same time, so as to coordinate the service order and resolve resource conflicts.

[0161] Optionally, a ninth time period can be set according to actual needs or experience. This application does not limit the specific length of the ninth time period.

[0162] In this embodiment, by continuously monitoring the user's recent behavior and comparing it with the original behavior patterns on which the skill pack relies, when habit drift is detected (i.e., the original behavior pattern is no longer significant), the corresponding skill pack is automatically marked as expired, its trigger priority is reduced, or user confirmation is initiated. This constructs a closed-loop skill lifecycle management mechanism, ensuring that the smart home control system can dynamically adapt to the user's ever-changing lifestyle habits and avoid outdated or no longer applicable behavior patterns from interfering with the user, thereby continuously providing accurate and personalized services in long-term use.

[0163] In some embodiments of this application, such as Figure 8As shown, when a trigger source is detected, the embodied intelligent agent controls the corresponding smart home device to perform corresponding actions based on the generated skill packs, which may include steps 801 to 802.

[0164] Step 801: When a trigger source is detected, the embodied agent determines the target skill pack that matches the trigger source from the generated skill packs.

[0165] Step 802: The embodied intelligent agent triggers the target skill pack to control the corresponding smart home device to perform the corresponding action.

[0166] In this embodiment, when a trigger source is detected, the embodied intelligent agent accurately matches and triggers the execution of the corresponding target skill package control device, which can transform the target skill package into an online, real-time responsive personalized service. This realizes a complete automated closed loop from "environmental perception" to "intelligent decision-making" to "precise execution," so that users can enjoy a proactive smart home experience that is highly compatible with their habits and current situation without having to manually configure or search for it.

[0167] In some embodiments of this application, before detecting the trigger source, the method further includes: When the embodied agent is activated, the embodied agent scans the skill library to read the metadata in each skill pack and builds a skill index table based on the metadata. The skill library includes at least one skill pack. The embodied intelligent agent detects the existence of a trigger source based on a skill index table and obtains the detection result; Upon detecting a trigger source, the embodied agent determines the target skill pack that matches the trigger source from the generated skill packs, including: If the detection result indicates the presence of a trigger source, the embodied agent, in response to the trigger source, retrieves the skill library via vector and identifies the target skill set that matches the trigger source.

[0168] The skill library includes skill packs generated based on behavioral patterns, as well as system-preset skill packs and user-defined skill packs.

[0169] The embodied agent can scan the skill library and build a lightweight skill index table based on the metadata of all skill packages in the skill library, and detect whether there is a trigger source based on the skill index table.

[0170] Table 1 shows an example of a skill index table, which uses a skill pack as an example for description.

[0171]

[0172] In this embodiment, by scanning the skill library and preloading metadata to build a lightweight skill index table upon startup of the embodied agent, and by quickly detecting trigger sources based on the index table and using vector retrieval for semantic matching at runtime, efficient management of massive skill packages, extremely low-latency trigger monitoring, and high-accuracy skill package matching are achieved. Metadata preloading ensures minimal system resident memory burden and fast startup, while the matching strategy combining structured condition filtering and semantic vector retrieval enables the system to respond to explicit event triggers in milliseconds and intelligently understand ambiguous or diverse natural language commands from users. This achieves an optimal balance between resource consumption, response speed, and service accuracy, providing a solid foundation for real-time, smooth adaptive services.

[0173] In some embodiments of this application, the skill index table includes at least one of time-based triggering conditions, sequence-based triggering conditions, state-based triggering conditions, and natural language descriptions for each skill package. The natural language descriptions are used to interpret the user intent represented by the corresponding behavioral patterns. The embodied agent detects the existence of trigger sources based on the skill index table and obtains detection results, including: The embodied intelligent agent periodically checks whether the first current time matches any time trigger condition in the skill index table. If the first current time matches any time trigger condition, it determines that there is a detection result of a trigger source. And / or, when subscribing to device state change events, the embodied agent detects whether the device state change event matches any sequence trigger condition or any state trigger condition in the skill index table. If the device state change event matches any sequence trigger condition or any state trigger condition, then the detection result of the existence of a trigger source is determined. And / or, in response to a user instruction, the embodied agent detects whether the user instruction matches any natural language description in the skill index table. If the user instruction matches the natural language description, the detection result indicates the existence of a trigger source.

[0174] It should be understood that different skill packs may include the same or different trigger conditions. For example, some skill packs may only include time-based trigger conditions and sequence-based trigger conditions, while others may only include sequence-based trigger conditions, and still others may include all four types of trigger conditions: time-based trigger conditions, sequence-based trigger conditions, state-based trigger conditions, and natural language descriptions.

[0175] The first current time can refer to the current system clock time obtained when the embodied intelligent agent periodically performs detection operations.

[0176] Time-triggered conditions can refer to the time period, date type, or periodic time rule that a skill pack must meet to be activated. For example, defining a skill pack's time-triggered condition as: the date is Monday to Sunday, and the time window is [19:00, 23:00] means that the skill pack meets the time-triggered condition within the time window from 7 pm to 11 pm every Monday to Sunday.

[0177] A sequence trigger condition can refer to the detection of a series of device actions occurring in a specific order and time interval required for a skill pack to be activated. For example, a sequence trigger condition for a skill pack can be defined as [living room light on, TV on], with the interval between the two device actions being within 5 minutes. When the event stream listened to by the embodied agent occurs in this order and sequence, a successful match is achieved.

[0178] A state trigger condition can refer to the requirement that a smart home device's state value reaches a specific threshold or is in a specific state for a skill package to be activated. For example, if the behavior pattern of a skill package is defined as {living room temperature > 30℃} → {turn on air conditioner}, then living room temperature > 30℃ is a state trigger condition. When the living room temperature reported by the temperature sensor meets this condition, the corresponding skill package will be triggered.

[0179] This application does not limit the specific type of the aforementioned user instructions. For example, the user instructions can be voice instructions or text instructions. The embodied intelligent agent can semantically match the user instructions with the natural language description of any skill package in the skill index table; if a match is found, the skill package is loaded.

[0180] In this embodiment, by defining a unified skill index table and implementing periodic time matching, real-time event stream sequence, state matching, and semantic-based user instruction matching based on this table, the embodied intelligent agent can simultaneously listen to multiple trigger sources such as time, device actions, device status, and natural language instructions with extremely low resource overhead (only needing to query the skill index table in memory). It can also respond instantly when the trigger source matches the trigger conditions of any skill package, thus laying a solid technical foundation for realizing millisecond-level, complete trigger perception and precise skill scheduling in the smart home control system.

[0181] In some embodiments, predictive models can be used to anticipate users' upcoming habits and proactively ask or execute them, thus achieving "predictive services." For example, the lights can be adjusted in advance to prepare for the user's usual TV viewing time.

[0182] In some embodiments of this application, such as Figure 9 As shown, when there are multiple target skill packs, the embodied intelligent agent triggers the target skill pack to control the corresponding smart home device to perform the corresponding action, which may include steps 901 to 904.

[0183] Step 901: The embodied agent detects whether there is a skill conflict between at least two of the multiple target skill packages, and if there is a skill conflict between at least two target skill packages, determines the conflict relationship between the at least two target skill packages.

[0184] Skill conflicts include at least one of resource conflicts, logical conflicts, and intentional conflicts.

[0185] Resource conflicts can refer to situations where at least two target skill packs use the same smart home device and the device actions are mutually exclusive. For example, two target skill packs simultaneously require "turn on the living room light" and "turn off the living room light".

[0186] A logical conflict can refer to a situation where the execution results of at least two target skill sets are semantically contradictory. For example, two target skill sets may simultaneously require "dimming the lights to 20%" and "brightening the lights to 80%".

[0187] Intent conflict can refer to at least two target skill sets reflecting conflicting user intents. For example, two target skill sets may simultaneously require triggering "movie mode" and "reading mode".

[0188] A conflict relationship can refer to a category of conflict that needs to be adjudicated between at least two target skill sets that are triggered simultaneously. Conflict relationships describe what type of skill conflict exists between which skill sets.

[0189] In some embodiments, an embodied agent can extract execution plans for multiple target skill packages, construct a device-action mapping table, combine a predefined library of mutually exclusive action pairs and lightweight semantic parsing, complete conflict detection and determine conflict relationships, and provide structured input for subsequent large model adjudication.

[0190] Step 902: The embodied agent generates prompt words based on the task objective, candidate skill list, conflict relationships, current environmental context, and recent user feedback.

[0191] The candidate skill list includes the skill names, skill descriptions, triggering conditions, execution plans, and triggering priorities of multiple target skill packages. Recent user feedback represents the user's acceptance or rejection of multiple target skill packages within the eighth time period, and the end time of the eighth time period is earlier than the second current time.

[0192] Optionally, an eighth time period can be set according to actual needs or experience. This application does not limit the specific length of the eighth time period.

[0193] The second current time can refer to the current system clock time obtained by the embodied agent when handling multi-skill conflict adjudication.

[0194] Optionally, the above prompts can be structured text.

[0195] The task objective defines the fundamental problem that the second major model needs to solve in this conflict resolution, namely "what to do," thereby initiating and guiding the subsequent complex semantic understanding and reasoning process, ensuring that the model's output is consistent with the system's expectations (resolving the conflict and coordinating execution). For example, the task objective could be "Please select the optimal execution plan for the current multi-skill conflict."

[0196] The current environmental context can refer to the context of a second current time, including but not limited to the second current time, date type, personnel presence status, environmental sensor data, etc.

[0197] Step 903: The embodied agent inputs the prompt words into the second large model to understand the skill intent of multiple target skill packages and obtain the adjudication result.

[0198] The ruling includes the set of skill packs to be executed and the execution order of each skill pack in the set.

[0199] In this embodiment, the embodied agent inputs the prompt words into the second model, which outputs a ruling based on a deep understanding of the skill intent.

[0200] In some embodiments, if the confidence level of the pruning result output by the second largest model is low (which can be determined by the logits output by the second largest model or by an explicit confidence field), the system can degrade to an interactive query, presenting the conflicting options and the reasons for the model's recommendation to the user for final confirmation.

[0201] The second major model mentioned above can be a locally deployed lightweight model (such as Qwen-2.5-7B) to ensure response speed and privacy. The adjudication prompts can be designed in a few-shot format, providing input and output examples of typical conflict cases to guide the second major model to output structured results that meet expectations. For scenarios with extremely high real-time requirements, explicit conflicts (such as the absolute priority of security skills) can be quickly processed in the behavior pattern mining process first, leaving only complex conflicts to be adjudicated by the second major model, achieving a balance between efficiency and intelligence. Through the above design, this embodiment deeply integrates the semantic understanding and reasoning capabilities of the large language model into the multi-skill adjudication process, enabling the smart home control system to make intelligent decisions that align with the user's true intentions even in complex situations.

[0202] Step 904: The embodied intelligent agent parses the execution plans of each skill package in the skill package set into skill execution instructions, and controls the corresponding smart home devices to execute the corresponding skill execution instructions in the execution order.

[0203] In some embodiments, after controlling the corresponding smart home devices to execute the corresponding skill execution instructions in the execution order, the execution results and subsequent user feedback (such as whether the user manually corrects the error) can be recorded to build the historical context for the next decision, forming a closed loop of continuous optimization.

[0204] In this embodiment, the large model adjudication method has significant advantages over traditional adjudication methods. At the semantic understanding level, the large model can identify the true intent behind skill descriptions. For example, "dimming the lights" might serve a "movie watching" need, while "brightening the lights" might serve a "reading" need. Even if there is a resource conflict between the two, the large model can make a more context-appropriate choice based on the current environmental context (e.g., whether the TV is on or the user is holding a book). In handling complex scenarios, the large model can output non-binary, refined solutions. For example, when "movie watching mode" and "energy-saving mode" are triggered simultaneously, the large model might suggest "dimming the lights to 30%" instead of completely rejecting one, achieving a balance between user comfort and energy-saving goals.

[0205] In some embodiments of this application, the decision result further includes the rationale for the decision, which includes the rationale for selecting the skill pack set and the rationale for executing each skill pack in the skill pack set in the execution order; after obtaining the decision result, it also includes: The embodied intelligent agent presents the rationale behind the decision to the user.

[0206] In this embodiment, the embodied intelligent agent directly presents the decision rationale output by the large model (e.g., "considering that it is evening and the TV is on, prioritize the movie-watching mode") to the user, which can significantly enhance the transparency of the system and the user's sense of trust.

[0207] In some embodiments of this application, after triggering the target skill pack, the method further includes: Record user behavior after the target skill pack is triggered; If the user behavior is a corrective behavior, and corrective behaviors for the target skill pack occur frequently, then the target skill pack will be deleted. A corrective behavior refers to an action used to correct the device action corresponding to the target skill pack.

[0208] The control device records user behavior each time a skill pack is triggered. For example, after the skill pack automatically turns on the living room light, does the user immediately turn it off manually? If "user correction" events occur frequently, such as more than five times, the skill pack is deleted.

[0209] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0210] Corresponding to the skill generation method for embodied intelligent agents described in the above embodiments, Figure 10 A schematic diagram of the structure of a skill generation device for embodied intelligent agents provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0211] Reference Figure 10 The device includes: The sequence acquisition module 1001 is used to acquire the first behavior sequence formed by the various interaction events between the user and the smart home device within a first time period; The pattern mining module 1002 is used to perform multi-dimensional mining on the first behavior sequence to mine multiple behavior patterns. The multiple behavior patterns include at least two of the following: periodic pattern, first sequence pattern, first association pattern, and scenario pattern. The periodic pattern represents a user behavior pattern with a time regularity. The first sequence pattern represents the temporal relationship between different device actions. The device action refers to the action performed by the smart home device. The first association pattern represents the association between the action execution condition and the device action. The scenario pattern represents the user's behavior scenario. The skill generation module 1003 is used to generate a corresponding skill pack based on any of the aforementioned behavior patterns. The embodied intelligent agent 1004 is used to control the corresponding smart home device to perform corresponding actions based on the generated skill packages when a trigger source is detected.

[0212] It should be understood that, Figure 10 The structural block diagram of the skill generation device for embodied intelligent agents shown includes modules for performing... Figures 2 to 9 The steps in the corresponding embodiments, and for Figures 2 to 9 The steps in the corresponding embodiments have been explained in detail in the above embodiments. Please refer to them for details. Figures 2 to 9 as well as Figures 2 to 9 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0213] Figure 11 This is a schematic diagram of the structure of the control device provided in an embodiment of this application. Figure 11 As shown, the control device of this embodiment includes: at least one processor 1100 ( Figure 11 (Only one is shown in the diagram), memory 1101, and computer program 1102 stored in the memory 1101 and executable on the at least one processor 1100, wherein when the processor 1100 executes the computer program 1102, it causes the control device to perform the steps in any of the above method embodiments.

[0214] The control device may include, but is not limited to, a processor 1100 and a memory 1101. Those skilled in the art will understand that... Figure 11 This is merely an example of a control device and does not constitute a limitation on the control device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0215] The processor 1100 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0216] In some embodiments, the memory 1101 may be an internal storage unit of the control device, such as a hard drive or memory. In other embodiments, the memory 1101 may be an external storage device of the control device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the control device. Furthermore, the memory 1101 may include both internal and external storage units of the control device. The memory 1101 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 1101 can also be used to temporarily store data that has been output or will be output.

[0217] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0218] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / control device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0219] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0220] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0221] In the embodiments provided in this application, it should be understood that the disclosed apparatus / control devices and methods can be implemented in other ways. For example, the apparatus / control device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0222] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0223] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for skill generation in embodied intelligent agents, characterized in that, include: Obtain the first behavioral sequence formed by the various interaction events between the user and smart home devices within the first time period; The first behavior sequence is mined from multiple dimensions to uncover multiple behavior patterns. These multiple behavior patterns include at least two of the following: periodic pattern, first sequence pattern, first association pattern, and scenario pattern. The periodic pattern represents a user behavior pattern with a time regularity. The first sequence pattern represents the temporal relationship between different device actions, where the device action refers to the action performed by the smart home device. The first association pattern represents the association between the action execution condition and the device action. The scenario pattern represents the user's behavior scenario. For any of the aforementioned behavior patterns, a corresponding skill set is generated based on the behavior pattern; Upon detecting a trigger source, the embodied intelligent agent controls the corresponding smart home device to perform the corresponding action based on the generated skill packages.

2. The skill generation method according to claim 1, characterized in that, The periodic patterns include periodic sequence patterns and / or periodic association patterns; The methods for mining the periodic sequence patterns include: For the second behavior sequence of each day in the first behavior sequence, the second behavior sequence is grouped according to different second time periods and date types to obtain each first group; Extract frequently occurring first action subsequences within each of the first groups, and determine the first action subsequences as the second sequence pattern; If the second sequence pattern appears stably within N consecutive observation periods, then the second sequence pattern is determined as the periodic sequence pattern, where N is an integer greater than 1; The methods for mining the periodic association patterns include: The first behavior sequence is organized into transactions according to a fixed time window, and a time tag is attached to each transaction. The time tag includes the time period corresponding to the transaction and the date type. Transactions with the same time tag are grouped into the same group to obtain each second group; The association between the action execution conditions and the device actions is mined from each of the second groups, and the association between the action execution conditions and the device actions mined from each of the second groups is determined as a second association pattern; If the support of the second correlation pattern remains stable over M consecutive observation periods and is concentrated in a certain time period, then the second correlation pattern is determined as the periodic correlation pattern, where M is an integer greater than 1.

3. The skill generation method according to claim 1, characterized in that, The methods for mining the first association pattern include: When the action execution condition is another device action, for the third time period in the first behavior sequence, the linkage relationship between different device actions is mined to obtain the inter-device association pattern; When the action execution condition is an environmental context condition, for the fourth time period in the first behavior sequence, the relationship between the environmental context condition and the device action is mined to obtain the context and device association pattern; When the action execution condition includes both another device state and the environmental context condition, for the fifth time period in the first behavior sequence, the correlation between the other device state and the environmental context condition and the device action is mined to obtain a first hybrid mode. The other device state refers to the operating state of another smart home device. When the action execution conditions simultaneously include the other device action and the environmental context conditions, for the sixth time period in the first behavior sequence, the correlation between the other device action and the environmental context conditions and the device action is mined to obtain the second hybrid mode; At least one of the device association mode, the context and device association mode, the first hybrid mode, and the second hybrid mode is the first association mode.

4. The skill generation method according to claim 1, characterized in that, The mining method for the first sequence pattern includes: For the seventh time period in the first behavior sequence, frequently occurring second action subsequences are extracted, the average time interval between each device action in the second action subsequence is recorded, and the second action subsequence is determined as the first sequence pattern.

5. The skill generation method according to claim 1, characterized in that, The methods for mining the scene patterns include: Divide the daily second behavior sequence in the first behavior sequence into multiple behavior segments; The multiple behavioral segments are vectorized to obtain multiple vectors; Clustering is performed on multiple vectors to obtain the behavioral scenario, and the behavioral scenario is determined as the scenario pattern.

6. The skill generation method according to any one of claims 1 to 5, characterized in that, For any of the aforementioned behavioral patterns, generating a corresponding skill set based on the behavioral pattern includes: The behavior pattern is input into the first model to perform intent recognition on the behavior pattern and generate a natural language description, which is used to explain the user intent represented by the behavior pattern. The skill set is generated based on the behavioral patterns and the natural language descriptions.

7. The skill generation method according to claim 6, characterized in that, After generating the skill pack, the process also includes: Based on the natural language description, a semantic description for the skill package is generated; The skill pack is pushed to the user, along with the semantic description; In response to the confirmation command for the skill pack, the skill pack is marked as activated. Alternatively, in response to a denial instruction for the skill pack, the skill pack may be marked as pending or obsolete.

8. The skill generation method according to any one of claims 1 to 5, characterized in that, Upon detecting a trigger source, the embodied intelligent agent, based on the generated skill packages, controls the corresponding smart home devices to perform corresponding actions, including: Upon detecting a trigger source, the embodied agent determines a target skill pack that matches the trigger source from among the generated skill packs; The embodied intelligent agent triggers the target skill package to control the corresponding smart home device to perform the corresponding action.

9. The skill generation method according to claim 8, characterized in that, Before the trigger source is detected, it also includes: When the embodied agent is activated, the embodied agent scans the skill library to read the metadata in each skill package and constructs a skill index table based on the metadata. The skill library includes at least one of the skill packages. The embodied intelligent agent detects the existence of a trigger source based on the skill index table and obtains the detection result; When a trigger source is detected, the embodied agent determines a target skill pack matching the trigger source from the generated skill packs, including: If the detection result indicates the presence of a trigger source, then in response to the trigger source, the embodied agent retrieves the skill library via vector and determines the target skill pack that matches the trigger source.

10. The skill generation method according to claim 9, characterized in that, The skill index table includes at least one of the following: time-triggered conditions, sequence-triggered conditions, state-triggered conditions, and natural language descriptions for each skill package. The natural language descriptions are used to interpret the user intent represented by the corresponding behavioral patterns. The embodied intelligent agent detects the existence of a trigger source based on the skill index table and obtains the detection result, including: The embodied intelligent agent periodically detects whether the first current time matches any of the time triggering conditions in the skill index table. If the first current time matches any of the time triggering conditions, it is determined that there is a detection result of a trigger source. And / or, when subscribing to device state change events, the embodied agent detects whether the device state change event matches any of the sequence trigger conditions or any of the state trigger conditions in the skill index table. If the device state change event matches any of the sequence trigger conditions or any of the state trigger conditions, then a detection result indicating the existence of a trigger source is determined. And / or, in response to a user instruction, the embodied agent detects whether the user instruction matches any of the natural language descriptions in the skill index table; if the user instruction matches the natural language description, then a detection result indicating the existence of a trigger source is determined.

11. The skill generation method according to claim 8, characterized in that, When there are multiple target skill packages, the embodied intelligent agent triggers the target skill package to control the corresponding smart home device to perform corresponding actions, including: The embodied intelligent agent detects whether there is a skill conflict between at least two of the multiple target skill packages, and determines the conflict relationship between the at least two target skill packages if there is a skill conflict between them. The skill conflict includes at least one of resource conflict, logical conflict and intent conflict. The resource conflict refers to at least two target skill packages using the same smart home device and the device actions are mutually exclusive. The logical conflict refers to at least two target skill packages having semantically contradictory execution results. The intent conflict refers to at least two target skill packages reflecting contradictory user intents. The embodied intelligent agent generates prompt words based on the task objective, the candidate skill list, the conflict relationship, the current environmental context, and the user's recent feedback. The candidate skill list includes the skill names, skill descriptions, triggering conditions, execution plans, and triggering priorities of multiple target skill packages. The user's recent feedback represents the user's acceptance or rejection of multiple target skill packages within an eighth time period. The end time of the eighth time period is earlier than the second current time. The embodied intelligent agent inputs the prompt words into the second large model to perform skill intent understanding on multiple target skill packs through the second large model, and obtains a decision result. The decision result includes the set of skill packs to be executed and the execution order of each skill pack in the skill pack set. The embodied intelligent agent parses the execution plan of each skill package in the skill package set into skill execution instructions, and controls the corresponding smart home device to execute the corresponding skill execution instructions according to the execution order.

12. The skill generation method according to claim 11, characterized in that, The decision result also includes the reasons for the decision, which include the reasons for selecting the set of skill packs and the reasons for executing each skill pack in the set of skill packs in the execution order; Following the determination of the ruling, the following is also included: The embodied intelligent agent presents the rationale for the decision to the user.

13. The skill generation method according to claim 8, characterized in that, After the embodied agent triggers the target skill pack, the following is also included: Record user behavior after the target skill pack is triggered; If the user behavior is a corrective behavior, and the corrective behavior for the target skill pack occurs frequently, then the target skill pack is deleted. The corrective behavior refers to the behavior used to correct the device action corresponding to the target skill pack.

14. The skill generation method according to any one of claims 1 to 5, characterized in that, After generating the corresponding skill pack, the following is also included: The system continuously monitors recent user behavior. If, based on the user's recent behavior, it detects that the behavior pattern corresponding to a certain skill pack is no longer significant, then the skill pack is marked as expired, and the trigger priority of the skill pack is reduced. Alternatively, the system may ask the user whether to disable a certain skill pack. The user's recent behavior refers to the user's interaction events with the smart home device during the ninth time period, and the end time of the ninth time period is earlier than the third current time.

15. The skill generation method according to any one of claims 1 to 5, characterized in that, Obtain the first behavioral sequence formed by various interaction events between the user and smart home devices within a first time period, including: The interactive events are cleaned and aligned to obtain the processed interactive events. The processed interactive events are sorted by time to obtain the structured first behavior sequence.

16. A control device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the control device to implement the skill generation method as described in any one of claims 1 to 15.

17. A smart home control system, characterized in that, It includes a control device and at least one smart home device, the control device performing the skill generation method as described in any one of claims 1 to 15.

Images