A home scene control method, a ternary architecture system and an electronic device

Abstract

The application provides a home scene control method, a ternary architecture system and electronic equipment, and belongs to the technical field of smart home control. The method comprises the following steps: obtaining the behavior track intention and the risk degree value of each person through the monitoring image and the home state data under a home scene, and further identifying a plurality of conflict events existing in the home scene based on the same; and through the combination simulation of the decision schemes between the conflict events, the optimal target decision scheme group can be automatically screened out under the multi-objective constraint of the minimum safety risk value, the minimum scheme convenience loss and the maximum scheme emergency response degree, and the home equipment and the user are simultaneously affected. Compared with the prior art, the application realizes the balance among the safety risk, the convenience and the emergency response capability through the comprehensive identification of the multiple conflict events and the optimal decision of the multiple targets, and meets the demand of the collaborative operation of the multiple persons and the multiple devices in the whole-house intelligent scene.

Description

Technical Field

[0001] This application relates to the field of smart home control technology, and in particular to a control method for home scenarios, a ternary architecture system, and electronic devices. Background Technology

[0002] Controlling home environments typically refers to the process of managing a smart home system by sensing environmental data, device operating status, and user behavior, and then outputting device control commands and human-computer interaction information to achieve safe, comfortable, and efficient operation. Home spaces often involve multiple users, various home devices, and complex environmental variables, which can easily lead to conflicts between different user behaviors and device operating states. Examples include conflicts in user behavior trajectories, conflicts in multiple user needs, conflicts in user device permissions, and conflicts in security behaviors caused by environmental factors.

[0003] To achieve basic control of home scenarios, traditional home control methods typically employ a singular decision-making model, formulating control strategies independently for single events or devices, lacking comprehensive consideration of multiple conflicting events. Because they cannot achieve a comprehensive balance when multiple conflicting events coexist, traditional methods often excessively sacrifice user convenience to reduce security risks, or ignore dangerous events requiring immediate emergency response in order to maintain convenience. Therefore, by failing to find a balance between security risks, convenience sacrifices, and emergency response capabilities, the final control decisions suffer from bias, poor stability, and insufficient adaptability. Consequently, traditional home control methods struggle to meet the needs of multi-person, multi-device collaborative operation in whole-house smart home scenarios. Summary of the Invention

[0004] The main purpose of this application is to propose a control method, a three-element architecture system and an electronic device for home scenarios. This application effectively solves the problems of one-sided decision-making, poor stability and insufficient adaptability of traditional home control methods in scenarios with multiple conflicting events, through multi-conflict event identification, multi-objective optimization decision-making and collaborative control of device and human-computer interaction.

[0005] To achieve the above objectives, this application proposes a method for controlling a home scene, the method comprising: Acquire several surveillance images in a home scene, and determine the home space corresponding to each surveillance image and the home status data corresponding to the home space; wherein, the home scene contains several different human objects; For each person in each of the aforementioned surveillance images, based on the home status data and the image features in the surveillance images, the behavioral trajectory intention of the person and the behavioral trajectory risk level value corresponding to the behavioral trajectory intention are generated; Based on the image features, the intent of each behavior trajectory, the risk level value of each behavior trajectory, and the home status data, several conflict events of different conflict types are generated in the home scenario. Based on preset decision constraints, the decision schemes between each conflict event are combined and simulated. When a candidate decision scheme group satisfies the minimum safety risk value, the minimum loss of convenience, and the maximum emergency response level, the candidate decision scheme group is taken as the target decision scheme group. The target decision scheme group includes: several target equipment control commands and several target human-computer interaction mechanisms. The home appliances in the home scene are controlled according to the control commands of each target device; based on the human-computer interaction mechanism of each target, several interactive reminder messages are generated for information transmission with the human object.

[0006] Furthermore, the image features include: the facial features of the person; the behavioral trajectory intent includes: the intention of the movement area to represent the endpoint of the person's movement trajectory and the intention of the action tendency to represent the degree of avoidance tendency during the movement of the person; the conflict event corresponds to a behavioral attribute label; The process of generating the conflict event includes: Based on the facial features, obtain the historical behavioral habits data and device operation permissions of each of the aforementioned individuals; For each of the aforementioned human objects, based on the historical behavioral habit data, the device operation permissions, the home status data, and the behavioral trajectory risk level value, the behavioral trajectory intent is subjected to scene matching degree verification, permission matching degree verification, device conflict relationship verification, and habit matching degree verification, and the target verification result corresponding to the human object is output; wherein, the target verification result is used to characterize the degree of adaptation between the human object's behavior and the spatiotemporal scene, device permissions, device operating status, and historical behavioral habits; Within the same time window, several behavioral trajectory intentions that have overlapping areas and whose target collision probability is greater than a preset collision probability threshold are combined into a first conflict event of trajectory collision type; wherein, the target collision probability is calculated from the avoidance tendency degree between each of the action tendency intentions. The target inspection results and the risk level values ​​of each behavior trajectory corresponding to the first conflict event are weighted and summed to output the behavior attribute label of the first conflict event; The behavioral attribute tags include: safety attribute tags indicating that the behavior will threaten personal safety or equipment safety; emergency attribute tags indicating that the behavior needs to be responded to in order to avoid further damage; or convenience attribute tags indicating that the behavior attribute is a need behavior of a person or object.

[0007] Furthermore, the conflict types also include: device operation types; device operation permissions include: having operation permissions or not having operation permissions; the behavioral trajectory intent also includes: device operation intent used to represent the user's operational needs for home appliances; wherein, different operational needs correspond to different values ​​of operational rationality. The process of generating the conflict event also includes: Several behavioral trajectory intentions that have no device operation permission and whose operation rationality value is less than a preset operation rationality threshold are combined into a second conflict event whose conflict type is device operation. The target inspection results and the risk level values ​​of each behavior trajectory corresponding to the second conflict event are weighted and summed to output the behavior attribute label of the second conflict event.

[0008] Furthermore, the process of generating the first conflict event includes: For each stated behavioral trajectory intention, the minimum circumcircle corresponding to the endpoint pointing region is determined based on the region shape. For each stated behavioral trajectory intention, the center of the minimum circumcircle is adjusted according to the degree of avoidance tendency and the preset center offset, and the radius of the minimum circumcircle is adjusted according to the degree of avoidance tendency and the preset radius offset. Then, the adjusted minimum circumcircle is generated based on the adjusted center and the adjusted radius. Multiple character objects whose adjusted minimum circumcircles overlap within the same time window are combined into a first candidate character combination; For each of the first candidate combinations, the overlapping area between each of the adjusted minimum circumcircles is taken as the target area, and the maximum inscribed circle of the target area is determined; the center distance between the center of the maximum inscribed circle and the center of each of the adjusted minimum circumcircles is calculated, and the maximum center distance is taken as the center distance to be detected; when it is determined that the center distance to be detected is less than a preset center distance threshold, the first candidate combination is taken as the second candidate combination. For each second candidate character combination, the target collision probability is calculated based on the degree of avoidance tendency among the various action tendency intentions; when the target collision probability is determined to be greater than a preset collision probability threshold, the second candidate character combination is marked as the target character combination; For each group of target individuals, the intentional behavioral trajectories of each individual in the group of target individuals are combined into the first conflict event.

[0009] Furthermore, the process of determining the target decision scheme group includes: Based on the behavioral attribute tags corresponding to the first conflict event and the second conflict event, the priorities corresponding to the first conflict event and the second conflict event are determined respectively; wherein, the priority of the safety attribute tag is greater than the priority of the emergency attribute tag; the priority of the emergency attribute tag is greater than the priority of the convenience attribute tag; Based on a pre-defined mapping rule base between conflict and solution, several candidate decision schemes corresponding to the first conflict event and several candidate decision schemes corresponding to the second conflict event are determined; wherein, the candidate decision schemes include: several candidate device control commands and several candidate machine interaction mechanisms; Based on the priorities corresponding to the first conflict event and the second conflict event, the candidate decision schemes are traversed until several candidate decision scheme groups are output; wherein, during each traversal, a candidate decision scheme is extracted from each candidate decision scheme corresponding to the first conflict event and each candidate decision scheme corresponding to the second conflict event, and combined into a candidate decision scheme group. Based on preset decision constraints, each candidate decision scheme group is simulated. When a candidate decision scheme group satisfies the minimum safety risk value, the minimum convenience loss of the scheme, and the maximum emergency response level of the scheme, the output is the target decision scheme group.

[0010] Furthermore, the process of generating the group of candidate decision schemes includes: A tree structure is constructed with the combination of conflict events corresponding to the first conflict event and the second conflict event as the root node, and the first conflict event and the second conflict event as different first-level child nodes; wherein, the priority of the first conflict event is used as the node attribute of the first-level child node corresponding to the first conflict event; the priority of the second conflict event is used as the node attribute of the first-level child node corresponding to the second conflict event. Starting from the root node, several different first-level child nodes are randomly selected from the tree structure to generate a first node path group; wherein, the first node path group contains several different first node paths; the first node path contains one or more first-level child nodes. The first node path is filtered according to a preset path selection rule. Then, according to each of the candidate decision schemes, the node paths in the filtered first node path group are expanded into secondary nodes to construct several target node paths to be simulated. The preset path selection rule is used to characterize that the priority of the previous node in the node path is greater than the priority of the next node. For each target node path to be simulated, extract a second-level child node corresponding to each first-level child node in the target node path to be simulated, generate a candidate decision scheme group, until the extraction is completed, and output several candidate decision scheme groups.

[0011] Furthermore, the construction process of several target node paths to be simulated includes: According to the preset path selection rules, delete the first node paths in the first node path group that do not conform to the preset path selection rules, and mark the first node paths that have not been deleted as second node paths. For each second node path, the first-level child node with the smallest relative distance to the root node is taken as the first target child node. Then, the candidate decision scheme corresponding to the first target child node is expanded into the second-level child nodes corresponding to the first target child node to construct the third node path; wherein, the first target child node corresponds to multiple different second-level child nodes. For each of the third node paths, the candidate device control instructions in the second-level sub-nodes corresponding to the first target sub-node are taken as the first candidate device control instructions. The first candidate device control instructions are identified, and it is determined whether the action corresponding to the first candidate device control instructions is an interception behavior. If yes, the third node path is marked as the initial node path to be simulated. If no, the second-level sub-nodes corresponding to the first candidate device control instructions are deleted from the third node path, and then the initial node path to be simulated is constructed. The interception behavior is used to indicate the action instructions to prevent the occurrence of security risks. For each initial node path to be simulated, the initial node path to be simulated is adjusted according to the candidate decision schemes corresponding to the first-level child nodes that have not been expanded into second-level nodes in the initial node path to be simulated, thereby constructing the target node path to be simulated.

[0012] Furthermore, the step of adjusting the initial path to be simulated based on the candidate decision schemes corresponding to the first-level child nodes that have not been expanded into second-level nodes in the initial path to be simulated, and constructing the target path to be simulated, includes: The first-level child nodes that have not been expanded into second-level nodes in the initial simulated node path are taken as the second target child nodes. Then, the candidate decision schemes corresponding to the second target child nodes are expanded into second-level child nodes corresponding to the second target child nodes to construct the fourth node path. For each of the fourth node paths, the candidate device control instructions in the second-level sub-nodes corresponding to the second target sub-node are used as second candidate device control instructions. The second candidate device control instructions are identified to determine whether the action corresponding to the second candidate device control instructions is a required behavior. If yes, the fourth node path is marked as the target node path to be simulated. If no, the second-level sub-nodes corresponding to the second candidate device control instructions are deleted from the fourth node path, thereby constructing the target node path to be simulated. The required behavior is used to indicate action instructions that can satisfy the character's needs. The initial node path to be simulated that does not have any unexpanded secondary nodes is directly marked as the target node path to be simulated.

[0013] To achieve the above objectives, another aspect of this application proposes a three-element architecture system for home scenarios, the three-element architecture system comprising: a terminal device, an edge server, and a cloud server; The edge device is used to collect several monitoring images in a home scene, and then send each monitoring image to the edge server. The edge server is used to determine the home space corresponding to each monitoring image and the home status data corresponding to the home space; wherein, the home scene includes several different human objects; The edge server is also used to generate, for each person in each of the monitoring images, the behavioral trajectory intention of the person and the behavioral trajectory risk level value corresponding to the behavioral trajectory intention, based on the home status data and the image features in the monitoring images; The edge server is also used to generate several conflict events of different conflict types in the home scene based on the image features, the intention of each behavior trajectory, the risk level value of each behavior trajectory, and the home status data, and then send the conflict events of each different conflict type to the cloud server. The cloud server is used to perform combined simulations of decision schemes between each conflict event based on preset decision constraints. When a candidate decision scheme group satisfies the minimum security risk value, the minimum loss of convenience, and the maximum emergency response level, the candidate decision scheme group is taken as the target decision scheme group. The target decision scheme group includes: several target device control commands and several target human-computer interaction mechanisms. The cloud server is also used to send control instructions for each of the target devices to the end device, so that the end device can control the home devices in the home scene according to the control instructions for each of the target devices; The cloud server is also used to send each of the target human-computer interaction mechanisms to the edge server, so that the edge server generates a number of interactive reminder messages for information transmission with the human object based on each of the target human-computer interaction mechanisms, and the edge server sends the interactive reminder messages to the corresponding terminal device for execution output.

[0014] Another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned home scene control method.

[0015] The embodiments of this application include at least the following beneficial effects: This application provides a home scene control method, a three-element architecture system, and an electronic device. The home scene control method of this application acquires several monitoring images in the home scene, and analyzes and generates the behavioral trajectory intention and risk level value of each person based on the image features of the monitoring images and home status data, thereby achieving a comprehensive perception of potential conflicts between person behavior and home status; furthermore, it performs fusion analysis on image features, person behavioral trajectory intention, behavioral trajectory risk level value, and home status data to automatically identify multiple types of conflict events existing in the home scene, in order to solve the problem that traditional methods cannot comprehensively consider multiple conflict events and in To address the issue of balancing multiple objectives, this invention introduces a multi-objective optimization decision-making mechanism based on preset decision constraints. By simulating the combination of candidate decision schemes among various conflicting events, it can predict the impact of different decision combinations on the overall home environment in advance. Furthermore, under the constraints of minimizing safety risk, minimizing convenience loss, and maximizing emergency response capability, it automatically selects the optimal target decision scheme group, achieving a balance between safety risk, convenience, and emergency response capability. Finally, by outputting the target device control commands and target human-computer interaction mechanisms from the target decision scheme group, it simultaneously acts on home devices and users, achieving collaborative optimization of device control and user reminders in multi-person, multi-device scenarios. Therefore, this invention effectively solves the problems of one-sided decision-making, poor stability, and insufficient adaptability of traditional home control methods in scenarios with multiple conflicting events, through comprehensive identification of multiple conflicting events, multi-objective optimization decision-making, and collaborative control of devices and human-computer interaction. It meets the needs of multi-person, multi-device collaborative operation in whole-house smart home scenarios. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart of a home scene control method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the tree structure corresponding to different conflict events provided in an embodiment of this application; Figure 3 This is a schematic diagram of the path structure of the third node path provided in an embodiment of this application; Figure 4 This is a schematic diagram of the path structure of the target node to be simulated provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a three-element architecture system for a home scene provided in one embodiment of this application; Figure 6 This is a schematic diagram of the module structure of a ternary architecture system provided in another embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0018] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0019] As used in this application, the terms "several", "each", and "multiple" include, "several" as one, two, or more; "multiple" as one, two, or more; "each" as each of the corresponding multiples; and "any" as any one of the multiples.

[0020] In existing technologies, traditional home control systems mostly adopt a device-level single-point automation model, which can only achieve independent control of a single device. It lacks the ability to perform global correlation analysis of human behavior, device operating status, and environmental data within the home space. It cannot proactively identify complex conflicts caused by multi-device linkage and human interaction, and can only passively issue alarms after a conflict occurs. For example, it cannot anticipate the risk of collisions when a child approaches an oven or an elderly person carries hot soup, or the conflict between the automatic cooling of an air conditioner and the full-load operation of a humidifier in terms of temperature and humidity control targets. This results in delayed safety protection and passive conflict response. Furthermore, traditional technologies often have simplistic and rigid solutions for home conflicts, failing to consider the balance between minimizing safety risks, minimizing the loss of convenience, and achieving adequate emergency response levels. This can easily lead to over-protection that reduces user experience or insufficient protection that leaves safety hazards.

[0021] In view of this, this application provides a home scene control method, a three-element architecture system, and an electronic device. This application can proactively identify multiple types of conflict events through multi-dimensional fusion analysis of image features, behavioral trajectory intentions, and home status data, achieving early perception and classification of conflicts. Furthermore, through candidate decision scheme combination simulation and verification of preset decision constraints, it selects a target decision scheme group that simultaneously meets the requirements of lowest safety risk, lowest convenience loss, and qualified emergency response level, achieving the optimal balance between safety and experience. This can meet the needs of multi-person and multi-device collaborative operation in a whole-house smart scene.

[0022] Figure 1 This is an optional flowchart of a home scene control method provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S1 to S5: Step S1: Acquire several surveillance images in a home scene, and determine the home space corresponding to each surveillance image and the home status data corresponding to the home space; wherein, the home scene contains several different human objects; Step S2: For each person in each of the monitoring images, based on the home status data and the image features in the monitoring images, generate the behavioral trajectory intention of the person and the behavioral trajectory risk level value corresponding to the behavioral trajectory intention. Step S3: Based on the image features, the intent of each behavior trajectory, the risk level value of each behavior trajectory, and the home status data, generate several conflict events of different conflict types in the home scene; Step S4: Based on preset decision constraints, perform combined simulations of decision schemes for each conflict event. When a candidate decision scheme group satisfies the minimum safety risk value, the minimum loss of convenience, and the maximum emergency response level, the candidate decision scheme group is taken as the target decision scheme group. The target decision scheme group includes: several target equipment control commands and several target human-machine interaction mechanisms. Step S5: Control the home devices in the home scene according to the control instructions of each target device; generate a number of interactive reminder messages for information transmission with the human object based on the human-computer interaction mechanism of each target.

[0023] In illustrative terms, the home scene described in this embodiment of the invention refers to a complete set of living scenes that take the family living space as the carrier and include several physical partitions, multiple human objects, various home appliances, and dynamic environmental variables.

[0024] This invention can identify multiple living spaces within a home scene based on surveillance images captured by various cameras. It then extracts multi-dimensional home status data for each space to further deduce the behavioral intentions of individuals within the current home scene and quantify the corresponding risk levels, thereby enabling the early identification of potentially dangerous behaviors. For example, in this embodiment, a pre-trained deep learning model can divide the surveillance images into multiple image blocks. A self-attention mechanism captures global image features, and a cross-modal attention mechanism is introduced to establish a mapping between image features and home status data. Intention classification and risk quantification are integrated into a joint prediction task to derive the behavioral intentions of individuals and their corresponding risk levels.

[0025] Since traditional methods cannot identify conflict events between multiple people, devices, and spaces in a home setting, adjustments are often made passively only after a conflict occurs. To address this issue, step S3 of this embodiment can further identify multiple types of conflict events based on multi-dimensional fusion analysis of image features, behavioral intentions, risk values, and home status data, such as device operation conflicts, human behavior conflicts, and space usage conflicts.

[0026] In real-world home environments, different types of conflict events can influence each other, and the operational status of different devices can trigger chain reactions. A single decision may trigger new conflicts or risks. This invention, through a combined simulation mechanism, can effectively handle the complex coupling relationships between multiple people, devices, and spaces in a whole-house smart home scenario. By combining and simulating decision schemes for each conflict event, the impact of different decision combinations on the overall home scenario can be predicted in advance, avoiding mutual interference between decisions and improving the stability and reliability of system decisions. Specifically, when combining and simulating candidate schemes, this embodiment of the invention follows a triple objective of minimizing safety risk, minimizing convenience loss, and meeting emergency response standards. Ultimately, a target decision scheme group that simultaneously satisfies the above three objectives can be selected, achieving an optimal balance between safety and user experience, and ensuring that the final decision scheme has sufficient responsiveness. Finally, this embodiment of the invention can also generate interactive reminder information based on a human-computer interaction mechanism while executing device control commands, ensuring both the effectiveness of control and enhancing the user's right to know and user experience.

[0027] Therefore, the embodiments of the present invention significantly improve the security, reliability, and user experience of whole-house intelligent systems in complex scenarios through combined simulation of multiple conflict events and a multi-objective optimization decision-making mechanism. Indicatively, by implementing the present invention, both the security protection capabilities of whole-house intelligent systems can be improved, while also ensuring user convenience and experience.

[0028] For step S1, in some embodiments, the home space corresponding to each monitoring image actually refers to the partitioning of various space types within the residence, such as kitchen, living room, bedroom, balcony, and bathroom; the people involved can be: residents or visitors, such as children, the elderly, adults, and visitors; the home devices can be: various smart / non-smart devices, such as ovens, air conditioners, humidifiers, cameras, and smart door locks; furthermore, a home scene where a child is near the oven in the kitchen, an elderly person is carrying hot soup from the kitchen to the living room, the air conditioner is in cooling mode, and the kitchen floor is slippery can constitute a typical home scene that requires intelligent control.

[0029] For step S2, the present invention achieves the discrimination of behavioral intent from human action recognition through deep correlation and quantitative analysis of multi-dimensional features. In some embodiments, the home status data includes: space usage type, environmental data, and operating data of several different home devices; the image features include: facial features, permission level features, limb features, and trajectory features of each human object; the limb features include: action temporal change rate features and limb posture category features; the trajectory features include: human dwelling area features, human dwelling duration features, and endpoint area features used to characterize the trajectory direction. The specific process for generating the behavioral trajectory intent of the person and the corresponding behavioral trajectory risk level value based on the home status data and the image features in the surveillance images is as follows: Based on the home status data corresponding to the home space, as well as the facial features, permission level features, body features and trajectory features of the person, target scene association features are generated to represent the adaptation relationship between the person and the home space. Based on the target scene association features, home status data, action temporal change rate features, body posture category features, character dwelling area features, character dwelling duration features, and endpoint area features, the behavior trajectory of the character object is temporally modeled, and the behavior trajectory intent of the character object is output. Based on the behavioral trajectory intent, target scene association features, trajectory features, and user permission level features, a multi-dimensional evaluation index system is constructed to determine the risk level of the behavioral trajectory. This multi-dimensional evaluation index system includes the following indicators: intent risk benchmark value, scene security adaptation coefficient, trajectory risk coefficient, and permission violation coefficient. The intent risk benchmark value is determined by the risk level corresponding to the behavioral trajectory intent; the scene security adaptation coefficient is determined based on the quantified value of the target scene association features; the trajectory risk coefficient is determined based on the user's dwell area features, dwell time features, and destination area features; and the permission violation coefficient is determined based on the matching degree between the user's permission level features and the behavioral trajectory intent. Based on the preset risk quantification rules for home scenarios, each indicator in the multi-dimensional assessment indicator system is quantified and assigned a value, and the quantified indicators are weighted and fused to obtain an initial risk level value. Based on the environmental data and the operating data of each home appliance in the home status data, the initial risk level value is calibrated a second time to generate the behavioral trajectory risk level value corresponding to the behavioral trajectory intention.

[0030] As an illustration, image feature extraction from surveillance images can be based on a combined architecture of lightweight object detection models, human pose estimation algorithms, and multi-object tracking algorithms. After extracting the feature vector of a person's facial features, it can be simultaneously linked to the user profile database of the whole-house smart system to match the person's identity, such as a child, elderly person, adult, or visitor within the family. Based on the identity matching results, corresponding access permission levels for home spaces and devices are assigned to derive the person's access permission level characteristics. For example, an adult has high access permission to kitchen devices, a child has low access permission to kitchen devices, and a visitor has limited access permission to all devices in the house. This access permission level characteristic serves to distinguish between legal and illegal / risky actions.

[0031] This invention can also classify posture categories, such as standing, bending over, reaching out to touch, or walking while holding an object, by identifying the coordinate positions of key joints of the human body, thereby obtaining the limb posture category characteristics of each person; and by calculating the displacement change rate of key joints between continuous monitoring frames, the dynamic degree of limb movements can be quantified. A normalized value of 0-1 can be used to represent this, so as to obtain the temporal change rate characteristics of each person's movements. For example, when the temporal change rate of a child quickly reaching out to touch the oven is 0.8, it is marked as a high-dynamic temporal change rate characteristic, and when the temporal change rate of an elderly person slowly carrying soup is 0.3, it is marked as a low-dynamic temporal change rate characteristic.

[0032] Therefore, by introducing temporal modeling and combining target scene association features, limb dynamic features, and trajectory features, this embodiment of the invention can completely capture the continuous sequence of a person's actions, thereby accurately deducing the intent of the behavioral trajectory. Based on the constructed multi-dimensional evaluation index system, which covers four core dimensions—intent risk benchmark value, scene security adaptation coefficient, trajectory risk coefficient, and permission violation coefficient—it can respectively correspond to the risk attributes of the behavior itself, the adaptability of the person and the scene, the risk characteristics of the trajectory, and the permission compliance of the behavior, thus achieving comprehensive coverage of risks.

[0033] In a preferred embodiment, when generating target scene association features, instead of simply concatenating all features, four types of scene association features are calculated separately through modular and dimensional feature analysis and fusion. This allows complex home scenes to be broken down into multiple quantifiable and combinable association dimensions, enabling a more accurate determination of the compatibility relationships between people, spaces, devices, and the environment within a complex home scene. The target scene association features include: a first scene association feature, a second scene association feature, a third scene association feature, and a fourth scene association feature; the limb features also include: coordinate sequence features of limb joints and limb movement amplitude features; the trajectory features also include: temporal coordinate point set features of the person's movement and the person's movement speed features; the home status data also includes: device operation permissions of home devices; the environmental data includes: space temperature, space humidity, and weather data. Therefore, when generating target scene association features to represent the adaptation relationship between a person and a home space based on the home status data corresponding to the home space, as well as the facial features, permission level features, body features, and trajectory features of the person, the specific process includes: The features of space use type, access level, human dwelling area, human dwelling duration, destination area and body posture category are fused together, and a first scene association feature is generated based on the fused features to represent the degree of fit between space use type and human behavior. Based on the operation data of various home appliances, device operation permissions, coordinate sequence characteristics of limb joints, action temporal change rate characteristics, and limb posture category characteristics, operation action features are generated to represent whether a person has device operation actions or not. Based on the operational action characteristics and the limb movement amplitude characteristics, an action risk characteristic is generated to quantify the risk level of the person's current action. Based on the operation action characteristics, the action risk characteristics, and the permission level characteristics of the human object, a second scene association feature is generated to characterize the reasonableness between the human behavior and the operation of home devices. Based on the facial features, the emotional state features of the person are extracted, and based on the coordinate sequence features of limb joints, the amplitude features of limb movements, the rate of change of movement time, the limb posture category features, the time coordinate point set features of the person's movement, and the movement speed features of the person, the movement coordination features are extracted to quantify the movement coordination of the person. Based on the emotional state features, the action coordination features, the character's dwelling area features, and the character's dwelling duration features, a third scene association feature is generated to characterize the fit between the character's state and the character's behavior. Based on the type of space use, space temperature, and space humidity, the first environmental hazard coefficient corresponding to the home space is extracted; Based on the space use type, the first environmental hazard coefficient, and weather data, a second environmental hazard coefficient is generated to quantify the degree of environmental threat posed by weather to the home space. Based on the first scene association features, the second scene association features, the first environmental risk coefficient, and the second environmental risk coefficient, a fourth scene association feature is generated to characterize the safety between the home environment state and human behavior.

[0034] In illustrative terms, the first scene association feature represents the degree of fit between the space's purpose and the person's behavior. By using this feature, we can determine whether it's reasonable for a person to do something in that space, providing a basis for scene rationality in subsequent intention judgments and significantly reducing misjudgments. For example, if an adult stays in the kitchen and approaches the stove, it indicates a high degree of fit between the space's purpose and behavior, so the first scene association feature value can be close to or equal to 1. If a child stays in the kitchen and approaches the stove, the fit is low, so the first scene association feature value is lower, perhaps 0.2 or 0.1.

[0035] The second scenario association feature represents the reasonableness of a person's behavior and equipment operation, thus distinguishing whether a person is operating the equipment normally or touching it dangerously. By combining equipment permissions, key movements, and the range of motion, it is possible to accurately determine whether a person has the ability and permission to operate the equipment. For example, when an adult reaches out to open an oven door with a normal range of motion and a high degree of permission matching, the second scenario association feature value is high. When a child reaches out to touch the oven door, but has a low level of permission and abnormal range of motion, the second scenario association feature value is low, indicating that the person's action poses a higher risk.

[0036] The third-scene correlation feature represents the degree of consistency between a person's state and behavior. It incorporates features such as the person's emotional state, motor coordination, and movement speed to determine whether the person is in an abnormal state, such as loss of balance before a fall or dangerous behavior caused by emotional agitation. For example, when an elderly person has low motor coordination, slows down their movement speed, and increases their dwell time, the third-scene correlation feature value is low, indicating a potential risk of falling. If the person is an adult, with stable emotions and coordinated movements, the third-scene correlation feature value is high.

[0037] The fourth scenario-related feature represents the safety of the home environment and human behavior. It integrates data such as spatial temperature, humidity, and weather to determine whether the environment will amplify the risks of human behavior, achieving a joint assessment of environmental and behavioral risks for a more comprehensive risk evaluation. For example, when the kitchen floor is damp, an elderly person approaches the kitchen, and the environmental hazard level is high, the fourth scenario-related feature value is low, indicating low safety. Conversely, when the living room temperature is comfortable, the floor is dry, and adults are moving around normally, the fourth scenario-related feature value is high.

[0038] By calculating the aforementioned four types of scenario association features separately, complex home scenarios can be decomposed into four interpretable association dimensions. This allows the system to comprehensively assess behavioral risks from four perspectives: space, equipment, person status, and environment. Moreover, each type of scenario association feature targets a specific risk source, avoiding information confusion and misjudgments caused by feature stacking in traditional methods. Therefore, this embodiment of the invention directly establishes the association mapping between people, space, and equipment. For example, it clarifies the high-risk association relationships between children, the kitchen, and a running oven to avoid misjudgments caused by isolated data.

[0039] For step S3, in some embodiments, the present invention can generate two different conflict events of different conflict types; illustratively, a conflict event of the trajectory collision type is marked as a first conflict event, and a conflict event of the device operation type is marked as a second conflict event; each of the different conflict events corresponds to a behavior attribute label.

[0040] For the first conflict event, we have: The image features include: the facial features of the person; the behavioral trajectory intent includes: the intention of the moving area to represent the endpoint of the person's movement trajectory and the intention of the action tendency to represent the degree of avoidance tendency during the movement of the person. Therefore, the generation process of the first conflict event includes: Based on the facial features, obtain the historical behavioral habits data and device operation permissions of each of the aforementioned individuals; For each of the aforementioned human objects, based on the historical behavioral habit data, the device operation permissions, the home status data, and the behavioral trajectory risk level value, the behavioral trajectory intent is subjected to scene matching degree verification, permission matching degree verification, device conflict relationship verification, and habit matching degree verification, and the target verification result corresponding to the human object is output; wherein, the target verification result is used to characterize the degree of adaptation between the human object's behavior and the spatiotemporal scene, device permissions, device operating status, and historical behavioral habits; Within the same time window, several behavioral trajectory intentions that have overlapping areas and whose target collision probability is greater than a preset collision probability threshold are combined into a first conflict event of trajectory collision type; wherein, the target collision probability is calculated from the avoidance tendency degree between each of the action tendency intentions. The target inspection results and the risk level values ​​of each behavior trajectory corresponding to the first conflict event are weighted and summed to output the behavior attribute label of the first conflict event; The behavioral attribute tags include: safety attribute tags indicating that the behavior will threaten personal safety or equipment safety; emergency attribute tags indicating that the behavior needs to be responded to in order to avoid further damage; or convenience attribute tags indicating that the behavior attribute is a need behavior of a person or object.

[0041] In illustrative terms, this embodiment of the invention combines historical behavioral habits associated with facial features, device operation permissions, home status data, and behavioral trajectory risk level values ​​to perform multi-dimensional verification of behavioral trajectory intent and output target verification results. Then, based on the spatial overlap characteristics of movement area intent, preliminary screening of trajectory conflicts is completed. Next, the target collision probability calculated from the avoidance tendency is combined to achieve accurate determination of trajectory collision-type conflicts. Simultaneously, a corresponding behavioral attribute label is generated by weighted summation of the target verification results and the behavioral trajectory risk level values. It is evident that this embodiment of the invention not only achieves quantitative identification of spatial overlap risks in human movement trajectories but also combines multi-dimensional information such as user historical habits, scene permissions, and device status to complete behavioral rationality verification. Finally, it can further categorize each behavior in the first conflict event into behavioral attribute labels representing threats to personal and equipment safety, emergency response needs, or routine convenience needs.

[0042] Furthermore, regarding the second conflict event, we have: The conflict types also include: device operation types; device operation permissions include: having operation permissions or not having operation permissions; the behavioral trajectory intent also includes: device operation intent used to represent the user's operational needs for home appliances; wherein, different operational needs correspond to different values ​​of operational rationality. Therefore, the process for generating the second conflict event, whose conflict type is device operation, also includes: Several behavioral trajectory intentions that have no device operation permission and whose operation rationality value is less than a preset operation rationality threshold are combined into a second conflict event whose conflict type is device operation. The target inspection results and the risk level values ​​of each behavior trajectory corresponding to the second conflict event are weighted and summed to output the behavior attribute label of the second conflict event.

[0043] In illustrative terms, in this embodiment of the invention, by jointly determining both device operation permissions and the reasonableness value of the operation, behavioral trajectories that lack corresponding operation permissions and whose operational requests are significantly less reasonable can be defined as device operation conflicts. This avoids misjudgments caused by relying solely on a single permission determination. Therefore, this embodiment of the invention can identify unauthorized device operation behaviors, ensuring the safety of home device use and protecting user privacy boundaries. Furthermore, it can detect abnormal, falsely triggered, or unintentional device operation requests through the reasonableness value of the operation, thereby improving the accuracy and reliability of device operation conflict identification.

[0044] Furthermore, in this embodiment of the invention, the generation of the second conflict event reuses the target inspection result and the behavioral trajectory risk level value already output by the trajectory collision conflict judgment step. The behavioral attribute label of the second conflict event is generated by weighted summation, so that the trajectory collision type and the equipment operation type of conflict events have the same dimension and the same standard attribute labeling system. This ensures that the definitions of the three types of labels, namely safety, emergency, and convenience, are consistent across different conflict types. This allows the invention to accurately distinguish the behavioral type of conflict events while identifying illegal equipment operation.

[0045] Furthermore, in some embodiments, when determining the first conflict event based on the intention to move to a specific region and the intention to act, the specific process is as follows: For each stated behavioral trajectory intention, the minimum circumcircle corresponding to the endpoint pointing region is determined based on the region shape. For each stated behavioral trajectory intention, the center of the minimum circumcircle is adjusted according to the degree of avoidance tendency and the preset center offset, and the radius of the minimum circumcircle is adjusted according to the degree of avoidance tendency and the preset radius offset. Then, the adjusted minimum circumcircle is generated based on the adjusted center and the adjusted radius. Multiple character objects whose adjusted minimum circumcircles overlap within the same time window are combined into a first candidate character combination; For each of the first candidate combinations, the overlapping area between each of the adjusted minimum circumcircles is taken as the target area, and the maximum inscribed circle of the target area is determined; the center distance between the center of the maximum inscribed circle and the center of each of the adjusted minimum circumcircles is calculated, and the maximum center distance is taken as the center distance to be detected; when it is determined that the center distance to be detected is less than a preset center distance threshold, the first candidate combination is taken as the second candidate combination. For each second candidate character combination, the target collision probability is calculated based on the degree of avoidance tendency among the various action tendency intentions; when the target collision probability is determined to be greater than a preset collision probability threshold, the second candidate character combination is marked as the target character combination; For each group of target individuals, the intentional behavioral trajectories of each individual in the group of target individuals are combined into the first conflict event.

[0046] In some embodiments, the adjustment of the minimum circumcircle and the calculation of the target collision probability are quantified in the following way: Center adjustment formula: O adj =O orig +k a *ΔO preset Among them, O orig O is the original center. adj k is the adjusted center of the circle. a The coefficient corresponding to the degree of avoidance tendency, ΔO preset This is a preset center offset vector; Radius adjustment formula: R adj =R orig *(1+k a *ΔR preset ); where R orig R is the original radius. adj The adjusted radius, ΔR preset This is the preset radius offset; Target collision probability formula: Where N is the number of character objects. For the first The coefficient corresponding to the degree of avoidance tendency of each individual. The probability of collision with the target.

[0047] It is understandable that the endpoint of a person's movement trajectory often points to non-standard geometric shapes such as irregular polygons, scattered areas, or blurred coverage areas in real-world scenarios. Directly performing overlap determination or distance calculation on irregular areas requires complex polygon intersection operations and boundary traversal comparisons, resulting in cumbersome computational logic and a large computational load. However, this invention employs a minimum circumscribed circle, which is the smallest standard circle that can completely enclose the target irregular area. This circle can be quickly and uniquely determined using the region's extreme coordinates. Therefore, this invention performs a circle drawing operation on the irregular area pointed to by the endpoint of the person's trajectory, which can significantly simplify the complex calculation of spatial overlap in irregular areas. Since the overlap and distance calculation of the circle only require basic arithmetic operations using the center coordinates and radius values, the calculation steps are greatly reduced compared to the intersection determination of irregular polygons. In the scenario of real-time image analysis and parallel determination of multiple people in smart homes, this invention can reduce the computational load on edge server devices and improve the real-time performance and response speed of conflict recognition.

[0048] Furthermore, the degree of avoidance tendency directly corresponds to the collision risk of a character. The lower the avoidance tendency, the weaker the character's awareness and behavior in actively avoiding obstacles and other characters. The higher the probability of deviating from the planned endpoint area and expanding the contact range during actual movement, the greater the potential collision risk. Conversely, the higher the avoidance tendency, the lower the collision risk. This invention can transform the dimensionless avoidance tendency into a geometric correction parameter with spatial dimensions. By shifting the center of the circle and changing its center position, the radius can be enlarged or reduced to adjust the circle's coverage area. This allows behavioral risk characteristics to be embedded into the spatial geometric model, so that the geometry no longer simply represents the planned area but also the actual possible risk expansion range. Therefore, this invention can expand the circular coverage area for characters with low avoidance and high risk, including objects that do not overlap in the original planned area but are prone to contact during actual movement within the overlap determination range. This solves the problem that fixed geometric areas cannot reflect behavioral differences and high-risk collision events are missed.

[0049] Furthermore, the present invention can also have a two-stage judgment structure based on the adjusted circumscribed circle for coarse screening and the center distance of the largest inscribed circle for fine screening. This can quickly screen out irrelevant objects that have no spatial intersection. Then, by verifying the center distance of the largest inscribed circle, pseudo-conflict combinations that only have slight edge contact but do not have the possibility of actual collision are eliminated, thereby retaining only the second candidate object combination whose center distance to be detected is less than the preset center distance threshold.

[0050] Specifically, the first candidate character combination only indicates that the adjusted circumcircles overlap, but some first candidate character combinations may only have slight contact at the edges of the circles, lacking the spatial conditions for a real collision, and thus constituting a pseudo-collision. To solve this problem, this embodiment of the invention constructs a maximum inscribed circle based on the common overlapping area of ​​multiple adjusted circumcircles. This inscribed circle is the largest circle completely within the overlapping range of all circumcircles, representing the core common risk area of ​​multiple character trajectories. The distance from the center of this inscribed circle to the center of each adjusted circumcircle is calculated, and the maximum value is used for threshold determination. This allows it to determine whether the trajectory centers of each character are concentrated close to the core common risk area, thereby retaining the real potential collision combinations with highly concentrated trajectory centers and a large overlapping area of ​​the common risk area, i.e., the second candidate character combinations. Therefore, the embodiments of the present invention can, on the one hand, transform irregular spatial regions into standard geometric shapes, greatly simplifying the calculation difficulty of spatial overlap and improving the judgment efficiency; on the other hand, they can directly integrate behavioral collision risks into the adjustment of circle parameters, allowing high-risk behaviors to correspond to a larger coverage area, improving the sensitivity of conflict recognition. At the same time, by quickly eliminating irrelevant objects through coarse screening and filtering edge pseudo-overlaps through fine screening, the missed judgments and false judgments are effectively reduced, making the recognition of trajectory collision conflicts more reliable.

[0051] In some embodiments, the intention to move the region includes: the coordinates of the destination region and the coordinate set of the movement path; the intention to move includes: no tendency to avoid or tendency to avoid. Therefore, in the process of generating the first conflict event based on the intention of moving area and the intention of action tendency, the following scheme can also be adopted for generation, specifically: Based on the coordinates of the endpoint regions of different characters, calculate the overlap ratio of the endpoint regions between any two characters within the same time window; Based on the coordinate set of the movement paths of different character objects, calculate the path overlap ratio between any two character objects within the same time window; Multiple character objects whose endpoint region overlap ratio is greater than a preset region overlap ratio threshold and whose path overlap ratio is greater than a preset path overlap ratio threshold are marked as candidate trajectory interaction character combinations. For each candidate trajectory interaction character combination, the collision probability of the candidate trajectory interaction character combination is calculated based on the action tendencies and intentions of each character object in the candidate trajectory interaction character combination. Candidate trajectory interactive character combinations with collision probabilities greater than a preset collision probability threshold are used as target trajectory interactive character combinations. For each target trajectory interaction character combination, the behavioral trajectory intents corresponding to the character objects in the target trajectory interaction character combination are combined into the first conflict event.

[0052] In illustrative terms, in this embodiment of the invention, even in complex scenarios such as living rooms or corridors with multiple people, multiple paths, and multiple devices, the invention calculates the overlap ratio of the endpoint areas of people and the overlap ratio of their movement paths, and combines this with the calculation of the collision probability based on the intention of their actions. This enables the accurate identification of the first conflict event between multiple people in a home setting, improving the real-time performance and reliability of conflict identification.

[0053] In some embodiments, the target verification results when performing four-layer verification on the behavioral trajectory intent of each character object in the embodiments of the present invention include the following four results: first verification result, second verification result, third verification result and fourth verification result; Therefore, when performing scene matching verification, permission matching verification, device conflict relationship verification, and habit matching verification on the behavioral trajectory intent based on the historical behavioral habit data, the device operation permissions, the home status data, and the behavioral trajectory risk level value, and outputting the target verification result corresponding to the person object, the specific process is as follows: Based on the type of space use, environmental data, and behavioral trajectory intent, the rationality between behavioral trajectory intent and home space scene is verified to generate a first verification result; wherein, the first verification result is used to quantify the degree of rationality between the person's behavior and the home space scene; Based on device operation permissions and behavioral trajectory intent, the matching degree between the person's behavior and device permissions is verified, and a second verification result is generated; wherein, the second verification result includes: complete unauthorized access, partial unauthorized access, or complete permission matching; Based on the operational data of each home appliance, the intent of the behavioral trajectory, and the risk level value of the corresponding behavioral trajectory intent, the conflict relationship between the behavioral trajectory intent and the operating status of the appliances is verified, and a third verification result is generated. The third verification result includes: no appliance conflict or appliance conflict. When the third verification result is that appliance conflict exists, the third verification result also includes the name of the target home appliance that conflicts with the behavior of the person. Based on the historical behavioral habit data and behavioral trajectory intent corresponding to the person, the consistency between the behavioral trajectory intent and the historical habits is verified to generate a fourth verification result; wherein, the fourth verification result is used to quantify the habit matching degree between the person's behavior and historical habits.

[0054] In illustrative terms, embodiments of the present invention can comprehensively verify the rationality and security of a person's trajectory intent from four dimensions: space, permissions, device, and habits. It can accurately determine whether the behavior is risky, whether the permissions are matched, whether the behavior conflicts with the environment, and whether the behavior conforms to the user's habits. Based on the multi-dimensional verification mechanism, it can effectively identify potential risks and avoid misjudgments caused by single-dimensional judgments, thereby providing a reliable basis for subsequent conflict event identification.

[0055] In a preferred embodiment, the home scenario is a living room connected to a balcony. The living room is used as a leisure activity area, and the balcony is used for drying clothes and caring for plants. Current environmental data shows an outdoor temperature of 32°C and moderate indoor humidity. Clothes dried at high temperatures are being dried on the balcony, and the living room air conditioner is in cooling mode. After acquiring real-time monitoring images of this home scenario, two individuals are identified through image features: child A and adult B. Child A has low access privileges, with device operation permissions limited to the living room light switch. Historical behavior shows that child A enters the balcony area infrequently and has no record of touching the clothes drying or the plants on the balcony. Adult B has high access privileges, possessing operation permissions for all home devices. Historical behavior shows that adult B frequently enters the balcony between 3 PM and 4 PM to collect clothes and water plants.

[0056] Furthermore, we can deduce that child A's behavioral trajectory intention is: to quickly run from the living room sofa area to the balcony clothes drying rack; the intended area of ​​movement is the area where the balcony clothes drying rack is located; the action tendency intention is no avoidance tendency, i.e., low degree of avoidance; the equipment operation intention is to want to touch the hot clothes drying rack on the balcony. Adult B's behavioral trajectory intention is: to walk from the living room / dining area to the balcony green plant area; the intended area of ​​movement is the area where the balcony green plant area is located; the action tendency intention is some avoidance tendency, i.e., high degree of avoidance; the equipment operation intention is to want to use the balcony water tap to water the plants. First, the behavioral trajectories and intentions of the two individuals are verified for scene matching, permission matching, device conflict, and habit matching, and the respective target verification results are output. For child A, in terms of scenario matching, their activities and intentions to operate related equipment in the high-temperature drying rack area on the balcony do not match the purpose of the balcony space and the high-temperature environment, with a quantitative matching degree of 20%, which is lower than the corresponding threshold and is judged as low scenario matching. In terms of permission matching, they only have permission to operate the living room lighting. Entering the high-risk area of ​​the balcony and having the intention to operate related equipment constitutes an overreach of permission. In terms of equipment conflict, their intention to approach hot clothing and operate related equipment could easily cause burns, and accidentally touching the faucet could also cause the floor to become slippery, indicating a conflict of equipment. In terms of habit matching, they rarely enter the balcony in the past and have no habit of operating related equipment. The consistency between their behavior and equipment operation intentions and their historical habits is only 10%. Due to their low avoidance level and lack of initiative to avoid risks, the risk level can be further increased. Based on the comprehensive four-layer verification and avoidance level assessment, child A is ultimately judged to have high-risk behavior, and the risk level value of their behavior trajectory is quantified as 90 points out of 100. For adult B, in terms of scenario matching, their intention to move towards the balcony greenery area and operate the watering equipment is consistent with the purpose of the balcony space and adapts to the indoor and outdoor environment, with a quantitative matching degree of 95%, which is judged as a high scenario matching degree. In terms of permission matching, they have high permissions, and their intention to enter the balcony and operate the watering equipment are in accordance with permission requirements, which is a complete permission matching. In terms of equipment conflict verification, their activities in the balcony greenery area and watering operations do not conflict with the equipment operation status and pose no safety risks, which is judged as no equipment conflict. In terms of habit matching, their intention to enter the balcony and operate the watering equipment during this period is consistent with historical habits by 90%, and their degree of avoidance is high, showing an awareness and tendency to actively avoid risks, which can further reduce the risk level. Based on the comprehensive four-layer verification and avoidance degree assessment, adult B's behavior is finally judged as low risk, and the risk level value of the behavior trajectory is quantified as 20 points.

[0057] Furthermore, the areas where the endpoints of the two individuals' trajectories point are transformed into minimum circumscribed circles to standardize the irregular regions and simplify calculations. Then, the parameters of the circles are dynamically adjusted based on the risk level and avoidance degree of the two individuals. At this point, the lower the avoidance degree, the higher the risk and the larger the coverage area. Subsequently, potential collision combinations are quickly coarsely screened out by the overlap of the adjusted circumscribed circles. Then, a fine screening is performed based on the maximum center distance of the inscribed circles in the overlapping areas to eliminate pseudo-collisions with slight edge overlap. Finally, the collision probability is quantified by combining the risk level value and the avoidance degree to confirm that there is a risk of trajectory collision between child A and adult B, thus generating the first conflict event.

[0058] Furthermore, since child A has no habit of playing in the balcony area and does not have the right to use the related equipment, and since he has the intention to touch the high-risk equipment while playing in the high-risk area of ​​the balcony, his behavior is not reasonable and meets the conditions for generating a conflict related to equipment operation. Therefore, a second conflict event can be generated. Adult B has the right to play on the balcony, his behavior is highly reasonable, and his intention to use the water tap to water plants is reasonable. Therefore, it does not constitute a conflict.

[0059] Finally, this invention performs a weighted summation of the target inspection results and behavioral trajectory risk level values ​​for the two types of conflict events, and outputs corresponding behavioral attribute labels. The first conflict event, where child A and adult B face a risk of trajectory collision, is classified as a safety event; the second conflict event, where child A is active in a high-risk area on the balcony and faces a risk of touching high-risk equipment, is classified as an emergency event. In summary, this invention achieves comprehensive perception and accurate judgment of multiple types of conflicts in complex home environments by combining multi-dimensional verification, geometrical precise judgment, intent recognition, and risk quantification.

[0060] For step S4, the present invention can automatically select a set of target decision schemes that are safest, most convenient and meet emergency requirements from a large number of candidate decision schemes in complex home scenarios with multiple conflicting events, through preset decision constraints and combined simulation mechanisms, thereby realizing safe, reliable and humanized control of the whole-house intelligent system.

[0061] In some embodiments, the preset decision constraints include: rigid constraints on safety risk thresholds, constraints on meeting emergency response levels, constraints on adaptability upper limits for convenience, and constraints on device coordination to characterize the absence of conflicting device control commands. The process of determining the target decision scheme group includes: Based on the behavioral attribute tags corresponding to the first conflict event and the second conflict event, the priorities corresponding to the first conflict event and the second conflict event are determined respectively; wherein, the priority of the safety attribute tag is greater than the priority of the emergency attribute tag; the priority of the emergency attribute tag is greater than the priority of the convenience attribute tag; Based on a pre-defined mapping rule base between conflict and solution, several candidate decision schemes corresponding to the first conflict event and several candidate decision schemes corresponding to the second conflict event are determined; wherein, the candidate decision schemes include: several candidate device control commands and several candidate machine interaction mechanisms; Based on the priorities corresponding to the first conflict event and the second conflict event, the candidate decision schemes are traversed until several candidate decision scheme groups are output; wherein, during each traversal, a candidate decision scheme is extracted from each candidate decision scheme corresponding to the first conflict event and each candidate decision scheme corresponding to the second conflict event, and combined into a candidate decision scheme group. Based on rigid constraints of safety risk threshold, emergency response level compliance constraints, convenience upper limit adaptation constraints, and equipment coordination constraints used to characterize the non-conflictualization of equipment control commands, each candidate decision scheme group is simulated. When a candidate decision scheme group satisfies the minimum safety risk value, the minimum convenience loss of the scheme, and the maximum emergency response level of the scheme, the output is the target decision scheme group.

[0062] Specifically, the process of generating several candidate decision scheme groups includes: A tree structure is constructed with the combination of conflict events corresponding to the first conflict event and the second conflict event as the root node, and the first conflict event and the second conflict event as different first-level child nodes; wherein, the priority of the first conflict event is used as the node attribute of the first-level child node corresponding to the first conflict event; the priority of the second conflict event is used as the node attribute of the first-level child node corresponding to the second conflict event. Starting from the root node, several different first-level child nodes are randomly selected from the tree structure to generate a first node path group; wherein, the first node path group contains several different first node paths; the first node path contains one or more first-level child nodes. The first node path is filtered according to a preset path selection rule. Then, according to each of the candidate decision schemes, the node paths in the filtered first node path group are expanded into secondary nodes to construct several target node paths to be simulated. The preset path selection rule is used to characterize that the priority of the previous node in the node path is greater than the priority of the next node. For each target node path to be simulated, extract a second-level child node corresponding to each first-level child node in the target node path to be simulated, generate a candidate decision scheme group, until the extraction is completed, and output several candidate decision scheme groups.

[0063] Furthermore, the construction process of several target node paths to be simulated includes: According to the preset path selection rules, delete the first node paths in the first node path group that do not conform to the preset path selection rules, and mark the first node paths that have not been deleted as second node paths. For each second node path, the first-level child node with the smallest relative distance to the root node is taken as the first target child node. Then, the candidate decision scheme corresponding to the first target child node is expanded into the second-level child nodes corresponding to the first target child node to construct the third node path; wherein, the first target child node corresponds to multiple different second-level child nodes. For each of the third node paths, the candidate device control instructions in the second-level sub-nodes corresponding to the first target sub-node are taken as the first candidate device control instructions. The first candidate device control instructions are identified, and it is determined whether the action corresponding to the first candidate device control instructions is an interception behavior. If yes, the third node path is marked as the initial node path to be simulated. If no, the second-level sub-nodes corresponding to the first candidate device control instructions are deleted from the third node path, and then the initial node path to be simulated is constructed. The interception behavior is used to indicate the action instructions to prevent the occurrence of security risks. For each initial node path to be simulated, the initial node path to be simulated is adjusted according to the candidate decision schemes corresponding to the first-level child nodes that have not been expanded into second-level nodes in the initial node path to be simulated, thereby constructing the target node path to be simulated. The specific process of constructing the path of the target node to be simulated is as follows: For each initial node path to be simulated, the first-level child node in the initial node path to be simulated that has not been expanded into a second-level node is taken as the second target child node. Then, the candidate decision scheme corresponding to the second target child node is expanded into the second-level child node corresponding to the second target child node to construct the fourth node path. For each of the fourth node paths, the candidate device control instructions in the second-level sub-nodes corresponding to the second target sub-node are used as second candidate device control instructions. The second candidate device control instructions are identified to determine whether the action corresponding to the second candidate device control instructions is a required behavior. If yes, the fourth node path is marked as the target node path to be simulated. If no, the second-level sub-nodes corresponding to the second candidate device control instructions are deleted from the fourth node path, thereby constructing the target node path to be simulated. The required behavior is used to indicate action instructions that can satisfy the character's needs. The initial node path to be simulated that does not have any unexpanded secondary nodes is directly marked as the target node path to be simulated.

[0064] It is understood that the embodiments of the present invention achieve multi-objective optimization decision-making for complex scenarios through mechanisms such as priority sorting, tree structure modeling, path filtering, action type recognition, and combined simulation.

[0065] In a preferred embodiment, when selecting a set of target decision schemes from a large number of candidate decision schemes for multiple conflict events, which minimizes the safety risk, convenience loss, and emergency response capability, the present invention can automatically select the target decision scheme set from the large number of candidate decision scheme sets using the Particle Swarm Optimization (PSO) algorithm. Illustratively, the PSO algorithm is an iterative optimization method based on swarm intelligence. Its core idea is to simulate the information sharing and collaborative optimization behavior among individuals during bird foraging, and to efficiently search for the optimal solution in the solution space by continuously iterating and updating the particle position and velocity. In this invention, each particle represents a set of candidate decision schemes. By learning its own historical optimal position and the group's historical optimal position, the particle gradually converges towards the target decision scheme set that is safest, most convenient, and meets emergency response requirements, ultimately outputting the optimal solution that satisfies all preset decision constraints.

[0066] Specifically, based on the safety risk value, the loss of convenience of the solution, and the emergency response level of the solution, this invention constructs the following objective function: ; in, The value of the objective function. For the candidate decision-making scheme group, This is the function used to calculate the safety risk value. For the function used to calculate the convenience of the scheme, when The larger the value, the greater the loss of convenience in the solution (i.e., The smaller, The function used to calculate the emergency response level of a plan. , , They are respectively , , The corresponding weighting coefficients.

[0067] Indicatively, in the process of multi-objective optimization, this invention actually minimizes... This leads to a candidate decision scheme group X that satisfies the minimum safety risk value, the minimum convenience loss of the scheme, and the maximum emergency response level of the scheme, and marks this candidate decision scheme as the final target decision scheme.

[0068] Furthermore, the function used to calculate the safety risk value for: ; in, The number of conflict events For the first One conflict event, Indicates the first The total risk level value of all behavioral trajectories in a conflict event, corresponding to the intent of each behavioral trajectory. For the first The residual risk coefficient of a candidate decision-making scheme group for a conflict event is obtained by quantitatively weighting the control commands of each candidate device and the candidate-machine interaction mechanism in the candidate decision-making scheme group. (Illustratively, through...) It can comprehensively consider the risks and intervention effects of all conflict events to obtain the overall security risk value of candidate decision scheme X.

[0069] Functional expression for calculating the convenience of a solution for: ; in, The number of people or objects involved in the conflict. For the first Individual objects, Indicates the first The degree of matching of the historical habits of each person is measured on a scale of 0-100. For the candidate decision-making group, the first The degree to which the needs of each individual target are met is quantitatively weighted and calculated through the interaction mechanism of each candidate in the candidate decision-making scheme group. (Illustratively, through...) It can measure how well candidate decision-making solutions X meet user habits and needs; the higher the value, the more convenient it is.

[0070] Function used to calculate the emergency response level of a plan for: ; in, The number of conflict events with the behavior attribute tag "emergency". For the first A conflict event with a behavioral attribute tag classified as an emergency. For the first Each behavioral attribute tag represents the emergency response level for emergency-related conflict events, which can be quantified by a numerical level of 1–5, with higher levels indicating greater urgency. For candidate decision-making scheme group X, the first Each behavioral attribute label represents the response compliance coefficient for emergency-type conflict events. A value of 1 indicates that the standard has been met. A value of 0 indicates that the standard is not met. This is illustrative, using... It can measure whether candidate decision-making scheme X can handle emergency conflict events in a timely and effective manner. The higher the value, the higher the emergency response level of the candidate decision-making scheme.

[0071] When using the Particle Swarm Optimization (PSO) algorithm to automatically select a target decision set from a large number of candidate decision sets, the specific process is as follows: Several particles are randomly generated. Each candidate decision scheme group code corresponds to one particle. Each dimension of the particle corresponds to the candidate decision scheme number of a conflict event. According to the speed and position update rules of the particle swarm optimization algorithm, the particles are iteratively updated until convergence. Then, the decision scheme group corresponding to the globally optimal particle is taken as the target decision scheme group with the minimum safety risk value, the minimum convenience loss of the scheme, and the maximum emergency response degree of the scheme.

[0072] In one embodiment, the present invention may also select a non-dominated sorting genetic algorithm for multi-objective optimization decision-making. Illustratively, the non-dominated sorting genetic algorithm (NSGA) is a multi-objective intelligent optimization algorithm based on genetic evolution and non-dominated sorting. It is primarily applicable to scenarios with multiple mutually constrained optimization objectives that are difficult to quantify individually. For example, in this invention, the three parallel optimization objectives are minimizing safety risk, minimizing solution convenience loss, and maximizing emergency response capability. By simulating the evolutionary process of natural selection, crossover, and mutation in biology, and combining non-dominated sorting with crowding calculation mechanisms, it selects a Pareto optimal solution set that considers all optimization objectives from a massive pool of candidate solutions, ultimately outputting the optimal decision scheme that satisfies preset constraints, thus adapting to the multi-objective optimization decision-making requirements of this invention under multiple conflicting events.

[0073] In a preferred embodiment, the home scenario is a kitchen and dining room connected. The kitchen is designated as a cooking area, and the dining room as a dining area. Current environmental data shows an outdoor temperature of 28°C and high ground humidity. After acquiring the monitoring images, the invention identifies two individuals: child C and elderly person D. Child C has low access privileges, and historical behavior shows a low frequency of entering the kitchen area; elderly person D has medium access privileges, and historical behavior shows a high frequency of entering the kitchen to get water or retrieve items during the specified time period.

[0074] Child C's behavioral trajectory intention is: to run quickly towards the oven area and intend to open the oven door. The intention of the movement area includes the coordinates of the destination area where the oven is located and the corresponding set of movement path coordinates. The action tendency intention is: no tendency to avoid, low degree of avoidance tendency. The equipment operation intention is: to open the oven.

[0075] The intention of the elderly D's behavior trajectory is: to walk from the restaurant to the kitchen sink area and turn on the faucet to get water. The intention of his movement area includes the area where the sink is located as the destination area. The intention of his action tendency is to avoid the sink, and the degree of avoidance tendency is high. The intention of equipment operation is to turn on the faucet.

[0076] First, the present invention can calculate the corresponding behavioral attribute tags based on the conflict events generated in the above-described scenario, and determine the priority of each conflict event accordingly. Then, based on a preset mapping rule base, multiple candidate decision schemes are generated for each conflict event. The candidate decision schemes can provide multiple possible solutions for each conflict. For example, when a conflict event involves a child approaching an oven, the selectable candidate decision schemes include: turning off the oven, issuing an alarm, locking the oven door, and reminding parents.

[0077] The tree structure of conflict events constructed in this embodiment of the invention is as follows: root node: a combination of all conflict events; first-level child nodes: conflict events; the number of first-level child nodes is the number of conflict events; the node attribute of the first-level child nodes: the priority of the conflict events. By constructing a tree structure, the hierarchical and priority relationships between conflicts can be reflected, thereby structuring the complex problem of multiple concurrent conflicts and facilitating subsequent path search and solution combination.

[0078] Furthermore, in this embodiment of the invention, several first-level child nodes can be randomly selected from the root node to generate multiple first-node paths. Based on the path selection rules—that is, the principle that the priority of the preceding node is higher than that of the following node—first-node paths that do not conform to the rules are deleted. This ensures that the conflict events remaining in the second-node paths are arranged in descending order of priority, preventing low-priority conflicts from being processed first and causing high-priority conflicts to be ignored, and further reducing the computational load of subsequent combination simulations.

[0079] After generating the second node path, the reason for first expanding the highest-priority first target child node to generate the third node path, and then generating the path to be simulated based on the third node path, is to ensure that the entire decision-making process always follows the principle of prioritizing the handling of high-priority conflicts. That is, by prioritizing the expansion of candidate decision schemes corresponding to high-priority conflicts, interception behaviors that can directly prevent danger from occurring can be introduced at the forefront of the decision path, thereby ensuring the safety and effectiveness of all subsequent decision paths and avoiding the inability to resolve high-priority conflicts in a timely and effective manner due to the early handling of low-priority conflicts.

[0080] After generating the third node path, this invention further determines whether the first candidate device control command is an interception behavior. This allows for the further screening of key actions that can eliminate or significantly reduce security risks, pruning candidate schemes that lack interception capabilities in advance. This reduces the number of subsequent paths to be simulated, improves decision-making efficiency, and reduces computational complexity. Subsequently, after obtaining the initial path to be simulated, this invention expands the initial path to be simulated for first-level sub-nodes that have not been expanded to second-level nodes, and generates the target path to be simulated. This allows for the introduction of non-interception-related action commands relevant to user needs, while ensuring that high-priority conflicts have been handled. This ensures that the final generated candidate decision scheme group meets both security requirements and user experience and convenience, achieving a balance between security and convenience. Therefore, this embodiment of the invention, by first expanding the first target sub-node to generate the third node path and then generating the path to be simulated, not only ensures the security and effectiveness of the decision path but also reduces the number of invalid paths through early pruning, significantly improving the speed and reliability of subsequent decision simulation. It can quickly and accurately output the optimal target decision scheme group in complex home environments with multiple conflicts.

[0081] Indicatively, the execution order of each scheme in the candidate decision scheme group generated by this invention is consistent with the extraction order. The execution order of the second-level nodes extracted first from the target simulated node path is higher than the execution order of the second-level nodes extracted later. Schemes corresponding to high-priority conflicts, such as reminding the elderly to perform trajectory collision conflicts, will be executed first, while schemes corresponding to low-priority conflicts, such as locking the oven door to perform equipment operation conflicts, will be executed later. This ensures that high-risk conflicts are handled first, avoiding secondary risks.

[0082] The two types of conflict events derived from the above-mentioned home scenario where the kitchen and dining room are connected are: The first conflict event is a trajectory collision between child C and elderly person D; child C's intention is to open the oven, which involves high-risk equipment operation and unauthorized access, thus generating the second conflict event, an equipment operation-related conflict event. Therefore, the tree structure constructed based on these two different conflict events is as follows: Figure 2 As shown.

[0083] Furthermore, starting from the root node, several first-level child nodes are randomly selected to generate a first node path group, including the following first node paths: First node path 1 is: first conflict event; First node path 2 is: second conflict event; The order of first node path 3 is: first conflict event, second conflict event; The order of first node path 4 is: second conflict event, first conflict event. According to the preset path selection rule that the priority of the preceding node must be higher than that of the following node, the first node path 4 is deleted, resulting in the following second node paths: Second node path 1 is: First conflict event; Second node path 2 is: Second conflict event; The order of second node path 3 is: First conflict event, Second conflict event; For each second-node path, the first-level child node with the smallest distance from the root node is selected as the first target child node. In this embodiment, when the first target child node is a first-level child node 1, it corresponds to the first conflict event between child C and elderly person D, and its candidate decision schemes are expanded to second-level child nodes. Illustratively, the candidate decision schemes for the first conflict event include: sending a reminder to elderly person D that there is a child ahead, controlling the kitchen door light to turn red, giving a voice prompt to child C to slow down, and activating a floor slippery warning. When the first target child node is a first-level child node 2, it corresponds to the second conflict event for child C, and its candidate decision schemes are expanded to second-level child nodes. Illustratively, the candidate decision schemes for the second conflict event include locking the oven door, triggering an alarm on the parent's app, turning off the oven power, and activating the exhaust system's strong fan. At this point, three third-node paths are constructed, such as... Figure 3 As shown.

[0084] Furthermore, it is determined whether the candidate device control commands of the second-level child nodes of the first target child node in the aforementioned third node path are interception behaviors. In this embodiment of the invention, all candidate device control commands of the second-level child nodes are set to be actions capable of preventing collisions. Therefore, all second-level child nodes are retained, and all third node paths are marked as initial node paths to be simulated.

[0085] For the unexpanded second target child node in the initial node path to be simulated, such as Figure 3 As shown, the first-level node 2 in the third node path 2, which has been marked as the initial simulated node path, can be used as the second target child node. The candidate decision schemes for the second conflict event corresponding to the first-level node 2 include: locking the oven door, parent APP alarm, turning off the oven power, and starting the exhaust system's strong fan. Further, it is determined whether the action corresponding to the second candidate device control command of the second-level child node of the above-mentioned second target child node is the required behavior. If turning off the oven power and starting the exhaust system's strong fan do not meet the user's needs, then the second-level child nodes containing the forced shutdown of the oven power and the activation of the exhaust system's strong fan are deleted. The initial simulated node path after deleting the above two second-level child nodes is marked as the target simulated node path. In this embodiment of the invention, three target simulated node paths are constructed, as follows: Figure 4 As shown.

[0086] Indicative, in Figure 4The path 3 of the target node to be simulated retains the options of forcibly shutting off the oven power and activating the strong exhaust system to preserve a more complete decision space and avoid the loss of the optimal solution due to premature pruning. Although shutting off the oven power and activating the strong exhaust system are determined to be non-required actions in the fourth node path, they are still effective safety actions. In some extreme scenarios, such as when the oven has malfunctioned or is emitting smoke, these actions may be the only option to prevent a fire. Therefore, by retaining the non-required actions in the target node path 3, this embodiment of the invention preserves more possibilities for subsequent multi-objective simulations. During multi-objective simulation, this invention needs to calculate the convenience loss value of each option. Retaining these non-required actions, which are high in safety but low in convenience, allows for comparison with options that balance safety and convenience, thereby more accurately selecting the optimal option with the lowest safety risk and lowest convenience loss.

[0087] Furthermore, second-level child nodes are extracted level by level from the path of the target node to be simulated. When extracting the second-level node corresponding to a first-level node, only one solution corresponding to the second-level node is extracted, thus generating multiple solution groups. For illustration, the following solution groups may exist: Solution Group 1: Send a reminder to the elderly D that there is a child ahead; Solution Group 2: The kitchen door light turns red; Solution Group 3: Send a reminder to the elderly D that there is a child ahead and lock the oven door; Solution Group 4: The kitchen door light turns red and locks the oven door; Solution Group 5: Activate a slippery floor warning and alert the parent's app. For illustration, the remaining combined solutions are extracted sequentially according to the path logic, forming a total of 16 valid solution groups.

[0088] Subsequently, all scheme groups were simulated and verified based on preset decision constraints. In an optional embodiment, the optimal target decision scheme group was finally selected as scheme group 3. This scheme group, by reminding the elderly D and locking the oven door, minimizes safety risks, ensures optimal convenience loss, and is fully adapted to the high-risk scenario characteristics of the kitchen and dining room, making the emergency response level of the scheme higher than the preset emergency response level. Illustratively, in this embodiment, reminding the elderly and locking the oven door in scheme group 3 simultaneously meet the following requirements: preventing children from colliding with the elderly, preventing children from accidentally operating the oven, not affecting the normal use of the kitchen, and having a high emergency response level, thus becoming the optimal scheme.

[0089] It is understood that the embodiments of the present invention require that the preceding node has a higher priority than the following node through preset path filtering rules, thus ensuring the rationality of the processing order. In this embodiment, the first conflict event in which a child and an elderly person may collide has a higher priority than the second conflict event; therefore, the first node path 4 is deleted, ensuring safety as the priority.

[0090] By expanding the first target sub-node and identifying interception behaviors, high-risk conflicts can be ensured to be controlled most effectively. Sending a reminder to the elderly D that there are children ahead, turning the kitchen doorway light red, giving a voice prompt to child C to slow down, and activating a wet floor warning are all interception behaviors. These are actions specifically designed to prevent danger from occurring. By identifying and retaining these actions, the system can ensure that high-risk conflicts are handled in the most timely and effective manner.

[0091] Extending the second target sub-node and identifying required actions aims to balance user needs with system convenience. For conflicts of secondary priority, this invention prioritizes solutions that meet user needs without excessive intervention, avoiding a decline in user experience due to over-control. The required action identification mechanism can distinguish between actions that meet user needs and actions that force intervention. In this embodiment, locking the oven door and issuing alarms via the parent's app are required actions that meet safety requirements without disrupting normal user operation, and are therefore retained; while forcibly shutting off the oven power and using the strong exhaust fan are considered excessive interventions and are therefore removed.

[0092] For step S5, after determining the target decision scheme group in step S4, all target device control commands contained therein are sent to the corresponding home appliances, which then execute specific control actions to achieve automatic adjustment of the home scene. Simultaneously with device control, reminders, prompts, or explanations related to the current scene are sent to the user based on the target human-computer interaction mechanism, thus achieving a closed loop between automatic device control and human-computer interaction feedback. Since device control can directly prevent danger from occurring, such as locking the oven or turning off the gas, while interactive reminders allow users to be aware of risks in a timely manner, the home scene control process of this invention can significantly improve home safety while also considering user convenience and avoiding excessive intervention.

[0093] Please see Figure 5 This application also provides a three-element architecture system for home scenarios, the three-element architecture system including: end-side devices, edge servers, and cloud servers; The edge device is used to collect several monitoring images in a home scene, and then send each monitoring image to the edge server. The edge server is used to determine the home space corresponding to each monitoring image and the home status data corresponding to the home space; wherein, the home scene includes several different human objects; The edge server is also used to generate, for each person in each of the monitoring images, the behavioral trajectory intention of the person and the behavioral trajectory risk level value corresponding to the behavioral trajectory intention, based on the home status data and the image features in the monitoring images; The edge server is also used to generate several conflict events of different conflict types in the home scene based on the image features, the intention of each behavior trajectory, the risk level value of each behavior trajectory, and the home status data, and then send the conflict events of each different conflict type to the cloud server. The cloud server is used to perform combined simulations of decision schemes between each conflict event based on preset decision constraints. When a candidate decision scheme group satisfies the minimum security risk value, the minimum loss of convenience, and the maximum emergency response level, the candidate decision scheme group is taken as the target decision scheme group. The target decision scheme group includes: several target device control commands and several target human-computer interaction mechanisms. The cloud server is also used to send control instructions for each of the target devices to the end device, so that the end device can control the home devices in the home scene according to the control instructions for each of the target devices; The cloud server is also used to send each of the target human-computer interaction mechanisms to the edge server, so that the edge server generates a number of interactive reminder messages for information transmission with the human object based on each of the target human-computer interaction mechanisms, and the edge server sends the interactive reminder messages to the corresponding terminal device for execution output.

[0094] In a preferred embodiment, such as Figure 6 The schematic diagram of the three-element architecture system shown illustrates the module structure. The edge devices include: a monitoring image acquisition device, a home automation device, and an interactive output device. The monitoring image acquisition device captures several monitoring images of the home environment and sends them to the edge server. The home automation device receives control commands from the cloud server to control home appliances, such as locking the oven door or adjusting lights. The interactive output device receives interactive notifications from the edge server and outputs them to the user, such as voice prompts, light alerts, and app push notifications.

[0095] The edge server comprises an image analysis module, a behavior intent generation module, and a conflict recognition module. The image analysis module receives monitoring images from the edge devices and determines the corresponding home space and home status data for each image, such as space usage, environmental data, and equipment operation data. The behavior intent generation module combines home status data with image features to generate the behavioral trajectory intent of a person or object and its corresponding risk level. The conflict recognition module identifies conflict events based on image features, behavioral intent, risk levels, and home status data, and then sends the results to the cloud server. The conflict recognition module also receives target human-computer interaction mechanisms from the cloud server, generates interactive reminder information, and sends it to the edge device's interactive output device.

[0096] The cloud server comprises a decision constraint module, a solution combination simulation module, and a target solution output module. The decision constraint module provides preset decision constraints, such as security thresholds, emergency levels, convenience limits, and device collaboration constraints, as the basis for solution selection. The solution combination simulation module simulates the combination of candidate decision solutions for conflict events to verify whether they meet multi-objective optimization conditions. The target solution output module selects target decision solution groups that meet the conditions, sends target device control commands to the edge device, and sends the target human-machine interaction mechanism to the edge server.

[0097] It is understood that the content of the above method embodiments is applicable to the embodiments of this ternary architecture system. The specific functions implemented by the embodiments of this ternary architecture system are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0098] It should be noted that the ternary architecture system described above is merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the ternary architecture system embodiment provided by this invention, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0099] Those skilled in the art will understand that, for convenience and simplicity, the specific working process of the ternary architecture system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0100] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned home scene control method. This electronic device can include any smart terminal such as a tablet computer or an in-vehicle computer.

[0101] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0102] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting various parts of the terminal device via various interfaces and lines.

[0103] The memory can be used to store the computer program. The processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, SmartMediaCard (SMC), Secure Digital (SD) card, FlashCard, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0104] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned home scene control method.

[0105] It is understood that the content of the above method embodiments is applicable to the present computer storage medium embodiments. The specific functions implemented by the present computer storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0106] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for controlling a home scene.

[0107] It is understood that the content of the above method embodiments is applicable to the embodiments of this computer program product. The specific functions implemented by the embodiments of this computer program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0108] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, the ternary architecture system, and the functional modules / units in the device can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0109] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for controlling a home scene, characterized in that, The method includes: Acquire several surveillance images in a home scene, and determine the home space corresponding to each surveillance image and the home status data corresponding to the home space; wherein, the home scene contains several different human objects; For each person in each of the aforementioned surveillance images, based on the home status data and the facial features of the person in the surveillance image, a behavioral trajectory intention is generated for the person. This behavioral trajectory intention includes: a movement area intention representing the endpoint of the person's movement trajectory; an action tendency intention representing the degree of avoidance tendency during the person's movement; and a device operation intention representing the person's operational needs regarding home appliances. Different operational needs correspond to different operational rationality values. Several behavioral trajectory intentions that have overlapping areas within the same time window and whose target collision probability is greater than a preset collision probability threshold are combined into a first conflict event of trajectory collision type; wherein, the target collision probability is calculated from the avoidance tendency degree between each of the action tendency intentions. Based on the facial features, device operation permissions for each of the aforementioned individuals are obtained; wherein, the device operation permissions include: having operation permissions or not having operation permissions. Several behavioral trajectory intentions that have no device operation permission and whose operation rationality value is less than a preset operation rationality threshold are combined into a second conflict event with the conflict type of device operation. Based on preset decision constraints, a combination simulation is performed on the decision schemes between each first conflict event and each second conflict event. When a candidate decision scheme group satisfies the minimum safety risk value, the minimum convenience loss of the scheme, and the maximum emergency response level of the scheme, the candidate decision scheme group is taken as the target decision scheme group; wherein, the target decision scheme group includes: a number of target equipment control commands and a number of target human-computer interaction mechanisms. The home appliances in the home scene are controlled according to the control commands of each target device; based on the human-computer interaction mechanism of each target, several interactive reminder messages are generated for information transmission with the human object.

2. The home scene control method according to claim 1, characterized in that, The intent behind the behavioral trajectory corresponds to a risk level value for that behavioral trajectory. Both the first conflict event and the second conflict event correspond to a behavior attribute tag; The generation of the behavioral attribute tags includes: Based on the facial features, obtain historical behavioral habit data for each of the aforementioned individuals; For each of the aforementioned human objects, based on the historical behavioral habit data, the device operation permissions, the home status data, and the behavioral trajectory risk level value, the behavioral trajectory intent is subjected to scene matching degree verification, permission matching degree verification, device conflict relationship verification, and habit matching degree verification, and the target verification result corresponding to the human object is output; wherein, the target verification result is used to characterize the degree of adaptation between the human object's behavior and the spatiotemporal scene, device permissions, device operating status, and historical behavioral habits; The target inspection results and the risk level values ​​of each behavior trajectory corresponding to the first conflict event are weighted and summed to output the behavior attribute label of the first conflict event; The target inspection results and the risk level values ​​of each behavior trajectory corresponding to the second conflict event are weighted and summed to output the behavior attribute label of the second conflict event; The behavioral attribute tags include: safety attribute tags indicating that the behavior will threaten personal safety or equipment safety; emergency attribute tags indicating that the behavior needs to be responded to in order to avoid further damage; or convenience attribute tags indicating that the behavior attribute is a need behavior of a person or object.

3. The home scene control method according to claim 2, characterized in that, The generation process of the first conflict event includes: For each stated behavioral trajectory intention, the minimum circumcircle corresponding to the endpoint pointing region is determined based on the region shape. For each stated behavioral trajectory intention, the center of the minimum circumcircle is adjusted according to the degree of avoidance tendency and the preset center offset, and the radius of the minimum circumcircle is adjusted according to the degree of avoidance tendency and the preset radius offset. Then, the adjusted minimum circumcircle is generated based on the adjusted center and the adjusted radius. Multiple character objects whose adjusted minimum circumcircles overlap within the same time window are combined into a first candidate character combination; For each of the first candidate combinations, the overlapping area between each of the adjusted minimum circumcircles is taken as the target area, and the maximum inscribed circle of the target area is determined; the center distance between the center of the maximum inscribed circle and the center of each of the adjusted minimum circumcircles is calculated, and the maximum center distance is taken as the center distance to be detected; when it is determined that the center distance to be detected is less than a preset center distance threshold, the first candidate combination is taken as the second candidate combination. For each second candidate character combination, the target collision probability is calculated based on the degree of avoidance tendency among the various action tendency intentions; when the target collision probability is determined to be greater than a preset collision probability threshold, the second candidate character combination is marked as the target character combination; For each group of target individuals, the intentional behavioral trajectories of each individual in the group of target individuals are combined into the first conflict event.

4. The home scene control method according to claim 3, characterized in that, The process of determining the target decision scheme group includes: Based on the behavioral attribute tags corresponding to the first conflict event and the second conflict event, the priorities corresponding to the first conflict event and the second conflict event are determined respectively; wherein, the priority of the safety attribute tag is greater than the priority of the emergency attribute tag; the priority of the emergency attribute tag is greater than the priority of the convenience attribute tag; Based on a pre-defined mapping rule base between conflict and solution, several candidate decision schemes corresponding to the first conflict event and several candidate decision schemes corresponding to the second conflict event are determined; wherein, the candidate decision schemes include: several candidate device control commands and several candidate machine interaction mechanisms; Based on the priorities corresponding to the first conflict event and the second conflict event, the candidate decision schemes are traversed until several candidate decision scheme groups are output; wherein, during each traversal, a candidate decision scheme is extracted from each candidate decision scheme corresponding to the first conflict event and each candidate decision scheme corresponding to the second conflict event, and combined into a candidate decision scheme group. Based on preset decision constraints, each candidate decision scheme group is simulated. When a candidate decision scheme group satisfies the minimum safety risk value, the minimum convenience loss of the scheme, and the maximum emergency response level of the scheme, the output is the target decision scheme group.

5. The home scene control method according to claim 4, characterized in that, The process of generating several candidate decision scheme groups includes: A tree structure is constructed with the combination of conflict events corresponding to the first conflict event and the second conflict event as the root node, and the first conflict event and the second conflict event as different first-level child nodes; wherein, the priority of the first conflict event is used as the node attribute of the first-level child node corresponding to the first conflict event; the priority of the second conflict event is used as the node attribute of the first-level child node corresponding to the second conflict event. Starting from the root node, several different first-level child nodes are randomly selected from the tree structure to generate a first node path group; wherein, the first node path group contains several different first node paths; the first node path contains one or more first-level child nodes. The first node path is filtered according to a preset path selection rule. Then, according to each of the candidate decision schemes, the node paths in the filtered first node path group are expanded into secondary nodes to construct several target node paths to be simulated. The preset path selection rule is used to characterize that the priority of the previous node in the node path is greater than the priority of the next node. For each target node path to be simulated, extract a second-level child node corresponding to each first-level child node in the target node path to be simulated, generate a candidate decision scheme group, until the extraction is completed, and output several candidate decision scheme groups.

6. The home scene control method according to claim 5, characterized in that, The process of constructing the paths of several target nodes to be simulated includes: According to the preset path selection rules, delete the first node paths in the first node path group that do not conform to the preset path selection rules, and mark the first node paths that have not been deleted as second node paths. For each second node path, the first-level child node with the smallest relative distance to the root node is taken as the first target child node. Then, the candidate decision scheme corresponding to the first target child node is expanded into the second-level child nodes corresponding to the first target child node to construct the third node path; wherein, the first target child node corresponds to multiple different second-level child nodes. For each of the third node paths, the candidate device control instructions in the second-level sub-nodes corresponding to the first target sub-node are taken as the first candidate device control instructions. The first candidate device control instructions are identified, and it is determined whether the action corresponding to the first candidate device control instructions is an interception behavior. If yes, the third node path is marked as the initial node path to be simulated. If no, the second-level sub-nodes corresponding to the first candidate device control instructions are deleted from the third node path, and then the initial node path to be simulated is constructed. The interception behavior is used to indicate the action instructions to prevent the occurrence of security risks. For each initial node path to be simulated, the initial node path to be simulated is adjusted according to the candidate decision schemes corresponding to the first-level child nodes that have not been expanded into second-level nodes in the initial node path to be simulated, thereby constructing the target node path to be simulated.

7. The home scene control method according to claim 6, characterized in that, The step of adjusting the initial path to be simulated based on the candidate decision schemes corresponding to the first-level child nodes that have not been expanded into second-level nodes in the initial path to be simulated, and constructing the target path to be simulated, includes: The first-level child nodes that have not been expanded into second-level nodes in the initial simulated node path are taken as the second target child nodes. Then, the candidate decision schemes corresponding to the second target child nodes are expanded into second-level child nodes corresponding to the second target child nodes to construct the fourth node path. For each of the fourth node paths, the candidate device control instructions in the second-level sub-nodes corresponding to the second target sub-node are used as second candidate device control instructions. The second candidate device control instructions are identified to determine whether the action corresponding to the second candidate device control instructions is a required behavior. If yes, the fourth node path is marked as the target node path to be simulated. If no, the second-level sub-nodes corresponding to the second candidate device control instructions are deleted from the fourth node path, thereby constructing the target node path to be simulated. The required behavior is used to indicate action instructions that can satisfy the character's needs. The initial node path to be simulated that does not have any unexpanded secondary nodes is directly marked as the target node path to be simulated.

8. A ternary architecture system for home scenarios, characterized in that, The three-element architecture system includes: end-side devices, edge servers, and cloud servers; The edge device is used to collect several monitoring images in a home scene, and then send each monitoring image to the edge server. The edge server is used to determine the home space corresponding to each monitoring image and the home status data corresponding to the home space; wherein, the home scene includes several different human objects; The edge server is further configured to, for each person in each of the monitored images, generate the behavioral trajectory intent of the person based on the home status data and the facial features of the person in the monitored image; wherein, the behavioral trajectory intent includes: a movement area intent representing the endpoint of the person's movement trajectory, an action tendency intent representing the degree of avoidance tendency during the person's movement, and a device operation intent representing the person's operational needs for home appliances; wherein, different operational needs correspond to different operational rationality values; The edge server is further configured to combine several behavioral trajectory intentions that have overlapping areas within the same time window and whose target collision probability between several action tendency intentions is greater than a preset collision probability threshold into a first conflict event of trajectory collision type; wherein, the target collision probability is calculated from the avoidance tendency degree between each action tendency intention. The edge server is further configured to obtain device operation permissions for each person based on the facial features; wherein, the device operation permissions include: having operation permissions or not having operation permissions. The edge server is further configured to combine several behavioral trajectory intentions of the device operation permission as no operation permission and the operation reasonableness value is less than a preset operation reasonableness threshold into a second conflict event of the device operation type, and then send the first conflict event and the second conflict event of each different conflict type to the cloud server. The cloud server is used to perform combined simulations of decision schemes between each first conflict event and each second conflict event based on preset decision constraints. When a candidate decision scheme group satisfies the minimum security risk value, the minimum convenience loss of the scheme, and the maximum emergency response level of the scheme, the candidate decision scheme group is taken as the target decision scheme group. The target decision scheme group includes: a number of target device control commands and a number of target human-computer interaction mechanisms. The cloud server is also used to send control instructions for each of the target devices to the end device, so that the end device can control the home devices in the home scene according to the control instructions for each of the target devices; The cloud server is also used to send each of the target human-computer interaction mechanisms to the edge server, so that the edge server generates a number of interactive reminder messages for information transmission with the human object based on each of the target human-computer interaction mechanisms, and the edge server sends the interactive reminder messages to the corresponding terminal device for execution output.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement a home scene control method according to any one of claims 1 to 7.

Images