Smart home interaction system based on motion sensing technology

Abstract

The invention relates to the technical field of somatosensory interaction, in particular to a somatosensory technology-based smart home interaction system, which comprises a data acquisition module, a skeleton behavior thermodynamic diagram model construction module, a skeleton behavior thermodynamic diagram model construction module, a control right decision module and an execution module, the data acquisition module is used for acquiring somatosensory data of continuous frames of each target individual in a target area; the execution module is used for triggering an interaction command and executing service control; a decision basis is converted into historical behavior accumulation from an instantaneous spatial relationship, a robust interaction system which is insensitive to visual occlusion and ID hopping and loyalty to historical behaviors of a user is successfully constructed, identity drift is effectively resisted, the stability of control authority is maintained, the attribution of interaction control authority is reliably judged, wrong switching and preemption are prevented, and the user experience is improved. A hidden identity inertia model is established, intention continuity is achieved, and the accuracy of somatosensory interaction of the smart home and user experience are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of motion-sensing interaction technology, and more particularly to a smart home interaction system based on motion-sensing technology. Background Technology

[0002] Currently, the main control methods for smart home devices include voice control, remote terminal control, and touchscreen control. Voice control requires users to issue clear and accurate commands, and the recognition rate decreases in noisy environments; remote terminal control requires users to operate their mobile phones, which involves certain steps and time costs; touchscreen control requires users to have close contact with the device, which is not suitable for scenarios where it is remote or inconvenient to touch the device. With the widespread deployment of motion recognition technology in smart home systems, users can perform contactless control operations by recognizing their own posture and movements, such as using gestures to turn lights on and off, or waving to control the volume of speakers.

[0003] However, in real-life scenarios where multiple people use the system, such as family gatherings or living room environments where multiple people watch TV, motion recognition systems often face the following technical challenges: when two users move quickly, overlap, or overlap and occlude each other, skeleton recognition is prone to losing IDs or misidentifying new IDs, causing identity drift; after the skeleton ID is incorrectly identified as another user, their actions may be misjudged by the system as command input, resulting in the loss of the original user's control rights. Although traditional target tracking methods can alleviate ID drift, they still cannot reliably determine the ownership of interactive control under conditions of strong occlusion and simultaneous partial actions of multiple people. Summary of the Invention

[0004] To address the technical problems existing in the background art, this invention proposes a smart home interaction system based on motion sensing technology, the specific solution of which is as follows: A smart home interaction system based on motion sensing technology includes: The data acquisition module is used to collect continuous frames of somatosensory data for each target individual within the target area; The skeleton identity allocation model construction module extracts the spatial coordinates of key points of the human skeleton from the somatosensory data of each target individual to form skeleton coordinate data. Based on the recognition algorithm, it assigns a skeleton identity ID to the somatosensory data of each target individual, forming a binding relationship between the skeleton identity ID and the skeleton coordinate data, and constructs the skeleton identity allocation model. The skeleton behavior heatmap model building module constructs a skeleton behavior heatmap model. Within a preset time window, historical interaction behavior data is constructed based on the skeleton identity ID, and a skeleton behavior heatmap model is constructed based on the historical interaction behavior data. The skeleton behavior heatmap model is used to represent the behavior heatmap of the skeleton identity ID in a spatial region. The control decision module, for multiple skeleton identity IDs identified in the current frame, performs the following steps to determine the ownership of interaction control when an interaction action triggers an intent: Extract the spatial region where the interactive action is located; Obtain the behavior heatmap corresponding to each skeleton identity ID from the skeleton behavior heatmap model, and base the behavior heatmap on the behavior heatmap value; Select the ID with the highest heat value for the behavior from among multiple IDs, and use that ID as the control ID for the current interaction action. The execution module is used to trigger interactive commands and execute business controls. Once the control ownership ID is determined, its corresponding interaction action mode is matched with the preset control rules, and the corresponding smart home device control command is triggered.

[0005] Furthermore, a skeleton behavior heatmap model is constructed based on historical interaction behavior data, as follows: Historical interaction behavior data corresponding to the skeleton identity ID is read, and a skeleton behavior heatmap model is constructed based on the following steps: Discretize the spatial location of the skeleton in the historical interaction behavior data into two-dimensional grid cells; The behavior weight value is calculated based on the frequency of interaction behaviors occurring within a two-dimensional grid cell using a weight value formula. A behavior heatmap is generated based on the behavior weight values, where each skeleton identity ID corresponds to a behavior heatmap, thus constructing a skeleton behavior heatmap model.

[0006] Furthermore, the formula for the weight value is: in, For the intensity of interactive actions, The time decay coefficient, For position indication function, Let be the spatial coordinates of the i-th interactive event. Use its event timestamp.

[0007] Furthermore, the construction process of the skeleton behavior heatmap model also includes the following steps: Each time a valid interactive action is detected in a target individual, its thermal value on the surrounding grid cells is calculated based on a two-dimensional Gaussian distribution function, with its spatial location coordinates as the center. The thermal values ​​are then accumulated to form the behavioral thermal value. The peak value of the Gaussian distribution function is set to different values ​​according to the type of interactive action; The thermal value of the behavior decreases exponentially over time.

[0008] Furthermore, in the control decision module, if the difference in behavioral heatmap values ​​of multiple skeleton identity IDs within a specified spatial area does not exceed a preset threshold, the following priority judgment criteria are used to determine the control ownership ID: Prioritize skeleton identity IDs that have been confirmed in the most recent successful interaction; If there is no interaction history, select the skeleton identity ID in the current frame that is most similar to the target action motion features and historical action trajectory.

[0009] Furthermore, in the execution module, if the selected control ownership ID is stably tracked for N consecutive frames before the interactive action is triggered and no identity switch occurs, then the execution of the control command is allowed.

[0010] Furthermore, it also includes: performing occlusion recovery detection on the skeleton identity ID, and when the skeleton identity ID reappears within a preset recovery time window after temporarily disappearing due to occlusion, its historical heat map information and last control ownership status are used preferentially.

[0011] Furthermore, the behavior heatmap is a two-dimensional array structure that supports recursive iteration in a sliding time window manner, and performs an exponential decay function operation on each heatmap unit value when each time iteration node is reached.

[0012] Compared with the prior art, the present invention can achieve at least the following beneficial effects: This invention, through the construction of a skeleton behavior heatmap model, records the historical behavioral inertia of each skeleton identity ID in spatial and temporal dimensions. When occlusion causes a brief ID drift or jump, because the newly misidentified ID has no historical heat accumulation in the target area, while the original user has a long-term accumulated high heat value, control will still stably belong to the original user. This is like establishing a behavioral credit account for each user, which will not be cleared due to a brief identification failure, thus greatly improving the identity retention rate and permission continuity. By establishing priority control rights for old users, it effectively prevents the accidental switching of control rights due to others passing by or temporarily intervening, solving the decision-making dilemma of traditional systems when multiple people interact at close range. The skeleton behavior heatmap model, through weighting, attenuation, and spatial aggregation of historical interaction behaviors, precipitates the user's habitual paths and frequently used control points into a visual control territory map. The continuation and respect for the user's historical control intentions and behaviors ensures the contextual coherence of the interaction process, making the system's behavior more in line with human expectations and habits. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a system principle block diagram of the present invention. Detailed Implementation

[0014] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0015] Example 1, please refer to... Figure 1 As shown, the present invention provides a smart home interaction system based on motion sensing technology, including a data acquisition module, a skeleton behavior heatmap model construction module, a control decision module, and an execution module; The data acquisition module is used to collect continuous frames of somatosensory data for each target individual within the target area; It should be noted that motion-sensing data refers to raw data collected by one or more motion-sensing recognition devices that reflects the spatial posture and motion state of a target individual (usually a user). In practical applications, these motion-sensing recognition devices may include, but are not limited to, depth cameras, RGB-D sensors, or visual recognition systems based on ordinary cameras. The motion-sensing data typically includes RGB image sequences, depth image sequences, and / or point cloud data calculated based on these images, provided by these devices. The purpose of acquiring continuous frames is to capture the temporal information of the user's motion, providing a data foundation for subsequent skeleton tracking and identity continuity maintenance.

[0016] The skeleton identity allocation model construction module extracts the spatial coordinates of key points of the human skeleton from the somatosensory data of each target individual to form skeleton coordinate data. Based on the recognition algorithm, it assigns a skeleton identity ID to the somatosensory data of each target individual, forming a binding relationship between the skeleton identity ID and the skeleton coordinate data, and constructs the skeleton identity allocation model. It should be noted that the skeleton identity assignment model is not a pre-trained machine learning model with fixed parameters, but rather a set of detection-tracking-association logical processes and data management mechanisms that are dynamically executed during system runtime.

[0017] It should be noted that skeleton key point extraction: "Extracting the spatial coordinates of key human skeleton points from motion sensing data" refers to processing each frame of motion sensing data using a skeleton estimation algorithm (e.g., the core algorithm integrated from open-source libraries such as MediaPipe Pose or OpenPose). This processing can identify the pixel coordinates of key joints of the human body from the image and typically convert them into three-dimensional spatial coordinates within the target area (i.e., the "skeleton coordinate data") using camera intrinsics and depth information. These key points may include, but are not limited to, the nose, neck, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles.

[0018] Skeleton Identity ID Allocation and Binding: The process of "assigning a skeleton identity ID to each target individual based on the somatosensory data of the recognition algorithm" typically includes two stages in practice: New target detection and ID creation: In the initial frame or when a new target enters the scene, the system creates a unique identifier (i.e., skeleton ID) for each detected independent skeleton and binds this ID to the coordinates of all key points of the skeleton to form an initial mapping relationship.

[0019] Cross-frame target tracking and ID preservation: In subsequent consecutive frames, the system uses target tracking algorithms (such as trajectory prediction-based algorithms, like Kalman filtering combined with the Hungarian algorithm for data association) to match the skeleton detected in the new frame with the existing skeleton IDs from the previous frame. Successfully matched skeletons inherit their original IDs, thus ensuring the continuation of the "binding relationship between the skeleton ID and the skeleton coordinate data" over time. This mechanism is fundamental to overcoming visual occlusion and ID jumps.

[0020] The skeleton behavior heatmap model construction module constructs historical interaction behavior data based on skeleton identity ID within a preset time window, and constructs a skeleton behavior heatmap model based on the historical interaction behavior data. The skeleton behavior heatmap model is used to represent the behavior heatmap of skeleton identity ID in a spatial region.

[0021] It should be noted that the historical interaction behavior data is a structured dataset organized in a time series, with its core data items originating from the skeleton coordinate data continuously output by the skeleton identity assignment model. Specifically, for each successfully tracked skeleton identity ID, the system records the following information within the "preset time window" (e.g., the past 10 or 30 seconds): the timestamp of each frame, the spatial coordinates of specific key points of the skeleton in that frame (such as wrist key points used for gesture recognition), and the amplitude and type of the action (such as waving, raising, pointing, etc.) calculated based on skeleton kinematics. These data collectively constitute the historical interaction behavior data of a single skeleton identity ID.

[0022] A skeleton behavior heatmap model is constructed based on historical interaction behavior data, as follows: Historical interaction behavior data corresponding to the skeleton identity ID is read, and a skeleton behavior heatmap model is constructed based on the following steps: Discretize the spatial location of the skeleton in the historical interaction behavior data into two-dimensional grid cells; It should be noted that "discretizing the spatial location of the skeleton into two-dimensional grid cells" aims to quantize continuous physical space into computable discrete units for statistical and visualization purposes. In practice, the system uses the target area (such as the entire living room floor plan) as a base map and divides it into a regular grid of M rows x N columns. Each grid cell... This corresponds to a fixed area in physical space. The spatial location where any interaction event occurs ( All data can be categorized into a specific grid cell through coordinate mapping. This gridding process is the cornerstone of transforming continuous user behavior trajectories into discrete spatial statistical data.

[0023] The behavior weight value is calculated based on the frequency of interaction behaviors occurring within a two-dimensional grid cell using a weight value formula. The formula for the weight value is: in, This represents the intensity of the interactive action. It's a quantitative indicator of the importance of the interactive behavior. The system can assign different values ​​based on the type of action recognized. For example, a clear, large-amplitude "switch" gesture can be assigned a higher intensity value (e.g., A=1.0), while a small, ambiguous hand movement is assigned a lower intensity value (e.g., A=0.2). It is the time decay coefficient, which determines how quickly the heat dissipates. It is the time difference between the current time and the time the interaction event occurred. This factor ensures that interactions that occurred a long time ago contribute far less to the current heatmap value than recent interactions, allowing the heatmap to dynamically reflect the user's latest interaction tendencies.

[0024] For position indication function, Let be the spatial coordinates of the i-th interactive event. Use its event timestamp.

[0025] It should be noted that the calculation process of the weight value formula can be understood as follows: for a certain skeleton identity ID(k), in the grid cell... The final heat value It is the sum of the contributions of all historical interaction events that occurred within that grid cell, and the contribution value of each event is equal to its action intensity. The result after appropriate time decay. This is achieved by traversing all grid cells and calculating for each cell. The system then generates a complete, quantified behavioral heatmap for each skeleton identity ID.

[0026] A behavior heatmap is generated based on the behavior weight values, where each skeleton identity ID corresponds to a behavior heatmap, thus constructing a skeleton behavior heatmap model.

[0027] The construction process of the skeleton behavior heatmap model also includes the following steps: Each time a valid interactive action is detected in a target individual, its thermal value on the surrounding grid cells is calculated based on a two-dimensional Gaussian distribution function, with its spatial location coordinates as the center. The thermal values ​​are then accumulated to form the behavioral thermal value. The peak value of the Gaussian distribution function is set to different values ​​according to the type of interactive action; The behavioral heat value decays exponentially over time, reflecting the timeliness of behavioral control intensity.

[0028] The behavior heatmap is a two-dimensional array structure that supports recursive iteration in a sliding time window manner. At each time iteration node, an exponential decay function operation is performed on each heatmap unit value, so that the behavior heatmap has spatial and temporal continuity.

[0029] It should be noted that the operation of "calculating the heat value of a target individual on surrounding grid cells based on a two-dimensional Gaussian distribution function, centered on its spatial location coordinates, and accumulating these heat values ​​to form the behavior heat value" aims to simulate the diffusion of the influence of a single interaction behavior in physical space, rather than a momentary event limited to a single coordinate point. In specific implementation, the system does not simply assign all weight values ​​to the single grid cell where the interaction event occurred, but rather uses the location of the event (…) Using a grid as the center, calculate the value of a two-dimensional Gaussian distribution function over the entire grid, and then accumulate these values ​​proportionally to the corresponding grid cells.

[0030] It should be noted that the peak value of the Gaussian distribution function is set to different values ​​based on the type of interaction action. This mechanism enables differentiated modeling of different interaction intentions. The system predefines a mapping relationship between action type and peak value intensity. For example, a clear and decisive "skip song" gesture may be assigned a higher peak value (e.g., Ai=1.0), indicating a strong control intention; while a more casual, possibly unintentional "wave" gesture is assigned a lower peak value (e.g., Ai=0.3), indicating a weaker control intention or possibly a mis-touch. Through this differentiated setting, the heatmap can more accurately reflect the user's real and effective control habits, rather than a simple superposition of all movements.

[0031] It should be noted that the exponential decay of the behavioral heatmap values ​​over time, and the exponential decay function operation performed on each heatmap unit value at each time iteration node, together constitute the dynamic update and forgetting mechanism of the heatmap model, ensuring that it can reflect the latest user behavior patterns. In practice, the system does not need to continuously recalculate all historical data within the entire time window. Instead, it performs a uniform decay calculation on each unit value of the entire heatmap at a fixed time interval (i.e., "iteration node", such as once per second).

[0032] It should be noted that the "behavioral heatmap is a two-dimensional array structure," and its collaboration with the "sliding time window method" and the aforementioned decay mechanism is key to achieving the "spatial and temporal continuity" of the behavioral heatmap. The two-dimensional array directly maps the discrete grid of space in memory, ensuring the continuity of spatial relationships. In the preferred implementation described in this paper, the "sliding time window" does not refer to a fixed-length FIFO (first-in, first-out) queue storing the original data, but is implicitly implemented through the aforementioned global, periodic exponential decay operation. Each decay operation continuously weakens the contributions of earlier historical data, thus effectively equivalent to a time window focusing on recent history. This combination of "spatial gridded storage" and "temporal exponential decay" generates a "control heatfield" that reflects both the user's habitual activity areas in space (spatial continuity) and dynamically tracks changes in their behavioral patterns (temporal continuity).

[0033] The control decision module, for multiple skeleton identity IDs identified in the current frame, performs the following steps to determine the ownership of interaction control when an interaction action triggers an intent: Extract the spatial region where the interactive action is located; It should be noted that "extracting the spatial area where the interactive action is located" refers to dynamically determining a physical spatial range associated with the identified specific interactive intent. This spatial area is not always fixed; its definition is closely related to the location and type of smart home devices. For example: When the interactive action is "waving towards the TV", the spatial area can be predefined as a rectangular area in front of the TV screen.

[0034] When the interaction action is "making a gesture to adjust the lights", the spatial area can be defined as the entire room partition where the user is located when the gesture is recognized, or the controllable range of a specific light fixture.

[0035] The system maintains a mapping table of "interactive devices - spatial regions". When a predefined interactive action pattern (i.e. "interactive action triggering intent") is identified, the corresponding one or more grid cells are locked by looking up the table, forming the "spatial region where the interactive action is located" to be determined.

[0036] Obtain the behavior heatmap corresponding to each skeleton identity ID from the skeleton behavior heatmap model, and base the behavior heatmap on the behavior heatmap value; Select the ID with the highest heat value for the behavior from among multiple IDs, and use that ID as the control ID for the current interaction action. It should be noted that the above together constitute the core decision-making logic of this invention for solving the ID drift and preemption problems. The key is that the attribution of control does not depend on which skeleton ID in the current frame is spatially closest to the interacting device, but rather on the control "inertia" or "territorial rights" accumulated from historical behavior. Specifically: The system extracts the heat values ​​of all grid cells within a defined "spatial region" from the behavior heatmap maintained independently for each skeleton ID. For each ID, the maximum, average, or integral value of the heat values ​​within that region can be taken as its comprehensive "behavioral heat value" for that region.

[0037] The system simply compares the overall heat value of all candidate IDs and assigns control to the ID with the highest value. This mechanism effectively ensures that a user who has been controlling the TV from the sofa position for a long time will not have their control easily taken away, even if they are briefly blocked by another user passing through the living room, because their historical behavior has accumulated a higher heat value in the sofa area.

[0038] It should be noted that the condition "if the difference in behavioral heatmap values ​​of multiple skeleton IDs within a specified spatial area does not exceed a preset threshold" is to handle a boundary scenario where the historical interaction activity of multiple users in that area is very similar, making it difficult to clearly distinguish them based on heatmap values. The "preset threshold" is an empirical value determined through debugging based on actual scenarios, and its purpose is to define a "fuzzy interval." When the difference between the heatmap champion and runner-up is less than this threshold, the system considers that heatmap values ​​alone are insufficient for reliable decision-making, thus activating the following alternative priority judgment logic to increase the system's robustness.

[0039] In the control decision module, if the difference in behavioral heatmap values ​​of multiple skeleton identity IDs within a specified spatial area does not exceed a preset threshold, the following priority judgment criteria are applied to determine the control ownership ID: Prioritize skeleton identity IDs that have been confirmed in the most recent successful interaction; If there is no interaction history, select the skeleton identity ID in the current frame that is most similar to the target action motion features and historical action trajectory.

[0040] It should be noted that the priority determination is based on a two-tiered, progressive security strategy: "Prioritize skeleton IDs that have been confirmed in the most recent successful interaction": This strategy aims to maintain the continuity of the interaction session. The system internally records the skeleton ID of the last successfully executed control command. When the popularity values ​​are similar, this ID is used first, which can effectively prevent control from switching back and forth between multiple users due to minor fluctuations in continuous operation, thereby improving the consistency of the experience.

[0041] "If there is no interaction history, select the skeleton ID whose target action motion features in the current frame are most similar to the historical action trajectory": This strategy is used to handle the extreme case where none of the candidate IDs have a successful interaction history (e.g., two people simultaneously enter a new room and make their first interaction). In this case, the system calculates the similarity between the action that triggered the interaction intention in the current frame (e.g., a waving trajectory) and the typical action patterns (e.g., waving amplitude, speed features) of each candidate ID in recent historical frames. Selecting the most similar one is based on the reasonable inference that "the action patterns of the same individual are consistent," which provides a fallback mechanism for the system when long-term historical data is lacking.

[0042] It should be noted that the entire control decision-making module design embodies the progressive judgment idea of ​​this invention in decision-making, from "primary basis" to "auxiliary basis". It primarily relies on "behavioral heatmaps" with long-term historical significance. When the heatmap cannot provide a clear decision, it then introduces short-term "interaction continuity" and instantaneous "action similarity" as supplementary criteria, thereby ensuring the high reliability and robustness of the control attribution determination in a progressive manner.

[0043] The execution module is used to trigger interactive commands and execute business controls. Once the control ownership ID is determined, its corresponding interaction action mode is matched with the preset control rules, and the corresponding smart home device control command is triggered.

[0044] In the execution module, if the selected control ownership ID is stably tracked for N consecutive frames before the interactive action is triggered and no identity switch occurs, the control command is allowed to be executed, thereby achieving the screening purpose of preventing newly entering individuals from having short-term preemptive control.

[0045] It should be noted that the condition in the execution module, "when the selected control owner ID is stably tracked for N consecutive frames before the interaction is triggered and no identity switch occurs," is a crucial security check step. Its core purpose is to ensure that the interaction command originates from a stable and trusted identity entity, thereby effectively filtering out momentary interference and preemption by newly entering users. The specific implementation is as follows: The value N in N consecutive frames is a configurable system parameter whose setting needs to take into account both the system's frame rate and the requirements for "stability". For example, in a system of 30 frames per second, setting N=15 means that the control owner ID must be continuously and accurately tracked for half a second before the command is triggered.

[0046] Stable tracking without identity switching means that when the system traces the history of the ID across the most recent N frames, it confirms that the skeleton data is consistent and that there has been no instance of the tracking algorithm temporarily losing the ID and then reassigning a new one. This mechanism effectively identifies and prevents two types of risks: New user preemption: A new user entering the field of view is assigned a new skeleton identity ID. Since this ID does not have a tracking history of "N consecutive frames", even if it makes an interactive action, it will be blocked by this condition and cannot immediately gain control.

[0047] ID jump error triggering: In complex scenarios with multiple people occluding or intersecting, the tracking algorithm may experience a brief ID jump (i.e., user A's ID is incorrectly identified as user B's ID). If the ID after the jump happens to be the ID with the highest heat value, but the condition of "N consecutive frames" is not met, the instruction will not be executed, thus avoiding erroneous operations caused by momentary algorithm errors.

[0048] It also includes: performing occlusion recovery detection on the skeleton identity ID. When the skeleton identity ID reappears within a preset recovery time window after temporarily disappearing due to occlusion, its historical heat map information and the last control ownership status are used first to avoid generating a new control identity ID and causing command ownership errors.

[0049] It should be noted that the "occlusion recovery detection for skeleton identity ID" mechanism is an important guarantee for solving the unavoidable visual occlusion problem in practical applications and maintaining the continuity of user experience. The specific workflow of this mechanism is as follows: The system maintains a state machine and a timer for each active skeleton identity ID.

[0050] When an ID temporarily disappears from the video stream due to occlusion (such as being obscured by furniture or other people), the system does not immediately destroy all its data. Instead, it marks the ID as "pending" and initiates a "preset recovery time window" (e.g., 2-3 seconds). This time window is set based on statistics of typical occlusion durations.

[0051] Within this time window, if the system detects a skeleton again whose position, size, and movement trajectory highly match the previously disappeared ID, the system will determine that this is the reappearance of the same user, rather than a new user.

[0052] At this point, the system will prioritize using its historical heatmap information and the last control attribution status. This means: The system directly restores all previously accumulated behavioral heatmap data for the user, thus preserving their control "territory".

[0053] If the user had control before disappearing, their control will also be restored.

[0054] The core of this mechanism, which "avoids the generation of new control identity IDs that could lead to incorrect command attribution," lies in its departure from the simplistic "what you see is what you get" logic of motion-sensing systems. Instead, it introduces identity presumption based on historical continuity. For example, if a user gets up from the sofa to get a water glass, briefly leaves the camera's field of view, and then returns, the system can correctly identify them as the original user and restore their control privileges, rather than treating them as a new visitor without the authority to control the device.

[0055] It should be noted that the aforementioned stability verification and occlusion recovery mechanisms, together with the control authority determination model based on behavior heatmaps, constitute a multi-layered and in-depth security protection system. The heatmap model solves the problem of "who has more authority," while stability verification ensures that the entity executing the command is "real and reliable," and the occlusion recovery mechanism guarantees the "spatiotemporal continuity" of the user's identity. The three work together to fundamentally improve the accuracy, robustness, and user experience of smart home interactions in complex multi-user scenarios.

[0056] In summary, by shifting the decision-making basis from instantaneous spatial relationships to historical behavioral accumulation, a robust interactive system that is insensitive to visual occlusion and ID changes and loyal to users' historical behavior has been successfully constructed. This system effectively combats identity drift, maintains the stability of control permissions, reliably determines the ownership of interactive control, prevents accidental switching and preemption, establishes an implicit identity inertia model, achieves intent continuity, and significantly improves the accuracy of smart home haptic interaction and user experience.

[0057] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.

[0058] In the embodiments provided by this invention, it should be understood that the disclosed system or method can be implemented in other ways. For example, the embodiments of the invention described above are merely illustrative; for instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation.

[0059] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0060] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0061] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the basic characteristics of the present invention.

[0062] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A smart home interaction system based on motion sensing technology, characterized in that, include: The data acquisition module is used to collect continuous frames of somatosensory data for each target individual within the target area; The skeleton identity allocation model construction module extracts the spatial coordinates of key points of the human skeleton from the somatosensory data of each target individual to form skeleton coordinate data. Based on the recognition algorithm, it assigns a skeleton identity ID to the somatosensory data of each target individual, forming a binding relationship between the skeleton identity ID and the skeleton coordinate data, and constructs the skeleton identity allocation model. The skeleton behavior heatmap model building module constructs a skeleton behavior heatmap model. Within a preset time window, historical interaction behavior data is constructed based on the skeleton identity ID, and a skeleton behavior heatmap model is constructed based on the historical interaction behavior data. The skeleton behavior heatmap model is used to represent the behavior heatmap of the skeleton identity ID in a spatial region. The control decision module, for multiple skeleton identity IDs identified in the current frame, performs the following steps to determine the ownership of interaction control when an interaction action triggers an intent: Extract the spatial region where the interactive action is located; Obtain the behavior heatmap corresponding to each skeleton identity ID from the skeleton behavior heatmap model, and base the behavior heatmap on the behavior heatmap value; Select the ID with the highest heat value for the behavior from among multiple IDs, and use that ID as the control ID for the current interaction action. The execution module is used to trigger interactive commands and execute business controls. Once the control ownership ID is determined, its corresponding interaction action mode is matched with the preset control rules, and the corresponding smart home device control command is triggered.

2. The smart home interaction system based on motion sensing technology as described in claim 1, characterized in that: A skeleton behavior heatmap model is constructed based on historical interaction behavior data, as follows: Historical interaction behavior data corresponding to the skeleton identity ID is read, and a skeleton behavior heatmap model is constructed based on the following steps: Discretize the spatial location of the skeleton in the historical interaction behavior data into two-dimensional grid cells; The behavior weight value is calculated based on the frequency of interaction behaviors occurring within a two-dimensional grid cell using a weight value formula. A behavior heatmap is generated based on the behavior weight values, where each skeleton identity ID corresponds to a behavior heatmap, thus constructing a skeleton behavior heatmap model.

3. The smart home interaction system based on motion sensing technology as described in claim 2, characterized in that: The formula for the weight value is: in, For the intensity of interactive actions, The time decay coefficient, For position indication function, Let be the spatial coordinates of the i-th interactive event. Use its event timestamp.

4. The smart home interaction system based on motion sensing technology as described in claim 2, characterized in that: The construction process of the skeleton behavior heatmap model also includes the following steps: Each time a valid interactive action is detected in a target individual, its thermal value on the surrounding grid cells is calculated based on a two-dimensional Gaussian distribution function, with its spatial location coordinates as the center. The thermal values ​​are then accumulated to form the behavioral thermal value. The peak value of the Gaussian distribution function is set to different values ​​according to the type of interactive action; The thermal value of the behavior decreases exponentially over time.

5. The smart home interaction system based on motion sensing technology as described in claim 1, characterized in that: In the control decision module, if the difference in behavioral heatmap values ​​of multiple skeleton identity IDs within a specified spatial area does not exceed a preset threshold, the following priority judgment criteria are applied to determine the control ownership ID: Prioritize skeleton identity IDs that have been confirmed in the most recent successful interaction; If there is no interaction history, select the skeleton identity ID in the current frame that is most similar to the target action motion features and historical action trajectory.

6. The smart home interaction system based on motion sensing technology as described in claim 1, characterized in that: In the execution module, if the selected control owner ID is stably tracked for N consecutive frames before the interactive action is triggered and no identity switch occurs, then the execution of the control command is allowed.

7. The smart home interaction system based on motion sensing technology as described in claim 1, characterized in that: Also includes: The skeleton identity ID is subjected to occlusion recovery detection. When the skeleton identity ID reappears within a preset recovery time window after temporarily disappearing due to occlusion, its historical heat map information and last control ownership status are used first.

8. The smart home interaction system based on motion sensing technology as described in claim 1, characterized in that: The behavior heatmap is a two-dimensional array structure that supports recursive iteration in a sliding time window manner, and performs an exponential decay function operation on each heatmap unit value when each time iteration node is reached.

Images