An end-cloud collaborative home security method and system

Abstract

This invention relates to the technical field of home safety, and discloses a home safety method and system based on edge-cloud collaboration. The method includes: collecting home scene behavior data sequences using multi-source sensing devices, and performing target association on the home scene behavior data sequences; extracting target micro-exploration events using a multi-dimensional behavior trajectory determination algorithm, and generating event feature vectors for the target micro-exploration events; constructing a sliding time window, performing sliding aggregation on the target micro-exploration events, and combining the event feature vectors of the target micro-exploration events in the aggregated events to generate a dangerous exploration evolution intensity vector for the aggregated events; selecting aggregated events with a trend growth rate higher than a preset stable rate threshold as home dangerous events, and generating danger status labels for home dangerous events for graded early warning. This invention achieves early identification, dynamic evolution modeling, and graded early warning of children's dangerous exploration behaviors, and improves real-time performance and identification accuracy through an edge-cloud collaborative architecture.

Description

Technical Field

[0001] This invention relates to the field of home security, and more particularly to a home security method and system that integrates edge and cloud computing. Background Technology

[0002] With the development of smart home and IoT technologies, home safety monitoring based on visual perception and multi-source sensing is gradually becoming an important means of protecting children's safety. In the home environment, children often show interest in potentially dangerous items such as furniture high up, electrical appliances, drawer doors, and kitchen utensils, exploring them by touching, pulling, climbing, or moving around the target area. However, in the early stages, these behaviors often manifest as low-intensity, intermittent, and discontinuous exploratory behaviors, such as short periods of time spent near the object, repeated approaching, slight touching, or tentative operations. These behaviors are usually difficult to identify as dangerous at the level of a single incident and are easily confused with children's normal activities (such as searching for toys or wandering around), making it difficult to identify potential risks in a timely manner.

[0003] In existing technologies, such as patent CN117746339A, a home security monitoring system and method based on thermal imaging and identity recognition technology, the system collects thermal imaging data of residents by setting up a thermal imaging module, an identity recognition module, and a security judgment module. It then matches the monitored objects with the identity recognition data to analyze the security status of specific objects. This method can accurately identify target objects in multi-person environments and analyze their status through thermal imaging data, thus improving the accuracy and reliability of home security monitoring to some extent. However, it mainly relies on thermal imaging data for security judgment and lacks in-depth modeling of behavioral trajectories and the interaction between people and the environment, making it difficult to identify early exploratory behaviors such as children approaching dangerous objects.

[0004] To address this issue, this invention proposes an edge-cloud collaborative home safety method that can identify children's early exploration behavior towards dangerous objects and characterize the risk evolution process over time. It can identify potential risks and provide early warnings before dangerous behavior actually occurs, providing data support for children's behavioral science research. Summary of the Invention

[0005] This invention provides a home safety method and system with edge-cloud collaboration. Since existing technologies often rely on single-frame detection or simple tracking, target loss or mismatch is prone to occur. Step S1 constructs a target association mechanism by fusing identity tags and 3D spatial information, achieving consistent target matching across frames. This solves the problem of inaccurate target association in scenarios with multiple people and objects, improving the continuity and reliability of behavioral data. Step S2 constructs a multi-dimensional behavioral trajectory determination model that includes distance changes, displacement amplitude, direction changes, and contact features, enabling the identification of early exploratory behaviors such as children approaching dangerous objects, solving the problem of delayed risk identification. Step S3 aggregates multiple micro-exploration events through a sliding time window and constructs a dangerous exploration evolution intensity vector, introducing danger accumulation, trend, and acceleration indicators, solving the problem of not being able to describe the dynamic evolution process of risk. Step S4 determines dangerous events based on trend growth rate and generates dangerous status labels by combining comprehensive risk scores, achieving risk classification and dynamic early warning, solving the problems of single risk assessment and inaccurate early warning.

[0006] To achieve the above objectives, the present invention provides a home security method with end-to-end cloud collaboration, comprising the following steps: S1: Collect home scene behavior data sequences using multi-source sensing devices, perform target association on the home scene behavior data sequences, and obtain target-associated home scene behavior association data; S2: Based on the home scene behavior association data, extract the target micro-exploration event using a multi-dimensional behavior trajectory determination algorithm, and generate the event feature vector of the target micro-exploration event; S3: Construct a sliding time window, perform sliding aggregation on the target micro-exploration events to obtain aggregated events, and combine the event feature vectors of the target micro-exploration events in the aggregated events to generate the danger exploration evolution intensity vector of the aggregated events; S4: Based on the trend growth rate of the danger exploration evolution intensity vector, select aggregated events with a trend growth rate higher than a preset stable rate threshold as home danger events, generate danger status labels for the home danger events, and conduct graded early warnings based on the danger status labels.

[0007] As a further improvement of the present invention: Furthermore, step S1 involves collecting home scene behavior data sequences using multi-source sensing devices, including: The multi-source sensing device includes a 4K high-definition smart camera and a TOF depth sensor. The 4K high-definition smart camera is used to capture images in a home scene and identify facial identity tags in the images. The TOF depth sensor is used to obtain the depth distance between pixels in the image and the TOF depth sensor, and convert it into the three-dimensional coordinates of the pixels. The home scene behavior data sequence consists of continuous images in the home scene and image information, wherein the image information includes identity tags in the images and the three-dimensional coordinates of pixels in the images. The types of identity tags include children and adults.

[0008] Furthermore, the target association of the home scene behavior data sequence in step S1 also includes: S11: Perform environmental target detection on the images in the home scene behavior data sequence to obtain the target category and target bounding box of the environmental target in the image, wherein the target category includes people, furniture and dangerous objects in the home; S12: Based on the three-dimensional coordinates of the pixels in the image, calculate the centroid of the environmental target whose target category is a person, and obtain the three-dimensional coordinates of the target bounding box of the environmental target whose target category is furniture and dangerous objects in the home. Use the centroid as the target information of the environmental target whose target category is a person, and use the three-dimensional coordinates of the target bounding box as the target information of the environmental target whose target category is furniture and dangerous objects in the home. S13: Perform association matching on environmental targets between adjacent images in the home scene behavior data sequence, and generate association matching labels for the same environmental targets between adjacent images, wherein the same environmental targets have consistent association matching labels; S14: The home scene behavior data sequence is constructed by combining the collection timestamps corresponding to the images, the target categories, identity tags, target information, and associated matching tags of the environmental targets in the images.

[0009] Further, in step S2, the target micro-exploration events are extracted using a multi-dimensional behavior trajectory determination algorithm based on the home scene behavior association data, including: S21: Extract images of environmental targets with the identity tag of children from the home scene behavior association data, sort the extracted images according to the collection timestamp order to form multiple image sequences, wherein each image sequence contains multiple images with consecutive collection timestamps; S22: Based on the target category, identity label, target information, and associated matching label of the environmental target in the image, calculate the distance change, displacement amplitude, direction change angle, and contact duration of the environmental target with the identity label of child relative to the dangerous object in the home, and use them as the child exploration behavior feature vector of the environmental target with the identity label of child in the image; S23: Based on the child exploration behavior feature vectors of all images in the image sequence, calculate the weak exploration judgment value of the image sequence, and select image sequences whose weak exploration judgment values ​​exceed the preset behavior benchmark threshold as target micro-exploration events.

[0010] Furthermore, the generation of the event feature vector of the target micro-exploration event in step S2 further includes: The mean distance between the centroids of environmental targets labeled as adults and environmental targets labeled as children in the image of the target micro-exploration event is calculated as the monitoring distance of the image; Calculate the average monitoring distance of all images in the target micro-exploration event, and use it as the monitoring information of the target micro-exploration event; The proportion of images of environmental targets with adult identity labels, guardianship information, average approach speed of children approaching dangerous objects in the home, and approach frequency of children approaching dangerous objects in the home are obtained as the event feature vector of the target micro-exploration event.

[0011] Furthermore, in step S3, constructing a sliding time window to perform sliding aggregation of the target micro-exploration events includes: The target micro-exploration events are sorted according to the acquisition timestamp order of the starting images in the target micro-exploration events. A sliding time window with a fixed length and a fixed sliding step size is constructed. The sorted target micro-exploration events are slid through, and multiple target micro-exploration events falling into the same sliding time window are aggregated to form an aggregated event.

[0012] Furthermore, in step S3, the generation of the danger exploration evolution intensity vector of the aggregated event by combining the event feature vectors of the target micro-exploration events in the aggregated event also includes: S31: Normalize the event feature vector of the target micro-exploration event in the aggregated event, and calculate the single event danger intensity of the target micro-exploration event using an exponential weighting method; Specifically, the normalized event feature vector of the m-th group of target micro-exploration events in the aggregated events is: : ; in, M represents the total number of target micro-exploration events in the aggregated event. The numbers represent the proportion of images with the normalized identity label of adults in the m-th group of target micro-exploration events, the guardianship information, the average approach speed of children approaching dangerous objects in the home, and the approach frequency of children approaching dangerous objects in the home, respectively. The formula for calculating the single-event hazard intensity of the m-th group of target micro-exploration events in the aggregated events is as follows: ; ; in, This represents the single-event hazard intensity of the m-th target micro-exploration event in the aggregated event. This represents the weighted sum of the normalized event feature vectors of the m-th group of target micro-exploration events in the aggregated events. All represent weighting coefficients; S32: Based on the single-event hazard intensity of the target micro-exploration event, calculate the overall hazard accumulation index, hazard trend term, and hazard acceleration term of the aggregated event; Specifically, the formulas for calculating the overall hazard accumulation index, hazard trend term, and hazard acceleration term of the aggregated event are as follows: ; ; ; in, These represent the overall cumulative risk index, risk trend term, and risk acceleration term of the aggregated event, respectively. Indicates the time weighting coefficient. This represents the time-weighted term for the m-th group of target micro-exploration events in the aggregated events, ensuring that the individual event hazard intensity of later-occurring target micro-exploration events accounts for a higher proportion of the overall hazard accumulation index. Represents an exponential function with the natural constant as its base; S33: The overall hazard accumulation index, hazard trend term, and hazard acceleration term of the aggregated event are concatenated into a hazard exploration evolution intensity vector.

[0013] Furthermore, the formula for calculating the trend growth rate of the danger exploration evolution intensity vector is as follows: ; in, This represents the rate of increase in the trend of the danger exploration evolution intensity vector F. ,in In order, they are the overall hazard accumulation index, the hazard trend term, and the hazard acceleration term. All of these represent modulation parameters.

[0014] Furthermore, step S4, which involves selecting aggregated events with a trend growth rate exceeding a preset stable rate threshold as home hazard events and generating hazard status labels for these events, also includes: S41: Extract the overall cumulative risk index, the maximum value of the single-event risk intensity, and the trend growth rate of the home hazard events, and calculate the comprehensive risk score of the home hazard events; S42: Based on the comprehensive risk score of the home hazard event, generate a hazard status label for the home hazard event using a graded threshold method.

[0015] The present invention also proposes an end-to-cloud collaborative home security system, which includes an end-side module, a cloud-based analysis module, and an application interaction terminal, to realize the end-to-cloud collaborative home security method described above.

[0016] Compared with existing technologies, this invention proposes a home security method and system with edge-cloud collaboration, which has the following beneficial effects: First, this invention normalizes the event feature vector of target micro-exploration events and introduces an exponential weighting function to construct the single-event danger intensity, achieving nonlinear fusion of different types of behavioral characteristics and effectively enhancing sensitivity to high-risk behaviors (such as rapid approach and unsupervised behavior). Based on this, by introducing a time decay weight to weighted accumulate the single-event danger intensity, an overall danger accumulation index is obtained, making behaviors closer to the current moment contribute more to the overall risk assessment, thereby improving the response capability to real-time risks. Furthermore, this invention constructs danger trend and danger acceleration terms through first-order and second-order differences respectively, characterizing the direction and rate of risk evolution. It can identify the gradual process of risk from low to high and sudden acceleration processes. Then, by splicing multi-dimensional risk indicators to form a danger exploration evolution intensity vector, it achieves continuous evolution modeling of children's dangerous exploration behavior, providing high-precision, multi-dimensional data support for subsequent dangerous event judgment and graded early warning.

[0017] Meanwhile, this invention calculates the trend growth rate, which is a quantitative measure of the speed at which children's dangerous exploration behavior evolves into a higher risk state based on the current risk level and the trend of change. The larger the value, the faster the children's dangerous exploration behavior evolves into a high-risk state, and the risk shows a continuous or accelerating growth trend, thereby effectively reflecting the degree of evolution and urgency of potential dangerous behavior. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a cloud-edge collaborative home security method according to an embodiment of the present invention. Figure 2 This is a diagram of an end-to-cloud collaborative architecture provided in an embodiment of the present invention; Figure 3 This is a comparison chart of home safety monitoring and early warning provided in an embodiment of the present invention.

[0019] In the diagram: 101, End-side module; 102, Cloud analysis module; 103, Application interaction terminal; 11, Multi-source sensing device; 12, Computing module; 13, Communication module; 14, Guardian terminal.

[0020] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] The realization of the objectives, functional characteristics, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0022] This invention provides a home security method and system with edge-cloud collaboration. The executing entity of this edge-cloud collaborative home security method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this invention: a server, a terminal, etc. In other words, this edge-cloud collaborative home security method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0023] Reference Figure 1 As shown, Embodiment 1 of the present invention is as follows: An edge-cloud collaborative home security method includes the following steps: S1: Collect home scene behavior data sequences using multi-source sensing devices, perform target association on the home scene behavior data sequences, and obtain target-associated home scene behavior association data.

[0024] Specifically, this involves using multi-source sensing devices to collect behavioral data sequences in home settings, including: As an embodiment of the present invention, the multi-source sensing device includes a 4K high-definition smart camera and a TOF depth sensor; The 4K high-definition smart camera integrates infrared night vision and a face recognition module to collect images in real time in home scenarios and identify the identity tags of faces in the images. The identity tag types include children and adults. The TOF depth sensor uses the image area captured by the 4K high-definition smart camera as the detection area. During the image capture process, the transmitter of the TOF depth sensor synchronously emits modulated near-infrared light into the detection area. After the light comes into contact with the monitored target (such as children, adults, furniture, dangerous items, etc.), it is reflected back to the receiver. The time difference of flight of the light from emission to reception is calculated by a dedicated chip, and the depth distance from the TOF depth sensor to each pixel in the captured image is calculated by combining the speed of light formula. Based on the pixel coordinates of the pixels in the captured image and the intrinsic and extrinsic parameters of the 4K high-definition smart camera, the spatial geometric projection algorithm is used to fuse and calculate the pixel coordinates and the depth distance from the TOF depth sensor to each pixel in the captured image, so as to obtain the three-dimensional coordinates of any pixel in the captured image of the 4K high-definition smart camera, and realize the real-time spatial positioning of the monitored target in the image. Specifically, a unified timestamp synchronization command is sent to all multi-source sensing devices through the home smart gateway, so that the acquisition time of the 4K high-definition smart camera and the TOF depth sensor are accurately aligned, and the images of the home scene and the three-dimensional coordinates of the pixels in the images are acquired at the same acquisition frequency. Optionally, the acquisition frequency is 10 frames / second. The multi-source sensing device includes a 4K high-definition smart camera and a TOF depth sensor. The 4K high-definition smart camera is used to capture images in a home scene and identify facial identity tags in the images. The TOF depth sensor is used to obtain the depth distance between pixels in the image and the TOF depth sensor, and convert it into the three-dimensional coordinates of the pixels. The home scene behavior data sequence consists of continuous images in the home scene and image information, wherein the image information includes identity tags in the images and the three-dimensional coordinates of pixels in the images. The types of identity tags include children and adults.

[0025] Step S1, which involves target association of the home scene behavior data sequence, also includes: S11: Perform environmental target detection on the images in the home scene behavior data sequence to obtain the target category and target bounding box of the environmental target in the image, wherein the target category includes people, furniture and dangerous objects in the home; Specifically, a deep learning detection model is used for environmental target detection. The deep learning detection model includes, but is not limited to, YOLO series models (such as YOLOv5, YOLOv8), Faster R-CNN model, DETR model or MobileNet-SSD lightweight model, which are used to detect and classify people, furniture and potentially dangerous items in home scene images. As an embodiment of the present invention, the dangerous objects in the home include sharp dangerous objects (such as knives, scissors, etc.) and high-temperature dangerous objects (such as kettles, stoves, etc.). S12: Based on the three-dimensional coordinates of the pixels in the image, calculate the centroid of the environmental target whose target category is a person, and obtain the three-dimensional coordinates of the target bounding box of the environmental target whose target category is furniture and dangerous objects in the home. Use the centroid as the target information of the environmental target whose target category is a person, and use the three-dimensional coordinates of the target bounding box as the target information of the environmental target whose target category is furniture and dangerous objects in the home. S13: Perform association matching on environmental targets between adjacent images in the home scene behavior data sequence, and generate association matching labels for the same environmental targets between adjacent images, wherein the same environmental targets have consistent association matching labels; In one embodiment of the present invention, for environmental targets of the category of people between adjacent images, the association score of the environmental targets from adjacent images is calculated by obtaining the identity label and centroid of the environmental targets and using an association scoring function. When the association score is higher than a preset scoring threshold (default setting is 0.8), the two environmental targets are determined to be the same environmental targets in adjacent images. The expression of the association scoring function is as follows: ; in, This indicates the correlation scoring function. If the target category between adjacent images is a person in the environment, then... Indicate environmental goals Correlation ratings between them These represent the environmental objectives in order. Identity tags, if Consistent, then =1, otherwise =0, Indicate environmental goals The Euclidean distance between the centroids of the two points. This indicates the distance control parameter, with default settings. It is 5 meters. Represents an exponential function with the natural constant as its base; Specifically, the association score represents the degree of spatial consistency between two environmental targets in adjacent images under the constraint of identity consistency; For environmental targets in adjacent images that are of the same category and are furniture or dangerous objects in the home, the area of ​​the target bounding box and the center pixel coordinates are compared. When the deviation between the center pixel coordinates and the area of ​​the target bounding box is lower than the preset coordinate distance threshold (default setting is 10 pixels) and the preset area threshold (default setting is 20 pixels), respectively, the two environmental targets are determined to be the same environmental targets in adjacent images. S14: The home scene behavior data sequence is constructed by combining the collection timestamps corresponding to the images, the target categories, identity tags, target information, and associated matching tags of the environmental targets in the images.

[0026] It should be noted that this invention constructs an association scoring function based on the fusion of identity tags and spatial distance to determine the consistency of human targets in adjacent images. While ensuring the consistency of identity tags, an exponential decay mechanism is introduced to constrain spatial displacement, thereby effectively achieving accurate modeling of the continuity of targets across frames. Simultaneously, for furniture and dangerous objects, matching is performed based on the center position and area changes of the target bounding box, further improving the association stability of static or slowly moving targets. The above method can effectively reduce target mismatch and loss problems in complex home scenarios, improve the accuracy and continuity of target trajectory construction, provide a reliable data foundation for subsequent children's behavior analysis and danger exploration identification, and significantly enhance the system's adaptability and robustness to multi-target dynamic interaction scenarios.

[0027] S2: Based on the home scene behavior association data, extract the target micro-exploration event using a multi-dimensional behavior trajectory determination algorithm, and generate the event feature vector of the target micro-exploration event.

[0028] Specifically, based on the aforementioned home scene behavior association data, a multi-dimensional behavior trajectory determination algorithm is used to extract target micro-exploration events, including: S21: Extract images of environmental targets with the identity tag of children from the home scene behavior association data, sort the extracted images according to the collection timestamp order to form multiple image sequences, wherein each image sequence contains multiple images with consecutive collection timestamps; S22: Based on the target category, identity label, target information, and associated matching label of the environmental target in the image, calculate the distance change, displacement amplitude, direction change angle, and contact duration of the environmental target with the identity label of child relative to the dangerous object in the home, and use them as the child exploration behavior feature vector of the environmental target with the identity label of child in the image; Specifically, the shortest distance between the centroid of the environmental target labeled as a child in the image and the three-dimensional coordinates of the bounding box of the home hazard is calculated, and this distance is used as the distance between the environmental target labeled as a child in the image and the home hazard. The formulas for calculating the change in distance, displacement amplitude, direction change angle, and contact mark of the environmental target labeled as a child relative to the home hazard are as follows: ; ; ; ; in, This represents the change in distance between an environmental target labeled "child" and a dangerous object in the home, as shown in the t-th image of the image sequence. A positive value indicates proximity to a dangerous object near the home. This represents the distance between an environmental object labeled "child" and a dangerous object in the home in the t-th image of the image sequence. This represents the distance between an environmental object labeled "child" and a dangerous object in the home in the (t-1)th image of the image sequence. Let represent the centroid of the environmental object labeled "child" in the t-th image of the image sequence. Let represent the centroid of the environmental object labeled "child" in the (t-1)th image of the image sequence. Let represent the centroid of the environmental object labeled "child" in the (t-2)th image of the image sequence. Indicates the center of mass The Euclidean distance between them This represents the displacement magnitude of the environmental target labeled "child" in the t-th image of the image sequence. Represents the inverse cosine function. Represents the L2 norm. This represents the angle of change of orientation of the environmental target labeled "child" in the t-th image of the image sequence. This represents the contact identifier of an environmental target labeled as a child in the t-th image of the image sequence. Indicates the contact distance threshold, default setting. It is 0.2 meters. N represents the total number of images in the image sequence; S23: Based on the child exploration behavior feature vectors of all images in the image sequence, calculate the weak exploration judgment value of the image sequence, and select image sequences whose weak exploration judgment values ​​exceed the preset behavior benchmark threshold as target micro-exploration events.

[0029] Specifically, the formula for calculating the weak exploration determination value of the image sequence is as follows: ; ; in, Indicates distance control parameters, This represents the weak exploration judgment value of the image sequence. It represents 180 degrees. This represents a normalized index indicating the change in distance, displacement, and directional change angle of an environmental target labeled as a child relative to a dangerous object in the home in the t-th image of the image sequence. Indicates selection The maximum value in, This indicates the distance change control parameter (default setting is 1 meter). This represents the trend enhancement coefficient, the default setting. It is 0.4.

[0030] Specifically, the weak exploration judgment value reflects a comprehensive measure of the intensity of a child's exploratory approach behavior and contact tendency relative to dangerous objects in the home in a continuous image sequence. The closer the weak exploration judgment value is to 0, the more it indicates that the child basically does not approach the dangerous objects in the home, or the behavior is random and has no obvious exploratory intention. It should be noted that this invention constructs multi-dimensional behavioral features that integrate distance change, displacement amplitude, and directional change angle, and introduces contact markers for trend enhancement. This allows for the unified quantification of children's behavior in continuous image sequences, effectively characterizing the intensity of children's exploratory behavior relative to dangerous objects. Specifically, by taking the positive component of distance change, it achieves accurate capture of proactive approach behavior; by normalizing displacement amplitude and directional change angle, it improves the ability to identify behavioral activity and tentativeness; and by combining this with a contact behavior proportion enhancement term... Its sensitivity to high-risk behaviors enables it to identify subtle, hidden exploratory behaviors from continuous time-series data, avoiding false positives or false negatives caused by relying solely on single-frame judgments. This provides a reliable basis for the identification and evolution analysis of subsequent dangerous exploratory events, significantly improving the ability to detect early-stage risky behaviors and provide early warnings.

[0031] The generation of the event feature vector of the target micro-exploration event in step S2 further includes: The mean distance between the centroids of environmental targets labeled as adults and environmental targets labeled as children in the image of the target micro-exploration event is calculated as the monitoring distance of the image; Calculate the average monitoring distance of all images in the target micro-exploration event, and use it as the monitoring information of the target micro-exploration event; The proportion of images of environmental targets with adult identity labels, guardianship information, average approach speed of children approaching dangerous objects in the home, and approach frequency of children approaching dangerous objects in the home are obtained as the event feature vector of the target micro-exploration event.

[0032] Specifically, the average approach speed of the child towards the dangerous object in the home is The frequency with which children approach dangerous objects in the home is... .

[0033] S3: Construct a sliding time window, perform sliding aggregation on the target micro-exploration events to obtain aggregated events, and combine the event feature vectors of the target micro-exploration events in the aggregated events to generate the danger exploration evolution intensity vector of the aggregated events.

[0034] Specifically, a sliding time window is constructed to aggregate the target micro-exploration events, including: The target micro-exploration events are sorted according to the acquisition timestamp order of the starting images in the target micro-exploration events. A sliding time window with a fixed length and a fixed sliding step size is constructed. The sorted target micro-exploration events are slid through, and multiple target micro-exploration events falling into the same sliding time window are aggregated to form an aggregated event.

[0035] Step S3, which combines the event feature vectors of the target micro-exploration events in the aggregated event to generate the danger exploration evolution intensity vector of the aggregated event, further includes: S31: Normalize the event feature vector of the target micro-exploration event in the aggregated event, and calculate the single event danger intensity of the target micro-exploration event using an exponential weighting method; Specifically, the normalized event feature vector of the m-th group of target micro-exploration events in the aggregated events is: : ; in, M represents the total number of target micro-exploration events in the aggregated event. The numbers represent the proportion of images with the normalized identity label of adults in the m-th group of target micro-exploration events, the guardianship information, the average approach speed of children approaching dangerous objects in the home, and the approach frequency of children approaching dangerous objects in the home, respectively. The formula for calculating the single-event hazard intensity of the m-th group of target micro-exploration events in the aggregated events is as follows: ; ; in, This represents the single-event hazard intensity of the m-th target micro-exploration event in the aggregated event. This represents the weighted sum of the normalized event feature vectors of the m-th group of target micro-exploration events in the aggregated events. All represent weighting coefficients, default settings. The values ​​are 0.3, 0.25, 0.25, and 0.2, respectively. S32: Based on the single-event hazard intensity of the target micro-exploration event, calculate the overall hazard accumulation index, hazard trend term, and hazard acceleration term of the aggregated event; Specifically, the formulas for calculating the overall hazard accumulation index, hazard trend term, and hazard acceleration term of the aggregated event are as follows: ; ; ; in, These represent the overall cumulative risk index, risk trend term, and risk acceleration term of the aggregated event, respectively. This represents the time weighting coefficient, set by default. It is 0.2. This represents the time-weighted term of the m-th group of target micro-exploration events in the aggregated events, which makes the single-event hazard intensity of later target micro-exploration events account for a higher proportion in the overall hazard accumulation index; S33: The overall hazard accumulation index, hazard trend term, and hazard acceleration term of the aggregated event are concatenated into a hazard exploration evolution intensity vector.

[0036] S4: Based on the trend growth rate of the danger exploration evolution intensity vector, select aggregated events with a trend growth rate higher than a preset stable rate threshold as home danger events, generate danger status labels for the home danger events, and conduct graded early warnings based on the danger status labels.

[0037] The formula for calculating the trend growth rate of the danger exploration evolution intensity vector is as follows: ; in, This represents the rate of increase in the trend of the danger exploration evolution intensity vector F. ,in In order, they are the overall hazard accumulation index, the hazard trend term, and the hazard acceleration term. All represent modulation parameters, default settings. The values ​​are 0.65, 0.35, and 0.5, respectively.

[0038] Step S4, which involves selecting aggregated events with a trend growth rate higher than a preset stable rate threshold as home hazard events and generating hazard status labels for these events, also includes: Specifically, the preset stable rate threshold is 0.15; S41: Extract the overall cumulative risk index, the maximum value of the single-event risk intensity, and the trend growth rate of the home hazard events, and calculate the comprehensive risk score of the home hazard events; Specifically, the formula for calculating the comprehensive risk score of the aforementioned home-related dangerous events is as follows: ; in, A comprehensive risk score indicating dangerous events at home. These represent, in order, the overall cumulative risk index of home-related hazardous events, the maximum value of the risk intensity of a single event, and the trend growth rate. All represent risk weighting coefficients, default settings. The values ​​are 0.3, 0.3, and 0.4 respectively. S42: Based on the comprehensive risk score of the home hazard event, generate a hazard status label for the home hazard event using a graded threshold method.

[0039] Specifically, the formula for generating hazard status labels based on grading thresholds is as follows: ; in, Indicates the overall risk score The corresponding danger status label, This indicates the grading threshold; the default setting is... They are 0.4 and 0.7 respectively; As a preferred embodiment of the present invention, the average proportion (rate) of the number of environmental targets with the identity label of adults after normalization processing of all target micro-exploration events in the home hazard event is extracted to construct an enhanced label for the home hazard event. Specifically, the label types of the enhanced label include sudden danger type and unsupervised danger type. If the trend growth rate of the home hazard event is higher than the high rate threshold (default setting is 0.22), then the enhanced label of the home hazard event is sudden danger type. If the average proportion (rate) of the number of images of the home hazard event is lower than the preset monitoring threshold (default setting is 0.2), then the enhanced label of the home hazard event is unsupervised type. Further, the enhanced label of the home hazard event may not exist or may have two types of enhanced labels. Based on the danger status labels of the aforementioned home-related dangerous events, basic warning levels are set for low risk, medium risk, and high risk, respectively: alert level, warning level, and emergency level. The alert level warning strategy is to record the event log and display a warning message on the guardian's terminal interface. The warning level warning strategy is to send a warning notification to the guardian's terminal and activate a local voice reminder (such as "Please pay attention to children's behavior"). The emergency level warning strategy is to trigger continuous voice warnings and activate remote video streaming (cloud linkage). Furthermore, based on the number of enhanced tags for the home hazard events, the basic warning level of the hazard status tag is increased, wherein the number of increased levels is equal to the number of enhanced tags for the home hazard events.

[0040] It should be noted that this invention constructs a comprehensive risk score by integrating the overall risk accumulation index, the peak intensity of the single event risk intensity, and the trend growth rate, thereby achieving a unified quantification of the intensity and evolution mode of dangerous behavior; it uses graded thresholds to generate hazard status labels, making risk assessment clearly hierarchical; at the same time, this invention further introduces enhanced labels for sudden danger and unsupervised danger, and dynamically upgrades the warning level based on the number of labels, thereby achieving a key response to high-growth risks and scenarios lacking supervision.

[0041] Example 2: An edge-cloud collaborative home security system to implement the edge-cloud collaborative home security method as described in Example 1, referring to... Figure 2 The edge-cloud collaborative architecture diagram includes an edge module 101, a cloud analysis module 102, and an application interaction terminal 103. The edge module includes a multi-source sensing device 11 and a computing module 12. The application interaction terminal 103 includes a communication module 13 and a guardian terminal 14 (such as a mobile phone, computer, etc.). The communication module 13 is used to transmit messages to the cloud analysis module 102 (such as starting the video stream of the multi-source sensing device 11 for remote viewing). The end-side module 101 is used to collect home scene behavior data sequences using multi-source sensing devices 11. The calculation module 12 performs target association on the home scene behavior data sequences to obtain target-associated home scene behavior association data. The multi-dimensional behavior trajectory determination algorithm is used to extract target micro-exploration events and generate event feature vectors of the target micro-exploration events. The cloud analysis module 102 is used to construct a sliding time window, perform sliding aggregation on the target micro-exploration events to obtain aggregated events, and combine the event feature vectors of the target micro-exploration events in the aggregated events to generate the danger exploration evolution intensity vector of the aggregated events. Based on the trend growth rate of the danger exploration evolution intensity vector, aggregated events with a trend growth rate higher than a preset stable rate threshold are selected as home danger events, and danger status labels of the home danger events are generated. Based on the danger status labels, graded warnings are issued. The application interaction terminal 103 is used to execute the early warning strategy and receive early warning information.

[0042] Example 3: As another embodiment of the present invention, a 1:1 simulated home experimental scenario (including living room, children's room, and kitchen, with typical dangerous items such as high and low cabinets, knife racks, and electrical appliances) is built, and the multi-source sensing device and edge-cloud collaborative architecture described in the present invention are deployed; at the same time, a traditional method monitoring platform (containing only ordinary cameras and using a fixed threshold detection algorithm) is built. We recruited 50 children aged 3-6 (core monitoring subjects for home safety) and adults. We designed 5 types of typical children's micro-exploration behaviors (short-term stay, repeated approach, slight touch, tentative pulling, wandering around dangerous objects), 3 types of normal children's behaviors (playing, looking for toys, wandering around), and 2 types of adult interference behaviors (adults touching dangerous objects, adults engaging in activities in dangerous areas). For each behavior, we designed 20 sets of repeated experiments, generating a total of 2000 sets of behavioral samples. In the same simulated scenario, this method and traditional methods are used simultaneously to monitor all behavioral samples in real time, and the accuracy of early warning triggering and the rate of missed early warnings are recorded, as shown in the example below. Figure 3The comparison chart of home safety monitoring and early warning shown in the figure demonstrates that the method described in this invention has higher early warning triggering accuracy and a lower rate of missed early warnings compared to traditional methods.

[0043] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in terms of the scope of the patent invention.

[0044] It should be noted that the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0045] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0046] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A home security method with edge-cloud collaboration, characterized in that, The method includes: S1: Collect home scene behavior data sequences using multi-source sensing devices, perform target association on the home scene behavior data sequences, and obtain target-associated home scene behavior association data; S2: Based on the home scene behavior association data, extract the target micro-exploration event using a multi-dimensional behavior trajectory determination algorithm, and generate the event feature vector of the target micro-exploration event; S3: Construct a sliding time window, perform sliding aggregation on the target micro-exploration events to obtain aggregated events, and combine the event feature vectors of the target micro-exploration events in the aggregated events to generate the danger exploration evolution intensity vector of the aggregated events; S4: Based on the trend growth rate of the danger exploration evolution intensity vector, select aggregated events with a trend growth rate higher than a preset stable rate threshold as home danger events, generate danger status labels for the home danger events, and conduct graded early warnings based on the danger status labels.

2. The home security method with end-to-end cloud collaboration as described in claim 1, characterized in that, Step S1 involves collecting home scene behavior data sequences using multi-source sensing devices, including: The multi-source sensing device includes a 4K high-definition smart camera and a TOF depth sensor. The 4K high-definition smart camera is used to capture images in a home scene and identify facial identity tags in the images. The TOF depth sensor is used to obtain the depth distance between pixels in the image and the TOF depth sensor, and convert it into the three-dimensional coordinates of the pixels. The home scene behavior data sequence consists of continuous images in the home scene and image information, wherein the image information includes identity tags in the images and the three-dimensional coordinates of pixels in the images. The types of identity tags include children and adults.

3. The home security method with end-to-end cloud collaboration as described in claim 2, characterized in that, Step S1, which involves target association of the home scene behavior data sequence, also includes: S11: Perform environmental target detection on the images in the home scene behavior data sequence to obtain the target category and target bounding box of the environmental target in the image, wherein the target category includes people, furniture and dangerous objects in the home; S12: Based on the three-dimensional coordinates of the pixels in the image, calculate the centroid of the environmental target whose target category is a person, and obtain the three-dimensional coordinates of the target bounding box of the environmental target whose target category is furniture and dangerous objects in the home. Use the centroid as the target information of the environmental target whose target category is a person, and use the three-dimensional coordinates of the target bounding box as the target information of the environmental target whose target category is furniture and dangerous objects in the home. S13: Perform association matching on environmental targets between adjacent images in the home scene behavior data sequence, and generate association matching labels for the same environmental targets between adjacent images, wherein the same environmental targets have consistent association matching labels; S14: The home scene behavior data sequence is constructed by combining the collection timestamps corresponding to the images, the target categories, identity tags, target information, and associated matching tags of the environmental targets in the images.

4. The home security method with end-to-end cloud collaboration as described in claim 1, characterized in that, In step S2, target micro-exploration events are extracted based on the home scene behavior association data using a multi-dimensional behavior trajectory determination algorithm, including: S21: Extract images of environmental targets with the identity tag of children from the home scene behavior association data, sort the extracted images according to the collection timestamp order to form multiple image sequences, wherein each image sequence contains multiple images with consecutive collection timestamps; S22: Based on the target category, identity label, target information, and associated matching label of the environmental target in the image, calculate the distance change, displacement amplitude, direction change angle, and contact duration of the environmental target with the identity label of child relative to the dangerous object in the home, and use them as the child exploration behavior feature vector of the environmental target with the identity label of child in the image; S23: Based on the child exploration behavior feature vectors of all images in the image sequence, calculate the weak exploration judgment value of the image sequence, and select image sequences whose weak exploration judgment values ​​exceed the preset behavior benchmark threshold as target micro-exploration events.

5. A home security method with end-to-end cloud collaboration as described in claim 4, characterized in that, The generation of the event feature vector of the target micro-exploration event in step S2 further includes: The mean distance between the centroids of environmental targets labeled as adults and environmental targets labeled as children in the image of the target micro-exploration event is calculated as the monitoring distance of the image; Calculate the average monitoring distance of all images in the target micro-exploration event, and use it as the monitoring information of the target micro-exploration event; The proportion of images of environmental targets with adult identity labels, guardianship information, average approach speed of children approaching dangerous objects in the home, and approach frequency of children approaching dangerous objects in the home are obtained as the event feature vector of the target micro-exploration event.

6. The home security method with end-to-end cloud collaboration as described in claim 1, characterized in that, Step S3 involves constructing a sliding time window and performing sliding aggregation on the target micro-exploration events, including: The target micro-exploration events are sorted according to the acquisition timestamp order of the starting images in the target micro-exploration events. A sliding time window with a fixed length and a fixed sliding step size is constructed. The sorted target micro-exploration events are slid through, and multiple target micro-exploration events falling into the same sliding time window are aggregated to form an aggregated event.

7. A home security method with end-to-end cloud collaboration as described in claim 6, characterized in that, Step S3, which combines the event feature vectors of the target micro-exploration events in the aggregated event to generate the danger exploration evolution intensity vector of the aggregated event, further includes: S31: Normalize the event feature vector of the target micro-exploration event in the aggregated event, and calculate the single event danger intensity of the target micro-exploration event using an exponential weighting method; S32: Based on the single-event hazard intensity of the target micro-exploration event, calculate the overall hazard accumulation index, hazard trend term, and hazard acceleration term of the aggregated event; S33: The overall hazard accumulation index, hazard trend term, and hazard acceleration term of the aggregated event are concatenated into a hazard exploration evolution intensity vector.

8. A home security method with end-to-end cloud collaboration as described in claim 1, characterized in that, The formula for calculating the trend growth rate of the danger exploration evolution intensity vector is as follows: ； in, This represents the rate of increase in the trend of the danger exploration evolution intensity vector F. ,in In order, they are the overall hazard accumulation index, the hazard trend term, and the hazard acceleration term. All of these represent modulation parameters.

9. A home security method with end-to-end cloud collaboration as described in claim 8, characterized in that, Step S4, which involves selecting aggregated events with a trend growth rate higher than a preset stable rate threshold as home hazard events and generating hazard status labels for these events, also includes: S41: Extract the overall cumulative risk index, the maximum value of the single-event risk intensity, and the trend growth rate of the home hazard events, and calculate the comprehensive risk score of the home hazard events; S42: Based on the comprehensive risk score of the home hazard event, generate a hazard status label for the home hazard event using a graded threshold method.

10. A home security system with edge-cloud collaboration, characterized in that, The home security system includes an edge module, a cloud analysis module, and an application interaction terminal to realize a cloud-edge collaborative home security method as described in any one of claims 1-9.

Images