Intelligent door and window self-adaptive opening and closing method based on AI behavior recognition

Abstract

The invention discloses an intelligent door and window self-adaptive opening and closing method based on AI behavior recognition, and relates to the field of computer vision, and the method comprises the steps: collecting a monitoring video image in real time, and obtaining an image frame sequence of each portrait region; joint points are extracted and classified and numbered, coordinates of all the joint points on each frame of image are obtained, and the motion smoothness of each joint point is quantified; on the basis of the motion smoothness of the joint points, calculating the motion trail correlation between various joint point pairs in each portrait area; acquiring a reference value of the normal behavior action based on the monitoring video image of the historical normal behavior action; based on the reference value of the normal behavior action, quantifying the abnormal degree of each portrait area in the current monitoring video image; based on the abnormal degree, abnormal behaviors are recognized, and door and window opening and closing control is executed. Abnormal behavior actions can be accurately recognized, the false alarm rate of abnormal behavior detection is reduced, and self-adaptive intelligent opening and closing of doors and windows are achieved.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to an intelligent door and window adaptive opening and closing method based on AI behavior recognition. Background Technology

[0002] The development of intelligent door and window automatic control systems stems from the continuous pursuit of home security and convenience. Existing intelligent anti-theft systems detect suspicious individuals by installing cameras to capture images of potential intrusion points, such as fences, doors, and windows. When sensors detect abnormal activity, the control center triggers an alarm. However, current intelligent anti-theft systems have limited detection range and experience a high number of false alarms, making accurate analysis and judgment based on specific situations impossible.

[0003] Existing anti-theft systems primarily rely on motion sensors or simple image recognition for anomaly detection. For example, traditional infrared or radar-based solutions can only detect moving objects but cannot understand motion behavior, making it difficult to distinguish between people, animals, and tree branch movements, resulting in a false alarm rate exceeding 30%. Traditional image recognition solutions that combine moving target detection with boundary crossing alarms lack semantic analysis of actions, easily misinterpreting pedestrians approaching the fence as intruders. Basic behavior recognition solutions, such as simple behavior recognition schemes based on Open-Pose joint point models and trajectory analysis, do not consider the correlation of joint movements, making it difficult to distinguish the physical characteristics of different action types, easily identifying behaviors such as running and jumping as abnormal, such as misinterpreting a passerby waving as climbing. Common commercial anti-theft systems, such as Hikvision's boundary crossing detection system, rely solely on simple spatial rules and lack action posture analysis, easily misinterpreting normal pedestrian activities near the fence as abnormal, such as children playing.

[0004] Existing technical solutions have not solved the problem of identifying subtle differences between normal and abnormal behaviors. This often leads to existing intelligent anti-theft systems confusing abnormal behaviors with normal ones, resulting in false alarms. For example, actions such as climbing over fences are easily confused with pedestrians running or jumping, causing alarms to be triggered falsely. Therefore, in order to reduce the false alarm rate and achieve adaptive intelligent opening and closing control of doors and windows, it is necessary to more accurately distinguish and identify the actions and behaviors of pedestrians within the monitoring range in order to accurately determine whether there are abnormal behaviors such as climbing or climbing over fences. Summary of the Invention

[0005] To more accurately distinguish and identify abnormal behaviors such as climbing and vaulting, and to achieve adaptive intelligent opening and closing of doors and windows, thus solving the problem of high false alarm rates caused by existing intelligent anti-theft systems easily confusing pedestrian behavior, this invention provides an intelligent door and window adaptive opening and closing method based on AI behavior recognition. The technical solution is as follows: In a first aspect, the present invention provides an intelligent door and window adaptive opening and closing method based on AI behavior recognition, the steps of which include: real-time acquisition of monitoring video images and extraction of human image regions for marking and tracking, to obtain an image frame sequence for each human image region; extraction of key points within the image frame sequence of human image regions and classification and numbering, obtaining the coordinates of all key points on each frame image, and quantifying the motion smoothness of each key point; based on the motion smoothness of the key points, calculating the motion trajectory correlation between various key point pairs within each human image region; based on monitoring video images of historical normal behavior actions, obtaining a baseline value of normal behavior actions; based on the baseline value of normal behavior actions, quantifying the degree of abnormality of each human image region in the current monitoring video image; and based on the degree of abnormality, identifying abnormal behavior and executing door and window opening and closing control.

[0006] Preferably, the monitoring video images and corresponding timestamps are acquired in real time by a camera, and the bounding box of the human figure region in each frame of the monitoring video image is extracted by the YOLO algorithm. When multiple human figures appear in the monitoring video image, a marker ID is assigned to each human figure that appears in the monitoring range by combining the SORT algorithm, and the movement trajectory of each human figure is tracked to obtain the image frame sequence of each human figure region.

[0007] Preferably, based on the image frame sequence of the human image region, the HR-Net network is used to extract the key points in all human image regions in each frame image, and the key points in each human image region are classified and numbered in the same order; the coordinates of all key points in each frame image are obtained, and the difference vector between the coordinates of a key point in the current frame image and the coordinates of the same key point in the next frame image is used as the moving speed of the corresponding key point in the current frame image. The moving speed of the same key point in two adjacent frames image is calculated in turn to obtain the moving speed sequence of the corresponding key point. Similarly, the moving speed sequence of all key points in each human image region is obtained. The mean movement speed of each joint is obtained based on the movement speed sequence of the joints. The magnitude of the vector difference between the movement speed of a certain joint in each frame and the mean movement speed is calculated. The squares of the magnitudes are accumulated. The ratio between the accumulated value and the number of images in the corresponding human portrait region image frame sequence is reverse mapped. The reverse mapped value is then normalized, and the normalized value is used as the motion smoothness of the joint. Similarly, the motion smoothness of all joints is obtained.

[0008] Preferably, based on the movement speed sequence of the joints, the vertical component sequence of the movement speed sequence of all joints in the vertical direction is obtained. Based on the calculation formula of Pearson correlation coefficient, the Pearson correlation coefficient of the vertical component sequence between each pair of joints in the same portrait area is obtained. The absolute value of the difference in motion smoothness between two joints in the same portrait area is back-mapped using the natural exponential function. The product of the back-mapped value and the corresponding Pearson correlation coefficient of the two joints is used as the motion trajectory correlation between the two joints. Similarly, the motion trajectory correlation between each pair of joints in each portrait area is calculated.

[0009] Preferably, historical data of surveillance video images is acquired, and a sequence of consecutive frames containing normal human behavior is extracted from the historical data. Similar to the processing method of real-time acquired surveillance video images, the YOLO algorithm is used to extract the human image region, and the HR-Net network is used to extract the key points in the human image region of each historical image as historical normal key points and classify and number them. Based on the sequence of consecutive frames containing normal human behavior, the coordinates of each key point in each historical image are obtained, and the historical motion smoothness of each historical normal key point and the correlation of historical motion trajectories between various pairs of historical normal key points are calculated.

[0010] Preferably, based on the correlation of historical motion trajectories between various historical normal joint point pairs, the value obtained by subtracting the correlation of historical motion trajectories from 1 is used as the distance metric between the corresponding historical normal joint point pairs. Similarly, the distance matrix of all historical normal joint point pairs is obtained. Using the bottom-up method in hierarchical clustering and setting the number of final clusters, all historical normal joint points are clustered and merged into final clusters. The correlation of historical motion trajectories between all historical normal joint point pairs in each final cluster is extracted and accumulated to obtain the accumulated historical correlation value corresponding to each final cluster. The ratio of the accumulated historical correlation value to the number of historical normal joint point pairs in the corresponding final cluster is used as the historical correlation mean of the final cluster. The historical correlation mean of each final cluster is used as the benchmark value of the corresponding joint point under normal behavior.

[0011] Preferably, based on the classification number of the key points and the division range of historical normal key points by the final cluster, the key points of each portrait area in the real-time acquired monitoring video image are divided into real-time key point groups corresponding to each final cluster. The motion trajectory correlation between all key point pairs in each real-time key point group in each portrait area is extracted and accumulated to obtain the correlation accumulation value corresponding to each real-time key point group. The ratio of the correlation accumulation value to the number of key point pairs in the corresponding real-time key point group is used as the real-time correlation mean of the real-time key point group. Similarly, the real-time correlation mean of all real-time key point groups in each portrait area is calculated.

[0012] Preferably, the ratio between the mean of the correlation between all relevant nodes and motion trajectory within a real-time key point group and the standard deviation of the motion smoothness of all relevant nodes is used as the difference weight between the real-time key point group and the corresponding final cluster. Similarly, the difference weights corresponding to all real-time key point groups within each portrait region are calculated. The absolute value of the difference between the real-time correlation mean of each real-time key point group within the same portrait region and the historical correlation mean of the corresponding final cluster is calculated. The product of the absolute value corresponding to each real-time key point group and the corresponding difference weight is used as the correlation difference feature value of each real-time key point group. The correlation difference feature values ​​of all real-time key point groups within the same portrait region are accumulated to obtain the total correlation difference value of the portrait region. At the same time, the difference weights corresponding to all real-time key point groups within the portrait region are accumulated to obtain the total difference weight value of the portrait region. The ratio of the total correlation difference value to the total difference weight value is used as the anomaly degree of the portrait region. Similarly, the anomaly degree of each portrait region in the current monitoring video image is calculated.

[0013] Preferably, an anomaly judgment threshold is set according to the actual application scenario or experience value. Based on the degree of anomaly of each portrait area in the current monitoring video image, the degree of anomaly of each portrait area is compared with the anomaly judgment threshold. When the degree of anomaly of all portrait areas is less than the anomaly judgment threshold, it is determined that there is no abnormal behavior of any pedestrian in the current monitoring video image, and the current state of the doors and windows is maintained. When the degree of anomaly of any portrait area is greater than or equal to the anomaly judgment threshold, it is determined that the corresponding pedestrian in the current monitoring video image has abnormal behavior. In response to the identified abnormal behavior, the corresponding portrait area is marked as a suspicious person, and the doors and windows are closed and locked, and an alarm is triggered to notify the management personnel.

[0014] Secondly, the present invention provides an intelligent door and window adaptive opening and closing system based on AI behavior recognition, used to implement the above-mentioned intelligent door and window adaptive opening and closing method based on AI behavior recognition, comprising: a processor, a memory, a communication interface, a camera, and a door and window opening and closing control device. The processor stores computer program instructions for implementing the above-mentioned intelligent door and window adaptive opening and closing method based on AI behavior recognition, and the communication interface is communicatively connected to the camera and the door and window opening and closing control device.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention leverages the significant vertical variations in climbing and vaulting movements to analyze joint motion correlation in the vertical direction, calculate the trajectory correlation between joint pairs, and enhance feature discrimination by combining the differences in motion smoothness between joint pairs in consecutive frame image sequences. This improves the sensitivity of abnormal motion detection. Simultaneously, by clustering historical data to establish a baseline group for normal walking, and calculating the degree of abnormality through weighted differences, it effectively avoids errors caused by individual differences, making the quantification of abnormality more accurate. This allows for more accurate analysis of pedestrian actions and behaviors within the monitored area, enabling more precise identification and differentiation of abnormal behaviors such as climbing and vaulting, thereby reducing the false alarm rate of abnormal behavior detection and achieving adaptive intelligent opening and closing of doors and windows. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the implementation of an AI-based adaptive opening and closing method for smart doors and windows. Figure 2 This is a structural block diagram of an AI-based intelligent door and window adaptive opening and closing system. Detailed Implementation

[0017] The technical features of the present invention will be further described in detail below with reference to the accompanying drawings so that those skilled in the art can understand them.

[0018] An AI-based behavior recognition-based method for adaptive opening and closing of smart doors and windows, the implementation process of which is as follows: Figure 1 As shown, the specific implementation steps are as follows: Step S1: Real-time acquisition of surveillance video images and extraction of human image regions for marking and tracking, resulting in an image frame sequence for each human image region; Specifically, surveillance video images and corresponding timestamps are captured in real time by cameras. The YOLO algorithm (You-Only-Look-Once, a single-stage object detector) is used to extract the bounding boxes of human figures in each frame of the surveillance video image. To avoid errors in trajectory analysis caused by multiple pedestrians within the monitoring range, when multiple human figures appear in the surveillance video image, the SORT algorithm (Simple-Online-and-Real-time-Tracking, a multi-object tracker) is used to assign a tag ID to each human figure appearing within the monitoring range, and the movement trajectory of each human figure is tracked to obtain the image frame sequence of each human figure region, so as to facilitate subsequent action and behavior analysis.

[0019] Step S2: Extract the key points in the image frame sequence of the human portrait region and classify and number them, obtain the coordinates of all key points on each frame image, and quantify the motion smoothness of each key point. After obtaining the image frame sequence and corresponding timestamp for each portrait area, the continuous frame image sequence containing the same portrait area is analyzed. If it is just normal entry and exit or pedestrian passing by, then the action posture should be normal walking, which is quite different from the action posture of abnormal behaviors such as climbing or jumping over fences. Therefore, the action posture can be estimated by extracting the key points of the portrait and analyzing the motion trajectory of the key points.

[0020] Specifically, based on the image frame sequence of the human image region, the HR-Net (High-Resolution-Net) network is used to extract the joints of all human image regions in each frame. The joints of each human image region are classified and numbered in the same order, such as left shoulder numbered 1, right shoulder numbered 2, left elbow numbered 3, right elbow numbered 4, and so on, as well as left and right wrists, left and right hip joints, left and right knee joints, left and right ankles, etc. The joint numbering is consistent in each frame of the human image region image frame sequence, and all joints in the same human image region in any two adjacent frames correspond one-to-one. The coordinates of all joints in each frame are obtained, and the difference vector between the coordinates of a certain joint in the current frame and the coordinates of the same joint in the next frame is used as the movement speed of the corresponding joint in the current frame. Among them, the same portrait area is in the first Intra-frame key points The coordinates are as In the Intra-frame key points The coordinates are as Key points In the The speed of movement in a frame image is denoted as , Here Given a two-dimensional vector, the movement speed of the same joint point in two adjacent frames is calculated sequentially to obtain the movement speed sequence of the corresponding joint point. Similarly, the movement speed sequence of all joint points in each portrait region is obtained.

[0021] Furthermore, if it is normal walking, the motion trajectory of each joint should be relatively smooth during the movement. The smoothness of the motion of each joint can be quantified by using the motion speed sequence of a single joint in consecutive frame images. At the same time, since the motion speed of the joint is a two-dimensional vector, the variance can be calculated by using the magnitude of the vector difference to quantify the smoothness of the motion of a single joint. Specifically, based on the movement speed sequence of the joints, the mean movement speed of each joint is obtained. The magnitude of the vector difference between the movement speed of a certain joint in each frame and the corresponding mean movement speed is calculated. The squares of the magnitudes are accumulated. The ratio between the accumulated value and the number of images in the corresponding human portrait region image frame sequence is reverse mapped. The reverse mapped value is then normalized, and the normalized value is used as the motion smoothness of that joint. Similarly, the motion smoothness of all joints is obtained. Among them, key points within the same portrait area The degree of motion smoothness in consecutive frame images is The calculation formula is as follows: In the formula, Indicates key points within the same portrait area The average speed of movement in consecutive frames of images. Indicates key points In the Movement speed in frame image Represents the magnitude of a vector. This indicates the number of frames in a sequence of images representing the same human portrait region, i.e., the total number of consecutive frames representing the same human portrait region. The normalization function is expressed as: During normal walking, the movement trajectory of each joint is relatively smooth, and the variance of the movement speed of each joint is small. However, when climbing or vaulting, the variance of the movement speed of each joint is large. The smoothness of the joint's movement is inversely proportional to the variance of the movement speed. Therefore, in a continuous frame image, the larger the variance of the movement speed, the lower the smoothness of the movement of that joint, and vice versa.

[0022] Step S3: Based on the motion smoothness of the joints, calculate the correlation of motion trajectories between various joint pairs within each portrait region; After obtaining the motion smoothness of any single joint point within the same portrait region in consecutive frames, relying solely on the motion smoothness of a single joint point is insufficient to quantify the difference between the current motion trajectory of that joint point and the corresponding joint point's motion trajectory during normal walking. Further analysis of the correlation between the motion trajectories of various joint points within the same portrait region is needed to more accurately distinguish pedestrian behavior postures. Since the motion trajectories of various joint points in the human body exhibit significant correlation during normal walking, the corresponding pedestrian's action posture can be analyzed by calculating the correlation between the motion trajectory correlations of various joint point pairs within the same portrait region. This involves calculating the correlation between the motion trajectory correlations of various joint point pairs within the portrait region. During the process, if the overall correlation of the movement speed sequence of the joints is directly calculated, the correlation value is small, which makes the difference between normal walking and abnormal movements such as climbing and vaulting not significant enough. However, there is a more obvious difference between normal walking and abnormal movements such as climbing and vaulting. Specifically, during climbing and vaulting, the changes of the joints of the human body in the vertical direction are more significant, while the changes in the horizontal direction are relatively small. Therefore, when calculating the correlation of the movement trajectory of each pair of joints in the same human image area, calculating the vertical direction separately will make the abnormal behavior more obvious and improve the detection sensitivity for abnormal movements. Specifically, based on the movement speed sequence of the joints, the vertical component sequence of the movement speed sequence of all joints in the vertical direction is obtained. Based on the calculation formula of Pearson correlation coefficient, the Pearson correlation coefficient of the vertical component sequence between each pair of joints in the same portrait area is obtained. The natural exponential function is used to back-map the absolute value of the difference in motion smoothness between two joints in the same portrait area. The product of the back-mapped value and the corresponding Pearson correlation coefficient of the two joints is used as the motion trajectory correlation between the two joints. Similarly, the motion trajectory correlation between each pair of joints in each portrait area is calculated. Within the same portrait area, key points and key points The correlation between their motion trajectories is The calculation formula is as follows: In the formula, Indicates key points The vertical component sequence of the moving velocity sequence. Indicates key points The vertical component sequence of the moving velocity sequence. Indicates key points and key points Pearson correlation coefficient between longitudinal component sequences Indicates key points The degree of motion smoothness in consecutive frames of images Indicates key points The degree of motion smoothness in consecutive frames of images Represents the natural exponential function; In statistics, the Pearson correlation coefficient, also known as the Pearson product-moment correlation coefficient (PPMCC), is used to measure the linear correlation between two variables. Its value ranges between -1 and 1. The Pearson correlation coefficient between the longitudinal component sequences corresponding to two joints within the same human image region can quantify the similarity of the vertical movement speed sequences of these two joints. The larger the correlation value, the greater the correlation between the movement trajectories of the two joints, the higher the similarity, and the more likely their movement posture is normal walking. The smaller the correlation value, the lower the similarity between the movement trajectories of the two joints, and the greater the difference between their movement posture and normal walking. The value of reflects the difference in the smoothness of motion between two joints; the greater the difference, the smaller the correlation between the motion trajectories of the two joints.

[0023] Step S4: Based on the surveillance video images of historical normal behaviors, obtain the baseline value of normal behaviors; After calculating the motion trajectory correlation between various key point pairs within each portrait area, it is necessary to judge the person's posture to determine whether there are any abnormal behaviors. However, since there are many key point pairs, directly averaging the motion trajectory correlation between various key point pairs will dilute key information and reduce the distinction between normal walking and abnormal behaviors such as climbing and vaulting. Therefore, it is necessary to analyze the motion trajectory correlation between each key point pair during normal walking as a benchmark, and group historical normal key points with high correlation, and quantify the possibility of the portrait posture in the monitoring area being abnormal behaviors such as climbing and vaulting by key point group, that is, quantify its degree of abnormality. Specifically, historical data of surveillance video images is acquired, and a sequence of consecutive frames containing normal human behavior is extracted from the historical data. Similar to the processing method for real-time acquired surveillance video images, the surveillance video images are processed according to steps S1 to S3. The YOLO algorithm is used to extract the human image region, and the HR-Net network is used to extract the key points within the human image region of each historical frame as historical normal key points, which are then classified and numbered. The order of classification and numbering of historical normal key points is the same as the order of classification and numbering of key points in the real-time surveillance video images, and the corresponding human body parts are consistent. Based on the sequence of consecutive frames containing normal human behavior, the coordinates of each key point in each historical frame are obtained, and the historical motion smoothness of each historical normal key point and the correlation of historical motion trajectories between various pairs of historical normal key points are calculated.

[0024] Furthermore, based on the correlation of historical motion trajectories between various historical normal joint point pairs, the value obtained by subtracting the correlation of historical motion trajectories from 1 is used as the distance metric between the corresponding historical normal joint point pairs. Similarly, the distance matrix of all historical normal joint point pairs is obtained. Using the bottom-up method in hierarchical clustering and setting the number of final clusters, all historical normal joint points are clustered and merged into final clusters. The correlation of historical motion trajectories between all historical normal joint point pairs in each final cluster is extracted and accumulated to obtain the accumulated historical correlation value corresponding to each final cluster. The ratio of the accumulated historical correlation value to the number of historical normal joint point pairs in the corresponding final cluster is used as the historical correlation mean of the final cluster. The historical correlation mean of each final cluster is used as the baseline value of the corresponding joint point under normal behavior. The process of clustering all historical normal joints using the bottom-up method in hierarchical clustering is as follows: The number of final clusters is set to 3, corresponding to the upper limb, lower limb, and trunk joint groups respectively. The number of final clusters can be further subdivided based on the actual situation. Each historical normal joint is considered as a class. Based on the distance matrix of all historical normal joint pairs, all historical normal joints are clustered, and the two classes with the smallest distance are merged into a new class. The inter-class distance between each class is repeatedly calculated, and the two classes with the smallest inter-class distance are merged sequentially until all historical normal joints are merged into 3 final clusters. Each final cluster represents a group of historical normal joints whose movement trajectories are highly correlated during normal walking. Since the distance metric between historical normal joint pairs is inversely proportional to the correlation of historical motion trajectories between historical normal joint pairs, where historical normal joints... and The distance metric is The calculation formula is: In the formula, Historical normal juncture and The correlation of historical movement trajectories between them; the final formula for calculating the inter-class distance between clusters is as follows: In the formula, Indicates the first The final cluster and the first Inter-class distance between the final clusters; Indicates the first The mean of the correlation of historical motion trajectories among each pair of historical normal keypoints in the final cluster is used to reflect the correlation of the historical motion trajectories among each pair of historical normal keypoints in the final cluster. The correlation level among historical normal key points in each final cluster; Indicates the first The standard deviation of the smoothness of historical motion among historical normal keypoints in the final cluster is used to reflect the degree of smoothness of historical motion. The larger the standard deviation of the difference in movement speed among all historical normal keypoints in the final cluster, the better. The greater the difference in motion smoothness between historical normal joints in the final cluster; The value can reflect the first The overall similarity of the motion of each historical normal key point in the final cluster; Similarly, Indicates the first The mean correlation of historical motion trajectories among each pair of historical normal keypoints in the final cluster. Indicates the first The standard deviation of the smoothness of historical motion among historical normal key points in the final cluster. The value can reflect the first The overall similarity of the motion of each historical normal key point in the final cluster; No. The formula for calculating the historical correlation mean of the final clusters is as follows: In the formula, Indicates the first The historical correlation mean of the final cluster also indicates the number of clusters. Each final cluster contains the baseline values ​​of historical normal keypoints under normal behavior. Indicates the first Historical normal key points in the final cluster and Correlation between their historical movement trajectories Indicates the first The number of historical normal keypoint pairs contained in each final cluster.

[0025] Step S5: Based on the baseline value of normal behavior, quantify the degree of abnormality of each portrait area in the current monitoring video image; In quantifying the degree of anomaly in each portrait area of ​​the current surveillance video image, due to the subtle differences in the movement characteristics of each person, the degree of anomaly cannot be directly quantified by the difference between the movement characteristics of each historical normal joint point in the historical data and the movement characteristics of each joint point in the real-time data. However, since the numbering rules of the portrait joint points are consistent, and the joint point numbers of the portrait areas in the historical data and the real-time monitoring data correspond to each other, the joint points with high correlation can be grouped. The joint points of each portrait area in the real-time monitoring data can be grouped according to the clustering results of the historical data, and the degree of anomaly of each portrait movement posture in the real-time monitoring data can be quantified by the joint point group. Specifically, based on the classification number of the key points and the division range of historical normal key points by the final cluster, the key points of each portrait area in the real-time acquired monitoring video image are divided into real-time key point groups corresponding to each final cluster. The motion trajectory correlation between all key point pairs in each real-time key point group within each portrait area is extracted and accumulated to obtain the correlation accumulation value corresponding to each real-time key point group. The ratio of the correlation accumulation value to the number of key point pairs in the corresponding real-time key point group is used as the real-time correlation mean of the real-time key point group. Similarly, the real-time correlation mean of all real-time key point groups within each portrait area is calculated. Among them, the corresponding number within the same portrait area The first final cluster The real-time correlation mean of a group of real-time key points The calculation formula is as follows: In the formula, Indicates the first image within the same portrait area Joints in a real-time joint group and key points Correlation of motion trajectories between them Indicates the first image within the same portrait area The number of key pairs contained in a real-time key group.

[0026] Furthermore, the ratio between the mean of the correlation between all relevant nodes and motion trajectories within a real-time keypoint group and the standard deviation of the motion smoothness of all relevant nodes is used as the difference weight between that real-time keypoint group and the corresponding final cluster. Similarly, the difference weights corresponding to all real-time keypoint groups within each portrait region are calculated. The absolute value of the difference between the real-time correlation mean of each real-time keypoint group within the same portrait region and the historical correlation mean of the corresponding final cluster is calculated. The product of the absolute value corresponding to each real-time keypoint group and the corresponding difference weight is used as the correlation difference feature value of each real-time keypoint group. The correlation difference feature values ​​of all real-time keypoint groups within the same portrait region are summed to obtain the total correlation difference value of the portrait region. At the same time, the difference weights corresponding to all real-time keypoint groups within the portrait region are summed to obtain the total difference weight value of the portrait region. The ratio of the total correlation difference value to the total difference weight value is used as the anomaly degree of the portrait region. Similarly, the anomaly degree of each portrait region in the current monitoring video image is calculated. The formula for calculating the degree of abnormality in a single portrait region is as follows: In the formula, This indicates the degree of anomaly in a single human image region within the current surveillance video image. This indicates the number of final clusters, and also the number of real-time keypoint groups contained in a single portrait region. Indicates the first image within the same portrait area The mean real-time correlation of a group of real-time key points. Indicates the first The historical correlation mean of the final clusters, Indicates the first image within the same portrait area Difference weights for each real-time key group; The value of reflects the difference between the pedestrian's movement and posture in real-time monitoring data and the normal walking movement of pedestrians in historical data. The larger the value, the greater the possibility that the pedestrian's movement and posture in real-time monitoring data is abnormal. The value of the difference weight is related to the above-mentioned clustering results of historical data. The overall similarity of the historical normal key point movements in each final cluster is consistent, that is: The larger the value of the difference weight, the stronger the difference. The smaller the difference in motion characteristics between historical normal keypoints in the final cluster, the better. The higher the motion correlation between historical normal joints in the final cluster, the more likely the corresponding joints should be grouped together for analysis. Within the same portrait region, the... The greater the anomaly of a real-time key group, the greater its contribution.

[0027] Step S6: Based on the degree of abnormality, identify abnormal behavior and execute the opening and closing control of doors and windows; Specifically, the anomaly detection threshold should be set based on the actual application scenario or empirical values. Generally, the anomaly detection threshold can be set to a value of [value missing]. It can also be trained with a large amount of data to adjust the value of the anomaly judgment threshold. By flexibly setting the anomaly judgment threshold, normal behaviors such as running can be filtered out, reducing the probability of false judgment. Based on the degree of anomaly of each portrait area in the current monitoring video image, the degree of anomaly of each portrait area is compared with the anomaly judgment threshold. When the degree of anomaly of all portrait areas is less than the anomaly judgment threshold, it is determined that there is no abnormal behavior of all pedestrians in the current monitoring video image, and the current state of doors and windows is maintained. When the degree of anomaly of any portrait area is greater than or equal to the anomaly judgment threshold, it is determined that the corresponding pedestrian in the current monitoring video image has abnormal behavior. In response to the identified abnormal behavior, the corresponding portrait area is marked as a suspicious person, and at the same time, the doors and windows are closed and locked, and an alarm is triggered to notify the management personnel.

[0028] This invention also discloses an AI-based adaptive opening and closing system for smart doors and windows, used to implement the aforementioned AI-based adaptive opening and closing method for smart doors and windows. The system structure is as follows: Figure 2 As shown, it includes: a processor, a memory, a communication interface, a camera, and a door and window opening and closing control device. The processor stores computer program instructions for implementing the above-mentioned AI behavior recognition-based intelligent door and window adaptive opening and closing method. The communication interface is communicatively connected to the camera and the door and window opening and closing control device.

[0029] The embodiments included in this invention are merely descriptions of preferred embodiments of the invention and are not limited to the precise structures described above and shown in the accompanying drawings. Various modifications and changes can be made without departing from the scope of protection. Any variations and improvements made by those skilled in the art to the technical solutions of this invention without departing from the design concept of this invention should fall within the scope of protection of this invention.

Claims

1. A smart door and window adaptive opening and closing method based on AI behavior recognition, characterized in that: Real-time acquisition of surveillance video images and extraction of human portrait regions for marking and tracking, resulting in an image frame sequence for each human portrait region; extraction of key points within the image frame sequence of human portrait regions and classification and numbering, obtaining the coordinates of all key points on each frame image, and quantifying the motion smoothness of each key point; based on the motion smoothness of the key points, calculation of the motion trajectory correlation between various key point pairs within each human portrait region. Based on historical surveillance video images of normal behaviors, obtain baseline values ​​for normal behaviors. Based on the baseline values ​​of normal behavior, the degree of abnormality of each portrait area in the current surveillance video image is quantified; Based on the degree of abnormality, identify abnormal behavior and implement door and window opening and closing control; Specifically, based on the coordinates of the joints, the movement speed sequence and motion smoothness of all joints are calculated. Based on the longitudinal component sequence of the joint movement speed sequence, the Pearson correlation coefficients of the longitudinal component sequences between each pair of joints within the same portrait area are obtained. The natural exponential function is used to back-map the absolute value of the difference in motion smoothness between two joints within the same portrait area. The product of the back-mapped value and the corresponding Pearson correlation coefficients of the two joints is used as the motion trajectory correlation between the two joints. Similarly, the motion trajectory correlation between each pair of joints within each portrait area is calculated.

2. The AI-based adaptive opening and closing method for intelligent doors and windows according to claim 1, characterized in that, The real-time acquisition of surveillance video images and extraction of human image regions for marking and tracking, to obtain an image frame sequence for each human image region, includes: acquiring surveillance video images and corresponding timestamps in real time through a camera; using the YOLO algorithm to extract the bounding box of the human image region in each frame of the surveillance video image; when multiple human images appear in the surveillance video image, using the SORT algorithm to assign a marker ID to each human image appearing within the monitoring range; tracking the movement trajectory of each human image to obtain an image frame sequence for each human image region.

3. The AI-based adaptive opening and closing method for intelligent doors and windows according to claim 1, characterized in that, The process of extracting and classifying key points within the image frame sequence of the human portrait region, obtaining the coordinates of all key points in each frame, and quantifying the smoothness of motion of each key point includes: based on the image frame sequence of the human portrait region, using the HR-Net network to extract key points within all human portrait regions in each frame, classifying and numbering key points in each human portrait region in the same order; obtaining the coordinates of all key points in each frame, using the difference vector between the coordinates of a key point in the current frame and the coordinates of that key point in the next frame as the moving speed of the corresponding key point in the current frame, calculating the moving speed of the same key point in adjacent frames sequentially to obtain the moving speed sequence of the corresponding key point, and similarly obtaining the moving speed sequence of all key points within each human portrait region.

4. The AI-based adaptive opening and closing method for intelligent doors and windows according to claim 3, characterized in that, The quantification of the motion smoothness of each joint point further includes: obtaining the average motion speed of each joint point based on the motion speed sequence of the joint points; calculating the magnitude of the vector difference between the motion speed of a certain joint point in each frame image and the corresponding average motion speed; accumulating the squares of the magnitudes; performing a reverse mapping between the ratio of the accumulated value and the number of images in the corresponding portrait region image frame sequence; normalizing the reverse mapped value; and using the normalized value as the motion smoothness of the joint point; similarly, the motion smoothness of all joint points is obtained.

5. The AI-based adaptive opening and closing method for intelligent doors and windows according to any one of claims 1 to 4, characterized in that, The process of obtaining baseline values ​​for normal behavior based on historical normal behavior in surveillance video images includes: acquiring historical data of surveillance video images; extracting a sequence of consecutive frames containing normal behavior of a single person from the historical data; processing the data in the same way as real-time acquired surveillance video images; extracting the human image region using the YOLO algorithm; extracting the key points within the human image region of each historical frame using the HR-Net network as historical normal key points and classifying and numbering them; and based on the sequence of consecutive frames containing normal behavior of a single person, obtaining the coordinates of each key point in each historical frame; calculating the smoothness of historical motion of each historical normal key point and the correlation of historical motion trajectories between various pairs of historical normal key points.

6. The AI-based adaptive opening and closing method for intelligent doors and windows according to claim 5, characterized in that, The method of obtaining baseline values ​​for normal behavior based on surveillance video images of historical normal behavior further includes: using the correlation of historical motion trajectories between various historical normal joint point pairs as the distance metric between corresponding historical normal joint point pairs, and similarly obtaining the distance matrix of all historical normal joint point pairs; using the bottom-up method in hierarchical clustering and setting the number of final clusters, merging all historical normal joint points into final clusters; extracting and accumulating the correlation of historical motion trajectories between all historical normal joint point pairs in each final cluster to obtain the historical correlation accumulation value corresponding to each final cluster; using the ratio of the historical correlation accumulation value to the number of historical normal joint point pairs in the corresponding final cluster as the historical correlation mean of the final cluster; and using the historical correlation mean of each final cluster as the baseline value of the corresponding joint point under normal behavior.

7. The AI-based adaptive opening and closing method for intelligent doors and windows according to claim 6, characterized in that, The method of quantifying the degree of abnormality of each portrait region in the current monitoring video image based on the baseline value of normal behavior includes: dividing the joints of each portrait region in the real-time acquired monitoring video image into real-time joint groups corresponding to each final cluster according to the classification number of the joints and the division range of historical normal joints by the final cluster; extracting and accumulating the motion trajectory correlation between all relevant node pairs in the real-time joint group in each portrait region to obtain the correlation accumulation value corresponding to each real-time joint group; and taking the ratio of the correlation accumulation value to the number of joint pairs in the corresponding real-time joint group as the real-time correlation mean of the real-time joint group; similarly, calculating the real-time correlation mean of all real-time joint groups in each portrait region.

8. The AI-based adaptive opening and closing method for intelligent doors and windows according to claim 7, characterized in that, The method of quantifying the degree of anomaly in each portrait region in the current monitoring video image based on the baseline value of normal behavior and actions further includes: using the ratio between the mean of the correlation between all relevant nodes and motion trajectories within a certain real-time key point group and the standard deviation of the motion smoothness of all relevant nodes as the difference weight between the real-time key point group and the corresponding final cluster; similarly, calculating the difference weight corresponding to all real-time key point groups within each portrait region; calculating the absolute value of the difference between the real-time correlation mean of each real-time key point group within the same portrait region and the historical correlation mean of the corresponding final cluster; multiplying the absolute value corresponding to each real-time key point group by the corresponding difference weight as the correlation difference feature value of each real-time key point group; summing the correlation difference feature values ​​of all real-time key point groups within the same portrait region to obtain the total correlation difference value of the portrait region; simultaneously, summing the difference weights corresponding to all real-time key point groups within the portrait region to obtain the total difference weight value corresponding to the portrait region; and using the ratio of the total correlation difference value to the total difference weight value as the degree of anomaly of the portrait region; similarly, calculating the degree of anomaly of each portrait region in the current monitoring video image.

9. The AI-based adaptive opening and closing method for intelligent doors and windows according to claim 1, characterized in that, The method of identifying abnormal behavior and controlling the opening and closing of doors and windows based on the degree of abnormality includes: setting an abnormality judgment threshold according to the actual application scenario or experience value; comparing the degree of abnormality of each portrait area in the current monitoring video image with the abnormality judgment threshold; when the degree of abnormality of all portrait areas is less than the abnormality judgment threshold, it is determined that there is no abnormal behavior among all pedestrians in the current monitoring video image, and the current state of doors and windows is maintained; when the degree of abnormality of any portrait area is greater than or equal to the abnormality judgment threshold, it is determined that the corresponding pedestrian in the current monitoring video image has abnormal behavior; in response to the identified abnormal behavior, the corresponding portrait area is marked as a suspicious person, and the doors and windows are controlled to close and lock, and an alarm is triggered to notify the management personnel.

10. An intelligent door and window adaptive opening and closing system based on AI behavior recognition, characterized in that: It includes a processor, a memory, a communication interface, a camera, and a door and window opening and closing control device. The processor stores computer program instructions for implementing the AI ​​behavior recognition-based intelligent door and window adaptive opening and closing method according to any one of claims 1 to 9. The communication interface is communicatively connected to the camera and the door and window opening and closing control device.

Images