Elder behavior monitoring method and device based on rule mining, equipment and medium

Abstract

The invention relates to the technical field of intelligent nursing, solves the problem that abnormal behaviors of old people cannot be accurately predicted and identified in the prior art, and provides an old people behavior monitoring method and device based on rule mining, equipment and a medium. The method comprises the following steps: according to historical video stream data, acquiring behavior characteristic data of a target old person in different time periods; mining the action feature data in different time periods according to a time sequence matching algorithm, and obtaining a behavior pattern of the target old person and corresponding behavior rule data; establishing a normal behavior pattern library according to the behavior pattern and the corresponding behavior rule data; according to real-time video stream data and the normal behavior pattern library, judging whether the target old person has an abnormal behavior or not; and when the abnormal behavior exists, giving an alarm according to the abnormal behavior. The method can improve the comprehensive behavior analysis capability, thereby improving the accuracy and stability of abnormal behavior recognition.

Description

Technical Field

[0001] This invention relates to the field of intelligent care technology, and in particular to a method, device, equipment and medium for monitoring elderly behavior based on pattern mining. Background Technology

[0002] Human behavior detection refers to the use of technology to monitor and analyze the daily behavior of the elderly in order to achieve real-time monitoring and attention to their living conditions. As the aging population continues to grow, the health status of the elderly has become a focus of social concern. The development of elderly behavior detection technology can help the elderly better manage their daily lives, improve their quality of life, and also promptly detect abnormal situations in their lives, reducing accidents and incidents and increasing their sense of security.

[0003] Some existing technologies monitor the activities of the elderly in real time by deploying sensor nodes, such as pressure sensors and infrared sensors, in their living environment. This method is simple and easy to implement, but requires a large number of sensors and has limitations in sensor accuracy and reliability. Other methods utilize cameras to capture video data of the elderly and analyze their behavior through image processing and pattern recognition techniques. This method enables non-contact monitoring, eliminates the need for the elderly to wear devices, and provides more detailed behavioral information. For example, patent CN116682179A discloses a method and system for detecting elderly behavior. This system constructs an abnormal behavior recognition model to determine whether the elderly person's behavior matches the physical characteristics of a fall. It uses video surveillance equipment to acquire the elderly person's movement data in real time, processes the data, and extracts feature information from each frame of the video stream. The acquired movement data is then input into the behavior recognition model to obtain corresponding behavior results and probability values. Finally, based on the behavior results and probability values, it determines whether the elderly person's behavior is abnormal and issues a timely alert after a fall. However, the behavior of the elderly is diverse and complex. Existing technological solutions lack comprehensive behavioral analysis capabilities and systems cannot identify complex behavioral patterns and rules, such as the daily routines and activity habits of the elderly, and cannot accurately predict and identify abnormal behaviors. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, device, equipment and medium for monitoring elderly behavior based on pattern mining, in order to solve the problem that the prior art cannot accurately predict and identify abnormal behavior of the elderly.

[0005] In a first aspect, embodiments of the present invention provide a method for monitoring elderly behavior based on pattern mining, the method comprising: Based on historical video stream data, behavioral characteristic data of the target elderly person are obtained at different time periods, wherein the historical video stream data includes the target elderly person; The action feature data of the target elderly person are mined by time series matching algorithm to obtain the behavior pattern and corresponding behavior regularity data of the target elderly person. Based on the behavioral patterns and corresponding behavioral pattern data, a normal behavior pattern library is established, wherein the normal behavior pattern library includes standard behavior models corresponding to each of the behavioral patterns. Based on real-time video stream data and the normal behavior pattern database, it is determined whether the target elderly person exhibits abnormal behavior; When abnormal behavior is detected, an alarm will be triggered based on the abnormal behavior.

[0006] Preferably, the step of obtaining behavioral data of the target elderly person at different time periods based on historical video stream data includes... The historical video stream data is preprocessed to obtain the target video stream data, wherein the preprocessing includes one or more of noise reduction processing, contrast adjustment and brightness adjustment; The target elderly person in the video stream data is located and tracked to obtain the key skeletal points of the target elderly person; Based on the skeletal key points, obtain the body posture data of the target elderly person; Based on the body posture data, behavioral characteristic data for different time periods are obtained, wherein the behavioral characteristic data includes the range of motion, speed, and rate of change of speed of each joint of the target elderly person.

[0007] Preferably, the step of mining action feature data from different time periods using a time series matching algorithm to obtain the target elderly person's behavior patterns and corresponding behavioral regularity data includes: The video stream data is segmented based on the behavioral feature data to obtain several video segments; Based on the behavioral feature data, the behavioral patterns of each video segment are labeled, wherein the behavioral patterns include standing, walking, sitting, eating, watching TV, and sleeping. Based on the labeled video segments and the corresponding behavioral feature data, obtain the behavioral feature time series corresponding to each behavioral pattern; Based on the time series matching algorithm and the time series of each behavioral feature, the behavioral patterns and corresponding behavioral regularity data of the target elderly are obtained.

[0008] Preferably, the time series matching algorithm is a dynamic time warping algorithm, and the step of obtaining the behavioral patterns and corresponding behavioral regularity data of the target elderly person based on the time series matching algorithm and the time series of each behavioral feature includes: The behavioral feature time series is input into a pre-built time series model to obtain the probability distribution data of each behavioral pattern, wherein the time series model is built based on an LSTM model; The similarity between the time series of each behavioral feature is obtained based on the dynamic time warping algorithm. Based on the similarity and probability distribution data, the behavioral patterns and corresponding behavioral regularities of the target elderly person are obtained.

[0009] Preferably, a normal behavior pattern library is established based on the said behavior pattern and the corresponding behavior pattern data, including: Modeling is performed based on the probability distribution data of each behavior pattern to obtain an initial model for each behavior pattern, wherein the initial model is constructed based on the SVM classification model; Based on the time series of behavioral characteristics of each behavioral pattern, train the corresponding initial model to obtain the standard behavioral model of each behavioral pattern; Statistical analysis of the similarity between the time series of the aforementioned behavioral features reveals the periodicity of each behavioral pattern. Based on the standard behavioral models of each behavioral pattern and the periodic patterns described therein, the normal behavioral pattern library is obtained.

[0010] Preferably, determining whether the target elderly person exhibits abnormal behavior based on real-time video stream data and the normal behavior pattern database includes: Based on the real-time video stream data, obtain the real-time behavioral characteristics time series of the target elderly person; Based on the normal behavior pattern library, obtain the standard behavior model and deviation threshold corresponding to the behavior pattern of the real-time behavior feature time series; Calculate the degree of deviation between the real-time behavioral feature data and the corresponding standard behavioral model; When the degree of deviation is greater than the degree of deviation threshold, the target elderly person exhibits abnormal behavior; The threshold for the degree of deviation varies among different behavioral models.

[0011] Preferably, the step of triggering an alarm based on the abnormal behavior when abnormal behavior occurs includes: Based on the real-time behavior feature time series corresponding to the abnormal behavior, the real-time video stream data is segmented to obtain the video segment where the abnormality occurred; Determine whether a preset privacy scenario exists in the video segment where the anomaly occurred; If present, the preset privacy scene in the video segment where the anomaly occurred is occluded to obtain the target anomaly video; An alarm will be triggered based on the abnormal video of the target.

[0012] Secondly, embodiments of the present invention provide an elderly behavior monitoring device based on pattern mining, the device comprising: The feature data acquisition module is used to acquire behavioral feature data of the target elderly person at different time periods based on historical video stream data, wherein the historical video stream data includes the target elderly person; The pattern mining module is used to mine action feature data from different time periods using a time series matching algorithm to obtain the target elderly person's behavior patterns and corresponding behavior regularity data. The pattern library establishment module is used to establish a normal behavior pattern library based on the behavior patterns and corresponding behavior pattern data, wherein the normal behavior pattern library includes standard behavior models corresponding to each of the behavior patterns; The judgment module is used to determine whether the target elderly person has abnormal behavior based on real-time video stream data and the normal behavior pattern library; The alarm module is used to trigger an alarm based on the abnormal behavior when such behavior is detected.

[0013] Thirdly, embodiments of the present invention provide an electronic device, including: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect described above.

[0014] Fourthly, embodiments of the present invention provide a storage medium storing computer program instructions, which, when executed by a processor, implement the method of the first aspect described above.

[0015] In summary, the beneficial effects of the present invention are as follows: The elderly behavior monitoring method, device, equipment, and medium based on pattern mining provided in this invention can acquire behavioral characteristic data of target elderly people at different time periods through historical video data, providing a data foundation for subsequent behavioral pattern mining and pattern analysis. By mining historical behavioral data through time series matching algorithms, behavioral patterns and regularities of the elderly can be discovered, such as daily habits and activity patterns, thereby establishing an understanding of normal behavior and providing a benchmark for identifying abnormal behavior. The mined behavioral patterns and regularities are established as a normal behavior pattern library, forming a model of normal elderly behavior. Such a model library can serve as a basis for judging whether elderly behavior is abnormal, helping to improve the accuracy and stability of abnormal behavior identification. By comparing real-time video stream data with the established normal behavior pattern library, it is possible to determine in real time whether the target elderly person exhibits abnormal behavior. This allows for timely detection of abnormal states in the elderly, providing timely warnings and feedback for providing assistance and care. When the system detects abnormal behavior, it can issue an alarm in a timely manner, reminding relevant personnel to handle and intervene. This helps prevent accidents or emergencies involving the elderly and provides timely help and support. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0017] Figure 1 This is a flowchart illustrating the elderly behavior monitoring method based on pattern mining, according to an embodiment of the present invention.

[0018] Figure 2 This is another flowchart illustrating the method for monitoring elderly behavior based on pattern mining, as described in this invention.

[0019] Figure 3 This is another flowchart illustrating the method for monitoring elderly behavior based on pattern mining, as described in this invention.

[0020] Figure 4 This is a schematic diagram of the elderly behavior monitoring device based on pattern mining, according to an embodiment of the present invention.

[0021] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0022] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the invention.

[0023] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0024] Example 1 Please see Figure 1 This invention provides a method for monitoring elderly behavior based on pattern mining, the method comprising: S1. Based on historical video stream data, obtain behavioral characteristic data of the target elderly person at different time periods, wherein the historical video stream data includes the target elderly person; Specifically, the system first collects and stores historical video stream data. This video data covers the daily activities of the elderly at different time periods, encompassing various behaviors such as walking, sitting, eating, and resting. For these video clips of the target elderly individuals, the system extracts and analyzes behavioral feature data. This behavioral feature data may include information such as the elderly person's body posture, movement trajectory, and activity area. This feature data can be extracted using computer vision techniques, such as skeletal keypoint detection and target tracking. Finally, these behavioral characteristic data are organized and divided according to different time periods to form a set of behavioral characteristic data of the target elderly in different time periods. These data will become the basis for subsequent behavioral pattern mining and modeling, and will help the system understand the daily behavioral habits and activity patterns of the target elderly.

[0025] Preferably, such as Figure 2As shown, the step of obtaining behavioral data of the target elderly person at different time periods based on historical video stream data includes... S11. Preprocess the historical video stream data to obtain target video stream data, wherein the preprocessing includes one or more of noise reduction processing, contrast adjustment and brightness adjustment; Specifically, in this step, the historical video stream data first needs to be preprocessed for subsequent analysis and processing. The main purpose of preprocessing is to improve video quality, reduce noise, and improve the analyzability of the image. Specifically, preprocessing may include one or more of the following operations: Noise reduction: By removing noise or interference from the video, visual clutter in the image is reduced, thereby improving the accuracy of subsequent processing steps.

[0026] Contrast Adjustment: Adjust the contrast of the video image to enhance details and features in the image, making the target more clearly visible.

[0027] Brightness Adjustment: The brightness of the video image is adjusted to make the brightness level more uniform, thereby improving the stability and reliability of subsequent processing steps. Through these preprocessing operations, historical video stream data will be optimized, providing a more reliable data foundation for subsequent target detection and behavior analysis.

[0028] S12. Locate and track the target elderly person in the video stream data to obtain the skeletal key points of the target elderly person; Specifically, this step first uses object detection algorithms to identify the location and bounding box of the elderly target in the video frames. Commonly used object detection algorithms include deep learning-based methods such as YOLO (You Only Look Once) and Faster R-CNN (Faster Region Convolutional Neural Network). These algorithms can accurately locate target objects in the video. Next, object tracking algorithms are used to track the elderly target's position in consecutive video frames. Object tracking algorithms can predict the target's position in the current frame based on its position in the previous frame and make corresponding corrections. Commonly used object tracking algorithms include Kalman filters, optical flow methods, and deep learning methods. Once the elderly target's position is determined, the system performs skeletal keypoint detection on the target region. This step is typically implemented using deep learning models, such as YOLOv8-pose, OpenPose, and PoseNet. These models can identify key points of the human body in the image, such as the head, shoulders, elbows, wrists, hips, knees, and ankles. Finally, the system tracks the detected skeletal keypoints to ensure accurate capture of the elderly target's body posture and movements in consecutive video frames. Tracking can be achieved by matching the correspondence between key points. Common methods include Kalman filters and optical flow.

[0029] S13. Obtain the body posture data of the target elderly person based on the skeletal key points; Specifically, after obtaining the key skeletal points of the target elderly person, the next step is to acquire their body posture data based on these key points. This posture data can include information such as body posture, joint angles, and postural stability. First, the location information of key skeletal points is used to build the human skeletal structure. This step typically involves connecting the key points according to the human anatomy to form the skeletal connections, such as connecting the shoulder to the wrist, or the hip to the ankle.

[0030] Next, the established skeletal structure is used to calculate the target elderly person's body posture. This involves calculating the angles, rotation directions, and postures of various joints based on the location information of key points. Posture calculation typically uses mathematical models and geometric principles to infer the body's posture based on the relative positions between key points.

[0031] Then, stability analysis is performed on the calculated posture data. This includes detecting the stability and balance of the posture to determine whether the target elderly person is in a stable posture. This typically involves monitoring and analyzing joint angles and the position of the body's center of gravity.

[0032] Finally, the acquired posture data is matched with predefined movement patterns to identify the current movement or behavior of the target elderly person. Movement recognition can be achieved by comparing the similarity between posture data and known movement patterns, or by using machine learning algorithms for pattern recognition and classification to obtain body posture data. S14. Based on the body posture data, obtain behavioral characteristic data for different time periods, wherein the behavioral characteristic data includes the range of motion, speed, and rate of change of speed of each joint of the target elderly person.

[0033] After obtaining the target elderly person's body posture data, the next step is to calculate behavioral characteristic data for different time periods based on this posture data. This behavioral characteristic data can provide various indicators about the elderly person's behavior, such as range of motion, speed, and rate of change of speed. First, the posture data of each joint of the target elderly person is analyzed over different time periods to determine their range of motion by measuring the range of motion of the joints in three-dimensional space, such as the maximum angle change of the joint or the range of motion of the joint within a specific time period. The movement speed of each joint of the target elderly person is calculated based on continuous posture data. This includes calculating the rate of change of angle of the joint per unit time, usually in degrees per second or radians per second. By comparing the movement speed over different time periods, the level of the elderly person's motor activity and behavioral changes can be understood. The rate of change of joint movement speed is analyzed to determine whether the elderly person's motor behavior has suddenly changed or accelerated, achieved by calculating the first or second derivative of the movement speed to obtain information on the rate of change of movement speed. Through the above steps, behavioral characteristic data for different time periods can be calculated based on the target elderly person's body posture data, including indicators such as joint range of motion, movement speed, and rate of change of speed. This data can be used to analyze the elderly person's behavioral patterns and activity levels, thereby achieving the purpose of behavioral monitoring and anomaly detection.

[0034] S2. Mining action feature data from different time periods using a time series matching algorithm to obtain the target elderly person's behavior patterns and corresponding behavior regularity data; Specifically, firstly, the system selects an appropriate time series matching algorithm, including Dynamic Time Warping (DTW), template-based methods, and statistical methods. These algorithms can be used to compare the similarity between behavioral feature sequences within different time periods, thereby identifying patterns and regularities. The selected time series matching algorithm is used to mine historical behavioral feature data. It takes the behavioral feature data of the target elderly person within different time periods as input and analyzes the similarities and differences between these data. By analyzing the similarities and differences, the system can discover the behavioral patterns and regularities of the target elderly person within different time periods. These behavioral patterns include, but are not limited to, walking, sitting, eating, and resting. Regularities may manifest as common characteristics in the frequency, duration, and sequence of specific behaviors, such as the elderly person's daily activity patterns and preferred behavioral sequences. Behavioral pattern data can include information such as the frequency of occurrence of each behavioral pattern, the distribution of duration, the temporal relationship between behaviors, and the statistical characteristics of behavioral patterns within different time periods.

[0035] Preferably, such as Figure 3 As shown, the step of mining action feature data from different time periods using a time series matching algorithm to obtain the target elderly person's behavior patterns and corresponding behavioral regularity data includes: S21. The video stream data is segmented according to the behavioral feature data to obtain several video segments; The input video stream data is segmented based on pre-extracted behavioral feature data to obtain a series of video segments, each corresponding to the elderly person's activities in different behavioral states. The purpose of video segmentation is to divide the entire video stream into segments with clear behavioral characteristics for subsequent behavioral pattern recognition and analysis.

[0036] In one embodiment, step S21 specifically includes: S211. Based on the skeletal key points of the target elderly person, calculate the spatial position of each joint in each frame, and obtain the body posture data of the target elderly person based on the changes in the spatial position of each joint in consecutive frames. In this step, skeletal keypoints can be understood as the localization results of discrete feature points such as the head, shoulders, elbows, hips, knees, and ankles in the image. Each frame of the video provides the location of these keypoints. Based on the pixel coordinates or mapped spatial coordinates of these skeletal keypoints, the specific position of each joint in the current frame can be calculated, thereby reconstructing the elderly person's body structure and posture in the current frame. Furthermore, by comparing the positions of the same joint in adjacent frames, the continuous displacement and posture changes of that joint on the time axis can be obtained. Integrating this series of information that changes over time yields the target elderly person's body posture data. In this way, the original video images can be transformed into posture sequences suitable for subsequent analysis, laying a data foundation for subsequent behavioral feature extraction and pattern mining.

[0037] S212. Based on the body posture data, calculate the range of motion, speed and rate of change of speed of each joint within a preset time window to obtain the corresponding behavioral feature data. In this step, the body posture data already provides the positional changes of each joint in each frame, allowing for statistical analysis of these positional sequences within a preset time window. For example, by statistically analyzing the maximum and minimum values ​​of each joint position within a fixed-length time window, the range of motion of the joint can be calculated. The movement velocity of the joint can be calculated using the positional differences and time intervals between adjacent frames. Further calculation of the change in velocity yields the rate of velocity change. Organizing these ranges of motion, velocities, and rates of velocity change according to joints and time windows constitutes the behavioral feature data for that time period. Through this processing step, the original posture trajectory is compressed into behavioral features with clear physical meaning, which is beneficial for distinguishing different types of action patterns.

[0038] S213. Based on the behavioral feature data, calculate the motion intensity index of each frame in the video stream data, and perform time series analysis on the motion intensity index to obtain the motion intensity change curve; In this step, the behavioral characteristics scattered across various joints need to be integrated into an indicator representative of the overall range of motion of the human body, namely, a movement intensity indicator. Within each frame or short time slice, the range of motion, velocity, and rate of change of velocity of multiple joints can be comprehensively calculated to obtain a numerical value reflecting the overall level of motor activity. Arranging these values ​​in chronological order constitutes a sequence of movement intensity changes over time. Time series analysis of this sequence reveals the overall trend and local fluctuations of movement intensity over time, i.e., the movement intensity change curve. The movement intensity change curve visually reflects the changes in an elderly person from rest to activity, from activity to rest, or from small movements to large movements, facilitating the segmentation of subsequent behavioral segments.

[0039] S214. Detect the inflection point of motion intensity change according to the motion intensity change curve, and divide the video stream data according to the time interval between adjacent inflection points and the motion intensity index meets the preset stability condition to obtain multiple candidate behavior segments. This stage utilizes the shape characteristics of the motion intensity change curve to determine the start and end boundary points of the action. Specifically, by detecting the first-order change trend of the motion intensity change curve, it identifies inflection points where the curve changes from rising to falling, from falling to rising, or from rapid change to flattening. These inflection points are considered potential boundary points of behavioral change. Then, between adjacent inflection points, it checks whether the motion intensity within that time interval meets preset stability conditions, such as whether the fluctuation amplitude remains within a certain range, or whether it remains at a sustained high or low intensity. When these stability conditions are met, the video segment corresponding to that time interval is classified as a candidate behavioral segment.

[0040] S215. Based on the behavioral feature data within each candidate behavioral segment, cluster the consecutive frames to obtain multiple micro-behavioral segments with similar behavioral features. In this step, each candidate behavior segment is further subdivided using behavioral feature data. By clustering the behavioral feature vectors of consecutive frames or small time slices within the segment, frames with similar range of motion, speed, and rate of change of speed can be grouped into the same category, thus forming multiple micro-behavioral segments with similar behavioral features. In this way, the multiple detailed actions that might originally be contained within a candidate behavior segment, such as from standing to walking slowly, or from walking to pausing, will be divided into different micro-behavioral regions. The purpose of this step is to further structure the coarse-grained candidate segments, making the action composition within each segment clearer.

[0041] S216. Based on the continuity and clustering categories of micro-behavioral segments on the time axis, adjacent micro-behavioral segments with the same category are merged to obtain a structured video segment sequence. After clustering, multiple scattered micro-behavioral segments within the same candidate segment may belong to the same cluster category. To avoid excessive fragmentation, at this stage, adjacent micro-behavioral segments belonging to the same cluster category are merged according to their chronological order on the timeline. This integrates consecutively occurring micro-behaviors with similar behavioral characteristics into longer, structured segments. The resulting structured video segment sequence not only maintains the differences between different types of actions but also avoids noise caused by excessive subdivision, improving segment-level stability and interpretability.

[0042] S217. Based on the structured video segment sequence and its corresponding behavioral feature data, perform behavioral pattern annotation on each video segment and construct a behavioral feature time series corresponding to each behavioral pattern.

[0043] Using the aforementioned structured video segment sequence as the analysis object, and combining the behavioral feature data corresponding to each segment, each video segment is assigned a specific behavioral pattern label, such as walking, sitting down, standing up, bending over, etc. Labeling can be done based on existing behavioral pattern recognition models or corrected using manual labeling results. Subsequently, for each behavioral pattern, the behavioral features corresponding to its occurrence at different times are arranged in chronological order to construct a time series of behavioral features corresponding to each behavioral pattern. This time series preserves both the temporal order of the behavior and the joint motion features related to the behavior.

[0044] Through the processing flow described in S211 to S217, this embodiment, based on the original skeletal keypoint and behavioral features, introduces a multi-level segmentation mechanism, from motion intensity change curves to candidate behavior segment division, and then to micro-behavior clustering and structured segment merging, to achieve automated and refined structured segmentation of video stream data. On the one hand, compared to simply segmenting according to a fixed time window, this embodiment can adaptively determine the behavior boundary based on the actual changes in the elderly's motion intensity, thereby more accurately corresponding to the real action process and reducing feature ambiguity caused by mixed actions within segments. On the other hand, by performing clustering and merging processing within candidate segments, this embodiment suppresses the interference of noise and occasional micro-movements while ensuring behavioral differences, making the final structured video segment sequence more stable and clearer. The behavioral feature time series constructed based on this structured video segment sequence is more conducive to subsequent behavioral pattern recognition and individualized pattern mining, improving the accuracy and robustness of behavioral pattern modeling, providing a more reliable feature basis for abnormal behavior detection, thereby helping to reduce false alarm rate and false negative rate, and improving the overall performance of the elderly behavior monitoring system.

[0045] S22. Based on the behavioral feature data, label the behavioral pattern of each video segment, wherein the behavioral pattern includes standing, walking, sitting, eating, watching TV, and sleeping. Each video segment is labeled based on pre-extracted behavioral feature data to determine the specific behavioral pattern corresponding to each segment. These behavioral patterns typically include various daily activities that the elderly may exhibit, including but not limited to standing, walking, sitting, eating, watching TV, and sleeping.

[0046] S23. Based on the labeled video segments and the corresponding behavioral feature data, obtain the behavioral feature time series corresponding to each behavioral pattern; Based on the labeled video clips and corresponding behavioral feature data, they were organized into time series data. These time series data describe the behavioral characteristics of the elderly under different behavioral patterns, such as the range of motion and speed of each joint, to facilitate subsequent behavioral pattern mining and analysis.

[0047] S24. Based on the time series matching algorithm and the time series of each behavioral feature, obtain the behavioral pattern and corresponding behavioral regularity data of the target elderly person.

[0048] Specifically, time series matching algorithms are used to analyze and mine the extracted behavioral feature time series to identify the behavioral patterns of the target elderly and infer behavioral regularities. By comparing the similarity and trends between different behavioral feature time series, the system can identify common behavioral patterns of the elderly and analyze their behavioral regularities, such as the frequency and duration of activities.

[0049] Preferably, the time series matching algorithm is a dynamic time warping algorithm, and the step of obtaining the behavioral patterns and corresponding behavioral regularity data of the target elderly person based on the time series matching algorithm and the time series of each behavioral feature includes: S241. Input the behavioral feature time series into a pre-constructed time series model to obtain the probability distribution data of each behavioral pattern, wherein the time series model is constructed based on an LSTM model; Specifically, the extracted behavioral feature time series is input into a pre-built time series model for processing. The time series model used here is based on a Long Short-Term Memory (LSTM) network, a deep learning model suitable for processing and predicting time series data. By inputting the behavioral feature time series into the LSTM model, the system can learn the features of different behavioral patterns and generate corresponding probability distribution data for subsequent behavioral pattern recognition and pattern analysis.

[0050] S242. Obtain the similarity between the time series of each behavioral feature according to the dynamic time warping algorithm; The Dynamic Time Warping (DTW) algorithm will be used to calculate the similarity between time series of different behavioral features. DTW is an algorithm used to measure the similarity between two time series. It takes into account the non-linear alignment relationship between the time series, and is therefore suitable for handling situations where the time series data have different lengths and speeds. By calculating similarity, the system can assess the degree of similarity between time series of different behavioral features, thereby identifying similar behavioral patterns.

[0051] S234. Based on the similarity and probability distribution data, obtain the behavioral pattern and corresponding behavioral regularity data of the target elderly person.

[0052] By combining similarity and probability distribution data, the system comprehensively assesses the behavioral patterns and regularities of the target elderly person. Through analysis of similarity and probability distribution data, the system can identify the behavioral patterns of the target elderly person in different time periods and infer corresponding behavioral patterns, such as the frequency and duration of activities.

[0053] S3. Based on the behavioral patterns and corresponding behavioral pattern data, establish a normal behavior pattern library, wherein the normal behavior pattern library includes standard behavior models corresponding to each of the behavioral patterns. Based on previously mined behavioral patterns and data, the various behavioral activities that the target elderly may exhibit are categorized and summarized, such as walking, sitting, eating, watching TV, and sleeping. Then, for each behavioral pattern, the system establishes a corresponding standard behavioral model. The standard behavioral model describes and defines the typical manifestations of the behavioral pattern, reflecting its general characteristics and patterns in a specific context. For example, for the walking behavior pattern, the standard behavioral model might include features such as gait, walking speed, and stride length; for the eating behavior pattern, it might include features such as actions during meals, duration, and frequency. The purpose of establishing a normal behavior pattern database is to provide a reference standard. The system can compare the real-time monitored elderly behavior with these standard behavioral models to determine whether the elderly's behavior is normal. The establishment of the normal behavior pattern database enables the system to more accurately identify and distinguish between normal and abnormal behaviors in the elderly, improving the accuracy and reliability of anomaly detection.

[0054] Preferably, a normal behavior pattern library is established based on the said behavior pattern and the corresponding behavior pattern data, including: S31. Model the behavior patterns based on the probability distribution data of each behavior pattern to obtain the initial model of each behavior pattern, wherein the initial model is constructed based on the SVM classification model. Specifically, the system will utilize the probability distribution data of behavioral patterns to build an initial model. First, for each behavioral pattern, the system will construct an initial model using a Support Vector Machine (SVM) classification model. SVM is a supervised learning algorithm that can be used for classification and regression analysis. By using the probability distribution data of behavioral patterns as input, the system trains the SVM classification model to distinguish between normal and abnormal behaviors. This model construction helps the system accurately identify the normal behaviors of the elderly.

[0055] S32. Train the corresponding initial model based on the time series of behavioral features of each behavioral pattern to obtain the standard behavioral model of each behavioral pattern; Specifically, the system uses time series data of behavioral features from each behavior pattern to train the corresponding initial model, thereby obtaining a standard behavioral model for each pattern. By training the model, the system can learn the typical characteristics and patterns of each behavior pattern, such as joint range of motion and speed. Such standard behavioral models provide a detailed description of normal behavior, which is helpful for subsequent detection and identification of abnormal behavior.

[0056] In one embodiment, step S32 includes: S321. Based on the time series of behavioral features corresponding to each behavioral pattern, extract the start and end times of each occurrence of the behavioral pattern to obtain the time stamp data of each behavioral pattern. Specifically, the time series of behavioral features corresponding to each behavioral pattern provides a continuous change process of the behavior on the timeline, including the characteristic change trajectory of the behavior's start, duration, and end. Based on these behavioral feature time series, each identified behavioral pattern is divided into segments. The start time is determined by the point when the behavior transitions from a non-pattern to the current pattern, and the end time is determined by the point when the behavior ends in the current pattern and transitions to another pattern. By traversing all historical data, multiple start and end times can be extracted for each behavioral pattern. These two types of time information are associated with the behavioral pattern number and organized into structured time-stamped data. The time-stamped data clearly records the occurrence position and duration interval of each behavioral pattern on the timeline.

[0057] S322. Based on the time stamp data, using the day as the time unit, map the occurrence time and duration of each behavior pattern within a day to a unified daily time axis, count the number of occurrences and total duration of each time period, and obtain the first cycle pattern parameters. This step, based on time-stamped data, first converts the start and end times of each behavior to a "day" time scale, constructing a unified daily time axis with a 24-hour cycle. Then, it maps the behavior intervals to specific moments and time periods on this time axis. Next, according to a preset time granularity, such as every 15 minutes or every 30 minutes, the day is divided into several time periods. The frequency of each behavior pattern within each time period and its cumulative duration within that period are then counted. Through these statistics, the typical occurrence times and activity levels of each behavior pattern on the daily scale can be obtained. These statistical results are organized into first-period regularity parameters to describe the temporal distribution characteristics of the behavior pattern within a day, such as a high incidence in the morning and a low incidence at night.

[0058] S323. Based on the time stamp data, using the week as the time unit, aggregate the data of multiple natural days according to the week dimension, and statistically analyze the frequency and duration distribution of each behavioral pattern on different dates and at different times of the week to obtain the second cycle regularity parameter. In this stage, the time scale is further expanded to the "weekly" level. Based on time-stamped data, the records of behavior over multiple days are divided into natural weeks and organized according to the dimensions of Monday to Sunday. For each behavioral pattern, within each specific day of the week, combined with the time period division on the daily time axis, the frequency and duration of the behavior are statistically analyzed to obtain the distribution of the behavior across different dates and times of the week. For example, the differences in activity levels between weekdays and weekends, or the differences in activity levels between morning and evening on certain specific days of the week, can be identified. These statistical results are compiled into a second periodicity parameter to characterize the periodicity of behavioral patterns on a weekly scale.

[0059] S324. Based on the time stamp data, using months as the time unit, aggregate the statistical results of multiple natural weeks according to the natural month dimension, and statistically analyze the frequency of occurrence and cumulative duration of each behavioral pattern in different date intervals to obtain the third cycle pattern parameter. This step extends to the monthly scale. Building upon the aforementioned weekly statistics, data from multiple natural weeks are aggregated by natural month. The frequency and cumulative duration of each behavioral pattern within different date intervals (e.g., the first 10 days, middle 10 days, and last 10 days of each month, or predefined key date intervals) are statistically analyzed. This allows us to capture changes in behavioral patterns over longer time scales, such as increased medical visits at the beginning of the month or increased activity during certain regular events in the middle of the month. The statistical results are compiled into a third-period pattern parameter to describe the periodicity and phased characteristics of the behavioral pattern on the monthly scale.

[0060] S325. Based on the first periodic regularity parameter, the second periodic regularity parameter and the third periodic regularity parameter, perform multi-scale modeling of the time distribution of each behavior pattern to obtain a multi-scale periodic regularity model of each behavior pattern. Finally, the first-cycle regularity parameters at the daily scale, the second-cycle regularity parameters at the weekly scale, and the third-cycle regularity parameters at the monthly scale are comprehensively modeled. Specifically, the frequency of occurrence and duration at each scale are normalized and used as features. Joint analysis of the activity patterns of behavioral patterns at different time scales is then performed to construct a multi-scale periodic regularity model describing the multi-level temporal distribution characteristics of the behavior within the day, week, and month. This model can take the form of a rule table, a statistical model, or a data-driven model, explicitly defining the high-probability occurrence intervals, low-probability occurrence intervals, and temporal distribution preferences of the behavioral pattern at different time scales. Using this multi-scale periodic regularity model, in subsequent anomaly detection, the degree of deviation between the current behavior and its normal temporal distribution pattern can be determined based on the current time's position on the daily, weekly, and monthly scales, thus providing a more refined characterization of the behavioral's temporal regularity.

[0061] This embodiment not only considers the occurrence patterns of behavioral patterns on a single time scale, but also systematically models the temporal distribution of behavioral patterns at the daily, weekly, and monthly levels. On the one hand, by using time-stamped data, the start and end times of each behavior are uniformly incorporated into a multi-level statistical framework of daily, weekly, and monthly. This allows behavioral patterns to move beyond the simple description of "certain behaviors occurring at a certain time each day," and instead capture the rhythmic differences between weekdays and weekends, different weeks, and different monthly intervals, forming a time profile that more closely reflects the actual living habits of the target elderly person. On the other hand, the multi-scale periodic pattern model provides rich temporal priors for subsequent anomaly detection. When determining whether a particular behavior is abnormal, the expected probability of its occurrence at the current time of day, current day of the week, and current monthly position can be comprehensively considered, significantly improving the ability to identify abnormal temporal behaviors and reducing false alarms and false negatives caused by ignoring long-term rhythmic changes. Overall, this embodiment, through multi-scale periodic pattern modeling, enables the normal behavior pattern library to not only possess standard behavior models in the feature space but also detailed rhythmic constraints in the time dimension. This improves the accuracy and robustness of individualized behavior pattern mining, and enhances the practical value and reliability of the elderly behavior monitoring system.

[0062] S33. Perform statistical analysis on the similarity between the time series of each behavioral feature to obtain the periodicity of each behavioral pattern; Statistical analysis of the similarity between time series of various behavioral characteristics reveals the periodic patterns of each behavior. By analyzing the degree of similarity and trends among different behavioral characteristic time series, the system can reveal the periodic patterns of each behavior, such as frequency and duration. These periodic patterns can help the system better understand the normal behavioral patterns of the elderly.

[0063] S34. Based on the standard behavioral models of each behavioral pattern and the periodicity, the normal behavioral pattern library is obtained.

[0064] A normal behavior pattern library will be constructed based on standard behavior models and periodic patterns for each behavior pattern. This library will contain detailed descriptions of each normal behavior pattern, including typical characteristics and periodic patterns. The establishment of the normal behavior pattern library provides a reference standard for the system, which can be used to identify and judge whether the elderly's behavior is normal, thereby enabling timely detection and alarm of abnormal behavior.

[0065] S4. Based on the real-time video stream data and the normal behavior pattern database, determine whether the target elderly person exhibits abnormal behavior; Specifically, this step acquires real-time video stream data captured by the camera. This video stream data contains behavioral information about the target elderly person. Each frame of the real-time video stream is processed to extract behavioral feature data of the target elderly person, such as joint range of motion, speed, and posture. The extracted behavioral feature data is compared and matched with standard behavioral models in a normal behavior pattern library. For each behavioral pattern, the system calculates the similarity or deviation between the real-time behavioral feature data and the corresponding standard behavioral model. Based on a similarity or deviation threshold, the system determines whether the current behavior matches a normal behavior pattern. If the matching result of the real-time behavioral feature data with any behavioral pattern does not meet the preset threshold condition, the system determines that the target elderly person exhibits abnormal behavior.

[0066] Preferably, determining whether the target elderly person exhibits abnormal behavior based on real-time video stream data and the normal behavior pattern database includes: S41. Based on the real-time video stream data, obtain the real-time behavioral characteristics time series of the target elderly person; Real-time video stream data is used to extract real-time behavioral characteristics of the target elderly person. By processing and analyzing images in the real-time video stream, the system can obtain behavioral characteristic data of the elderly person, such as the range of motion, speed, and rate of change of speed of each joint. This behavioral characteristic data will be used for subsequent abnormal behavior detection and identification. The method of obtaining the time series of behavioral characteristics from real-time video stream data is the same as in the previous embodiments, and will not be repeated here. S42. Based on the normal behavior pattern library, obtain the standard behavior model and deviation threshold corresponding to the behavior pattern of the real-time behavior feature time series; Based on a normal behavior pattern library, the system will obtain standard behavior models and deviation thresholds corresponding to real-time behavior feature time series patterns. For each behavior pattern, the system will retrieve the corresponding standard behavior model from the model library and set a corresponding deviation threshold to determine whether the behavior is abnormal. Different behavior models have different deviation thresholds because different behaviors may correspond to different levels of importance and risk. Setting different thresholds allows for more precise control and management of the abnormality of different behaviors based on actual circumstances. Some behaviors may be more critical to the health and safety of the elderly, such as falls or sudden falls, while some daily activities such as walking and standing may be relatively less urgent. Therefore, for behaviors with higher importance, a lower deviation threshold can be set to improve the sensitivity and accuracy of detection. Some behaviors may have higher risks and may lead to serious consequences, such as falls that could result in fractures or other serious injuries. In contrast, some behavioral abnormalities may be relatively minor and will not cause significant impact. Therefore, for behaviors with higher risk, a lower deviation threshold can be set to ensure timely detection and handling of abnormal situations. For example, suppose there are two behavior patterns: falling and standing. For high-risk, high-importance behaviors like falling, a lower deviation threshold can be set to ensure timely detection of any potential anomalies; for example, a threshold of 0.1. Conversely, for relatively low-risk, low-importance behaviors like standing, a higher deviation threshold can be set to reduce false alarms; for example, a threshold of 0.5. These settings can be adjusted according to the characteristics of different behaviors to balance detection sensitivity and accuracy.

[0067] S43. Calculate the degree of deviation between the real-time behavioral feature data and the corresponding standard behavioral model; The degree of deviation between real-time behavioral feature data and the corresponding standard behavioral model can be calculated using various distance or similarity metrics. Common methods include Euclidean distance, Manhattan distance, cosine similarity, and Mahalanobis distance. These methods quantify the degree of similarity or difference between real-time behavioral data and the standard behavioral model, thereby determining whether abnormal behavior exists. By comparing the differences between real-time data and the standard model, the system can assess the similarity between current behavior and normal behavior and calculate the degree of deviation. This degree of deviation will be used to determine whether the behavior is abnormal.

[0068] In one embodiment, step S43 specifically includes: S411. Based on the real-time behavioral feature time series and the standard behavioral model in the normal behavioral pattern library, calculate the feature distance between the real-time behavioral feature and the corresponding standard behavioral model in the feature space to obtain the first deviation index. In this step, firstly, based on the real-time behavior feature time series extracted from the real-time video stream, the behavior features at the current moment or within the current time window are represented as feature vectors, such as data including the range of motion, velocity, and rate of velocity change of each joint. Simultaneously, a standard behavior model corresponding to the current behavior pattern is retrieved from the normal behavior pattern library. This standard behavior model also provides the feature range of the behavior under normal conditions in the form of feature vectors or feature distributions.

[0069] After obtaining the real-time feature vector and the standard model feature vector, the feature distance between them is calculated based on a distance metric in the feature space, such as Euclidean distance, Manhattan distance, or Mahalanobis distance. This feature distance reflects the degree of deviation of the current behavioral features from the standard behavioral features; the larger the distance, the more significant the difference between the real-time behavior and normal behavior. Through the above calculation, this feature distance is used as the first deviation index, providing a basic quantitative result at the spatial feature level for subsequent comprehensive deviation assessment.

[0070] S412. Based on the multi-scale periodic law model, obtain the expected distribution of the occurrence time of real-time behavior on multiple time scales such as day, week, and month, and calculate the time deviation of the occurrence time of real-time behavior relative to the expected distribution to obtain the second deviation index. In this step, the multi-scale periodic pattern model constructed above is first used to obtain the expected time distribution information of the current behavior pattern on the daily, weekly and monthly scales from the normal behavior pattern library, including the high frequency occurrence intervals and low frequency occurrence intervals in different time periods, different days of the week and different date intervals.

[0071] Then, the specific time of the current behavior is mapped to the aforementioned multiple time scales, such as the specific time period of the day, the current day of the week, and the date position within the current month. The time of the current behavior is compared with the high-probability time interval of that behavior in the multi-scale periodic pattern model, calculating its deviation on the intraday time axis, the intraweek date dimension, and the monthly date interval. This deviation can be quantified using methods such as time distance, probability difference, or distribution shift. These time deviations are then combined to obtain a second deviation index. This second deviation index reflects whether the current behavior deviates from the elderly person's existing individualized daily routine and circadian rhythm in the time dimension.

[0072] S413. Based on the continuous change sequence of real-time behavior patterns and common behavior sequence patterns obtained from historical statistics, match the current behavior sequence with common behavior sequences, calculate the degree of deviation in the behavior sequence structure, and obtain the third deviation index. This step focuses on the rationality of the behavior sequence structure. First, based on the behavior recognition results obtained from real-time monitoring, the continuous changes in the elderly's behavior patterns over a period of time are recorded to form a real-time behavior pattern sequence. For example, behaviors such as getting up, walking, sitting down, and resting are arranged in chronological order to constitute the current behavior sequence.

[0073] Meanwhile, based on historical data, statistical analysis was conducted on the long-term behavioral patterns of the elderly, revealing several common behavioral sequence patterns. For example, after getting up, they usually wash up and eat immediately; after getting up at night, they usually engage in short-term activities before returning to rest. These high-frequency behavioral sequences were stored as normal behavioral chains.

[0074] Subsequently, the current real-time behavioral sequence is matched with these common behavioral sequences. Methods such as sequence matching, edit distance, or pattern matching can be used to measure the degree of structural deviation of the current behavioral sequence from common behavioral chains. The greater the deviation, the more significant the difference between the current behavioral chain and the individual's past behavioral habits. Through this matching process, the deviation in the behavioral sequence structure is quantified into a third deviation index, which reflects the degree of abnormality in the sequence organization of the behavior.

[0075] S414. Based on the fluctuations in speed, range of motion, and rate of change of speed in the real-time behavioral characteristic time series, and combined with the fluctuation range of each behavioral pattern under normal conditions, calculate the range deviation of the current fluctuation amplitude to obtain the fourth deviation index. This step focuses on the volatility characteristics of behavioral features. First, based on the real-time behavioral feature time series, statistical analysis is performed on the changes of key features such as speed, range of motion, and rate of change of speed within the current time window. For example, the maximum value, minimum value, standard deviation, or other volatility indicators of the features are calculated.

[0076] Then, the fluctuation range information of the corresponding behavior pattern under normal conditions is extracted from the normal behavior pattern library, that is, the usual fluctuation range and fluctuation level of each feature under normal conditions, which is used as a reference range. The fluctuation degree of the real-time feature is compared with the corresponding normal fluctuation range to determine whether the current fluctuation amplitude exceeds the normal range, and to what extent.

[0077] By comprehensively evaluating the fluctuation deviations of various characteristics, the deviation of the current behavior from normal behavior in terms of stability, intensity, or jitter can be obtained. This deviation is quantified as the fourth deviation index, used to reflect whether the current behavior is abnormally intense or abnormally sluggish, or otherwise inconsistent with the normal fluctuation pattern.

[0078] S415. Normalize the first deviation index, the second deviation index, the third deviation index and the fourth deviation index, and perform weighted fusion of each deviation index according to the preset feature importance weight or the fusion parameters obtained through training to obtain the comprehensive deviation degree. First, the four types of deviation indices obtained above are normalized to unify their numerical ranges to a comparable interval, so as to avoid the adverse effects of different indices' dimensions and numerical ranges on the fusion results. For example, methods such as maximum-minimum normalization or standard deviation normalization can be used to convert each deviation index into a dimensionless standardized value.

[0079] Subsequently, based on the importance of each deviation indicator in the judgment of abnormal behavior, corresponding weights are assigned to different indicators. These weights can be pre-set based on experience or obtained by training on historical labeled data, making the fusion process more in line with actual application scenarios. After obtaining the normalized values ​​and corresponding weights of each deviation indicator, the four types of deviation indicators are weighted and summed or otherwise fused to obtain a single comprehensive deviation degree.

[0080] This comprehensive deviation assessment considers multiple dimensions, including spatial distance, deviation from temporal cycle patterns, deviation from behavioral sequence structure, and deviation from characteristic fluctuations. It is one of the core quantitative indicators used to ultimately determine whether behavior is abnormal. This multi-indicator weighted fusion approach avoids the bias inherent in single-indicator assessments, thus comprehensively reflecting the overall degree of deviation of the current behavior from normal behavioral patterns.

[0081] This embodiment no longer relies solely on a single feature distance or simple threshold to determine whether behavior is abnormal. Instead, it employs a comprehensive deviation assessment of real-time behavior from four different dimensions: feature space similarity, deviation from temporal cycle patterns, deviation from behavioral sequence structure, and deviation from feature fluctuation characteristics. On one hand, the feature distance index ensures consistency between the current action and the standard behavioral model at the motion feature level. On the other hand, the time deviation index based on multi-scale periodic patterns and the sequence deviation index based on common behavioral chains enable the system to identify abnormal behaviors that significantly violate an individual's daily routines in terms of time and behavioral organization. Simultaneously, the fluctuation deviation index focuses on behavioral stability and drastic abnormal changes, helping to identify acute risk behaviors. Finally, by normalizing and weighting the multiple deviation indices, the resulting comprehensive deviation level more accurately characterizes the overall degree of behavioral deviation, mitigating the problem of single indicators being susceptible to noise interference. This helps reduce false alarm and false negative rates, improves the reliability and robustness of abnormal behavior detection, and ultimately enhances the practical effectiveness of the elderly behavior monitoring system in real-world applications.

[0082] S44. When the degree of deviation is greater than the degree of deviation threshold, the target elderly person exhibits abnormal behavior; wherein, the degree of deviation threshold is different for different behavior models.

[0083] The system compares the calculated deviation with a set deviation threshold. If the deviation exceeds the threshold, the system determines that the elderly person is exhibiting abnormal behavior. This mechanism helps the system promptly identify and alert to abnormal behavior, ensuring the safety of the elderly person.

[0084] S5. When abnormal behavior is detected, an alarm will be triggered based on the abnormal behavior.

[0085] First, once abnormal behavior is confirmed in the target elderly person, an alarm mechanism will be triggered. Alarms can be triggered in several ways, such as sending alerts to relevant personnel, including family members, caregivers, or medical staff, so they can take timely action. The system can also send alert messages to relevant personnel via SMS or telephone, alerting them to the abnormal situation. Application notifications: If the system has a corresponding mobile application, it can send alert notifications to users through the application. In some cases, the system may automatically trigger emergency assistance procedures, such as calling emergency services or notifying community care facilities. Alarm notifications may include key information such as the type of abnormal behavior, the time and location of the occurrence, so that relevant personnel can accurately understand the situation.

[0086] Through this step, the system can promptly alert relevant personnel to help them take appropriate action and ensure the safety and health of the target elderly person.

[0087] Preferably, the step of triggering an alarm based on the abnormal behavior when abnormal behavior occurs includes: S51. Based on the real-time behavior feature time series corresponding to the abnormal behavior, the real-time video stream data is extracted to obtain the video segment where the abnormality occurred; After detecting abnormal behavior in the target elderly person, the system extracts relevant video segments from the real-time video stream data based on the corresponding real-time behavioral feature time series. First, the system determines the time point when the abnormal behavior occurs, based on the abnormal behavior information detected in the real-time behavioral feature time series. Once the time point of the abnormal behavior is detected, the system needs to determine the range of video segments to be extracted. This typically includes a period of time before and after the abnormal behavior occurs, providing a complete picture of the abnormal behavior process. Based on the determined time range, the system extracts relevant video segments from the real-time video stream data. The extracted segments should cover a sufficiently long period before and after the abnormal behavior to ensure that the entire abnormal behavior process is captured.

[0088] S52. Determine whether a preset privacy scenario exists in the video segment where the anomaly occurred; After monitoring elderly individuals for abnormal behavior, the system captures relevant video clips of this behavior. To protect the elderly's privacy, the system needs to determine the privacy scenarios within these video clips and take appropriate action as necessary. These preset privacy scenarios typically involve privacy-sensitive behaviors or activities of the elderly, such as personal hygiene, dressing, bathing, and using the toilet. The system analyzes the content of the video clips to determine if preset privacy scenarios exist. For example, by detecting the background, objects, actions, or behavioral patterns in the video, it can determine whether the elderly person is in a private place such as a toilet or bedroom. In addition to scene recognition, the system can also determine whether privacy-sensitive behaviors are involved by detecting the elderly person's state, such as clothing, location, or actions. Privacy scenario determination can also be based on time and location restrictions. For example, activities at night or during specific time periods, as well as activities in specific rooms, may be more likely to involve privacy. S53. If it exists, the preset privacy scene in the video segment where the anomaly occurred is occluded to obtain the target anomaly video. Based on the preset privacy scenario (such as a private place like a bathroom or bedroom, or actions involving personal hygiene), determine the areas that need to be obscured. This may include parts of the elderly person's body, parts of their clothing that are not properly covered, or specific background elements. Choose an appropriate obscuration method as needed. Common obscuration methods include blurring, mosaicking, and blackout processing. Blurring makes the privacy-sensitive area less clear, mosaicking divides the area into smaller pieces, and reducing clarity. The processed video clip is then output as the target anomalous video, which no longer contains privacy-sensitive content while retaining necessary information about abnormal behavior. S54. Issue an alarm based on the abnormal video of the target.

[0089] The system triggers an alarm on processed abnormal video clips, sending them to relevant guardians, medical institutions, or security personnel. This alarm ensures that relevant personnel are promptly aware of the elderly person's unusual behavior and can take appropriate intervention measures. These steps not only help in the timely detection and handling of abnormal behavior but also provide complete and accurate information for subsequent processing and decision-making, while protecting the elderly person's privacy and safety.

[0090] Example 2 Please see Figure 4 This invention provides a device for monitoring elderly behavior based on pattern mining, the device comprising: The feature data acquisition module is used to acquire behavioral feature data of the target elderly person at different time periods based on historical video stream data, wherein the historical video stream data includes the target elderly person; The pattern mining module is used to mine action feature data from different time periods using a time series matching algorithm to obtain the target elderly person's behavior patterns and corresponding behavior regularity data. The pattern library establishment module is used to establish a normal behavior pattern library based on the behavior patterns and corresponding behavior pattern data, wherein the normal behavior pattern library includes standard behavior models corresponding to each of the behavior patterns; The judgment module is used to determine whether the target elderly person has abnormal behavior based on real-time video stream data and the normal behavior pattern library; The alarm module is used to trigger an alarm based on the abnormal behavior when such behavior is detected.

[0091] It should be noted that each module and unit in the elderly behavior monitoring device based on pattern mining in this embodiment corresponds one-to-one with each step in the elderly behavior monitoring based on pattern mining in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned elderly behavior monitoring method based on pattern mining, and will not be repeated here.

[0092] Example 3 In addition, combined Figure 1 The pattern-mining-based elderly behavior monitoring method described in this embodiment of the invention can be implemented by an electronic device. Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention is shown.

[0093] Electronic devices may include processors and memory storing computer program instructions.

[0094] Specifically, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0095] The memory may include a large-capacity storage device for data or instructions. For example, and not limitingly, the memory may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include removable or non-removable (or fixed) media. Where appropriate, the memory may be internal or external to a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0096] The processor reads and executes computer program instructions stored in memory to implement any of the pattern-mining-based elderly behavior monitoring methods in the above embodiments.

[0097] In one example, the electronic device may also include a communication interface and a bus. For example, Figure 5 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0098] The communication interface is mainly used to enable communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0099] A bus, including hardware, software, or both, couples components of an electronic device together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, a bus may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0100] Example 4 Furthermore, in conjunction with the pattern-mining-based elderly behavior monitoring method in the above embodiments, this invention can be implemented using a computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the pattern-mining-based elderly behavior monitoring methods in the above embodiments.

[0101] In summary, the elderly behavior monitoring method, device, equipment, and medium based on pattern mining provided in this invention can acquire behavioral characteristic data of the target elderly person at different time periods through historical video data. This provides a data foundation for subsequent behavioral pattern mining and pattern analysis. Mining historical behavioral data using time series matching algorithms can reveal behavioral patterns and regularities of the elderly person, such as daily habits and activity patterns, thereby establishing an understanding of normal behavior and providing a benchmark for identifying abnormal behavior. Establishing the mined behavioral patterns and regularities into a normal behavior pattern library can form a model of normal behavior in the elderly person. Such a model library can serve as a basis for judging whether the elderly person's behavior is abnormal, helping to improve the accuracy and stability of abnormal behavior identification. By comparing real-time video stream data with the established normal behavior pattern library, it is possible to determine in real time whether the target elderly person exhibits abnormal behavior. This allows for timely detection of abnormal states in the elderly person, providing timely warnings and feedback for providing assistance and care. When the system detects abnormal behavior, it can issue an alarm in a timely manner, reminding relevant personnel to handle and intervene. This helps prevent accidents or emergencies involving the elderly person and provides timely help and support.

[0102] It should be noted that all actions involving the acquisition of signals, information, or data in this invention are performed in compliance with the relevant data protection laws and regulations of the locality and with authorization from the owner of the corresponding device. It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0103] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0104] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0105] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A method for monitoring elderly behavior based on pattern mining, characterized in that, The method includes: Based on historical video stream data, behavioral characteristic data of the target elderly person are obtained at different time periods, wherein the historical video stream data includes the target elderly person; The action feature data of the target elderly person are mined by time series matching algorithm to obtain the behavior pattern and corresponding behavior regularity data of the target elderly person. Based on the behavioral patterns and corresponding behavioral pattern data, a normal behavior pattern library is established, wherein the normal behavior pattern library includes standard behavior models corresponding to each of the behavioral patterns. Based on real-time video stream data and the normal behavior pattern database, it is determined whether the target elderly person exhibits abnormal behavior; When abnormal behavior is detected, an alarm will be triggered based on the abnormal behavior.

2. The method for monitoring elderly behavior based on pattern mining according to claim 1, characterized in that, The process involves acquiring behavioral data of the target elderly person at different time periods based on historical video stream data, including... The historical video stream data is preprocessed to obtain the target video stream data, wherein the preprocessing includes one or more of noise reduction processing, contrast adjustment and brightness adjustment; The target elderly person in the video stream data is located and tracked to obtain the key skeletal points of the target elderly person; Based on the skeletal key points, obtain the body posture data of the target elderly person; Based on the body posture data, behavioral characteristic data for different time periods are obtained, wherein the behavioral characteristic data includes the range of motion, speed, and rate of change of speed of each joint of the target elderly person.

3. The method for monitoring elderly behavior based on pattern mining according to claim 2, characterized in that, The step of mining action feature data from different time periods using a time series matching algorithm to obtain the target elderly person's behavior patterns and corresponding behavioral regularity data includes: The video stream data is segmented based on the behavioral feature data to obtain several video segments; Based on the behavioral feature data, label the behavioral pattern of each video segment; Based on the labeled video segments and the corresponding behavioral feature data, obtain the behavioral feature time series corresponding to each behavioral pattern; Based on the time series matching algorithm and the time series of each behavioral feature, the behavioral patterns and corresponding behavioral regularity data of the target elderly are obtained.

4. The method for monitoring elderly behavior based on pattern mining according to claim 3, characterized in that, The time series matching algorithm is a dynamic time warping algorithm. The step of obtaining the target elderly person's behavioral patterns and corresponding behavioral regularity data based on the time series of the time series matching algorithm and each behavioral feature includes: The behavioral feature time series is input into a pre-built time series model to obtain the probability distribution data of each behavioral pattern, wherein the time series model is built based on an LSTM model; The similarity between the time series of each behavioral feature is obtained based on the dynamic time warping algorithm. Based on the similarity and probability distribution data, the behavioral patterns and corresponding behavioral regularities of the target elderly person are obtained.

5. The method for monitoring elderly behavior based on pattern mining according to claim 4, characterized in that, Based on the aforementioned behavioral patterns and corresponding behavioral regularity data, a normal behavioral pattern library is established, including: Modeling is performed based on the probability distribution data of each behavior pattern to obtain an initial model for each behavior pattern, wherein the initial model is constructed based on the SVM classification model; Based on the time series of behavioral characteristics of each behavioral pattern, train the corresponding initial model to obtain the standard behavioral model of each behavioral pattern; Statistical analysis of the similarity between the time series of the aforementioned behavioral features reveals the periodicity of each behavioral pattern. Based on the standard behavioral models of each behavioral pattern and the periodic patterns described therein, the normal behavioral pattern library is obtained.

6. The method for monitoring elderly behavior based on pattern mining according to any one of claims 1-5, characterized in that, The step of determining whether the target elderly person exhibits abnormal behavior based on real-time video stream data and the normal behavior pattern database includes: Based on the real-time video stream data, obtain the real-time behavioral characteristics time series of the target elderly person; Based on the normal behavior pattern library, obtain the standard behavior model and deviation threshold corresponding to the behavior pattern of the real-time behavior feature time series; Calculate the degree of deviation between the real-time behavioral feature data and the corresponding standard behavioral model; When the degree of deviation is greater than the degree of deviation threshold, the target elderly person exhibits abnormal behavior; The threshold for the degree of deviation varies among different behavioral models.

7. The method for monitoring elderly behavior based on pattern mining according to claim 6, characterized in that, The provision of triggering an alarm based on abnormal behavior includes: Based on the real-time behavior feature time series corresponding to the abnormal behavior, the real-time video stream data is segmented to obtain the video segment where the abnormality occurred; Determine whether a preset privacy scenario exists in the video segment where the anomaly occurred; If present, the preset privacy scene in the video segment where the anomaly occurred is occluded to obtain the target anomaly video; An alarm will be triggered based on the abnormal video of the target.

8. A device for monitoring elderly behavior based on pattern mining, characterized in that, The device includes: The feature data acquisition module is used to acquire behavioral feature data of the target elderly person at different time periods based on historical video stream data, wherein the historical video stream data includes the target elderly person; The pattern mining module is used to mine action feature data from different time periods using a time series matching algorithm to obtain the target elderly person's behavior patterns and corresponding behavior regularity data. The pattern library establishment module is used to establish a normal behavior pattern library based on the behavior patterns and corresponding behavior pattern data, wherein the normal behavior pattern library includes standard behavior models corresponding to each of the behavior patterns; The judgment module is used to determine whether the target elderly person has abnormal behavior based on real-time video stream data and the normal behavior pattern library; The alarm module is used to trigger an alarm based on the abnormal behavior when such behavior is detected.

9. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method as described in any one of claims 1-7.

10. A storage medium storing computer program instructions thereon, characterized in that, The method as described in any one of claims 1-7 is implemented when the computer program instructions are executed by the processor.

Images