Abnormal behavior detection method and device, electronic equipment and storage medium

Abstract

The application discloses an abnormal behavior detection method and device, electronic equipment and a storage medium. The abnormal behavior detection method comprises the following steps: obtaining monitoring data for a target user; preprocessing the monitoring data to obtain a data sequence; inputting the data sequence into a behavior recognition model, determining a behavior sequence of the target user according to the output of the behavior recognition model; inputting the data sequence and the behavior sequence into an anomaly detection model, and determining whether the target user has an abnormal behavior according to the output of the anomaly detection model. The technical scheme provided by the application can timely discover the abnormal behavior of the user, timely intervene, delay the development of the disease, and save the treatment cost.

Description

Technical Field

[0001] This application relates to the field of computer application technology, and in particular to an abnormal behavior detection method, device, electronic device, and storage medium. Background Technology

[0002] With the increasing aging of the population, the number of Alzheimer's disease patients (ADP) is gradually increasing. Alzheimer's disease is a progressive neurodegenerative disease with insidious onset, clinically characterized by comprehensive dementia manifestations such as memory impairment, aphasia, apraxia, agnosia, visuospatial skill impairment, executive dysfunction, and personality and behavioral changes.

[0003] Alzheimer's disease typically progresses through three stages: mild, moderate, and severe. Most Alzheimer's patients are overlooked in the mild stage, and by the time they seek medical attention, the disease has usually progressed to the moderate or severe stage. The moderate and severe stages are irreversible and incur high treatment costs. If a patient is diagnosed in the mild cognitive impairment stage and receives timely intervention, the progression of the disease can be slowed, and treatment costs can be reduced.

[0004] Therefore, how to detect abnormal behavior in the elderly in a timely manner is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide an abnormal behavior detection method, device, electronic device, and storage medium to detect abnormal user behavior in a timely manner.

[0006] To solve the above-mentioned technical problems, this application provides the following technical solution:

[0007] Firstly, an abnormal behavior detection method is provided, including:

[0008] Obtain monitoring data for the target users;

[0009] The monitoring data is preprocessed to obtain a data sequence;

[0010] The data sequence is input into the behavior recognition model, and the behavior sequence of the target user is determined based on the output of the behavior recognition model.

[0011] The data sequence and the behavior sequence are input into the anomaly detection model, and the output of the anomaly detection model is used to determine whether the target user has abnormal behavior.

[0012] Optionally, the behavior recognition model and / or the anomaly detection model includes at least one of the following model structures:

[0013] Multi-head convolutional neural network-long short-term memory network model structure;

[0014] Attention mechanism - convolutional neural network - long short-term memory network model structure.

[0015] Optionally, in the multi-head-convolutional neural network-long short-term memory network model structure, multiple convolutional neural networks are connected in parallel. Each convolutional neural network receives and processes the input sequence. The outputs of the multiple convolutional neural networks enter the fully connected layer, which merges and interprets the data before inputting it into the long short-term memory network to output the prediction result.

[0016] Optionally, in the attention mechanism-convolutional neural network-long short-term memory network model structure, the convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer connected in sequence. The convolutional neural network receives the input sequence and extracts features from the input sequence. The long short-term memory network learns the data change pattern based on the features extracted by the convolutional neural network. After the attention part iterates to obtain the weights of different features, the output results of the long short-term memory network are re-weighted, and the prediction results are output.

[0017] Optionally, the preprocessing of the monitoring data to obtain a data sequence includes:

[0018] The monitoring data is then standardized.

[0019] The standardized monitoring data is segmented using a sliding window to obtain a data sequence.

[0020] Optionally, obtaining monitoring data for the target user includes:

[0021] Obtain raw monitoring data for the target users;

[0022] The raw monitoring data is processed by filling in missing values ​​and / or reducing noise to obtain monitoring data for the target user.

[0023] Optionally, the behavior recognition model and the anomaly detection model are obtained in advance through the following steps:

[0024] Obtain a sample dataset, which includes multiple sets of monitoring sample data, each set of monitoring sample data corresponding to a user sample;

[0025] The sample dataset is divided into a training dataset and a test dataset;

[0026] The initial recognition model is trained using the training dataset, and the model parameters of the initial recognition model are adjusted. The initial recognition model is the initial model of the behavior recognition model.

[0027] The initial detection model is trained using the training dataset and the output of the initial recognition model, and the model parameters of the initial detection model are adjusted. The initial detection model is the initial model of the anomaly detection model.

[0028] The trained initial recognition model and initial detection model are tested using the test dataset. If the test results meet the preset requirements, the trained initial recognition model is determined as the behavior recognition model, and the trained initial detection model is determined as the anomaly detection model.

[0029] Secondly, an abnormal behavior detection device is provided, comprising:

[0030] The first acquisition module is used to acquire monitoring data for the target user;

[0031] The second acquisition module is used to preprocess the monitoring data to obtain a data sequence;

[0032] The first determining module is used to input the data sequence into the behavior recognition model and determine the behavior sequence of the target user based on the output of the behavior recognition model.

[0033] The second determining module is used to input the data sequence and the behavior sequence into the anomaly detection model, and determine whether the target user has abnormal behavior based on the output of the anomaly detection model.

[0034] Thirdly, an electronic device is provided, comprising:

[0035] Memory, used to store computer programs;

[0036] A processor for implementing the steps of the abnormal behavior detection method as described in the first aspect when executing the computer program.

[0037] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the abnormal behavior detection method as described in the first aspect.

[0038] Fifthly, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium and adapted to be read and executed by a processor to cause a computer device having the processor to perform the steps of the abnormal behavior detection method as described in the first aspect.

[0039] By applying the technical solution provided in the embodiments of this application, after obtaining monitoring data for the target user, the monitoring data is preprocessed to obtain a data sequence. The data sequence is then identified using a behavior recognition model to determine the target user's behavior sequence. The data sequence and behavior sequence are input into an anomaly detection model. Based on the output of the anomaly detection model, it can be determined whether the target user exhibits abnormal behavior. Through the joint detection of the behavior recognition model and the anomaly detection model, abnormal user behavior can be detected in a timely manner, allowing for timely intervention, which helps to delay the progression of the disease and save treatment costs.

[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram illustrating one application scenario of an embodiment of this application;

[0043] Figure 2 This is a flowchart illustrating the implementation of an abnormal behavior detection method in an embodiment of this application.

[0044] Figure 3 This is a schematic diagram of the convolutional neural network feature extraction process in an embodiment of this application;

[0045] Figure 4 This is a schematic diagram of a model structure in an embodiment of this application;

[0046] Figure 5 This is a schematic diagram of data segmentation in an embodiment of this application;

[0047] Figure 6 This is a schematic diagram of the model acquisition process in an embodiment of this application;

[0048] Figure 7 This is a schematic diagram illustrating the performance of a conventional CNN-LSTM model in a comparison of models in the embodiments of this application;

[0049] Figure 8 This is a schematic diagram illustrating the performance of the MH-CNN-LSTM model in the model comparison of the embodiments of this application;

[0050] Figure 9This is another schematic diagram illustrating the performance of a conventional CNN-LSTM model in the model comparison of the embodiments of this application;

[0051] Figure 10 This is another schematic diagram illustrating the performance of the MH-CNN-LSTM model in the model comparison of the embodiments of this application;

[0052] Figure 11 This is a schematic diagram illustrating an application example in an embodiment of this application;

[0053] Figure 12 This is a schematic diagram of the structure of an abnormal behavior detection device in an embodiment of this application;

[0054] Figure 13 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0055] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] The core of this application is to provide an abnormal behavior detection method, which can be applied to scenarios such as monitoring abnormal behavior of users, especially the elderly, or protecting the health of users, especially the elderly.

[0057] As the aging population intensifies, elderly people, especially those living alone, may exhibit various unusual behaviors at home, such as falls. If these are not detected in time, they can cause serious harm. Furthermore, abnormalities in the sequence of daily behaviors may indicate certain chronic diseases, such as Alzheimer's disease. Early detection of these unusual behaviors can effectively reduce potential health risks.

[0058] After obtaining monitoring data for the target user, the monitoring data is preprocessed to obtain a data sequence. The data sequence is then identified using a behavior recognition model to determine the target user's behavior sequence. The data sequence and behavior sequence are then input into an anomaly detection model. Based on the output of the anomaly detection model, it can be determined whether the target user exhibits abnormal behavior. The joint detection by the behavior recognition model and the anomaly detection model can promptly detect abnormal user behavior and intervene in a timely manner, which helps to delay the progression of the disease and save on treatment costs.

[0059] To facilitate understanding, an application scenario of the technical solution of this application will be introduced below. See [link / reference] Figure 1Wearable devices, cameras, and other monitoring equipment are deployed in the target user's daily life environment. These monitoring devices can communicate with the application server to transmit monitoring data. The application server obtains monitoring data for the target user through the monitoring devices, preprocesses the data to obtain a data sequence, and inputs the data sequence into a behavior recognition model. The behavior recognition model can perform human behavior recognition, such as identifying behavior 1, behavior 2, ..., behavior N. Based on the output of the behavior recognition model, the target user's behavior sequence can be determined. Then, the data sequence and behavior sequence are input into an anomaly detection model. Based on the output of the anomaly detection model, it can be determined whether the target user exhibits abnormal behavior.

[0060] It should be noted that the above explanation uses a single application server as an example. However, in practical applications, the application server can be replaced by an application server cluster or a distributed cluster composed of multiple application servers. Correspondingly, in Figure 1 In this context, the application server can also be replaced by an application platform consisting of multiple application servers.

[0061] See Figure 2 The diagram shown is an implementation flowchart of an abnormal behavior detection method provided in this application embodiment. The method may include the following steps:

[0062] S210: Obtain monitoring data for the target user.

[0063] In this embodiment, the target user is any user subject to abnormal behavior detection, including elderly users or other user groups. The target user can carry a non-invasive everyday wearable device, which may embed inertial sensors such as magnetometers, three-axis accelerometers, gyroscopes, and indoor positioning systems. Image acquisition devices such as cameras can also be deployed in the target user's living environment. These can all be referred to as monitoring devices. The monitoring devices can continuously monitor various states of the target user and obtain monitoring data.

[0064] Monitoring equipment can be used to obtain monitoring data for target users.

[0065] S220: Preprocess the monitoring data to obtain a data sequence.

[0066] Understandably, monitoring the status of a target user is a continuous process, with corresponding monitoring data being generated continuously. After obtaining the monitoring data for the target user, the data can be preprocessed to obtain a data sequence. The data sequence includes multiple sets of data that are correlated over time.

[0067] S230: Input the data sequence into the behavior recognition model, and determine the target user's behavior sequence based on the output of the behavior recognition model.

[0068] In this embodiment of the application, a behavior recognition model can be pre-trained to perform human behavior recognition. The input of the behavior recognition model can be a data sequence, and the output can be the probability of each behavior recognized.

[0069] By obtaining monitoring data for the target user and preprocessing the monitoring data to obtain a data sequence, the data sequence can be input into the behavior recognition model. Through the processing of the behavior recognition model, the output of the behavior recognition model can be obtained, which can be understood as obtaining the probability of each behavior corresponding to each group of data included in the data sequence.

[0070] Based on the output of the behavior recognition model, the behavior sequence of the target user can be determined, such as behavior 1, behavior 2, ..., behavior N.

[0071] Optionally, based on the output of the behavior recognition model, the behavior with the highest probability corresponding to each group of data in the data sequence can be obtained first, and the sequence composed of the behavior with the highest probability can be determined as the behavior sequence of the target user.

[0072] Optionally, the probability of each behavior corresponding to each group of data in the data sequence can be obtained first based on the output of the behavior recognition model. For each behavior corresponding to a certain group of data, if the probability of the behavior is the highest and greater than or equal to the set probability threshold, then it is determined that the group of data actually corresponds to the behavior. If the probability of the behavior is the highest but less than the probability threshold, then it is considered that the group of data does not actually correspond to any behavior, and the behavior corresponding to the group of data is not included in the determined target user's behavior sequence.

[0073] S240: Input the data sequence and behavior sequence into the anomaly detection model, and determine whether the target user has abnormal behavior based on the output of the anomaly detection model.

[0074] In this embodiment, an anomaly detection model can be pre-trained to identify abnormal behavior. Its input can be a data sequence and a behavior sequence (containing behavior labels), and its output can be the probability of detected normal or abnormal behavior, or a determination of detected normal or abnormal behavior. The anomaly detection model can detect whether the order of behaviors in a behavior sequence is normal, and it can also detect whether specific behaviors in a behavior sequence are normal. For example, if a behavior in a behavior sequence does not conform to a normal behavior order, it can be considered an abnormal behavior; or, if a behavior in a behavior sequence does not conform to normal behavior, it can be considered an abnormal behavior.

[0075] After determining the target user's behavioral sequence using a behavior recognition model, the data sequence and behavioral sequence can be further input into an anomaly detection model. Through processing by the anomaly detection model, its output can be obtained. Based on the output of the anomaly detection model, it can be determined whether the target user exhibits abnormal behavior.

[0076] The joint action recognition model and anomaly detection model can be understood as a classifier chain, with the action recognition model as the first classifier and the anomaly detection model as the second. The recognized action sequences and data sequences can be used as inputs to the anomaly detection model. The output of the action recognition model is added as input to the anomaly detection model. The input space of the anomaly detection model is (x, y, z, label), where (x, y, z) is the data sequence corresponding to the monitoring data obtained from the monitoring equipment, i.e., the actual input features, and also the input space of the first classifier. "Label" represents the action label, which is the output of the first classifier. For the second classifier, "label" is used as input along with (x, y, z). That is, for the action recognition model, each input has three instances; for the anomaly detection model, each input has four instances. The advantage of using a classifier chain is that it maintains the correlation between the output labels of different classifiers.

[0077] By applying the method provided in the embodiments of this application, after obtaining monitoring data for the target user, the monitoring data is preprocessed to obtain a data sequence. The data sequence is then identified using a behavior recognition model to determine the target user's behavior sequence. The data sequence and behavior sequence are input into an anomaly detection model. Based on the output of the anomaly detection model, it can be determined whether the target user exhibits abnormal behavior. The joint detection by the behavior recognition model and the anomaly detection model can promptly detect the user's abnormal behavior and intervene in a timely manner, which helps to delay the progression of the disease and save treatment costs.

[0078] In some embodiments of this application, the behavior recognition model and / or anomaly detection model may include at least one of the following model structures:

[0079] Multi-head convolutional neural network-long short-term memory network model structure;

[0080] Attention mechanism - convolutional neural network - long short-term memory network model structure.

[0081] In the embodiments of this application, the behavior recognition model and / or anomaly detection model can be a multi-headed (MH)-convolutional neural network (CNN)-long short-term memory (LSTM) model structure.

[0082] Alternatively, the behavior recognition model and / or anomaly detection model can be an attention mechanism-convolutional neural network-long short-term memory network model structure.

[0083] Alternatively, behavior recognition models and / or anomaly detection models can be a combination of multi-head convolutional neural network-long short-term memory network model structures and attention mechanism-convolutional neural network-long short-term memory network model structures.

[0084] Convolutional neural networks are particularly effective at automatically extracting and learning features from time-series data, such as... Figure 3 As shown, after different inputs to the convolutional neural network generate feature maps through each convolutional kernel, pooling operations are performed on the feature maps to reduce the amount of data and lower the computational complexity of the model.

[0085] The appropriate model structure can be chosen based on actual needs. Using either of the above two model structures helps improve the accuracy of behavior recognition models and / or anomaly detection models.

[0086] In some embodiments of this application, in the multi-head convolutional neural network-long short-term memory network model structure, multiple convolutional neural networks are connected in parallel. Each convolutional neural network receives and processes the input sequence. The outputs of the multiple convolutional neural networks enter the fully connected layer, and after being merged and interpreted by the fully connected layer, they are input into the long short-term memory network to output the prediction result.

[0087] In this embodiment, the multi-head convolutional neural network-long short-term memory network model structure can include four parts: an input part, a convolutional neural network part, a long short-term memory network part, and an output part.

[0088] The convolutional neural network (CNN) component can include multiple CNNs connected in parallel, such as three CNNs connected in parallel. Each CNN receives and processes the input sequence, allowing the multi-head architecture to read and understand sequence data under different inputs, thus obtaining the outputs of multiple CNNs and extracting obvious behavioral feature vectors. The outputs of multiple CNNs enter a fully connected layer, which merges and interprets the data before feeding it into a Long Short-Term Memory (LSTM) network, which then outputs the prediction result.

[0089] It should be noted that if the behavior recognition model includes a multi-head convolutional neural network-long short-term memory network model structure, then the above input sequence refers to the data sequence, and the prediction result refers to the probability of each behavior being recognized. If the anomaly detection model includes a multi-head convolutional neural network-long short-term memory network model structure, then the above input sequence refers to both the data sequence and the behavior sequence, and the prediction result refers to the probability of an abnormal behavior or the determination of an abnormal behavior.

[0090] The above-described multi-head convolutional neural network-long short-term memory network model structure can improve the accuracy of behavior recognition or abnormal behavior detection.

[0091] In some embodiments of this application, in the attention mechanism-convolutional neural network-long short-term memory network model structure, the convolutional neural network may include a convolutional layer, a pooling layer and a fully connected layer connected in sequence. The convolutional neural network receives the input sequence and extracts features from the input sequence. The long short-term memory network learns the data change pattern based on the features extracted by the convolutional neural network. After attention is applied, the weights of different features are iterated and the output results of the long short-term memory network are re-weighted to output the prediction result.

[0092] In this embodiment, the attention mechanism-convolutional neural network-long short-term memory network model structure comprises five parts: an input part, a convolutional neural network (CNN) part, a long short-term memory network (LSTM) part, an attention part, and an output part, as follows: Figure 4 .

[0093] A convolutional neural network (CNN) consists of convolutional layers, pooling layers, and fully connected layers connected in sequence. The CNN receives an input sequence and extracts features from it. Specifically, features are first extracted through convolutional layers, then convolved with one-dimensional convolutional kernels. Next, pooling layers reduce the dimensionality of the features, and finally, fully connected layers transform the features into a one-dimensional structure, yielding the output features. The Long Short-Term Memory (LSTM) network learns the patterns of data variation based on the features extracted by the previous CNN step. Then, the attention mechanism iteratively assigns weights to different features, re-weighting the output of the LSTM network before finally outputting the prediction result. The LSTM network can include multiple sequentially connected LSTM units. The output of each LSTM unit serves as the input to the attention mechanism, which uses the SoftMax function as its normalization exponential function, and its output is the attention weight α. i .

[0094] It should be noted that if the behavior recognition model includes an attention mechanism-convolutional neural network-long short-term memory network model structure, then the above input sequence refers to the data sequence, and the prediction result refers to the probability of each behavior being recognized. If the anomaly detection model includes an attention mechanism-convolutional neural network-long short-term memory network model structure, then the above input sequence refers to both the data sequence and the behavior sequence, and the prediction result refers to the probability of an abnormal behavior or the determination of an abnormal behavior.

[0095] The above-mentioned attention mechanism-convolutional neural network-long short-term memory network model structure can improve the accuracy of behavior recognition or abnormal behavior detection.

[0096] In some embodiments of this application, preprocessing the monitoring data to obtain a data sequence may include the following steps:

[0097] Step 1: Standardize the monitoring data;

[0098] Step 2: Use a sliding window to segment the standardized monitoring data to obtain a data sequence.

[0099] For ease of description, the two steps above will be explained together.

[0100] In this embodiment of the application, after obtaining monitoring data for the target user, appropriate preprocessing can be performed based on the behavior recognition model and anomaly detection model adopted.

[0101] The monitoring data can be standardized first, for example, using a center scaling method. This is because monitoring data from wearable devices, cameras, and other monitoring equipment have different scales or types, so a center scaling method can be used to rescale the distribution of the values. Here, the monitoring data values ​​are suitable for an appropriate mean and standard deviation (SD).

[0102] The standardization process for monitoring data is as follows:

[0103]

[0104] Among them, the average value

[0105] Standard deviation

[0106] x represents the monitoring data before standardization;

[0107] y represents the standardized monitoring data;

[0108] X i This represents the i-th monitoring data;

[0109] n represents the total number of monitoring data.

[0110] Monitoring data is continuously collected and recorded. After standardization, the standardized data can be segmented using a sliding window to obtain data sequences, which maintain the temporal relationship between the monitoring data. For example... Figure 5 As shown, the monitoring data segmented using a sliding window consists of N matrices, each of size T×I, where T is the length of the time window and I is the input instance, which can be understood as the data within that time window.

[0111] After obtaining monitoring data for the target users, the monitoring data is first standardized, and then the standardized monitoring data is segmented using a sliding window, so that the obtained data sequence can be better adapted as input for behavior recognition model and anomaly detection model.

[0112] In some embodiments of this application, obtaining monitoring data for a target user may include the following steps:

[0113] The first step: Obtain raw monitoring data for the target users;

[0114] The second step is to perform missing value imputation and / or noise reduction on the raw monitoring data to obtain monitoring data for the target user.

[0115] For ease of description, the two steps above will be explained together.

[0116] In this embodiment, wearable devices, cameras, and other monitoring devices can wirelessly transmit the collected raw data to an application server, thereby enabling the application server to obtain the raw monitoring data for the target user. Since the raw monitoring data often contains some noise and data loss may occur during communication, missing value imputation and / or noise reduction processing can be performed on the raw monitoring data before it is determined as the monitoring data for the target user.

[0117] Optionally, missing value imputation can be performed on the original monitoring data. This can be done by representing the missing data with NaN (Not a Number) or "0", thereby making inferences about the missing data.

[0118] Optionally, the raw monitoring data acquired from sensors such as accelerometers, magnetometers, and gyroscopes can be filtered to eliminate noise. Different noise data processing methods can be selected for different types of sensors. For example, for sensors such as accelerometers, inertial filters can be used to filter the data and remove some noise, the principle of which is as follows:

[0119]

[0120]

[0121] Where y(k) is the kth output of the filter;

[0122] x(k) is the k-th input to the filter;

[0123] a is the filter coefficient, a = T / (T f +T);

[0124] T is the time constant of the filtering stage;

[0125] T f The sampling period.

[0126] Typically, the sampling period T f Much smaller than the time constant T of the filtering stage, T and T f The choice of can be determined based on specific circumstances, as long as the filtered signal does not produce obvious ripples.

[0127] After obtaining the raw monitoring data for the target users, the raw monitoring data is first processed by filling in missing values ​​and / or reducing noise, so that the obtained monitoring data is more complete and has better performance.

[0128] In some embodiments of this application, when it is determined that the target user is exhibiting abnormal behavior, the method may further include the following steps:

[0129] Output alarm information, including information on abnormal behavior.

[0130] Optionally, alerts can be sent via SMS, social media, telephone, or notification sounds. The recipients of these alerts can be the target user or their associated users, such as family members or healthcare workers. Alert information can include details of abnormal behavior, such as the specific type of abnormal behavior or the circumstances under which it occurred. This allows the target user or their associated users to be promptly informed of the target user's status and to intervene in a timely manner.

[0131] In some embodiments of this application, the behavior recognition model and the anomaly detection model can be obtained in advance through the following steps:

[0132] Step 1: Obtain the sample dataset. The sample dataset includes multiple sets of monitoring sample data, and each set of monitoring sample data corresponds to a user sample.

[0133] Step 2: Divide the sample dataset into a training dataset and a test dataset;

[0134] Step 3: Train the initial recognition model using the training dataset, and adjust the model parameters of the initial recognition model. The initial recognition model is the initial model of the behavior recognition model.

[0135] Step 4: Train the initial detection model using the training dataset and the output of the initial recognition model, and adjust the model parameters of the initial detection model. The initial detection model is the initial model of the anomaly detection model.

[0136] Step 5: Test the trained initial recognition model and initial detection model using the test dataset. If the test results meet the preset requirements, determine the trained initial recognition model as the behavior recognition model and the trained initial detection model as the anomaly detection model.

[0137] For ease of description, the five steps above will be explained together.

[0138] In this embodiment of the application, a sample dataset can be obtained, which includes multiple sets of monitoring sample data, each set of monitoring sample data corresponding to a user sample. For example, there are ten user samples, each with corresponding monitoring sample data, for a total of ten sets of monitoring sample data.

[0139] The sample dataset may include the original data for user samples after missing value imputation and / or noise reduction, as well as the data after standardization and data segmentation.

[0140] After obtaining the sample dataset, it can be divided into a training dataset and a test dataset, for example, in an 8:2 ratio. The data in the training dataset is used only for training, while the data in the test dataset is not used for training but is used to test the trained model and analyze and evaluate its capabilities based on relevant evaluation metrics.

[0141] An initial model for the behavior recognition model, i.e., the initial recognition model, can be pre-built. The initial recognition model has the same structure as the behavior recognition model. At the same time, an initial model for the anomaly detection model, i.e., the initial detection model, can be pre-built. The initial detection model has the same structure as the anomaly detection model.

[0142] The initial recognition model can be trained using the training dataset. During the training process, the model parameters of the initial recognition model can be adjusted. Then, the initial detection model can be trained using the training dataset and the output of the initial recognition model. During the training process, the model parameters of the initial detection model can be adjusted.

[0143] Training can be stopped once the set threshold number of training iterations is reached or the model converges to a preset condition, thus obtaining the initial recognition model and initial detection model after training.

[0144] The trained initial recognition and detection models are tested using a test dataset. If the test results meet preset requirements, the trained initial recognition model can be designated as the behavior recognition model, and the trained initial detection model as the anomaly detection model. If the test results do not meet the preset requirements, the initial recognition and detection models can be trained again, for example, by increasing the training dataset. Preset requirements can include accuracy requirements, precision requirements, recall requirements, etc.

[0145] One possible model acquisition process is as follows: Figure 6 As shown, after obtaining the training dataset, a multi-head model for Human Activity Recognition (HAR) can be trained using the training dataset. After training, a multi-head model for HAR is obtained, forming the first process. The multi-head model for HAR is then applied to the training dataset to obtain its output. This output, along with the training dataset, is used to train a multi-head model for Abnormal Activity Recognition (AAR). After training, a multi-head model for AAR is obtained, forming the second process. This forms a classifier chain, with each process consisting of a multi-head model, each with its own input and output. These models improve the accuracy and generalization ability of the classifier by learning different features and patterns.

[0146] The above process yields behavior recognition and anomaly detection models. Applying these models to real-world scenarios enables human behavior recognition and abnormal behavior detection.

[0147] This application's embodiments utilize the linkage of behavior recognition models and anomaly detection models to detect abnormal behavior, which can reduce false alarm rates, improve reliability, and is more suitable for monitoring the health and behavior of the elderly. Furthermore, a remote behavior monitoring system employing non-invasive wearable devices, deep learning information technology, and broadcast notifications can provide an efficient, reliable, and low-cost solution for assessing users' daily behavior.

[0148] For ease of understanding, the technical solutions provided in the embodiments of this application will be described again below with specific examples.

[0149] 1) First, data collection and acquisition are carried out, and the data is preprocessed.

[0150] 1.1) The data in this example comes from 10 volunteers aged 60. The 10 volunteers performed 9 daily actions and 3 different types of fall actions. The sensors collected the data at a rate of 20Hz. The data was generated by the accelerometer every 50ms. Combined with the human behavior posture dataset WISDM, the behavior of approximately 24,000 to 36,000 sample users can be obtained.

[0151] 1.2) The sample data is preprocessed. After filling missing values ​​and reducing noise, the center scaling method is used to obtain the standardized data.

[0152] 1.3) The sample dataset is split into training and testing datasets in an 8:2 ratio, as shown in Table 1:

[0153] Table 1

[0154]

[0155]

[0156] 2) This example primarily employs the MH-CNN-LSTM model architecture, which can be built using Keras (an open-source artificial neural network library). Keras is an efficient application programming interface (API) for deep neural network architectures. The MH model is implemented as a supervised model. This work uses the cross-entropy loss function, which calculates the error between the predicted and true labels. The gradient-based algorithm Adam is used for optimization, and the learning rate for each individual model can be determined based on the parameters. In this example, a sliding window method is used to segment the sensor data. The window size is 200, the stride is 20, and other parameters are shown in Table 2.

[0157] Table 2

[0158]

[0159] By training the model using the training dataset and testing the model using the test dataset, we can obtain the behavior recognition model and the anomaly detection model.

[0160] After 15 iterations, the conventional CNN-LSTM model and the MH-CNN-LSTM model were tested and compared. The accuracy and loss values ​​of the behavior recognition were obtained as follows: Figure 7 and Figure 8 As shown. For anomaly detection, both the MH-CNN-LSTM model and the regular CNN-LSTM model are used. The model's result indicates whether the detected behavior is normal or abnormal. The F1-score (a weighted harmonic average of model precision and recall) is shown below. Figure 9and Figure 10 As shown, the MH-CNN-LSTM model demonstrates high accuracy and precision in both action recognition and anomaly detection.

[0161] like Figure 11 As shown, the entire abnormal behavior detection process can include a raw data layer, an observation state layer, and an anomaly detection layer. Raw monitoring data for the target user can be collected using sensors such as accelerometers, gyroscopes, and azimuth sensors. The indoor location of the target user, such as in the bedroom, bathroom, living room, dining room, or kitchen, can be determined using an indoor positioning system.

[0162] After imputing missing values ​​and / or reducing noise in the raw monitoring data, monitoring data for the target user can be obtained. Preprocessing the monitoring data yields a data sequence. The MH-CNN-LSTM algorithm can then be used to perform action recognition on the data sequence, resulting in real-time action data.

[0163] Movements can be categorized into static movements, dynamic movements, and falling movements. Static movements can include sitting, standing, and lying down; dynamic movements can include walking, running, sitting down, standing up, going upstairs, and going downstairs; and falling movements can include falling forward, falling backward, and falling sideways.

[0164] By observing real-time actions and their locations, the state can be determined to identify any specific behavioral abnormalities. If such abnormalities are found, guardians or medical personnel are notified. If no specific behavioral abnormalities are found, a behavioral sequence, or behavioral activity chain, can be formed. The MH-CNN-LSTM algorithm can then be used to detect abnormal behaviors in the elderly. If the sequence of behaviors is abnormal, such as repeatedly eating or forgetting to eat, guardians or medical personnel are notified, and the notification message can include a description of the abnormality. This allows for continuous monitoring of the elderly's daily activities in their everyday living environments.

[0165] For example, the abnormal behaviors identified through behavioral sequences are shown in Table 3:

[0166] Table 3

[0167]

[0168]

[0169] In summary, this application's embodiments introduce deep learning technology and integrate multiple sensors for multi-level sequential abnormal behavior recognition, achieving higher accuracy than traditional machine learning methods. It also provides a low-cost, high-efficiency solution for real-time monitoring and abnormal behavior detection in the daily lives of the elderly. This addresses the issue of Alzheimer's disease only being diagnosed in high-cost clinical settings, enabling early identification of Alzheimer's patients and reducing medical costs for the elderly. It achieves remote real-time monitoring and abnormal behavior recognition of the elderly in daily life; proposes a deep learning-based method to identify abnormal behaviors and their sequence in the elderly; and focuses more on the abnormality of behavioral sequence compared to existing technologies, resulting in more accurate identification of abnormal behaviors in the elderly and improved anomaly recognition accuracy.

[0170] Corresponding to the above method embodiments, this application also provides an abnormal behavior detection device. The abnormal behavior detection device described below can be referred to in correspondence with the abnormal behavior detection method described above.

[0171] See Figure 12 As shown, the abnormal behavior detection device 1200 may include the following modules:

[0172] The first acquisition module 1210 is used to acquire monitoring data for the target user;

[0173] The second acquisition module 1220 is used to preprocess the monitoring data to obtain a data sequence;

[0174] The first determining module 1230 is used to input the data sequence into the behavior recognition model and determine the behavior sequence of the target user based on the output of the behavior recognition model.

[0175] The second determining module 1240 is used to input the data sequence and behavior sequence into the anomaly detection model, and determine whether the target user has abnormal behavior based on the output of the anomaly detection model.

[0176] By applying the apparatus provided in the embodiments of this application, after obtaining monitoring data for a target user, the monitoring data is preprocessed to obtain a data sequence. The data sequence is then identified using a behavior recognition model to determine the target user's behavior sequence. The data sequence and behavior sequence are input into an anomaly detection model. Based on the output of the anomaly detection model, it can be determined whether the target user exhibits abnormal behavior. Through the joint detection of the behavior recognition model and the anomaly detection model, abnormal user behavior can be detected in a timely manner, allowing for timely intervention, which helps to delay the progression of the disease and save treatment costs.

[0177] In some embodiments of this application, the behavior recognition model and / or anomaly detection model includes at least one of the following model structures:

[0178] Multi-head convolutional neural network-long short-term memory network model structure;

[0179] Attention mechanism - convolutional neural network - long short-term memory network model structure.

[0180] In some embodiments of this application, in the multi-head-convolutional neural network-long short-term memory network model structure, multiple convolutional neural networks are connected in parallel. Each convolutional neural network receives and processes the input sequence. The outputs of the multiple convolutional neural networks enter the fully connected layer, which merges and interprets the data before inputting it into the long short-term memory network to output the prediction result.

[0181] In some embodiments of this application, in the attention mechanism-convolutional neural network-long short-term memory network model structure, the convolutional neural network includes a convolutional layer, a pooling layer and a fully connected layer connected in sequence. The convolutional neural network receives the input sequence and extracts features from the input sequence. The long short-term memory network learns the data change pattern based on the features extracted by the convolutional neural network. After the attention part iterates to obtain the weights of different features, the output result of the long short-term memory network is re-weighted and the prediction result is output.

[0182] In some embodiments of this application, the second obtaining module 1220 is specifically used for:

[0183] Standardize the monitoring data;

[0184] The standardized monitoring data is segmented using a sliding window to obtain a data sequence.

[0185] In some embodiments of this application, the first obtaining module 1210 is specifically used for:

[0186] Obtain raw monitoring data for the target users;

[0187] The raw monitoring data is imputed for missing values ​​and / or denoised to obtain monitoring data for the target user.

[0188] In some embodiments of this application, the abnormal behavior detection device 1200 further includes a third acquisition module, used to pre-obtain a behavior recognition model and an anomaly detection model through the following steps:

[0189] Obtain a sample dataset, which includes multiple sets of monitoring sample data, each set of monitoring sample data corresponding to a user sample;

[0190] The sample dataset is divided into a training dataset and a test dataset;

[0191] The initial recognition model is trained using the training dataset, and the model parameters of the initial recognition model are adjusted. The initial recognition model is the initial model of the behavior recognition model.

[0192] The initial detection model is trained using the training dataset and the output of the initial recognition model. The model parameters of the initial detection model are adjusted. The initial detection model is the initial model of the anomaly detection model.

[0193] The initial recognition model and the initial detection model after training were tested using a test dataset. If the test results met the preset requirements, the initial recognition model after training was determined as the behavior recognition model and the initial detection model after training was determined as the anomaly detection model.

[0194] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0195] Corresponding to the above method embodiments, this application also provides an electronic device, including:

[0196] Memory, used to store computer programs;

[0197] A processor is used to implement the steps of the above-described abnormal behavior detection method when executing a computer program.

[0198] like Figure 13 The diagram shows the structural composition of an electronic device, which may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0199] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0200] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiment of the abnormal behavior detection method.

[0201] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0202] Obtain monitoring data for the target users;

[0203] The monitoring data is preprocessed to obtain a data sequence;

[0204] The data sequence is input into the behavior recognition model, and the behavior sequence of the target user is determined based on the output of the behavior recognition model.

[0205] The data sequence and behavior sequence are input into the anomaly detection model, and the output of the anomaly detection model is used to determine whether the target user has abnormal behavior.

[0206] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0207] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0208] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0209] Of course, it should be noted that, Figure 13 The structure shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than Figure 13 More or fewer components as shown, or combinations of certain components.

[0210] Corresponding to the above method embodiments, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described abnormal behavior detection method.

[0211] Furthermore, it should be noted that this application also provides a computer program product or computer program, which may include computer instructions that can be stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the abnormal behavior detection method described in the preceding embodiments. Therefore, this will not be repeated here. Additionally, the beneficial effects of using the same method will not be repeated here either. For technical details not disclosed in the embodiments of the computer program product or computer program involved in this application, please refer to the description of the method embodiments of this application.

[0212] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0213] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0214] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0215] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for detecting abnormal behavior, characterized in that, include: Obtain monitoring data for the target users; The monitoring data is preprocessed to obtain a data sequence; The data sequence is input into the behavior recognition model, and the behavior sequence of the target user is determined based on the output of the behavior recognition model. The data sequence and the behavior sequence are input into the anomaly detection model. Based on the output of the anomaly detection model, it is determined whether the target user exhibits abnormal behavior. The anomaly detection model performs anomaly detection based on the order of behaviors in the behavior sequence and the specific behaviors in the behavior sequence. The behavior recognition model is the first classifier, and the anomaly detection model is the second classifier. The preprocessing of the monitoring data to obtain a data sequence includes: The monitoring data is then standardized. The standardized monitoring data is segmented using a sliding window to obtain a data sequence; The behavior recognition model and / or the anomaly detection model includes at least one of the following model structures: Multi-head convolutional neural network-long short-term memory network model structure; Attention mechanism - Convolutional neural network - Long Short-Term Memory network model structure; The anomaly detection model performs anomaly detection based on the order of behaviors in the behavior sequence and the specific behaviors in the behavior sequence, including: The anomaly detection model detects whether the order of behaviors in the behavior sequence is normal and whether the specific behaviors in the behavior sequence are normal.

2. The method according to claim 1, characterized in that, In the multi-head-convolutional neural network-long short-term memory network model structure, multiple convolutional neural networks are connected in parallel. Each convolutional neural network receives and processes the input sequence. The outputs of multiple convolutional neural networks enter the fully connected layer, which merges and interprets the data before inputting it into the long short-term memory network to output the prediction result.

3. The method according to claim 1, characterized in that, In the attention mechanism-convolutional neural network-long short-term memory network model structure, the convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer connected in sequence. The convolutional neural network receives the input sequence and extracts features from the input sequence. The long short-term memory network learns the data change pattern based on the features extracted by the convolutional neural network. After the attention part iterates to obtain the weights of different features, the output results of the long short-term memory network are re-weighted and the prediction results are output.

4. The method according to claim 1, characterized in that, The acquisition of monitoring data for the target user includes: Obtain raw monitoring data for the target users; The raw monitoring data is processed by filling in missing values ​​and / or reducing noise to obtain monitoring data for the target user.

5. The method according to any one of claims 1 to 4, characterized in that, The behavior recognition model and the anomaly detection model are obtained in advance through the following steps: Obtain a sample dataset, which includes multiple sets of monitoring sample data, each set of monitoring sample data corresponding to a user sample; The sample dataset is divided into a training dataset and a test dataset; The initial recognition model is trained using the training dataset, and the model parameters of the initial recognition model are adjusted. The initial recognition model is the initial model of the behavior recognition model. The initial detection model is trained using the training dataset and the output of the initial recognition model, and the model parameters of the initial detection model are adjusted. The initial detection model is the initial model of the anomaly detection model. The trained initial recognition model and initial detection model are tested using the test dataset. If the test results meet the preset requirements, the trained initial recognition model is determined as the behavior recognition model, and the trained initial detection model is determined as the anomaly detection model.

6. An abnormal behavior detection device, characterized in that, include: The first acquisition module is used to acquire monitoring data for the target user; The second acquisition module is used to preprocess the monitoring data to obtain a data sequence; The first determining module is used to input the data sequence into the behavior recognition model and determine the behavior sequence of the target user based on the output of the behavior recognition model. The second determining module is used to input the data sequence and the behavior sequence into the anomaly detection model, and determine whether the target user has abnormal behavior based on the output of the anomaly detection model; wherein, the anomaly detection model performs anomaly detection based on the behavior sequence order and specific behavior in the behavior sequence, the behavior recognition model is the first classifier, and the anomaly detection model is the second classifier; The second acquisition module is also used for: Standardize the monitoring data; The standardized monitoring data is segmented using a sliding window to obtain a data sequence; The behavior recognition model and / or anomaly detection model includes at least one of the following model structures: Multi-head convolutional neural network-long short-term memory network model structure; Attention mechanism - Convolutional neural network - Long Short-Term Memory network model structure; The anomaly detection model performs anomaly detection based on the order of behaviors in the behavior sequence and the specific behaviors in the behavior sequence, including: The anomaly detection model detects whether the order of behaviors in the behavior sequence is normal and whether the specific behaviors in the behavior sequence are normal.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the abnormal behavior detection method as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the abnormal behavior detection method as described in any one of claims 1 to 5.

Images