Abnormal behavior detection method and device, electronic equipment and storage medium

Abstract

The invention discloses an abnormal behavior detection method and device, electronic equipment and a storage medium, and is used for solving the problem that an existing abnormal behavior detection method is relatively high in false alarm rate and missing report rate. Comprising the following steps: acquiring position information and centroid height information of a target object acquired by a radar, and initial time information and residence time information of the target object reaching the position; inputting the position information and the centroid height information of the target object and the initial time information of the target object reaching the position into a time prediction model to obtain predicted residence time information of the target object at the position, the time prediction model is obtained by training a behavior sample of the target object according to a preset neural network model; and obtaining an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information.

Description

Technical Field

[0001] The present application relates to the field of radar monitoring technology, and in particular to an abnormal behavior detection method, device, electronic device and storage medium. Background Art

[0002] At present, the number of elderly people in China is gradually increasing. As they age, their physical functions decline, and the probability of accidents (such as falls) increases greatly, which greatly endangers the health and lives of the elderly. Immediate detection and warning of abnormal events will increase the chances of saving the lives of the elderly. In this context, millimeter-wave radars have begun to be widely used in indoor care scenarios.

[0003] However, the current abnormal behavior detection method based on millimeter-wave radar generally adopts target-based acceleration motion, simply determines abnormal events based on fall behavior, and attempts to use laboratory fall feature data to match real abnormal events in actual scenarios. In actual indoor care scenarios, firstly, there is a big gap between fall behavior and laboratory data, and it is difficult to accurately capture abnormal events in actual scene applications. Secondly, indoor abnormal events are not simply fall behaviors. Only by capturing fall behaviors can only cover a small part of indoor abnormal behaviors. Therefore, existing methods have defects in the accuracy and breadth of abnormal event detection, resulting in high false alarm and missed alarm rates. Summary of the invention

[0004] In order to solve the problem of high false alarm rate and missed alarm rate in existing abnormal behavior detection methods, the embodiments of the present application provide an abnormal behavior detection method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present application provides an abnormal behavior detection method, comprising:

[0006] Acquire the location information, centroid height information, initial time information of the target object arriving at the location, and residence time information collected by the radar;

[0007] Inputting the location information, the centroid height information and the initial time information of the target object arriving at the location into a time prediction model to obtain the predicted residence time information of the target object at the location, wherein the time prediction model is obtained by training the behavior samples of the target object according to a preset neural network model;

[0008] An abnormal behavior detection result is obtained according to the difference between the residence time information and the predicted residence time information.

[0009] In one embodiment, before inputting the position information, the centroid height information and the initial time information of the target object arriving at the position into the time prediction model, the method further includes:

[0010] Determine the target area to which the position of the target object belongs according to the position information of each pre-marked area and the position information of the target object;

[0011] Acquire historical centroid height information of the target object when it moves normally in the target area;

[0012] The centroid height information and the historical centroid height information are clustered to obtain a clustering result.

[0013] In one implementation, obtaining an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information specifically includes:

[0014] An abnormal behavior detection result is obtained according to the difference between the residence time information and the predicted residence time information and the clustering result.

[0015] In one implementation, obtaining an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information and the clustering result specifically includes:

[0016] Determining whether the centroid height information is abnormal according to the clustering result;

[0017] determining whether the residence time information is abnormal according to a difference between the residence time information and the predicted residence time information;

[0018] If it is determined that the centroid height information is normal and the residence time information is normal, or if it is determined that the centroid height information is abnormal and the residence time information is normal, then it is determined that the behavior of the target object is normal;

[0019] If it is determined that the centroid height information is abnormal and the residence time information is abnormal, or if it is determined that the centroid height information is normal and the residence time information is abnormal, it is determined that the behavior of the target object is abnormal.

[0020] In one implementation, determining whether the residence time information is abnormal according to the difference between the residence time information and the predicted residence time information specifically includes:

[0021] If it is determined that the difference between the residence time information and the predicted residence time information is less than a preset threshold, then it is determined that the residence time information is normal;

[0022] If it is determined that the difference between the residence time information and the predicted residence time information is greater than or equal to the preset threshold, it is determined that the residence time information is abnormal.

[0023] In one embodiment, the time prediction model is obtained by training in the following manner:

[0024] Acquire a behavior sample of the target object's activities in each pre-marked area collected by the radar, wherein the behavior sample is the stay information of the target object at different positions in each pre-marked area, and the stay information includes the stay position information of the target object in each pre-marked area, the centroid height information of the target object at the stay position, the initial time information of the target object arriving at the stay position, and the stay time information of the target object at the stay position;

[0025] For each behavior sample of the target object, input the target object's stay position information, the target object's center of mass height information at the stay position, and the target object's initial time information when it arrives at the stay position into the preset neural network model to obtain the target object's predicted stay time information at the stay position;

[0026] Determining a prediction error based on the predicted stay time information of the target object at the stay location and the stay time information of the target object at the stay location;

[0027] According to the prediction error, various parameters included in the preset neural network model are adjusted until the model converges to obtain the trained time prediction model.

[0028] In a second aspect, an embodiment of the present application provides an abnormal behavior detection device, including:

[0029] A first acquisition module is used to acquire the position information, centroid height information, initial time information of the target object arriving at the position and residence time information of the target object collected by the radar;

[0030] A prediction module, used to input the location information, centroid height information and initial time information of the target object arriving at the location into a time prediction model to obtain the predicted residence time information of the target object at the location, wherein the time prediction model is obtained by training the behavior samples of the target object according to a preset neural network model;

[0031] The detection module is used to obtain an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information.

[0032] In one embodiment, the device further comprises:

[0033] a determination module, used for determining the target area to which the position of the target object belongs based on the position information of each pre-marked area and the position information of the target object before inputting the position information, the centroid height information and the initial time information of the target object arriving at the position into the time prediction model;

[0034] A second acquisition module is used to acquire historical centroid height information of the target object when it moves normally in the target area;

[0035] The clustering module is used to cluster the centroid height information and the historical centroid height information to obtain a clustering result.

[0036] In one implementation, the detection module is specifically configured to obtain an abnormal behavior detection result based on a difference between the residence time information and the predicted residence time information and the clustering result.

[0037] In one embodiment, the detection module is specifically used to determine whether the centroid height information is abnormal based on the clustering result; determine whether the residence time information is abnormal based on the difference between the residence time information and the predicted residence time information; if it is determined that the centroid height information is normal and the residence time information is normal, or, if it is determined that the centroid height information is abnormal and the residence time information is normal, then determine that the behavior of the target object is normal; if it is determined that the centroid height information is abnormal and the residence time information is abnormal, or, if it is determined that the centroid height information is normal and the residence time information is abnormal, then determine that the behavior of the target object is abnormal.

[0038] In one embodiment, the detection module is specifically used to determine that the residence time information is normal if it is determined that the difference between the residence time information and the predicted residence time information is less than a preset threshold; if it is determined that the difference between the residence time information and the predicted residence time information is greater than or equal to the preset threshold, then determine that the residence time information is abnormal.

[0039] In one embodiment, the prediction module is specifically used to train and obtain the time prediction model in the following manner: obtaining behavior samples of the target object's activities in pre-marked areas collected by radar, the behavior samples being the stay information of the target object at different positions in the pre-marked areas, the stay information including the stay position information of the target object in the pre-marked areas, the center of mass height information of the target object at the stay position, the initial time information of the target object arriving at the stay position, and the stay time information of the target object at the stay position; for each behavior sample of the target object, the stay position information of the target object, the center of mass height information of the target object at the stay position, and the initial time information of the target object arriving at the stay position are input into the preset neural network model to obtain the predicted stay time information of the target object at the stay position; determining the prediction error based on the predicted stay time information of the target object at the stay position and the stay time information of the target object at the stay position; adjusting the various parameters contained in the preset neural network model based on the prediction error until the model converges to obtain the trained time prediction model.

[0040] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the abnormal behavior detection method described in the present application when executing the program.

[0041] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps in the abnormal behavior detection method described in the present application are implemented.

[0042] The beneficial effects of this application are as follows:

[0043] The abnormal behavior detection method, device, electronic device and storage medium provided in the embodiments of the present application obtain the position information, center of mass height information, initial time information and residence time information of the target object collected by radar, input the position information, center of mass height information and initial time information of the target object arriving at the position into a time prediction model, and obtain the predicted residence time information of the target object at the position. The time prediction model is obtained by training the behavior samples of the target object according to a preset neural network model, and then, according to the difference between the residence time information and the predicted residence time information, the abnormal behavior detection result is obtained. In the embodiment of the present application, the position information, centroid height information, initial time information of the target object arriving at the position and residence time information collected by the radar are obtained, and the residence time is the length of time the target object has currently stayed at the position. The position information, centroid height information and initial time information of the target object arriving at the position are input into a time prediction model obtained in advance based on the behavior sample of the target object and the preset neural network model training, and the initial time of the target object arriving at the position and the normal residence time corresponding to maintaining the centroid height at the position are predicted, wherein the time prediction model is used to predict the residence time of the target object at a certain position, and the target object's residence time is obtained according to the difference between the actual residence time of the target object at the position and the predicted residence time. Abnormal behavior detection results. Compared with the prior art, the present application jointly judges the abnormal behavior of the target object based on the three dimensions of the target object's position, center of mass height and residence time, comprehensively considers the spatial characteristics and time characteristics, and predicts the target object's residence time based on the time prediction model pre-trained based on the target object's behavior samples. The time prediction model trained according to the target object's behavior habits can more accurately predict the target object's residence time based on the target object's current position, the target object's initial time when it arrives at the position, and the center of mass height maintained at the position. Therefore, based on the difference between the predicted residence time and the actual residence time of the target object, it is possible to more accurately judge whether the target object's behavior is abnormal, thereby improving the accuracy of abnormal event detection and reducing the false alarm rate and missed alarm rate.

[0044] Other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 A schematic diagram of an application scenario of the abnormal behavior detection method provided in an embodiment of the present application;

[0047] Figure 2 A flowchart of an abnormal behavior detection method provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a process for clustering the current centroid height information and the historical centroid height information of a target object provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the training process of the time prediction model provided in the embodiment of the present application;

[0050] Figure 5 A schematic diagram of a process for obtaining abnormal behavior detection results provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of an abnormal behavior detection device provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to solve the problem of high false alarm rate and missed alarm rate in existing abnormal behavior detection methods, the embodiments of the present application provide an abnormal behavior detection method, device, electronic device and storage medium.

[0054] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0055] First reference Figure 1, which is a schematic diagram of an application scenario of the abnormal behavior detection method provided in an embodiment of the present application, and may include a radar 101 and a server 102. The radar 101 may monitor the target object in real time, collect the target object's location information, centroid height information, initial time information of the target object arriving at the location, and residence time information, and upload them to the server 102. The server 102 inputs the target object's location information, centroid height information, and initial time information of the target object arriving at the location into a time prediction model to obtain the predicted residence time information of the target object at the location. The time prediction model is obtained by training the behavior sample of the target object according to a preset neural network model. The server 102 obtains the abnormal behavior detection result according to the difference between the current target object's residence time information at the location and the predicted residence time information, thereby improving the accuracy of abnormal event detection and reducing the false alarm rate and missed alarm rate.

[0056] The server 102 may be an independent physical server or a cloud server that provides basic cloud computing services such as cloud servers, cloud databases, and cloud storage. The radar 101 may be, but is not limited to, a millimeter wave radar. The server 102 and the radar 101 may be connected via a network, which is not limited in the present embodiment.

[0057] Based on the above application scenarios, the following will refer to the attached Figures 2 to 5 The exemplary embodiments of the present application are described in more detail. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in any way. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0058] like Figure 2 As shown, it is a schematic diagram of the implementation process of the abnormal behavior detection method provided in the embodiment of the present application. The abnormal behavior detection method can be applied to the above-mentioned server 102, and specifically may include the following steps:

[0059] S21, obtaining the position information, centroid height information, initial time information of the target object arriving at the position, and residence time information collected by the radar.

[0060] During specific implementation, the target object is located in an indoor environment, and the radar is installed in the indoor environment where the target object is located. The radar can collect point cloud data of the target object in the indoor environment in real time, and send the collected point cloud data of the target object in the indoor environment to the server.

[0061] The server conducts indoor household survey in advance, and obtains the location information of the target object, that is, the location coordinates of the target object, according to the deployment position of the radar, the distance of the target object from the radar, and the angle information of the target object. The server obtains the center of mass height information of the target object from the point cloud data of the target object in the indoor environment, as well as the initial time information and residence time information of the target object arriving at the current position collected by the radar, wherein the residence time of the target object at the current position refers to the length of time the target object has stayed at the position from the time it arrived at the position to the current time. The center of mass height of the target object can reflect the current posture of the target object, such as standing, sitting, squatting, lying, etc. The target object is a monitored person, such as an elderly person living alone, etc., which is not limited to this in the embodiments of the present application.

[0062] In order to improve the accuracy of abnormal behavior detection, in one embodiment, after obtaining the current position information, center of mass height information, initial time information and residence time information of the target object at the current position, the server can also cluster the center of mass height information of the target object with the historical center of mass height information of the target object when it normally moves in the target area to which the position belongs, so as to determine whether the current center of mass height information of the target object is abnormal.

[0063] When implementing it specifically, you can follow the following steps: Figure 3 The process shown in the figure performs clustering processing on the current centroid height information and the historical centroid height information of the target object, including the following steps:

[0064] S31 . Determine the target area to which the position of the target object belongs based on the position information of each pre-marked area and the position information of the target object.

[0065] In specific implementation, the server queries the position coordinate range of the area where the position coordinates of the target object are located according to the position coordinate ranges of the pre-marked areas and the position coordinates of the target object, and obtains the target area to which the position of the target object belongs.

[0066] S32. Obtain historical centroid height information of the target object when it moves normally in the target area.

[0067] During specific implementation, the server records the historical centroid height information of the target object each time it moves normally in different areas, and the server obtains the historical centroid height information of the target object when it moves normally in the target area.

[0068] S33, clustering the centroid height information and the historical centroid height information to obtain a clustering result.

[0069] In specific implementation, the server may use a preset clustering algorithm to cluster the current centroid height information of the target object with the historical centroid height information of the target object to obtain a clustering result. The preset clustering algorithm may be, but is not limited to, a K-means clustering algorithm or DBSCAN (Density-Based Spatial Clustering of Applications with Noise), and any other clustering algorithm may also be used, which is not limited in the present embodiment of the application.

[0070] In an embodiment of the present application, by clustering the current center of mass height information of the target object with the historical center of mass height information of the target object when it is acting normally, it is possible to determine whether the center of mass height information of the target object in the area to which the current position belongs is abnormal by determining whether the current center of mass height information of the target object is far away from the cluster center. Thus, combined with the difference between the actual residence time information of the target object at the current position and the predicted residence time information, it is possible to more accurately determine whether the current behavior of the target object is abnormal.

[0071] S22: Input the location information, centroid height information and initial time information of the target object arriving at the location into a time prediction model to obtain the predicted stay time information of the target object at the location.

[0072] In specific implementation, the server pre-trains a time prediction model, which is obtained by training the behavior samples of the target object according to a preset neural network model. The time prediction model is used to predict the normal stay time information of the target object at the location based on the location information, center of mass height information and the initial time information of the target object arriving at the location.

[0073] Specifically, you can follow the following Figure 4 The process shown in the figure trains the time prediction model, including the following steps:

[0074] S41, obtaining the behavior samples of the target object's activities in each pre-marked area collected by the radar.

[0075] In specific implementation, the server pre-marks different areas of the room where the target object is located, such as: bedroom (master bedroom, second bedroom, etc.), living room, kitchen, bathroom, etc. The bedroom may include a bed, an open area, etc., the living room may include a sofa area, an open area, etc., and the bathroom may include a toilet area, a washing area, a shower area, etc., which is not limited in the embodiment of the present application. After marking each area in the room, the server records the location information of each area, that is, records the location coordinates of each area.

[0076] The server receives behavior samples of the target object's activities in pre-marked areas collected by the radar. The behavior samples of the target object's activities in the pre-marked areas are the stay information of the target object at different positions in the pre-marked areas. The stay information includes the stay position information of the target object in the pre-marked areas, the center of mass height information of the target object at the stay position, the initial time information of the target object arriving at the stay position, and the stay time information of the target object at the stay position.

[0077] Specifically, the radar can collect behavioral samples of the target object's activities in pre-marked areas 24 hours a day over multiple days in history, that is, collect the position changes of the target object in each pre-marked area indoors 24 hours a day over multiple days in history, the center of mass height changes, and the initial time information and residence time information of the target object arriving at each stop position.

[0078] The server uses the behavior samples of the target object collected by the radar in pre-marked areas 24 hours a day over multiple days in history as training samples. In this way, the time prediction model trained based on the historical behavior samples of the target object is more in line with the behavioral habits of the target object, making the residence time predicted by the model more accurate.

[0079] S42. For each behavior sample of the target object, the target object's stop position information, the target object's center of mass height information at the stop position, and the initial time information of the target object arriving at the stop position are input into a preset neural network model to obtain the predicted stop time information of the target object at the stop position.

[0080] In specific implementation, for each behavior sample of the target object, the server inputs the target object's stop location information, the target object's centroid height information at the stop location, and the target object's initial time information when it arrives at the stop location into a preset neural network model to obtain the target object's predicted stop time information at the stop location. The preset neural network model may be, but is not limited to, an LSTM (Long Short-Term Memory) model or a CNN (Convolutional Neural Networks) model, and any other neural network model may also be used, which is not limited in the present application embodiment.

[0081] S43: Determine a prediction error according to the predicted stay time information of the target object at the stay location and the stay time information of the target object at the stay location.

[0082] During specific implementation, the server may use a preset loss function to determine the prediction error between the predicted residence time information of the target object at the residence location and the actual residence time information of the target object at the residence location, wherein the preset loss function may be, but is not limited to, a mean square error (MSE) function, and may also use any other loss function, which is not limited to the embodiments of the present application.

[0083] S44. Adjust various parameters included in the preset neural network model according to the prediction error until the model converges to obtain a trained time prediction model.

[0084] In specific implementation, the server adjusts various parameters contained in the preset neural network model to minimize the loss function according to the prediction error between the predicted stay time information of the target object at the stay location corresponding to each behavior sample of the target object and the actual stay time information of the target object at the stay location until the model converges to obtain the trained time prediction model.

[0085] The present application embodiment takes the LSTM model as an example to illustrate the training process of the time prediction model:

[0086] The LSTM model in the embodiment of the present application may include: an input layer, at least two LSTM layers and an output layer (i.e., a fully connected layer). Taking two LSTM layers as an example, the first LSTM layer may include 128 units and the second LSTM layer may include 64 units to capture complex patterns and long-term and short-term dependencies in time series data.

[0087] After the server obtains the behavior samples of the target object's activities in each pre-marked area, it extracts the target object's stop location information, the target object's centroid height information at the stop location, the target object's initial time information when it arrives at the stop location (specifically accurate to: year, month, day, hour, minute), and the target object's stay time information at the stop location from each behavior sample. The server standardizes the three feature information extracted from each behavior sample, namely, the target object's stop location information, the target object's centroid height information at the stop location, and the target object's initial time information when it arrives at the stop location, so that each feature information is on a similar scale, thereby improving the numerical stability of the model training. Then, the above three standardized feature information are input into the input layer of the LSTM model. The dimensions of the input data include time steps (Timesteps) and feature data (Number of Features), etc. When the three feature information reaches the LSTM layer, the current hidden state is obtained. The formula is described as follows:

[0088] h t =LSTM(x t ,ht-1 ,c t-1 )

[0089] Among them, h t represents the current hidden state, x t represents the input data, h t-1 represents the hidden state of the previous time step, c t-1 Represents the state of the memory cell at the previous time step.

[0090] Furthermore, the predicted residence time information of the target object at the residence location is output through the output layer (ie, the fully connected layer).

[0091] When training the LSTM model, the mean square error function can be used as the loss function to measure the prediction error between the predicted residence time information of the target object at the residence location and the actual residence time information of the target object at the residence location. The formula of the mean square error function is as follows:

[0092]

[0093] Among them, y i is the actual stay time information of the target object at the i-th stop location, is the predicted residence time information of the target object at the i-th residence position, i=1, 2, ..., n, where n is the number of behavior samples of the target object.

[0094] The server can, but is not limited to, use the Adam optimizer to update the LSTM model parameters. The Adam optimizer is an adaptive learning rate optimization algorithm that can accelerate the model convergence speed. Training is performed based on the historical behavior sample data of the target object, and the LSTM model parameters are continuously adjusted through the back propagation algorithm to minimize the loss function until the model converges, thereby obtaining a trained time prediction model.

[0095] The probability prediction during model training is as follows:

[0096] For each set of input data, including the standardized target object's stop position, the target object's centroid height information at the stop position, and the target object's initial time information at the stop position, multiple predictions are performed to generate multiple predicted stop time samples. Assuming that N simulations are performed, the result of each simulation is Among them, x is the input data, θ i is the parameter of the model in the i-th simulation. Then, statistical analysis is performed on the generated multiple predicted residence time samples to calculate the probability distribution of the predicted residence time. The distribution characteristics can be described by calculating the mean and standard deviation of the predicted residence time samples:

[0097]

[0098] Among them, μ represents the sample mean of the predicted residence time, σ 2 represents the sample variance of the predicted residence time.

[0099] Based on the distribution of the predicted stay time samples, a frequency histogram can be drawn or a probability distribution, such as a normal distribution, can be fitted. The frequency histogram can show the frequency of occurrence of each stay time interval, thereby obtaining the probability of different predicted stay times.

[0100] Specifically, the predicted residence time samples are segmented, the number of predicted residence time samples in each residence time interval is counted, and the number of predicted residence time samples in each residence time interval is divided by the total number of predicted residence time samples to obtain the probability of the predicted residence time samples in each residence time interval.

[0101] During implementation, the server inputs the acquired current position information, current centroid height information and initial time information of the target object arriving at the current position into the trained time prediction model to obtain the predicted residence time information of the target object at the current position.

[0102] S23. Obtain an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information.

[0103] During specific implementation, the server obtains the abnormal behavior detection result according to the difference between the residence time information at the current location and the predicted residence time information, as well as the clustering result.

[0104] Specifically, you can follow the following Figure 5 The process shown in the figure obtains the abnormal behavior detection result, including the following steps:

[0105] S51. Determine whether the centroid height information is abnormal based on the clustering result.

[0106] During specific implementation, if the server determines that the clustering result is that the current center of mass height information of the target object is far away from the cluster center, that is, the current center of mass height information of the target object in the target area and the historical center of mass height information of the target object in the target area are not classified in the same category, then it can be determined that the current center of mass height information of the target object in the target area is abnormal; if the clustering result is determined that the current center of mass height information of the target object is within the range of the category represented by the cluster center, that is, the current center of mass height information of the target object in the target area and the historical center of mass height information of the target object in the target area are classified in the same category, then it can be determined that the current center of mass height information of the target object in the target area is normal.

[0107] S52: Determine whether the residence time information is abnormal based on the difference between the residence time information and the predicted residence time information.

[0108] In specific implementation, if the difference between the determined residence time information and the predicted residence time information is less than a preset threshold, the residence time information is determined to be normal; if the difference between the determined residence time and the predicted residence time information is greater than or equal to the preset threshold, the residence time information is determined to be abnormal.

[0109] Specifically, if the server determines that the difference between the current residence time information and the predicted residence time information of the target object at the current location is less than a preset threshold, the server determines that the current residence time information of the target object at the current location is normal; if the server determines that the difference between the current residence time information and the predicted residence time information of the target object at the current location is greater than or equal to the preset threshold, the server determines that the current residence time information of the target object at the current location is abnormal. The preset thresholds for different areas can be set according to needs. For example, the preset threshold corresponding to the open space area of ​​the living room can be set to 5 minutes, the preset threshold corresponding to the sofa area of ​​the living room can be set to 20 minutes, and can also be set to any other time length, which is not limited in the embodiments of the present application.

[0110] S53: If it is determined that the centroid height information is normal and the residence time information is normal, or if it is determined that the centroid height information is abnormal and the residence time information is normal, it is determined that the behavior of the target object is normal.

[0111] In specific implementation, there are two situations in which the behavior of the target object is judged to be normal:

[0112] In one case, if it is determined that the current centroid height information of the target object is normal and the current residence time information of the target object at the current position is normal, then it is determined that the behavior of the target object is normal.

[0113] Another situation is: if the centroid height information is determined to be abnormal but the residence time information is normal, then the behavior of the target object can be determined to be normal. In this case, although the posture of the target object at the current position is abnormal, the residence time has not reached the abnormal condition, and the behavior of the target object is not determined to be abnormal, which can reduce the false alarm rate.

[0114] S54: If it is determined that the centroid height information is abnormal and the residence time information is abnormal, or if it is determined that the centroid height information is normal and the residence time information is abnormal, it is determined that the behavior of the target object is abnormal.

[0115] In specific implementation, there are two situations in which the abnormal behavior of the target object is determined:

[0116] One situation is: if the current centroid height information of the target object and the current residence time information of the target object at the current position are both abnormal, it can be determined that the behavior of the target object is abnormal.

[0117] Another situation is that although the current centroid height information of the target object is normal, the current residence time information of the target object at the current position is abnormal, and the behavior of the target object is determined to be abnormal. In this case, although the posture of the target object at the current position is normal, but the residence time has reached the abnormal condition, the current behavior of the target object is determined to be abnormal first to reduce the false alarm rate.

[0118] Furthermore, when the target object's behavior is detected to be abnormal, the server sends an early warning message to the management terminal device so that the corresponding caregiver can rescue the target object. The server can also send a voice message to the target object's care device to confirm whether the target object needs rescue. If it is determined that the target object is currently in a safe state, the early warning message may not be sent to the management terminal. If it is determined that the target object is in a dangerous state, a early warning message is sent to the management terminal.

[0119] For example, assuming that the current position of the target object is in the sofa area of ​​the object, the centroid height reflects that the posture of the target object is lying (i.e., lying), the initial time when the target object arrives at the sofa area is 8:30 in the morning, and the current stay time is 3 hours. The server inputs the current position information, centroid height information and the initial time information of the target object arriving at the sofa area into the time prediction model, and obtains the predicted stay time information of the target object in the sofa area as 2.5 hours, and the difference between the current stay time and the predicted stay time is 0.5 hours. Assuming that the preset threshold corresponding to the sofa area is 20 minutes, and the difference between the current stay time and the predicted stay time is greater than 20 minutes, it can be judged that the current stay time information is abnormal. The server clusters the current centroid height information with the historical centroid height information when the target object acts normally in the sofa area to obtain the clustering result. According to the clustering result, the current centroid height information is normal. Since the current stay time information is abnormal, it can be judged that the current behavior of the target object is abnormal, thereby avoiding underreporting.

[0120] For another example, assuming that the current position of the target object is in the open space area of ​​the living room, the centroid height reflects that the target object is sitting, the initial time when the target object arrives at the sofa area is 1:30 pm, and the current stay time is 5 minutes. The server inputs the current position information, centroid height information and the initial time information when the target object arrives at the sofa area into the time prediction model, and obtains the predicted stay time information of the target object in the open space area of ​​the living room as 2 minutes, and the difference between the current stay time and the predicted stay time is 3 hours. Assuming that the preset threshold corresponding to the open space area of ​​the living room is 5 minutes, and the difference between the current stay time and the predicted stay time is less than 5 minutes, it can be judged that the current stay time information is normal. The server clusters the current centroid height information with the historical centroid height information when the target object acts normally in the sofa area to obtain a clustering result. According to the clustering result, the current centroid height information is abnormal. Since the current stay time information is normal, it can be judged that the current behavior of the target object is normal to avoid false alarms.

[0121] The abnormal behavior detection method provided in the embodiment of the present application obtains the position information, center of mass height information, initial time information and residence time information of the target object collected by radar, and inputs the position information, center of mass height information and initial time information of the target object arriving at the position into a time prediction model to obtain the predicted residence time information of the target object at the position. The time prediction model is obtained by training the behavior samples of the target object according to a preset neural network model, and then, according to the difference between the residence time information and the predicted residence time information, the abnormal behavior detection result is obtained. In the embodiment of the present application, the position information, centroid height information, initial time information of the target object arriving at the position and residence time information collected by the radar are obtained, and the residence time is the length of time the target object has currently stayed at the position. The position information, centroid height information and initial time information of the target object arriving at the position are input into a time prediction model obtained in advance based on the behavior sample of the target object and the preset neural network model training, and the initial time of the target object arriving at the position and the normal residence time corresponding to maintaining the centroid height at the position are predicted, wherein the time prediction model is used to predict the residence time of the target object at a certain position, and the target object's residence time is obtained according to the difference between the actual residence time of the target object at the position and the predicted residence time. Abnormal behavior detection results. Compared with the prior art, the present application jointly judges the abnormal behavior of the target object based on the three dimensions of the target object's position, center of mass height and residence time, comprehensively considers the spatial characteristics and time characteristics, and predicts the target object's residence time based on the time prediction model pre-trained based on the target object's behavior samples. The time prediction model trained according to the target object's behavior habits can more accurately predict the target object's residence time based on the target object's current position, the target object's initial time when it arrives at the position, and the center of mass height maintained at the position. Therefore, based on the difference between the predicted residence time and the actual residence time of the target object, it is possible to more accurately judge whether the target object's behavior is abnormal, thereby improving the accuracy of abnormal event detection and reducing the false alarm rate and missed alarm rate.

[0122] Based on the same inventive concept, an embodiment of the present application also provides an abnormal behavior detection device. Since the principle of solving the problem by the above abnormal behavior detection device is similar to that of the above abnormal behavior detection method, the implementation of the above device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0123] like Figure 6 As shown, it is a schematic diagram of the structure of the abnormal behavior detection device provided in the embodiment of the present application, and the abnormal behavior detection device can be applied to Figure 1 The server 102 shown may include:

[0124] The first acquisition module 61 is used to acquire the position information, the centroid height information, the initial time information of the target object arriving at the position and the residence time information collected by the radar;

[0125] A prediction module 62 is used to input the location information, the centroid height information and the initial time information of the target object arriving at the location into a time prediction model to obtain the predicted residence time information of the target object at the location, wherein the time prediction model is obtained by training the behavior samples of the target object according to a preset neural network model;

[0126] The detection module 63 is used to obtain an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information.

[0127] In one embodiment, the device further comprises:

[0128] a determination module, used for determining the target area to which the position of the target object belongs based on the position information of each pre-marked area and the position information of the target object before inputting the position information, the centroid height information and the initial time information of the target object arriving at the position into the time prediction model;

[0129] A second acquisition module is used to acquire historical centroid height information of the target object when it moves normally in the target area;

[0130] The clustering module is used to cluster the centroid height information and the historical centroid height information to obtain a clustering result.

[0131] In one implementation, the detection module 63 is specifically configured to obtain an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information and the clustering result.

[0132] In one embodiment, the detection module 63 is specifically used to determine whether the centroid height information is abnormal based on the clustering result; determine whether the residence time information is abnormal based on the difference between the residence time information and the predicted residence time information; if it is determined that the centroid height information is normal and the residence time information is normal, or, if it is determined that the centroid height information is abnormal and the residence time information is normal, then determine that the behavior of the target object is normal; if it is determined that the centroid height information is abnormal and the residence time information is abnormal, or, if it is determined that the centroid height information is normal and the residence time information is abnormal, then determine that the behavior of the target object is abnormal.

[0133] In one embodiment, the detection module 63 is specifically used to determine that the residence time information is normal if it is determined that the difference between the residence time information and the predicted residence time information is less than a preset threshold; if it is determined that the difference between the residence time information and the predicted residence time information is greater than or equal to the preset threshold, then determine that the residence time information is abnormal.

[0134] In one embodiment, the prediction module 62 is specifically used to train and obtain the time prediction model in the following manner: obtaining behavior samples of the target object's activities in pre-marked areas collected by radar, the behavior samples being the stay information of the target object at different positions in the pre-marked areas, the stay information including the stay position information of the target object in the pre-marked areas, the center of mass height information of the target object at the stay position, the initial time information of the target object arriving at the stay position, and the stay time information of the target object at the stay position; for each behavior sample of the target object, the stay position information of the target object, the center of mass height information of the target object at the stay position, and the initial time information of the target object arriving at the stay position are input into the preset neural network model to obtain the predicted stay time information of the target object at the stay position; determining the prediction error based on the predicted stay time information of the target object at the stay position and the stay time information of the target object at the stay position; adjusting each parameter contained in the preset neural network model based on the prediction error until the model converges to obtain the trained time prediction model.

[0135] Based on the same technical concept, the present application embodiment also provides an electronic device 700, referring to Figure 7 As shown, the electronic device 700 is used to implement the abnormal behavior detection method described in the above method embodiment. The electronic device 700 of this embodiment may include: a memory 701, a processor 702, and a computer program stored in the memory and executable on the processor, such as an abnormal behavior detection program. When the processor executes the computer program, the steps in each of the above abnormal behavior detection method embodiments are implemented.

[0136] The specific connection medium between the memory 701 and the processor 702 is not limited in the embodiment of the present application. Figure 7 In the embodiment, the memory 701 and the processor 702 are connected via a bus 703. The bus 703 is Figure 7 The connection between other components is shown by bold lines, which is only for schematic illustration and is not intended to be limiting. The bus 703 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0137] The memory 701 may be a volatile memory, such as a random-access memory (RAM); the memory 701 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 701 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 701 may be a combination of the above memories.

[0138] Processor 702 is used to implement the abnormal behavior detection method provided in the embodiment of the present application.

[0139] An embodiment of the present application also provides a computer-readable storage medium that stores computer-executable instructions required to execute the above-mentioned processor, which includes a program required to execute the above-mentioned processor.

[0140] In some possible implementations, various aspects of the abnormal behavior detection method provided in the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the abnormal behavior detection method according to various exemplary implementations of the present application described above in this specification.

[0141] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0145] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0146] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting abnormal behavior, characterized in that: include: Acquire the location information, centroid height information, initial time information of the target object arriving at the location, and residence time information collected by the radar; Inputting the location information, the centroid height information and the initial time information of the target object arriving at the location into a time prediction model to obtain the predicted residence time information of the target object at the location, wherein the time prediction model is obtained by training the behavior samples of the target object according to a preset neural network model; An abnormal behavior detection result is obtained according to the difference between the residence time information and the predicted residence time information.

2. The method according to claim 1, characterized in that Before inputting the position information, the centroid height information and the initial time information of the target object arriving at the position into the time prediction model, the method further includes: Determine the target area to which the position of the target object belongs according to the position information of each pre-marked area and the position information of the target object; Acquire historical centroid height information of the target object when it moves normally in the target area; The centroid height information and the historical centroid height information are clustered to obtain a clustering result.

3. The method according to claim 2, characterized in that Obtaining an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information, specifically including: An abnormal behavior detection result is obtained according to the difference between the residence time information and the predicted residence time information and the clustering result.

4. The method according to claim 3, characterized in that Obtaining an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information and the clustering result, specifically including: Determining whether the centroid height information is abnormal according to the clustering result; determining whether the residence time information is abnormal according to a difference between the residence time information and the predicted residence time information; If it is determined that the centroid height information is normal and the residence time information is normal, or if it is determined that the centroid height information is abnormal and the residence time information is normal, then it is determined that the behavior of the target object is normal; If it is determined that the centroid height information is abnormal and the residence time information is abnormal, or if it is determined that the centroid height information is normal and the residence time information is abnormal, it is determined that the behavior of the target object is abnormal.

5. The method according to claim 4, characterized in that Determining whether the residence time information is abnormal according to the difference between the residence time information and the predicted residence time information specifically includes: If it is determined that the difference between the residence time information and the predicted residence time information is less than a preset threshold, then it is determined that the residence time information is normal; If it is determined that the difference between the residence time information and the predicted residence time information is greater than or equal to the preset threshold, it is determined that the residence time information is abnormal.

6. The method according to claim 1, characterized in that The time prediction model is obtained by training in the following way: Acquire a behavior sample of the target object's activities in each pre-marked area collected by the radar, wherein the behavior sample is the stay information of the target object at different positions in each pre-marked area, and the stay information includes the stay position information of the target object in each pre-marked area, the centroid height information of the target object at the stay position, the initial time information of the target object arriving at the stay position, and the stay time information of the target object at the stay position; For each behavior sample of the target object, input the target object's stay position information, the target object's center of mass height information at the stay position, and the target object's initial time information when it arrives at the stay position into the preset neural network model to obtain the target object's predicted stay time information at the stay position; Determining a prediction error based on the predicted stay time information of the target object at the stay location and the stay time information of the target object at the stay location; According to the prediction error, various parameters included in the preset neural network model are adjusted until the model converges to obtain the trained time prediction model.

7. An abnormal behavior detection device, characterized in that: include: A first acquisition module is used to acquire the position information, centroid height information, initial time information of the target object arriving at the position and residence time information of the target object collected by the radar; A prediction module, used to input the location information, centroid height information and initial time information of the target object arriving at the location into a time prediction model to obtain the predicted residence time information of the target object at the location, wherein the time prediction model is obtained by training the behavior samples of the target object according to a preset neural network model; The detection module is used to obtain an abnormal behavior detection result according to the difference between the residence time information and the predicted residence time information.

8. The device according to claim 7, characterized in that The prediction module is specifically used to train and obtain the time prediction model in the following manner: obtaining behavior samples of the target object's activities in pre-marked areas collected by radar, the behavior samples being the stay information of the target object at different positions in the pre-marked areas, the stay information including the stay position information of the target object in the pre-marked areas, the centroid height information of the target object at the stay position, the initial time information of the target object arriving at the stay position and the stay time information of the target object at the stay position; for each behavior sample of the target object, inputting the stay position information of the target object, the centroid height information of the target object at the stay position and the initial time information of the target object arriving at the stay position into the preset neural network model to obtain the predicted stay time information of the target object at the stay position; determining the prediction error based on the predicted stay time information of the target object at the stay position and the stay time information of the target object at the stay position; adjusting each parameter contained in the preset neural network model based on the prediction error until the model converges to obtain the trained time prediction model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the abnormal behavior detection method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the abnormal behavior detection method according to any one of claims 1 to 6 are implemented.

Images