Abnormal behavior detection method and device

Abstract

The embodiments of the present application provide a method and device for detecting abnormal behavior, relating to the field of artificial intelligence. The method comprises: obtaining target point cloud data from a target area. Based on the target point cloud data, the body posture type of the target person and the target position of the target person in the target area are output. When the target position and body posture type meet the abnormal behavior conditions, an alarm is triggered. Compared to the prior art method of analyzing abnormal behavior by collecting videos or images of the elderly at home, the present application uses point cloud data for abnormal behavior detection, which reasonably avoids the privacy issues of the elderly.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular to a method and device for detecting abnormal behavior. Background Art

[0002] With the increasing aging of society, the safety of elderly people living alone at home has received widespread attention. Due to years of neglect, a lack of ability to cope with accidental injuries and dangers, and a lack of social support networks, elderly people living alone face significant safety risks at home. Accidents can easily go unnoticed, leading to death. Therefore, to address the safety of elderly people living alone at home, abnormal behavior detection is necessary.

[0003] Related technologies generally use cameras to collect video or image data of the elderly at home, and then analyze and identify the videos and images to determine whether the elderly in the videos or images have exhibited abnormal behavior. However, this method can easily infringe on the privacy of the elderly. Summary of the Invention

[0004] The embodiments of the present application provide an abnormal behavior detection method, apparatus, device, and storage medium for detecting abnormal behavior of elderly people living alone without infringing on their privacy.

[0005] In one aspect, an embodiment of the present application provides a method for detecting abnormal behavior, the method comprising:

[0006] Obtain target point cloud data from target area detection;

[0007] Based on the target point cloud data, outputting a human body posture type of the target human body and a target position of the target human body in the target area;

[0008] When the target position and the human body posture type meet abnormal behavior conditions, an alarm is triggered.

[0009] Optionally, it also includes:

[0010] If the target position is the set position and the time the target person is at the target position is greater than a preset threshold, it is determined that the target person's behavior is abnormal and an alarm is triggered.

[0011] In one aspect, an embodiment of the present application provides an abnormal behavior detection device, comprising:

[0012] A preprocessing module is used to obtain target point cloud data from target area detection;

[0013] A recognition module, configured to output a human body posture type of a target human body and a target position of the target human body in the target area based on the target point cloud data;

[0014] The discrimination module is used to trigger an alarm when the target position and the human body posture type meet abnormal behavior conditions.

[0015] Optionally, the preprocessing module is specifically used to:

[0016] The original point cloud data obtained from the target area detection is filtered to obtain the target point cloud data.

[0017] Optionally, the identification module is specifically configured to:

[0018] Determine a human body detection frame in the target point cloud data and a human body posture type corresponding to the human body detection frame through a deep learning model;

[0019] The human body detection frame is position-matched with the area map of the target area to determine the target position of the target human body corresponding to the human body detection frame in the target area.

[0020] Optionally, the preprocessing module is specifically used to:

[0021] Adding the original point cloud data to the first collection;

[0022] Iteratively filtering the data points in the first set to obtain target point cloud data, wherein each iteration process includes the following steps:

[0023] Selecting n data points from the first set and performing plane fitting on the n data points to obtain a target plane, where n is an integer greater than 0;

[0024] Determining the distance between each data point in the first set and the target plane;

[0025] Adding data points whose distance from the target plane is greater than or equal to a first threshold to the second set, and adding data points whose distance from the target plane is less than the first threshold to the third set;

[0026] If the number of data points in the second set is greater than a second threshold, the data points in the first set and the third set are deleted, and the data points in the second set are moved to the first set.

[0027] Optionally, the preprocessing module is further configured to:

[0028] If the number of data points in the third set is greater than a second threshold, a difference set is calculated between the first set and the third set, the data points retained in the first set are used as target point cloud data, and the iterative filtering ends.

[0029] Optionally, the preprocessing module is specifically used to:

[0030] Downsampling is performed on the original point cloud data to obtain a plurality of downsampled data points.

[0031] Clustering the multiple downsampled data points to determine discrete data points among the multiple downsampled data points;

[0032] The discrete data points in the plurality of downsampled data points are filtered to obtain target point cloud data.

[0033] Optionally, the deep learning model includes at least one multi-task recognition module, wherein one multi-task recognition module includes a human detection module, a human posture recognition module, and a feature fusion module, wherein the human detection module is used to extract human detection frame features, the human posture recognition module is used to extract human posture features, and the feature fusion module is used to fuse the human detection frame features with the human posture features;

[0034] The identification module is specifically used for:

[0035] Performing feature extraction on the target point cloud data to obtain a first data feature;

[0036] Using the at least one multi-task recognition module, sequentially processing the first data features to obtain second data features;

[0037] Based on the second data feature, a human body detection frame in the target point cloud data and a human body posture type corresponding to the human body detection frame are determined.

[0038] Optionally, the discrimination module is specifically configured to:

[0039] If the target position is a set position and the human body posture type corresponding to the human body detection frame is a set human body posture type, it is determined that the target human body behavior is abnormal and an alarm is triggered.

[0040] Optionally, the discrimination module is specifically configured to:

[0041] If the target position is the set position, the human body posture type corresponding to the human body detection frame is the first set human body posture type, and the human body posture type of the target human body in the previous detection cycle is the second set human body posture type, then the target human body behavior is determined to be abnormal and an alarm is triggered.

[0042] Optionally, the discrimination module is further configured to:

[0043] If the target position is the set position and the time the target person is at the target position is greater than a preset threshold, it is determined that the target person's behavior is abnormal and an alarm is triggered.

[0044] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned workflow generation method when executing the program.

[0045] On the one hand, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the program is run on the computer device, the computer device executes the steps of the above-mentioned workflow generation method.

[0046] In an embodiment of the present application, a terminal device obtains raw point cloud data from a target area detection and filters the raw point cloud data to obtain target point cloud data. Using a deep learning model, a human body detection frame and the human body posture type corresponding to the human body detection frame are determined in the target point cloud data. The human body detection frame is then positionally matched with a regional map of the target area to determine the target position of the target person corresponding to the human body detection frame within the target area. If, based on the target position and the human body posture type corresponding to the human body detection frame, the target person's behavior is determined to be abnormal, an alarm is triggered. Compared to the prior art method of analyzing abnormal behavior by collecting videos or images of elderly people at home, this application utilizes point cloud data for abnormal behavior detection, effectively avoiding privacy concerns for the elderly. The use of a filtering method reduces the amount of point cloud data, significantly shortens the training time of the deep learning model, and improves the training accuracy of the model. A deep learning model is used to detect the point cloud data, determine the human body detection frame and human body posture type, and then determine whether the human behavior is abnormal based on the human body detection frame and human posture type, thereby improving the accuracy of abnormal behavior detection using point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A schematic diagram of a system architecture provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of a process for obtaining a terminal device's posture trajectory and constructing a target area map provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of a target area map with manually labeled functional attributes provided in an embodiment of the present application;

[0051] Figure 4 A flowchart of an abnormal behavior detection method provided in an embodiment of the present application;

[0052] Figure 5 A flowchart of a statistical filtering method provided in an embodiment of the present application;

[0053] Figure 6 A schematic diagram of a flow chart of a method for removing planar point cloud data provided in an embodiment of the present application;

[0054] Figure 7 A schematic diagram of a flow chart of a plane fitting method provided in an embodiment of the present application;

[0055] Figure 8 A schematic diagram of a down-sampling structure provided in an embodiment of the present application;

[0056] Figure 9 A schematic diagram of a process flow of a Euclidean clustering algorithm provided in an embodiment of the present application;

[0057] Figure 10 A schematic diagram of the structure of a deep learning model provided in an embodiment of the present application;

[0058] Figure 11 A flowchart of an abnormal behavior detection method provided in an embodiment of the present application;

[0059] Figure 12 A schematic diagram of the structure of an abnormal behavior detection device provided in an embodiment of the present application;

[0060] Figure 13 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and beneficial effects of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0062] For ease of understanding, the terms involved in the embodiments of this application are explained below.

[0063] SLAM (Simultaneous Localization and Mapping) is a technology used to solve the positioning, navigation, and mapping problems of mobile robots operating in a location environment. Specifically, the robot starts from an unknown position in an unknown environment and locates itself based on its position and map. Simultaneously, it constructs an incremental map based on its positioning, enabling autonomous positioning and navigation.

[0064] Point cloud data refers to a collection of vectors in a three-dimensional coordinate system obtained by automatically measuring points on an object's surface using 3D scanning equipment. 3D scanning equipment includes lidar, stereo cameras, and time-of-flight cameras.

[0065] refer to Figure 1 , which is a system architecture diagram applicable to an embodiment of the present application, and the system architecture includes at least a terminal device 101 and a server 102.

[0066] The terminal device 101 is installed with a target application for abnormal behavior detection, which can be a pre-installed client, a web application, or a small program embedded in other applications, etc. The terminal device 101 can be a sweeping robot, a robot, etc., but is not limited thereto.

[0067] Server 102 is the background server of the target application and provides services for the target application. Server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0068] The terminal device 101 constructs a target area map in response to a target area map construction operation submitted by a user in an abnormal behavior detection application.

[0069] Specifically, if Figure 2As shown, terminal device 101 uses environmental sensors to acquire raw environmental data from various objects in the measurement environment. This data can be laser scanning data, point cloud data, or other such data. Environmental sensors include various sensors that assist the robot in perceiving its surroundings, such as laser range finders, light sensors, and temperature and humidity sensors. Terminal device 101 uses posture sensors to acquire the robot's position and posture. Position includes the X, Y, and Z axes of the spatial coordinate system; posture includes the angle rx between a point in space and the X axis, the angle ry between the point in space and the Y axis, and the angle rz between the point in space and the Z axis.

[0070] The environmental information acquired by the environmental sensor and posture sensor, along with the robot's posture information, are used as input data. The terminal device 101 sends the collected input data to the server 102, which uses visual odometry to perform inter-frame motion estimation and landmark delineation. Landmark delineation involves determining the landmarks corresponding to the point cloud data at each moment. Landmarks are fixed points in three-dimensional space, and representative feature points in the point cloud data can be used as landmarks. Inter-frame motion estimation estimates the posture trajectory of the terminal device 101 using landmarks at adjacent moments, thereby solving the positioning problem of the terminal device 101.

[0071] Since the input data received by the visual odometry generally contains a lot of noise, when using the input data at adjacent moments to estimate the position trajectory of the terminal device 101, if there is an error in the estimation result at one moment, it will cause cumulative errors in the subsequent position estimation, resulting in an increasingly larger error.

[0072] To solve the above problems, it is necessary to optimize the position trajectory of the terminal device 101 initially obtained by the visual odometry. The server 102 performs global state estimation on the obtained position trajectory to determine a more accurate position trajectory of the terminal device 101 and a target area map.

[0073] To further improve the accuracy of detecting the position trajectory of the terminal device 101 and the target area map, loop closure detection can also be performed on the results of the global state estimation. Loop closure detection determines whether the current position of the terminal device 101 has been visited before by detecting the similarity between the current scene and historical scenes, thereby correcting the position trajectory of the terminal device 101.

[0074] Therefore, executing both global state estimation and loop detection in the server 102 can produce more accurate results than executing the global state estimation technique alone. Finally, the server 102 outputs the obtained position trajectory of the terminal device 101 and the target area map to the terminal device 101.

[0075] It should be noted that the above-mentioned processing steps such as inter-frame motion estimation, landmark delineation, global state estimation and loop detection are not limited to execution on the server, but can also be adjusted to be executed on the terminal device 101 according to actual needs. This application does not make specific limitations on this.

[0076] The target area map presented by the terminal device 101 is divided into zones, and functional attributes are marked according to each zone, such as kitchen, living room, bathroom, bedroom, etc.

[0077] For example, if Figure 3 As shown, a target area map is constructed by SLAM technology, and the target area map is manually partitioned, and the corresponding functional attributes are marked for each partition, namely, bedroom 1, bedroom 2, living room, kitchen, and bathroom.

[0078] When performing abnormal behavior detection, the terminal device 101 first obtains target point cloud data from the target area. Based on the target point cloud data, it then outputs the target person's posture type and the target position within the target area. When the target position and posture type meet the abnormal behavior criteria, an alarm is triggered. A help message is then sent to the guardian or community management staff via the communication application installed on the terminal device 101.

[0079] based on Figure 1 The system architecture diagram, the embodiment of the present application provides a process of an abnormal behavior detection method, such as Figure 4 As shown, the process of this method can be Figure 1 The server 102 shown executes the following steps:

[0080] Step S401: Detect and obtain target point cloud data from the target area.

[0081] In a possible implementation, original point cloud data obtained from target area detection is filtered to obtain target point cloud data.

[0082] Specifically, the target area can include multiple functional areas (such as a kitchen and a restroom), a single functional area (such as a kitchen), or a portion of a functional area (such as a portion of a kitchen). The target area is detected using equipment such as lidar, stereo cameras, and time-of-flight cameras to obtain raw point cloud data.

[0083] You can use one or more of the following methods to filter the original point cloud data to obtain the target point cloud data:

[0084] Method 1: Filter the original point cloud data to remove noise points, outliers, holes and other data points in the point cloud data.

[0085] Method 2: Perform plane detection on the original point cloud data and remove the plane point clouds in the original point cloud data.

[0086] Method 3: Downsample the original point cloud data.

[0087] Method 4: cluster the original point cloud data to determine the discrete data points in the original point cloud data; and filter the discrete data points.

[0088] Step S402: Based on the target point cloud data, output the human body posture type of the target human body and the target position of the target human body in the target area.

[0089] In one possible implementation, a deep learning model is used to determine a human body detection frame and the human body posture type corresponding to the human body detection frame in the target point cloud data. The human body detection frame is then positionally matched with a region map of the target area to determine the target position of the target person corresponding to the human body detection frame within the target area.

[0090] Specifically, the deep learning model includes at least one multi-task recognition module, wherein each multi-task recognition module includes a human detection module, a human posture recognition module, and a feature fusion module. A human detection frame in the target point cloud data and the human posture type corresponding to the human detection frame are determined based on the multi-task recognition module in the deep learning model.

[0091] Obtain the position information of the human detection frame in the detection coordinate system, where the detection coordinate system is a coordinate system constructed with the terminal device as the origin. Based on the mapping relationship between the regional map coordinate system and the detection coordinate system, and the position of the human detection frame in the detection coordinate system, match the human detection frame with the regional map of the target area to determine the target position of the target person in the target area.

[0092] In the embodiments of this application, a filtering method is used to reduce the amount of point cloud data, significantly reducing the training time of the deep learning model while improving the training accuracy of the model. A deep learning model is used to detect point cloud data, determine human body detection frames and human posture types, and then determine whether human behavior is abnormal based on the human body detection frames and human posture types, thereby improving the accuracy of detecting abnormal behavior using point cloud data.

[0093] Step S403: triggering an alarm when the target position and the human body posture type meet abnormal behavior conditions.

[0094] In an embodiment of the present application, a terminal device obtains target point cloud data from target area detection. Based on this target point cloud data, the target person's body posture type and the target person's target position within the target area are output. When the target position and body posture type meet abnormal behavior conditions, an alarm is triggered. Compared to the existing technology that analyzes abnormal behavior by collecting videos or images of the elderly at home, this application uses point cloud data for abnormal behavior detection, which reasonably avoids the privacy issues of the elderly.

[0095] Optionally, in the above step S401, the original point cloud data is filtered to remove noise points, outliers, holes and other data points in the point cloud data.

[0096] Specifically, statistical filtering can be used to remove noise points, outliers, holes and other data points in the original point cloud data. Figure 5 The statistical filtering method provided in an embodiment of the present application is exemplified, and the method includes:

[0097] Step S501: Calculate the average distance from each data point in the point cloud data to its nearest k data points, where k is an integer greater than 0.

[0098] Specifically, a data point A is selected from the point cloud data, and the data points in the point cloud data except point A are traversed. The distance between data point A and the remaining data points is recorded, and the distances are arranged in ascending order. The distances of the first k data points are averaged to obtain the average distance from data point A to its nearest k data points.

[0099] Repeat the above process for each data point in the point cloud data to obtain the average distance between each data point and its nearest k data points.

[0100] Step S502: construct a scatter plot based on the average distance between each data point and its nearest k data points.

[0101] Step S503: Fitting the scatter plot using a Gaussian distribution function.

[0102] Specifically, the scatter plot is approximately Gaussian distributed, and the position points in the scatter plot are fitted with a Gaussian distribution function to determine the mean, variance, and standard deviation of the Gaussian distribution function corresponding to the minimum fitting error.

[0103] In step S504, based on the fitted Gaussian distribution function, the target position points outside the range of 3 times the standard deviation in the scatter plot are determined, and the data points corresponding to the target position points in the original point cloud data are removed.

[0104] In the embodiments of the present application, since the average distance from each data point to its k nearest data points in the point cloud data is roughly Gaussian, a statistical filtering algorithm can be used to simply, efficiently, and accurately filter out most noise points, outliers, holes, and other isolated points in the point cloud data. Furthermore, dense areas in the point cloud data contain a large amount of information, while isolated points contain less information. The statistical filtering algorithm can effectively reduce the amount of point cloud data while retaining most of the information in the point cloud data.

[0105] Optionally, in the above step S401, the embodiment of the present application can perform plane detection on the original point cloud data, and then remove the plane point cloud data in the original point cloud data to obtain the target point cloud data, specifically as follows: Figure 6 As shown, the following steps are included:

[0106] Step S601: adding original point cloud data to the first set.

[0107] Step S602: select n data points from the first set, and perform plane fitting on the n data points to obtain a target plane, where n is an integer greater than 0.

[0108] Step S603: Determine the distance between each data point in the first set and the target plane, and execute steps S604 and S607 respectively.

[0109] Step S604: Add data points whose distance from the target plane is greater than or equal to the first threshold to the second set.

[0110] Step S605 , determining whether the number of data points in the second set is greater than a second threshold, if so, executing step S606 , otherwise ending.

[0111] Step S606: Delete the data points in the first set and the third set, and move the data points in the second set to the first set, and jump to step S602.

[0112] Step S607: Add data points whose distances to the target plane are less than a first threshold to the third set.

[0113] Step S608, determine whether the number of data points in the third set is greater than the second threshold, if so, execute step S609, otherwise end.

[0114] Step S609: Calculate the difference between the first set and the third set.

[0115] Step S610: Use the data points retained in the first set as target point cloud data, and end.

[0116] according to Figure 6The method shown can find a plane in the original point cloud data and delete the plane point cloud data. In order to delete multiple plane point cloud data in the original point cloud data, Figure 6 The above steps are repeated for a preset number of times until all the plane point cloud data in the original point cloud data are deleted.

[0117] In the embodiments of the present application, since the raw point cloud data contains at least one plane, and plane point cloud data generally contains background information such as walls, rather than information about the target human body, the method of performing plane detection on the raw point cloud data and removing the plane point cloud data from the raw point cloud data to obtain the target point cloud data can effectively remove background information from the raw point cloud data, significantly reducing the amount of raw point cloud data. Furthermore, since most background information is invalid, removing this invalid information does not affect the accuracy of the deep learning model results.

[0118] Optionally, in the above step S602, Figure 7 The following is an example of a plane fitting process provided in an embodiment of the present application, including the following steps:

[0119] Step S701: Add the original point cloud data to set A.

[0120] Step S702, determine whether the current number of iterations is less than an iteration threshold T, where T is an integer greater than 0. If so, execute step S703, otherwise end.

[0121] Step S703: randomly select n data points from set A to construct a plane model, where n is an integer greater than 0.

[0122] Step S704 : determining the distance between each data point in the original point cloud data and the plane model.

[0123] Step S705 , obtaining the distance corresponding to the i-th data point in the original point cloud data, where i=1.

[0124] Step S706 , determining whether i is less than the number of original point cloud data, if so, executing step S707 , otherwise executing step S710 .

[0125] Step S707 , determining whether the distance corresponding to the i-th data is less than the distance threshold D, if so, executing step S708 , otherwise executing step S709 .

[0126] Step S708: Add the i-th data point to set B.

[0127] Step S709, i=i+1, jump to step S706.

[0128] Step S710: Determine the number of data points in set B to be m, where m is an integer greater than or equal to 0.

[0129] Step S711, determine whether m is greater than the quantity threshold M, if so, execute step S712, otherwise execute step S714.

[0130] Step S712: save m and the corresponding plane model into the candidate set.

[0131] Step S713: Delete the data points in set A and move the data points in set B to set A, and jump to step S703.

[0132] Step S714, determine whether the current number of iterations is equal to the iteration threshold T, if so, execute step S715, otherwise jump to step S703.

[0133] Step S715: Select the plane model corresponding to the largest m from the candidate set as the optimal plane model, and end.

[0134] Optionally, in the above step S401 , downsampling processing is performed on the original point cloud data to obtain a plurality of downsampled data points.

[0135] Specifically, the height, width, and depth of the original point cloud data are H, W, and D, respectively. Each voxel is a small cube, and the height, width, and depth of each small cube are h, w, and d, respectively. The original point cloud data is divided according to the size of each voxel, that is, a three-dimensional voxel grid is created for the original point cloud data according to the size of each voxel. There are H / h voxels in the height direction of the three-dimensional voxel grid, W / w voxels in the width direction of the three-dimensional voxel grid, and D / d voxels in the depth direction of the three-dimensional voxel grid. Therefore, after dividing the original point cloud data according to voxels, the number of voxels obtained is (H / h)*(W / w)*(D / d).

[0136] The pose of all data points within each voxel in the 3D voxel grid is determined, and the poses of all data points within each voxel are averaged to obtain the center of gravity of each voxel. The center of gravity of each voxel is used to represent the corresponding voxel information, rather than all points within the voxel, thus completing the downsampling process.

[0137] For example, if Figure 8 As shown in the figure, the height, width, and depth of the original point cloud data are first determined to be H, W, and D, respectively. The height, width, and depth of each voxel are h, w, and d, respectively. The original point cloud data is divided into voxels to obtain a 3D voxel grid. The pose of all data points within each voxel in the 3D voxel grid is averaged, which is the center of gravity of the voxel. The center of gravity is used to represent the corresponding voxel information, completing the downsampling process and obtaining multiple downsampled data points.

[0138] In the embodiments of the present application, since the amount of raw point cloud data is very large, directly inputting the raw point cloud data into the deep learning model for human body detection and human body posture discrimination will consume a lot of graphics processing unit (GPU) computing power. Therefore, by downsampling the raw point cloud data, the amount of raw point cloud data can be effectively reduced. At the same time, the downsampling operation does not destroy the geometric structure of the point cloud data itself, and the posture of the point cloud data remains intact, thus not affecting the accuracy of the deep learning model results.

[0139] Optionally, in the above step S401, the multiple down-sampled data points are clustered to determine discrete data points among the multiple down-sampled data points; and the discrete data points among the multiple down-sampled data points are filtered to obtain target point cloud data.

[0140] Specifically, in point cloud data processing, commonly used clustering algorithms include Euclidean clustering, density clustering, and super-volume clustering. Due to the large amount of point cloud data, in order to reduce the time consumption of the clustering algorithm and achieve the clustering effect in the fastest way, the embodiment of this application uses the Euclidean clustering algorithm to cluster the downsampled data points. Figure 9 The process of the Euclidean clustering algorithm provided in the embodiment of the present application is exemplified, including:

[0141] Step S901: Add the downsampled data points to the fourth set, and set the fifth set to an empty set.

[0142] Step S902: randomly select a data point P from the fourth set.

[0143] Step S903, determine whether the data point P is in the fifth set, if so, jump to step S902, otherwise, execute step S904.

[0144] Step S904: Add the data point P to the fifth set.

[0145] Step S905: Use the KDTree algorithm to determine the K data points closest to the data point P.

[0146] Step S906 , obtaining the distance corresponding to the i-th data point among the K data points, where i=1.

[0147] Step S907, determine whether i is less than or equal to K, if so, execute step S908, otherwise execute step S911.

[0148] Step S908 , determining whether the distance corresponding to the i-th data point is less than a distance threshold D, if so, executing step S909 , otherwise executing step S910 .

[0149] Step S909: add the i-th data point to the sixth set.

[0150] Specifically, a data point whose distance from the data point P is not less than the distance threshold D is a discrete data point.

[0151] Step S910, i=i+1, jump to step S907.

[0152] Step S911, determine whether the data points in the sixth set increase, if so, execute step S912, otherwise execute step S913.

[0153] Step S912: Delete the data points in the fourth set, and copy the data points in the sixth set to the fourth set, and jump to S902.

[0154] Step S913: Calculate the difference between the downsampled data points and the sixth set to obtain the seventh set.

[0155] Step S914, determine whether the number of data points in the seventh set is greater than the set threshold N, if so, execute step S915, otherwise end.

[0156] Step S915: Delete the data points in the fourth set, move the data points in the seventh set to the fourth set, and jump to S902.

[0157] In the embodiment of the present application, Euclidean clustering can effectively distinguish instances in the point cloud data, retain these instance data points, and delete the discrete data points in the point cloud data. Using the point cloud data without the discrete data points as the input data of the model is more conducive to improving the accuracy of the deep learning model.

[0158] Optionally, in the above step S402, the deep learning model includes at least one multi-task recognition module, wherein one multi-task recognition module includes a human body detection module, a human body posture recognition module and a feature fusion module, wherein the human body detection module is used to extract human body detection frame features, the human body posture recognition module is used to extract human body posture features, and the feature fusion module is used to fuse human body detection frame features and human body posture features.

[0159] Feature extraction is performed on the target point cloud data to obtain a first data feature. The first data feature is then processed sequentially using at least one multi-task recognition module to obtain a second data feature. Based on the second data feature, a human body detection frame in the target point cloud data and the human posture type corresponding to the human body detection frame are determined.

[0160] In order to more clearly introduce the deep learning model of the embodiment of this application, Figure 10The structure of a deep learning model provided in an embodiment of the present application is exemplified, which includes a resnet18 module, a multi-task recognition module (human body detection module 1, human body posture recognition module 1 and feature fusion module 1), a human body detection module 2, and a human body posture recognition module 2.

[0161] Each human detection module includes at least one convolution module C and a loss function. Each task identification module includes at least one convolution module C and a loss function.

[0162] The present invention provides a method for calling the deep learning model, which includes:

[0163] The resnet18 module is used to extract features from the input target point cloud data to obtain the first data feature. The feature extraction module in the present invention can adopt other modules and is not limited here.

[0164] The first data feature is input into the human detection module 1 in the task identification module to extract the human detection frame feature The first data feature is input into the human posture recognition module 1 in the task recognition module to extract the human posture feature

[0165] The human body detection frame features and human posture characteristics Input into feature fusion module 1, perform feature fusion, and obtain the second data feature.

[0166] Based on the second data feature, the human body detection frame in the target point cloud data and the human body posture type corresponding to the human body detection frame are determined. Specifically, the second data feature is input into the human body detection module 2 to extract the human body detection frame feature. The second data feature is input into the human posture recognition module 2 to extract the human posture feature

[0167] In the embodiment of the present application, the human body detection module and the human body posture recognition module in the multi-task recognition module respectively extract the human body detection frame features and the human body posture features. The two modules extract corresponding features respectively, and the features do not interfere with each other, which is conducive to improving the accuracy of the model. At the same time, the feature fusion module in the multi-task recognition module can fuse the human body detection frame features and the human body posture features, increasing the diversity of features. Since the human body detection module and the human body posture recognition module in each multi-task recognition module have a loss function, it can effectively avoid the gradient vanishing problem during model training, and can play the role of regularization constraint, so that the middle layer of the model also has recognition ability.

[0168] Optionally, in the above step S403, in a possible implementation, if the target position is a set position and the human posture type corresponding to the human body detection frame is a set human posture type, it is determined that the target human behavior is abnormal and an alarm is triggered.

[0169] Specifically, the location is set to a specific functional area on the target area map, such as a kitchen or storage room. Human posture types include standing, sitting, lying, squatting, and other types. The posture type can be set to lying, or other posture types can be set as needed. If the target person appears in an area such as the kitchen or storage room and the posture type is lying, indicating that the target person may fall, the target person's behavior is considered abnormal and an alarm is triggered.

[0170] In one possible implementation, if the target position is a set position, the human body posture type corresponding to the human body detection frame is the first set human body posture type, and the human body posture type of the target human body in the previous detection cycle is the second set human body posture type, then the target human body behavior is determined to be abnormal and an alarm is triggered.

[0171] Specifically, the location is set to a specific functional area on the target area map, such as a kitchen or storage room. The first set human posture type is standing, sitting, squatting, etc. The second set human posture type is lying. If the target person appears in an area such as a kitchen or storage room, and the first set human posture type is one of standing, sitting, or squatting, and the second set human posture type is lying, indicating that the person may fall, the target person's behavior is determined to be abnormal and an alarm is triggered.

[0172] In addition to determining whether the target human body behaves abnormally based on the target position and the human posture type corresponding to the human body detection frame, it is also possible to determine whether the target human body behaves abnormally based on the target position and time information.

[0173] In a possible implementation, if the target position is a set position and the time the target person is at the target position is greater than a preset threshold, it is determined that the target person's behavior is abnormal.

[0174] Specifically, the location is set as a specific functional area of ​​the target area map, which may be a toilet. When the target person stays in the toilet for a time exceeding a preset threshold, it is determined that the target person's behavior is abnormal and an alarm is triggered.

[0175] After determining that the target person's behavior is abnormal, the terminal device uses the voice module to send a voice inquiry to the elderly person living alone, asking if they need help. If the elderly person responds affirmatively, or if there is no response from the elderly person within a certain time threshold, a call for help will be sent to the guardian or community management staff through the communication module.

[0176] In order to better explain the embodiment of the present application, the following describes the process of an abnormal behavior detection method provided by the embodiment of the present application by taking a specific implementation scenario as an example. Figure 11 As shown:

[0177] Step S1101, collect original point cloud data through laser radar equipment, and execute steps S1102 and S1109 respectively.

[0178] Step S1102: Perform statistical filtering on the original point cloud data.

[0179] Step S1103: perform plane detection and remove plane point cloud data.

[0180] Step S1104: perform downsampling processing.

[0181] Step S1105 : Perform Euclidean clustering processing to obtain target point cloud data.

[0182] Step S1106: input the target point cloud data into the deep learning model.

[0183] Step S1107: Obtain a human body detection frame through a deep learning model.

[0184] Step S1108: Obtain the human body posture type through the deep learning model.

[0185] Step S1109: construct a target area map using SLAM technology.

[0186] Step S1110, judging whether the target human body has abnormal behavior according to the human body detection frame and the target area map, if so, executing step S1112, otherwise ending.

[0187] Step S1111, based on the human body detection frame, human body posture type and target area map, determine whether the target human body has abnormal behavior. If so, execute step S1112, otherwise end.

[0188] Step S1112: The terminal device initiates a voice inquiry.

[0189] Step S1113, determine whether the elderly person living alone responds, if so, execute step S1114, otherwise execute step S1115.

[0190] Step S1114: determine whether the elderly person living alone needs help based on the elderly person's answer. If so, execute step S1115; otherwise, end.

[0191] Step S1115: The terminal device sends a request for help to the guardian or community manager, and the process ends.

[0192] In the embodiment of the present application, point cloud data is collected for processing instead of video or image data, which can effectively avoid privacy issues for the elderly. Filtering the original point cloud data can greatly reduce the amount of point cloud data, and at the same time filter out invalid information in the point cloud data, which is beneficial to improving the accuracy of the deep learning model. Secondly, in the present application, the multi-task recognition module in the deep learning model extracts and fuses the human body detection frame features and human body posture features respectively, which increases the diversity of features and can effectively improve the accuracy of the model. In addition, in the present application, the abnormal behavior of the target human body is judged according to the human body posture type corresponding to the target position and the human body detection frame, or according to the target position and residence time, and the various abnormal behavior situations are comprehensively and reasonably considered, thereby improving the accuracy of abnormal behavior detection for elderly people living alone.

[0193] Based on the same technical concept, the embodiment of the present application provides an abnormal behavior device, such as Figure 12 As shown, the apparatus 1200 includes:

[0194] The pre-processing module 1201 is used to obtain target point cloud data from target area detection;

[0195] The recognition module 1202 is configured to output a human body posture type of a target human body and a target position of the target human body in the target area based on the target point cloud data;

[0196] The identification module 1203 is configured to trigger an alarm when the target position and the human body posture type meet abnormal behavior conditions.

[0197] Optionally, the preprocessing module 1201 is specifically configured to:

[0198] The original point cloud data obtained from the target area detection is filtered to obtain the target point cloud data.

[0199] Optionally, the identification module 1202 is specifically configured to:

[0200] Determine a human body detection frame in the target point cloud data and a human body posture type corresponding to the human body detection frame through a deep learning model;

[0201] The human body detection frame is position-matched with the area map of the target area to determine the target position of the target human body corresponding to the human body detection frame in the target area.

[0202] Optionally, the preprocessing module 1201 is specifically configured to:

[0203] Adding the original point cloud data to the first collection;

[0204] Iteratively filtering the data points in the first set to obtain target point cloud data, wherein each iteration process includes the following steps:

[0205] Selecting n data points from the first set and performing plane fitting on the n data points to obtain a target plane, where n is an integer greater than 0;

[0206] Determining the distance between each data point in the first set and the target plane;

[0207] Adding data points whose distance from the target plane is greater than or equal to a first threshold to the second set, and adding data points whose distance from the target plane is less than the first threshold to the third set;

[0208] If the number of data points in the second set is greater than a second threshold, the data points in the first set and the third set are deleted, and the data points in the second set are moved to the first set.

[0209] Optionally, the preprocessing module 1201 is further configured to:

[0210] If the number of data points in the third set is greater than a second threshold, a difference set is calculated between the first set and the third set, the data points retained in the first set are used as target point cloud data, and the iterative filtering ends.

[0211] Optionally, the preprocessing module 1201 is specifically configured to:

[0212] Downsampling is performed on the original point cloud data to obtain a plurality of downsampled data points.

[0213] Clustering the multiple downsampled data points to determine discrete data points among the multiple downsampled data points;

[0214] The discrete data points in the plurality of downsampled data points are filtered to obtain target point cloud data.

[0215] Optionally, the deep learning model includes at least one multi-task recognition module, wherein one multi-task recognition module includes a human detection module, a human posture recognition module, and a feature fusion module, wherein the human detection module is used to extract human detection frame features, the human posture recognition module is used to extract human posture features, and the feature fusion module is used to fuse the human detection frame features with the human posture features;

[0216] The identification module 1202 is specifically configured to:

[0217] Performing feature extraction on the target point cloud data to obtain a first data feature;

[0218] Using the at least one multi-task recognition module, sequentially processing the first data features to obtain second data features;

[0219] Based on the second data feature, a human body detection frame in the target point cloud data and a human body posture type corresponding to the human body detection frame are determined.

[0220] Optionally, the identification module 1203 is specifically configured to:

[0221] If the target position is a set position and the human body posture type corresponding to the human body detection frame is a set human body posture type, it is determined that the target human body behavior is abnormal and an alarm is triggered.

[0222] Optionally, the identification module 1203 is specifically configured to:

[0223] If the target position is the set position, the human body posture type corresponding to the human body detection frame is the first set human body posture type, and the human body posture type of the target human body in the previous detection cycle is the second set human body posture type, then the target human body behavior is determined to be abnormal and an alarm is triggered.

[0224] Optionally, the discrimination module 1203 is further configured to:

[0225] If the target position is the set position and the time the target person is at the target position is greater than a preset threshold, it is determined that the target person's behavior is abnormal and an alarm is triggered.

[0226] Based on the same technical concept, the embodiment of the present application provides a computer device, which can be a terminal or a server, such as Figure 13 As shown, it includes at least one processor 1301 and a memory 1302 connected to the at least one processor. The specific connection medium between the processor 1301 and the memory 1302 is not limited in the embodiment of the present application. Figure 13 For example, the processor 1301 and the memory 1302 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, and the like.

[0227] In the embodiment of the present application, the memory 1302 stores instructions that can be executed by at least one processor 1301. The at least one processor 1301 can execute the steps included in the above-mentioned workflow generation method by executing the instructions stored in the memory 1302.

[0228] The processor 1301 is the control center of the computer device. It can connect the various parts of the computer device using various interfaces and lines. It generates workflows by running or executing instructions stored in the memory 1302 and calling data stored in the memory 1302. Optionally, the processor 1301 may include one or more processing units. The processor 1301 may integrate an application processor and a modem processor. The application processor mainly processes the operating system, user interface, and application programs, while the modem processor mainly processes wireless communications. It is understood that the modem processor may not be integrated into the processor 1301. In some embodiments, the processor 1301 and the memory 1302 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0229] The processor 1301 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0230] Memory 1302 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1302 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. Memory 1302 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1302 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0231] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned workflow generation method.

[0232] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0233] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0234] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0235] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0236] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for detecting abnormal behavior, characterized in that: Applications in robots include: Obtain target point cloud data from target area detection; Determine a human body detection frame in the target point cloud data and a human body posture type corresponding to the human body detection frame through a deep learning model; Positionally matching the human body detection frame with the area map of the target area to determine the target position of the target human body corresponding to the human body detection frame in the target area; wherein the area map of the target area is obtained by determining the posture trajectory of the robot using input data at adjacent moments, and then performing global state estimation and loop closure detection on the posture trajectory, wherein the input data includes environmental information acquired by the robot and the position and posture of the robot; wherein loop closure detection refers to determining whether the robot has visited the current scene based on the similarity between the current scene and historical scenes, and the loop closure detection is used to correct the posture trajectory of the robot; When the target position and the human body posture type meet abnormal behavior conditions, an alarm is triggered.

2. The method according to claim 1, wherein The step of obtaining target point cloud data from target area detection includes: The original point cloud data obtained from the target area detection is filtered to obtain the target point cloud data.

3. The method according to claim 2, wherein The filtering of the original point cloud data obtained from the target area detection to obtain the target point cloud data includes: Adding the original point cloud data to the first collection; Iteratively filtering the data points in the first set to obtain target point cloud data, wherein each iteration process includes the following steps: Selecting n data points from the first set and performing plane fitting on the n data points to obtain a target plane, where n is an integer greater than 0; Determining the distance between each data point in the first set and the target plane; Adding data points whose distance from the target plane is greater than or equal to a first threshold to the second set, and adding data points whose distance from the target plane is less than the first threshold to the third set; If the number of data points in the second set is greater than a second threshold, the data points in the first set and the third set are deleted, and the data points in the second set are moved to the first set.

4. The method according to claim 3, wherein Also includes: If the number of data points in the third set is greater than a second threshold, a difference set is calculated between the first set and the third set, the data points retained in the first set are used as target point cloud data, and the iterative filtering ends.

5. The method according to claim 2, wherein The filtering of the original point cloud data obtained from the target area detection to obtain the target point cloud data includes: Performing downsampling processing on the original point cloud data to obtain a plurality of downsampled data points; Clustering the multiple downsampled data points to determine discrete data points among the multiple downsampled data points; The discrete data points in the plurality of downsampled data points are filtered to obtain target point cloud data.

6. The method according to claim 1, wherein The deep learning model includes at least one multi-task recognition module, wherein one multi-task recognition module includes a human body detection module, a human body posture recognition module and a feature fusion module, wherein the human body detection module is used to extract human body detection frame features, the human body posture recognition module is used to extract human body posture features, and the feature fusion module is used to fuse the human body detection frame features with the human body posture features; The determining, by a deep learning model, a human body detection frame in the target point cloud data and a human body posture type corresponding to the human body detection frame includes: Performing feature extraction on the target point cloud data to obtain a first data feature; Using the at least one multi-task recognition module, sequentially processing the first data features to obtain second data features; Based on the second data feature, a human body detection frame in the target point cloud data and a human body posture type corresponding to the human body detection frame are determined.

7. The method according to any one of claims 1 to 6, characterized in that: When the target position and the human body posture type meet abnormal behavior conditions, triggering an alarm includes: If the target position is a set position and the human body posture type corresponding to the human body detection frame is a set human body posture type, it is determined that the target human body behavior is abnormal and an alarm is triggered.

8. The method according to any one of claims 1 to 6, characterized in that: When the target position and the human body posture type meet abnormal behavior conditions, triggering an alarm includes: If the target position is the set position, the human body posture type corresponding to the human body detection frame is the first set human body posture type, and the human body posture type of the target human body in the previous detection cycle is the second set human body posture type, then the target human body behavior is determined to be abnormal and an alarm is triggered.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images