Fall detection method, device and equipment based on laser point cloud

Abstract

The invention discloses a tumble detection method, device and equipment based on laser point cloud. The method comprises the steps of obtaining multi-frame point cloud data including current-frame point cloud data, for each frame of point cloud data, identifying key parts of a human body from the point cloud data, determining key points of the key parts, calculating the descending displacement of the key points in the vertical direction within the duration of the multi-frame point cloud data, and calculating the descending displacement of the key points in the vertical direction according to the descending displacement. Whether the displacement is larger than or equal to a threshold value or not is judged, when the displacement is larger than or equal to the threshold value, it is judged that a tumble event occurs, and a tumble alarm is given out, non-inductive detection can be achieved, the method is easily accepted by old people, human body biological characteristics cannot be collected through point cloud data, and the privacy leakage risk does not exist. Besides, whether a falling event occurs or not is judged by identifying the key part of the human body and according to the descending displacement of the key part in the vertical direction, and falling action modeling does not need to be performed on the whole human body, so that the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to laser point cloud technology, and in particular to a fall detection method, device and equipment based on laser point cloud. Background Art

[0002] Elderly people are prone to falling due to their declining physical functions. If the fall is not discovered and treated in time, it may cause permanent loss of body tissues or even death. Therefore, fall detection for the elderly is very necessary.

[0003] An existing fall detection method uses a wearable device (e.g., a smart watch) to detect the fall action. However, due to the cumbersome use process of wearable devices, such as charging, wearing, and equipment maintenance, it is difficult for the elderly to accept. Another fall detection method uses a camera to capture video images to track the trajectory of the human body and detect the fall action. However, due to the privacy protection issues of the video images captured by the camera, it cannot be applied to some private places (e.g., toilets, bathrooms). Summary of the invention

[0004] The present invention provides a fall detection method, device and equipment based on laser point cloud to achieve non-sensing detection, and the point cloud data will not collect human biological characteristics, and there is no risk of privacy leakage.

[0005] In a first aspect, the present invention provides a fall detection method based on laser point cloud, comprising:

[0006] Obtain multi-frame point cloud data including the current frame point cloud data;

[0007] For each frame of point cloud data, identifying key parts of a human body from the point cloud data;

[0008] Determine the key points of the key parts;

[0009] Calculate the vertical displacement of the key point within the duration of multiple frames of point cloud data;

[0010] Determining whether the displacement is greater than or equal to a threshold;

[0011] When the displacement is greater than or equal to a threshold, it is determined that a fall event has occurred, and a fall alarm is issued.

[0012] Optionally, for each frame of point cloud data, identifying key parts of a human body from the point cloud data includes:

[0013] Eliminating environmental point cloud data from the point cloud data and retaining human body point cloud data;

[0014] Performing encoding processing on the human body point cloud data to obtain encoding features representing the semantic similarity between points in the human body point cloud data;

[0015] Performing pooling processing on the encoded features to obtain global features of the human body point cloud data;

[0016] Concatenate the encoding feature and the global feature in the spatial dimension to obtain a fusion feature;

[0017] The fused features are semantically segmented to identify key parts of the human body.

[0018] Optionally, encoding the human body point cloud data to obtain encoding features representing the semantic similarity between points in the human body point cloud data includes:

[0019] Performing embedding processing on the human body point cloud data to convert the human body point cloud data into embedding features;

[0020] Processing the embedded features based on a self-attention mechanism to obtain attention features of multiple different feature scales;

[0021] Concatenate multiple attention features of different feature scales in the spatial dimension to obtain concatenated features;

[0022] The concatenated features are linearized, batch normalized and nonlinearly activated to obtain encoding features that characterize the semantic similarity between points in the human body point cloud data.

[0023] Optionally, embedding the human body point cloud data to convert the human body point cloud data into embedding features includes:

[0024] Performing linearization, batch normalization, and nonlinear activation on the human body point cloud data to obtain regularized human body point cloud features;

[0025] Downsampling the regularized human body point cloud features to obtain downsampled point cloud features;

[0026] For each sampling point in the downsampled point cloud features, determining the sampling point from the regularized human body point cloud features;

[0027] Calculate the K neighboring points closest to the sampling point from the regularized human body point cloud features, and determine the K neighborhood features of each sampling point;

[0028] Fusing the downsampled point cloud features with K-neighborhood features of all the sampling points to obtain neighborhood fusion features;

[0029] The neighborhood fusion features are linearized, batch normalized and nonlinearly activated to obtain embedded features of the human body point cloud data.

[0030] Optionally, the embedded features are processed based on a self-attention mechanism to obtain attention features of multiple different feature scales, including:

[0031] The embedded feature is input into a self-attention module for processing. The self-attention module includes multiple cascaded self-attention units, the input feature of the first self-attention unit is the embedded feature, the input feature of the i-th self-attention unit is the output feature of the i-1-th self-attention unit, and each of the self-attention units is used to output attention features of different feature scales, where i is a positive integer greater than 1.

[0032] Optionally, the processing process of the self-attention unit is:

[0033] Perform linear transformation on the input features to obtain the query matrix, key matrix and value matrix;

[0034] Calculate the dot product of the query matrix and the key matrix and normalize them to obtain an attention weight matrix;

[0035] Calculate the dot product of the attention weight matrix and the value matrix to obtain a self-attention matrix;

[0036] Calculate the difference between the input feature and the self-attention matrix to obtain a deviation matrix;

[0037] Linearizing, batch normalizing, and nonlinearly activating the deviation matrix to obtain a deviation compensation matrix of the input feature;

[0038] The sum of the input feature and the bias compensation matrix is ​​calculated to obtain the output feature of the self-attention unit.

[0039] Optionally, determining the key points of the key parts includes:

[0040] Clustering the point cloud data of the key parts, and calculating the cluster center of the point cloud data of the key parts;

[0041] The cluster center is used as the key point of the key part.

[0042] Optionally, the fall detection method based on laser point cloud also includes:

[0043] When the displacement is less than the threshold, the next frame of point cloud data is received as the current frame of point cloud data, and the step of obtaining multiple frames of point cloud data including the current frame of point cloud data is returned to be executed.

[0044] In a second aspect, the present invention further provides a fall detection device based on laser point cloud, comprising:

[0045] A point cloud data acquisition module is used to acquire multiple frames of point cloud data including the current frame point cloud data;

[0046] A key part identification module, for identifying key parts of a human body from the point cloud data for each frame of point cloud data;

[0047] A key point determination module, used to determine the key points of the key parts;

[0048] A displacement calculation module is used to calculate the vertical displacement of the key point within the duration of multiple frames of point cloud data;

[0049] A judging module, used to judge whether the displacement is greater than or equal to a threshold;

[0050] The fall alarm module is used to determine that a fall event has occurred and issue a fall alarm when the displacement is greater than or equal to a threshold.

[0051] In a third aspect, the present invention further provides an electronic device, comprising:

[0052] one or more processors;

[0053] A storage device for storing one or more programs;

[0054] When the one or more programs are executed by the one or more processors, the one or more processors implement the laser point cloud-based fall detection method provided in the first aspect of the present invention.

[0055] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the laser point cloud-based fall detection method provided in the first aspect of the present invention.

[0056] The fall detection method based on laser point cloud provided by the present invention obtains multiple frames of point cloud data including the current frame point cloud data, identifies the key parts of the human body from the point cloud data for each frame of point cloud data, determines the key points of the key parts, calculates the vertical displacement of the key points within the duration of the multiple frames of point cloud data, and determines whether the displacement is greater than or equal to the threshold. When the displacement is greater than or equal to the threshold, it is determined that a fall event has occurred, and a fall alarm is issued, which can achieve non-sensing detection and is easily accepted by the elderly. The point cloud data will not collect human biological features, and there is no risk of privacy leakage. In addition, by identifying the key parts of the human body and judging whether a fall event has occurred based on the vertical displacement of the key parts of the key parts, there is no need to model the fall action of the entire human body, thereby improving the detection efficiency.

[0057] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 A flow chart of a fall detection method based on laser point cloud provided by the present invention;

[0060] Figure 2 A schematic diagram of the structure of a key part identification model provided by the present invention;

[0061] Figure 3 A schematic diagram of an embedding process provided by the present invention;

[0062] Figure 4 A processing flow chart of a self-attention unit provided by the present invention;

[0063] Figure 5 A schematic diagram of the structure of a fall detection device based on laser point cloud provided by the present invention;

[0064] Figure 6 The present invention provides a schematic structural diagram of an electronic device.

[0065] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0066] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0067] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0068] Figure 1 This is a flow chart of a fall detection method based on laser point cloud provided by the present invention. This embodiment can be applied to human fall detection using laser radar, and is particularly applicable to human fall detection in a privacy space. The method can be executed by a fall detection device based on laser point cloud provided by an embodiment of the present invention. The device can be implemented by software and / or hardware, and is usually configured in an electronic device, such as Figure 1 As shown, the fall detection method based on laser point cloud may include the following steps:

[0069] S101, acquiring multiple frames of point cloud data including the current frame of point cloud data.

[0070] Exemplarily, in an embodiment of the present invention, a laser radar can be set indoors, for example, in a bedroom, bathroom, toilet, living room, etc., and the embodiment of the present invention is not limited here. The laser radar continuously emits lasers, scans the surrounding environment, and receives lasers reflected by external objects to obtain point cloud data. Specifically, the laser radar emits a laser beam including multiple (for example, 32 lines or 64 lines) lasers, and receives reflected lasers reflected back from the surface of the object. The laser radar accurately measures the propagation time of the laser from emission to reflection. Given that the speed of light is known, the propagation time can be converted into a measurement of distance. Combined with the height of the laser radar and the laser scanning angle, the three-dimensional coordinates of each light spot (i.e., a point in the point cloud) in the coordinate system where the laser radar is located can be accurately calculated.

[0071] In an embodiment of the present invention, the electronic device (which may be a server) continuously receives point cloud data uploaded by the laser radar. When receiving the current frame point cloud data, multiple frames of point cloud data including the current frame point cloud data (i.e., the current frame point cloud data and the previous multiple frames of point cloud data) are obtained.

[0072] S102: For each frame of point cloud data, identify key parts of the human body from the point cloud data.

[0073] In an embodiment of the present invention, for each frame of point cloud data, semantic segmentation can be used to identify key parts of the human body from the point cloud data. Exemplarily, the key part can be the head. During a fall, the displacement change of the head in the vertical direction is the most obvious. Therefore, the head is selected as the key part. Of course, in other embodiments of the present invention, other parts can also be selected as key parts, such as the torso, which is not limited in the embodiment of the present invention.

[0074] In some embodiments of the present invention, the point cloud data may be processed in combination with a deep learning algorithm to identify key parts of a human body. Exemplarily, for each frame of point cloud data, identifying key parts of a human body from the point cloud data includes:

[0075] S1021. Eliminate environmental point cloud data from the point cloud data, and retain human point cloud data.

[0076] For each frame of point cloud data, the environmental point cloud data is removed from the point cloud data, and the human point cloud data is retained. Exemplarily, the laser radar can scan the environment when no one is indoors, obtain environmental point cloud data, and pre-store it in the electronic device. During the detection process, it is only necessary to compare the received point cloud data with the pre-stored environmental point cloud data and remove the environmental point cloud data from the point cloud data. After removing the environmental point cloud data, the human point cloud data is retained, avoiding the influence of the environmental point cloud data on subsequent detection results. At the same time, the amount of data is reduced and the processing efficiency is improved.

[0077] S1022, encoding the human body point cloud data to obtain encoding features that represent the semantic similarity between points in the human body point cloud data.

[0078] In an embodiment of the present invention, the human body point cloud data is encoded and converted into high-dimensional encoding features, which can characterize the semantic similarity between points in the human body point cloud data. Exemplarily, the disordered human body point cloud data can be vectorized first to obtain a vector representation (or feature) of the human body point cloud data, and then the vector can be input into the encoder for encoding processing to obtain encoding features that characterize the semantic similarity between points in the human body point cloud data. The encoder can adopt encoder structures and other structures commonly used in the field of deep learning, and the embodiments of the present invention are not limited here.

[0079] Figure 2 A schematic diagram of the structure of a key part identification model provided by the present invention, such as Figure 2 As shown, the key part recognition model mainly includes an encoding part (Encoder) and a semantic recognition part (Segmentation), the encoding part is used to encode the human body point cloud data, and the semantic recognition part is used to recognize the key parts. Figure 2, the process of encoding the human body point cloud data and obtaining the encoding features is as follows:

[0080] 1. Embed the human point cloud data and convert it into embedded features.

[0081] refer to Figure 2 , the human point cloud data is input into an embedding layer (Embedding) for embedding processing, and the discrete and non-continuous human point cloud data is converted into continuous embedding features (i.e., vectors or matrices).

[0082] Figure 3 A schematic diagram of an embedding process provided by the present invention, such as Figure 3 As shown, the embedding process is:

[0083] 1.1. Linearize, batch normalize and nonlinearly activate the human point cloud data to obtain regularized human point cloud features.

[0084] The human body point cloud data is input into an LBR unit for processing. The LBR unit includes a linearization layer, a batch normalization layer, and a Rule function activation layer connected in sequence. The human body point cloud data is processed by linearization, then batch normalization, and finally the Rule function is used for nonlinear activation to obtain regularized human body point cloud features.

[0085] 1.2. Downsample the regularized human body point cloud features to obtain downsampled point cloud features.

[0086] The regularized human body point cloud features are downsampled (Sampling) to obtain the downsampled point cloud features P1. Exemplarily, in some embodiments of the present invention, the far point sampling algorithm (FPS) can be used for downsampling. The far point sampling algorithm is an algorithm for downsampling point cloud data. Its purpose is to select a subset from a large-scale point set so that the point cloud in the subset can represent the shape characteristics of the original point cloud as much as possible. The advantage of the far point sampling algorithm is that it can maintain the coverage and shape characteristics of the point cloud because it gives priority to those points that are relatively isolated geometrically. This makes the algorithm particularly suitable for non-uniformly distributed point cloud data, especially when the point cloud contains details and is unevenly distributed.

[0087] 1.3. For each sampling point in the downsampled point cloud features, determine the sampling point from the regularized human body point cloud features.

[0088] In the embodiment of the present invention, for each sampling point in the downsampled point cloud feature P1, the regularized human body point cloud feature is traced back to determine the sampling point from the regularized human body point cloud feature.

[0089] 1.4. Calculate the K nearest points to the sampling point from the regularized human body point cloud features and determine the K neighborhood features of each sampling point.

[0090] The K nearest points to the sampling point are calculated from the regularized human body point cloud features, and the K neighborhood features of each sampling point are determined. For example, Figure 3 As shown, the K-nearest neighbor algorithm (KNN) can be used to calculate the K neighboring points closest to the sampling point from the regularized human body point cloud features, and determine the K-neighborhood feature P2 of each sampling point.

[0091] 1.5. Fuse the downsampled point cloud features and the K-neighborhood features of all sampling points to obtain the neighborhood fusion features.

[0092] In the embodiment of the present invention, the downsampled point cloud features and the K-neighborhood features of all sampling points are fused to obtain neighborhood fusion features. For example, Figure 3 As shown in FIG. 1 , the downsampled point cloud features are replicated (Repeat) K times to obtain a matrix with the same spatial dimension as the matrix composed of the K neighborhood features of all sampling points. Then, the matrix is ​​added to the matrix composed of the K neighborhood features of all sampling points, and then concatenated with the features after being replicated K times to obtain the neighborhood fusion feature P3.

[0093] 1.6. Linearize, batch normalize and nonlinearly activate the neighborhood fusion features to obtain the embedded features of the human point cloud data.

[0094] In the embodiment of the present invention, the neighborhood fusion features are linearized, batch normalized and nonlinearly activated to obtain the embedded features of the human body point cloud data. For example, Figure 3 As shown in Figure 1, the neighborhood fusion features are input into an LBR unit for linearization, batch normalization, and nonlinear activation to obtain the embedded features P4 of the human point cloud data.

[0095] In the embodiment of the present invention, the embedded features of the human body point cloud data fuse the K-neighborhood features of the sampling points and introduce local feature information, which is beneficial to improving the accuracy of subsequent key part recognition.

[0096] 2. The embedded features are processed based on the self-attention mechanism to obtain attention features of multiple different feature scales.

[0097] In the embodiment of the present invention, the embedded features are processed based on the self-attention mechanism to capture the correlation between the elements in the embedded features, and obtain attention features of multiple different feature scales, which is conducive to improving the accuracy of subsequent key part recognition. For example, Figure 2As shown in the figure, the embedded feature is input into the self-attention module for processing. The self-attention module includes multiple cascaded self-attention units Attention. The input feature of the first self-attention unit is the embedded feature, the input feature of the i-th self-attention unit is the output feature of the i-1-th self-attention unit, and the output feature of each self-attention unit is also used as the attention feature output by the self-attention unit. Each self-attention unit outputs attention features of different feature scales, and i is a positive integer greater than 1.

[0098] Each self-attention unit can process the input features using the common self-attention processing method, namely: perform linear transformation on the input features respectively to obtain the query matrix, key matrix and value matrix, calculate the dot product of the query matrix and the key matrix and normalize them to obtain the attention weight matrix, calculate the dot product of the attention weight matrix and the value matrix to obtain the self-attention matrix, and finally calculate the sum of the self-attention matrix and the input features to obtain the output features of the self-attention unit.

[0099] In the embodiment of the present invention, in order to better capture the details and local geometric structures in the point cloud and thus improve the accuracy of subsequent key part recognition, the self-attention unit is improved. Figure 4 A processing flow chart of a self-attention unit provided by the present invention, such as Figure 4 As shown in Figure 2, the processing process of the self-attention unit is:

[0100] (1) Perform linear transformation on the input features to obtain the query matrix, key matrix and value matrix.

[0101] In the embodiment of the present invention, firstly, three linear transformations are performed on the input features to obtain a query matrix Query, a key matrix Key, and a value matrix Value, respectively.

[0102] (2) Calculate the dot product of the query matrix and the key matrix and normalize them to obtain the attention weight matrix.

[0103] For example, Figure 4 As shown in Figure 1, after calculating the dot product of the query matrix and the key matrix, the dot product is input into an LBR unit for linearization, batch normalization, and nonlinear activation to obtain the attention weight matrix.

[0104] (3) Calculate the dot product of the attention weight matrix and the value matrix to obtain the self-attention matrix.

[0105] For example, Figure 4 As shown, the dot product of the attention weight matrix and the value matrix is ​​calculated to obtain the self-attention matrix.

[0106] (4) Calculate the difference between the input features and the self-attention matrix to obtain the deviation matrix.

[0107] For example, Figure 4 As shown in Figure 1, the difference between the input feature and the self-attention matrix is ​​calculated to obtain the deviation matrix. The deviation matrix represents the offset between the input feature and the self-attention matrix.

[0108] (5) Linearize, batch normalize and nonlinearly activate the bias matrix to obtain the bias compensation matrix of the input features.

[0109] For example, Figure 4 As shown, the deviation matrix is ​​input into an LBR unit for linearization, batch normalization and nonlinear activation to obtain the deviation compensation matrix of the input features.

[0110] (6) Calculate the sum of the input features and the bias compensation matrix to obtain the output features of the self-attention unit.

[0111] For example, Figure 4 As shown in Figure 1, the sum of the input features and the bias compensation matrix is ​​calculated to obtain the output features of the self-attention unit. That is, the calculated bias compensation matrix is ​​used to adjust the input features, so that the model can better capture the details and local geometric structures in the point cloud, thereby improving the accuracy of subsequent key part recognition.

[0112] 3. Concatenate multiple attention features of different feature scales in the spatial dimension to obtain concatenated features.

[0113] Continue to refer Figure 2 , the attention features of multiple different feature scales output by multiple self-attention units are concatenated in the spatial dimension (represented by C in the figure) to obtain the concatenated features.

[0114] 4. Linearize, batch normalize and nonlinearly activate the concatenated features to obtain the encoding features that represent the semantic similarity between points in the human point cloud data.

[0115] like Figure 2 As shown, the concatenated features are input into an LBR unit for linearization, batch normalization, and nonlinear activation to obtain encoding features that represent the semantic similarity between points in the human point cloud data.

[0116] S1023. Perform pooling processing on the encoded features to obtain global features of the human body point cloud data.

[0117] like Figure 2As shown in the semantic recognition part, the encoded features are pooled (MA-Pooling) to obtain the global features of the human point cloud data. Exemplarily, in an embodiment of the present invention, the encoded features are first input into a maximum pooling layer (Max-Pooling), and the output result of the maximum pooling layer is input into an average pooling layer (Average-Pooling), and finally the global features of the human point cloud data are obtained.

[0118] S1024: Concatenate the coded features and the global features in the spatial dimension to obtain fused features.

[0119] In the embodiment of the present invention, the coding feature and the global feature are concatenated in the spatial dimension to obtain a fused feature. Since the coding feature contains a large amount of local feature information, the fused feature finally obtained combines the local feature information and the global feature information, which is conducive to improving the recognition accuracy of subsequent key parts.

[0120] S1025. Perform semantic segmentation on the fused features to identify key parts of the human body.

[0121] like Figure 2 As shown, the fused features are semantically segmented to identify the key parts of the human body. Exemplarily, the fused features are input into an LBRD unit for processing, the LBRD unit includes a linearization layer (Linear), a batch normalization layer (Batch Normalization), a Rule function activation layer and a dropout layer (Dropout) connected in sequence, and the features output by the LBRD unit are then sent to an LBR unit for processing, the LBR unit includes a linearization layer (Linear), a batch normalization layer (Batch Normalization) and a Rule function activation layer connected in sequence, and the features output by the LBR unit are then sent to the linearization layer (Linear) for processing to obtain various parts of the human body, and determine the key parts of the human body, such as the head.

[0122] S103, determine the key points of the key parts.

[0123] After identifying the key parts of the human body, determine the key points of the key parts. Exemplarily, the geometric center or centroid of the key parts can be calculated as the key points of the key parts, and the embodiments of the present invention are not limited here. Exemplarily, in a specific embodiment of the present invention, the point cloud data of the key parts are clustered, and the cluster center of the point cloud data of the key parts is calculated, and the cluster center is used as the key point of the key parts. The clustering algorithm can adopt K-means, FCM, etc., and the embodiments of the present invention are not limited here.

[0124] S104, calculating the vertical downward displacement of the key point within the duration of the multi-frame point cloud data.

[0125] Exemplarily, each frame of point cloud data carries a timestamp, and the duration of the multi-frame point cloud data can be calculated based on the difference between the timestamps of the last frame and the first frame in the multi-frame point cloud data. Based on the coordinate information of the key points in the multi-frame point cloud data in the point cloud coordinates, after coordinate system conversion, the coordinate information of the key points in the multi-frame point cloud data in the world coordinates is calculated, and the coordinate value in the vertical direction is determined, and then the vertical downward displacement of the key points is calculated based on the coordinate value in the vertical direction.

[0126] S105: Determine whether the displacement is greater than or equal to a threshold.

[0127] In the embodiment of the present invention, the calculated vertical downward displacement of the key point is compared with a preset threshold to determine whether the displacement is greater than or equal to the threshold.

[0128] S106: Determine that a fall event has occurred and issue a fall alarm.

[0129] When the displacement is greater than or equal to the threshold, a fall event is determined to have occurred, and a fall alarm is sent to the guardian's mobile terminal so that the guardian can discover it in time and arrive at the scene for treatment.

[0130] The prior art has a method of identifying various parts of the human body, establishing a human motion model through the key points of each part, and analyzing whether a fall event has occurred through the motion model. However, the modeling process has a large amount of calculation, so the detection efficiency is low. The present invention identifies the key parts of the human body and determines whether a fall event has occurred based on the vertical displacement of the key parts of the key parts. There is no need to perform motion modeling on the entire human body, thereby improving the detection efficiency.

[0131] S107, receiving the next frame of point cloud data as the current frame of point cloud data.

[0132] When the displacement is less than the threshold, it is considered that no fall event has occurred, and the next frame of point cloud data is continued to be received as the current frame of point cloud data, and the step of obtaining multiple frames of point cloud data including the current frame of point cloud data is returned to continue the above fall detection process.

[0133] The fall detection method based on laser point cloud provided by the present invention obtains multiple frames of point cloud data including the current frame point cloud data, identifies the key parts of the human body from the point cloud data for each frame of point cloud data, determines the key points of the key parts, calculates the vertical displacement of the key points within the duration of the multiple frames of point cloud data, and determines whether the displacement is greater than or equal to the threshold. When the displacement is greater than or equal to the threshold, it is determined that a fall event has occurred, and a fall alarm is issued, which can achieve non-sensing detection and is easily accepted by the elderly. The point cloud data will not collect human biological features, and there is no risk of privacy leakage. In addition, by identifying the key parts of the human body and judging whether a fall event has occurred based on the vertical displacement of the key parts of the key parts, there is no need to model the fall action of the entire human body, thereby improving the detection efficiency.

[0134] Figure 5 A schematic diagram of a fall detection device based on laser point cloud provided by the present invention is shown in FIG. Figure 5 As shown, the fall detection device based on laser point cloud includes:

[0135] The point cloud data acquisition module 201 is used to acquire multiple frames of point cloud data including the current frame of point cloud data;

[0136] A key part identification module 202, for identifying key parts of a human body from each frame of point cloud data;

[0137] A key point determination module 203 is used to determine the key points of the key parts;

[0138] The displacement calculation module 204 is used to calculate the vertical displacement of the key point within the duration of multiple frames of point cloud data;

[0139] A judging module 205 is used to judge whether the displacement is greater than or equal to a threshold;

[0140] The fall alarm module 206 is used to determine that a fall event occurs and issue a fall alarm when the displacement is greater than or equal to a threshold.

[0141] In some embodiments of the present invention, the key part identification module 202 includes:

[0142] A point cloud elimination submodule, used to eliminate environmental point cloud data from the point cloud data and retain human point cloud data;

[0143] The encoding submodule is used to encode the human body point cloud data to obtain encoding features that characterize the semantic similarity between points in the human body point cloud data;

[0144] A pooling processing submodule, used for performing pooling processing on the coded features to obtain global features of the human body point cloud data;

[0145] A feature splicing submodule, used for splicing the encoding feature and the global feature in the spatial dimension to obtain a fusion feature;

[0146] The semantic segmentation submodule is used to perform semantic segmentation on the fused features to identify key parts of the human body.

[0147] In some embodiments of the present invention, the encoding submodule includes:

[0148] An embedding processing unit, used for performing embedding processing on the human body point cloud data to convert the human body point cloud data into embedding features;

[0149] An attention processing unit, used for processing the embedded features based on a self-attention mechanism to obtain attention features of multiple different feature scales;

[0150] A feature concatenation unit is used to concatenate multiple attention features of different feature scales in the spatial dimension to obtain a concatenated feature;

[0151] The encoding feature determination unit is used to perform linearization, batch normalization and nonlinear activation on the splicing features to obtain encoding features that characterize the semantic similarity between points in the human body point cloud data.

[0152] In some embodiments of the invention, the embedded processing unit comprises:

[0153] A regularization subunit, used for performing linearization, batch normalization and nonlinear activation on the human body point cloud data to obtain regularized human body point cloud features;

[0154] A downsampling subunit, used for downsampling the regularized human body point cloud features to obtain downsampled point cloud features;

[0155] A sampling point determination subunit, configured to determine, for each sampling point in the downsampled point cloud feature, the sampling point from the regularized human body point cloud feature;

[0156] A neighborhood calculation subunit, used to calculate the K neighboring points closest to the sampling point from the regularized human body point cloud features, and determine the K neighborhood features of each of the sampling points;

[0157] A feature fusion subunit, used to fuse the downsampled point cloud features and the K-neighborhood features of all the sampling points to obtain a neighborhood fusion feature;

[0158] The embedded feature determination subunit is used to perform linearization, batch normalization and nonlinear activation on the neighborhood fusion features to obtain the embedded features of the human body point cloud data.

[0159] In some embodiments of the invention, the attention processing unit comprises:

[0160] An attention processing subunit is used to input the embedded feature into a self-attention module for processing. The self-attention module includes multiple cascaded self-attention units. The input feature of the first self-attention unit is the embedded feature, the input feature of the i-th self-attention unit is the output feature of the i-1-th self-attention unit, and each of the self-attention units is used to output attention features of different feature scales, where i is a positive integer greater than 1.

[0161] In some embodiments of the present invention, the processing process of the self-attention unit is:

[0162] Perform linear transformation on the input features to obtain the query matrix, key matrix and value matrix;

[0163] Calculate the dot product of the query matrix and the key matrix and normalize them to obtain an attention weight matrix;

[0164] Calculate the dot product of the attention weight matrix and the value matrix to obtain a self-attention matrix;

[0165] Calculate the difference between the input feature and the self-attention matrix to obtain a deviation matrix;

[0166] Linearizing, batch normalizing, and nonlinearly activating the deviation matrix to obtain a deviation compensation matrix of the input feature;

[0167] The sum of the input feature and the bias compensation matrix is ​​calculated to obtain the output feature of the self-attention unit.

[0168] In some embodiments of the present invention, the key point determination module 203 includes:

[0169] A clustering submodule, used to cluster the point cloud data of the key parts and calculate the cluster center of the point cloud data of the key parts;

[0170] The key point determination submodule is used to use the cluster center as the key point of the key part.

[0171] In some embodiments of the present invention, the fall detection device based on laser point cloud further includes:

[0172] The return execution module is used to receive the next frame of point cloud data as the current frame of point cloud data when the displacement is less than the threshold, and return to execute the step of obtaining multiple frames of point cloud data including the current frame of point cloud data.

[0173] The above-mentioned fall detection device based on laser point cloud can execute the fall detection method based on laser point cloud provided by the aforementioned embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the fall detection method based on laser point cloud.

[0174] Figure 6 A schematic diagram of the structure of an electronic device provided by the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0175] like Figure 6 As shown, the electronic device includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0176] A number of components in the electronic device are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0177] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a fall detection method based on a laser point cloud.

[0178] In some embodiments, the laser point cloud-based fall detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the laser point cloud-based fall detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the laser point cloud-based fall detection method in any other appropriate manner (e.g., by means of firmware).

[0179] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0180] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0181] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0182] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0183] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0184] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0185] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements a fall detection method based on a laser point cloud as provided in any embodiment of the present application.

[0186] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0187] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0188] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A fall detection method based on laser point cloud, characterized in that: include: Obtain multi-frame point cloud data including the current frame point cloud data; For each frame of point cloud data, identifying key parts of a human body from the point cloud data; Determine the key points of the key parts; Calculate the vertical displacement of the key point within the duration of multiple frames of point cloud data; Determining whether the displacement is greater than or equal to a threshold; When the displacement is greater than or equal to a threshold, it is determined that a fall event has occurred, and a fall alarm is issued.

2. The fall detection method based on laser point cloud according to claim 1, characterized in that: For each frame of point cloud data, key parts of the human body are identified from the point cloud data, including: Eliminating environmental point cloud data from the point cloud data and retaining human body point cloud data; Performing encoding processing on the human body point cloud data to obtain encoding features representing the semantic similarity between points in the human body point cloud data; Performing pooling processing on the encoded features to obtain global features of the human body point cloud data; Concatenate the encoding feature and the global feature in the spatial dimension to obtain a fusion feature; The fused features are semantically segmented to identify key parts of the human body.

3. The fall detection method based on laser point cloud according to claim 2, characterized in that: The human body point cloud data is encoded to obtain encoding features representing the semantic similarity between points in the human body point cloud data, including: Performing embedding processing on the human body point cloud data to convert the human body point cloud data into embedding features; Processing the embedded features based on a self-attention mechanism to obtain attention features of multiple different feature scales; Concatenate multiple attention features of different feature scales in the spatial dimension to obtain concatenated features; The concatenated features are linearized, batch normalized and nonlinearly activated to obtain encoding features that characterize the semantic similarity between points in the human body point cloud data.

4. The fall detection method based on laser point cloud according to claim 3, characterized in that: Embedding the human body point cloud data to convert the human body point cloud data into embedded features includes: Performing linearization, batch normalization, and nonlinear activation on the human body point cloud data to obtain regularized human body point cloud features; Downsampling the regularized human body point cloud features to obtain downsampled point cloud features; For each sampling point in the downsampled point cloud features, determining the sampling point from the regularized human body point cloud features; Calculate the K neighboring points closest to the sampling point from the regularized human body point cloud features, and determine the K neighborhood features of each sampling point; Fusing the downsampled point cloud features with K-neighborhood features of all the sampling points to obtain neighborhood fusion features; The neighborhood fusion features are linearized, batch normalized and nonlinearly activated to obtain embedded features of the human body point cloud data.

5. The fall detection method based on laser point cloud according to claim 3, characterized in that: The embedded features are processed based on the self-attention mechanism to obtain multiple attention features of different feature scales, including: The embedded feature is input into a self-attention module for processing. The self-attention module includes multiple cascaded self-attention units, the input feature of the first self-attention unit is the embedded feature, the input feature of the i-th self-attention unit is the output feature of the i-1-th self-attention unit, and each of the self-attention units is used to output attention features of different feature scales, where i is a positive integer greater than 1.

6. The fall detection method based on laser point cloud according to claim 5, characterized in that: The processing process of the self-attention unit is: Perform linear transformation on the input features to obtain the query matrix, key matrix and value matrix; Calculate the dot product of the query matrix and the key matrix and normalize them to obtain an attention weight matrix; Calculate the dot product of the attention weight matrix and the value matrix to obtain a self-attention matrix; Calculate the difference between the input feature and the self-attention matrix to obtain a deviation matrix; Linearizing, batch normalizing, and nonlinearly activating the deviation matrix to obtain a deviation compensation matrix of the input feature; The sum of the input feature and the bias compensation matrix is ​​calculated to obtain the output feature of the self-attention unit.

7. The fall detection method based on laser point cloud according to any one of claims 1 to 6, characterized in that: Determine the key points of the key parts, including: Clustering the point cloud data of the key parts, and calculating the cluster center of the point cloud data of the key parts; The cluster center is used as the key point of the key part.

8. The fall detection method based on laser point cloud according to any one of claims 1 to 6, characterized in that: Also includes: When the displacement is less than the threshold, the next frame of point cloud data is received as the current frame of point cloud data, and the step of obtaining multiple frames of point cloud data including the current frame of point cloud data is returned to be executed.

9. A fall detection device based on laser point cloud, characterized in that: include: A point cloud data acquisition module is used to acquire multiple frames of point cloud data including the current frame point cloud data; A key part identification module, for identifying key parts of a human body from the point cloud data for each frame of point cloud data; A key point determination module, used to determine the key points of the key parts; A displacement calculation module is used to calculate the vertical displacement of the key point within the duration of multiple frames of point cloud data; A judging module, used to judge whether the displacement is greater than or equal to a threshold; The fall alarm module is used to determine that a fall event has occurred and issue a fall alarm when the displacement is greater than or equal to a threshold.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the laser point cloud-based fall detection method as described in any one of claims 1-8.

Images