Fall detection model training based on radar data, fall detection method and device

Abstract

The present application discloses a training method for a fall detection model based on radar data, which obtains radar spectrum data including multiple frames of radar signals collected by the radar, and the radar spectrum data corresponds to a classification label of fall or non-fall. According to the signal strength of each frame of radar signal, radar signals greater than a first preset threshold are extracted from the radar spectrum data to form a torso movement interval, and the starting position and / or end position of the torso movement interval are corrected to generate an action interval. The classification label corresponding to each frame of radar signal in the action interval is determined according to the classification label corresponding to the radar data. A plurality of action intervals are spliced ​​into training data, and the training data includes a preset number of frames of radar signals and each frame of radar signal corresponds to a classification label. The fall detection model is trained based on the training data, and the fall detection model can predict the classification result of each frame of radar signal. In this way, the obtained fall detection model has good real-time performance and high accuracy.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a training method for a fall detection model based on radar data, a fall detection method, device and equipment based on radar data. Background Art

[0002] With the development of society and economy, more and more people are concerned about their health. Falling can cause serious damage to the body. Therefore, when a user falls, especially the elderly, they hope to be notified of the user's fall in a timely manner so that relevant personnel can be notified quickly and help can be provided.

[0003] Currently, trained fall detection models can be used to detect whether a user has fallen. However, these models suffer from poor real-time performance, making them difficult to meet the demands of real-time and high-precision applications, and thus failing to meet user needs. Summary of the Invention

[0004] In order to solve the above problems, the embodiments of the present application provide a training method for a fall detection model based on radar data, a fall detection method, device and equipment based on radar data, which can detect whether a user falls in real time to meet user needs.

[0005] To achieve the above objectives, the technical solutions provided in the embodiments of the present application are as follows:

[0006] An embodiment of the present application provides a method for training a fall detection model based on radar data, the method comprising:

[0007] Acquire radar spectrum data, the radar spectrum data including multiple frames of radar signals, the radar spectrum data corresponding to classification labels, the classification labels including fall or non-fall;

[0008] Extracting radar signals with signal strength greater than a first preset threshold from the radar spectrum data to form a torso movement interval;

[0009] Correcting the starting position and / or ending position of the trunk movement interval to generate a movement interval;

[0010] Determine, according to the classification label corresponding to the radar spectrum data, a classification label corresponding to each frame of radar signal in the action interval;

[0011] splicing the plurality of action intervals into training data, wherein the training data includes a preset number of radar signal frames, and each radar signal frame of the training data corresponds to a classification label;

[0012] The training data is input into the fall detection model to obtain the predicted classification result corresponding to each frame of the radar signal of the training data. The loss value is calculated based on the predicted classification result corresponding to each frame of the radar signal of the training data and the classification label corresponding to each frame of the radar signal of the training data. The fall detection model is adjusted according to the loss value. The training data is input into the fall detection model to obtain the predicted classification result corresponding to each frame of the radar signal of the training data and subsequent steps are repeatedly performed until the preset conditions are met to generate a trained fall detection model; the fall detection model includes a frame feature attention extraction network, an LSTM timing network and a timing output network connected in sequence.

[0013] In one possible implementation, the frame feature attention extraction network includes a first fully connected layer, the LSTM timing network includes a second fully connected network, a third fully connected network and a two-layer LSTM network, and the timing output network includes a fourth fully connected network and a fifth fully connected network;

[0014] The training data is input into the first fully-connected layer, the output of the first fully-connected layer is multiplied by the training data and then input into the second fully-connected network, the output of the second fully-connected network is input into the third fully-connected network, the output of the third fully-connected network is input into the two-layer LSTM network, the output of the two-layer LSTM network is input into the fourth fully-connected network, and the output of the fourth fully-connected network is input into the fifth fully-connected network to obtain a prediction classification result corresponding to each frame of radar signal of the training data.

[0015] In a possible implementation, the fall detection model further includes an attention transfer network, and the attention transfer network includes the sixth fully connected network and the seventh fully connected network;

[0016] The output of the first fully connected layer is multiplied by the training data and then input into the sixth fully connected network. The output of the sixth fully connected network is input into the seventh fully connected network. The output of the seventh fully connected network is multiplied by the output of the two-layer LSTM network and then input into the fourth fully connected network.

[0017] In a possible implementation, calculating the loss value according to the predicted classification result corresponding to each frame of radar signal in the training data and the classification label corresponding to each frame of radar signal in the training data includes:

[0018] The loss value is calculated based on the predicted classification result corresponding to each frame of radar signal in the training data, the classification label corresponding to each frame of radar signal in the training data, and the loss value weight of the classification label; wherein the loss value weight of the fall classification label is greater than the loss value weight of the non-fall classification label.

[0019] In a possible implementation, the modifying the starting position and / or ending position of the torso motion interval to generate the motion interval includes:

[0020] Correcting the starting position and / or ending position of the trunk movement interval according to a movement start and end point correction algorithm to generate a first movement interval;

[0021] and / or,

[0022] Correcting the starting position and / or ending position of the trunk motion interval according to a morphological start-end point correction algorithm to generate a second motion interval;

[0023] When only the first action interval is generated, the first action interval is determined as the action interval; when only the second action interval is generated, the second action interval is determined as the action interval; when the first action interval and the second action interval are generated at the same time, the average value of the starting positions of the first action interval and the second action interval is determined as the starting position of the action interval, and the average value of the ending positions of the first action interval and the second action interval is determined as the ending position of the action interval.

[0024] In a possible implementation, the step of correcting the starting position and / or ending position of the torso motion interval according to the motion start and end point correction algorithm to generate the first motion interval includes:

[0025] When the classification label corresponding to the radar spectrum data is fall, a random position is selected in a first interval from the starting position of the radar spectrum data to the first target position as the starting position of the first action interval, and / or a random position is selected in a second interval from the ending position of the torso action interval to the ending position of the radar spectrum data as the ending position of the first action interval;

[0026] When the classification label corresponding to the radar spectrum data is non-fall, a random position is selected within a first interval from the starting position of the radar spectrum data to the first target position as the starting position of the first action interval, and / or a random position is selected within a third interval from the second target position to the ending position of the radar spectrum data as the ending position of the first action interval;

[0027] The first target position is a position that is a first preset distance behind the starting position of the torso movement interval; and the second target position is a position that is a second preset distance before the ending position of the torso movement interval.

[0028] In a possible implementation, the step of correcting the starting position and / or ending position of the trunk motion interval according to a morphological start-end point correction algorithm to generate a second motion interval includes:

[0029] In a fourth interval from a position a third preset distance before the starting position of the torso movement interval to a position a fourth preset distance after the starting position of the torso movement interval, a position where the first-order derivative of the signal strength is the largest is selected as the third target position; in a fifth interval from the third target position to the starting position of the torso movement interval, the first position where the second-order derivative of the signal strength is less than a second preset threshold is selected as the starting position of the second movement interval;

[0030] and / or,

[0031] In the sixth interval from the position of the fifth preset distance before the end position of the torso movement interval to the position of the sixth preset distance after the end position of the torso movement interval, the position where the first-order derivative of the signal strength is the largest is selected as the fourth target position; in the seventh interval from the fourth target position to the end position of the torso movement interval, the first position where the second-order derivative of the signal strength is less than the second preset threshold is selected as the end position of the second movement interval.

[0032] In a possible implementation, after modifying the starting position and / or ending position of the torso motion interval to generate the motion interval, the method further includes:

[0033] When the classification label corresponding to the radar spectrum data is fall, the eighth interval from the starting position of the action interval to the fifth target position and the ninth interval from the sixth target position to the end position of the action interval are removed from the action interval, and a new action interval is generated.

[0034] The present application also provides a radar data-based fall detection method, which includes:

[0035] Acquiring radar spectrum data to be classified, wherein the radar spectrum data to be classified includes a preset number of frames of radar signals;

[0036] The radar spectrum data to be classified is input into a fall detection model to obtain a classification result corresponding to each frame of radar signal of the radar spectrum data to be classified, wherein the classification result includes fall or non-fall; the fall detection model is trained according to the above-mentioned training method of the fall detection model based on radar data.

[0037] The present application also provides a radar data-based fall detection model training device, which includes:

[0038] an acquisition unit, configured to acquire radar spectrum data, the radar spectrum data including multiple frames of radar signals, the radar spectrum data corresponding to classification labels, the classification labels including fall or non-fall;

[0039] a composition unit, configured to extract radar signals having signal strength greater than a first preset threshold from the radar spectrum data to form a torso movement interval;

[0040] a correction unit, configured to correct the starting position and / or ending position of the trunk movement interval to generate a movement interval;

[0041] a determining unit, configured to determine a classification label corresponding to each frame of radar signal in the action interval according to the classification label corresponding to the radar spectrum data;

[0042] a splicing unit, configured to splice the plurality of motion intervals into training data, wherein the training data includes a preset number of radar signal frames, and each radar signal frame of the training data corresponds to a classification label;

[0043] A training unit is used to input the training data into a fall detection model, obtain a predicted classification result corresponding to each frame of radar signal of the training data, calculate a loss value based on the predicted classification result corresponding to each frame of radar signal of the training data and the classification label corresponding to each frame of radar signal of the training data, adjust the fall detection model based on the loss value, repeatedly input the training data into the fall detection model, obtain the predicted classification result corresponding to each frame of radar signal of the training data, and subsequent steps until a preset condition is met, thereby generating a trained fall detection model; the fall detection model includes a frame feature attention extraction network, an LSTM timing network, and a timing output network connected in sequence.

[0044] The present application also provides a radar data-based fall detection device, comprising:

[0045] A first acquiring unit is configured to acquire radar spectrum data to be classified, wherein the radar spectrum data to be classified includes a preset number of frames of radar signals;

[0046] A second acquisition unit is used to input the radar spectrum data to be classified into a fall detection model to obtain a classification result corresponding to each frame of radar signal of the radar spectrum data to be classified, wherein the classification result includes fall or non-fall; the fall detection model is trained according to the above-mentioned training method for the fall detection model based on radar data.

[0047] An embodiment of the present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the training method for a fall detection model based on radar data, or the fall detection method based on radar data, as described above, is implemented.

[0048] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes the training method of the fall detection model based on radar data as described above, or the fall detection method based on radar data as described above.

[0049] It can be seen that the embodiments of the present application have the following beneficial effects:

[0050] An embodiment of the present application provides a training method for a fall detection model based on radar data, which obtains radar spectrum data including multiple frames of radar signals collected by the radar, and the radar spectrum data corresponds to a classification label of fall or non-fall. According to the signal strength of each frame of radar signal, radar signals greater than a first preset threshold are extracted from the radar spectrum data to form a torso movement interval, and the starting position and / or end position of the torso movement interval is corrected to generate an action interval. The classification label corresponding to each frame of radar signal in the action interval is determined according to the classification label corresponding to the radar data. Based on this, a plurality of action intervals are spliced ​​into training data, and the training data includes a preset number of frames of radar signals and each frame of radar signal corresponds to a classification label. Furthermore, the fall detection model is trained and parameters are adjusted based on the training data, and the fall detection model can predict the classification result of each frame of radar signal. In this way, the trained fall detection model can detect the fall or non-fall classification result corresponding to each frame of radar signal. After improving the detection frequency and accuracy of the fall detection model to each sampling frame of the radar signal, it can detect whether the user has fallen in real time with high accuracy, which can meet the needs of actual scenarios and user demands. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of an exemplary application scenario provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of a method for training a fall detection model based on radar data provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a radar Doppler wave spectrum provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the data format of training data provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of a fall detection model provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of the structure of another fall detection model provided in an embodiment of the present application;

[0057] Figure 7a A schematic diagram of a torso movement interval correction provided in an embodiment of the present application;

[0058] Figure 7b Another schematic diagram of the correction of the trunk movement interval provided in an embodiment of the present application;

[0059] Figure 8 A flowchart of a fall detection method based on radar data provided in an embodiment of the present application;

[0060] Figure 9 A schematic diagram of the structure of a radar data-based fall detection model training device provided in an embodiment of the present application;

[0061] Figure 10 A schematic structural diagram of a radar data-based fall detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0063] In order to facilitate understanding and explanation of the technical solutions provided by the embodiments of the present application, the background technology of the present application will be described below.

[0064] With the development of society and economy, more and more people are concerned about their health. Falling can cause serious damage to the body, especially in the case of elderly people. Therefore, when a user falls, it is hoped that the user's falling action can be promptly notified so that relevant personnel can be quickly notified for assistance.

[0065] Currently, trained fall detection models can be used to detect whether a user has fallen. However, these models suffer from poor real-time performance, making them difficult to meet the demands of real-time and high-precision applications, and thus failing to meet user needs.

[0066] Based on this, an embodiment of the present application provides a training method for a fall detection model based on radar data, which obtains radar spectrum data including multiple frames of radar signals collected by the radar, and the radar spectrum data corresponds to a classification label of fall or non-fall. According to the signal strength of each frame of radar signal, radar signals greater than a first preset threshold are extracted from the radar spectrum data to form a torso movement interval, and the starting position and / or end position of the torso movement interval is corrected to generate an action interval. The classification label corresponding to each frame of radar signal in the action interval is determined according to the classification label corresponding to the radar data. Based on this, multiple action intervals are spliced ​​into training data, and the training data includes a preset number of frames of radar signals and each frame of radar signal corresponds to a classification label. Furthermore, the fall detection model is trained and parameters are adjusted based on the training data, and the fall detection model can predict the classification result of each frame of radar signal. In this way, the trained fall detection model can detect the fall or non-fall classification result corresponding to each frame of radar signal. After improving the detection frequency and accuracy of the fall detection model to each sampling frame of the radar signal, it can detect whether the user has fallen in real time with high accuracy, which can meet the needs of actual scenarios and user demands.

[0067] In order to facilitate understanding of the training method of the fall detection model based on radar data provided in the embodiment of the present application, the following Figure 1 See the example scenario shown. Figure 1 As shown in FIG, this figure is a schematic diagram of a framework of an exemplary application scenario provided by an embodiment of the present application. As an optional example, the method can be applied to a terminal device or a server.

[0068] In practical applications, radar spectrum data collected by a radar is obtained. Specifically, radar spectrum data includes only data on a single user action, which can be a fall or a non-fall action. Therefore, the radar spectrum data is assigned a classification label of fall or non-fall. Radar spectrum data includes multiple frames of radar signals.

[0069] To extract clearly indicative torso motion intervals, radar signals greater than a first preset threshold are extracted from the radar spectrum data based on the signal strength of each frame. Radar signals greater than the first preset threshold constitute torso motion intervals. Furthermore, to further refine the torso motion intervals, the starting and / or ending positions of the torso motion intervals are corrected to generate motion intervals. The motion intervals are the corrected torso motion intervals.

[0070] Based on the classification labels corresponding to the radar data, the classification labels corresponding to each radar signal frame within the action interval can be determined. After obtaining multiple corresponding action intervals and the classification labels corresponding to each radar signal frame within each action interval based on multiple sets of radar spectrum data, the multiple action intervals are concatenated to form training data. The training data includes a preset number of radar signal frames, each with a corresponding classification label.

[0071] The training data is then input into the fall detection model, and the model outputs a predicted classification result for each radar signal frame of the training data. A loss value is calculated based on the predicted classification result and the classification label for each radar signal frame of the training data, and the model parameters of the fall detection model are adjusted based on the loss value. As an optional example, the fall detection model includes a sequentially connected frame feature attention extraction network, an LSTM time series network, and a time series output network.

[0072] If the pre-set conditions are met during training, a trained fall detection model is obtained. If the pre-set conditions are not met, the training data is input into the fall detection model, and the predicted classification results corresponding to each frame of radar signal in the training data are obtained, along with the subsequent steps. The trained fall detection model is then obtained until the pre-set conditions are met.

[0073] Those skilled in the art will understand that Figure 1 The framework diagram shown is only an example in which the embodiments of the present application can be implemented. The scope of application of the embodiments of the present application is not limited by any aspect of the framework.

[0074] To facilitate understanding of the present application, a training method for a fall detection model based on radar data provided in an embodiment of the present application is described below with reference to the accompanying drawings.

[0075] See also Figure 2 As shown in FIG, this figure is a flow chart of a method for training a fall detection model based on radar data provided by an embodiment of the present application. The method can be applied to a terminal device or a server, and specifically, can be implemented by software in the terminal device or the server. Figure 2 As shown, the method may include S201-S206:

[0076] S201: Acquire radar spectrum data, where the radar spectrum data includes multiple frames of radar signals. The radar spectrum data has corresponding classification labels, where the classification labels include fall or non-fall.

[0077] Radar detects moving objects by transmitting signals of a certain frequency and receiving return signals. In this embodiment, radar detects users at a preset sampling frequency. Radar can only detect moving objects and provides almost no detection feedback for stationary objects. When the user is stationary, the radar can determine that the user is stationary based on the transmitted and returned signals, and the radar does not collect user motion signals. When the user is moving, the radar can determine that the user is moving based on the transmitted and returned signals, and the radar collects user motion signals. The user motion signal is the return signal received by the radar.

[0078] After the radar processes the received user motion signal, it generates radar spectrum data. Furthermore, training data is obtained based on the radar spectrum data, and the fall detection model in this embodiment is trained using this training data. The radar spectrum data can be in multiple groups, each group containing multiple frames of radar signals. As an optional example, a sampling frame is set every 30ms.

[0079] As an optional example, a Wi-Fi radar can be selected. The Wi-Fi radar can be installed at any location in the user's room that is convenient for collecting user movements, and there is no limitation here.

[0080] The following describes the specific process of obtaining radar spectrum data based on the user motion signal received by the radar.

[0081] The user motion signal received by the radar is an inherently complex time series, resulting from a time delay and frequency shift after the motion signal has been transmitted. The motion signal consists of signals returned from K parts of the user's body. The superposition of these signals can be used to approximate the scattered signal x[n] for the entire body. Body part information includes torso motion information and limb information.

[0082] The calculation formula for x[n] is as follows:

[0083]

[0084] Among them, R n,i is the distance range from the radar to the ith body part at time n, f c is the emission frequency, c is the speed of light, and j is the imaginary part. i is the square root of the received signal power, amplitude a i The calculation formula is as follows:

[0085]

[0086] Where G is the antenna gain, λ and P t are the wavelength and power of the transmitted signal, σ iis the radar cross section RCS of the i-th body part, L s and L a are system loss and atmospheric loss respectively.

[0087] See also Figure 3 , Figure 3 A schematic diagram of a radar Doppler spectrum provided in an embodiment of the present application. The scattered signal x[n] of the entire human body is processed and converted to obtain the following: Figure 3 The radar Doppler spectrum diagram shown. The radar Doppler signal spectrum data in the radar Doppler spectrum diagram is the input data form of the fall detection model of this embodiment. In this embodiment, the radar Doppler signal spectrum data is simply referred to as radar spectrum data.

[0088] like Figure 3 As shown in the figure, the data format of radar spectrum data is a two-dimensional matrix format. The horizontal axis is the time axis, the unit is ms, representing the sampling frame or time frame; the vertical axis is the Doppler frequency, the unit is HZ. Figure 3 As shown, when the user is moving, the Doppler frequency corresponding to the time frame in the radar Doppler spectrum has a specific value and a spectrum is displayed, indicating that the user is moving. If the Doppler frequency corresponding to the time frame in the radar Doppler spectrum does not have a specific value, the spectrum is not displayed, indicating that the user is stationary. The Doppler frequency is the difference between the frequency of the signal received by the radar and the frequency of the signal transmitted. The faster the user moves, the higher the Doppler frequency. The faster the movement changes, the faster the Doppler frequency changes. In addition, the Doppler frequency is positive when the user moves toward the radar, and negative when the user moves away from the radar.

[0089] In this embodiment, the radar spectrum data specifically includes only one user action. That is, a set of radar spectrum data does not include the user's second action. For example, a set of radar spectrum data has a size of 256×205, where 256 represents the time axis dimension of the radar spectrum data, consisting of multiple sampling frames of action data, and 256 is the number of sampling frames. 205 represents the length of the radar data vector for each 30ms sampling frame, and 205 also represents 205 frequency points, representing 205 different frequency values.

[0090] The color depth of each of the 205 frequency points in each sampling frame on the Doppler wave spectrum represents the signal strength of the point. The closer the color is to red, the stronger the reflected wave intensity is ( Figure 3 (The color image is not shown.) Furthermore, the signal strength is positively correlated with the volume of the object covered by the radar signal, or the radar cross section (RCS). For example, the larger the RCS, the higher the radar reflected wave energy intensity.

[0091] The action represented by the action data included in the radar spectrum data can be a fall action or a non-fall action. Therefore, the radar spectrum data corresponds to a classification label of fall or non-fall. Here, the fall action can be exemplarily understood as a fall action generated by the user, and the non-fall action can be exemplarily understood as a movement generated by the user but not a fall action.

[0092] S202: Extract radar signals with signal strength greater than a first preset threshold from the radar spectrum data to form a torso motion interval.

[0093] After acquiring radar spectrum data, it needs to be processed. Specifically, since body part information includes torso motion information and limb information, and the user's torso has the largest RCS, the signal strength of the limb information during movement is weaker than that of the torso. To improve the accuracy of the trained fall detection model, the limb information is removed from the movement, retaining only the torso motion information, which is a significant indicator of fall detection.

[0094] In a specific implementation, radar signals with signal strength greater than a first preset threshold are extracted from the radar spectrum data. Radar signals with signal strength greater than the first preset threshold are used to represent torso motion information. As an optional example, the first preset threshold is 80, and radar signals with signal strength greater than 80 are extracted from the radar spectrum data.

[0095] In one possible implementation, the present application provides a specific implementation method for extracting radar signals having signal strength greater than a first preset threshold from radar spectrum data to form a torso motion interval, including:

[0096] Selecting the maximum signal strength among multiple signal strengths of each frame of radar signal;

[0097] Comparing the maximum signal strength corresponding to each frame of radar signal with a first preset threshold frame by frame, obtaining a first frame of radar signal greater than the first preset threshold and a last frame of radar signal greater than the first preset threshold;

[0098] The interval between the first frame radar signal and the last frame radar signal is determined as the torso motion interval.

[0099] It can be understood that the maximum value of each column in each sampling frame with a height of 205 is first taken. That is, the maximum signal strength value among the 205 signal strengths of each frame of radar signal is selected. Each sampling frame has 205 frequency points, each frequency point corresponds to a signal strength value, and each sampling frame corresponds to the signal strength values ​​of 205 frequency points. Each sampling frame is a frame of radar signal.

[0100] Radar signals with signal strength greater than a first preset threshold indicate torso movements, while radar signals less than or equal to the first preset threshold indicate non-torso movements. The position of the first radar signal frame with signal strength greater than the first preset threshold is considered the starting position (starting time) of the torso movement, recorded as "original start". The position of the last radar signal frame with signal strength greater than the first preset threshold is considered the ending position (ending time) of the torso movement, recorded as "original end".

[0101] It is understandable that the interval between original start and original end [original start: original end] is determined as the trunk motion interval. The trunk motion waveform is composed of the maximum signal intensity corresponding to each sampling frame in the trunk motion interval.

[0102] S203: Correct the starting position and / or ending position of the trunk movement interval to generate a movement interval.

[0103] To make the trunk movement interval more accurate, the starting or ending position of the trunk movement interval is corrected, or both are corrected to generate the movement interval. The starting position of the movement interval is recorded as the final start, and the ending position is recorded as the final end.

[0104] In one possible implementation, an embodiment of the present application provides a specific implementation method for correcting the starting position and / or ending position of a torso motion interval to generate a motion interval. Please see below for details.

[0105] S204: Determine the classification label corresponding to each frame of radar signal in the action interval according to the classification label corresponding to the radar spectrum data.

[0106] Specifically, when the classification label corresponding to the radar spectrum data is fall, the label corresponding to the action interval in the radar spectrum data is determined to be fall, and the classification label corresponding to each frame of radar signal in the action interval is determined to be fall, which can be represented by 0. When the classification label corresponding to the radar spectrum data is non-fall, the label corresponding to the action interval in the radar spectrum data is determined to be non-fall, and the classification label corresponding to each frame of radar signal in the action interval is determined to be non-fall, which can be represented by 1.

[0107] In practical applications, the first and last parts of a motion interval labeled "fall" are not completely certain. This interval can be described as the "fall motion uncertainty interval." To make the motion interval more precise, it is necessary to determine the location and size of the "fall motion uncertainty interval" so that it can be removed from the motion interval.

[0108] As can be seen, there are two uncertainty intervals for a fall action within an action interval: one at the beginning of the action interval and the other at the end of the action interval. The signal label corresponding to each radar signal frame in the fall action uncertainty interval can be set to an uncertain label, represented by -1.

[0109] In a possible implementation, after the start position and / or end position of the trunk motion interval is corrected to generate the motion interval, the method further includes:

[0110] When the classification label corresponding to the radar spectrum data is fall, the eighth interval from the starting position of the action interval to the fifth target position and the ninth interval from the sixth target position to the end position of the action interval are removed from the action interval, and a new action interval is generated.

[0111] The eighth and ninth intervals are both uncertainty intervals for falling movements. The eighth interval is located at the beginning of the movement interval, and the ninth interval is located at the end of the movement interval.

[0112] In this embodiment, the starting position of the movement interval is final start, and the ending position of the movement interval is final end. The starting position of the trunk movement interval is original start, and the ending position of the trunk movement interval is original end.

[0113] As an optional example, the fifth target position is (original end – final start) – (original end – max (final start, original start)) / / 4. Here, “ / / ” indicates rounding after division. As an optional example, the sixth target position is -(final end – original end).

[0114] Therefore, the eighth interval is [final start: (original end–final start)–(original end–max(final start, original start)) / / 4], and the ninth interval is [-(final end–original end): final end].

[0115] Then, the label for each radar signal frame in the eighth and ninth intervals is -1. Furthermore, the eighth and ninth intervals, each corresponding to a label of -1, are removed from the action interval to regenerate the action interval. As a result, the regenerated action interval is a more accurate action interval, with each radar signal frame in the regenerated action interval having a classification label of 0 or 1.

[0116] S205: Splicing the multiple action intervals into training data, where the training data includes a preset number of radar signal frames, and each radar signal frame of the training data corresponds to a classification label.

[0117] When there are multiple sets of radar spectrum data, after executing S202-S204 for each set of spectrum data, the action intervals within each set of radar spectrum data can be obtained. Furthermore, the multiple action intervals are concatenated to form training data. The size of the training data must be the same as the input data size of the fall detection model. The concatenated training data includes a preset number of radar signal frames. Each radar signal frame in the training data is assigned a classification label: fall or non-fall, i.e., 0 or 1.

[0118] Since the size of the training data needs to be the same as the size of the input data of the fall detection model, in actual applications, the preset number can be determined according to the size of the input data of the fall detection model, and then the number of action intervals can be determined based on the preset number. Figure 4 , Figure 4 A data format diagram of training data provided in an embodiment of the present application. Figure 4 As shown, the training data is spliced ​​together from multiple motion intervals. The preset number of frames is 2000, and the training data size is 2000 × 205. Here, 205 is the length of the radar data vector for each 30ms sampling frame, and the 205 points represent 205 different frequency values. 2000 is the time axis dimension composed of the multiple motion data sampling frames, and 2000 is the number of frames.

[0119] It is understood that when the total length of the spliced ​​frames of multiple action intervals is greater than 2000, the multiple actions are spliced ​​together and the input data length with a total frame length of 2000 is truncated to obtain training data of size 2000 × 205. When the total length of the spliced ​​frames of multiple action intervals is less than 2000, a frame padding operation can be performed to obtain an input data length with a total frame length of 2000. This data splicing operation is performed until all action intervals are traversed to obtain multiple sets of training data.

[0120] In this way, the training data obtained can not only meet the input size of the fall detection model, but also meet the random form of splicing different action intervals, so that the fall detection model trained by the training data can ignore the splicing form of the input data and only detect the specific sampling frames in the input data.

[0121] S206: Input the training data into the fall detection model to obtain the predicted classification results corresponding to each frame of radar signal of the training data, calculate the loss value based on the predicted classification results corresponding to each frame of radar signal of the training data and the classification label corresponding to each frame of radar signal of the training data, adjust the fall detection model based on the loss value, and repeat the steps of inputting the training data into the fall detection model to obtain the predicted classification results corresponding to each frame of radar signal of the training data and subsequent steps until the preset conditions are met to generate a trained fall detection model.

[0122] The training data is input into the fall detection model, and the predicted classification results corresponding to each radar signal frame of the training data are obtained from the output of the fall detection model. Furthermore, a loss value is calculated based on the predicted classification results corresponding to each radar signal frame of the training data and the classification labels corresponding to each radar signal frame of the training data. The fall detection model is adjusted based on the loss value, specifically by adjusting the model parameters of the fall detection model. A detailed description of the structure of the fall detection model is provided below.

[0123] Before adjusting the fall detection model, it is necessary to first determine whether the preset conditions are met. When the preset conditions are met, the fall detection model training is completed and the trained fall detection model is obtained. When the preset conditions are not met, the fall detection model is adjusted according to the loss value until the preset conditions are met. As an optional example, the preset condition is that the preset number of training times is reached. As another optional example, the preset condition is that the loss value reaches a preset loss threshold. The preset number of training times and the preset loss threshold can be set according to actual conditions and are not restricted here.

[0124] In one possible implementation, the embodiment of the present application provides a method for calculating a loss value based on a predicted classification result corresponding to each frame of radar signal in the training data and a classification label corresponding to each frame of radar signal in the training data, including:

[0125] The loss value is calculated based on the predicted classification result corresponding to each frame of radar signal in the training data, the classification label corresponding to each frame of radar signal in the training data, and the loss value weight of the classification label; among which, the loss value weight of the fall classification label is greater than the loss value weight of the non-fall classification label.

[0126] Specifically, before calculating the loss value, a loss function is constructed and the loss value is calculated using the loss function. For example, the loss function is a cross entropy loss function. The loss value weights of the classification labels are set in the loss function, that is, the loss value weights of the fall classification label and the loss value weights of the non-fall classification label are set. When setting the loss value weights of the classification labels, the loss value weight of the fall classification label is set to be greater than the loss value weight of the non-fall classification label. For example, the loss value weight of the fall classification label is 1.5, and the loss value weight of the non-fall classification label is 1.2.

[0127] It is understandable that since the time a person falls is shorter than the time they do not fall, the amount of non-fall data in the training data is greater than the amount of fall data. The data consisting of multiple frames of radar signals with a classification label of fall is fall data, and the data consisting of multiple frames of radar signals with a classification label of non-fall is non-fall data. Therefore, the loss value weight of the fall classification label is set to be greater than the loss value weight of the non-fall classification label to balance the proportion of the loss value of fall data and non-fall data in the loss function, and the resulting loss value is a loss value that can adapt to actual fall scenarios.

[0128] As an optional example, the setting of 1.5 can be the initial loss value weight of the fall classification label, and the setting of 1.2 can be the initial loss value of the non-fall classification label. The Adam optimizer can also be used to update and optimize the weights in the loss function to obtain the optimal weight solution of the fall classification label and the optimal weight solution of the non-fall classification label.

[0129] Based on the contents of S201-S206, it can be seen that the embodiment of the present application provides a training method for a fall detection model based on radar data, which obtains radar spectrum data including multiple frames of radar signals collected by the radar, and the radar spectrum data corresponds to a classification label of fall or non-fall. According to the signal strength of each frame of radar signal, radar signals greater than a first preset threshold are extracted from the radar spectrum data to form a torso movement interval, and the starting position and / or end position of the torso movement interval are corrected to generate an action interval. The classification label corresponding to each frame of radar signal in the action interval is determined according to the classification label corresponding to the radar data. Based on this, a plurality of action intervals are spliced ​​into training data, and the training data includes a preset number of frames of radar signals and each frame of radar signal corresponds to a classification label. Furthermore, the fall detection model is trained and the parameters are adjusted based on the training data, and the fall detection model can predict the classification result of each frame of radar signal. In this way, the trained fall detection model can detect the fall or non-fall classification result corresponding to each frame of radar signal. After improving the detection frequency and accuracy of the fall detection model to each sampling frame of the radar signal, it can detect whether the user has fallen in real time with high accuracy, which can meet the needs of actual scenarios and user demands.

[0130] In actual applications, training data is obtained to train the fall detection model, and the trained fall detection model is tested based on the verification data. For example, there are 2456 fall data and 2671 non-fall data in the training data. There are 556 fall data and 628 non-fall data in the verification data. After the fall detection model is trained based on the training data and the trained fall detection model is obtained, the fall detection model is verified based on the verification data. Furthermore, the verification results can be evaluated by accuracy, precision, recall and F1-score. For example, the accuracy of the verification data obtained is 97.21%, the precision is 96.92%, the recall is 89.43%, and the F1-score is 93.02%. It can be seen that the trained fall detection model can accurately and in real time detect whether the user has fallen, meeting the needs of the user.

[0131] See also Figure 5 , Figure 5 This is a structural diagram of a fall detection model provided in an embodiment of the present application. Figure 5 As shown, as an optional example, the fall detection model includes a frame feature attention extraction network, a long short-term memory (LSTM) timing network, and a timing output network connected in sequence.

[0132] The training data is fed into the frame feature attention extraction network to obtain its output. The output of the frame feature attention extraction network is fed into the long short-term memory (LSTM) timing network to obtain its output. The output of the LSTM timing network is fed into the timing output network to obtain the predicted classification result corresponding to each frame of radar signal output by the timing output network.

[0133] As an optional example, the fall detection model also includes an attention transfer network. The output of the frame feature attention extraction network is also used to input the attention transfer network to obtain the output of the attention transfer network. The output of the attention transfer network is then input into the time series output network.

[0134] See also Figure 6 , Figure 6 This is a structural diagram of another fall detection model provided in an embodiment of the present application. Figure 6 As shown, in one or more embodiments, the frame feature attention extraction network includes a first fully connected layer. The LSTM timing network includes a second fully connected network, a third fully connected network, and a two-layer LSTM network. The timing output network includes a fourth fully connected network and a fifth fully connected network.

[0135] Specifically, the training data is input into the first fully connected layer, producing its output. The first fully connected layer is used to extract the intrinsic information of each radar signal frame within the training data. Multiplying the output of the first fully connected layer with the training data enhances the fall detection model's ability to identify each radar signal frame. The resulting product is then input into the second fully connected network, producing its output.

[0136] The output of the second fully connected network is input into the third fully connected network to obtain the output of the third fully connected network. The output of the third fully connected network is input into the two-layer LSTM network to obtain the output of the two-layer LSTM network. The output of the two-layer LSTM network is input into the fourth fully connected network to obtain the output of the fourth fully connected network. The output of the fourth fully connected network is input into the fifth fully connected network to obtain the predicted classification result for each frame of radar signal in the training data.

[0137] In one or more embodiments, the second fully-connected network includes a second fully-connected layer, a first dropout layer, and a first activation function, which are sequentially connected. The third fully-connected network includes a third fully-connected layer, a second dropout layer, and a second activation function, which are sequentially connected. The fourth fully-connected network includes a fourth fully-connected layer, a third dropout layer, and a third activation function, which are sequentially connected. The fifth fully-connected network includes a fifth fully-connected layer, a fourth dropout layer, and a fourth activation function, which are sequentially connected. It is understood that adding a dropout layer after a fully-connected layer can prevent model overfitting and improve the model's generalization ability.

[0138] As an optional example, the fall detection model further includes an attention transfer network. In one or more embodiments, the attention transfer network includes a sixth fully connected network and a seventh fully connected network.

[0139] The output of the first fully connected layer is multiplied by the training data and then simultaneously fed into the sixth fully connected network to obtain the output of the sixth network. The output of the sixth network is then fed into the seventh network to obtain the output of the seventh network. Multiplying the output of the seventh network with the output of the two-layer LSTM network enhances the understanding of each radar signal frame. The resulting product is then fed into the fourth fully connected network.

[0140] In one or more embodiments, the sixth fully connected network includes a sixth fully connected layer, a fifth dropout layer, and a fifth activation function connected in sequence. The seventh fully connected network includes a seventh fully connected layer, a sixth dropout layer, and a sixth activation function connected in sequence.

[0141] In a possible implementation, the embodiment of the present application provides a specific implementation method for modifying the starting position and / or ending position of the torso movement interval in S203 to generate the movement interval, including: A1 and / or A2. Specifically:

[0142] A1: Correct the starting position and / or ending position of the trunk movement interval according to the movement start and end point correction algorithm to generate a first movement interval.

[0143] The movement start and end point correction algorithm is used to correct the start and / or end positions of the torso movement interval using random values. Please see A11-A12 below for details.

[0144] A2: The starting position and / or ending position of the trunk motion interval is corrected according to the morphological start-end point correction algorithm to generate a second motion interval.

[0145] The morphological start-end point correction algorithm calculates the radar signal's signal strength slope and inflection point based on the first-order derivative slope and second-order derivative inflection point in a mathematical formula. This algorithm then corrects the start and / or end positions of the torso motion interval based on these slopes and inflection points. See A21-A22 below for details.

[0146] As an optional example, only step A1 may be performed, that is, only the starting position and / or ending position of the torso motion interval is corrected according to the motion start and end point correction algorithm. After executing step A1 and generating the first motion interval, the first motion interval is determined as the motion interval.

[0147] As another optional example, step A2 may be performed, that is, only the starting position and / or ending position of the trunk motion interval is corrected according to the morphological start-end point correction algorithm. After step A2 is performed to generate the second motion interval, the second motion interval is determined as the motion interval.

[0148] As another optional example, steps A1 and A2 may be performed. After generating the first action interval and the second action interval, the average value of the starting positions of the first action interval and the second action interval is determined as the starting position of the action interval, i.e., the final start. The average value of the ending positions of the first action interval and the second action interval is determined as the ending position of the action interval, i.e., the final end. In this way, using two algorithms to correct the starting position and / or ending position of the torso action interval can make the obtained action interval more accurate.

[0149] In one possible implementation, the embodiment of the present application provides a specific implementation method for correcting the starting position and / or ending position of the torso movement interval according to the movement start and end point correction algorithm in A1 to generate the first movement interval, including:

[0150] A11: When the classification label corresponding to the radar spectrum data is fall, a random position is selected within a first interval from the starting position of the radar spectrum data to the first target position as the starting position of the first action interval, and / or a random position is selected within a second interval from the ending position of the torso action interval to the ending position of the radar spectrum data as the ending position of the first action interval.

[0151] See also Figure 7a , Figure 7a This is a schematic diagram of a trunk movement interval correction provided by an embodiment of the present application. Figure 7a As shown in the figure, the starting position of the radar spectrum data is recorded as O1, the ending position of the radar spectrum data is recorded as O2, and the frame length of the radar spectrum data is recorded as length. The starting position of the torso movement interval is recorded as original start, abbreviated as OS; the ending position of the torso movement interval is recorded as original end, abbreviated as OE.

[0152] Determine the first target position, denoted as a1. The first target position is a predetermined distance after the start position of the torso motion interval. As an optional example, the predetermined distance is one-quarter the length of the radar spectrum data frame, i.e., 1 / 4 length. In this case, the position at OS + 1 / 4 length is a1, and the first interval is [O1, OS + 1 / 4 length].

[0153] The starting position of the first motion interval is determined to be a random position in the first interval, denoted as FS1, that is, FS1∈random[O1,OS+1 / 4length]. In addition, the ending position of the first motion interval is determined to be a random position in the second interval from the end position of the torso motion interval to the end position of the radar spectrum data, denoted as FE1, that is, FE1∈random[OE,O2].

[0154] It is understandable that, since the classification label corresponding to the radar spectrum data is a fall, the last moment of the action needs to be retained when intercepting the end point, that is, the position where FE1 is OE or after OE.

[0155] A12: When the classification label corresponding to the radar spectrum data is non-fall, a random position is selected within the first interval from the starting position of the radar spectrum data to the first target position as the starting position of the first action interval, and / or a random position is selected within the third interval from the second target position to the ending position of the radar spectrum data as the ending position of the first action interval.

[0156] See also Figure 7b , Figure 7b This is another schematic diagram of the correction of the trunk movement interval provided in the embodiment of the present application. Figure 7b As shown, the second target position is determined, denoted as a2. The second target position is the position of the second preset distance before the end position of the torso action interval. As an optional example, the second preset distance is one-third of the frame data length of the radar spectrum data, that is, 1 / 3length, then the position of OE-1 / 3length is a2, and the third interval is [OE-1 / 3length, O2]. The starting position of the second action interval is determined to be a random position in the first interval, denoted as FS2, that is, FS2∈random[O1,OS+1 / 4length]. In addition, the end position of the second action interval is determined to be a random position in the third interval, denoted as FE2, that is, FE2∈random[OE-1 / 3length, O2].

[0157] It is understandable that since the classification label corresponding to the radar spectrum data is non-fall, it is not necessary to retain the last moment of the action when intercepting the end point.

[0158] In one possible implementation, the present application embodiment provides a specific implementation method for generating a second action interval by correcting the starting position and / or ending position of the torso action interval according to the morphological start-end point correction algorithm in A2, including A21 and / or A22. Specifically:

[0159] A21: In the fourth interval from the position at the third preset distance before the starting position of the torso movement interval to the position at the fourth preset distance after the starting position of the torso movement interval, select the position where the first-order derivative of the signal strength is the largest as the third target position; in the fifth interval from the third target position to the starting position of the torso movement interval, select the first position where the second-order derivative of the signal strength is less than the second preset threshold as the starting position of the second movement interval.

[0160] As an optional example, the third preset distance is the distance encompassed by the 10 sampling points before the start of the torso motion interval, i.e., within 0.3 seconds before the start of the torso motion interval. The fourth preset distance is the distance encompassed by the 10 sampling points after the start, i.e., within 0.3 seconds after the start of the torso motion interval. Within the fourth interval, the location with the maximum first-order derivative of the signal strength is searched for as the third target location, denoted as d1.

[0161] In the fifth interval from the third target position to the start of the torso movement interval, the first position where the second derivative of the signal strength is less than the second preset threshold is selected as the start of the second movement interval. That is, the inflection point of the second derivative of the signal strength is calculated in the fifth interval from point d1 to the start of the torso movement interval. For example, the second threshold can be set to 0.01. The first position in the fifth interval where the inflection point of the second derivative of the signal strength is less than 0.01 is selected as the start of the second movement interval.

[0162] A22: In the sixth interval from the position at the fifth preset distance before the end position of the torso movement interval to the position at the sixth preset distance after the end position of the torso movement interval, select the position where the first-order derivative of the signal strength is the largest as the fourth target position; in the seventh interval from the fourth target position to the end position of the torso movement interval, select the first position where the second-order derivative of the signal strength is less than the second preset threshold as the end position of the second movement interval.

[0163] As an optional example, the fifth preset distance is the distance encompassed by the 10 sampling points before the end of the torso motion interval, i.e., within 0.3 seconds before the end of the torso motion interval. The sixth preset distance is the distance encompassed by the 10 sampling points after the end of the torso motion interval, i.e., within 0.3 seconds after the end of the torso motion interval. Within the sixth interval, the location with the maximum first-order derivative of the signal strength is searched for as the fourth target location, denoted as d2.

[0164] In the seventh interval from the fourth target position to the end of the torso movement interval, the first position where the second derivative of the signal strength is less than the second preset threshold is selected as the end of the second movement interval. That is, the inflection point of the second derivative of the signal strength is calculated in the seventh interval from point d2 to the start of the torso movement interval. For example, the second threshold can be set to 0.01, and the first position in the seventh interval where the inflection point of the second derivative of the signal strength is less than 0.01 is selected as the end of the second movement interval.

[0165] Based on A1-A2, it can be seen that after obtaining the trunk movement interval, the starting position and / or ending position of the trunk movement interval can be corrected to obtain a more accurate movement interval.

[0166] See also Figure 8 As shown in FIG, this figure is a flow chart of a fall detection method based on radar data provided by an embodiment of the present application, as shown in FIG. Figure 8 As shown, the method may include S801-S802:

[0167] S801: Obtain radar spectrum data to be classified, where the radar spectrum data to be classified includes a preset number of frames of radar signals.

[0168] It can be seen that the size of the radar spectrum data to be classified is also 2000×205.

[0169] S802: Input the radar spectrum data to be classified into the fall detection model to obtain a classification result corresponding to each frame of radar signal of the radar spectrum data to be classified, where the classification result includes fall or non-fall; the fall detection model is trained according to the training method of the fall detection model based on radar data in any of the above embodiments.

[0170] The technical details of S801-S802 can be found in the above embodiment and will not be repeated here. It can be seen that the trained fall detection model can detect and classify each radar signal frame as a fall or non-fall. By increasing the detection frequency and accuracy of the fall detection model to detect each sampling frame of the radar signal, it can detect whether the user has fallen in real time with high accuracy, which can meet the needs of actual scenarios and users.

[0171] Based on the training method of a fall detection model based on radar data provided in the above-mentioned method embodiment, the embodiment of the present application also provides a training device for a fall detection model based on radar data. The training device for a fall detection model based on radar data will be described below in conjunction with the accompanying drawings.

[0172] See also Figure 9 As shown in FIG, this figure is a structural diagram of a training device for a fall detection model based on radar data provided by an embodiment of the present application. Figure 9 As shown, the training device of the fall detection model based on radar data includes:

[0173] An acquisition unit 901 is configured to acquire radar spectrum data, the radar spectrum data including multiple frames of radar signals, the radar spectrum data corresponding to a classification label, the classification label including fall or non-fall;

[0174] A forming unit 902 is configured to extract radar signals having signal strength greater than a first preset threshold from the radar spectrum data to form a torso motion interval;

[0175] a correction unit 903, configured to correct the starting position and / or ending position of the trunk movement interval to generate a movement interval;

[0176] A determining unit 904 is configured to determine a classification label corresponding to each radar signal frame in the action interval according to the classification label corresponding to the radar spectrum data;

[0177] a splicing unit 905 for splicing the plurality of motion intervals into training data, wherein the training data includes a preset number of radar signal frames, and each radar signal frame of the training data corresponds to a classification label;

[0178] A training unit 906 is configured to input the training data into a fall detection model, obtain a predicted classification result corresponding to each frame of radar signal of the training data, calculate a loss value based on the predicted classification result corresponding to each frame of radar signal of the training data and the classification label corresponding to each frame of radar signal of the training data, adjust the fall detection model based on the loss value, repeatedly input the training data into the fall detection model, obtain a predicted classification result corresponding to each frame of radar signal of the training data, and subsequent steps until a preset condition is met, thereby generating a trained fall detection model; the fall detection model includes a frame feature attention extraction network, an LSTM timing network, and a timing output network connected in sequence.

[0179] In one possible implementation, the frame feature attention extraction network includes a first fully connected layer, the LSTM timing network includes a second fully connected network, a third fully connected network and a two-layer LSTM network, and the timing output network includes a fourth fully connected network and a fifth fully connected network;

[0180] The training data is input into the first fully-connected layer, the output of the first fully-connected layer is multiplied by the training data and then input into the second fully-connected network, the output of the second fully-connected network is input into the third fully-connected network, the output of the third fully-connected network is input into the two-layer LSTM network, the output of the two-layer LSTM network is input into the fourth fully-connected network, and the output of the fourth fully-connected network is input into the fifth fully-connected network to obtain a prediction classification result corresponding to each frame of radar signal of the training data.

[0181] In a possible implementation, the fall detection model further includes an attention transfer network, and the attention transfer network includes the sixth fully connected network and the seventh fully connected network;

[0182] The output of the first fully connected layer is multiplied by the training data and then input into the sixth fully connected network. The output of the sixth fully connected network is input into the seventh fully connected network. The output of the seventh fully connected network is multiplied by the output of the two-layer LSTM network and then input into the fourth fully connected network.

[0183] In a possible implementation, the training unit 906 includes a computing subunit;

[0184] The calculation subunit is used to calculate the loss value based on the predicted classification result corresponding to each frame of radar signal in the training data, the classification label corresponding to each frame of radar signal in the training data, and the loss value weight of the classification label; wherein the loss value weight of the fall classification label is greater than the loss value weight of the non-fall classification label.

[0185] In a possible implementation, the correction unit 903 includes a first generation subunit and / or a second generation subunit, and a determination subunit;

[0186] A first generating subunit is configured to correct the starting position and / or ending position of the trunk movement interval according to a movement start and end point correction algorithm to generate a first movement interval;

[0187] A second generating subunit is configured to correct the starting position and / or ending position of the trunk motion interval according to a morphological start-end point correction algorithm to generate a second motion interval;

[0188] a determination subunit, configured to, when only the first action interval is generated, determine the first action interval as the action interval; when only the second action interval is generated, determine the second action interval as the action interval; and, when both the first action interval and the second action interval are generated simultaneously, determine the average of the starting positions of the first action interval and the second action interval as the starting position of the action interval, and determine the average of the ending positions of the first action interval and the second action interval as the ending position of the action interval.

[0189] In a possible implementation, the first generating subunit includes:

[0190] a first selection subunit configured to, when the classification label corresponding to the radar spectrum data is fall, select a random position within a first interval between a starting position of the radar spectrum data and a first target position as a starting position of a first motion interval, and / or select a random position within a second interval between an ending position of the torso motion interval and an ending position of the radar spectrum data as an ending position of the first motion interval;

[0191] a second selection subunit, configured to, when the classification label corresponding to the radar spectrum data is non-fall, select a random position within a first interval from the starting position of the radar spectrum data to the first target position as the starting position of a first action interval, and / or select a random position within a third interval from the second target position to the ending position of the radar spectrum data as the ending position of the first action interval;

[0192] The first target position is a position that is a first preset distance behind the starting position of the torso movement interval; and the second target position is a position that is a second preset distance before the ending position of the torso movement interval.

[0193] In a possible implementation, the second generating subunit includes a third selecting subunit and / or a fourth selecting subunit;

[0194] a third selection subunit configured to select, within a fourth interval from a position a third preset distance before the starting position of the torso movement interval to a position a fourth preset distance after the starting position of the torso movement interval, a position at which the first-order derivative of the signal strength is the largest as the third target position; and, within a fifth interval from the third target position to the starting position of the torso movement interval, select, as the starting position of the second movement interval, the first position at which the second-order derivative of the signal strength is less than a second preset threshold.

[0195] The fourth selection subunit is used to select the position with the largest first-order derivative of signal strength as the fourth target position in the sixth interval from the position at the fifth preset distance before the end position of the torso movement interval to the position at the sixth preset distance after the end position of the torso movement interval; and select the first position where the second-order derivative of signal strength is less than the second preset threshold as the end position of the second movement interval in the seventh interval from the fourth target position to the end position of the torso movement interval.

[0196] In a possible implementation, the apparatus further includes:

[0197] A removal unit is configured to correct the starting position and / or ending position of the torso motion interval to generate a motion interval. When the classification label corresponding to the radar spectrum data is a fall, the eighth interval from the starting position of the motion interval to the fifth target position and the ninth interval from the sixth target position to the ending position of the motion interval are removed from the motion interval to regenerate the motion interval.

[0198] Based on the radar data-based fall detection method provided in the above-mentioned method embodiment, the embodiment of the present application also provides a radar data-based fall detection device. The radar data-based fall detection device will be described below with reference to the accompanying drawings.

[0199] See also Figure 10 As shown in FIG, this figure is a schematic diagram of the structure of a fall detection device based on radar data provided by an embodiment of the present application. Figure 10 As shown, the training device of the fall detection model based on radar data includes:

[0200] The first acquisition unit 1001 is configured to acquire radar spectrum data to be classified, where the radar spectrum data to be classified includes a preset number of frames of radar signals;

[0201] A second acquisition unit 1002 is configured to input the radar spectral data to be classified into a fall detection model to obtain a classification result corresponding to each frame of radar signal of the radar spectral data to be classified, where the classification result includes a fall or non-fall; the fall detection model is trained according to the radar data-based fall detection model training method described in any of the above embodiments.

[0202] In addition, an embodiment of the present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the processor executes the computer program, it implements the training method of the fall detection model based on radar data as described in any of the above embodiments, or the fall detection method based on radar data as described in any of the above embodiments.

[0203] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes the training method of the fall detection model based on radar data as described in any of the above embodiments, or the fall detection method based on radar data as described in any of the above embodiments.

[0204] Based on this, the embodiment of the present application provides a training device for a fall detection model based on radar data, a fall detection device, equipment and storage medium based on radar data, which obtains radar spectrum data including multiple frames of radar signals collected by the radar, and the radar spectrum data corresponds to a classification label of fall or non-fall. According to the signal strength of each frame of radar signal, radar signals greater than a first preset threshold are extracted from the radar spectrum data to form a torso movement interval, and the starting position and / or end position of the torso movement interval is corrected to generate an action interval. The classification label corresponding to each frame of radar signal in the action interval is determined according to the classification label corresponding to the radar data. Based on this, multiple action intervals are spliced ​​into training data, and the training data includes a preset number of frames of radar signals and each frame of radar signal corresponds to a classification label. Furthermore, the fall detection model is trained and parameters are adjusted based on the training data, and the fall detection model can predict the classification result of each frame of radar signal. In this way, the trained fall detection model can detect the fall or non-fall classification result corresponding to each frame of radar signal. After improving the detection frequency and accuracy of the fall detection model to each sampling frame of the radar signal, it can detect whether the user has fallen in real time with high accuracy, which can meet the needs of actual scenarios and user demands.

[0205] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0206] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0207] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0208] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0209] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a fall detection model based on radar data, characterized in that: The method comprises: Acquire radar spectrum data, the radar spectrum data including multiple frames of radar signals, the radar spectrum data corresponding to classification labels, the classification labels including fall or non-fall; Extracting radar signals with signal strength greater than a first preset threshold from the radar spectrum data to form a torso movement interval; Correcting the starting position and / or ending position of the trunk movement interval to generate a movement interval; Determine, according to the classification label corresponding to the radar spectrum data, a classification label corresponding to each frame of radar signal in the action interval; splicing the plurality of action intervals into training data, wherein the training data includes a preset number of radar signal frames, and each radar signal frame of the training data corresponds to a classification label; Inputting the training data into the fall detection model, obtaining a predicted classification result corresponding to each frame of radar signal of the training data, calculating a loss value based on the predicted classification result corresponding to each frame of radar signal of the training data and the classification label corresponding to each frame of radar signal of the training data, adjusting the fall detection model based on the loss value, repeatedly inputting the training data into the fall detection model, obtaining a predicted classification result corresponding to each frame of radar signal of the training data, and subsequent steps until a preset condition is met, thereby generating a trained fall detection model; the fall detection model includes a frame feature attention extraction network, an LSTM timing network, and a timing output network connected in sequence; The step of modifying the starting position and / or ending position of the trunk motion interval to generate the motion interval includes: Correcting the starting position and / or ending position of the trunk movement interval according to a movement start and end point correction algorithm to generate a first movement interval; and / or, Correcting the starting position and / or ending position of the trunk motion interval according to a morphological start-end point correction algorithm to generate a second motion interval; When only the first action interval is generated, the first action interval is determined as the action interval; when only the second action interval is generated, the second action interval is determined as the action interval; when the first action interval and the second action interval are generated at the same time, the average value of the starting positions of the first action interval and the second action interval is determined as the starting position of the action interval, and the average value of the ending positions of the first action interval and the second action interval is determined as the ending position of the action interval.

2. The method according to claim 1, characterized in that The frame feature attention extraction network includes a first fully connected layer, the LSTM timing network includes a second fully connected network, a third fully connected network and a double-layer LSTM network, and the timing output network includes a fourth fully connected network and a fifth fully connected network; The training data is input into the first fully-connected layer, the output of the first fully-connected layer is multiplied by the training data and then input into the second fully-connected network, the output of the second fully-connected network is input into the third fully-connected network, the output of the third fully-connected network is input into the two-layer LSTM network, the output of the two-layer LSTM network is input into the fourth fully-connected network, and the output of the fourth fully-connected network is input into the fifth fully-connected network to obtain a prediction classification result corresponding to each frame of radar signal of the training data.

3. The method according to claim 2, characterized in that The fall detection model further includes an attention transfer network, wherein the attention transfer network includes a sixth fully connected network and a seventh fully connected network; The output of the first fully connected layer is multiplied by the training data and then input into the sixth fully connected network. The output of the sixth fully connected network is input into the seventh fully connected network. The output of the seventh fully connected network is multiplied by the output of the two-layer LSTM network and then input into the fourth fully connected network.

4. The method according to any one of claims 1 to 3, characterized in that The calculating the loss value according to the predicted classification result corresponding to each frame of radar signal of the training data and the classification label corresponding to each frame of radar signal of the training data includes: The loss value is calculated based on the predicted classification result corresponding to each frame of radar signal in the training data, the classification label corresponding to each frame of radar signal in the training data, and the loss value weight of the classification label; wherein the loss value weight of the fall classification label is greater than the loss value weight of the non-fall classification label.

5. The method according to claim 1, wherein The step of correcting the starting position and / or ending position of the trunk motion interval according to the motion start and end point correction algorithm to generate a first motion interval includes: When the classification label corresponding to the radar spectrum data is fall, a random position is selected in a first interval from the starting position of the radar spectrum data to the first target position as the starting position of the first action interval, and / or a random position is selected in a second interval from the ending position of the torso action interval to the ending position of the radar spectrum data as the ending position of the first action interval; When the classification label corresponding to the radar spectrum data is non-fall, a random position is selected within a first interval from the starting position of the radar spectrum data to the first target position as the starting position of the first action interval, and / or a random position is selected within a third interval from the second target position to the ending position of the radar spectrum data as the ending position of the first action interval; The first target position is a position that is a first preset distance behind the starting position of the torso movement interval; and the second target position is a position that is a second preset distance before the ending position of the torso movement interval.

6. The method according to claim 1, characterized in that The step of correcting the starting position and / or ending position of the trunk motion interval according to the morphological start-end point correction algorithm to generate a second motion interval includes: In a fourth interval from a position a third preset distance before the starting position of the torso movement interval to a position a fourth preset distance after the starting position of the torso movement interval, a position where the first-order derivative of the signal strength is the largest is selected as the third target position; in a fifth interval from the third target position to the starting position of the torso movement interval, the first position where the second-order derivative of the signal strength is less than a second preset threshold is selected as the starting position of the second movement interval; and / or, In the sixth interval from the position of the fifth preset distance before the end position of the torso movement interval to the position of the sixth preset distance after the end position of the torso movement interval, the position where the first-order derivative of the signal strength is the largest is selected as the fourth target position; in the seventh interval from the fourth target position to the end position of the torso movement interval, the first position where the second-order derivative of the signal strength is less than the second preset threshold is selected as the end position of the second movement interval.

7. The method according to any one of claims 1-3, 5-6, characterized in that: After modifying the starting position and / or ending position of the trunk motion interval to generate the motion interval, the method further includes: When the classification label corresponding to the radar spectrum data is fall, the eighth interval from the starting position of the action interval to the fifth target position and the ninth interval from the sixth target position to the end position of the action interval are removed from the action interval, and a new action interval is generated.

8. The method according to claim 4, characterized in that After modifying the starting position and / or ending position of the trunk motion interval to generate the motion interval, the method further includes: When the classification label corresponding to the radar spectrum data is fall, the eighth interval from the starting position of the action interval to the fifth target position and the ninth interval from the sixth target position to the end position of the action interval are removed from the action interval, and a new action interval is generated.

9. A fall detection method based on radar data, characterized in that: The method comprises: Acquiring radar spectrum data to be classified, wherein the radar spectrum data to be classified includes a preset number of frames of radar signals; The radar spectrum data to be classified is input into a fall detection model to obtain a classification result corresponding to each frame of radar signal of the radar spectrum data to be classified, wherein the classification result includes fall or non-fall; the fall detection model is trained according to the training method of the fall detection model based on radar data according to any one of claims 1-8.

10. A training device for a fall detection model based on radar data, characterized in that: The device comprises: an acquisition unit, configured to acquire radar spectrum data, the radar spectrum data including multiple frames of radar signals, the radar spectrum data corresponding to classification labels, the classification labels including fall or non-fall; a composition unit, configured to extract radar signals having signal strength greater than a first preset threshold from the radar spectrum data to form a torso movement interval; a correction unit, configured to correct the starting position and / or ending position of the trunk movement interval to generate a movement interval; a determining unit, configured to determine a classification label corresponding to each frame of radar signal in the action interval according to the classification label corresponding to the radar spectrum data; a splicing unit, configured to splice the plurality of motion intervals into training data, wherein the training data includes a preset number of radar signal frames, and each radar signal frame of the training data corresponds to a classification label; A training unit is configured to input the training data into a fall detection model, obtain a predicted classification result corresponding to each frame of radar signal of the training data, calculate a loss value based on the predicted classification result corresponding to each frame of radar signal of the training data and the classification label corresponding to each frame of radar signal of the training data, adjust the fall detection model based on the loss value, and repeatedly perform the steps of inputting the training data into the fall detection model, obtaining the predicted classification result corresponding to each frame of radar signal of the training data, and subsequent steps until a preset condition is met, thereby generating a trained fall detection model; the fall detection model includes a frame feature attention extraction network, an LSTM time series network, and a time series output network connected in sequence; The correction unit includes a first generation subunit and / or a second generation subunit, and a determination subunit; A first generating subunit is configured to correct the starting position and / or ending position of the trunk movement interval according to a movement start and end point correction algorithm to generate a first movement interval; A second generating subunit is configured to correct the starting position and / or ending position of the trunk motion interval according to a morphological start-end point correction algorithm to generate a second motion interval; a determination subunit, configured to, when only the first action interval is generated, determine the first action interval as the action interval; when only the second action interval is generated, determine the second action interval as the action interval; and, when both the first action interval and the second action interval are generated simultaneously, determine the average of the starting positions of the first action interval and the second action interval as the starting position of the action interval, and determine the average of the ending positions of the first action interval and the second action interval as the ending position of the action interval.

11. A fall detection device based on radar data, characterized in that: The device comprises: A first acquiring unit is configured to acquire radar spectrum data to be classified, wherein the radar spectrum data to be classified includes a preset number of frames of radar signals; A second acquisition unit is configured to input the radar spectral data to be classified into a fall detection model to obtain a classification result corresponding to each frame of radar signal of the radar spectral data to be classified, wherein the classification result includes a fall or non-fall; the fall detection model is trained according to the training method for a fall detection model based on radar data according to any one of claims 1 to 8.

12. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for training a fall detection model based on radar data as described in any one of claims 1 to 8 or the method for fall detection based on radar data as described in claim 9 is implemented.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on a terminal device, cause the terminal device to execute the method for training a fall detection model based on radar data as described in any one of claims 1 to 8, or the method for fall detection based on radar data as described in claim 9.

Images