Intelligent fall detection method and system based on millimeter wave radar
By using millimeter wave radar and deep learning technology in fall detection, time-frequency characteristic analysis of linear frequency modulation wave signals and electromagnetic wave reflected signals, the problem of privacy violations and identification difficulties in the existing technology is solved, and more accurate and reliable Fall warning prompts provide a safer monitoring environment.
Patent Information
- Application Number
- CN202411212293.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-30
AI Technical Summary
The existing technology has problems of privacy violations and difficulty in identifying under insufficient light or obstruction of sight in the guardian fall detection, resulting in false alarms or missed reports, and cannot provide timely and accurate fall warnings.
The millimeter wave radar is used to emit linear frequency modulation wave signals to the target human body, and the time-frequency characteristic analysis of the signal is performed through deep learning artificial intelligence technology. The line-of-visibility and non-contact characteristics of the millimeter wave radar are used to achieve intelligent recognition and fall of the target human body posture. Warning prompt.
It improves the accuracy and reliability of fall detection, protects user privacy, provides a safer and smart living environment, and reduces false alarms and missed reports.
Smart Images

Figure CN118731940B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of millimeter wave radar technology, and more specifically, to an intelligent fall detection method and system based on millimeter wave radar. Background Art
[0002] At present, some special groups, such as the elderly and the disabled, require additional supervision and care. In daily supervision work, falls have become the main cause of accidental injuries to the wards. Especially in areas with limited space and slippery floors such as bathrooms and kitchens, the risk of wards accidentally falling is relatively high. After the ward falls, emergency assistance for the ward often needs to be carried out in a short time. A slight delay may cause serious physical injuries or even endanger life.
[0003] In the current market, there are products that use cameras for remote monitoring or use smart cameras with integrated fall detection functions to monitor falls of the ward. However, these solutions still have some limitations in practical applications. For example, camera-based monitoring systems may infringe on the privacy of the ward, and in the case of insufficient light or obstructed vision, they may not accurately identify falls, resulting in false alarms or missed alarms.
[0004] At present, millimeter-wave radar technology is increasingly used in the fields of human body sensing and motion detection, especially in terms of privacy protection and environmental adaptability. Therefore, we look forward to a smart fall detection method based on millimeter-wave radar, which can detect the fall behavior of the guardian more accurately, reliably and timely, so as to take safety measures in time. Summary of the invention
[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an intelligent fall detection method and system based on millimeter wave radar, which transmits a linear frequency modulation wave signal to the target human body through the millimeter wave radar, and receives the electromagnetic wave signal reflected by the target human body, and uses artificial intelligence technology based on deep learning to perform time-frequency feature analysis on the linear frequency modulation wave signal and the electromagnetic wave reflection signal, and then realizes intelligent recognition of the target human body posture based on the feature difference between the two, so as to provide timely fall warning prompts. By utilizing the non-line-of-sight and non-contact characteristics of millimeter wave radar, the present application can improve the accuracy and reliability of fall detection while protecting user privacy, providing a safer and smarter living environment for the protected person.
[0006] According to one aspect of the present application, a method for intelligent fall detection based on millimeter wave radar is provided, which includes:
[0007] collecting, by an oscilloscope, a linear frequency modulation wave signal emitted by the millimeter wave radar and an electromagnetic wave reflection signal received by the millimeter wave radar;
[0008] Performing signal noise reduction processing on the electromagnetic wave reflection signal to obtain a noise-reduced electromagnetic wave reflection signal;
[0009] Respectively extracting the time-frequency characteristics of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain a time-frequency characteristic graph of the linear frequency modulation wave signal and a time-frequency characteristic graph of the electromagnetic wave reflection signal after noise reduction;
[0010] Performing autocorrelation attention enhancement on the linear frequency modulation wave signal time-frequency characteristic diagram and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction to obtain an enhanced linear frequency modulation wave signal time-frequency characteristic diagram and an enhanced time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction;
[0011] Based on the characteristic difference between the enhanced linear frequency modulation wave signal time-frequency characteristic diagram and the enhanced noise-reduced electromagnetic wave reflection signal time-frequency characteristic diagram, it is determined whether to generate a fall warning prompt.
[0012] According to another aspect of the present application, there is provided a millimeter wave radar-based intelligent fall detection system, which includes:
[0013] A signal acquisition module, used for acquiring a linear frequency modulation wave signal emitted by the millimeter wave radar and an electromagnetic wave reflection signal received by the millimeter wave radar through an oscilloscope;
[0014] A noise reduction module, used for performing signal noise reduction processing on the electromagnetic wave reflection signal to obtain a noise-reduced electromagnetic wave reflection signal;
[0015] A time-frequency feature extraction module, used to extract the time-frequency features of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain a time-frequency feature graph of the linear frequency modulation wave signal and a time-frequency feature graph of the electromagnetic wave reflection signal after noise reduction;
[0016] An autocorrelation attention enhancement module is used to perform autocorrelation attention enhancement on the linear frequency modulation wave signal time-frequency characteristic diagram and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction to obtain an enhanced linear frequency modulation wave signal time-frequency characteristic diagram and an enhanced time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction;
[0017] The warning prompt generating module is used to determine whether to generate a fall warning prompt based on the characteristic difference between the time-frequency characteristic diagram of the enhanced linear frequency modulation wave signal and the time-frequency characteristic diagram of the enhanced electromagnetic wave reflection signal after noise reduction.
[0018] The present application has at least the following technical effects: Compared with the prior art, the present application provides a method and system for intelligent fall detection based on millimeter wave radar, which transmits a linear frequency modulation wave signal to the target human body through the millimeter wave radar, and receives the electromagnetic wave signal reflected by the target human body, and uses artificial intelligence technology based on deep learning to perform time-frequency feature analysis on the linear frequency modulation wave signal and the electromagnetic wave reflection signal, and then realizes intelligent recognition of the target human body posture based on the feature difference between the two, so as to provide timely fall warning prompts. By utilizing the non-line-of-sight and non-contact characteristics of millimeter wave radar, the present application can improve the accuracy and reliability of fall detection while protecting user privacy, and provide a safer and smarter living environment for the guardian. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0020] Figure 1 is a flow chart of an intelligent fall detection method based on millimeter wave radar according to an embodiment of the present application;
[0021] Figure 2 Schematic diagram of data flow of a millimeter-wave radar-based intelligent fall detection method according to an embodiment of the present application;
[0022] Figure 3 is a flowchart of sub-step S3 of the intelligent fall detection method based on millimeter wave radar according to an embodiment of the present application;
[0023] Figure 4 is a block diagram of an intelligent fall detection system based on millimeter-wave radar according to an embodiment of the present application. DETAILED DESCRIPTION
[0024] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.
[0025] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.
[0026] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.
[0027] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.
[0028] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.
[0029] In the current market, there are related products that use cameras for remote monitoring or use smart cameras with integrated fall detection functions to monitor falls of the ward. However, these solutions still have some limitations in practical applications. For example, the camera-based monitoring system may infringe on the privacy of the ward, and in the case of insufficient light or blocked vision, it may not be able to accurately identify falls, resulting in false alarms or missed alarms. At present, millimeter-wave radar technology is increasingly used in the fields of human body sensing and motion detection, especially in terms of privacy protection and environmental adaptability. Therefore, a millimeter-wave radar-based intelligent fall detection method is expected to solve the above technical problems.
[0030] In the technical solution of the present application, a method for intelligent fall detection based on millimeter-wave radar is proposed. Figure 1 Flow chart of a method for intelligent fall detection based on millimeter-wave radar according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of a fall intelligent detection method based on millimeter wave radar according to an embodiment of the present application. Figure 1 and Figure 2As shown, according to the embodiment of the present application, the intelligent fall detection method based on millimeter wave radar includes the following steps: S1, collecting the linear frequency modulation wave signal emitted by the millimeter wave radar and the electromagnetic wave reflection signal received by the millimeter wave radar through an oscilloscope; S2, performing signal noise reduction processing on the electromagnetic wave reflection signal to obtain the electromagnetic wave reflection signal after noise reduction; S3, respectively extracting the time-frequency characteristics of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain the linear frequency modulation wave signal time-frequency characteristic diagram and the electromagnetic wave reflection signal time-frequency characteristic diagram after noise reduction; S4, respectively performing autocorrelation attention enhancement on the linear frequency modulation wave signal time-frequency characteristic diagram and the electromagnetic wave reflection signal time-frequency characteristic diagram after noise reduction to obtain the enhanced linear frequency modulation wave signal time-frequency characteristic diagram and the enhanced electromagnetic wave reflection signal time-frequency characteristic diagram after noise reduction; S5, determining whether to generate a fall warning prompt based on the feature difference between the enhanced linear frequency modulation wave signal time-frequency characteristic diagram and the enhanced electromagnetic wave reflection signal time-frequency characteristic diagram after noise reduction.
[0031] In particular, the S1 collects the linear frequency modulation wave signal emitted by the millimeter wave radar and the electromagnetic wave reflection signal received by the millimeter wave radar through an oscilloscope. Specifically, the millimeter wave radar has the advantages of strong penetration, high anti-interference ability, good privacy protection, etc., and can realize non-contact accurate monitoring of the human body. Especially in complex situations such as changes in ambient light and line of sight obstruction, it can still stably capture the slight movement changes of the human body, thereby improving the accuracy and timeliness of fall detection. Specifically, the linear frequency modulation wave signal emitted by the millimeter wave radar generates reflection after interacting with the human body during the propagation process. Due to the Doppler frequency shift caused by human body movement, the electromagnetic wave reflection signal can carry rich human body posture and motion state information. For example, when the target human body falls, the human body's movement is more chaotic and random, and the acceleration and speed change greatly, which will cause the electromagnetic wave reflection signal to have a wider spectrum width and an increased frequency change rate relative to the linear frequency modulation wave signal. Based on this, in the technical solution of the present application, the linear frequency modulation wave signal and the electromagnetic wave reflection signal are compared and analyzed for time-frequency characteristics to intelligently identify the fall event of the target human body.
[0032] In particular, S2 performs signal noise reduction processing on the electromagnetic wave reflection signal to obtain the electromagnetic wave reflection signal after noise reduction. Considering that in the actual environment, the electromagnetic wave reflection signal is often interfered by various noises, such as environmental background noise, signal fading caused by multipath propagation, etc., thereby reducing the accuracy of human posture recognition. Therefore, in the technical solution of the present application, the electromagnetic wave reflection signal is further subjected to signal noise reduction processing to eliminate the noise components therein and enhance the signal-to-noise ratio of the signal, thereby obtaining the electromagnetic wave reflection signal after noise reduction. In an embodiment of the present application, an adaptive filter or a noise reduction method based on deep learning can be used to perform signal noise reduction processing on the electromagnetic wave reflection signal.
[0033] In particular, the S3 extracts the time-frequency characteristics of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain a time-frequency characteristic diagram of the linear frequency modulation wave signal and a time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction. In particular, in a specific example of the present application, Figure 3 As shown, the S3 includes: S31, using wavelet analysis to process the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain the time-frequency diagram of the linear frequency modulation wave signal and the time-frequency diagram of the electromagnetic wave reflection signal after noise reduction; S32, respectively extracting time-frequency features of the linear frequency modulation wave signal time-frequency diagram and the time-frequency diagram of the electromagnetic wave reflection signal after noise reduction to obtain the time-frequency feature diagram of the linear frequency modulation wave signal and the time-frequency feature diagram of the electromagnetic wave reflection signal after noise reduction.
[0034] Specifically, the S31 uses wavelet analysis to process the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain a time-frequency diagram of the linear frequency modulation wave signal and a time-frequency diagram of the electromagnetic wave reflection signal after noise reduction. Among them, in order to convert the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction into a representation in the time-frequency domain, so as to intuitively reveal the spectrum changes of the signal caused by human body movement, the present application uses a wavelet analysis method to process the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction. Specifically, wavelet analysis is an effective time-frequency analysis tool that can effectively reveal the local characteristics of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction in the time domain and frequency domain, generate a time-frequency diagram of the linear frequency modulation wave signal and a time-frequency diagram of the electromagnetic wave reflection signal after noise reduction, thereby clearly showing the frequency distribution and changes of the signal at different time points, which is helpful for the subsequent time-frequency feature difference analysis.
[0035] Specifically, in S32, the time-frequency feature extraction is performed on the time-frequency diagram of the linear frequency modulation wave signal and the time-frequency diagram of the electromagnetic wave reflection signal after noise reduction to obtain the time-frequency feature diagram of the linear frequency modulation wave signal and the time-frequency feature diagram of the electromagnetic wave reflection signal after noise reduction. In a specific example of the present application, the time-frequency diagram of the linear frequency modulation wave signal and the time-frequency diagram of the electromagnetic wave reflection signal after noise reduction are respectively input into a signal time-frequency feature extractor based on a hole convolutional neural network model to obtain the time-frequency feature diagram of the linear frequency modulation wave signal and the time-frequency feature diagram of the electromagnetic wave reflection signal after noise reduction. Considering that although the wavelet analysis method provides a time-frequency feature representation of the signal, there may be certain limitations in directly using the time-frequency diagram for feature difference analysis, such as the complexity of feature selection and the large amount of calculation. Therefore, the present application further adopts a hole convolutional neural network model as an extractor of the signal time-frequency features to automatically learn and extract the time-frequency correlation information in the linear frequency modulation wave signal time-frequency diagram and the electromagnetic wave reflection signal time-frequency diagram after noise reduction, and generate a linear frequency modulation wave signal time-frequency feature diagram and a electromagnetic wave reflection signal time-frequency feature diagram after noise reduction. Among them, the hole convolutional neural network expands the receptive field by introducing holes (i.e. sparse sampling) in the convolution kernel, which can effectively capture the long-distance dependence and local details of the signal in the time-frequency domain while keeping the computational complexity relatively low, and more deeply understand the time-domain dynamic characteristics and frequency distribution characteristics of the signal, thereby improving the accuracy and robustness of feature expression.
[0036] Specifically, in other specific examples of the present application, the time-frequency characteristics of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction can also be extracted respectively in other ways to obtain the time-frequency characteristic diagram of the linear frequency modulation wave signal and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction, for example: input the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction; use time-frequency analysis methods such as short-time Fourier transform (STFT) or continuous wavelet transform (CWT) to decompose the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction into time-frequency domain; extract amplitude, phase, frequency and time characteristics from the time-frequency representation respectively to obtain the time-frequency characteristic diagram of the linear frequency modulation wave signal and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction.
[0037] In particular, the S4 performs autocorrelation attention enhancement on the time-frequency feature graph of the linear frequency modulation wave signal and the time-frequency feature graph of the electromagnetic wave reflection signal after noise reduction to obtain the enhanced time-frequency feature graph of the linear frequency modulation wave signal and the enhanced time-frequency feature graph of the electromagnetic wave reflection signal after noise reduction. In a specific example of the present application, the time-frequency feature graph of the linear frequency modulation wave signal and the time-frequency feature graph of the electromagnetic wave reflection signal after noise reduction are respectively input into the cross-channel adaptive feature space structure enhanced self-attention module to obtain the enhanced time-frequency feature graph of the linear frequency modulation wave signal and the enhanced time-frequency feature graph of the electromagnetic wave reflection signal after noise reduction. In order to further enhance the distinguishability and expression ability of the features, the present application further introduces a cross-channel adaptive feature space structure enhanced self-attention module to perform feature enhancement processing on the time-frequency feature graph of the linear frequency modulation wave signal and the time-frequency feature graph of the electromagnetic wave reflection signal after noise reduction. Specifically, the cross-channel adaptive feature space structure enhanced self-attention module first performs layer normalization processing on the input feature graph to eliminate the scale differences between different levels, and ensure the stability and consistency of the subsequent feature processing process. Then, multi-layer convolution operations are used to capture the channel context association information and spatial structure information of the feature map, and then the self-attention mechanism is introduced. The channel context association information is used as the query and the spatial structure information is used as the key and value to perform global cross-channel association interaction of features. While retaining the spatial structure of the feature map, the deep fusion and exchange of channel association information between feature structures is promoted based on the feature spatial structure, thereby effectively enhancing the expression depth and richness of the features, and obtaining the time-frequency feature map of the enhanced linear frequency modulation wave signal and the time-frequency feature map of the electromagnetic wave reflection signal after enhanced noise reduction, providing more accurate and distinctive feature representation for fall event recognition.
[0038] In an embodiment of the present application, the linear frequency modulation wave signal time-frequency feature map and the time-frequency feature map of the electromagnetic wave reflection signal after noise reduction are respectively input into the cross-channel adaptive feature space structure enhanced self-attention module to obtain the enhanced linear frequency modulation wave signal time-frequency feature map and the enhanced time-frequency feature map of the electromagnetic wave reflection signal after noise reduction, including: performing layer normalization on the linear frequency modulation wave signal time-frequency feature map to obtain a normalized linear frequency modulation wave signal time-frequency feature map; performing point convolution processing on the normalized linear frequency modulation wave signal time-frequency feature map to obtain a linear frequency modulation wave signal time-frequency channel context association representation feature map; performing convolution encoding on the linear frequency modulation wave signal time-frequency channel context association representation feature map to obtain a linear frequency modulation wave signal time-frequency space context association representation feature map; performing channel-space global interactive attention fusion on the linear frequency modulation wave signal time-frequency channel context association representation feature map and the linear frequency modulation wave signal time-frequency space context association representation feature map to obtain the enhanced linear frequency modulation wave signal time-frequency feature map.
[0039] Among them, the channel-space global interactive attention fusion is performed on the linear frequency modulation wave signal time-frequency channel context association representation feature map and the linear frequency modulation wave signal time-frequency space context association representation feature map to obtain the enhanced linear frequency modulation wave signal time-frequency feature map, including: copying the linear frequency modulation wave signal time-frequency space context association representation feature map to obtain a backup linear frequency modulation wave signal time-frequency space context association representation feature map; reshaping the linear frequency modulation wave signal time-frequency channel context association representation feature map, the linear frequency modulation wave signal time-frequency space context association representation feature map and the backup linear frequency modulation wave signal time-frequency space context association representation feature map to obtain a linear frequency modulation wave signal time-frequency channel context association representation feature matrix, a linear frequency modulation wave signal time-frequency space context association representation feature matrix and A backup linear frequency modulation wave signal time-frequency space context association representation feature matrix; calculating the cross-channel cross covariance matrix between the linear frequency modulation wave signal time-frequency channel context association representation feature matrix and the linear frequency modulation wave signal time-frequency space context association representation feature matrix; using the Softmax function to activate the cross-channel cross covariance matrix to obtain the linear frequency modulation wave signal time-frequency feature global interactive attention matrix; calculating the product between the backup linear frequency modulation wave signal time-frequency space context association representation feature matrix and the linear frequency modulation wave signal time-frequency feature global interactive attention matrix to obtain the attention enhancement linear frequency modulation wave signal time-frequency feature representation matrix; and reshaping the attention enhancement linear frequency modulation wave signal time-frequency feature representation matrix to obtain the enhanced linear frequency modulation wave signal time-frequency feature map.
[0040] In summary, in the above embodiment, the linear frequency modulation wave signal time-frequency feature map and the time-frequency feature map of the electromagnetic wave reflection signal after noise reduction are respectively input into the cross-channel adaptive feature space structure enhanced self-attention module to obtain the enhanced linear frequency modulation wave signal time-frequency feature map and the enhanced time-frequency feature map of the electromagnetic wave reflection signal after noise reduction, including: processing the linear frequency modulation wave signal time-frequency feature map with the following autocorrelation attention enhancement formula to obtain the enhanced linear frequency modulation wave signal time-frequency feature map, wherein the autocorrelation attention enhancement formula is:
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
[0049]
[0050] in, represents the time-frequency characteristic diagram of the linear frequency modulation wave signal, Representation layer normalization operation, represents the time-frequency characteristic diagram of the normalized linear frequency modulation wave signal, represents point convolution, represents a time-frequency channel context association representation feature map of the linear frequency modulation wave signal, Represents convolution processing based on 3×3 convolution kernel, represents a time-frequency-space contextual representation feature map of the linear frequency modulation wave signal, Indicates a copy operation. A time-frequency-space contextual representation feature map representing the backup linear frequency modulation wave signal, represents the feature shape reshaping, , and They respectively represent the linear frequency modulation wave signal time-frequency channel context association representation feature matrix, the linear frequency modulation wave signal time-frequency space context association representation feature matrix and the backup linear frequency modulation wave signal time-frequency space context association representation feature matrix, represents the cross-channel cross covariance matrix, is the scaling factor, represents the normalized exponential function, represents the matrix multiplication operation, The time-frequency characteristic diagram of the enhanced linear frequency modulation wave signal is represented.
[0051] In particular, the S5 determines whether to generate a fall warning prompt based on the feature difference between the enhanced linear frequency modulation wave signal time-frequency feature graph and the enhanced noise reduction electromagnetic wave reflection signal time-frequency feature graph. In a specific example of the present application, the differential feature graph between the enhanced linear frequency modulation wave signal time-frequency feature graph and the enhanced noise reduction electromagnetic wave reflection signal time-frequency feature graph is first calculated as a posture information characterization feature graph; that is, in order to reveal the feature difference between the enhanced linear frequency modulation wave signal time-frequency feature graph and the enhanced noise reduction electromagnetic wave reflection signal time-frequency feature graph, the differential feature graph between the two is further calculated by a differential calculation method, so as to intuitively display the time-frequency domain feature changes caused by human body movement. Further, the differential feature graph is used as a posture information characterization feature graph, and is input into a classifier-based posture recognizer to obtain a recognition result, and the recognition result is used to indicate whether the posture type is a fall. That is, the classifier model is used to perform feature analysis on the posture information characterization feature graph to determine whether the current posture information meets the fall feature. Specifically, during the training process of the classifier, supervised learning is performed on the classifier by using a large number of labeled fall and non-fall samples, so that it can learn the mapping relationship between different posture features and fall events, so that in the test phase, the classifier can accurately infer the posture type of the target human body based on the feature pattern of the input posture information representation feature map. Furthermore, when the recognition result output by the classifier points to a fall, the system will immediately trigger the alarm mechanism, generate a fall warning prompt, and notify relevant personnel or equipment so that rescue measures can be taken in time.
[0052] In a preferred embodiment, the step of passing the posture information representation feature graph through a classifier-based posture recognizer to obtain a recognition result comprises:
[0053] Calculating a characteristic mean of the posture information characterization feature map, and dividing the characteristic mean by a difference between a maximum characteristic value and a minimum characteristic value of the posture information characterization feature map to obtain a posture information characterization distribution characterization value;
[0054] Subtracting the attitude information characterization distribution characterization value from one and dividing by the attitude information characterization distribution characterization value to obtain an attitude information characterization distribution modulation value;
[0055] Activating the posture information representation feature map through a probabilistic function to obtain a probabilistic posture information representation feature map;
[0056] After performing point subtraction between the probabilistic posture information characterization feature graph and the posture information characterization distribution modulation value, taking the absolute value and calculating the negative of the logarithmic value with base 2 to obtain the probabilistic posture information characterization distribution modulation information feature graph;
[0057] After dividing the posture information characterization distribution characterization value by one minus the difference of each eigenvalue of the probabilistic posture information characterization feature map, summing all eigenvalues of the probabilistic posture information characterization feature map and dividing by the scale of the posture information characterization feature map to obtain a probabilistic posture information characterization distribution modulation bias value;
[0058] The probabilistic posture information characterization distribution modulation information feature graph and the product of the probabilistic posture information characterization distribution modulation bias value and the weight as a hyperparameter are multiplied to obtain an optimized posture information characterization feature graph;
[0059] And the optimized posture information representation feature map is input into the classifier-based posture recognizer to obtain the recognition result.
[0060] The optimized posture information representation feature graph is expressed as:
[0061]
[0062] And among them:
[0063]
[0064] in, represents the feature mean of the feature graph representing the posture information, and They respectively represent the maximum eigenvalue and the minimum eigenvalue in the posture information representation feature graph, represents the distribution representation value of the posture information, represents a probabilistic posture information representation feature map obtained after the posture information representation feature map is activated by a probabilistic function, The first eigenvalues, represents the logarithmic function with base 2, is the weight as a hyperparameter, is the scale of the posture information representation feature map, that is, the width of the feature matrix of the posture information representation feature map multiplied by the height and then multiplied by the number of channels of the posture information representation feature map, The optimized posture information representation feature graph is represented.
[0065] Specifically, considering that the enhanced linear frequency modulation wave signal time-frequency feature map and the enhanced denoised electromagnetic wave reflection signal time-frequency feature map respectively express the image semantic space distribution channel cross-structure self-attention features represented by the two-dimensional image in the time-frequency domain determined by wavelet analysis of the linear frequency modulation wave signal and the electromagnetic wave reflection signal received by the millimeter wave radar, when calculating the differential feature map between the enhanced linear frequency modulation wave signal time-frequency feature map and the enhanced denoised electromagnetic wave reflection signal time-frequency feature map, the obtained posture information representation feature map will also cause insufficient feature differential coverage due to the difference in fine-grained cross-channel adaptive attention weights of the image, thereby leading to outlier class inference mapping deviation, affecting the accuracy of the recognition result obtained by the posture information representation feature map through the classifier-based posture recognizer.
[0066] Therefore, in the preferred example described above, the eigenvalue-based probability information distribution planning of the posture information characterization feature graph is performed by using the Bernoulli probability modulation distribution of the posture information characterization feature graph relative to the eigenvalue distribution, and the probability reverse mapping of the probability features of the posture information characterization feature graph as a whole is used as an extended coverage of the set mapping space of the posture information characterization feature graph to independently understand the interaction path between the intuitive probability information distribution and the abstract probability space mapping of the posture information characterization feature graph, so as to improve the accuracy of the recognition result obtained by the optimized posture information characterization representation vector through the classifier-based posture recognizer by avoiding the counterfactual reasoning mapping of the outlier feature distribution of the posture information characterization feature graph to the class regression probability.
[0067] In summary, the intelligent fall detection method based on millimeter wave radar according to the embodiment of the present application is explained, which transmits a linear frequency modulation wave signal to the target human body through the millimeter wave radar, and receives the electromagnetic wave signal reflected by the target human body, and uses artificial intelligence technology based on deep learning to perform time-frequency feature analysis on the linear frequency modulation wave signal and the electromagnetic wave reflection signal, and then realizes intelligent recognition of the target human body posture based on the feature difference between the two, so as to provide timely fall warning prompts. In this way, by utilizing the non-line-of-sight and non-contact characteristics of millimeter wave radar, it is possible to improve the accuracy and reliability of fall detection while protecting user privacy, and provide a safer and smarter living environment for the guardian.
[0068] Furthermore, a millimeter-wave radar-based intelligent fall detection system is also provided.
[0069] Figure 4 FIG. 1 is a block diagram of a fall intelligent detection system based on millimeter wave radar according to an embodiment of the present application. Figure 4As shown, according to the embodiment of the present application, the intelligent fall detection system 300 based on the millimeter wave radar includes: a signal acquisition module 310, which is used to acquire the linear frequency modulation wave signal emitted by the millimeter wave radar and the electromagnetic wave reflection signal received by the millimeter wave radar through an oscilloscope; a noise reduction module 320, which is used to perform signal noise reduction processing on the electromagnetic wave reflection signal to obtain the electromagnetic wave reflection signal after noise reduction; a time-frequency feature extraction module 330, which is used to respectively extract the time-frequency features of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain the time-frequency features of the linear frequency modulation wave signal Characteristic diagram and time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction; an autocorrelation attention enhancement module 340, which is used to perform autocorrelation attention enhancement on the time-frequency characteristic diagram of the linear frequency modulation wave signal and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction to obtain an enhanced time-frequency characteristic diagram of the linear frequency modulation wave signal and an enhanced time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction; an early warning prompt generation module 350, which is used to determine whether to generate a fall warning prompt based on the feature difference between the enhanced time-frequency characteristic diagram of the linear frequency modulation wave signal and the enhanced time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction.
[0070] As described above, the intelligent fall detection system 300 based on millimeter wave radar according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an intelligent fall detection algorithm based on millimeter wave radar. In one possible implementation, the intelligent fall detection system 300 based on millimeter wave radar according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the intelligent fall detection system 300 based on millimeter wave radar can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the intelligent fall detection system 300 based on millimeter wave radar can also be one of the many hardware modules of the wireless terminal.
[0071] Alternatively, in another example, the millimeter-wave radar-based intelligent fall detection system 300 and the wireless terminal may also be separate devices, and the millimeter-wave radar-based intelligent fall detection system 300 may be connected to the wireless terminal through a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.
[0072] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. Intelligent fall detection method based on millimeter wave radar, It is characterized in that include: collecting, by an oscilloscope, a linear frequency modulation wave signal emitted by the millimeter wave radar and an electromagnetic wave reflection signal received by the millimeter wave radar; Performing signal noise reduction processing on the electromagnetic wave reflection signal to obtain a noise-reduced electromagnetic wave reflection signal; Respectively extracting the time-frequency characteristics of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain a time-frequency characteristic graph of the linear frequency modulation wave signal and a time-frequency characteristic graph of the electromagnetic wave reflection signal after noise reduction; Performing autocorrelation attention enhancement on the linear frequency modulation wave signal time-frequency characteristic diagram and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction to obtain an enhanced linear frequency modulation wave signal time-frequency characteristic diagram and an enhanced time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction; Determining whether to generate a fall warning prompt based on a feature difference between the enhanced linear frequency modulation wave signal time-frequency feature graph and the enhanced noise-reduced electromagnetic wave reflection signal time-frequency feature graph; The time-frequency characteristics of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction are extracted respectively to obtain a time-frequency characteristic graph of the linear frequency modulation wave signal and a time-frequency characteristic graph of the electromagnetic wave reflection signal after noise reduction, including: Processing the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction by using wavelet analysis to obtain a time-frequency diagram of the linear frequency modulation wave signal and a time-frequency diagram of the electromagnetic wave reflection signal after noise reduction; Extracting time-frequency features from the time-frequency diagram of the linear frequency modulation wave signal and the time-frequency diagram of the electromagnetic wave reflection signal after noise reduction to obtain the time-frequency characteristic diagram of the linear frequency modulation wave signal and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction; The time-frequency feature extraction is performed on the time-frequency graph of the linear frequency modulation wave signal and the time-frequency graph of the electromagnetic wave reflection signal after noise reduction to obtain the time-frequency feature graph of the linear frequency modulation wave signal and the time-frequency feature graph of the electromagnetic wave reflection signal after noise reduction, including: Inputting the linear frequency modulation wave signal time-frequency diagram and the time-frequency diagram of the electromagnetic wave reflection signal after noise reduction into a signal time-frequency feature extractor based on a hole convolutional neural network model to obtain the linear frequency modulation wave signal time-frequency feature diagram and the time-frequency feature diagram of the electromagnetic wave reflection signal after noise reduction; The method of performing autocorrelation attention enhancement on the linear frequency modulation wave signal time-frequency characteristic diagram and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction to obtain an enhanced linear frequency modulation wave signal time-frequency characteristic diagram and an enhanced time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction comprises: Input the linear frequency modulation wave signal time-frequency feature map and the time-frequency feature map of the electromagnetic wave reflection signal after noise reduction into the cross-channel adaptive feature space structure enhanced self-attention module respectively to obtain the enhanced linear frequency modulation wave signal time-frequency feature map and the enhanced time-frequency feature map of the electromagnetic wave reflection signal after noise reduction; The time-frequency feature map of the linear frequency modulation wave signal and the time-frequency feature map of the electromagnetic wave reflection signal after noise reduction are respectively input into the cross-channel adaptive feature space structure enhanced self-attention module to obtain the enhanced time-frequency feature map of the linear frequency modulation wave signal and the enhanced time-frequency feature map of the electromagnetic wave reflection signal after noise reduction, including: Performing layer normalization on the linear frequency modulation wave signal time-frequency characteristic graph to obtain a normalized linear frequency modulation wave signal time-frequency characteristic graph; Performing point convolution processing on the normalized linear frequency modulation wave signal time-frequency feature map to obtain a linear frequency modulation wave signal time-frequency channel context association representation feature map; Convolutionally encoding the linear frequency modulation wave signal time-frequency channel context association representation feature map to obtain a linear frequency modulation wave signal time-frequency space context association representation feature map; Performing channel-space global interactive attention fusion on the linear frequency modulation wave signal time-frequency channel context association representation feature map and the linear frequency modulation wave signal time-frequency space context association representation feature map to obtain the enhanced linear frequency modulation wave signal time-frequency feature map; The step of performing channel-space global interactive attention fusion on the linear frequency modulation wave signal time-frequency channel context association representation feature map and the linear frequency modulation wave signal time-frequency space context association representation feature map to obtain the enhanced linear frequency modulation wave signal time-frequency feature map includes: Copying the linear frequency modulation wave signal time-frequency space context association representation feature map to obtain a backup linear frequency modulation wave signal time-frequency space context association representation feature map; Reshaping the linear frequency modulation signal time-frequency channel context association representation feature map, the linear frequency modulation signal time-frequency space context association representation feature map and the backup linear frequency modulation signal time-frequency space context association representation feature map to obtain a linear frequency modulation signal time-frequency channel context association representation feature matrix, a linear frequency modulation signal time-frequency space context association representation feature matrix and a backup linear frequency modulation signal time-frequency space context association representation feature matrix; Calculating a cross-channel cross-covariance matrix between the linear frequency modulation wave signal time-frequency channel context association representation feature matrix and the linear frequency modulation wave signal time-frequency space context association representation feature matrix; Use the Softmax function to activate the cross-channel cross-covariance matrix to obtain a global interactive attention matrix of the time-frequency features of the linear frequency modulation wave signal; Calculate the product of the backup linear frequency modulation wave signal time-frequency space context association representation feature matrix and the linear frequency modulation wave signal time-frequency feature global interactive attention matrix to obtain the attention reinforcement linear frequency modulation wave signal time-frequency feature representation matrix; Reshaping the time-frequency feature representation matrix of the attention-enhanced linear frequency modulation wave signal to obtain the time-frequency feature map of the enhanced linear frequency modulation wave signal; Wherein, determining whether to generate a fall warning prompt based on the characteristic difference between the enhanced linear frequency modulation wave signal time-frequency characteristic graph and the enhanced noise-reduced electromagnetic wave reflection signal time-frequency characteristic graph includes: Calculating a differential feature map between the enhanced linear frequency modulation wave signal time-frequency feature map and the enhanced noise-reduced electromagnetic wave reflection signal time-frequency feature map as a posture information representation feature map; Based on the posture information representation feature graph, identifying the posture type of the target human body; In response to the posture type being a fall, generating the fall warning prompt; Wherein, based on the posture information representation feature graph, identifying the posture type of the target human body includes: The posture information characterization feature map is input into a classifier-based posture recognizer to obtain a recognition result, and the recognition result is used to indicate whether the posture type is a fall.
2. A millimeter wave radar-based intelligent fall detection system, used to execute the intelligent fall detection method as claimed in claim 1, It is characterized in that include: A signal acquisition module, used for acquiring a linear frequency modulation wave signal emitted by the millimeter wave radar and an electromagnetic wave reflection signal received by the millimeter wave radar through an oscilloscope; A noise reduction module, used for performing signal noise reduction processing on the electromagnetic wave reflection signal to obtain a noise-reduced electromagnetic wave reflection signal; A time-frequency feature extraction module, used to extract the time-frequency features of the linear frequency modulation wave signal and the electromagnetic wave reflection signal after noise reduction to obtain a time-frequency feature graph of the linear frequency modulation wave signal and a time-frequency feature graph of the electromagnetic wave reflection signal after noise reduction; The autocorrelation attention enhancement module is used to perform autocorrelation attention enhancement on the time-frequency characteristic diagram of the linear frequency modulation wave signal and the time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction to obtain an enhanced time-frequency characteristic diagram of the linear frequency modulation wave signal and an enhanced time-frequency characteristic diagram of the electromagnetic wave reflection signal after noise reduction.
3. According to claim 2, the intelligent fall detection system based on millimeter wave radar, It is characterized in that Also includes: The warning prompt generating module is used to determine whether to generate a fall warning prompt based on the characteristic difference between the time-frequency characteristic diagram of the enhanced linear frequency modulation wave signal and the time-frequency characteristic diagram of the enhanced electromagnetic wave reflection signal after noise reduction.
Citation Information
Patent Citations
Dual-millimeter-wave radar human body tumble detection device and detection method
CN113447905A
Fall detection intelligent alarm method and system based on millimeter wave radar
CN115054233A
Clutter elimination method for millimeter wave radar fall detection based on time-frequency information matrix
CN116687392A
Millimeter wave radar anti-interference tumble detection method based on three-dimensional time-frequency information matrix
CN117805764A
Human body tumble detection method based on millimeter wave radar point cloud
CN118549926A
