Human body behavior recognition device and algorithm based on millimeter wave radar and ultrasound

Abstract

The invention relates to the technical field of human body behavior recognition, in particular to a human body behavior recognition device and algorithm based on millimeter wave radar and ultrasound, and the method comprises the steps of control management, data acquisition, data processing, data analysis, data storage and command output. Gesture actions of a user are obtained in an ultrasonic and millimeter wave mode, leakage of privacy of the user is reduced, more data can be conveniently obtained through different data obtaining modes, the privacy of the user is protected, meanwhile, the analysis accuracy is guaranteed, when a non-input person is detected, a camera shooting mode is adopted, and the user experience is improved. According to the method, non-input personnel images are continuously obtained, the images can be conveniently used as evidences, and the data acquisition accuracy is further improved by gathering millimeter waves and cleaning air at local positions.

Description

Technical Field

[0001] The present invention relates to the technical field of human behavior recognition, and in particular to a human behavior recognition device and algorithm based on millimeter wave radar and ultrasound. Background Art

[0002] Human behavior recognition is a biometric recognition technology based on human motion characteristics. It mainly determines and identifies individual identity, behavioral intentions and psychological states by analyzing and identifying human motions, postures and behaviors. In the field of smart homes, human behavior is generally identified through a video monitoring abnormal behavior method for smart home elderly care disclosed in the invention patent with announcement number CN106027978B and a smart home central control system disclosed in the invention patent with publication number CN117930728A.

[0003] However, it was found during use that the existing human behavior recognition is prone to leaking user privacy and is easily affected by changes in the surrounding environment and obstructions, resulting in poor practicality. Therefore, a human behavior recognition device and algorithm based on millimeter-wave radar and ultrasound are urgently needed to improve the above problems. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method of acquiring user gestures by means of ultrasound and millimeter waves, thereby reducing the leakage of user privacy, and obtaining more data through different data acquisition methods, thereby protecting user privacy and ensuring the accuracy of analysis. When non-entered persons are detected, the image of the non-entered persons is continuously obtained by video recording, which is convenient for use as evidence. In addition, a human behavior recognition device and algorithm based on millimeter wave radar and ultrasound is provided to further improve the accuracy of data acquisition by gathering millimeter waves and clearing the air in local locations.

[0005] The present invention provides a human behavior recognition algorithm based on millimeter wave radar and ultrasound, comprising:

[0006] Control and management: providing an interactive platform for users and millimeter-wave radar and ultrasonic human behavior recognition devices, verifying the user's identity information, and controlling and managing the millimeter-wave radar and ultrasonic human behavior recognition devices, allowing users to initialize and adjust the millimeter-wave radar and ultrasonic human behavior recognition devices;

[0007] Data acquisition: Use ultrasound and millimeter waves to acquire indoor environmental data, as well as user gestures, and take photos of non-entered personnel to acquire images of personnel;

[0008] Data processing: Record the body movements and images of users and their family members to facilitate data acquisition and protect the privacy of users and their family members. When users and their family members are photographed, the images are processed to retain only the hand position images, while all images of non-recorded persons are retained;

[0009] Data analysis: Use deep learning to analyze the processed data and determine the user's gestures. If multiple analysis errors occur or no analysis results are obtained, take a photo of the user's hand to obtain the user's hand image. Adjust the analysis method and re-analyze according to the user's hand condition and surrounding environment.

[0010] Data storage: storing processed data, user-entered data and analysis results;

[0011] Command output: Send the analysis results to the smart device for execution;

[0012] Preferably, the data is acquired:

[0013] Ultrasonic detection: After the ultrasonic wave is emitted, it radiates to the periphery in a fan shape in the air. During the propagation process, it encounters obstacles and generates modulation and reflection. The reflected echo signal carries characteristic information such as the distance, direction, and posture of the obstacle. When the wave source and the observer move relative to each other, the frequency and wavelength of the sound waves received by the observer will change. When the frequency emitted by the wave source remains unchanged, when the observer moves away from the wave source, the frequency of the received signal will decrease and the wavelength will become longer. When the observer moves closer to the wave source, the frequency of the received signal will increase and the wavelength will become shorter, thereby detecting the positions of static objects in the room and the users and their family members. In addition, due to the high frequency, the impact on the human body is reduced.

[0014] The relationship between the frequency fs of the source signal and the actual measured frequency fa is expressed as:

[0015]

[0016] Where c represents the propagation speed of the wave source signal in the air; v h represents the speed of the observer relative to the air. When the observer approaches the wave source, v h When the observer is far away from the source, the sign is “-”; v w Indicates the speed of the wave source relative to the air. When the wave source approaches the observer, v w When the wave source is far away from the observer, it takes a "+" sign. From the above formula, we can see that when the wave source and the observer are close to each other, f>f s ; On the contrary, f<f s

[0017] Millimeter wave detection: When the user and his family members' movements are detected by ultrasound, multiple chirp signals with the same time interval are continuously sent. Since the target is moving, the echo signal between two consecutive chirp signals will be affected by the Doppler effect caused by the target movement, resulting in a change in phase difference. According to the principle of the Doppler effect, the speed of the target can be inferred by measuring the change in phase difference, and the specific movement of the user's hand can be detected by millimeter waves.

[0018] Speed ​​calculation formula:

[0019]

[0020] Phase calculation formula:

[0021]

[0022] Image acquisition: When the millimeter wave detects non-users and user family members indoors, the active persons are photographed and images of the active persons are continuously acquired. When multiple analysis errors occur or no analysis results can be obtained, the user's hands are photographed.

[0023] Preferably, the data processing includes:

[0024] Data entry: The body movement information of the user and the user's family members is obtained by transmitting millimeter waves through a human behavior recognition device based on millimeter wave radar and ultrasound, and the images of the user and the user's family members are obtained by photographing the user and the user's family members through a human behavior recognition device based on millimeter wave radar and ultrasound;

[0025] Data storage: store the entered data;

[0026] Data comparison: when acquiring data through millimeter waves, the data acquired through millimeter waves are compared with the recorded data to determine whether the person is a user or a family member of the user. If the person is determined to be yes, the human behavior recognition device based on millimeter wave radar and ultrasound continues to detect the hand movements of the user and the family members of the user using millimeter waves. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound acquires images of the room and compares the acquired images with the recorded images to further determine whether the person is a user or a family member of the user. If the person is determined to be yes, image acquisition is stopped and millimeter waves are continued to be used to acquire data. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound continues to acquire images of active persons.

[0027] Image cropping: When multiple analysis errors occur or no analysis results can be obtained, the human behavior recognition device based on millimeter-wave radar and ultrasound obtains indoor images, then crops the images, retaining only the user's hand images and deleting the images of other positions.

[0028] Preferably, the data analysis:

[0029] Deep learning data processing: pre-process the sound waves to reduce the complexity and noise of the data while retaining the important information in the data, and then analyze and process the data through the pre-trained qualified Attention-CNN-BiLSTM model;

[0030] Preferably, the method for preprocessing the sound waves is:

[0031] Step 1: SG filtering

[0032] 1-1. Determine the size of the sliding window and the order of the polynomial. The size of the sliding window and the order of the polynomial need to be selected according to the characteristics of the signal and the expected degree of smoothness;

[0033] 1-2. Apply a sliding window to the signal data sequence. The size of the window is usually an odd number, including the center point and the data points on its left and right.

[0034] 1-3. Perform polynomial fitting on the data points in the window and use the least squares method to fit the data to obtain the best fitting polynomial coefficients;

[0035] 1-4. Use the fitting polynomial to estimate the smoothing value of the window center point;

[0036] 1-5. Move the window and repeat steps 2 to 4 to process the entire signal sequence;

[0037] Step 2: Short-time Fourier transform:

[0038] Perform FFT transformation on each frame to convert the time domain signal into the frequency domain signal. Through FFT transformation, the spectrum information of each frame is obtained;

[0039] Step 3: Mel spectrum characteristic analysis:

[0040] 3-1, pre-weighting, used to compensate for the high-frequency components of the suppressed signal and improve the signal quality, which is achieved by using a high-pass filter;

[0041] 3-2. Signal framing and windowing. The purpose of framing is to transform the signal into a continuous signal in a short time, so as to extract the signal frequency domain features using SFTT. Windowing can enhance harmonics, smooth edges, and eliminate edge effects when performing DFT on the edges.

[0042] 3-3. Calculate the power spectrum using the formula: Perform STFT, extract the signal frequency domain characteristic modulus value and square it to obtain the power spectrum.

[0043] 3-4. Mel filter bank filtering, convert the spectrum to the Mel scale, place a set of overlapping triangular filters on the Mel scale, the filter bank is usually set according to the frequency interval of the Mel scale, and its frequency response function H m (k) can be expressed as the formula:

[0044]

[0045] 3-5. Take the logarithm. The four behaviors can be identified only by the power spectrum. However, when the speeds of different behaviors are the same, it is very easy to cause misclassification by using only the power spectrum. In order to increase the difference in characteristics of different behaviors, the formula is used: The output of each filter is logarithmically calculated to obtain the logarithmic energy spectrum.

[0046] 3-6. Discrete cosine transform (DCT), through the formula: Perform discrete cosine transform on the output of the Mel filter bank after logarithmization, convert the spectrum into cepstral coefficients, extract MFCC features, usually only retain the first few cepstral coefficients after conversion, and the rest are discarded or regarded as noise;

[0047] Step 4: Dimensionality reduction:

[0048] 4-1. Initial variable standardization is done by subtracting the mean of each value of each variable and dividing by the standard deviation. After standardization is completed, all variables will be converted to the same scale;

[0049] 4-2. Calculate the covariance matrix to identify correlations. The purpose of this step is to understand how the variables of the input data set change relative to each other's means, observe whether there is any relationship between them, and remove redundant information. The covariance matrix is ​​a p×p symmetric matrix (where p is the dimension);

[0050] 4-3. Calculate the eigenvectors and eigenvalues ​​of the covariance matrix to identify the principal components. Eigenvectors and eigenvalues ​​always appear in pairs. Each eigenvector has an eigenvalue, and their number is equal to the dimension of the data. Sort the eigenvectors in descending order of eigenvalues, and obtain the principal components in order of importance;

[0051] 4-4. Create an eigenvector to decide which principal components to retain, sort the eigenvalues ​​in descending order, retain the important components and discard those that are less important (low eigenvalues), and form a vector matrix with the remaining components, namely the eigenvector;

[0052] 4-5. Restructure the data along the principal component axes, using the eigenvectors formed by the eigenvectors of the covariance matrix, and reorient the data from the original axes to the axes represented by the principal components by multiplying the transpose of the original data set by the transpose of the eigenvectors;

[0053] 4-6. The variance contribution rate can be used to determine the importance of the principal components by looking at the proportion of variance explained by each principal component. The number of principal components that can explain most of the total variance (such as 80% or 90%) can be selected as an appropriate choice to select the most representative features from among the many principal components.

[0054] Database: Provides training data for the model.

[0055] Preferably, the Attention-CNN-BiLSTM model is a combination of a convolutional neural network, a bidirectional long short-term memory network, and an attention mechanism.

[0056] Preferably, the data is stored:

[0057] Data classification: Classify the processed data, user-entered data, and analysis results:

[0058] Data storage: The classified data is stored for a limited period of time;

[0059] Data deletion: Data that exceeds the storage period will be deleted, and users can also delete data after logging in.

[0060] Preferably, the control management:

[0061] Human-computer interaction: Provide users with a platform to control human behavior recognition devices based on millimeter-wave radar and ultrasound

[0062] Information verification: verify the user's identity information;

[0063] Remote management: enables users to remotely control and manage human behavior recognition devices based on millimeter-wave radar and ultrasound.

[0064] A human behavior recognition device based on millimeter wave radar and ultrasound comprises a machine body, a guide rail, a control management module, a data processing module, a data analysis module, a data storage module, an ultrasonic sensor, a millimeter wave radar, a camera, an anti-interference mechanism and a shielding mechanism. An electric slider is arranged on the machine body, and the machine body is slidably installed on the guide rail through the electric slider. The control management module, the data processing module, the data analysis module and the data storage module are all installed in the machine body, the ultrasonic sensor, the millimeter wave radar and the camera are all installed on the machine body, the anti-sensing mechanism is installed in the machine body, and the shielding mechanism is installed on the camera. The control management module centrally controls each component, and the machine body slides on the guide rail to adjust its own position. The ultrasonic sensor emits ultrasonic waves to detect the positions of indoor static objects and users and their family members. After detecting human activities, the millimeter-wave radar emits millimeter waves to detect the hand movements of the user and the user's family members, the data processing module compares the detection results of the millimeter-wave radar with the input data, the data analysis module analyzes the processed results, and the data storage module stores the processed data, the data entered by the user and the analysis results. When there are multiple analysis errors or the analysis is fruitless, the anti-interference mechanism assists the millimeter-wave radar to obtain the hand movements of the user and the user's family members again, the shielding mechanism allows light to pass through, and the camera takes pictures of the user's hands to obtain the status image of the user's hands, thereby improving the analysis accuracy. When there are no users and user family members indoors, the camera continues to capture images of active people, thereby improving the practicality of the human behavior recognition device based on millimeter-wave radar and ultrasound.

[0065] Preferably, the anti-interference mechanism includes multiple groups of gathering plates, sleeves, telescopic motors and micro silent air compressors, the multiple groups of gathering plates are rotatably installed on the machine body around the millimeter-wave radar, and the rotational connection positions of the multiple groups of gathering plates and the machine body are provided with coil springs, the side walls of the multiple groups of gathering plates are provided with exhaust chambers, the outer walls of the multiple groups of gathering plates are provided with openings, and the openings on the multiple groups of gathering plates are communicated with the chambers inside them, the millimeter-wave radar is fixedly installed on the machine body through a bracket, and the sleeve is slidably installed on the machine body and the bracket, one end of the telescopic motor is installed on the bracket, and the other end of the telescopic motor is installed on the sleeve, and the exhaust port of the micro silent air compressor is communicated with the inside of the sleeve; through the micro The silent air compressor compresses the air and extends it through the telescopic motor, so that the sleeve squeezes the multiple groups of gathering plates, and then the multiple groups of gathering plates gather into a bucket shape. At the same time, the left end of the sleeve squeezes the outer walls of the multiple groups of gathering plates and covers the openings of the multiple groups of gathering plates. The millimeter waves emitted by the millimeter-wave radar are gathered by the multiple groups of gathering plates to increase the millimeter wave intensity. At the same time, the air is discharged into the sleeve through the micro silent air compressor. The air enters the cavity of the multiple groups of gathering plates and is then ejected through the left end of the multiple groups of gathering plates, blowing away the water vapor in front of the millimeter-wave radar in a short time, so that the millimeter-wave radar can obtain the user's hand motion data in time, thereby improving the practicality of the human behavior recognition device based on millimeter-wave radar and ultrasound.

[0066] Preferably, the shielding mechanism includes a first polarizer, a second polarizer, a gear ring, a silent motor, a gear and a counter, the first polarizer is fixedly installed on the left part of the camera, the second polarizer is rotatably installed on the left part of the camera, the gear ring is sleeved on the second polarizer, the silent motor is installed on the body, the gear is installed on the output shaft of the silent motor, and the gear and the silent motor are meshingly connected, the counter is installed below the gear, and a counting key is provided on the top of the counter, a reset key is provided on the bottom of the counter, and a display screen is provided on the left end of the counter; when using the camera, the silent motor is turned on, and the second polarizer rotates through the meshing transmission of the gear and the gear ring, and then the light passes through the second polarizer and the first polarizer, and then is photographed by the camera, and each time the gear rotates, the bottom of the gear squeezes the counting key of the counter, the counter counts, and displays it through the display screen, so that the user can timely understand the number of times the camera has been operated, and timely delete the captured images involving meanings, thereby improving the practicality of the human behavior recognition device based on millimeter wave radar and ultrasound.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] 1. Use ultrasound and millimeter waves to obtain user gestures to reduce the leakage of user privacy. And through different data acquisition methods, it is convenient to obtain more data, while protecting user privacy and ensuring the accuracy of analysis;

[0069] 2. When a non-entered person is detected, the image of the non-entered person is continuously obtained by video recording, which is convenient for use as evidence;

[0070] 3. Further improve the accuracy of data acquisition by focusing millimeter waves and cleaning the air in local locations. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a schematic diagram of the flow structure of the human behavior recognition algorithm based on millimeter wave radar and ultrasound of the present invention;

[0072] Figure 2 It is a schematic diagram of the process structure of data acquisition in the present invention;

[0073] Figure 3 It is a schematic diagram of the flow structure of data processing of the present invention;

[0074] Figure 4 It is a schematic diagram of the process structure of the data analysis of the present invention;

[0075] Figure 5 It is a schematic diagram of the process structure of data storage in the present invention;

[0076] Figure 6 It is a schematic diagram of the process structure during initialization of the present invention;

[0077] Figure 7 It is a structural diagram of the Attention-CNN-BiLSTM model of the present invention.

[0078] Figure 8 It is a schematic diagram of the axonometric structure of the human behavior recognition based on millimeter wave radar and ultrasound of the present invention;

[0079] Fig. 9 It is a schematic diagram of the front cross-sectional structure of the human behavior recognition based on millimeter wave radar and ultrasound of the present invention;

[0080] Fig.10 The present invention Figure 8 A schematic diagram of the enlarged structure of part A;

[0081] Fig.11 The present invention n Sample SG filtering schematic;

[0082] Fig.12 This is a schematic diagram of the effects of the four ultrasonic behaviors after using SG filtering in the present invention;.

[0083] Markings in the accompanying drawings: 1. body; 2. guide rail; 3. control management module; 4. data processing module; 5. data analysis module; 6. data storage module; 7. ultrasonic sensor; 8. millimeter wave radar; 9. camera; 10. gathering plate; 11. sleeve; 12. telescopic motor; 13. micro silent air compressor; 14. first polarizer; 15. second polarizer; 16. gear ring; 17. silent motor; 18. gear; 19. counter. DETAILED DESCRIPTION

[0084] In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0085] like Figures 1 to 7 As shown, a human behavior recognition algorithm based on millimeter wave radar and ultrasound includes:

[0086] Control and management: providing an interactive platform for users and millimeter-wave radar and ultrasonic human behavior recognition devices, verifying the user's identity information, and controlling and managing the millimeter-wave radar and ultrasonic human behavior recognition devices, allowing users to initialize and adjust the millimeter-wave radar and ultrasonic human behavior recognition devices;

[0087] Data acquisition: Use ultrasound and millimeter waves to acquire indoor environmental data, as well as user gestures, and take photos of non-entered personnel to acquire images of personnel;

[0088] Data processing: Record the body movements and images of users and their family members to facilitate data acquisition and protect the privacy of users and their family members. When users and their family members are photographed, the images are processed to retain only the hand position images, while all images of non-recorded persons are retained;

[0089] Data analysis: Use deep learning to analyze the processed data and determine the user's gestures. If multiple analysis errors occur or no analysis results can be obtained, take a photo of the user's hand to obtain the user's hand image. Adjust the analysis method and re-analyze according to the user's hand condition and surrounding environment (for example, if the user's hand has water stains, oil stains, or is wearing gloves, etc.);

[0090] Data storage: storing processed data, user-entered data and analysis results;

[0091] Command output: Send the analysis results to the smart device for execution;

[0092] Preferably, the data is acquired:

[0093] Ultrasonic detection: After the ultrasonic wave is emitted, it radiates to the periphery in a fan shape in the air. During the propagation process, it encounters obstacles and generates modulation and reflection. The reflected echo signal carries characteristic information such as the distance, direction, and posture of the obstacle. When the wave source and the observer move relative to each other, the frequency and wavelength of the sound waves received by the observer will change. When the frequency emitted by the wave source remains unchanged, when the observer moves away from the wave source, the frequency of the received signal will decrease and the wavelength will become longer. When the observer moves closer to the wave source, the frequency of the received signal will increase and the wavelength will become shorter, thereby detecting the positions of static objects in the room and the users and their family members. In addition, due to the high frequency, the impact on the human body is reduced.

[0094] The relationship between the frequency fs of the source signal and the actual measured frequency fa is expressed as:

[0095]

[0096] Where c represents the propagation speed of the wave source signal in the air; v h represents the speed of the observer relative to the air. When the observer approaches the wave source, v h When the observer is far away from the source, the sign is “-”; v w Indicates the speed of the wave source relative to the air. When the wave source approaches the observer, v w When the wave source is far away from the observer, it takes a "+" sign. From the above formula, we can see that when the wave source and the observer are close to each other, f>f s ; On the contrary, f<f s

[0097] Millimeter wave detection: When the user and his family members' movements are detected by ultrasound, multiple chirp signals with the same time interval are continuously sent. Since the target is moving, the echo signal between two consecutive chirp signals will be affected by the Doppler effect caused by the target movement, resulting in a change in phase difference. According to the principle of the Doppler effect, the speed of the target can be inferred by measuring the change in phase difference, and the specific movement of the user's hand can be detected by millimeter waves.

[0098] Speed ​​calculation formula:

[0099]

[0100] Phase calculation formula:

[0101]

[0102] Image acquisition: When the millimeter wave detects non-users and user family members indoors, the active persons are photographed and images of the active persons are continuously acquired. When multiple analysis errors occur or no analysis results can be obtained, the user's hands are photographed.

[0103] Preferably, the data processing includes:

[0104] Data entry: The body movement information of the user and the user's family members is obtained by transmitting millimeter waves through a human behavior recognition device based on millimeter wave radar and ultrasound, and the images of the user and the user's family members are obtained by photographing the user and the user's family members through a human behavior recognition device based on millimeter wave radar and ultrasound;

[0105] Data storage: store the entered data;

[0106] Data comparison: when acquiring data through millimeter waves, the data acquired through millimeter waves are compared with the recorded data to determine whether the person is a user or a family member of the user. If the person is determined to be yes, the human behavior recognition device based on millimeter wave radar and ultrasound continues to detect the hand movements of the user and the family members of the user using millimeter waves. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound acquires images of the room and compares the acquired images with the recorded images to further determine whether the person is a user or a family member of the user. If the person is determined to be yes, image acquisition is stopped and millimeter waves are continued to be used to acquire data. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound continues to acquire images of active persons.

[0107] Image cropping: When multiple analysis errors occur or no analysis results can be obtained, the human behavior recognition device based on millimeter-wave radar and ultrasound obtains indoor images, then crops the images, retaining only the user's hand images and deleting the images of other positions.

[0108] Preferably, the data analysis:

[0109] Deep learning data processing: pre-process the sound waves to reduce the complexity and noise of the data while retaining the important information in the data, and then analyze and process the data through the pre-trained qualified Attention-CNN-BiLSTM model;

[0110] Preferably, the method for pre-processing the sound waves is:

[0111] Step 1: SG filtering

[0112] 1-1. Determine the size of the sliding window and the order of the polynomial. The size of the sliding window and the order of the polynomial need to be selected according to the characteristics of the signal and the expected degree of smoothness;

[0113] 1-2. Apply a sliding window to the signal data sequence. The size of the window is usually an odd number, including the center point and the data points on its left and right.

[0114] 1-3. Perform polynomial fitting on the data points in the window and use the least squares method to fit the data to obtain the best fitting polynomial coefficients;

[0115] 1-4. Use the fitting polynomial to estimate the smoothing value of the window center point;

[0116] 1-5. Move the window and repeat steps 2 to 4 to process the entire signal sequence;

[0117] like Fig.11 and Fig.12 As shown, x n Taking samples as an example, the basic idea behind least squares polynomial smoothing is shown. Least squares smoothing is illustrated by locally fitting a quadratic polynomial (solid line) to five input samples: · represents the input sample, represents the least squares output sample, and x represents the effective impulse response sample (weighting constant);

[0118] Assume that the filter window width is N s =2m+1, all measurement points in the window can be expressed as Use k-1 order polynomial to fit the points in the window to get the formula:

[0119]

[0120] In the formula, x represents the fitted data, α i represents the fitting coefficient, t m represents the sampling point, k-1 represents the number of fitting points, and each filter window with a width of n can be represented by n of the above formulas, which together form a k-dimensional linear equation system, which can be expressed as a matrix equation:

[0121]

[0122] , use the least squares method to solve the fitting coefficient α i The filtered signal can be fitted.

[0123] Step 2: Short-time Fourier transform:

[0124] Perform FFT transformation on each frame to convert the time domain signal into the frequency domain signal. Through FFT transformation, the spectrum information of each frame is obtained;

[0125] Step 3: Mel spectrum characteristic analysis:

[0126] 3-1, pre-weighting, used to compensate for the high-frequency components of the suppressed signal and improve the signal quality, which is achieved by using a high-pass filter;

[0127] 3-2. Signal framing and windowing. The purpose of framing is to transform the signal into a continuous signal in a short time, so as to extract the signal frequency domain features using SFTT. Windowing can enhance harmonics, smooth edges, and eliminate edge effects when performing DFT on the edges.

[0128] 3-3. Calculate the power spectrum using the formula: Perform STFT, extract the signal frequency domain characteristic modulus value and square it to obtain the power spectrum.

[0129] 3-4. Mel filter bank filtering, convert the spectrum to the Mel scale, place a set of overlapping triangular filters on the Mel scale, the filter bank is usually set according to the frequency interval of the Mel scale, and its frequency response function H m (k) can be expressed as the formula:

[0130]

[0131] 3-5. Take the logarithm. The four behaviors can be identified only by the power spectrum. However, when the speeds of different behaviors are the same, it is very easy to cause misclassification by using only the power spectrum. In order to increase the difference in characteristics of different behaviors, the formula is used: The output of each filter is logarithmically calculated to obtain the logarithmic energy spectrum.

[0132] 3-6. Discrete cosine transform (DCT), through the formula: Perform discrete cosine transform on the output of the Mel filter bank after logarithmization, convert the spectrum into cepstral coefficients, extract MFCC features, usually only retain the first few cepstral coefficients after conversion, and the rest are discarded or regarded as noise;

[0133] Step 4: Dimensionality reduction:

[0134] 4-1. Initial variable standardization is done by subtracting the mean of each value of each variable and dividing by the standard deviation. After standardization is completed, all variables will be converted to the same scale;

[0135] 4-2. Calculate the covariance matrix to identify correlations. The purpose of this step is to understand how the variables of the input data set change relative to each other's means, observe whether there is any relationship between them, and remove redundant information. The covariance matrix is ​​a p×p symmetric matrix (where p is the dimension);

[0136] 4-3. Calculate the eigenvectors and eigenvalues ​​of the covariance matrix to identify the principal components. Eigenvectors and eigenvalues ​​always appear in pairs. Each eigenvector has an eigenvalue, and their number is equal to the dimension of the data. Sort the eigenvectors in descending order of eigenvalues, and obtain the principal components in order of importance;

[0137] 4-4. Create an eigenvector to decide which principal components to retain, sort the eigenvalues ​​in descending order, retain the important components and discard those that are less important (low eigenvalues), and form a vector matrix with the remaining components, namely the eigenvector;

[0138] 4-5. Restructure the data along the principal component axes, using the eigenvectors formed by the eigenvectors of the covariance matrix, and reorient the data from the original axes to the axes represented by the principal components by multiplying the transpose of the original data set by the transpose of the eigenvectors;

[0139] 4-6. The variance contribution rate can be used to determine the importance of the principal components by looking at the proportion of variance explained by each principal component. The number of principal components that can explain most of the total variance (such as 80% or 90%) can be selected as an appropriate choice to select the most representative features from among the many principal components.

[0140] Database: Provides training data for the model.

[0141] Specific parameters of the Attetnion-CNN-BiLSTM network model with attention mechanism:

[0142]

[0143]

[0144] Database: provides training data for the model;

[0145] The Attention-CNN-BiLSTM model is a combination of a convolutional neural network, a bidirectional long short-term memory network, and an attention mechanism;

[0146] The data is stored:

[0147] Data classification: Classify the processed data, user-entered data, and analysis results:

[0148] Data storage: The classified data is stored for a limited period of time;

[0149] Data deletion: Delete data that exceeds the storage period, and users can also delete data after logging in;

[0150] The control management:

[0151] Human-computer interaction: Provide users with a platform to control human behavior recognition devices based on millimeter-wave radar and ultrasound. Information verification: Verify the user's identity information;

[0152] Remote management: enables users to remotely control and manage human behavior recognition devices based on millimeter-wave radar and ultrasound.

[0153] Example 1

[0154] like Figures 8 to 10 As shown, a human behavior recognition device based on millimeter wave radar and ultrasound includes:

[0155] Control and management: providing an interactive platform for users and millimeter-wave radar and ultrasonic human behavior recognition devices, verifying the user's identity information, and controlling and managing the millimeter-wave radar and ultrasonic human behavior recognition devices, allowing users to initialize and adjust the millimeter-wave radar and ultrasonic human behavior recognition devices;

[0156] Data acquisition: Use ultrasound and millimeter waves to acquire indoor environmental data, as well as user gestures, and take photos of non-entered personnel to acquire images of personnel;

[0157] Data processing: Record the body movements and images of users and their family members to facilitate data acquisition and protect the privacy of users and their family members. When users and their family members are photographed, the images are processed to retain only the hand position images, while all images of non-recorded persons are retained;

[0158] Data analysis: Use deep learning to analyze the processed data and determine the user's gestures. If multiple analysis errors occur or no analysis results are obtained, take a photo of the user's hand to obtain the user's hand image. Adjust the analysis method and re-analyze according to the user's hand condition and surrounding environment.

[0159] Data storage: storing processed data, user-entered data and analysis results;

[0160] Command output: Send the analysis results to the smart device for execution;

[0161] The data acquisition:

[0162] Ultrasonic detection: Use ultrasonic waves to detect the location of static objects in the room and the user and his / her family members;

[0163] Millimeter wave detection: When the user and his / her family members' movements are detected by ultrasound, the specific movements of the user's hands are detected by millimeter waves;

[0164] Image acquisition: When the millimeter wave detects non-users and user family members in the room, the active persons are photographed and images of active persons are continuously acquired. When there are multiple analysis errors or no analysis results can be obtained, the user's hands are photographed;

[0165] The data processing includes:

[0166] Data entry: obtaining body movement information of the user and the user's family members by emitting millimeter waves through a human behavior recognition device based on millimeter wave radar and ultrasound, and obtaining images of the user and the user's family members by photographing the user and the user's family members through a human behavior recognition device based on millimeter wave radar and ultrasound;

[0167] Data storage: store the entered data;

[0168] Data comparison: when acquiring data through millimeter waves, the data acquired through millimeter waves are compared with the recorded data to determine whether the person is a user or a family member of the user. If the person is determined to be yes, the human behavior recognition device based on millimeter wave radar and ultrasound continues to detect the hand movements of the user and the family members of the user using millimeter waves. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound acquires images of the room and compares the acquired images with the recorded images to further determine whether the person is a user or a family member of the user. If the person is determined to be yes, image acquisition is stopped and millimeter waves are continued to be used to acquire data. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound continues to acquire images of active persons.

[0169] Image cropping: When multiple analysis errors occur or no analysis results are obtained, the human behavior recognition device based on millimeter-wave radar and ultrasound obtains indoor images, then crops the images to retain only the user's hand image and deletes the images of other positions;

[0170] The data analysis:

[0171] Deep learning data processing: pre-process the ultrasound to reduce the complexity and noise of the data while retaining the important information in the data, and then analyze and process the data through the pre-trained qualified Attention-CNN-BiLSTM model;

[0172] Database: provides training data for the model;

[0173] The Attention-CNN-BiLSTM model is a combination of a convolutional neural network, a bidirectional long short-term memory network, and an attention mechanism;

[0174] The data is stored:

[0175] Data classification: Classify the processed data, user-entered data, and analysis results:

[0176] Data storage: The classified data is stored for a limited period of time;

[0177] Data deletion: Delete data that exceeds the storage period, and users can also delete data after logging in;

[0178] The control management:

[0179] Human-computer interaction: Provide users with a platform to control human behavior recognition devices based on millimeter-wave radar and ultrasound. Information verification: Verify the user's identity information;

[0180] Remote management: enables users to remotely control and manage human behavior recognition devices based on millimeter-wave radar and ultrasound;

[0181] It includes a body 1, a guide rail 2, a control management module 3, a data processing module 4, a data analysis module 5, a data storage module 6, an ultrasonic sensor 7, a millimeter wave radar 8, a camera 9, an anti-interference mechanism and a shielding mechanism. The body 1 is provided with an electric slider, and the body 1 is slidably installed on the guide rail 2 through the electric slider. The control management module 3, the data processing module 4, the data analysis module 5 and the data storage module 6 are all installed in the body 1, the ultrasonic sensor 7, the millimeter wave radar 8 and the camera 9 are all installed on the body 1, the anti-sensing mechanism is installed in the body 1, and the shielding mechanism is installed on the camera 9;

[0182] The anti-interference mechanism includes multiple groups of gathering plates 10, sleeves 11, telescopic motors 12 and micro silent air compressors 13. The multiple groups of gathering plates 10 are rotatably mounted on the machine body 1 around the millimeter wave radar 8, and the rotational connection positions of the multiple groups of gathering plates 10 and the machine body 1 are provided with coil springs. Exhaust chambers are provided in the side walls of the multiple groups of gathering plates 10, and openings are provided on the outer walls of the multiple groups of gathering plates 10, and the openings on the multiple groups of gathering plates 10 are communicated with the chambers therein. The millimeter wave radar 8 is fixedly mounted on the machine body 1 through a bracket, and the sleeve 11 is slidably mounted on the machine body 1 and the bracket, one end of the telescopic motor 12 is mounted on the bracket, and the other end of the telescopic motor 12 is mounted on the sleeve 11, and the exhaust port of the micro silent air compressor 13 is communicated with the interior of the sleeve 11;

[0183] The shielding mechanism includes a first polarizing plate 14, a second polarizing plate 15, a gear ring 16, a silent motor 17, a gear 18 and a counter 19, wherein the first polarizing plate 14 is fixedly mounted on the left portion of the camera 9, the second polarizing plate 15 is rotatably mounted on the left portion of the camera 9, the gear ring 16 is sleeved on the second polarizing plate 15, the silent motor 17 is mounted on the body 1, the gear 18 is mounted on the output shaft of the silent motor 17, and the gear 18 is meshingly connected with the silent motor 17, the counter 19 is mounted below the gear 18, and a counting key is provided on the top of the counter 19, a reset key is provided on the bottom of the counter 19, and a display screen is provided on the left end of the counter 19;

[0184] The control management module 3 centrally controls each component, and the body 1 slides on the guide rail 2 to adjust its own position. The ultrasonic sensor 7 emits ultrasonic waves to detect the positions of indoor static objects and users and their family members. After detecting human activities, the millimeter wave radar 8 emits millimeter waves to detect the hand movements of users and their family members. The data processing module 4 compares the detection results of the millimeter wave radar 8 with the input data, and the data analysis module 5 analyzes the processed results. The data storage module 6 stores the processed data, the data input by the user, and the analysis results. When there are multiple analysis errors or no results, the anti-interference mechanism assists the millimeter wave radar 8 to obtain the user and his family members again. The hand movement of the user's family members turns on the silent motor 17, which is meshed with the gear 18 and the gear ring 16 to rotate the second polarizer 15. Then the light passes through the second polarizer 15 and the first polarizer 14, and the user's hand is photographed by the camera 9 to obtain the state image of the user's hand, thereby improving the analysis accuracy. When there are no users or family members in the room, the camera 9 continues to capture the images of active persons, and each time the gear 18 rotates, the bottom of the gear 18 squeezes the counting key of the counter 19, and the counter 19 counts and displays it on the display screen, so that the user can timely understand the number of times the camera 9 has been operated, and timely delete the captured images of interest, thereby improving the practicality of the human behavior recognition device based on millimeter wave radar and ultrasound.

[0185] Example 2

[0186] like Figures 8 to 10 As shown, a human behavior recognition device based on millimeter wave radar and ultrasound includes:

[0187] Control and management: providing an interactive platform for users and millimeter-wave radar and ultrasonic human behavior recognition devices, verifying the user's identity information, and controlling and managing the millimeter-wave radar and ultrasonic human behavior recognition devices, allowing users to initialize and adjust the millimeter-wave radar and ultrasonic human behavior recognition devices;

[0188] Data acquisition: Use ultrasound and millimeter waves to acquire indoor environmental data, as well as user gestures, and take photos of non-entered personnel to acquire images of personnel;

[0189] Data processing: Record the body movements and images of users and their family members to facilitate data acquisition and protect the privacy of users and their family members. When users and their family members are photographed, the images are processed to retain only the hand position images, while all images of non-recorded persons are retained;

[0190] Data analysis: Use deep learning to analyze the processed data and determine the user's gestures. If multiple analysis errors occur or no analysis results are obtained, take a photo of the user's hand to obtain the user's hand image. Adjust the analysis method and re-analyze according to the user's hand condition and surrounding environment.

[0191] Data storage: storing processed data, user-entered data and analysis results;

[0192] Command output: Send the analysis results to the smart device for execution;

[0193] The data acquisition:

[0194] Ultrasonic detection: Use ultrasonic waves to detect the location of static objects in the room and the user and his / her family members;

[0195] Millimeter wave detection: When the user and his / her family members' movements are detected by ultrasound, the specific movements of the user's hands are detected by millimeter waves;

[0196] Image acquisition: When the millimeter wave detects non-users and user family members in the room, the active persons are photographed and images of active persons are continuously acquired. When there are multiple analysis errors or no analysis results can be obtained, the user's hands are photographed;

[0197] The data processing includes:

[0198] Data entry: obtaining body movement information of the user and the user's family members by emitting millimeter waves through a human behavior recognition device based on millimeter wave radar and ultrasound, and obtaining images of the user and the user's family members by photographing the user and the user's family members through a human behavior recognition device based on millimeter wave radar and ultrasound;

[0199] Data storage: store the entered data;

[0200] Data comparison: when acquiring data through millimeter waves, the data acquired through millimeter waves are compared with the recorded data to determine whether the person is a user or a family member of the user. If the person is determined to be yes, the human behavior recognition device based on millimeter wave radar and ultrasound continues to detect the hand movements of the user and the family members of the user using millimeter waves. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound acquires images of the room and compares the acquired images with the recorded images to further determine whether the person is a user or a family member of the user. If the person is determined to be yes, image acquisition is stopped and millimeter waves are continued to be used to acquire data. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound continues to acquire images of active persons.

[0201] Image cropping: When multiple analysis errors occur or no analysis results are obtained, the human behavior recognition device based on millimeter-wave radar and ultrasound obtains indoor images, then crops the images to retain only the user's hand image and deletes the images of other positions;

[0202] The data analysis:

[0203] Deep learning data processing: pre-process the ultrasound to reduce the complexity and noise of the data while retaining the important information in the data, and then analyze and process the data through the pre-trained qualified Attention-CNN-BiLSTM model;

[0204] Database: provides training data for the model;

[0205] The Attention-CNN-BiLSTM model is a combination of a convolutional neural network, a bidirectional long short-term memory network, and an attention mechanism;

[0206] The data is stored:

[0207] Data classification: Classify the processed data, user-entered data, and analysis results:

[0208] Data storage: The classified data is stored for a limited period of time;

[0209] Data deletion: Delete data that exceeds the storage period, and users can also delete data after logging in;

[0210] The control management:

[0211] Human-computer interaction: Provide users with a platform to control human behavior recognition devices based on millimeter-wave radar and ultrasound. Information verification: Verify the user's identity information;

[0212] Remote management: enables users to remotely control and manage human behavior recognition devices based on millimeter-wave radar and ultrasound;

[0213] It includes a body 1, a guide rail 2, a control management module 3, a data processing module 4, a data analysis module 5, a data storage module 6, an ultrasonic sensor 7, a millimeter wave radar 8, a camera 9, an anti-interference mechanism and a shielding mechanism. The body 1 is provided with an electric slider, and the body 1 is slidably installed on the guide rail 2 through the electric slider. The control management module 3, the data processing module 4, the data analysis module 5 and the data storage module 6 are all installed in the body 1, the ultrasonic sensor 7, the millimeter wave radar 8 and the camera 9 are all installed on the body 1, the anti-sensing mechanism is installed in the body 1, and the shielding mechanism is installed on the camera 9;

[0214] The anti-interference mechanism includes multiple groups of gathering plates 10, sleeves 11, telescopic motors 12 and micro silent air compressors 13. The multiple groups of gathering plates 10 are rotatably mounted on the machine body 1 around the millimeter wave radar 8, and the rotational connection positions of the multiple groups of gathering plates 10 and the machine body 1 are provided with coil springs. Exhaust chambers are provided in the side walls of the multiple groups of gathering plates 10, and openings are provided on the outer walls of the multiple groups of gathering plates 10, and the openings on the multiple groups of gathering plates 10 are communicated with the chambers therein. The millimeter wave radar 8 is fixedly mounted on the machine body 1 through a bracket, and the sleeve 11 is slidably mounted on the machine body 1 and the bracket, one end of the telescopic motor 12 is mounted on the bracket, and the other end of the telescopic motor 12 is mounted on the sleeve 11, and the exhaust port of the micro silent air compressor 13 is communicated with the interior of the sleeve 11;

[0215] The shielding mechanism includes a first polarizing plate 14, a second polarizing plate 15, a gear ring 16, a silent motor 17, a gear 18 and a counter 19, wherein the first polarizing plate 14 is fixedly mounted on the left portion of the camera 9, the second polarizing plate 15 is rotatably mounted on the left portion of the camera 9, the gear ring 16 is sleeved on the second polarizing plate 15, the silent motor 17 is mounted on the body 1, the gear 18 is mounted on the output shaft of the silent motor 17, and the gear 18 is meshingly connected with the silent motor 17, the counter 19 is mounted below the gear 18, and a counting key is provided on the top of the counter 19, a reset key is provided on the bottom of the counter 19, and a display screen is provided on the left end of the counter 19;

[0216] Each component is centrally controlled by the control management module 3, and the body 1 slides on the guide rail 2 to adjust its own position. The ultrasonic sensor 7 emits ultrasonic waves to detect the positions of indoor static objects and users and their family members. After detecting human activities, the millimeter wave radar 8 emits millimeter waves to detect the hand movements of users and their family members. The detection result of the millimeter wave radar 8 is compared with the input data by the data processing module 4, and the processed result is analyzed by the data analysis module 5. The processed data, the data input by the user and the analysis result are stored by the data storage module 6. When multiple analysis errors or unsuccessful results are found, the air is compressed by the micro silent air compressor 13 and extended by the telescopic motor 12, so that the sleeve 11 squeezes the multiple groups of gathering plates 10, and then the multiple groups of gathering plates 10 gather into a bucket shape. At the same time, the left end of the sleeve 11 squeezes the outer wall of the multiple groups of gathering plates 10 and covers the openings of the multiple groups of gathering plates 10. The millimeter waves emitted by the millimeter wave radar 8 are Gathering increases the intensity of millimeter waves, and at the same time, the air is discharged into the sleeve 11 through the micro silent air compressor 13. The air enters the chamber of the multiple groups of gathering plates 10, and then is ejected through the left end of the multiple groups of gathering plates 10, blowing away the water vapor in front of the millimeter wave radar 8 in a short time. The millimeter wave radar 8 obtains the hand movements of the user and the user's family members again, turns on the silent motor 17, and the gear 18 is meshed with the gear ring 16 to rotate the second polarizer 15. Then the light passes through the second polarizer 15 and the first polarizer 14, and the user's hand is photographed by the camera 9 to obtain the state image of the user's hand, thereby improving the analysis accuracy. When there are no users and user family members in the room, the camera 9 continues to photograph and obtain the image of the active person, and each time the gear 18 rotates, the bottom of the gear 18 squeezes the counting key of the counter 19, and the counter 19 counts and displays it through the display screen, so that the user can timely understand the number of times the camera 9 has been operated, and timely delete the captured images involving meaning, thereby improving the practicality of the human behavior recognition device based on millimeter wave radar and ultrasound.

[0217] The human behavior recognition device based on millimeter wave radar and ultrasound of the present invention has common mechanical installation, connection or setting methods, which can be implemented as long as they can achieve their beneficial effects; the ultrasonic sensor 7 and the millimeter wave radar 8 are both provided with a transmitting end and a receiving end; the human behavior recognition device based on millimeter wave radar and ultrasound of the present invention and the telescopic motor 12, micro silent air compressor 13, silent motor 17 and counter 19 of the algorithm are purchased on the market, and technicians in the industry only need to install and operate them according to the accompanying instruction manual, without the need for creative labor by technicians in this field.

[0218] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A human behavior recognition algorithm based on millimeter wave radar and ultrasound, characterized in that: include: Control and management: providing an interactive platform for users and millimeter-wave radar and ultrasonic human behavior recognition devices, verifying the user's identity information, and controlling and managing the millimeter-wave radar and ultrasonic human behavior recognition devices, allowing users to initialize and adjust the millimeter-wave radar and ultrasonic human behavior recognition devices; Data acquisition: Use ultrasound and millimeter waves to acquire indoor environmental data, as well as user gestures, and take photos of non-entered personnel to acquire images of personnel; Data processing: Record the body movements and images of users and their family members to facilitate data acquisition and protect the privacy of users and their family members. When users and their family members are photographed, the images are processed to retain only the hand position images, while all images of non-recorded persons are retained; Data analysis: Use deep learning to analyze the processed data and determine the user's gestures. If multiple analysis errors occur or no analysis results are obtained, take a photo of the user's hand to obtain the user's hand image. Adjust the analysis method and re-analyze according to the user's hand condition and surrounding environment. Data storage: storing processed data, user-entered data and analysis results; Command output: Send the analysis results to the smart device for execution.

2. The human behavior recognition algorithm based on millimeter wave radar and ultrasound as claimed in claim 1, characterized in that: The data acquisition: Ultrasonic detection: Use ultrasonic waves to detect the location of static objects in the room and the user and his / her family members; Millimeter wave detection: When the user and his / her family members' movements are detected by ultrasound, the specific movements of the user's hands are detected by millimeter waves; Image acquisition: When the millimeter wave detects non-users and user family members indoors, the active persons are photographed and images of the active persons are continuously acquired. When multiple analysis errors occur or no analysis results can be obtained, the user's hands are photographed.

3. The human behavior recognition algorithm based on millimeter wave radar and ultrasound as claimed in claim 1, characterized in that: The data processing includes: Data entry: obtaining body movement information of the user and the user's family members by emitting millimeter waves through a human behavior recognition device based on millimeter wave radar and ultrasound, and obtaining images of the user and the user's family members by photographing the user and the user's family members through a human behavior recognition device based on millimeter wave radar and ultrasound; Data storage: store the entered data; Data comparison: when acquiring data through millimeter waves, the data acquired through millimeter waves are compared with the recorded data to determine whether the person is a user or a family member of the user. If the person is determined to be yes, the human behavior recognition device based on millimeter wave radar and ultrasound continues to detect the hand movements of the user and the family members of the user using millimeter waves. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound acquires images of the room and compares the acquired images with the recorded images to further determine whether the person is a user or a family member of the user. If the person is determined to be yes, image acquisition is stopped and millimeter waves are continued to be used to acquire data. If the person is determined to be no, the human behavior recognition device based on millimeter wave radar and ultrasound continues to acquire images of active persons. Image cropping: When multiple analysis errors occur or no analysis results can be obtained, the human behavior recognition device based on millimeter-wave radar and ultrasound obtains indoor images, then crops the images, retaining only the user's hand images and deleting the images of other positions.

4. The human behavior recognition algorithm based on millimeter wave radar and ultrasound as claimed in claim 1, characterized in that: The data analysis: Deep learning data processing: pre-process the sound waves to reduce the complexity and noise of the data while retaining the important information in the data, and then analyze and process the data through the pre-trained qualified Attention-CNN-BiLSTM model; Database: Provides training data for the model.

5. The human behavior recognition algorithm based on millimeter wave radar and ultrasound as claimed in claim 4, characterized in that: The Attention-CNN-BiLSTM model is a combination of a convolutional neural network, a bidirectional long short-term memory network, and an attention mechanism.

6. The human behavior recognition algorithm based on millimeter wave radar and ultrasound as claimed in claim 1, characterized in that: The data is stored: Data classification: Classify the processed data, user-entered data, and analysis results: Data storage: The classified data is stored for a limited period of time; Data deletion: Data that exceeds the storage period will be deleted, and users can also delete data after logging in.

7. The human behavior recognition algorithm based on millimeter wave radar and ultrasound as claimed in claim 1, characterized in that: The control management: Human-computer interaction: Provide users with a platform to control human behavior recognition devices based on millimeter-wave radar and ultrasound. Information verification: Verify the user's identity information; Remote management: enables users to remotely control and manage human behavior recognition devices based on millimeter-wave radar and ultrasound.

8. A human behavior recognition device based on millimeter wave radar and ultrasound, characterized in that: The invention comprises a machine body (1), a guide rail (2), a control management module (3), a data processing module (4), a data analysis module (5), a data storage module (6), an ultrasonic sensor (7), a millimeter wave radar (8), a camera (9), an anti-interference mechanism and a shielding mechanism. The machine body (1) is provided with an electric slider, the machine body (1) is slidably mounted on the guide rail (2) through the electric slider, the control management module (3), the data processing module (4), the data analysis module (5) and the data storage module (6) are all mounted in the machine body (1), the ultrasonic sensor (7), the millimeter wave radar (8) and the camera (9) are all mounted on the machine body (1), the anti-interference mechanism is mounted in the machine body (1), and the shielding mechanism is mounted on the camera (9).

9. The human behavior recognition device based on millimeter wave radar and ultrasound as claimed in claim 8, characterized in that: The anti-interference mechanism comprises a plurality of gathering plates (10), a sleeve (11), a telescopic motor (12) and a miniature silent air compressor (13); the plurality of gathering plates (10) are rotatably mounted on the machine body (1) around the millimeter wave radar (8); and the rotational connection positions between the plurality of gathering plates (10) and the machine body (1) are provided with coil springs; exhaust chambers are provided in the side walls of the plurality of gathering plates (10); openings are provided on the outer walls of the plurality of gathering plates (10); and the openings on the plurality of gathering plates (10) are communicated with the chambers therein; the millimeter wave radar (8) is fixedly mounted on the machine body (1) through a bracket, and the sleeve (11) is slidably mounted on the machine body (1) and the bracket; one end of the telescopic motor (12) is mounted on the bracket, and the other end of the telescopic motor (12) is mounted on the sleeve (11); and the exhaust port of the miniature silent air compressor (13) is communicated with the interior of the sleeve (11).

10. The human behavior recognition device based on millimeter wave radar and ultrasound as claimed in claim 8, characterized in that: The shielding mechanism comprises a first polarizing plate (14), a second polarizing plate (15), a gear ring (16), a mute motor (17), a gear (18) and a counter (19); the first polarizing plate (14) is fixedly mounted on the left side of the camera (9); the second polarizing plate (15) is rotatably mounted on the left side of the camera (9); the gear ring (16) is sleeved on the second polarizing plate (15); the mute motor (17) is mounted on the machine body (1); the gear (18) is mounted on the output shaft of the mute motor (17); the gear (18) and the mute motor (17) are meshingly connected; the counter (19) is mounted below the gear (18); a counting key is arranged on the top of the counter (19); a reset key is arranged on the bottom of the counter (19); and a display screen is arranged on the left end of the counter (19).

Images