Behavior recognition method and device, electronic device, and storage medium

Abstract

The application provides a behavior recognition method and device, electronic equipment and a storage medium, which include: cleaning an initial frequency domain channel response of a Wi-Fi signal to obtain a cleaned frequency domain channel response; determining a noise sample point vector based on the initial frequency domain channel response and the cleaned frequency domain channel response, and determining a noise covariance matrix corresponding to the noise sample point vector; performing LDL decomposition on the noise covariance matrix to obtain a unit lower triangular matrix of the noise covariance matrix; performing deinterference on the cleaned frequency domain channel response based on the unit lower triangular matrix to obtain a target matrix, and determining a target covariance matrix corresponding to the target matrix; performing eigenvalue decomposition on the target covariance matrix to obtain an observed Doppler velocity of a passive target; and determining a behavior recognition result of the passive target according to observed Doppler velocities at multiple observation times. The scheme can clean the frequency domain channel response, and realize behavior recognition based on the cleaned frequency domain channel response.

Description

Technical Field

[0001] This application relates to the field of behavior recognition technology, and in particular to a behavior recognition method and apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] A key development direction for next-generation wireless communication technology is the integration of communication sensing and radar. Future Wi-Fi signals will not only function as data transmission devices but will also possess the ability to sense passive targets (people or other targets without terminal devices). When passive targets move indoors, they affect the multipath channels of indoor Wi-Fi signals, causing changes in Channel State Information (CSI). Therefore, CSI can be used for behavior recognition of passive targets. CSI includes the Channel Frequency Response (CFR) of multipath transmission of Wi-Fi signals, where multiple channels include LOS (Line of Sight) channels and NLOS (Not Line of Sight) channels. Since indoor objects remain stationary and the CSI of the LOS channel is unaffected by passive targets, the channel state estimate of the NLOS channel within the CSI can be used to characterize the impact of passive target activity on the wireless channel, thereby enabling behavior recognition. Summary of the Invention

[0003] The purpose of this application is to provide a behavior recognition method, device, electronic device, and computer-readable storage medium for effectively cleaning the frequency domain channel response, thereby realizing behavior recognition based on the cleaned frequency domain channel response.

[0004] On the one hand, this application provides a behavior recognition method, including:

[0005] The initial frequency domain channel response of the Wi-Fi signal is cleaned to obtain the cleaned frequency domain channel response;

[0006] Based on the initial frequency domain channel response and the cleaned frequency domain channel response, the noise sampling point vector is determined, and the noise covariance matrix corresponding to the noise sampling point vector is determined.

[0007] The noise covariance matrix is ​​decomposed by LDL to obtain the unit lower triangular matrix of the noise covariance matrix;

[0008] Based on the unit lower triangular matrix, the frequency domain channel response after cleaning is de-interferenced to obtain the target matrix, and the target covariance matrix corresponding to the target matrix is ​​determined.

[0009] The target covariance matrix is ​​decomposed into eigenvalues ​​to obtain the observed Doppler velocity of the passive target.

[0010] The behavior recognition result of the passive target is determined based on the observed Doppler velocity at multiple observation times.

[0011] In one embodiment, the step of cleaning the initial frequency domain channel response of the Wi-Fi signal to obtain a cleaned frequency domain channel response includes:

[0012] The initial frequency domain channel response is conjugate-flipped to obtain the flipped frequency domain channel response.

[0013] The inverse Fourier transform is performed on the flipped frequency domain channel response to obtain the target channel impulse response; wherein the target channel impulse response includes multiple sampling points, and the time domain channel estimate value of each sampling point is a real number;

[0014] The target channel impulse response is sampled to obtain the specified channel impulse response;

[0015] A sliding window is used in the specified channel impulse response to search for multiple in-window sampling points corresponding to the maximum energy value; wherein, the energy value is the sum of the time-domain channel estimates of the in-window sampling points;

[0016] Set the time-domain channel estimate of the remaining sampling points other than the multiple in-window sampling points to zero, set the time-domain channel estimate of the sampling points below the noise threshold to zero, and set the maximum time-domain channel estimate of the multiple in-window sampling points to zero to obtain the cleaned channel impulse response.

[0017] The cleaned channel impulse response is subjected to a Fourier transform to obtain the cleaned frequency domain channel response.

[0018] In one embodiment, before setting the time-domain channel estimate of the remaining sampling points other than the plurality of in-window sampling points to zero, setting the time-domain channel estimate of the in-window sampling points below the noise threshold to zero, and setting the maximum time-domain channel estimate of the plurality of in-window sampling points to zero, the method further includes:

[0019] After finding multiple in-window sampling points corresponding to the maximum energy value, determine the maximum time-domain channel estimate of the remaining sampling points in the specified channel impulse response;

[0020] The maximum time-domain channel estimate is determined as the noise threshold.

[0021] In one embodiment, determining the behavior recognition result of the passive target based on the observed Doppler velocities at multiple observation times includes:

[0022] Based on the observed Doppler velocities at multiple observation times, an observed Doppler velocity-time spectrum is constructed;

[0023] A sub-spectral image generated within a specified time period is cropped from the observed Doppler velocity-time spectrum and used as the image to be identified;

[0024] The image to be identified is used as a trained behavior recognition model to obtain the behavior recognition result output by the behavior recognition model.

[0025] In one embodiment, before constructing the observation Doppler velocity-time spectrum based on the observation Doppler velocities at multiple observation times, the method further includes:

[0026] Multiple sample frequency domain channel responses of Wi-Fi signals are acquired, and each sample frequency domain channel response is converted into a sample Doppler velocity-time spectrum; wherein, the multiple sample frequency domain channel responses correspond to multiple preset behavior categories;

[0027] Multiple sample images are constructed based on the Doppler velocity-time spectrum of each sample; wherein each sample image is labeled with the behavior category label corresponding to the Doppler velocity-time spectrum of the sample.

[0028] The machine learning model is trained based on the multiple sample images to obtain the behavior recognition model.

[0029] In one embodiment, the cleaning module is further configured to:

[0030] The initial frequency domain channel response is conjugate-flipped to obtain the flipped frequency domain channel response.

[0031] The inverse Fourier transform is performed on the flipped frequency domain channel response to obtain the target channel impulse response; wherein the target channel impulse response includes multiple sampling points, and the time domain channel estimate value of each sampling point is a real number;

[0032] The target channel impulse response is sampled to obtain the specified channel impulse response;

[0033] A sliding window is used in the specified channel impulse response to search for multiple in-window sampling points corresponding to the maximum energy value; wherein, the energy value is the sum of the time-domain channel estimates of the in-window sampling points;

[0034] Set the time-domain channel estimate of the remaining sampling points other than the multiple in-window sampling points to zero, and set the time-domain channel estimate of the in-window sampling points below the noise threshold to zero to obtain the cleaned channel impulse response;

[0035] The cleaned channel impulse response is subjected to a Fourier transform to obtain the cleaned frequency domain channel response.

[0036] In one embodiment, the identification module is further configured to:

[0037] Based on the observed Doppler velocities at multiple observation times, an observed Doppler velocity-time spectrum is constructed;

[0038] A sub-spectral image generated within a specified time period is cropped from the observed Doppler velocity-time spectrum and used as the image to be identified;

[0039] The image to be identified is used as a trained behavior recognition model to obtain the behavior recognition result output by the behavior recognition model.

[0040] On the other hand, this application provides a behavior recognition device, including:

[0041] The cleaning module is used to clean the initial frequency domain channel response of the Wi-Fi signal to obtain the cleaned frequency domain channel response.

[0042] The determination module is used to determine the noise sampling point vector based on the initial frequency domain channel response and the cleaned frequency domain channel response, and to determine the noise covariance matrix corresponding to the noise sampling point vector;

[0043] The first decomposition module is used to perform LDL decomposition on the noise covariance matrix to obtain the unit lower triangular matrix of the noise covariance matrix.

[0044] The interference removal module is used to remove interference from the cleaned frequency domain channel response based on the unit lower triangular matrix, obtain the target matrix, and determine the target covariance matrix corresponding to the target matrix.

[0045] The second decomposition module is used to perform eigenvalue decomposition on the target covariance matrix to obtain the observed Doppler velocity of the passive target;

[0046] The identification module is used to determine the behavior identification result of the passive target based on the observed Doppler velocity at multiple observation times.

[0047] Furthermore, this application provides an electronic device, the electronic device comprising:

[0048] processor;

[0049] Memory used to store processor-executable instructions;

[0050] The processor is configured to execute the behavior recognition method described above.

[0051] In addition, this application also provides a computer-readable storage medium storing a computer program that can be executed by a processor to perform the above-described behavior recognition method.

[0052] The proposed solution, after cleaning the frequency domain channel response, can further eliminate the influence of other Wi-Fi signal transmitters in the application environment, thereby obtaining a target matrix that accurately characterizes the behavior of passive targets, and then accurately realizing behavior recognition based on the resolved multiple observed Doppler velocities. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly described below.

[0054] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0055] Figure 2 A flowchart illustrating a behavior recognition method provided in an embodiment of this application;

[0056] Figure 3 Provided for an embodiment of this application Figure 2 A detailed flowchart of step 210 is shown below;

[0057] Figure 4 Provided for an embodiment of this application Figure 2 A detailed flowchart of step 260;

[0058] Figure 5 A flowchart illustrating a training method for a behavior recognition model provided in an embodiment of this application;

[0059] Figure 6 A block diagram of a behavior recognition device provided in an embodiment of this application. Detailed Implementation

[0060] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0061] Similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0062] like Figure 1 As shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12. Figure 1Taking a processor 11 as an example, the processor 11 and memory 12 are connected via a bus 10. The memory 12 stores instructions that can be executed by the processor 11. These instructions are executed by the processor 11 to enable the electronic device 1 to perform all or part of the processes of the methods described in the following embodiments. In one embodiment, the electronic device 1 may be a router, a communication terminal device (e.g., a mobile phone, a tablet computer), etc., used to execute the behavior recognition method. The electronic device may be equipped with a transmitting antenna and a receiving antenna for transmitting and receiving Wi-Fi signals. The following description uses the electronic device as the execution subject.

[0063] The memory 12 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0064] This application also provides a computer-readable storage medium storing a computer program that can be executed by a processor 11 to perform the behavior recognition method provided in this application.

[0065] See Figure 2 The above is a flowchart illustrating a behavior recognition method provided in an embodiment of this application. Figure 2 As shown, the method may include steps 210-260.

[0066] Step 210: Clean the initial frequency domain channel response of the Wi-Fi signal to obtain the cleaned frequency domain channel response.

[0067] The electronic device using this application is pre-placed in a designated location within the indoor environment, and its position remains unchanged during application. Here, the designated location can be determined based on the actual environmental conditions. For example, the electronic device could be a router, placed in the center of the living room TV wall; or, the electronic device could be a network camera, placed in a corner of the room, etc.

[0068] Electronic devices can perform channel estimation on Wi-Fi signals to obtain an initial frequency domain channel response. Here, the initial frequency domain channel response is the unwashed frequency domain channel response. The initial frequency domain channel response includes frequency domain channel estimates at M sampling points, where M is a positive integer. The M sampling points include sampling points corresponding to the effective channel and sampling points corresponding to the noisy channel. The effective channel includes NLOS channels and LOS channels.

[0069] After cleaning the initial frequency domain channel response, the data from the noisy channel can be removed, resulting in the cleaned frequency domain channel response. The cleaned frequency domain channel response includes frequency domain channel estimates for M sampling points, and the frequency domain channel estimates for the sampling points corresponding to the noisy channel are zero.

[0070] Step 220: Based on the initial frequency domain channel response and the cleaned frequency domain channel response, determine the noise sampling point vector and the noise covariance matrix corresponding to the noise sampling point vector.

[0071] For example, an electronic device can determine the noise sampling point vector using the following formula (1):

[0072]

[0073] Where H(k) is the initial frequency domain channel response; Let E be the frequency domain channel response after cleaning; E is the identity matrix; n(k) is the noise sampling point vector; and k ranges from 1 to M.

[0074] In the noise sampling point vector, the frequency domain channel estimate corresponding to the noise sampling point is not zero, while the frequency domain channel estimate of other sampling points is zero.

[0075] After obtaining the noise sampling point vector, the electronic device can calculate the corresponding covariance matrix as the noise covariance matrix. For example, the electronic device can determine the noise covariance matrix using the following formula (2):

[0076] R nn =E[n T (k)n * (k)] (2)

[0077] Among them, R nn Let n(k) represent the noise covariance matrix; n(k) is the noise sampling point vector.

[0078] Step 230: Perform LDL decomposition on the noise covariance matrix to obtain the unit lower triangular matrix of the noise covariance matrix.

[0079] LDL decomposition is a form of Cholesky decomposition, used to decompose a matrix into a multiplication of a lower triangular matrix, a diagonal matrix, and a lower triangular matrix.

[0080] For example, an electronic device can be decomposed into LDL using the following formula (3):

[0081] R nn =LL * (3)

[0082] Among them, R nn Let L represent the noise covariance matrix; L is the decomposed unit lower triangular matrix; the unit diagonal matrix is ​​omitted in formula (3).

[0083] The unit lower triangular matrix is ​​a whitening matrix, which can characterize the impact of other transmitters on the multipath channel of Wi-Fi signals.

[0084] Step 240: Based on the unit lower triangular matrix, perform interference removal on the cleaned frequency domain channel response to obtain the target matrix, and determine the target covariance matrix corresponding to the target matrix.

[0085] After obtaining the unit lower triangular matrix, the electronic device can perform de-interference on the cleaned frequency domain channel response. For example, de-interference can be performed using the following formula (4):

[0086]

[0087] Where F is the target matrix obtained by decontaminating the interference; L is the decomposed unit lower triangular matrix; This is the frequency domain channel response after cleaning.

[0088] After obtaining the target matrix, the covariance matrix corresponding to the target matrix can be calculated using the following formula (5), which serves as the target covariance matrix:

[0089] R FF =E[F T F * (5)

[0090] Among them, R FF Let F represent the target covariance matrix; F is the target matrix.

[0091] Step 250: Perform eigenvalue decomposition on the target covariance matrix to obtain the observed Doppler velocity of the passive target.

[0092] Electronic devices can use algorithms such as MUSIC (multiple signal classification algorithm), compressed sensing algorithm, or other subspace algorithms such as ESPRIT (Estimation of Signal Parameters using Rotational Invariance Techniques) to perform eigenvalue decomposition on the target covariance matrix, thereby obtaining the observed Doppler velocity of the passive target relative to the electronic device.

[0093] Step 260: Determine the behavior recognition result of the passive target based on the observed Doppler velocity at multiple observation times.

[0094] Electronic devices can process the initial frequency domain channel response of Wi-Fi signals at multiple observation times to obtain the observed Doppler velocity at each observation time. The interval between adjacent observation times can be configured as needed; for example, each data packet corresponds to one observation time. The observed Doppler velocity at multiple consecutive observation times can characterize the impact of passive target behavior on the multipath channel within this time period. Based on the determined impact results, behavior classification can be performed to obtain the passive target behavior recognition result.

[0095] Through the above measures, after obtaining the frequency domain channel response through cleaning, the influence of other Wi-Fi signal transmitters in the application environment can be further eliminated, thereby obtaining a target matrix that accurately represents the behavior of passive targets. Then, based on the resolved multiple observed Doppler velocities, behavior recognition can be accurately achieved.

[0096] In one embodiment, when cleaning the initial frequency domain channel response, see... Figure 3 This is provided as an embodiment of the present application. Figure 2 A detailed flowchart of step 210 is shown below. Figure 3 As shown, when performing step 210, steps 211 to 216 can be performed.

[0097] Step 211: Perform conjugate flipping on the initial frequency domain channel response to obtain the flipped frequency domain channel response.

[0098] The initial frequency domain channel response includes M sampling points. After conjugate flipping, a flipped frequency domain channel response with 2M-1 sampling points is obtained. The flipped frequency domain channel response is symmetrical along the vertical line at the center. For example, the initial frequency domain channel response includes 64 sampling points, numbered from 0 to 63. During conjugate flipping, sampling points numbered 1 to 63 are copied before sampling point number 0. Sampling point number 63 is used as sampling point number -63, sampling point number 62 is used as sampling point number -62, sampling point number 61 is used as sampling point number -61, and so on.

[0099] Step 212: Perform an inverse Fourier transform on the flipped frequency domain channel response to obtain the target channel impulse response; wherein, the target channel impulse response includes multiple sampling points, and the time domain channel estimate value of each sampling point is a real number.

[0100] Electronic devices can perform an inverse fast fourier transform (IFFT) on the flipped frequency domain channel response to obtain the target channel impulse response (CIR) in the time domain. Since the flipped frequency domain channel response has undergone conjugate flipping and exhibits conjugate symmetry, the time domain channel estimate corresponding to each sampling point in the target channel impulse response is a real number.

[0101] Step 213: Sample the target channel impulse response to obtain the specified channel impulse response.

[0102] The electronic device can start from the first sampling point of the target channel impulse response and select a sampling point every other sampling point to obtain the specified channel impulse response. In this case, if the target channel impulse response includes 2M-1 sampling points, after sampling, the specified channel impulse response includes M sampling points.

[0103] Step 214: Perform a sliding window operation in the specified channel impulse response to search for multiple in-window sampling points corresponding to the maximum energy value; where the energy value is the sum of the time-domain channel estimates of the in-window sampling points.

[0104] The window length used for the sliding window can be configured empirically. For example, the window length can be the length of the cyclic prefix of the Wi-Fi signal. The step size of the sliding window can be 1.

[0105] Since the time-domain channel estimate values ​​of each sampling point in the specified channel impulse response are real numbers, the electronic device can calculate the sum of the time-domain channel estimates of the sampling points within the window during the sliding window process, and use it as the energy value within the window.

[0106] During the sliding window process, multiple energy values ​​can be calculated separately, and the maximum energy value is determined from these values. At this point, the multiple sampling points within the window corresponding to the maximum energy value include the sampling points corresponding to all valid channels.

[0107] Step 215: Set the time-domain channel estimate of the remaining sampling points other than the multiple in-window sampling points to zero, set the time-domain channel estimate of the sampling points below the noise threshold to zero, and set the maximum time-domain channel estimate of the multiple in-window sampling points to zero to obtain the cleaned channel impulse response.

[0108] After determining the multiple in-window sampling points corresponding to the maximum energy value, the remaining sampling points besides those in the window can be considered noise sampling points. At this point, the time-domain channel estimate of the remaining sampling points can be set to zero. Furthermore, noise sampling points that may exist among the multiple in-window sampling points can be distinguished using a noise threshold. The electronic device can compare each in-window sampling point to determine whether the time-domain channel estimate of the in-window sampling point is lower than the noise threshold. If so, the in-window sampling point can be considered a noise sampling point, and its time-domain channel estimate can be set to zero. If not, the time-domain channel estimate of the in-window sampling point can be retained.

[0109] For multiple sampling points within a window, the electronic device can determine the maximum time-domain channel estimate among the multiple sampling points within the window. This maximum time-domain channel estimate can be regarded as the time-domain channel estimate corresponding to the LOS channel. Therefore, the maximum time-domain channel estimate can be set to zero.

[0110] By performing step 215, the channel impulse response after cleaning can be obtained.

[0111] In one embodiment, before performing step 215, after finding multiple in-window sampling points corresponding to the maximum energy value, the electronic device can examine the time-domain channel estimates of the remaining sampling points in the specified channel impulse response, excluding these multiple in-window sampling points, and determine the maximum time-domain channel estimate. The electronic device can use this maximum time-domain channel estimate as a noise threshold.

[0112] Step 216: Perform a Fourier transform on the cleaned channel impulse response to obtain the cleaned frequency domain channel response.

[0113] Electronic devices can perform a fast Fourier transform (FFT) on the cleaned channel impulse response to obtain the cleaned frequency domain channel response.

[0114] By taking the above measures, since the time-domain channel estimates of each sampling point of the processed time-domain channel impulse response are real numbers, the data of the noisy channel and LOS channel can be removed without calculating the power-delay profile (PDP), and the cleaned frequency-domain channel response can be obtained, thus reducing the computational load of the cleaning work.

[0115] In one embodiment, behavior recognition can be achieved using a machine learning model, see [link to relevant documentation]. Figure 4 This is provided as an embodiment of the present application. Figure 2 A detailed flowchart of step 260 is shown below. Figure 4 As shown, when executing step 260, steps 261 to 263 can be executed.

[0116] Step 261: Construct the observation Doppler velocity-time spectrum based on the observed Doppler velocities at multiple observation times.

[0117] Electronic devices can construct an observational Doppler velocity-time spectrum based on the observed Doppler velocities at multiple consecutive observation times. The horizontal axis of the observational Doppler velocity-time spectrum represents time, and the vertical axis represents the observed Doppler velocity.

[0118] Step 262: Crop the sub-spectral image generated within a specified time period from the observed Doppler velocity-time spectrum as the image to be identified.

[0119] The specified time period can be configured as needed. For example, in a scenario where passive target behavior is detected, if the behavior to be identified can be completed within 3 seconds, the specified time period can be the most recent 3 seconds.

[0120] Electronic devices can crop sub-spectral images generated within a specified time period based on the horizontal axis coordinate of the Doppler velocity-time spectrum and use them as images to be identified.

[0121] Step 263: Use the image to be recognized as a trained behavior recognition model to obtain the behavior recognition result output by the behavior recognition model.

[0122] Among them, the behavior recognition model is a network model used to classify behavior categories. The network model can be any of the following: CNN (Convolutional Neural Networks), TCN (Temporal Convolutional Network), CNN plus RNN (Recurrent Neural Networks).

[0123] The electronic device inputs the image to be recognized into the behavior recognition model, and the behavior recognition model processes the image to obtain the behavior recognition result of the passive target.

[0124] The above measures can be used to quickly achieve behavior recognition with the help of machine learning.

[0125] In one embodiment, see Figure 5 This is a flowchart illustrating a training method for a behavior recognition model provided in an embodiment of this application. Figure 5 As shown, the behavior recognition model can be trained through the following steps 510 to 530.

[0126] Step 510: Obtain multiple sample frequency domain channel responses of the Wi-Fi signal, and convert each sample frequency domain channel response into a sample Doppler velocity-time spectrum; wherein, multiple sample frequency domain channel responses correspond to multiple preset behavior categories.

[0127] When a passive target continuously performs the same type of behavior, the corresponding frequency domain channel response can be collected. For example, in an application scenario requiring user gesture detection, where the behavior categories include waving, spreading hands, and clapping, people of different body types can be asked to continuously wave for half an hour in an indoor setting, and the frequency domain channel response for that half hour can be collected. This half hour's frequency domain channel response corresponds to the waving category. Similarly, people of different body types can be asked to continuously spread their hands for half an hour in an indoor setting, and the frequency domain channel response for that half hour can be collected. This half hour's frequency domain channel response corresponds to the spreading hand category. Likewise, people of different body types can be asked to continuously clap for half an hour in an indoor setting, and the frequency domain channel response for that half hour can be collected. This half hour's frequency domain channel response corresponds to the clapping category.

[0128] After obtaining the sample frequency domain responses corresponding to various behavior categories, the electronic device can determine the observed Doppler velocity from them, thereby constructing the sample Doppler velocity-time spectrum. In this case, the sample Doppler velocity-time spectrum corresponding to each behavior category can be obtained. The method for constructing the sample Doppler velocity-time spectrum can be referred to the relevant description above, and will not be repeated here.

[0129] Step 520: Construct multiple sample images based on the Doppler velocity-time spectrum of each sample; wherein, each sample image is labeled with the behavior category label corresponding to the sample Doppler velocity-time spectrum.

[0130] For the sample Doppler velocity-time spectrum corresponding to each behavior category, it can be cropped based on the length of a specified time period during application to obtain multiple sub-spectral images. Adding a behavior category label to each sub-spectral image yields the sample image. This method allows for the processing and generation of multiple sample images corresponding to each behavior category.

[0131] Step 530: Train the machine learning model based on multiple sample images to obtain the behavior recognition model.

[0132] After obtaining a large number of sample images, supervised learning can be performed on the machine learning model using these sample images. Through multiple rounds of iterative training, when the machine learning model converges, a trained behavior recognition model can be obtained.

[0133] The above measures can be used to train a behavior recognition model for subsequent behavior recognition.

[0134] Figure 6 This is a block diagram of a behavior recognition device according to an embodiment of the present invention, such as... Figure 6 As shown, the device may include:

[0135] The cleaning module 610 is used to clean the initial frequency domain channel response of the Wi-Fi signal to obtain the cleaned frequency domain channel response;

[0136] The determination module 620 is used to determine the noise sampling point vector based on the initial frequency domain channel response and the cleaned frequency domain channel response, and to determine the noise covariance matrix corresponding to the noise sampling point vector;

[0137] The first decomposition module 630 is used to perform LDL decomposition on the noise covariance matrix to obtain the unit lower triangular matrix of the noise covariance matrix.

[0138] The interference removal module 640 is used to remove interference from the cleaned frequency domain channel response based on the unit lower triangular matrix to obtain the target matrix and determine the target covariance matrix corresponding to the target matrix.

[0139] The second decomposition module 650 is used to perform eigenvalue decomposition on the target covariance matrix to obtain the observed Doppler velocity of the passive target.

[0140] The identification module 660 is used to determine the behavior identification result of the passive target based on the observed Doppler velocity at multiple observation times.

[0141] In one embodiment, the cleaning module 610 is further configured to:

[0142] The initial frequency domain channel response is conjugate-flipped to obtain the flipped frequency domain channel response.

[0143] The inverse Fourier transform is performed on the flipped frequency domain channel response to obtain the target channel impulse response; wherein the target channel impulse response includes multiple sampling points, and the time domain channel estimate value of each sampling point is a real number;

[0144] The target channel impulse response is sampled to obtain the specified channel impulse response;

[0145] A sliding window is used in the specified channel impulse response to search for multiple in-window sampling points corresponding to the maximum energy value; wherein, the energy value is the sum of the time-domain channel estimates of the in-window sampling points;

[0146] Set the time-domain channel estimate of the remaining sampling points other than the multiple in-window sampling points to zero, set the time-domain channel estimate of the sampling points below the noise threshold to zero, and set the maximum time-domain channel estimate of the multiple in-window sampling points to zero to obtain the cleaned channel impulse response.

[0147] The cleaned channel impulse response is subjected to a Fourier transform to obtain the cleaned frequency domain channel response.

[0148] In one embodiment, the cleaning module 610 is further configured to:

[0149] After finding multiple in-window sampling points corresponding to the maximum energy value, determine the maximum time-domain channel estimate of the remaining sampling points in the specified channel impulse response;

[0150] The maximum time-domain channel estimate is determined as the noise threshold.

[0151] In one embodiment, the identification module 660 is further configured to:

[0152] Based on the observed Doppler velocities at multiple observation times, an observed Doppler velocity-time spectrum is constructed;

[0153] A sub-spectral image generated within a specified time period is cropped from the observed Doppler velocity-time spectrum and used as the image to be identified;

[0154] The image to be identified is used as a trained behavior recognition model to obtain the behavior recognition result output by the behavior recognition model.

[0155] In one embodiment, the device further includes a training module 670, used for:

[0156] Multiple sample frequency domain channel responses of Wi-Fi signals are acquired, and each sample frequency domain channel response is converted into a sample Doppler velocity-time spectrum; wherein, the multiple sample frequency domain channel responses correspond to multiple preset behavior categories;

[0157] Multiple sample images are constructed based on the Doppler velocity-time spectrum of each sample; wherein each sample image is labeled with the behavior category label corresponding to the Doppler velocity-time spectrum of the sample.

[0158] The machine learning model is trained based on the multiple sample images to obtain the behavior recognition model.

[0159] The specific implementation process of the functions and roles of each module in the above-mentioned device can be found in the implementation process of the corresponding steps in the above-mentioned behavior recognition method, and will not be repeated here.

[0160] The apparatuses and methods disclosed in the several embodiments provided in this application can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0161] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0162] If a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

Claims

1. A behavior recognition method, characterized in that, include: The initial frequency domain channel response of the Wi-Fi signal is cleaned to obtain the cleaned frequency domain channel response; Based on the initial frequency domain channel response and the cleaned frequency domain channel response, the noise sampling point vector is determined, and the noise covariance matrix corresponding to the noise sampling point vector is determined. The noise covariance matrix is ​​decomposed by LDL to obtain the unit lower triangular matrix of the noise covariance matrix; Based on the unit lower triangular matrix, the frequency domain channel response after cleaning is de-interferenced to obtain the target matrix, and the target covariance matrix corresponding to the target matrix is ​​determined. The target covariance matrix is ​​decomposed into eigenvalues ​​to obtain the observed Doppler velocity of the passive target. The behavior recognition result of the passive target is determined based on the observed Doppler velocity at multiple observation times; The step of cleaning the initial frequency domain channel response of the Wi-Fi signal to obtain the cleaned frequency domain channel response includes: The initial frequency domain channel response is conjugate-flipped to obtain the flipped frequency domain channel response. The inverse Fourier transform is performed on the flipped frequency domain channel response to obtain the target channel impulse response; wherein the target channel impulse response includes multiple sampling points, and the time domain channel estimate value of each sampling point is a real number; The target channel impulse response is sampled to obtain the specified channel impulse response; A sliding window is used in the specified channel impulse response to search for multiple in-window sampling points corresponding to the maximum energy value; wherein, the energy value is the sum of the time-domain channel estimates of the in-window sampling points; Set the time-domain channel estimate of the remaining sampling points other than the multiple in-window sampling points to zero, set the time-domain channel estimate of the sampling points below the noise threshold to zero, and set the maximum time-domain channel estimate of the multiple in-window sampling points to zero to obtain the cleaned channel impulse response. The cleaned channel impulse response is subjected to a Fourier transform to obtain the cleaned frequency domain channel response.

2. The method according to claim 1, characterized in that, Before setting the time-domain channel estimate of the remaining sampling points other than the plurality of in-window sampling points to zero, setting the time-domain channel estimate of the in-window sampling points below the noise threshold to zero, and setting the maximum time-domain channel estimate of the plurality of in-window sampling points to zero, the method further includes: After finding multiple in-window sampling points corresponding to the maximum energy value, determine the maximum time-domain channel estimate of the remaining sampling points in the specified channel impulse response; The maximum time-domain channel estimate is determined as the noise threshold.

3. The method according to claim 1, characterized in that, The determination of the passive target's behavior recognition result based on the observed Doppler velocities at multiple observation times includes: Based on the observed Doppler velocities at multiple observation times, an observed Doppler velocity-time spectrum is constructed; A sub-spectral image generated within a specified time period is cropped from the observed Doppler velocity-time spectrum and used as the image to be identified; The image to be identified is used as a trained behavior recognition model to obtain the behavior recognition result output by the behavior recognition model.

4. The method according to claim 3, characterized in that, Before constructing the observation Doppler velocity-time spectrum based on the observation Doppler velocities at multiple observation times, the method further includes: Multiple sample frequency domain channel responses of Wi-Fi signals are acquired, and each sample frequency domain channel response is converted into a sample Doppler velocity-time spectrum; wherein, the multiple sample frequency domain channel responses correspond to multiple preset behavior categories; Multiple sample images are constructed based on the Doppler velocity-time spectrum of each sample; wherein each sample image is labeled with the behavior category label corresponding to the Doppler velocity-time spectrum of the sample. The machine learning model is trained based on the multiple sample images to obtain the behavior recognition model.

5. A behavior recognition device, characterized in that, include: The cleaning module is used to clean the initial frequency domain channel response of the Wi-Fi signal to obtain the cleaned frequency domain channel response. The determination module is used to determine the noise sampling point vector based on the initial frequency domain channel response and the cleaned frequency domain channel response, and to determine the noise covariance matrix corresponding to the noise sampling point vector; The first decomposition module is used to perform LDL decomposition on the noise covariance matrix to obtain the unit lower triangular matrix of the noise covariance matrix. The interference removal module is used to remove interference from the cleaned frequency domain channel response based on the unit lower triangular matrix, obtain the target matrix, and determine the target covariance matrix corresponding to the target matrix. The second decomposition module is used to perform eigenvalue decomposition on the target covariance matrix to obtain the observed Doppler velocity of the passive target; The identification module is used to determine the behavior identification result of the passive target based on the observed Doppler velocity at multiple observation times; The cleaning module is further configured to: The initial frequency domain channel response is conjugate-flipped to obtain the flipped frequency domain channel response. The inverse Fourier transform is performed on the flipped frequency domain channel response to obtain the target channel impulse response; wherein the target channel impulse response includes multiple sampling points, and the time domain channel estimate value of each sampling point is a real number; The target channel impulse response is sampled to obtain the specified channel impulse response; A sliding window is used in the specified channel impulse response to search for multiple in-window sampling points corresponding to the maximum energy value; wherein, the energy value is the sum of the time-domain channel estimates of the in-window sampling points; Set the time-domain channel estimate of the remaining sampling points other than the multiple in-window sampling points to zero, and set the time-domain channel estimate of the in-window sampling points below the noise threshold to zero to obtain the cleaned channel impulse response; The cleaned channel impulse response is subjected to a Fourier transform to obtain the cleaned frequency domain channel response.

6. The apparatus according to claim 5, characterized in that, The identification module is also used for: Based on the observed Doppler velocities at multiple observation times, an observed Doppler velocity-time spectrum is constructed; A sub-spectral image generated within a specified time period is cropped from the observed Doppler velocity-time spectrum and used as the image to be identified; The image to be identified is used as a trained behavior recognition model to obtain the behavior recognition result output by the behavior recognition model.

7. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; The processor is configured to execute the behavior recognition method according to any one of claims 1-4.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that can be executed by a processor to perform the behavior recognition method according to any one of claims 1-4.

Images