Method and apparatus for behavior recognition based on channel state information, electronic device, and storage medium

Abstract

The application provides a behavior recognition method and device based on channel state information, electronic equipment and storage medium, the method comprises: determining the real Doppler velocity of a passive target for the channel state information of each antenna element; generating a time-Doppler velocity spectrum according to the real Doppler velocities of multiple observation times; constructing an input parameter of a trained behavior recognition model according to the time-Doppler velocity spectrum; and processing the input parameter according to the behavior recognition model to obtain a behavior recognition result. The application scheme realizes accurate recognition of the behavior category of a passive target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of behavior recognition, and in particular to a behavior recognition method and device based on channel state information, an electronic device and a computer readable storage medium. BACKGROUND

[0002] An important development direction of next-generation wireless communication technology is communication sensing / radar integration. Future Wi-Fi signals will not only have data transmission functions, but also have sensing functions for passive targets. When a passive target (a person or other target without carrying a terminal device) is located in an indoor environment, its existence will affect indoor Wi-Fi signals, causing changes in the multipath channel. In this case, the channel state information (CSI) changes accordingly, and therefore the passive target can be recognized by means of channel state information. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a behavior recognition method and device based on channel state information, an electronic device and a storage medium, for accurately recognizing the behavior of a passive target by means of channel state information.

[0004] In one aspect, the present application provides a behavior recognition method based on channel state information, applied to a Wi-Fi transceiver system, the Wi-Fi transceiver system including a transmitter, a first receiver and a second receiver, the first receiver and the second receiver each including at least one antenna element, comprising:

[0005] For the channel state information of each antenna element, determining the true Doppler velocity of a passive target;

[0006] According to the true Doppler velocities at multiple observation times, generating a time-Doppler velocity spectrum;

[0007] According to the time-Doppler velocity spectrum, constructing an input parameter of a trained behavior recognition model;

[0008] According to the behavior recognition model, processing the input parameter to obtain a behavior recognition result.

[0009] In one embodiment, before the channel state information of each antenna element is determined to determine the true Doppler velocity of a passive target, the method further comprises:

[0010] Obtaining channel estimation information corresponding to each antenna element, and converting the channel estimation information into time-domain channel estimation information through inverse Fourier transform;

[0011] According to the time-domain channel estimation information, determining the power corresponding to multiple channels;

[0012] a plurality of windows with maximum power are divided according to the preset window length, and an average value of the power corresponding to the channels outside the windows is calculated as noise power;

[0013] a threshold power is determined by the noise power and a preset first adjustment coefficient;

[0014] a first window-in-channel and a second window-in-channel are selected from the plurality of window-in-channels, time-domain channel estimation values corresponding to the second window-in-channels are set to zero, and target time-domain channel estimation information is constructed based on the time-domain channel estimation values corresponding to the first window-in-channels and the time-domain channel estimation values corresponding to the second window-in-channels; wherein the second window-in-channels include window-in-channels with power not greater than the threshold power and a window-in-channel with maximum power; and the first window-in-channel is a window-in-channel other than the second window-in-channels;

[0015] the target time-domain channel estimation information of each antenna array element is respectively subjected to Fourier transform to obtain channel state information of each antenna array element in the frequency domain.

[0016] In an embodiment, the channel state information of each antenna array element is used to determine a real Doppler velocity of the passive target, including:

[0017] each receiver is respectively taken as a target receiver, and an antenna array element under the target receiver is taken as a target antenna array element;

[0018] from the channel state information of the target antenna array element, a smallest non-zero frequency domain channel estimation value is selected as a second adjustment coefficient;

[0019] the second adjustment coefficient is processed according to a preset amplification coefficient to obtain a third adjustment coefficient;

[0020] the channel state information is adjusted by the second adjustment coefficient to obtain first adjusted channel state information;

[0021] the channel state information of the antenna array element under the specified receiver is adjusted by the third adjustment coefficient to obtain second adjusted channel state information; wherein the specified receiver is a receiver other than the target receiver;

[0022] the first adjusted channel state information and the second adjusted channel state information are conjugate multiplied to obtain specified channel state information of the target antenna array element;

[0023] the real Doppler velocity of the passive target is determined according to the specified channel state information corresponding to each antenna array element.

[0024] In an embodiment, the determining the real Doppler velocity of the passive target according to the designated channel state information corresponding to each antenna element comprises:

[0025] constructing a corresponding covariance matrix for the designated channel state information corresponding to the first receiver and the second receiver, respectively;

[0026] performing eigenvalue decomposition on the covariance matrix corresponding to the first receiver to obtain a first observed Doppler velocity of the passive target relative to the first receiver;

[0027] performing eigenvalue decomposition on the covariance matrix corresponding to the second receiver to obtain a second observed Doppler velocity of the passive target relative to the second receiver;

[0028] determining the real Doppler velocity of the passive target according to the first observed Doppler velocity and the second observed Doppler velocity.

[0029] In an embodiment, the behavior recognition model is trained by a convolutional neural network;

[0030] The constructing an input parameter of the trained behavior recognition model according to the time-Doppler velocity spectrum comprises:

[0031] periodically cutting the time-Doppler velocity spectrum to obtain a sub-spectrum as an input parameter of the behavior recognition model.

[0032] In an embodiment, the behavior recognition model is trained by a hybrid model of a convolutional neural network and a recurrent neural network:

[0033] The constructing an input parameter of the trained behavior recognition model according to the time-Doppler velocity spectrum comprises:

[0034] cutting the time-Doppler velocity spectrum according to a preset cutting window length and a preset cutting step length to obtain a sequence of time-continuous sub-spectra as an input parameter of the behavior recognition model.

[0035] In an embodiment, before the processing the input parameter according to the behavior recognition model to obtain a behavior recognition result, the method further comprises:

[0036] inputting a sample sub-spectrum sequence in a sample data set into the hybrid model to obtain a predicted behavior category output by the hybrid model; wherein the sample data set comprises a plurality of sample sub-spectrum sequences, and each sample sub-spectrum sequence carries a corresponding behavior category label;

[0037] adjusting model parameters of the hybrid model according to a difference between the predicted behavior category and the behavior category label.

[0038] repeating the above process until the mixed model converges, to obtain a behavior recognition model.

[0039] In another aspect, the application provides a behavior recognition device based on channel state information, applied to a Wi-Fi transceiving system, the Wi-Fi transceiving system comprising a transmitter, a first receiver and a second receiver, the first receiver and the second receiver each comprising at least one antenna element, comprising:

[0040] a determination module configured to determine a real Doppler velocity of a passive target for channel state information of each antenna element;

[0041] a generation module configured to generate a time-Doppler velocity spectrum according to the real Doppler velocities of multiple observation times;

[0042] a construction module configured to construct an input parameter of a trained behavior recognition model according to the time-Doppler velocity spectrum;

[0043] a recognition module configured to process the input parameter according to the behavior recognition model to obtain a behavior recognition result.

[0044] Further, the application provides an electronic device, comprising:

[0045] a processor;

[0046] a memory for storing processor-executable instructions;

[0047] wherein the processor is configured to execute the above behavior recognition method based on channel state information.

[0048] In addition, the application provides a computer-readable storage medium, the storage medium storing a computer program, the computer program being executable by a processor to complete the above behavior recognition method based on channel state information.

[0049] According to the application, the real Doppler velocities of multiple observation times are determined according to channel state information, and a time-Doppler velocity spectrum representing the behavior state of a passive target is constructed with the real Doppler velocities, and then an input parameter constructed with the time-Doppler velocity spectrum is classified and processed by a behavior recognition model to obtain a behavior recognition result, thereby realizing accurate recognition of behavior categories. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments of the application.

[0051] Figure 1This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0052] Figure 2 A flowchart illustrating a behavior recognition method based on channel state information provided in an embodiment of this application;

[0053] Figure 3 A schematic diagram of a time-Doppler velocity spectrum provided for an embodiment of this application;

[0054] Figure 4 A flowchart illustrating a method for determining channel state information according to an embodiment of this application;

[0055] Figure 5 A schematic diagram illustrating a method for determining channel state information according to an embodiment of this application;

[0056] Figure 6 Provided for an embodiment of this application Figure 2 A detailed flowchart of step 210 is shown below;

[0057] Figure 7 This is a block diagram of a behavior recognition device based on channel state information provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0059] Similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0060] like Figure 1 As shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12. Figure 1 Taking a processor 11 as an example, the processor 11 and memory 12 are connected via a bus 10. The memory 12 stores instructions that can be executed by the processor 11. These instructions are executed by the processor 11 to enable the electronic device 1 to perform all or part of the processes of the methods described in the following embodiments. In one embodiment, the electronic device 1 may be a computing device in a Wi-Fi transceiver system. This computing device may be a transmitter or receiver in the Wi-Fi transceiver system, or a device that interfaces with a transmitter or receiver, for executing a behavior recognition method based on channel state information. The following description uses the electronic device as the execution subject to illustrate the scheme.

[0061] The memory 12 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0062] The application further provides a computer readable storage medium, the storage medium storing a computer program, the computer program being executable by the processor 11 to complete the behavior recognition method based on channel state information provided by the application.

[0063] In an indoor environment, the transmitter, the first receiver and the second receiver in the Wi-Fi transceiving system can be placed at specified positions in the indoor scene (for example, the indoor scene is a living room, the transmitter can be placed at the upper left corner of the living room, the first receiver can be placed at the upper right corner of the living room, and the second receiver can be placed at the lower left corner of the living room). In the process of transmitting the Wi-Fi signal by the transmitter and receiving the Wi-Fi signal by the first receiver and the second receiver, the electronic device can obtain the channel state information (CSI) through channel estimation and cleaning. Various objects (such as walls, furniture, electrical appliances, people, etc.) existing in the indoor scene can cause various reflection of the Wi-Fi signal, and therefore, there are multiple wireless channels between the transmitter and the receiver. When the passive target is in the scene, the behavior state of the passive target can affect the channel state information of the wireless channel, and therefore, the behavior recognition of the passive target in the indoor scene can be performed by means of the channel state.

[0064] Referring to Figure 2 The flowchart of the behavior recognition method based on channel state information provided by an embodiment of the application is shown in FIG. 2, which can include the following steps 210-240. Figure 2

[0065] Step 210: determining the real Doppler velocity of the passive target for the channel state information of each antenna element.

[0066] ​This application solution is applied to a Wi-Fi transceiver system, which includes a transmitter for transmitting Wi-Fi signals, a first receiver for receiving Wi-Fi signals, and a second receiver. The first receiver and the second receiver each include at least one antenna element, which is used to receive the Wi-Fi signals transmitted by the transmitter.

[0067] Electronic devices can obtain channel state information for each antenna element during the transmission of Wi-Fi signals from the transmitter to the first and second receivers through channel estimation and cleaning. Based on the channel state information of the antenna elements under the first receiver, the first observed Doppler velocity of the passive target relative to the first receiver can be determined. Based on the channel state information of the antenna elements under the second receiver, the second observed Doppler velocity of the passive target relative to the second receiver can be determined. By fusing the first and second observed Doppler velocities, the true Doppler velocity of the passive target can be obtained. Here, the true Doppler velocity characterizes the overall velocity of the passive target relative to the Wi-Fi transceiver system.

[0068] Step 220: Generate a time-Doppler velocity spectrum based on the real Doppler velocities at multiple observation times.

[0069] Electronic devices can periodically determine the true Doppler velocity of passive targets, thereby obtaining the true Doppler velocity at multiple consecutive observation times. Here, the observation time is the moment when the true Doppler velocity is determined in each period; the period duration can be set as needed, for example, the period duration can be the reception interval between two adjacent data packets.

[0070] See Figure 3 This is a schematic diagram of a time-Doppler velocity spectrum provided in an embodiment of this application, as shown below. Figure 3 As shown, the horizontal axis of the time-Doppler velocity spectrum represents the observation time, and the vertical axis represents the true Doppler velocity of the passive target. When the true Doppler velocity is positive, it indicates that the passive target is moving away from the Wi-Fi transceiver system; when the true Doppler velocity is negative, it indicates that the passive target is moving towards the Wi-Fi transceiver system. In practical applications, the true Doppler velocity may not be generated by the actual movement of the passive target, but may be generated by other more subtle behaviors (such as breathing, heartbeat, falling, etc.).

[0071] During the execution of the behavior recognition method, the electronic device can continuously determine the true Doppler velocity of the passive target, thereby continuously updating the time-Doppler velocity spectrum.

[0072] Step 230: Construct the input parameters of the trained behavior recognition model based on the time-Doppler velocity spectrum.

[0073] Step 240: processing the input parameters according to the behavior recognition model to obtain a behavior recognition result.

[0074] The behavior recognition model is used to identify a plurality of behavior categories of the passive target. The behavior recognition model can be trained by any one of a CNN (Convolutional Neural Networks), a TCN (Temporal Convolutional Network), a hybrid model of a CNN and an RNN (Recurrent Neural Network).

[0075] According to application needs, the behavior categories can be running, falling, walking, etc., or the behavior categories can be various gesture actions (such as waving hands, pushing hands, clapping hands, etc.), or the behavior categories can be behaviors for sleep detection (such as heartbeat, breathing, turning over, etc.).

[0076] After generating the time-Doppler velocity spectrum, the electronic device can construct the input parameters of the behavior recognition model according to the time-Doppler velocity spectrum. For example, if the electronic device detects the behavior category of the passive target in real time, the electronic device can construct the input parameters by intercepting the time-Doppler velocity spectrum in the recent period of time in the process of updating the time-Doppler velocity spectrum.

[0077] After obtaining the input parameters, the input parameters are input into the behavior recognition model, so as to obtain the behavior recognition result output by the behavior recognition model corresponding to the input parameters. The behavior recognition result is a predicted behavior category.

[0078] Through the above measures, the electronic device determines the real Doppler velocity at a plurality of observation times according to the channel state information, and constructs the time-Doppler velocity spectrum representing the behavior state of the passive target by using the real Doppler velocity. Then, the input parameters constructed by using the time-Doppler velocity spectrum are classified and processed by the behavior recognition model, so as to obtain the behavior recognition result, and the accurate recognition of the behavior category is realized.

[0079] In an embodiment, before determining the real Doppler velocity by using the channel state information of each antenna array element, the channel state information corresponding to each antenna array element needs to be determined. Referring to FIG. 4, Figure 4 The flowchart of the method for determining the channel state information provided by the embodiment of the present application is shown in FIG. 4, which can include the following steps 410 to 460. Figure 4

[0080] Step 410: obtaining the channel estimation information corresponding to each antenna array element, and converting the channel estimation information into time-domain channel estimation information by inverse Fourier transform.

[0081] ​For each antenna element, the electronic device can perform channel estimation on the pilot signal of the Wi-Fi signal by using a least square (LS) algorithm, and obtain channel estimation information. The channel estimation information includes frequency domain channel estimation values of a plurality of symbols on a plurality of subcarriers.

[0082] After obtaining the channel estimation information, the electronic device can convert the channel estimation information in the frequency domain to the time domain by using an inverse fast Fourier transform (IFFT), and obtain time domain channel estimation information. This process can be represented by the following equation (1):

[0083]

[0084] wherein, is the time domain channel estimation information; is the channel estimation information; q is an index of an antenna element of a receiver; k is a subcarrier index; and l is a symbol index.

[0085] In an embodiment, the number of sampling points in the channel estimation information determined by the electronic device is small. Before performing the IFFT, the number of subcarriers in the channel estimation information can be supplemented to a first specified number by using a boundary repetition method. The first specified number can be the number of sampling points that can be processed by a subsequent IFFT.

[0086] The electronic device can repeat the frequency domain channel estimation values of the first subcarrier and the last subcarrier in the channel estimation information a plurality of times, thereby expanding the number of subcarriers to the first specified number. Here, the number of repetitions of the first subcarrier and the last subcarrier in the channel estimation information can be the same. For example, when the channel estimation information includes 60 subcarriers and the first specified number is 128, the expansion process can be represented by the following equation (2):

[0087]

[0088] wherein m is 0 to 59, indicating the channel estimation information in the middle, including frequency domain channel estimation values of 60 subcarriers; q is an index of an antenna element of a receiver; k is a subcarrier index; and l is a symbol index. As shown in equation (2), the frequency domain channel estimation value of the first subcarrier is repeated 34 times, and the frequency domain channel estimation value of the last subcarrier is repeated 34 times, thereby obtaining the supplemented channel estimation information.

[0089] The electronic device can perform a circular shift operation on the supplemented channel estimation information, and then perform the IFFT.

[0090] Step 420: determining the power corresponding to a plurality of channels according to the time domain channel estimation information.

[0091] For the time domain channel estimation information of each antenna element, the electronic device can calculate the power corresponding to the time domain information. The power calculation manner can be represented by the following formula (3):

[0092]

[0093] wherein, P q, l k represents the power of the time domain channel of the lth symbol of the qth antenna element on the kth subcarrier; h q, l k represents the time domain channel estimation value of the lth symbol of the qth antenna element on the kth subcarrier.

[0094] Step 430: According to the preset window length, the multiple window-in channels with the maximum power are divided, and the average value of the power corresponding to the window-out channel is calculated as the noise power.

[0095] wherein, the window length can be configured as needed. In an embodiment, the window length can be determined according to the cyclic prefix length. Exemplarily, the relationship between the window length L and the cyclic prefix length CP can be represented as L≤2*CP-1.

[0096] For any antenna element, after calculating the power of each time domain channel, the electronic device can screen out multiple time domain channels with the maximum power as the window-in channels through the window length, and the remaining time domain channels are the window-out channels. The window-out channel can be considered as the channel where the noise is located, and the electronic device can calculate the average value of the power of the window-out channel as the noise power.

[0097] Step 440: Determine the threshold power through the noise power and a preset first adjustment coefficient.

[0098] Here, the first adjustment coefficient is used to convert the threshold power according to the noise power, which can be pre-configured according to experience.

[0099] The electronic device can multiply the noise power by the first adjustment coefficient to obtain the threshold power.

[0100] Step 450: Screen out a first window-in channel and a second window-in channel from the multiple window-in channels, set the time domain channel estimation value corresponding to the second window-in channel to zero, and construct target time domain channel estimation information based on the time domain channel estimation value corresponding to the first window-in channel and the time domain channel estimation value corresponding to the second window-in channel; wherein, the second window-in channel includes the window-in channel with the power not greater than the threshold power and the window-in channel with the maximum power; the first window-in channel is the window-in channel other than the second window-in channel.

[0101] The electronic device can select, from the multiple intra-window channels, intra-window channels with power no greater than a threshold power as second intra-window channels. For intra-window channels with power greater than the threshold power, the intra-window channel with the greatest power can be selected as a second intra-window channel. The remaining intra-window channels with power greater than the threshold power can be first intra-window channels.

[0102] Here, the second intra-window channels with power no greater than the threshold power can be considered as noise channels, and the electronic device can set time-domain channel estimation values corresponding to these second intra-window channels to zero, thereby performing noise removal. The second intra-window channel with the greatest power can be considered as a LOS (Line of Sight) channel, and setting the time-domain channel estimation value corresponding to this second intra-window channel to zero can remove the influence of the LOS background.

[0103] The electronic device can construct target time-domain channel estimation information based on time-domain channel estimation values of all first intra-window channels and time-domain channel estimation values of all second intra-window channels. The target time-domain channel estimation information further removes noise in the time domain and the LOS background, and can accurately represent the influence of passive target motion on NLOS (Not Line of Sight) channels.

[0104] For each antenna array element, corresponding target time-domain channel estimation information can be constructed.

[0105] In an embodiment, if the number of subcarriers in the channel estimation information is expanded, and a circular shift operation is performed on the expanded channel estimation information, when the target time-domain channel estimation information is determined in step 450, intermediate time-domain channel estimation information is constructed based on time-domain channel estimation values of all first intra-window channels and time-domain channel estimation values of all second intra-window channels.

[0106] The electronic device can perform zero padding on the intermediate time-domain channel estimation information, which is to add time-domain channels with zero time-domain channel estimation values in the intermediate time-domain channel estimation information. The electronic device can expand the number of channels in the intermediate time-domain channel estimation information to a second specified number by zero padding, thereby obtaining the target time-domain channel estimation information. The target time-domain channel estimation information includes time-domain channel estimation information corresponding to the second specified number of channels. In an embodiment, when the electronic device performs zero padding, a plurality of zeros can be added among the plurality of time-domain channel estimation values in the intermediate time-domain channel estimation information.

[0107] For example, the zero padding can be represented by the following formula (4):

[0108]

[0109] wherein n is a second designated number; q is an index of an antenna element of the receiver; and I is a symbol index. The example in equation (4) is to estimate the intermediate time-domain information containing 128 sampling points to 512 sampling points by zero padding. The actual number of sampling points can be configured as needed.

[0110] Step 460: performing a Fourier transform on the target time-domain channel estimation information of each antenna element respectively to obtain the channel state information of each antenna element in the frequency domain.

[0111] After obtaining the target time-domain channel estimation information corresponding to each antenna element, the electronic device can perform a Fast Fourier Transform on the target time-domain channel estimation information respectively, thereby obtaining the channel state information of each antenna element in the frequency domain.

[0112] In an embodiment, in the case of obtaining the target time-domain channel estimation information by zero padding the intermediate time-domain channel estimation information, the electronic device can perform a Fourier transform on the target time-domain channel information of each antenna element respectively to obtain the intermediate channel estimation result of each antenna element in the frequency domain.

[0113] The electronic device can perform a circular shift operation on each intermediate channel estimation result respectively, and sample the intermediate channel estimation result after the circular shift operation to obtain the channel state information of each antenna element. For example, a third designated number of frequency domain channel estimation values in the middle of the intermediate channel estimation result after the circular shift operation can be selected for sampling. Here, the third designated number can be configured as needed. For example, the second designated number is 512, and the third designated number can be 240.

[0114] Referring to Figure 5 , the schematic diagram of the method for determining the channel state information provided by an embodiment of the present application is shown in Figure 5 , for the channel estimation information of any symbol of any antenna element, the number of subcarriers in the channel estimation information is expanded to a first designated number by the boundary repetition method, so as to Figure 5For example, with a first specified quantity of 128, for any symbol, the frequency domain channel estimate of that symbol on the first subcarrier can be repeated 34 times, and the frequency domain channel estimate of that symbol on the 60th subcarrier can be obtained, thus obtaining the expanded channel estimate information corresponding to that symbol. Noise reduction processing is performed on the expanded channel estimate information corresponding to each symbol of each antenna element: a circular displacement operation is performed on the expanded channel estimate information, and then the channel estimate information is transformed into time domain channel estimate information through inverse Fourier transform. The power corresponding to each channel is determined based on the time domain channel estimate information. Multiple in-window channels with the highest power are divided according to a preset window length (e.g., 16), and the average power of the 112 out-of-window channels is calculated as the noise power. After determining the threshold power using the first adjustment coefficient and the noise power, the first in-window channel and the second in-window channel are determined based on the threshold power. The time domain channel estimate value corresponding to the second in-window channel is set to zero, and intermediate time domain channel estimate information is constructed using the time domain channel estimates values ​​of the first and second in-window channels. Zero-padding is performed on the intermediate time-domain channel estimation information so that the zero-padding time-domain channel estimation information includes a second specified number of zeros. Figure 5 The target time-domain channel estimation information is obtained by sampling 512 points. Each sampling point represents a time-domain channel estimation value corresponding to a channel. The target time-domain channel estimation information of each symbol of each antenna element is subjected to Fourier transform to obtain the intermediate channel estimation result in the frequency domain for each antenna. This intermediate channel estimation result includes 512 sampling points, each sampling point being a frequency-domain channel estimation value. A circular displacement operation is performed on each intermediate channel estimation result, and a third specified number of samples (inclusive) from the middle of the circular displacement intermediate channel estimation result are sampled. Figure 5 The middle part contains 240 sampling points, which serve as the channel state information for each antenna element.

[0115] In one embodiment, see Figure 6 This is provided as an embodiment of the present application. Figure 2 A detailed flowchart of step 210 is shown below. Figure 6 As shown, when performing step 210, steps 211 to 217 can be specifically performed.

[0116] Step 211: Designate each receiver as a target receiver and the antenna array element under the target receiver as the target antenna array element.

[0117] The electronic device can respectively take the first receiver and the second receiver as a target receiver, and take an antenna array element under the target receiver as a target antenna array element, to perform the subsequent steps 212 to 216. When taking the first receiver as the target receiver, the second receiver can be taken as a specified receiver. When taking the second receiver as the target receiver, the first receiver can be taken as the specified receiver.

[0118] Step 212: Selecting, from the channel state information of the target antenna array element, a minimum non-zero frequency domain channel estimation value as a second adjustment coefficient.

[0119] After selecting any antenna array element as the target antenna array element, the electronic device can select, from the frequency domain channel estimation values of a plurality of symbols included in the channel state information of the target antenna array element, a minimum non-zero frequency domain channel estimation value as a second adjustment parameter.

[0120] For example, the channel state information of the antenna array element of the first receiver can be denoted as Here, k represents a subcarrier index, and l represents a symbol index. The channel state information includes k*l frequency domain channel estimation values, from which the electronic device can select a minimum non-zero frequency domain channel estimation value as the second adjustment parameter γ.

[0121] Step 213: Processing the second adjustment coefficient according to a preset amplification coefficient to obtain a third adjustment coefficient.

[0122] The amplification coefficient can amplify the second adjustment coefficient, and the value of the amplification coefficient can be set as needed. For example, the amplification coefficient can be tens, hundreds, thousands, etc.

[0123] The electronic device can multiply the amplification coefficient by the second adjustment coefficient to obtain the third adjustment coefficient β.

[0124] Step 214: Adjusting the channel state information by the second adjustment coefficient to obtain first adjusted channel state information.

[0125] The electronic device can filter out non-zero frequency domain channel estimation values from all frequency domain channel estimation values of the channel state information of the target antenna array element, and subtract the second adjustment coefficient from each of the filtered frequency domain channel estimation values, to obtain the first adjusted channel state information.

[0126] For example, the channel state information of the antenna array element of the first receiver includes k*l frequency domain channel estimation values. After filtering out a plurality of non-zero frequency domain channel estimation values, the second adjustment coefficient is subtracted from each of the filtered frequency domain channel estimation values, to obtain the first adjusted channel state information.

[0127] The adjustment process can clean up the deviation caused by the phase in the channel state information.

[0128] Step 215: adjusting the channel state information of the antenna elements under the specified receiver with a third adjustment coefficient to obtain second adjusted channel state information; wherein the specified receiver is a receiver other than the target receiver.

[0129] For the channel state information of each antenna element under the specified receiver, the electronic device can add the frequency domain channel estimation values of multiple symbols in multiple subcarriers in the channel state information to the third adjustment coefficient respectively to obtain the second adjusted channel state information.

[0130] Step 216: conjugate multiplying the first adjusted channel state information and the second adjusted channel state information to obtain the specified channel state information of the target antenna element.

[0131] After obtaining the first adjusted channel state information and the second adjusted channel state information, the first adjusted channel state information and the second adjusted channel state information can be conjugate multiplied to obtain the specified channel state information of the target antenna element.

[0132] Taking the first receiver and the second receiver as the target receiver respectively, and taking the antenna elements under the target receiver as the target antenna element, the specified channel state information corresponding to the antenna elements of the first receiver and the second receiver can be respectively processed. After the above processing, the influence of the passive target on the channel state information is amplified in the specified channel state information.

[0133] Step 217: determining the real Doppler velocity of the passive target according to the specified channel state information corresponding to each antenna element.

[0134] After obtaining the specified channel state information corresponding to each antenna element, the electronic device can determine the real Doppler velocity of the passive target according to the multiple specified channel state information.

[0135] Through the above measures, the influence of the passive target on the channel state information in the presence of various behaviors can be amplified, the specified channel state information corresponding to each antenna element can be obtained, and then the real Doppler velocity of the passive target can be accurately obtained according to the specified channel state information of each antenna element.

[0136] In an embodiment, the electronic device can construct a corresponding covariance matrix for the specified channel state information corresponding to the first receiver and the second receiver respectively.

[0137] The way of constructing the covariance matrix for the specified channel state information corresponding to any receiver can be represented by the following formula (5):

[0138] R xx = E[XX H ] (5)

[0139] wherein X is a designated channel state information; E is a unit matrix; R xx is a covariance matrix.

[0140] After obtaining the covariance matrix corresponding to the first receiver, the electronic device can perform eigenvalue decomposition on the covariance matrix, thereby obtaining a first observed Doppler velocity of the passive target relative to the first receiver.

[0141] The electronic device can perform eigenvalue decomposition on the covariance matrix corresponding to the first receiver by using algorithms such as a MUSIC (Multiple Signal Classification Algorithm) algorithm, an ESPRIT (Estimation of Signal Parameters using Rotational Invariance Techniques) algorithm, and a compressed sensing algorithm, thereby obtaining a Doppler velocity peak caused by movement of the passive target, as the first observed Doppler velocity of the passive target relative to the first receiver.

[0142] Similarly, after obtaining the covariance matrix corresponding to the second receiver, the electronic device can perform eigenvalue decomposition on the covariance matrix, thereby obtaining a second observed Doppler velocity of the passive target relative to the second receiver.

[0143] When there are multiple moving passive targets in the indoor scene, for each passive target, the first observed Doppler velocity of the passive target relative to the first receiver and the second observed Doppler velocity of the passive target relative to the second receiver can be respectively analyzed.

[0144] The electronic device can fuse the first observed Doppler velocity and the second observed Doppler velocity, thereby determining the real Doppler velocity of the passive target.

[0145] In an embodiment, the behavior recognition model can be trained by a CNN model, and before training, a sample data set can be constructed. The electronic device can obtain a plurality of sample channel state information, and each channel state information corresponds to a behavior category.

[0146] For each behavior category to be identified subsequently, a person of different stature can continuously perform the action of the behavior category in the indoor environment, so as to obtain sample channel estimation information corresponding to the behavior category, and the sample channel estimation information can be converted into sample channel state information through cleaning. The electronic device can convert the sample channel state information into a sample time-Doppler velocity spectrum. The sample time-Doppler velocity spectrum is periodically cut to obtain a sample sub-spectrum. The period length of the cutting can be a preset empirical value, which can be equal to the interval length of the subsequent adjacent two observation times. For example, the period length is 2 seconds.

[0147] The electronic device can add a behavior category label to the sample sub-spectrum, which indicates the behavior category corresponding to the sample channel estimation information. Further, a sample data set can be constructed based on a plurality of sample sub-spectra carrying the behavior category label. After obtaining the sample data set, the CNN model can be supervised trained to obtain a behavior recognition model.

[0148] In this case, when constructing the input parameter, the electronic device can periodically cut the time-Doppler velocity spectrum to obtain a sub-spectrum, and take a single sub-spectrum as the input parameter of the behavior recognition model. Here, the period length of the cutting can be equal to the period length of the cutting of the sample time-Doppler velocity spectrum during the training.

[0149] In an embodiment, the behavior recognition model can be trained by a hybrid model of a convolutional neural network and a recurrent neural network. The recurrent neural network is used in the hybrid model to process the data input into the classification layer, so as to learn the correlation of different postures corresponding to the same behavior category in time. In an embodiment, the recurrent neural network can be LSTM (long short-term memory), GRU (Gate Recurrent Unit), etc.

[0150] In this case, when constructing the input parameter, the electronic device can cut the time-Doppler velocity spectrum according to a preset cutting window length and a preset cutting step length, obtain a plurality of sub-spectra continuous in time, and construct a sub-spectrum sequence from the continuous plurality of sub-spectra. Here, the number of sub-spectra in the sub-spectrum sequence can be preconfigured as needed, and the time length corresponding to the sub-spectrum sequence can be the time length required to complete a behavior (for example: 2 seconds). The cutting window length and the cutting step length can be preconfigured as needed, and the cutting step length is less than the cutting window length. For example, the cutting window length is 1 second, and the cutting step length is 0.1 second.

[0151] After constructing the sub-spectrum sequence, the sub-spectrum sequence can be taken as an input parameter, and the sub-spectrum sequence is processed by the behavior recognition model to obtain a behavior recognition result.

[0152] In an embodiment, the electronic device can train a behavior recognition model based on a mixture model. The electronic device can input a sample sub-spectrum sequence in a sample data set into the mixture model as input of the mixture model, and obtain a predicted behavior category output by the mixture model. The sample data set includes a plurality of sample sub-spectrum sequences, and each sample sub-spectrum sequence carries a corresponding behavior category label. Before training the mixture model, a plurality of sample time-Doppler velocity spectra can be constructed, each sample time-Doppler velocity spectrum corresponds to a behavior category, and each sample time-Doppler velocity spectrum can be processed according to a clipping window length and a clipping step length to obtain a plurality of sample sub-spectrum sequences, and a corresponding behavior category label can be added to each sample sub-spectrum sequence.

[0153] The electronic device can evaluate the difference between the behavior category label corresponding to each sample sub-spectrum sequence and the predicted behavior category by a preset loss function, obtain a function value, and adjust the model parameters of the mixture model according to the function value.

[0154] After adjusting the model parameters, the sample sub-spectrum sequence can be re-input into the mixture model to further adjust the model parameters. This process is repeated, and after repeated iterations, when the function value of the loss function tends to be stable, it can be determined that the mixture model converges, and a behavior recognition model is obtained.

[0155] Figure 7 is a block diagram of a behavior recognition device based on channel state information according to an embodiment of the present application, as shown in Figure 7 The device can include:

[0156] The determination module 710 is configured to determine the true Doppler velocity of the passive target for the channel state information of each antenna element.

[0157] The generation module 720 is configured to generate a time-Doppler velocity spectrum according to the true Doppler velocities of a plurality of observation times.

[0158] The construction module 730 is configured to construct an input parameter of a trained behavior recognition model according to the time-Doppler velocity spectrum.

[0159] The recognition module 740 is configured to process the input parameter according to the behavior recognition model to obtain a behavior recognition result.

[0160] The implementation process of the functions and roles of each module in the above device is specifically described in the implementation process of the corresponding steps in the above behavior recognition method based on channel state information, and will not be repeated here.

[0161] In several embodiments provided in the present application, the disclosed apparatus and method can also be implemented in other manners. The above described apparatus embodiments are merely illustrative. For example, the flowchart and block diagram in the accompanying drawings illustrate the possible implementation modes of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. In some alternative implementation modes, the functions noted in the blocks can occur in different orders from those noted in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for executing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0162] In addition, each functional module in the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0163] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A channel state information based behavior recognition method applied to a Wi-Fi transceiving system, the Wi-Fi transceiving system comprising a transmitter, a first receiver and a second receiver, characterized in that, The first receiver and the second receiver each include at least one antenna element, comprising: For each antenna element, the channel state information is used to determine the real Doppler velocity of the passive target; According to the real Doppler velocity at multiple observation times, a time-Doppler velocity spectrum is generated; According to the time-Doppler velocity spectrum, an input parameter of a trained behavior recognition model is constructed; According to the behavior recognition model, the input parameter is processed to obtain a behavior recognition result; Before the channel state information of each antenna element is used to determine the real Doppler velocity of the passive target, the method further comprises: Obtain the channel estimation information corresponding to each antenna element, and convert the channel estimation information into time-domain channel estimation information through inverse Fourier transform; According to the time-domain channel estimation information, the power corresponding to multiple channels is determined; According to the preset window length, the channels with the maximum power in multiple windows are divided, and the average value of the power corresponding to the channels outside the window is calculated as the noise power; Determine the threshold power by the noise power and a preset first adjustment coefficient; From the multiple window channels, a first window channel and a second window channel are selected, the time-domain channel estimation value corresponding to the second window channel is set to zero, and the target time-domain channel estimation information is constructed based on the time-domain channel estimation value corresponding to the first window channel and the time-domain channel estimation value corresponding to the second window channel; wherein the second window channel includes a window channel with a power not greater than the threshold power and a window channel with the maximum power; the first window channel is a window channel other than the second window channel; Perform Fourier transform on the target time-domain channel estimation information of each antenna element to obtain the channel state information of each antenna element in the frequency domain.

2. The method of claim 1, wherein, The channel state information of each antenna element is used to determine the real Doppler velocity of the passive target, comprising: Respectively, each receiver is taken as a target receiver, and the antenna element under the target receiver is taken as a target antenna element; From the channel state information of the target antenna element, the smallest non-zero frequency domain channel estimation value is selected as a second adjustment coefficient; According to a preset amplification coefficient, the second adjustment coefficient is processed to obtain a third adjustment coefficient; The second adjustment coefficient is used to adjust the channel state information to obtain first adjusted channel state information; The channel state information of the antenna element under the specified receiver is adjusted by the third adjustment coefficient to obtain second adjusted channel state information; wherein the specified receiver is a receiver other than the target receiver; The first adjusted channel state information and the second adjusted channel state information are conjugate multiplied to obtain the specified channel state information of the target antenna element; According to the specified channel state information corresponding to each antenna element, the real Doppler velocity of the passive target is determined.

3. The method of claim 2, wherein, The specified channel state information corresponding to each antenna element is used to determine the real Doppler velocity of the passive target, comprising: For the specified channel state information corresponding to the first receiver and the second receiver, a corresponding covariance matrix is constructed; perform eigenvalue decomposition on the covariance matrix corresponding to the first receiver to obtain a first observed Doppler velocity of the passive target relative to the first receiver; perform eigenvalue decomposition on the covariance matrix corresponding to the second receiver to obtain a second observed Doppler velocity of the passive target relative to the second receiver; determine the real Doppler velocity of the passive target according to the first observed Doppler velocity and the second observed Doppler velocity.

4. The method of claim 1, wherein, The behavior recognition model is trained by a convolutional neural network. The input parameters of the trained behavior recognition model are constructed according to the time-Doppler velocity spectrum, including: The time-Doppler velocity spectrum is periodically cut to obtain a sub-spectrum as an input parameter of the behavior recognition model.

5. The method of claim 1, wherein, The behavior recognition model is trained by a hybrid model of a convolutional neural network and a recurrent neural network. The input parameters of the trained behavior recognition model are constructed according to the time-Doppler velocity spectrum, including: The time-Doppler velocity spectrum is cut according to a preset cutting window length and a preset cutting step length to obtain a time-continuous sub-spectrum sequence as an input parameter of the behavior recognition model.

6. The method of claim 5, wherein, Before the input parameters are processed by the behavior recognition model to obtain a behavior recognition result, the method further includes: sample sub-spectrum sequences in a sample data set are input into the hybrid model to obtain a predicted behavior category output by the hybrid model; the sample data set includes a plurality of sample sub-spectrum sequences, and each sample sub-spectrum sequence carries a corresponding behavior category label; model parameters of the hybrid model are adjusted according to a difference between the predicted behavior category and the behavior category label; the above process is repeated until the hybrid model converges to obtain a behavior recognition model.

7. A channel state information based behavior recognition apparatus applied to a Wi-Fi transceiving system, the Wi-Fi transceiving system comprising a transmitter, a first receiver and a second receiver, characterized in that, The first receiver and the second receiver each include at least one antenna element, including: a determination module configured to determine a real Doppler velocity of a passive target for channel state information of each antenna element; a generation module configured to generate a time-Doppler velocity spectrum according to real Doppler velocities at a plurality of observation times; a construction module configured to construct input parameters of a trained behavior recognition model according to the time-Doppler velocity spectrum; an identification module configured to process the input parameters according to the behavior recognition model to obtain a behavior recognition result. The device is also used to: before determining the real Doppler velocity of the passive target based on the channel state information of each antenna element, obtain channel estimation information corresponding to each antenna element, and convert the channel estimation information into time-domain channel estimation information through inverse Fourier transform; determine the power corresponding to multiple channels according to the time-domain channel estimation information; divide multiple window-in channels with the maximum power according to a preset window length, and calculate the average value of the power corresponding to window-out channels as noise power; determine threshold power through the noise power and a preset first adjustment coefficient; select first window-in channels and second window-in channels from the multiple window-in channels, set the time-domain channel estimation value corresponding to the second window-in channels to zero, and construct target time-domain channel estimation information based on the time-domain channel estimation value corresponding to the first window-in channels and the time-domain channel estimation value corresponding to the second window-in channels; wherein the second window-in channels include window-in channels with power not greater than threshold power, and the window-in channel with the maximum power; the first window-in channels are window-in channels other than the second window-in channels; and perform Fourier transform on the target time-domain channel estimation information of each antenna element respectively to obtain the channel state information of each antenna element in the frequency domain.

8. An electronic device, comprising: The electronic device includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the behavior recognition method based on channel state information according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which can be executed by the processor to complete the behavior recognition method based on channel state information according to any one of claims 1-6.

Images