A wall-penetrating target behavior recognition method based on channel state information and a device thereof

Abstract

The application provides a wall-penetrating target behavior recognition method based on channel state information, which comprises the following steps: preprocessing the collected CSI signals; performing PCA data dimension reduction on the preprocessed CSI signals, removing redundant signals and irrelevant information, and extracting optimal subcarriers; performing first-order difference processing on the signals extracted after PCA dimension reduction, and then using a method based on a buffer sliding window to segment effective feature signal segments; converting the effective feature signal segments into feature images with time-frequency domain features through STFT, and inputting the feature images into a pre-trained SE-ResNet18 convolutional neural network for behavior recognition classification. The method can penetrate the wall to realize the behavior recognition of the human target behind the wall. Compared with the traditional deep learning network, the method adopts a small sample transfer learning method combined with a pre-trained model, and the recognition accuracy can reach 91.67% under the conditions of fewer training times, fewer iteration times and shorter training time, so that the behavior recognition classification task can be effectively completed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of human behavior perception, and particularly relates to a wall-penetrating target behavior recognition method based on channel state information and a device thereof. BACKGROUND

[0002] In recent years, human behavior perception technology is a major research hotspot in the Internet of Things, and can be applied to various scenes in life, such as smart home, virtual reality, health monitoring and social networks. Traditional behavior perception methods usually require users to collect data generated by human motion by wearing multiple sensors, which not only increases the inconvenience of the experiment, but also cannot be passively detected. Due to the limitations of these methods and the non-invasive nature of passive perception technology, the status of passive perception technology has become increasingly important.

[0003] The activity recognition technology based on WiFi is to capture the multipath superposition signal generated by the human target motion by using the WiFi device. The wireless signal mainly has two types, one is the signal strength (Received Signal Strength, RSS), and the other is the channel state information (Channel State Information, CSI). The most obvious difference between the two is that the latter can describe the channel characteristics in more detail than the former, has higher sensitivity, and has a larger detection range. In common behavior recognition, human activities can be roughly divided into two categories: coarse-grained activities and fine-grained activities. Coarse-grained activities usually refer to behaviors with large amplitudes such as walking, running, jumping, etc., and fine-grained activities usually refer to small-amplitude behaviors such as breathing, heartbeat, hand gestures, etc.

[0004] At present, the behavior recognition of the target is usually completed by using visual devices for detection, but this method is greatly affected by the visual distance and light conditions, and cannot realize the detection of the target behind the wall. Moreover, since the wall blocks the propagation of the wireless signal, it seriously interferes with the reception of the WiFi signal, which also increases the difficulty of processing the received signal and the accuracy of target behavior recognition. Therefore, the current behavior recognition research experiment scene is mainly in the indoor environment, and the data collected is mainly the human target feature data in the same space. However, since the wireless signal will be transmitted and diffracted when encountering obstacles in transmission, this characteristic can be used for passive detection of the human target behind the wall. Since different wall thicknesses and materials have different effects on the propagation of wireless signals, the human target behavior recognition in the wall-penetrating scene will be a great challenge.

[0005] Therefore, it is of great significance to propose a wall-penetrating target behavior recognition method based on channel state information and a device thereof. SUMMARY

[0006] In order to solve the problem of low accuracy of human target behavior recognition in the existing through-wall scene, the application provides a through-wall target behavior recognition method based on channel state information and a device thereof to solve the above technical defects.

[0007] In the first aspect, the application provides a through-wall target behavior recognition method based on channel state information, which comprises the following steps:

[0008] The collected CSI signals are preprocessed;

[0009] The preprocessed CSI signals are subjected to PCA data dimension reduction to remove redundant signals and irrelevant information and extract optimal subcarriers;

[0010] The signals extracted after PCA dimension reduction are subjected to first-order difference processing, and then the effective feature signal segments are segmented by using a method based on a buffer sliding window;

[0011] The effective feature signal segments are converted into feature images with time-frequency domain features by STFT, and are input into a pre-trained SE-ResNet18 convolutional neural network for behavior recognition classification.

[0012] Preferably, the method further comprises collecting multipath superimposed CSI signals of the target behind the wall by using a WiFi device.

[0013] Preferably, the preprocessing of the collected CSI signals comprises removing abnormal values of adjacent data points by a Hampel filter and removing medium and high frequency noise by a Butterworth low-pass filter to remove environmental background noise.

[0014] Preferably, the first-order difference processing of the signals extracted after PCA dimension reduction and the segmentation of the effective feature signal segments by using the method based on the buffer sliding window comprise:

[0015] Window length demarcation, the window length is divided according to 1 / 10 of the length of a sampling rate of 200;

[0016] Window mean value calculation and determination, the mean value of the first-order difference signal in each window is calculated, and for the threshold value, the mean value of the first-order difference signal of the complete data is used as the standard, and if it is greater than the threshold value, it is judged as a motion time slot, otherwise it is a stationary time slot;

[0017] Buffer window optimization, a buffer with a window length of 4 is demarcated, and for the window below the threshold value, if the number of windows exceeding the threshold value in the buffer reaches a certain number, the window is judged as a motion time slot;

[0018] Feature segment extraction, the windows exceeding the threshold value are combined to extract the start point and end point of the motion state.

[0019] Further preferably, it further comprises:

[0020] 8 length of the sliding window is drawn, if the window exceeds the threshold value in the sliding window reaches a certain number, the window is determined as motion time slot, using sliding window time slot correction scheme to solve the action in the middle of a feature less than the threshold value.

[0021] Preferably, it further comprises:

[0022] N t and N r respectively represent the number of transmitting and receiving antennas, K represents the number of subcarrier groups, the signal arriving at the receiving end through multipath transmission is represented as:

[0023] Y = HX + N

[0024] X and Y represent the transmission path vector of the transmitting signal and the receiving signal respectively, H and N are the CSI channel information matrix and additive white Gaussian noise respectively;

[0025] The CSI information matrix is represented as:

[0026] H = [H1, H2, H3……H k ]

[0027] Wherein, K represents the number of subcarrier groups divided according to the driver, each data packet will eventually get N t × N r × K CSI measurement value;

[0028] H i is the CSI on each subcarrier, which is represented as:

[0029]

[0030] Wherein, |H i | and ∠H i are the amplitude and phase of the i-th subcarrier respectively;

[0031] The CSI information matrix quantization estimation is H(f,t):

[0032]

[0033] Wherein, f represents the subcarrier frequency; t represents the current time; a k (f,t) represents the amplitude attenuation; P represents the set of propagation paths; τ k (t) represents the propagation delay; j represents the imaginary unit mark.

[0034] Preferably, it further comprises, the CSI signal received by the receiving end is divided into two parts:

[0035] H(f, t) = H stc (f) + H dyn (f, t)

[0036] Wherein, H stc (f) is a static component of the two static paths of the sight distance path of the signal after penetrating the wall and not affected by the human body and the path of reflection only through static obstacles; H dyn (f, t) is a dynamic component of the dynamic path of the signal after penetrating the wall and reflecting through the human body;

[0037] The dynamic component H dyn (f, t) is expressed as:

[0038]

[0039] c represents the speed of light; d k (t) represents the length of the dynamic path; a k (f, t) represents a part of the dynamic path; when the person is active, the length d k (t) of the dynamic path reflected through the human body will change with time t.

[0040] In a second aspect, the embodiments of the present application also provide a wall-penetrating target behavior recognition device based on channel state information, which comprises:

[0041] The acquisition module is used for acquiring the CSI signal;

[0042] The preprocessing module is used for preprocessing the acquired CSI signal;

[0043] The data dimension reduction module is used for performing PCA data dimension reduction on the preprocessed CSI signal, removing redundant signals and irrelevant information, and extracting optimal subcarriers;

[0044] The processing module is used for performing first-order difference processing on the extracted signal after PCA dimension reduction;

[0045] The segmentation module is used for segmenting the effective feature signal segment by using a method based on a buffer sliding window;

[0046] The conversion module is used for converting the effective feature signal segment into a feature image with time-frequency domain features through STFT;

[0047] The behavior recognition classification module is used for inputting the feature image with time-frequency domain features into a pre-trained SE-ResNet18 convolutional neural network for behavior recognition classification.

[0048] In a third aspect, an electronic device is provided, including: one or more processors; and a memory storing one or more programs configured to, working with the one or more processors, implement the method described in any implementation of the first aspect.

[0049] In a fourth aspect, a computer-readable storage medium is provided, having stored thereon a computer program configured to, working with a processor, implement the method described in any implementation of the first aspect.

[0050] Compared with the prior art, the beneficial results of the present application are:

[0051] (1) Compared with the general indoor human target behavior recognition, the method can penetrate the wall to realize the behavior recognition of the human target behind the wall. Secondly, compared with the traditional deep learning network, a large number of samples are usually needed to obtain high recognition accuracy. In the case of difficult CSI behavior data collection, the method uses small sample transfer learning combined with a pre-trained model, and the recognition accuracy can reach 91.67% under the condition of fewer training times, fewer iteration times and shorter training time, which can effectively complete the behavior recognition classification task. BRIEF DESCRIPTION OF DRAWINGS

[0052] The accompanying drawings are included to provide a further understanding of embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the present application. Other embodiments and many of the intended advantages of the present application will be readily appreciated as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings. The elements of the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding similar parts.

[0053] Figure 1 is an exemplary device architecture diagram in which an embodiment of the present application can be applied;

[0054] Figure 2 is a flowchart of a channel state information based wall-penetrating target behavior recognition method of an embodiment of the present application;

[0055] Figure 3 is a technical flowchart of a channel state information based wall-penetrating target behavior recognition method of an embodiment of the present application;

[0056] Figure 4 is a WiFi wall-penetrating signal scene diagram in a channel state information based wall-penetrating target behavior recognition method of an embodiment of the present application;

[0057] Figure 5A raw CSI signal graph in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0058] Figure 6 A Hampel abnormal point removal signal graph in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0059] Figure 7 A Butterworth low-pass filtering signal graph in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0060] Figure 8-1 、 Figure 8-2 A first-order difference signal graph and a first-order difference segmented signal graph in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application, respectively;

[0061] Figure 9 A time-frequency feature graph after segmentation of a first-order difference signal in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0062] Figure 10 An SE-ResNet18 transfer learning flowchart in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0063] Figure 11 An SE-ResNet18 network model structure in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0064] Figure 12 A schematic diagram of an SE-ResNet18 network model structure in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0065] Figure 13 A structure diagram of an SE-ResNet18 network model structure in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0066] Figures 14(a), 14(b), 14(c), and 14(d) are accuracy graphs of model training, accuracy graphs of model testing, loss value graphs of model training, and loss value graphs of model testing in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0067] Figure 15 A confusion matrix graph of behavior recognition classification in a channel state information-based through-wall target behavior recognition method of an embodiment of the present application;

[0068] Figure 16The behavior recognition classification result graph in the channel state information based through-wall target behavior recognition method of the embodiment of the present application;

[0069] Fig. 17(a), Fig. 17(b) are TSNE visualization scatter plot before classification, TSNE visualization scatter plot after classification in the channel state information based through-wall target behavior recognition method of the embodiment of the present application;

[0070] Figure 18 The different CNN model recognition accuracy and training time comparison graph in the channel state information based through-wall target behavior recognition method of the embodiment of the present application;

[0071] Figure 19 The structural schematic diagram of the channel state information based through-wall target behavior recognition device of the embodiment of the present application;

[0072] Figure 20 The structural schematic diagram of the computer device suitable for realizing the electronic equipment of the embodiment of the present application. DETAILED DESCRIPTION

[0073] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which are shown by way of illustration specific embodiments in which the application can be practiced. In this regard, directional terminology, such as "top," "bottom," "left," "right," "above," "below," etc., is used with reference to the orientation of the Figure(s) being described. Because components of embodiments can be positioned in a number of different orientations, the directional terminology is used for purposes of illustration and is in no way limiting. It is to be understood that other embodiments can be utilized and structural or logical changes can be made without departing from the scope of the present application. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present application is defined by the appended claims.

[0074] It should be understood that Figure 1 The number of terminal devices, networks and servers in the system is merely illustrative. Any number of terminal devices, networks and servers can be provided as needed.

[0075] Figure 1 An exemplary system architecture 100 for a method of processing information or for an apparatus for processing information to which embodiments of the application can be applied is shown.

[0076] As Figure 1 shown, the system architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium to provide a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or fiber optic cables, etc.

[0077] The user can use the terminal device 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0078] The terminal device 101, 102, 103 can be various electronic devices with communication functions, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0079] The server 105 can be a server providing various services, such as a background information processing server processing the verification request information sent by the terminal device 101, 102, 103. The background information processing server can analyze and process the received verification request information, and obtain the processing result (such as verification success information representing that the verification request is a legal request).

[0080] It should be noted that the method for processing information provided by the embodiment of the application is generally executed by the server 105, and correspondingly, the device for processing information is generally arranged in the server 105. In addition, the method for sending information provided by the embodiment of the application is generally executed by the terminal device 101, 102, 103, and correspondingly, the device for sending information is generally arranged in the terminal device 101, 102, 103.

[0081] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, to provide distributed services), or as a single software or multiple software modules, which is not limited here.

[0082] The application discloses a wall-penetrating target behavior recognition technology based on channel state information. The method uses a WiFi device to obtain a multipath superposition CSI signal to realize behavior recognition of a target behind a wall. The method first pre-processes the collected CSI signal, segments an effective behavior feature signal segment, converts the feature signal segment into a feature image with time-frequency domain features through STFT, and finally inputs the feature image into a pre-trained convolutional neural network for training and classification, so as to realize behavior recognition of the wall-penetrating target.

[0083] Compared with the general indoor human target behavior recognition, the method can penetrate the wall to realize the behavior recognition of the human target behind the wall. Secondly, compared with the traditional deep learning network, a large number of samples are usually needed to obtain high recognition accuracy. In the case of difficult CSI behavior data collection, the method uses a small sample transfer learning method combined with a pre-trained model, and the recognition accuracy can reach 91.67% with fewer training times.

[0084] Figure 2 The embodiment of the application discloses a channel state information based wall-penetrating target behavior recognition method, as shown in Figure 2 and Figure 3 , the method comprises the following steps:

[0085] S1, using a WiFi device to collect the multipath superposition CSI signal of the target behind the wall;

[0086] S2, pre-processing the collected CSI signal;

[0087] S3, performing PCA data dimensionality reduction on the pre-processed CSI signal, removing redundant signals and irrelevant information, and extracting optimal subcarriers;

[0088] S4, performing first-order difference processing on the signal extracted after PCA dimensionality reduction, and then using a buffer sliding window based method to segment the effective feature signal segment;

[0089] S5, converting the effective feature signal segment into a feature image with time-frequency domain features through STFT, and inputting it into a pre-trained SE-ResNet18 convolutional neural network for behavior recognition classification.

[0090] In one specific embodiment, the application proposes a channel state information based wall-penetrating target behavior recognition technology, which uses a WiFi device (5300 network card) with a directional antenna to collect CSI signals, as shown in Figure 4 , and pre-processes them, as shown in Figure 5 , Figure 6 and Figure 7As shown, the Hampel filter removes outliers of adjacent data points, the main purpose is to remove outliers of the original CSI signal, eliminate the influence of sudden noise points; Butterworth low-pass filter removes high-frequency noise, the frequency band of human activity is mainly concentrated in the 8-20Hz low-frequency band, and the noise contained in the CSI signal is mainly concentrated in the high-frequency part, therefore, the Butterworth low-pass filter can make the amplitude-frequency characteristic smooth in the passband and cut off the frequency band exceeding the cutoff frequency. Then, PCA dimension reduction processing is performed on all subcarriers, and the optimal subcarrier is extracted. Selecting such subcarriers can best reflect the target behavior characteristics and greatly reduce the computational complexity of data.

[0091] Then, the optimal subcarrier extracted by PCA is subjected to first-order difference processing, as shown in Figure 8-1 and Figure 8-2 The first-order difference is a transformation of time series composed of the difference between adjacent periods, which can well reflect the influence of human behavior on the CSI signal, and is convenient for subsequent behavior feature segmentation and extraction. The starting time and ending time of different behaviors are divided by using the method based on the buffer sliding window, and the effective feature signal segment is segmented, as shown in Figure 9 The main process is mainly divided into: window length determination, window mean calculation and determination, buffer window optimization, and feature segment extraction.

[0092] The main steps are as follows: select the window length of 1 / 10 of the sampling rate 200, and calculate the mean value of the first-order difference signal in each window. For the threshold value, the mean value of the first-order difference signal of the complete data is used as the standard, and if it is greater than the threshold value, it is judged as a motion time slot, otherwise it is a stationary time slot;

[0093] In order to extract the weak features of the start and end time of the motion, a buffer area of 4 window lengths is determined. For the window below the threshold value, if the number of windows exceeding the threshold value in the buffer area reaches a certain number, the window is determined as a motion time slot;

[0094] In order to solve the situation that a certain feature in the action is less than the threshold value, a sliding window time slot correction scheme is adopted, that is, 8 length sliding windows are determined, and if the number of windows exceeding the threshold value in the sliding window reaches a certain number, the window is determined as a motion time slot;

[0095] Finally, the windows exceeding the threshold value are merged, and the start point and end point of the motion state are extracted;

[0096] As shown in Figure 10The effective feature signal segment is converted into a feature image with time-frequency domain features through STFT, and is input into an SE-ResNet18 network pre-trained by ImageNet for behavior recognition classification. The model is pre-trained by using an ImageNet1000 dataset, and then is retrained by using the processed human behavior dataset, so that better classification effect can be achieved in a short time.

[0097] As Figure 11-13 shown, the SE-ResNet18 model is obtained by adding an SE-ResNet18 module to a ResNet18 model. The module can adaptively adjust the weight of each feature channel, so as to amplify important feature channels and enhance the directivity of the features.

[0098] In order to introduce the content of the application in detail, some concepts are further described or defined as follows:

[0099] Channel state information (CSI): the WiFi signal adopted by the system meets the wireless communication 802.11.n protocol, which divides data into multiple data streams through orthogonal frequency division multiplexing technology and transmits them on different frequency sub-channels. Different sub-channels are orthogonal to each other and do not affect each other. In the protocol, the channel characteristics between the sending end and the receiving end of the wireless signal from the physical layer can be obtained in the form of CSI through the OFDM technology. The CSI information can be obtained by using an Intel5300 network card. If N t and N r represent the number of antennas of the sending end and the receiving end respectively, and K represents the number of sub-carrier groups, then the signal transmitted to the receiving end through multipath transmission can be represented as:

[0100] Y=HX+N (1)

[0101] In formula (1), X and Y represent the transmission path vectors of the sending signal and the receiving signal respectively, and H and N are the CSI channel information matrix and the additive white Gaussian noise respectively. The CSI can be divided into different sub-carrier arrays according to the driver program of the receiving end hardware device, so the CSI information matrix can be represented as:

[0102] H=[H1,H2,H3……H k ] (2)

[0103] Wherein, K is the number of sub-carrier groups divided according to the driver, which is 56 in a 20Mhz bandwidth channel and 114 in a 40Mhz bandwidth, so that N t ×N r ×K CSI measurement values will be finally obtained for each data packet. Wherein H i is the CSI on each sub-carrier, which can be represented as:

[0104]

[0105] where |H i | and ∠H i are the amplitude and phase of the i-th subcarrier, respectively. Of course, the CSI matrix H(f, t) can also be quantized as H(f, t) = H

[0106]

[0107] where f is the subcarrier frequency; t is the current time; a k (f, t) is the amplitude attenuation; P is the set of propagation paths; τ k (t) is the propagation delay; j is the imaginary unit.

[0108] Multipath propagation model: In a typical through-wall indoor environment, when a WiFi signal propagates from a stationary transmitter to a receiver, there are multiple paths other than the main line-of-sight (LOS) path. The CSI received at the receiver is the linear superposition of the signals propagated through the multiple paths. These paths collectively constitute the multipath propagation of the wireless signal.

[0109] As for the above propagation model, a person in the room is also an obstacle. The person’s activities in the room cause the signal to reflect off the human body during propagation. When the person is active, the CSI received at the receiver can be divided into two parts:

[0110] H(f, t) = H stc (f) + H dyn (f, t) (5)

[0111] where H stc (f) is the static component composed of the LOS path after the signal passes through the wall and the paths that only reflect off static obstacles; H dyn (f, t) is the dynamic component composed of the paths that reflect off the human body after the signal passes through the wall. Of course, when the person is active, there are also parts of the body that do not change position. The paths that reflect off these parts of the body after the signal passes through the wall are counted as static components rather than dynamic components. For simplicity of analysis, this part of the signal is ignored. Referring to equation (4), the dynamic component H dyn (f, t) can be further expressed as follows:

[0112]

[0113] where c is the speed of light; d k (t) is the length of the dynamic path; a k(f,t) is a part of the dynamic path. The length d of the dynamic path reflected by the human body during activity. k (t) will change with time t, which indicates that the |H of the CSI signal in equation (3) i | and ∠H i It changes with people's activities.

[0114] Figure 14 shows the classification results of the SE-ResNet18 network model in activity classification. Figure 14(a) shows the accuracy of model training; Figure 14(b) shows the accuracy of model testing; Figure 14(c) shows the loss value of model training; and Figure 14(d) shows the loss value of model testing.

[0115] Figure 15 The image shows the confusion matrix of the classification results in the activity classification.

[0116] Figure 16 The image shows the classification results in the activity classification, including three parts: Precision, Recall, and F1-score.

[0117] Figure 17 shows a scatter plot of TSNE visualization before and after activity classification. It can be seen that the model can separate different behaviors well.

[0118] Figure 18 The image shows a comparison of the recognition accuracy and training time of different CNN models in activity classification. By comparing with each model, it can be found that the SE-ResNet18 model can achieve better classification results in a shorter time.

[0119] Secondly, embodiments of the present invention also disclose a wall-penetrating target behavior recognition device based on channel state information, such as... Figure 19 As shown, the device includes: a data acquisition module 191, a preprocessing module 192, a data dimensionality reduction module 193, a processing module 194, a segmentation module 195, a conversion module 196, and a behavior recognition and classification module 197.

[0120] In one specific implementation, the collection module 191 is configured to collect a CSI signal; the preprocessing module 192 is configured to preprocess the collected CSI signal; the data dimension reduction module 193 is configured to perform PCA data dimension reduction on the preprocessed CSI signal, remove redundant signals and irrelevant information, and extract optimal subcarriers; the processing module 194 is configured to perform first-order difference processing on the signals extracted after PCA dimension reduction; the segmentation module 195 is configured to segment effective feature signal segments by using a method based on a buffer sliding window; the conversion module 196 is configured to convert the effective feature signal segments into feature images with time-frequency domain features by STFT; and the behavior recognition classification module 197 is configured to input the feature images with time-frequency domain features into a pre-trained SE-ResNet18 convolutional neural network for behavior recognition classification.

[0121] Reference will now be made to the following description Figure 20 which shows a structure diagram of a computer device 2000 of an electronic device (e.g. Figure 1 a server or a terminal device) suitable for implementing embodiments of the present application. Figure 20 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of embodiments of the present application.

[0122] As shown in Figure 20 , the computer device 2000 includes a central processing unit (CPU) 2001 and a graphics processor (GPU) 2002, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 2003 or programs loaded from a storage portion 2009 into a random access memory (RAM) 2006. In the RAM 2004, various programs and data required for the operation of the device 2000 are also stored. The CPU 2001, the GPU 2002, the ROM 2003, and the RAM 2004 are connected to each other through a bus 2005. An input / output (I / O) interface 2006 is also connected to the bus 2005.

[0123] The following components are connected to the I / O interface 2006: an input portion 2007 including a keyboard, a mouse, and the like; an output portion 2008 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 2009 including a hard disk, and the like; and a communication portion 2010 including a network interface card such as a LAN card, a modem, and the like. The communication portion 2010 performs communication processing via a network such as the Internet. A drive 2011 can also be connected to the I / O interface 2006 as necessary. A removable medium 2012 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 2011 as necessary, so that a computer program read therefrom is installed in the storage portion 2009 as necessary.

[0124] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program in accordance with embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 2010, and / or installed from the removable medium 2012. When the computer program is executed by the central processing unit (CPU) 2001 and the graphics processor (GPU) 2002, the above-described functions defined in the methods of the present disclosure are performed.

[0125] It should be noted that the computer readable medium described in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable medium may, for example, be—but is not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor technology, or any suitable combination of the above. More specific examples of the computer readable medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution device, apparatus, or device. In the present disclosure, the computer readable signal medium can include a data signal that propagates in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that can be used to carry or store computer readable program code for use by or in connection with an instruction execution device, apparatus, or device. The program code contained on the computer readable medium can be transmitted or received over any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the above.

[0126] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0127] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of apparatuses, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0128] The modules described in the embodiments of the present application can be implemented through software or hardware. The modules described can also be set in a processor.

[0129] As another aspect, the application also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: pre-process the collected CSI signal; perform PCA data dimension reduction on the pre-processed CSI signal, remove redundant signals and irrelevant information, and extract optimal subcarriers; perform first-order difference processing on the signals extracted after PCA dimension reduction, and then use a method based on a buffer sliding window to segment effective feature signal segments; convert the effective feature signal segments into feature images with time-frequency domain features through STFT, and input them into a pre-trained SE-ResNet18 convolutional neural network for behavior recognition classification.

[0130] The above description is merely the preferred embodiments of the present application and the explanation of the applied technical principles. It should be understood by those skilled in the art that the scope of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features can be replaced with the technical features disclosed in the present application (but not limited to) having similar functions to form technical solutions.

Claims

1. A method for wall-penetrating target behavior recognition based on channel state information, characterized in that, The method comprises the following steps: The collected CSI signal is preprocessed; The preprocessed CSI signal is subjected to PCA data dimension reduction to remove redundant signals and irrelevant information and extract optimal subcarriers; The signal extracted after PCA dimension reduction is subjected to first-order difference processing, and then the effective feature signal segment is segmented by using a method based on a buffer sliding window; The effective feature signal segment is converted into a feature image with time-frequency domain features by STFT and input into a pre-trained SE-ResNet18 convolutional neural network for behavior recognition classification; N t and N r represent the number of transmit and receive antennas, respectively, then the signal arriving at the receiver through the multipath is represented as: Y = HX + N X and Y represent the transmission path vectors of the transmitted signal and the received signal, and H and N are the CSI channel information matrix and the additive Gaussian white noise, respectively; The CSI information matrix is represented as: H = [H1, H2, H3... H k ] where K represents the number of subcarrier groups divided according to the driver, each data packet will finally obtain N t × N r × K CSI measurement values; H i For the CSI on each subcarrier, it is denoted as: where |H i | and ∠H i are the amplitude and phase of the i-th subcarrier, respectively. The CSI information matrix is quantitatively estimated as H(f, t): where f denotes a subcarrier frequency; t denotes a current time; a k (f, t) denotes an amplitude attenuation; P denotes a set of paths of propagation; τ k (t) denotes a propagation delay; j denotes an imaginary unit marker. 2.The channel state information based through-the-wall target behavior recognition method according to claim 1, characterized in that, Further comprising collecting the multipath superposition CSI signal of the target behind the wall by using the WiFi device. 3.The channel state information based through-the-wall target behavior recognition method of claim 1, wherein, The preprocessing of the collected CSI signal comprises removing abnormal values of adjacent data points by a Hampel filter and removing medium and high frequency noise by a Butterworth low-pass filter to remove environmental background noise. 4.The channel state information based through-the-wall target behavior recognition method of claim 1, wherein, The signal extracted after PCA dimension reduction is subjected to first-order difference processing, and then the effective feature signal segment is segmented by using a method based on a buffer sliding window, comprising: Window length demarcation, the window length is demarcated by selecting 1 / 10 of the length of a sampling rate of 200; Window mean value calculation and determination, the mean value of the first-order difference signal in each window is calculated, and for the threshold value, the mean value of the first-order difference signal of the complete data is used as the standard, and if it is greater than the threshold value, it is determined as a motion time slot, and otherwise, it is determined as a stationary time slot; Buffer window optimization, a buffer with a length of 4 windows is demarcated, and for the window below the threshold value, if the number of windows exceeding the threshold value in the buffer reaches a certain number, the window is determined as a motion time slot; Feature segment extraction, the windows exceeding the threshold value are combined to extract the start point and end point of the motion state.

5. The channel state information based through-the-wall target behavior recognition method of claim 4, wherein, Further comprising: Demarcating 8 length sliding windows, if the number of windows exceeding the threshold value in the sliding window reaches a certain number, the window is determined as a motion time slot, and a sliding window time slot correction scheme is used to solve the case that a certain feature in the middle of the action is less than the threshold value. 6.The channel state information based through-the-wall target behavior recognition method of claim 1, wherein, Further comprising that the CSI signal received by the receiving end is divided into two parts: H(f, t) = H stc (f) + H dyn (f, t) where H stc (f) static component of the two static paths superimposed, the line-of-sight path after the signal passes through the wall and is not affected by the human body, and the path reflected only by static obstacles; dyn (f,t) dynamic component superimposed by the dynamic path after the signal passes through the wall and is reflected by the human body. Dynamic component H dyn (f, t) is expressed as: c represents the speed of light; d k (t) represents the length of the dynamic path; a k (f, t) represents a portion of the dynamic path; the length of the dynamic path d that passes through the human body when the person is active k (t) will change over time t. 7.A wall-penetrating target behavior recognition device based on channel state information, characterized in that, The device comprises the method according to any one of claims 1-6, and further comprises: The acquisition module is configured to acquire the CSI signal; The preprocessing module is configured to preprocess the acquired CSI signal; The data dimension reduction module is configured to perform PCA data dimension reduction on the preprocessed CSI signal to remove redundant signals and irrelevant information and extract optimal subcarriers; The processing module is configured to perform first-order difference processing on the signal extracted after PCA dimension reduction; The segmentation module is configured to segment the effective feature signal segment by using a method based on a buffer sliding window; The conversion module is configured to convert the effective feature signal segment into a feature image with time-frequency domain features by STFT; The behavior recognition classification module is configured to input the feature image with the time-frequency domain features into a pre-trained SE-ResNet18 convolutional neural network for behavior recognition classification.

8. An electronic device, comprising: one or more processors; storage configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method according to any one of claims 1-6. The program is executed by the processor to implement the method according to any one of claims 1-6.

Images