Through-wall living body activity monitoring method and system, terminal equipment and storage medium

Abstract

The invention is suitable for the technical field of communication, and provides a through-wall living body activity monitoring method and system, terminal equipment and a storage medium, and the method comprises the steps: obtaining first target CSI data according to initial CSI data; according to the first target CSI data, determining whether there is an active living body on the opposite side of the wall; if yes, obtaining second target CSI data according to the initial CSI data; acquiring third target CSI data and fourth target CSI data based on the action classifier and the second target CSI data; inputting the preprocessed third target CSI data into a first neural network to obtain a first result; inputting the preprocessed fourth target CSI data into a second neural network to obtain a second result; and finally, according to the first result and the second result, determining the action of the living body on the opposite side of the wall body. The behavior information of the user can be obtained in time, and then automatic regulation and control of the intelligent equipment are accurately carried out according to the behavior information of the user.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and in particular, relates to a method, system, terminal device and storage medium for monitoring living activity through walls. Background Art

[0002] The application of smart homes has greatly improved the comfort and convenience of the living environment. However, most of the smart devices currently rely too much on the instructions actively issued by the user during use. It is difficult to obtain the user's behavior information in a timely manner, and thus it is impossible to accurately perform automatic adjustments based on the user's behavior information. If the user's behavior information is obtained through a camera, it will not only increase the additional equipment cost, but also there will be a risk of privacy exposure. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a through-wall living activity monitoring method, system, terminal device and storage medium to solve the problem in the prior art that smart devices cannot obtain user behavior information in a timely manner and cannot accurately perform automatic adjustments based on user behavior information.

[0004] A first aspect of an embodiment of the present application provides a method for monitoring living body activities through a wall, which is implemented based on terminal devices located on both sides of a wall. The method for monitoring living body activities through a wall includes:

[0005] Obtaining first target CSI data according to the initial CSI data, wherein the initial CSI data includes multiple CSI data collected through multiple channels, each channel corresponding to one CSI data collected, and each CSI data includes N subcarriers, where N is a positive integer;

[0006] Determining, according to the first target CSI data, whether there is an active living body on the opposite side of the wall;

[0007] If there is an active living body on the opposite side of the wall, obtaining second target CSI data according to the initial CSI data;

[0008] Based on the action classifier and the second target CSI data, acquiring third target CSI data including the first action data and fourth target CSI data including the second action data;

[0009] Preprocessing the third target CSI data, and inputting the preprocessed third target CSI data into a first neural network to identify and classify the first action data to obtain a first result;

[0010] preprocessing the fourth target CSI data, and inputting the preprocessed fourth target CSI data into a second neural network to perform recognition and classification of second action data to obtain a second result;

[0011] The action of the living body on the opposite side of the wall is determined according to the first result and the second result.

[0012] A second aspect of an embodiment of the present application provides a through-wall living activity monitoring system, the through-wall living activity monitoring system comprising:

[0013] A first data acquisition module, configured to obtain first target CSI data according to initial CSI data, wherein the initial CSI data includes multiple CSI data collected through multiple channels, each channel correspondingly collects one CSI data, and each CSI data includes N subcarriers, where N is a positive integer;

[0014] A first judgment module, configured to determine whether there is an active living body on the opposite side of the wall according to the first target CSI data;

[0015] A second data acquisition module, configured to obtain second target CSI data according to the initial CSI data if there is an active living body on the opposite side of the wall;

[0016] A third data acquisition module, configured to acquire, based on the action classifier and the second target CSI data, third target CSI data including the first action data and fourth target CSI data including the second action data;

[0017] A first recognition module is used to preprocess the third target CSI data and input the preprocessed third target CSI data into a first neural network to recognize and classify the first action data to obtain a first result;

[0018] A second recognition module is used to preprocess the fourth target CSI data, and input the preprocessed fourth target CSI data into a second neural network to recognize and classify the second action data to obtain a second result;

[0019] The second judgment module is used to determine the action of the living body on the opposite side of the wall according to the first result and the second result.

[0020] A third aspect of an embodiment of the present application provides a terminal device, including a signal acquisition module, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the through-wall living activity monitoring method as described in the first aspect of the embodiment of the present application are implemented.

[0021] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the through-wall living activity monitoring method as described in the first aspect of the embodiment of the present application are implemented.

[0022] The first aspect of the embodiment of the present application provides a method for monitoring the activity of a living body through a wall, wherein first target CSI data is obtained according to initial CSI data, wherein the initial CSI data includes multiple CSI data collected through multiple channels, each channel corresponding to one CSI data, and each CSI data includes N subcarriers, where N is a positive integer; then, according to the first target CSI data, it is determined whether there is an active living body on the opposite side of the wall; if there is an active living body on the opposite side of the wall, second target CSI data is obtained according to the initial CSI data; and then, based on the action classifier and the second target CSI data, third target CSI data including the first action data and fourth target CSI data including the second action data are obtained. target CSI data; preprocessing the third target CSI data, and inputting the preprocessed third target CSI data into the first neural network to identify and classify the first motion data to obtain a first result; preprocessing the fourth target CSI data, and inputting the preprocessed fourth target CSI data into the second neural network to identify and classify the second motion data to obtain a second result; finally, according to the first result and the second result, determining the action of the living body on the opposite side of the wall, in this way, the user's behavior information can be obtained in time, and then the smart device can be accurately self-regulated according to the user's behavior information, without increasing additional equipment costs or the risk of exposing privacy.

[0023] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 This is a schematic diagram of a first flow chart of a method for monitoring living body activity through walls provided in an embodiment of the present application;

[0026] Figure 2 This is a second flow chart of the method for monitoring living body activity through walls provided in an embodiment of the present application;

[0027] Figure 31 is a third flow chart of the method for monitoring living body activity through walls provided in an embodiment of the present application;

[0028] Figure 4 This is a fourth flow chart of the method for monitoring living body activities through walls provided in an embodiment of the present application;

[0029] Figure 5 This is a fifth flow chart of the method for monitoring living body activity through walls provided in an embodiment of the present application;

[0030] Figure 6 is a schematic diagram of the structure of a through-wall living activity monitoring system provided in an embodiment of the present application;

[0031] Figure 7 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0033] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0034] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0035] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. "Multiple" means "two" or "more than two".

[0036] As a core application scenario of the Internet of Things, smart home has a broad market prospect. However, most traditional smart homes rely on the instructions actively issued by users to work, and the degree of intelligence and automation is not enough. Therefore, it is difficult to obtain user behavior information in a timely manner and accurately perform automatic regulation based on user behavior information.

[0037] The embodiment of the present application provides a method for monitoring the activity of a living body through a wall, wherein first target CSI data is obtained according to initial CSI data, wherein the initial CSI data includes multiple CSI data collected through multiple channels, each channel corresponding to one CSI data collected, and each CSI data includes N subcarriers, where N is a positive integer; then, according to the first target CSI data, it is determined whether there is an active living body on the opposite side of the wall; if there is an active living body on the opposite side of the wall, second target CSI data is obtained according to the initial CSI data; and then, based on the action classifier and the second target CSI data, third target CSI data including the first action data and fourth target CSI data including the second action data are obtained. CSI data; preprocessing the third target CSI data, and inputting the preprocessed third target CSI data into the first neural network to identify and classify the first action data to obtain a first result; preprocessing the fourth target CSI data, and inputting the preprocessed fourth target CSI data into the second neural network to identify and classify the second action data to obtain a second result; finally, according to the first result and the second result, determining the action of the living body on the opposite side of the wall, in this way, the user's behavior information can be obtained in time, and then the smart device can be accurately self-regulated according to the user's behavior information, without increasing additional equipment costs and without the risk of exposing privacy.

[0038] Embodiment 1

[0039] like Figure 1 As shown, the method for monitoring living body activity through a wall provided in the embodiment of the present application includes the following steps S1 to S7:

[0040] Step S1, obtain first target CSI data according to initial CSI data, and proceed to step S2.

[0041] In one embodiment, the initial CSI data includes multiple CSI data collected through multiple channels, each channel collects one CSI data, and each CSI data includes N subcarriers, where N is a positive integer.

[0042] In applications, channel state information (CSI) is the channel property of a communication link in wireless communications. It describes the combined impact of multiple effects such as scattering, fading, and power attenuation in the channel.

[0043] In applications, there are two types of CSI: instantaneous CSI and statistical CSI. Instantaneous CSI refers to the instantaneous state of the channel, which can be regarded as the impulse response of the digital filter. Instantaneous CSI can obtain the optimal received signal. Average CSI refers to the statistical characteristics of the channel over a period of time, which includes the distribution of environmental fading, average channel gain, line of sight (LOS) component and spatial correlation. Average CSI can optimize the signal transmission process. Instantaneous CSI is used in the embodiments of the present application.

[0044] In the application, the Wi-Fi network card in the terminal device can be used to collect initial CSI data at a certain sampling frequency. Specifically, the initial CSI data can be collected by using a Wi-Fi network card such as IntelAx210. The Wi-Fi network card can collect CSI data at a sampling rate of 200Hz. The collected initial CSI data includes two parts: phase data and amplitude data. Among them, the phase data is used to monitor whether there is an active human body, and the amplitude data is used to classify the movements of the monitored active human body.

[0045] In applications, the terminal device may work based on the IEEE802.11n protocol defined by the Institute of Electrical and Electronics Engineers (IEEE), or may work based on other communication protocols, which are only examples and do not constitute limitations.

[0046] In application, IEEE802.11n is a technology developed on the basis of IEEE802.11g and IEEE802.11a. It has a wide range of applications and its biggest feature is the rate improvement. The theoretical rate can reach up to 600Mbps. IEEE802.11n can operate in both 2.4GHz and 5GHz frequency bands.

[0047] In applications, due to factors such as different Wi-Fi network cards being in different positions or different receiving antennas in the same Wi-Fi network card being placed in different positions, the reflected signal strengths generated by human activities received by different antennas will be different. In order to improve the accuracy of human activity data monitoring, initial CSI data can be obtained from multiple channels.

[0048] In the application, the use of Wi-Fi signals to monitor human activities has high privacy security and strong privacy protection compared to capturing human activity information based on cameras. At the same time, it is not easily affected by light and is suitable for all-weather work.

[0049] In one embodiment, Figure 2As shown, step S1 includes the following steps S11 to S14:

[0050] Step S11, perform phase correction on the initial CSI data to obtain first CSI phase data, and proceed to step S12.

[0051] In the application, since the transmitting and receiving ends of the Wi-Fi network card are not accurately clock-synchronized, the collected initial CSI signal has a random phase deflection noise. This application can remove the random noise caused by the asynchronous clock of the network card by performing phase correction on the initial CSI data.

[0052] In an application, a CSI conjugate phase correction technique may be used to perform phase correction on initial CSI data, and then first CSI phase data may be obtained.

[0053] Step S12: Slice the first CSI phase data to obtain second CSI phase data, and proceed to step S13.

[0054] In the application, after obtaining the first CSI phase data, the first CSI phase data is sliced ​​so that the length of each segment after slicing is 1 second and the duty cycle is 50%. The data obtained after slicing is the second CSI phase data.

[0055] In applications, the duty ratio refers to the proportion of the power-on time to the total time in a pulse cycle.

[0056] Step S13: perform subcarrier fusion processing on the second CSI phase data to obtain third CSI phase data, and proceed to step S14.

[0057] In applications, principal component analysis (PCA) is a commonly used data analysis method. PCA can transform the original data into a set of linearly independent representations in each dimension through linear transformation. It can be used to extract the main characteristic components of the data and can be used for dimensionality reduction of high-dimensional data in unsupervised learning.

[0058] In applications, in high-dimensional space, the classifier is in danger of overfitting the training data set. Using PCA technology, the high-dimensional feature vector can be effectively reduced to a low dimension.

[0059] In the application, the sliced ​​second CSI phase data contains 57 subcarriers. Subcarrier fusion through PCA can reduce noise and reduce data dimension. The third CSI phase data can be obtained by performing subcarrier fusion processing on the second CSI phase data.

[0060] Step S14: extract features from the third CSI phase data to obtain first target CSI data.

[0061] In an application, after the third CSI phase data is obtained by performing subcarrier fusion through PCA, feature extraction may be performed on the third CSI phase data to obtain the first target CSI data.

[0062] In the application, the features of the time series after PCA dimensionality reduction are extracted and can be used for subsequent judgment of whether there are active living organisms.

[0063] In the application, feature extraction can be performed on the first 10 principal component data in the third CSI phase data to obtain features such as mean and variance. More features can be obtained according to actual needs, which are not limited here.

[0064] Step S2: Determine whether there is an active living body on the opposite side of the wall according to the first target CSI data, and proceed to step S3.

[0065] In application, the active living body may be an active human body or an animal other than a human being, which is not limited here.

[0066] In applications, Support Vector Machines (SVM) is a binary classification model whose basic model is a linear classifier with the largest margin defined in feature space; SVM also includes kernel techniques, which makes it a substantially nonlinear classifier.

[0067] In the application, after obtaining the first target CSI data, the SVM classifier can be used to determine whether there are signs of human activity in the time period included in each CSI slice, thereby determining whether there is an active human body on the opposite side of the wall.

[0068] In the application, SVM uses a linear kernel and soft margin. When multiple consecutive CSI slices are identified to contain human activity information, the start and end times of these CSI slices will be recorded and reported. At the same time, the CSI amplitude data within this start and end period is used for human motion classification.

[0069] In one embodiment, after step S2, the method further includes: if there is no active living body on the opposite side of the wall, returning to the step of obtaining first target CSI data according to the initial CSI data.

[0070] In the application, if the SVM classifier determines that the time period included in each CSI slice does not contain signs of human activity, it indicates that there is no active human body on the opposite side of the wall, and then the process returns to step S1.

[0071] Step S3: If there is an active living body on the opposite side of the wall, obtain second target CSI data according to the initial CSI data and proceed to step S4.

[0072] In the application, if human activities are detected in multiple consecutive CSI slices, it indicates that there is an active human body on the opposite side of the wall, and then the second target CSI data is obtained based on the initial CSI data.

[0073] In one embodiment, Figure 3 As shown, step S3 includes the following steps S31 to S32:

[0074] Step S31: if there is an active living body on the opposite side of the wall, obtaining first CSI amplitude data according to the initial CSI data;

[0075] Step S32: Perform feature extraction on the first CSI amplitude data to obtain second target CSI data.

[0076] In the application, when multiple consecutive CSI slices are identified to contain human activity information, the first CSI amplitude data within the start and end time periods of these CSI slices in the initial CSI data is immediately obtained, and then statistical features are extracted from the first CSI amplitude data to obtain second target CSI data.

[0077] Step S4: Based on the action classifier and the second target CSI data, obtain the third target CSI data including the first action data and the fourth target CSI data including the second action data, and proceed to step S5.

[0078] In the application, the action classifier includes but is not limited to a random forest classifier, and may also include other classifiers that can be used for action data classification.

[0079] In the application, the first motion data is walking motion data of a human body; the second motion data is other motions except walking, including sitting down, standing up, lying down, standing up from lying down, squatting and the like.

[0080] In the application, since the amplitude and duration of human walking movements are very different from other movements, a random forest classifier can be used to distinguish walking movements from other movements, and obtain the third target CSI data including the first movement data and the fourth target CSI data including the second movement data, which can reduce the difficulty and complexity of constructing subsequent neural networks.

[0081] Step S5, preprocess the third target CSI data, and input the preprocessed third target CSI data into the first neural network to identify and classify the first action data, obtain the first result, and enter step S6.

[0082] In the application, after obtaining the third target CSI data including the first action data, the third target CSI data is preprocessed to meet certain conditions and then input into a pre-designed first neural network to identify and classify the first action data, and obtain a first result.

[0083] In one embodiment, the first neural network is a one-dimensional convolutional neural network;

[0084] The first neural network includes 1 input layer, 2 convolutional layers, 2 pooling layers and 1 fully connected layer.

[0085] In applications, compared to traditional convolutional neural networks (CNNs) that perform convolution in two dimensions, one-dimensional convolutional neural networks only perform convolution in one dimension. In human motion classification applications, traditional CNNs also learn features between different subcarriers, which are generally considered to be irrelevant to human motion, while one-dimensional convolutional neural networks only extract features of changes in each subcarrier at different time points. Therefore, the use of a one-dimensional convolutional neural network in the first neural network can effectively reduce the overfitting of the neural network to the data and improve the generalization ability of the system.

[0086] In one embodiment, Figure 4 As shown, step S5 includes the following steps S51 to S52:

[0087] Step S51, performing subcarrier fusion, resampling and noise reduction processing on the third target CSI data;

[0088] Step S52: input the third target CSI data after subcarrier fusion, resampling and noise reduction processing into the first neural network to identify and classify the first action data to obtain a first result.

[0089] In the application, because the amplitude of human walking movements is large enough, the disturbance in the CSI data is more obvious. Therefore, when identifying the movement direction of walking movements, only the third target CSI data needs to be subjected to subcarrier fusion, resampling and noise reduction processing before being input into the first neural network for identification, thereby obtaining the first result including the walking direction.

[0090] In the application, the third target CSI data is first subjected to PCA subcarrier fusion, and the first 10 principal components are used for subsequent processing. Then, these principal components are resampled to the same data length and format, and then subjected to low-pass filter denoising to remove high-frequency noise. Finally, the processed data is input into the first neural network to identify and classify the first action data and obtain the first result.

[0091] Step S6, preprocess the fourth target CSI data, and input the preprocessed fourth target CSI data into the second neural network to identify and classify the second action data, obtain the second result, and enter step S7.

[0092] In the application, after obtaining the fourth target CSI data including the second action data, the fourth target CSI data is preprocessed to meet certain conditions and then input into a pre-designed second neural network to identify and classify the second action data, and obtain a second result.

[0093] In one embodiment, the second neural network is a one-dimensional convolutional neural network;

[0094] The second neural network includes 1 input layer, 4 convolutional layers, 2 pooling layers and 1 fully connected layer.

[0095] In applications, compared to traditional convolutional neural networks (CNNs) that perform convolution in two dimensions, one-dimensional convolutional neural networks only perform convolution in one dimension. In human motion classification applications, traditional CNNs also learn features between different subcarriers, which are generally considered to be irrelevant to human motion, while one-dimensional convolutional neural networks only extract features of changes in each subcarrier at different time points. Therefore, the use of a one-dimensional convolutional neural network in the second neural network can effectively reduce the overfitting of the neural network to the data and improve the generalization ability of the system.

[0096] In one embodiment, Figure 5 As shown, step S6 includes the following steps S61 to S62:

[0097] Step S61, performing subcarrier fusion, channel screening, resampling, noise reduction and feature enhancement processing on the fourth target CSI data;

[0098] Step S62: input the fourth target CSI data after subcarrier fusion, channel screening, resampling, noise reduction and feature enhancement processing into the second neural network to identify and classify the second action data to obtain a second result.

[0099] In the application, because the amplitude of actions other than walking, such as sitting down, standing up, lying down, standing up from lying down, and squatting, is small and the duration is short, when recognizing other actions such as sitting down, standing up, lying down, standing up from lying down, and squatting, the fourth target CSI data is subjected to subcarrier fusion, channel screening, resampling, noise reduction, and feature enhancement processing, and then input into the second neural network for recognition to obtain a second result including actions such as sitting down, standing up, lying down, standing up from lying down, and squatting.

[0100] In applications, since wireless communication uses free space as a transmission medium, electromagnetic waves carrying information will reach the receiving end through multiple paths, thus causing multipath interference. Therefore, under wall penetration conditions, the multipath interference of the channel is very strong. The signal captured by a Wi-Fi network card is usually a superposition of multiple channels. Since the paths of different channels are different, they interfere with the recognition of human movements. The embodiment of the present application can effectively select the most stable and most powerful main channel through an unsupervised channel screening algorithm, greatly reducing the interference of multipath effects.

[0101] In applications, the K-means algorithm has the advantages of fast convergence speed and better clustering effect. Therefore, the unsupervised channel screening algorithm based on the K-means algorithm can be used to screen out the main channel with the strongest power, and then cooperate with the Hampel filter to remove abnormal data in the CSI amplitude data, which can effectively reduce the interference caused by multipath channels.

[0102] In the application, feature enhancement is performed on the resampled and denoised CSI amplitude data to highlight the human motion information contained in the CSI amplitude data.

[0103] In the application, a sliding variance filter can be used to design the step size and window size according to the length of the CSI amplitude data segment, so as to further highlight the human motion information contained in the data, enhance the amplitude of the CSI amplitude data changing over time, and highlight the human motion information contained in the CSI amplitude data.

[0104] Step S7: Determine the action of the living body on the opposite side of the wall according to the first result and the second result.

[0105] In the application, after obtaining the first result and the second result, the user's behavior information is obtained by analyzing the first result and the second result, thereby realizing automatic adjustment and control of the smart terminal device, which is suitable for most usage scenarios of smart terminal devices and can also effectively reduce user expenses.

[0106] In application, the specific application scenario of the embodiment of the present application may be: when the first terminal device in the bedroom detects that the user is moving away, and the second terminal device in the living room detects that the user is approaching, the lights in the living room are controlled to turn on automatically; when the action of the user sitting down in the living room is detected, the TV is controlled to turn on actively.

[0107] In application, the specific application scenario of the embodiment of the present application may also be: when the first terminal device in the bedroom detects that the pet is moving away, and the second terminal device in the living room detects that the pet is approaching, the lights in the living room are controlled to turn on automatically or other smart devices in the living room are controlled to be automatically adjusted, which is not limited here.

[0108] In the application, the first terminal device and the second terminal device are both devices equipped with Wi-Fi network cards and capable of collecting CSI data.

[0109] In the application, the user can perform any action within the signal receiving range of the terminal device. Through the method of the embodiment of the present application, the terminal device can accurately identify the user's behavior information, thereby controlling the smart device to perform automatic regulation.

[0110] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0111] Embodiment 2

[0112] The embodiment of the present application also provides a through-wall living body activity monitoring system, which is used to execute the method steps in the above-mentioned through-wall living body activity monitoring method embodiment. The system can be a virtual appliance in a terminal device, which is run by a processor of the terminal device, or it can be the terminal device itself.

[0113] like Figure 6 As shown, the through-wall living activity monitoring system 100 provided in the embodiment of the present application includes a first data acquisition module 101, a first judgment module 102, a second data acquisition module 103, a third data acquisition module 104, a first identification module 105, a second identification module 106 and a second judgment module 107.

[0114] A first data acquisition module 101 is used to obtain first target CSI data according to initial CSI data, wherein the initial CSI data includes multiple CSI data collected through multiple channels, each channel corresponding to one CSI data is collected, and each CSI data includes N subcarriers, where N is a positive integer;

[0115] A first judgment module 102 is used to determine whether there is an active living body on the opposite side of the wall according to the first target CSI data;

[0116] A second data acquisition module 103 is configured to obtain second target CSI data according to the initial CSI data if there is an active living body on the opposite side of the wall;

[0117] A third data acquisition module 104, configured to acquire, based on the action classifier and the second target CSI data, third target CSI data including the first action data and fourth target CSI data including the second action data;

[0118] A first recognition module 105, configured to preprocess the third target CSI data, and input the preprocessed third target CSI data into a first neural network to recognize and classify the first action data to obtain a first result;

[0119] A second recognition module 106, configured to preprocess the fourth target CSI data, and input the preprocessed fourth target CSI data into a second neural network to recognize and classify the second action data to obtain a second result;

[0120] The second judgment module 107 is used to determine the action of the living body on the opposite side of the wall according to the first result and the second result.

[0121] In one embodiment, the first data acquisition module 101 is further configured to:

[0122] Performing phase correction on the initial CSI data to obtain first CSI phase data;

[0123] Slicing the first CSI phase data to obtain second CSI phase data;

[0124] Performing subcarrier fusion processing on the second CSI phase data to obtain third CSI phase data;

[0125] Feature extraction is performed on the third CSI phase data to obtain first target CSI data.

[0126] In one embodiment, the second data acquisition module 103 is further configured to:

[0127] If there is a moving living body on the opposite side of the wall, obtaining first CSI amplitude data according to the initial CSI data;

[0128] Perform feature extraction on the first CSI amplitude data to obtain second target CSI data.

[0129] In one embodiment, the first identification module 105 is further configured to:

[0130] Performing subcarrier fusion, resampling and noise reduction processing on the third target CSI data;

[0131] The third target CSI data after subcarrier fusion, resampling and noise reduction processing is input into the first neural network to identify and classify the first action data to obtain a first result.

[0132] In one embodiment, the second identification module 106 is further configured to:

[0133] Performing subcarrier fusion, channel screening, resampling, noise reduction, and feature enhancement processing on the fourth target CSI data;

[0134] The fourth target CSI data after subcarrier fusion, channel screening, resampling, noise reduction and feature enhancement processing is input into the second neural network to identify and classify the second action data to obtain a second result.

[0135] In application, each unit in the above system may be a software program module, or may be implemented by different logic circuits integrated in a processor or independent physical components connected to a processor, or may be implemented by multiple distributed processors.

[0136] Embodiment 3

[0137] like Figure 7 As shown, the embodiment of the present application further provides a terminal device 200, including: at least one processor 201 ( Figure 7 Only one processor is shown in the figure), a memory 202, a computer program 203 stored in the memory 202 and executable on at least one processor 201, and a signal acquisition module 204. When the processor 201 executes the computer program 203, the steps in the above-mentioned method embodiments are implemented.

[0138] In the application, the signal acquisition module may be a Wi-Fi network card, specifically a Wi-Fi network card including Intel Ax210 and the like, which is used to acquire initial CSI data at a certain sampling frequency.

[0139] In the application, the terminal device may include, but is not limited to, a processor, a memory, and a signal acquisition module. Figure 7 It is only an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as human-computer interaction devices, input and output devices, network access devices, etc. The network access device may include a communication module for the terminal device to communicate with the user terminal.

[0140] In applications, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. For example, the processor may be a timing controller (TCON). A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0141] In applications, the memory may be an internal storage unit of a terminal device in some embodiments, for example, a hard disk or memory of the terminal device. The memory may also be an external storage device of the terminal device in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the terminal device. The memory may also include both an internal storage unit of the terminal device and an external storage device. The memory is used to store operating systems, applications, boot loaders, data, and other programs, such as program codes of computer programs. The memory may also be used to temporarily store data that has been output or is to be output.

[0142] In the application, the communication module can be set as any device that can directly or indirectly perform long-distance wired or wireless communication with the user terminal according to actual needs. For example, the communication module can provide communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth, Zigbee, mobile communication networks, global navigation satellite systems (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., which are applied to network devices. The communication module may include an antenna, and the antenna may have only one array element or an antenna array including multiple array elements. The communication module can receive electromagnetic waves through the antenna, frequency modulate and filter the electromagnetic wave signals, and send the processed signals to the processor. The communication module can also receive the signal to be sent from the processor, frequency modulate and amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0143] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0144] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The functional modules in the embodiment can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. In addition, the specific names of the functional modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0145] The embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0146] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned various method embodiments.

[0147] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to the terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0148] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0149] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0150] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0151] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for monitoring living activity through a wall, characterized in that: It is implemented based on terminal devices located on both sides of the wall, including: Obtaining first target CSI data according to the initial CSI data, wherein the initial CSI data includes multiple CSI data collected through multiple channels, each channel corresponding to one CSI data collected, and each CSI data includes N subcarriers, where N is a positive integer; Determining, according to the first target CSI data, whether there is an active living body on the opposite side of the wall; If there is an active living body on the opposite side of the wall, obtaining second target CSI data according to the initial CSI data; Based on the action classifier and the second target CSI data, acquiring third target CSI data including the first action data and fourth target CSI data including the second action data; Preprocessing the third target CSI data, and inputting the preprocessed third target CSI data into a first neural network to perform recognition and classification of the first action data to obtain a first result; preprocessing the fourth target CSI data, and inputting the preprocessed fourth target CSI data into a second neural network to identify and classify the second action data to obtain a second result; The action of the living body on the opposite side of the wall is determined according to the first result and the second result.

2. The method for monitoring living body activity through a wall as claimed in claim 1, characterized in that: The first neural network and the second neural network are one-dimensional convolutional neural networks; The first neural network includes 1 input layer, 2 convolutional layers, 2 pooling layers and 1 fully connected layer; The second neural network includes 1 input layer, 4 convolutional layers, 2 pooling layers and 1 fully connected layer.

3. The method for monitoring living activity through a wall as claimed in claim 1 or 2, characterized in that: The obtaining, according to the initial CSI data, first target CSI data includes: Performing phase correction on the initial CSI data to obtain first CSI phase data; Slicing the first CSI phase data to obtain second CSI phase data; Performing subcarrier fusion processing on the second CSI phase data to obtain third CSI phase data; Perform feature extraction on the third CSI phase data to obtain first target CSI data.

4. The method for monitoring living body activity through a wall as claimed in claim 3, characterized in that: If there is an active living body on the opposite side of the wall, obtaining second target CSI data according to the initial CSI data includes: If there is an active living body on the opposite side of the wall, obtaining first CSI amplitude data according to the initial CSI data; Perform feature extraction on the first CSI amplitude data to obtain second target CSI data.

5. The method for monitoring living body activity through a wall as claimed in claim 4, characterized in that: The preprocessing of the third target CSI data and inputting the preprocessed third target CSI data into the first neural network to identify and classify the first action data to obtain the first result includes: Performing subcarrier fusion, resampling and noise reduction processing on the third target CSI data; The third target CSI data after subcarrier fusion, resampling and noise reduction processing is input into the first neural network to identify and classify the first action data to obtain a first result.

6. The method for monitoring living body activity through a wall as claimed in claim 5, characterized in that: The preprocessing of the fourth target CSI data and inputting the preprocessed fourth target CSI data into a second neural network to identify and classify the second action data to obtain a second result includes: Performing subcarrier fusion, channel screening, resampling, noise reduction, and feature enhancement processing on the fourth target CSI data; The fourth target CSI data after subcarrier fusion, channel screening, resampling, noise reduction and feature enhancement processing is input into the second neural network to identify and classify the second action data to obtain a second result.

7. The method for monitoring living body activity through a wall as claimed in claim 6, characterized in that: After determining whether there is an active living body on the opposite side of the wall according to the first target CSI data, the method further includes: If there is no active living body on the opposite side of the wall, the process returns to the step of obtaining the first target CSI data according to the initial CSI data.

8. A through-wall living activity monitoring system, characterized in that: The through-wall living activity monitoring system comprises: A first data acquisition module, configured to obtain first target CSI data according to initial CSI data, wherein the initial CSI data includes multiple CSI data collected through multiple channels, each channel correspondingly collects one CSI data, and each CSI data includes N subcarriers, where N is a positive integer; A first judgment module, configured to determine whether there is an active living body on the opposite side of the wall according to the first target CSI data; A second data acquisition module, configured to obtain second target CSI data according to the initial CSI data if there is an active living body on the opposite side of the wall; A third data acquisition module, configured to acquire, based on the action classifier and the second target CSI data, third target CSI data including the first action data and fourth target CSI data including the second action data; A first recognition module is used to preprocess the third target CSI data and input the preprocessed third target CSI data into a first neural network to recognize and classify the first action data to obtain a first result; A second recognition module is used to preprocess the fourth target CSI data, and input the preprocessed fourth target CSI data into a second neural network to recognize and classify the second action data to obtain a second result; The second judgment module is used to determine the action of the living body on the opposite side of the wall according to the first result and the second result.

9. A terminal device, comprising a signal acquisition module, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the through-wall living activity monitoring method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the through-wall living activity monitoring method as described in any one of claims 1 to 7 are implemented.

Images