A Human Motion Recognition Method and System Based on Wireless Channels

Abstract

The present invention provides a method and system for human action recognition based on a wireless channel. By combining the wireless signal propagation characteristics and beamforming technology, orthogonal beams covering the entire three-dimensional space are formed; the beams aligned with the human body are obtained by the difference between the beam domain reference channel state information and the current channel state information; the beam numbers and instantaneous Doppler frequencies are extracted from the residual channel data to form a feature vector; and then a machine learning classifier is trained to obtain a human action classifier, which can effectively classify a variety of human actions.

Description

Technical Field

[0001] The present invention belongs to the technical field related to human action recognition, and in particular, relates to a method and system for human action recognition based on a wireless channel. Background Art

[0002] The statements in this part merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of technology, human action recognition plays an important role in fields such as healthcare, virtual reality, and sports. For example, in the field of healthcare, the timely detection of a person's fall, especially for the elderly living alone, is crucial for subsequent treatment and rehabilitation. The existing human action recognition methods are mainly divided into two types, namely, action recognition schemes based on video surveillance and action recognition schemes based on wearable sensors. The latter scheme requires wearing devices such as acceleration sensors, which affects the comfort of users' daily lives and has problems such as limited device battery life. The former scheme can perform contactless recognition, but it involves user privacy issues and is difficult to install in bathrooms and toilets, which are the scenarios where falls occur most frequently. Summary of the Invention

[0004] To overcome the deficiencies of the above-mentioned prior art, the present invention provides a method and system for human action recognition based on a wireless channel, which combines the characteristics of wireless signal propagation and beamforming technology to form orthogonal beams covering the entire three-dimensional space; obtains the beams aligned with the human body through the difference between the beam domain reference channel state information and the current channel state information; extracts the beam number and instantaneous Doppler frequency in the residual channel data to form a feature vector; and then trains a machine learning classifier to obtain a human action classifier, which can effectively classify various human actions.

[0005] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions: A method for human action recognition based on a wireless channel, comprising the following steps:

[0006] Obtain the beam domain channel state information under the condition of no person and the beam domain channel state information under different human actions respectively;

[0007] Obtain the beam domain residual channel data based on the beam domain channel state information under the condition of no person and the beam domain channel state information under different human actions;

[0008] Extract the beam number and instantaneous Doppler frequency corresponding to the human action according to the residual channel data;

[0009] Input the obtained beam number and instantaneous Doppler frequency corresponding to the human action into the trained classifier for recognition, and output the human action recognition result.

[0010] Further, before obtaining the beam domain channel state information, setting system parameters is also included, including carrier frequency, horizontal distance between the transceiver, heights of the transmitting and receiving antennas, and antenna configuration.

[0011] Further, based on obtaining the beam domain channel state information, it specifically includes: performing beam domain horizontal and vertical sampling in a three-dimensional space under unmanned conditions or different human body movements; the beam domain horizontal sampling angle is determined by the number of columns of antennas in the transmitting antenna array and the spatial frequency in the horizontal direction; the beam domain vertical sampling angle is determined by the number of rows of antennas in the transmitting antenna array and the spatial frequency in the vertical direction.

[0012] Further, obtain the transmit beamforming matrix for forming P v ×P h orthogonal beams in the entire three-dimensional space. The specific calculation formula is:

[0013]

[0014]

[0015]

[0016] where denotes the Kronecker product, vectors b and a are the one-dimensional antenna array steering vectors in the vertical and horizontal directions of the transmitting array respectively, and are the spatial frequencies in the horizontal and vertical directions, P v and P h are the number of rows and columns of antennas in the transmitting antenna array respectively.

[0017] Further, the calculation formula for the beam domain channel state information under unmanned conditions is:

[0018]

[0019] where (·) * denotes conjugation, H Ref (t,f) is the 1×P dimensional spatial domain channel matrix at time t and frequency f, and P is the number of antennas;

[0020] The calculation formula for the beam domain channel state information under different human body movements is:

[0021]

[0022] where H(t,f) is the 1×P dimensional wireless channel matrix in the presence of people.

[0023] Further, extract the beam number and instantaneous Doppler frequency corresponding to the human body movement from the residual channel data, specifically including: differentiating the residual channel data with respect to time to obtain the instantaneous Doppler frequency under different movements, and extracting the beam horizontal number and beam vertical number corresponding to the beam with the strongest power in the residual channel data.

[0024] The second aspect of the present invention discloses a human body movement recognition system based on a wireless channel, including:

[0025] An acquisition module: respectively acquire the beam domain channel state information under unmanned conditions and the beam domain channel state information under different human body movements;

[0026] A residual channel calculation module: obtain the beam domain residual channel data based on the beam domain channel state information under unmanned conditions and the beam domain channel state information under different human body movements;

[0027] A data extraction module: extract the beam number and instantaneous Doppler frequency corresponding to the human body movement from the residual channel data;

[0028] An action recognition module: input the beam number and instantaneous Doppler frequency corresponding to the human body movement obtained above into a trained classifier for recognition, and output the human body movement recognition result.

[0029] The third aspect of the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps described in the above method.

[0030] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and computer instructions stored on the memory and running on the processor. When the computer instructions are run by the processor, the steps described in the above method are completed.

[0031] The above one or more technical solutions have the following beneficial effects:

[0032] The present invention utilizes the change of the beam domain wireless channel over time, extracts the instantaneous Doppler frequency caused by human body movement and the beam number corresponding to the human body position from the beam domain wireless channel residual data, and uses a pre-trained classifier to detect human body actions such as stillness, walking, and falling. By increasing the carrier frequency and the number of antennas, the action recognition accuracy of this method can be improved.

[0033] In the present invention, the user does not need to wear detection devices such as sensors, which improves the user's comfort. There is no need to rely on video devices such as cameras, and privacy is not involved.

[0034] The advantages of the additional aspects of the present invention will be partially given in the following description, partially will become obvious from the following description, or will be understood through the practice of the present invention. Brief Description of the Drawings

[0035] The attached drawings forming a part of this invention are used to provide a further understanding of the invention. The schematic embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation to the invention.

[0036] Figure 1 is a flowchart of human motion recognition of the present invention;

[0037] Figure 2 is a schematic diagram of the human motion recognition device in the first embodiment of the present invention;

[0038] Figure 3(a) is the multipath angle distribution of the reference channel in the first embodiment of the present invention;

[0039] Figure 3(b) is the beam distribution of the reference channel in the first embodiment of the present invention;

[0040] Figure 4(a) is the multipath angle distribution of the residual channel in the first embodiment of the present invention;

[0041] Figure 4(b) is the beam distribution of the residual channel in the first embodiment of the present invention;

[0042] Figure 5 is the confusion matrix of the human motion recognition result in the first example of the present invention. Detailed Description of the Embodiments

[0043] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0044] First Embodiment

[0045] As Figure 1 shown, this embodiment discloses a human motion recognition method based on a wireless channel, including the following steps:

[0046] Step 1: Obtain the beam domain channel state information under the condition of no person and the beam domain channel state information under different human motions respectively;

[0047] Step 2: Obtain the beam domain residual channel data based on the beam domain channel state information under the condition of no person and the beam domain channel state information under different human motions;

[0048] Step 3: Extract the beam numbers and instantaneous Doppler frequencies corresponding to human motions according to the residual channel data;

[0049] Step 4: Input the beam numbers and instantaneous Doppler frequencies corresponding to the human body movements obtained above into the trained classifier for recognition, and output the human body movement recognition results.

[0050] In this embodiment, by utilizing the change of the beam-domain wireless channel over time, the instantaneous Doppler frequency caused by human body movements and the beam numbers corresponding to the human body positions are extracted from the beam-domain wireless channel residual data. Using the pre-trained classifier, actions such as the human body being stationary, walking, or falling are detected.

[0051] As Figure 2 shown, before step 1 of this embodiment, it further includes: setting system parameters, including the carrier frequency, the horizontal distance between the transceiver ends, the height of the transceiver ends, and the antenna configuration. Among them, the transmitting end is a rectangular large-scale antenna array, which can be installed on the wall, and there are a total of P = P h ×P v antennas, where P v and P h are the number of rows and columns of the antennas in the transmitting antenna array respectively. The receiving end is a single antenna.

[0052] In step 1 of this embodiment, taking the acquisition of the beam-domain channel state information under the condition of no one, that is, the reference channel state, as an example for illustration, the acquisition of the parameter channel state information includes:

[0053] Performing beam-domain horizontal and vertical sampling on the three-dimensional space under the condition of no one, which is used to form P v ×P h orthogonal beams in the entire three-dimensional space. Each beam corresponds to the scatterers in the transmission environment. The beam horizontal sampling angle and the vertical sampling angle are obtained by the following formulas:

[0054]

[0055]

[0056] Among them, and are the spatial frequencies in the horizontal and vertical directions respectively. i = 1,..., P h , j = 1,..., P v , P v and P h are the number of rows and columns of the antennas in the transmitting antenna array respectively. The sampling angle resolution increases with the increase in the number of antennas.

[0057] Obtain the transmitting end beamforming matrix. The calculation formula of the transmitting end beamforming matrix is:

[0058]

[0059]

[0060]

[0061] wherein, denotes the Kronecker product, and the vectors a and b are respectively the one-dimensional antenna array steering vectors in the horizontal and vertical directions of the transmitting array, which are defined by the following formulas:

[0062]

[0063]

[0064] wherein, (·) T denotes the vector transpose, are respectively the horizontal direction spatial frequency and the vertical direction spatial frequency corresponding to the nth path, λ is the carrier wavelength, d h and d v are respectively the horizontal direction and vertical direction antenna spacings between adjacent antenna elements in the transmitting antenna array, and are respectively the elevation departure angle and the horizontal departure angle of the nth path in the transmission environment.

[0065] In this embodiment, the calculation formula of the beam domain reference channel state information is:

[0066]

[0067] wherein, (·) * denotes the conjugate, H Ref (t,f) is a 1×P dimensional reference channel matrix at time t and frequency f, and can be expressed as:

[0068]

[0069] wherein, N is the number of multipaths in the channel, β n and τ n are respectively the amplitude and delay of the nth path. Φ n is the initial phase of the nth path, which follows a uniform distribution within [0, 2π].

[0070] As shown in Fig. 3(a), each circle in the figure represents a transmission path, and its size represents the relative power of the multipath. It can be seen that, compared with the horizontal direction, the distribution of the multipath in the vertical direction is more concentrated. Fig. 3(b) is the reference channel beam distribution corresponding to the multipath in Fig. 3(a), and each beam corresponds to a unique serial number.

[0071] Obtain the beam domain channel state information, i.e., the current channel state information, under different human actions. The specific acquisition formula is:

[0072]

[0073] Among them, H(t, f) is a 1×P-dimensional wireless channel matrix in the presence of people, which can be expressed as follows:

[0074]

[0075] Among them, ν n is the Doppler frequency of the nth multipath. Assuming that the transmission environment is stationary, the multipath labels caused by human reflection are When a person walks or falls,[[]] For other multipaths, their Doppler frequency is 0.[[]]

[0076] In step 2 of this embodiment, the acquired current channel state information is subtracted from the reference channel state information to obtain residual channel data. The beam-domain residual channel is calculated by the following formula:

[0077]

[0078] In step 3 of this embodiment, channel features are extracted from the obtained residual channel data, specifically including taking the derivative of the residual channel data with respect to time to obtain the instantaneous Doppler frequency of the human body under different actions, and extracting the beam horizontal sequence number and beam vertical sequence number corresponding to the beam with the strongest power in the residual channel data.[[]]

[0079] Figure 4(a) shows the angular distribution in three-dimensional space of the multipaths caused only by human reflection, excluding the influence of scatterers in space. Figure 4(b) shows the beam-domain residual channel corresponding to Figure 4(a). This set of beams corresponds to the direction of the human body in the scenario, which is caused by human reflection. The is reconstructed from 1×P dimension to P v ×P h dimensional matrix. Let be the element with the largest absolute value in Let this element be located in the row column. The Doppler frequency caused by human movement can be estimated as where d(·) represents differentiation, is the phase at time t.[[]]

[0080] In step 4 of this embodiment, the beam horizontal sequence number obtained in step 3 beam vertical sequence number and the instantaneous Doppler frequency Perform normalization processing to map its value to the interval [0, 1]. Compose the normalized data into a feature vector and import it into a machine learning classifier, such as a Naive Bayes classifier, a Random Forest classifier, or a K-Nearest Neighbor classifier for training. Each group of data corresponds to one of the three actions: "fall", "static", and "walk".

[0081] During recognition, the obtained residual channel data from the instantaneous channel data is processed and then input into the trained machine learning classifier to obtain the corresponding action category.

[0082] Figure 5 The displayed confusion matrix is generated using the Naive Bayes classifier in this embodiment, where "1", "2", and "3" represent the three actions of "fall", "static", and "walk" respectively. The classification accuracy is approximately 96%, indicating that this method can effectively identify different human actions.

[0083] In summary, the present invention combines the wireless signal propagation characteristics and beamforming technology to form orthogonal beams covering the entire three-dimensional space. The beam aligned with the human body is obtained by the difference between the beam domain reference channel state information and the current channel state information. The beam number and instantaneous Doppler frequency are extracted from the residual channel data to form a feature vector. Then, the machine learning classifier is trained to obtain a human action classifier, which can effectively classify various human actions. The present invention makes up for the deficiencies of current human action recognition methods and can provide technical support for fields such as healthcare, virtual reality, and sports competitions.

[0084] Embodiment 2

[0085] This embodiment provides a human action recognition system based on a wireless channel, including:

[0086] Acquisition module: respectively acquire the beam domain channel state information under unoccupied conditions and the beam domain channel state information under different human actions;

[0087] Residual channel calculation module: obtain the beam domain residual channel data based on the beam domain channel state information under unoccupied conditions and the beam domain channel state information under different human actions;

[0088] Data extraction module: extract the beam number and instantaneous Doppler frequency corresponding to the human action according to the residual channel data;

[0089] Action recognition module: input the beam number and instantaneous Doppler frequency corresponding to the human action obtained above into the trained classifier for recognition and output the human action recognition result.

[0090] Embodiment 3

[0091] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0092] Embodiment 4

[0093] The purpose of this embodiment is to provide a computer-readable storage medium.

[0094] A computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above method are executed.

[0095] In the devices of the above Embodiments 2, 3, and 4, the steps involved correspond to those of Method Embodiment 1. For specific implementation manners, reference may be made to the relevant description part of Embodiment 1. The term "computer-readable storage medium" should be understood to include a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0096] Those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0097] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. A method for human action recognition based on a wireless channel, characterized in that, It includes the following steps: Obtain the beam domain channel state information under unmanned conditions and the beam domain channel state information under different human actions respectively; Obtain the beam domain channel state information, specifically including: performing beam domain horizontal and vertical sampling in the three-dimensional space under unmanned conditions or different human actions; the beam domain horizontal sampling angle is determined by the number of columns of antennas in the transmitting antenna array and the horizontal direction spatial frequency; the beam domain vertical sampling angle is determined by the number of rows of antennas in the transmitting antenna array and the vertical direction spatial frequency; Obtain the transmitting end beamforming matrix, which is used to form P in the entire three-dimensional space v ×P h orthogonal beams, the specific calculation formula is: Among them, denotes the Kronecker product, and the vectors b and a are respectively the one-dimensional antenna array steering vectors in the vertical and horizontal directions of the transmitting array, and are the spatial frequencies in the horizontal and vertical directions, and P v and P h are respectively the number of rows and columns of the antennas in the transmitting antenna array; The calculation formula for the beam domain channel state information under unmanned conditions is: Among them, (·) * denotes conjugation, and H Ref (t, f) is a 1×P dimensional spatial domain channel matrix at time t and frequency f, where P is the number of antennas; H Ref (t,f) is expressed as: where N is the number of multipaths in the channel, β n and τ n are the amplitude and delay of the n-th path respectively, and Φ n is the initial phase of the n-th path, which follows a uniform distribution within [0, 2π]; The calculation formula for the beam domain channel state information under different human actions is: Wherein, H(t,f) is a 1×P-dimensional wireless channel matrix in the presence of people; H(t,f) is expressed as: where ν n is the Doppler frequency of the n-th multipath. Assuming the transmission environment is stationary, the multipath label caused by human reflection is When a person walks or falls,[[]] For other multipaths, their Doppler frequency is 0; The calculation formula for the beam domain residual channel is: Obtain the beam domain residual channel data based on the beam domain channel state information under unmanned conditions and the beam domain channel state information under different human actions; Extract the beam numbers and instantaneous Doppler frequencies corresponding to human actions according to the residual channel data; Input the obtained beam numbers and instantaneous Doppler frequencies corresponding to human actions into the trained classifier for recognition, and output the human action recognition result.

2. The human motion recognition method based on a wireless channel according to claim 1, characterized in that Before obtaining the beam domain channel state information, it also includes setting system parameters, including carrier frequency, horizontal distance between the transceiver, heights of the transmitting and receiving antennas, and antenna configuration.

3. The human motion recognition method based on a wireless channel according to claim 1, wherein Extracting the beam numbers and instantaneous Doppler frequencies corresponding to human actions according to the residual channel data specifically includes: taking the derivative of the residual channel data with respect to time to obtain the instantaneous Doppler frequencies under different actions, and extracting the beam horizontal number and beam vertical number corresponding to the beam with the strongest power in the residual channel data.

4. A human motion recognition system based on a wireless channel, characterized in that, It includes: Obtaining module: Obtain the beam domain channel state information under unmanned conditions and the beam domain channel state information under different human actions respectively; Obtain the beam domain channel state information, specifically including: performing beam domain horizontal and vertical sampling in the three-dimensional space under unmanned conditions or different human actions; the beam domain horizontal sampling angle is determined by the number of columns of antennas in the transmitting antenna array and the horizontal direction spatial frequency; the beam domain vertical sampling angle is determined by the number of rows of antennas in the transmitting antenna array and the vertical direction spatial frequency; Obtain the transmit beamforming matrix for forming P orthogonal beams in the entire three-dimensional space. The specific calculation formula is as follows: v ×P h orthogonal beams, and the specific calculation formula is: wherein, vectors b and a are respectively one-dimensional antenna array steering vectors in the vertical and horizontal directions of the transmitting array, and are the spatial frequencies in the horizontal and vertical directions, and P v and P h are respectively the number of rows and columns of the antennas in the transmitting antenna array; The calculation formula for the beam domain channel state information under unmanned conditions is: Among them, (·) * denotes conjugation, and H Ref (t,f) is a 1×P dimensional spatial domain channel matrix at time t and frequency f, where P is the number of antennas; H Ref (t,f) is expressed as: where N is the number of multipaths in the channel, β n and τ n are the amplitude and delay of the n-th path respectively, and Φ n is the initial phase of the n-th path, which follows a uniform distribution within [0, 2π]; The calculation formula for the beam domain channel state information under different human actions is: Wherein, H(t,f) is a 1×P-dimensional wireless channel matrix in the presence of people; H(t,f) is expressed as: where ν n is the Doppler frequency of the n-th multipath. Assuming the transmission environment is stationary, the multipath label caused by human reflection is When a person walks or falls, For other multipaths, their Doppler frequency is 0; The calculation formula for the beam domain residual channel is: Residual channel calculation module: Obtain the beam domain residual channel data based on the beam domain channel state information under unmanned conditions and the beam domain channel state information under different human actions; Data extraction module: Extract the beam numbers and instantaneous Doppler frequencies corresponding to human actions according to the residual channel data; Action recognition module: Input the obtained beam numbers and instantaneous Doppler frequencies corresponding to human actions into the trained classifier for recognition, and output the human action recognition result.

5. The human motion recognition system based on a wireless channel according to claim 4, wherein In the data extraction module, beam numbers and instantaneous Doppler frequencies corresponding to human actions are extracted from the residual channel data, specifically including: taking the derivative of the residual channel data with respect to time to obtain the instantaneous Doppler frequencies under different actions, and extracting the beam horizontal number and beam vertical number corresponding to the beam with the strongest power in the residual channel data.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, the steps in a human action recognition method based on a wireless channel as described in any one of claims 1-3 are implemented.

7. A processing device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the steps in a human action recognition method based on a wireless channel as described in any one of claims 1-3 are implemented.

Images