Human body motion monitoring method, electronic device, and storage medium
By collecting and processing short-distance communication data and extracting human motion characteristics, the problems of privacy exposure and easy equipment fall off in video image algorithms and wearable device technology are solved, and more accurate sleep monitoring is achieved.
Patent Information
- Application Number
- PCT/CN2024/090851
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-08
- Filing Date
- 2024-04-30
- Publication Date
- 2024-11-14
AI Technical Summary
Existing sleep monitoring methods, especially video image algorithms and wearable device technologies, have problems such as privacy exposure and easy equipment fall off, resulting in reduced accuracy of sleep monitoring data.
By collecting short-distance communication data in the perception area, dividing it into multiple amplitude data sequences, and extracting the target data sequence according to the amplitude fluctuation degree, monitoring human movements based on these data, realizing action recognition and improving recognition accuracy.
Without exposing user privacy, the accuracy of recognition of human movements is improved, data inaccuracy problems caused by device shedding is avoided, and more reliable sleep monitoring data is provided.
Smart Images

Figure CN2024090851_14112024_PF_FP_ABST
Abstract
Description
Human motion monitoring method, electronic device and storage medium
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to Chinese patent application No. 202310512404.8 filed on May 8, 2023, the contents of which are incorporated herein by reference in their entirety. Technical Field
[0003] The present application relates to the technical field of motion detection. Background Art
[0004] With the development of communication and artificial intelligence technologies, sleep monitoring has gradually become an important monitoring method in healthcare, elderly health monitoring and other fields. The evaluation and monitoring data of users' sleep quality can reflect their health status.
[0005] Video image algorithms or wearable device technology are usually used to monitor users' sleep quality. However, when using video image algorithms, the camera equipment deployed will expose users' privacy information and pose a security risk. When users wear wearable devices to sleep, there will be problems such as the device easily falling off, which reduces the accuracy of sleep monitoring data.
[0006] Summary of the Invention
[0007] The present application provides a human motion monitoring method, electronic device and storage medium.
[0008] An embodiment of the present application provides a method for monitoring human motion, comprising: collecting short-distance communication data within a sensing area; dividing the short-distance communication data into multiple amplitude data sequences according to the data amplitude; extracting multiple target data sequences from the multiple amplitude data sequences according to the degree of amplitude fluctuation; and monitoring human motion based on the multiple target data sequences.
[0009] An embodiment of the present application provides an electronic device, comprising: one or more processors; a memory on which one or more programs are stored. When the one or more programs are executed by one or more processors, the one or more processors implement any one of the human motion monitoring methods in the embodiments of the present application.
[0010] An embodiment of the present application provides a readable storage medium, which stores a computer program. When the computer program is executed by a processor, any one of the human motion monitoring methods in the embodiments of the present application is implemented. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] FIG1 is a flow chart showing a method for monitoring human motion according to an embodiment of the present application.
[0012] FIG2 is a schematic diagram showing a sensing area according to an embodiment of the present application.
[0013] FIG3 is a schematic diagram of training a residual neural network using a preset training set according to an embodiment of the present application.
[0014] FIG4 is a flow chart of a data collection method during training of a residual neural network provided in an embodiment of the present application.
[0015] FIG5 is a schematic diagram of a process for training a residual neural network according to an embodiment of the present application.
[0016] FIG6 is a block diagram of the motion recognition system provided in an embodiment of the present application.
[0017] FIG7 is a schematic diagram of a process for recognizing human body movements according to an embodiment of the present application.
[0018] FIG8 shows a block diagram of the motion recognition device provided in an embodiment of the present application.
[0019] FIG9 shows a structural diagram of an exemplary hardware architecture of a computing device capable of implementing the human motion monitoring method and apparatus according to an embodiment of the present application. DETAILED DESCRIPTION
[0020] To make the purpose, technical solutions and advantages of this application more clear, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of this application can be combined with each other in any way.
[0021] Typically, a user's sleep quality is monitored using video image algorithms or wearable device technology. When using a video image algorithm, cameras are placed around the human body to capture the movements of the human body during sleep; each captured image is then processed. The positions of the main joints of the human body are identified, and whether the main joints of the human body have changed position in multiple frames of images is determined. If the video image algorithm captures this position change, it is used to identify the human body's turning over. However, during the implementation of the above method, it is necessary to deploy equipment for capturing images, which will expose the user's privacy information and reduce the user's safety.
[0022] When using wearable devices to monitor a user's sleep quality, since wearable devices contain multiple modules such as accelerometers, gyroscopes, and thin-film resistors, if a user turns over while wearing the wearable device to sleep, the wearable device will capture the data transmitted by these sensors and then identify the human body's movements based on this data. However, when a user wears a wearable device to sleep, there is a risk that the device will fall off, which not only interferes with the user's sleep but also reduces the accuracy of the captured sleep data.
[0023] The present application provides a human motion monitoring method, electronic device and storage medium to at least solve the above problems.
[0024] FIG1 is a flow chart illustrating a method for monitoring human motion according to an embodiment of the present application. The method can be applied to a motion recognition device. As shown in FIG1 , the method for monitoring human motion according to an embodiment of the present application includes, but is not limited to, steps S101 to S104.
[0025] In step S101, short-range communication data within a sensing area is collected.
[0026] The sensing area may be an area for sensing human motions that is constructed based on short-range communication data exchanged between a signal transmitting device and a signal receiving device.
[0027] In some exemplary embodiments, the signal transmitting device can be a terminal (e.g., a smartphone, tablet, etc.); the signal receiving device can be a network access device (e.g., a router, wireless network card, etc.). Through short-range communication data exchanged between the terminal and the network access device, accurate human movement within the sensing area can be detected. In one exemplary embodiment, the size of the sensing area can also be set based on the location and distance between the signal transmitting and receiving devices, or based on user needs.
[0028] In some exemplary embodiments, the signal transmitting device (or, signal receiving device) can be any one of the following devices: a wireless network card, a router, customer premises equipment (CPE), and other devices containing a short-range communication chip; the signal receiving device can be a terminal (such as a smart phone, tablet computer, etc.) or other devices containing a short-range communication chip.
[0029] In step S102, the short-distance communication data is divided into a plurality of amplitude data sequences according to the data amplitude.
[0030] In step S103, a plurality of target data sequences are extracted from the plurality of amplitude data sequences according to the amplitude fluctuation degree.
[0031] The amplitude fluctuation degree is used to characterize the degree of change between each data in the amplitude data sequence, and the amplitude fluctuation degree can be characterized by the degree of difference between different data.
[0032] In step S104 , human body motion is monitored based on the multiple target data sequences.
[0033] In this embodiment, by collecting short-distance communication data within the sensing area, data within the sensing area can be obtained without exposing the user's privacy; the short-distance communication data is divided into multiple amplitude data sequences according to the data amplitude to obtain multiple amplitude data sequences that can reflect the amplitude information; multiple target data sequences are extracted from the multiple amplitude data sequences according to the degree of amplitude fluctuation, so that the obtained multiple target data sequences can better reflect the characteristics of human body movements; and then human body movements are monitored based on the multiple target data sequences to improve the accuracy of human body movement recognition.
[0034] It should be noted that before the signal transmitting device and the signal receiving device perform wireless perception, the signal transmitting device needs to obtain certain attribute parameters of the signal receiving device in advance (such as the network address or media access control address (MAC) of the signal receiving device, etc.). After obtaining the above attribute parameters, the signal transmitting device and the signal receiving device can collect short-range communication data (such as channel state information (CSI) data, etc.).
[0035] In some exemplary embodiments, the human body's movements may include: at least one of turning over, leg stretching, hand stretching, head movement, foot movement, and hand movement during sleep, or other movements involving any movement of the human body.
[0036] For example, different levels can be set according to the amplitude of the movement. For example, the level of movements with larger amplitudes, such as turning over, leg stretching, and hand stretching, can be set to level one; and the level of movements with smaller amplitudes, such as head movement, foot movement, and hand movement, can be set to level two, so as to facilitate accurate identification of movements of different levels.
[0037] Furthermore, some abnormal movements can be set, such as kicking, leg shaking, hand shaking, etc. In some exemplary embodiments, when only turning over during sleep is monitored, other movements can be classified as abnormal. The above human movements are only examples, and specific settings can be made according to actual needs. Other movements not described are also within the scope of protection of this application and will not be repeated here.
[0038] In some exemplary embodiments, the sensing area may be a rectangular area, wherein the length of the rectangular area is the distance between the signal transmitting device and the signal receiving device, and the width of the rectangular area is a length determined based on the wavelength of the collected short-range communication signal, a preset adjustment coefficient, and the distance between the signal transmitting device and the signal receiving device.
[0039] It should be noted that the signal transmitting device and the signal receiving device can be on a horizontal plane at the same height (for example, the height difference between the position of the signal transmitting device and the position of the signal receiving device can be within a preset range (such as 5 cm, 10 cm, etc.)), and the difference in height between the horizontal plane and the horizontal plane of the human body (or the horizontal plane on which the human body lies) is within a preset difference range (for example, the preset difference range is within 20 cm, etc.) to improve the accuracy of recognition.
[0040] Figure 2 shows a schematic diagram of a sensing area provided by an embodiment of the present application. As shown in Figure 2, a sensing area 220 is established between a signal transmitting device 231 and a signal receiving device 232 based on a short-range communication protocol. The human body is located in a central area 211 of an entity 210 (e.g., a bed).
[0041] According to the device arrangement shown in Figure 2 , the distance between signal transmitting device 231 and signal receiving device 232 is H, and the distance W / 2 is extended to both sides along the line connecting signal transmitting device 231 and signal receiving device 232. The width W of the rectangular area can be expressed using formula (1), thereby forming a rectangular sensing area 220.
[0042] Wherein, H represents the connection distance between the signal transmitting device 231 and the signal receiving device 232, that is, the length of the rectangular area; W represents the width of the rectangular area; λ represents the wavelength of the short-range communication signal. Generally speaking, for a 2.4 GHz short-range communication signal, λ is 13 cm; and for a 5 GHz short-range communication signal, λ is 6 cm.
[0043] From a top-down perspective, the line connecting the signal transmitting device 231 and the signal receiving device 232 passes through the central area 211. The central area 211 is formed by shrinking the perimeter of the entity 210 inward by half its side length. That is, the central area 211 has the same center point as the entity 210, and the side length of the central area 211 is half the side length of the entity 210.
[0044] It should be noted that the above-mentioned rectangular area is only used as an example to explain the scope of the perception area 220, so as to monitor the sleeping movements of the human body in the perception area 220 and improve the accuracy of movement recognition. Other unspecified perception areas are also within the scope of protection of this application and will not be repeated here.
[0045] In some exemplary embodiments, a signal transmitting device and a signal receiving device perform short-range communication via at least one pair of transceiver antennas, and each pair of transceiver antennas carries multiple subcarriers.
[0046] The short-range communication may be at least one of WiFi, ZigBee, Bluetooth, or other short-range communications. If the signal transmitting device (or the signal receiving device) includes A transmitting antennas and the signal receiving device (or the signal transmitting device) includes B receiving antennas, then the number of pairs of the transmitting and receiving antennas is A*B, where A and B are both integers greater than or equal to 1.
[0047] In an exemplary embodiment, dividing the short-range communication data into multiple amplitude data sequences according to the data amplitude in step S102 may include: collecting channel state data transmitted on multiple subcarriers carried by each pair of transmitting and receiving antennas based on the number of sampling packets; extracting the amplitude data from the channel state data transmitted on the multiple subcarriers respectively, and constructing multiple amplitude data sequences based on the amplitude data in the channel state data transmitted on the multiple subcarriers.
[0048] The channel state data may be CSI data or channel frequency response (CFR) data used to identify human behavior.
[0049] CSI can be used to characterize the channel properties of a communication link. It describes the attenuation factor of a communication signal along each transmission path, such as the value of each element in the channel gain matrix. CFR data, on the other hand, refers to the response data generated by a signal receiving device when a pulse signal is sent from a signal transmitting device. Due to factors such as multipath delay spread and Doppler shift, signal receiving devices at different locations will generate different CFR data. CFR data can be used to describe response data in multipath propagation, such as phase-frequency response data and / or amplitude-frequency response data.
[0050] In an exemplary embodiment, the number of amplitude data sequences is the same as the number of subcarriers, the number of sampling packets is a number determined based on a preset sampling duration and a preset sampling frequency, and each pair of transceiver antennas includes a transmitting antenna on a signal transmitting device and a receiving antenna on a signal receiving device.
[0051] For example, if the preset sampling frequency is set to the frequency f of reading the CSI signal, and the preset sampling time T is the time length of a set of sleep turning monitoring sample data (such as CSI signal data and its corresponding label information, etc.), then the number of sampling packets = T / f.
[0052] It should be noted that by collecting channel state data transmitted on multiple subcarriers based on the number of sampling packets, richer data can be obtained; the amplitude data in the channel state data transmitted on multiple subcarriers are extracted separately, and based on the amplitude data in the channel state data transmitted on multiple subcarriers, multiple amplitude data sequences are constructed to facilitate the use of multiple amplitude data sequences to accurately identify the movements of the human body in the perception area.
[0053] In some exemplary embodiments, step S103 extracts multiple target data sequences from multiple amplitude data sequences based on the degree of amplitude fluctuation, including: filtering outlier data sequences in the multiple amplitude data sequences based on a preset filtering range to obtain multiple first filtered data sequences; filtering out noise sequences in the multiple first filtered data sequences based on a preset filtering method to obtain multiple second filtered data sequences; and performing differential processing on the multiple second filtered data sequences respectively to obtain multiple target data sequences.
[0054] Among them, differential processing can reflect the differences between each data in the second filtered data sequence, and determine whether there is a change in the action through the difference, so that the multiple target data sequences obtained can reflect the change information of the action and improve the accuracy of action recognition.
[0055] In some exemplary embodiments, outlier data sequences in multiple amplitude data sequences are filtered out based on a preset filtering range to obtain multiple first filtered data sequences, including: filtering multiple first data in each amplitude data sequence based on the preset filtering range; determining that the first data is outlier data in the amplitude data sequence when it is determined that the first data is not within the preset filtering range; and clearing outlier data sequences including at least one outlier data in each amplitude data sequence to obtain multiple first filtered data sequences.
[0056] Among them, outlier data is used to characterize discrete data that can reflect certain data patterns in the amplitude data sequence. For example, there are 5 data in the amplitude data sequence, and 4 of them can reflect an action feature (that is, these 4 data are within the preset filtering range), but the remaining 5th data has a completely different action feature from the other 4 data (that is, the 5th data is not within the preset filtering range). In this case, it can be determined that the 5th data is an outlier.
[0057] By removing outlier data sequences including at least one discrete data in each amplitude data sequence, the obtained multiple first filtered data sequences can reflect a unified action feature, thereby improving the accuracy of action recognition.
[0058] In some exemplary embodiments, before filtering multiple first data in each amplitude data sequence based on a preset filtering range, the human motion monitoring method also includes: constructing a window sequence based on a preset window coefficient and a preset number of adjacent data in the extracted amplitude data sequence; determining the mean corresponding to the window sequence and the standard deviation corresponding to the window sequence based on the individual data in the window sequence and the number of data in the window sequence; determining the preset filtering range based on the mean corresponding to the window sequence, the standard deviation corresponding to the window sequence and the preset coefficient.
[0059] For example, if the window coefficient is set to x, a preset number of adjacent data in the amplitude data sequence are randomly selected (for example, the i-th data and the x data to the left and right of the i-th data are extracted, thereby obtaining a window sequence including 2x+1 data, where i is an integer greater than or equal to 1 and less than m, and m represents the maximum number of sequences included in the amplitude data sequence). In this window sequence, the mean μ of the 2x+1 data and the standard deviation σ of the 2x+1 data are calculated; thus, the preset filtering range can be determined as [μ-α*σ,μ+α*σ], where α represents the preset coefficient (for example, the preset coefficient can be set to 2, 3, etc.).
[0060] By determining the preset filtering range in the above manner, the boundary value of the data to be filtered can be clearly identified, which facilitates the screening of multiple first data (ie, outlier data) in each amplitude data sequence, thereby improving the accuracy of data screening.
[0061] In some exemplary embodiments, noise sequences in multiple first filtered data sequences are filtered out based on a preset filtering method to obtain multiple second filtered data sequences, including: performing wavelet decomposition on each first filtered data sequence based on a preset wavelet basis function to obtain a multi-layer decomposition sequence corresponding to each first filtered data sequence; screening sub-data sequences in the multi-layer decomposition sequence corresponding to each first filtered data sequence based on a preset coefficient threshold to obtain multiple screened data sequences; and reconstructing the multiple screened data sequences to obtain multiple second filtered data sequences.
[0062] In some exemplary embodiments, the preset wavelet basis function may be a Haar function, a Daubechies function, a Biorthogonal function, or the like. Each layer of the decomposition sequence includes at least two sub-data sequences and wavelet coefficients corresponding to the sub-data sequences. In other words, each sub-data sequence corresponds to a wavelet coefficient, and each sub-data sequence can represent high-pass or low-pass filtering characteristics.
[0063] It should be noted that, in the process of performing wavelet decomposition on each first filtered data sequence based on a preset wavelet basis function, the larger the number of decomposition layers, the more obvious the different characteristics of the noise signal and the useful signal, and the more conducive to separating the noise signal from the useful signal. However, if the number of decomposition layers is set to a larger number, the degree of distortion of the multiple second filtered data sequences obtained by reconstructing the multiple filtered data sequences will be greater, which will affect the effect of removing the noise signal. Therefore, it is necessary to select an appropriate number of decomposition layers. For example, the number of decomposition layers is determined to be N, where N is a number determined based on the frequency band range and sampling frequency of the wavelet decomposition.
[0064] For example, if the N-layer decomposition sequence corresponding to each first filtered data sequence is set, the size of each frequency band is Where Fs represents the sampling frequency corresponding to the first filtered data sequence to be screened. For example, if the sampling frequency corresponding to the first filtered data sequence is determined to be 1000 Hz, the sampling theorem shows that the maximum frequency of the first filtered data sequence pair is 500 Hz; therefore, N can be set to 3.
[0065] Among them, the frequency band of the first-layer decomposition sequence is 250~500hz, and the approximate frequency band is less than 250hz; the frequency band of the second-layer decomposition sequence is 125~250hz, and the approximate frequency band is less than 125hz; the frequency band of the third-layer decomposition sequence is approximately 62.5~125hz, and the approximate frequency band is less than 62.5hz.
[0066] The above implementation method of performing wavelet decomposition on each first filtered data sequence based on a preset wavelet basis function to obtain a multi-layer decomposition sequence corresponding to each first filtered data sequence is only an example, and specific settings can be made according to actual conditions. Other undescribed implementation methods are also within the scope of protection of this application and will not be repeated here.
[0067] In some exemplary embodiments, when it is determined that the wavelet coefficient corresponding to the sub-data sequence is less than a preset coefficient threshold, the sub-data sequence corresponding to the wavelet coefficient is determined to be noise data, and the sub-data sequence (i.e., the noise sequence) whose wavelet coefficient is less than the preset coefficient threshold in each layer of the decomposition sequence is removed, thereby reducing noise interference.
[0068] Furthermore, the multiple filtered data sequences are reconstructed, for example, based on a preset arrangement order, the multiple filtered data sequences are sorted so that the obtained multiple second filtered data sequences can not only reflect the characteristics of the original first filtered data sequences, but also eliminate noise interference, thereby facilitating the subsequent acquisition of clean data sequences.
[0069] In some exemplary embodiments, each second filtered data sequence includes m second filtered data, where m is the number of sampling packets; performing differential processing on the multiple second filtered data sequences to obtain multiple target data sequences, including: for the multiple second data in each second filtered data sequence, determining the i-1th data in the target data sequence corresponding to the second filtered data sequence based on a preset differential coefficient and the difference between the i-th data and the i-1th data; repeating the above steps until i equals m, thereby obtaining multiple target data sequences. Wherein, i is an integer greater than or equal to 1 and less than or equal to m; the number of target data sequences is the same as the number of subcarriers carried by each pair of transceiver antennas, the number of data in the target data sequence is m-1, and each pair of transceiver antennas includes a transmitting antenna on a signal transmitting device and a receiving antenna on a signal receiving device.
[0070] For example, the default differential coefficient can be set to 1, and the m-2 , x m-1 ] is subjected to differential processing on the second filtered data sequence, multiple target data sequences [y0, y1, ..., y m-2 ]. Among them, y0=x1-x0, y1=x2-x1, ..., y m-2 =x m-1 -x m-2 .
[0071] The difference between the i-th data and the i-1-th data can reflect the change of the two data, thereby determining whether the human body's movements have changed at different times.
[0072] In some exemplary embodiments, determining a human body motion based on multiple target data sequences in step S104 includes: determining an input sequence of a target neural network based on the target data sequences corresponding to the multiple subcarriers; and inputting the input sequence of the target neural network into the target neural network for recognition to determine the human body motion. The target neural network is obtained by training a residual neural network using a preset training set, wherein the training set includes sample data sequences and motion annotation information corresponding to the sample data sequences.
[0073] For example, the target neural network can include multiple skip-connected residual blocks, which can alleviate the gradient vanishing problem caused by increasing the depth in the deep neural network, thereby improving the recognition accuracy of the data sequence.
[0074] In some exemplary embodiments, an input sequence of a target neural network is determined based on target data sequences corresponding to multiple subcarriers, including: respectively determining the standard deviations of the target data sequences corresponding to the multiple subcarriers; sorting the standard deviations of the target data sequences corresponding to the multiple subcarriers to obtain sorting results; based on the sorting results, extracting a preset number of target data sequences as the input sequence of the target neural network, and adjusting the data dimensions of the preset number of target data sequences so that the data dimensions of the preset number of target data sequences are the same as the data dimensions required by the target neural network; wherein the preset number is less than the maximum number of subcarriers.
[0075] It should be noted that in the actual communication process, the maximum number of subcarriers can be 256. Processing data on multiple carriers requires a long time for the target neural network. Therefore, the standard deviation of multiple data sequences to be trained transmitted on each subcarrier can be calculated separately, and then the standard deviations of the target data sequences corresponding to the multiple subcarriers are sorted to obtain the sorting results.
[0076] For example, the maximum number of subcarriers is 256, and the preset number can be set to 30, 35, etc. That is, based on the sorting result, the target data sequence corresponding to the first 30 (or first 35) subcarriers is selected as the input sequence of the target neural network. This reduces the processing time of the target neural network for data and speeds up the recognition of human body movements.
[0077] For example, if the data dimension of the n target data sequences is set to 4 dimensions (e.g., the number of transmitting antennas of the signal transmitting device, the number of receiving antennas of the signal receiving device, the number of subcarriers, and the number of sampling packets), and the data dimension required by the target neural network is 3 dimensions, the data dimension of the n target data sequences can be adjusted from 4 dimensions to 3 dimensions. For example, the data dimensions of the adjusted n target data sequences include: the number of sampling packets, the number of subcarriers, and the number of pairs of transmitting and receiving antennas. This makes the data dimension of the n target data sequences the same as the data dimension required by the target neural network, making it easier to use the target neural network to identify the n target data sequences and improve the recognition accuracy of the human body movements to be identified.
[0078] In some exemplary embodiments, before inputting the input sequence of the target neural network into the target neural network for recognition and determining the human body movement, the method further includes: training the residual neural network using a preset training set to obtain the target neural network. The training set includes sample data sequences and action annotation information corresponding to the sample data sequences.
[0079] For example, FIG3 is a schematic diagram of a residual neural network trained using a preset training set according to an embodiment of the present application. As shown in FIG3 , a sampling module 301, an amplitude extraction module 302, a filtering module 303, a differential processing module 304, an action labeling module 305, and a residual neural network 306 connected in communication can constitute a training system. The sampling module 301 is configured to collect multiple sample data sequences and use the multiple sample data sequences as data in the training set.
[0080] The amplitude extraction module 302 is configured to extract the amplitude of each sample data sequence in the input training set. Since the sampling value of the CSI signal is a complex number, the amplitude value of the CSI signal needs to be extracted as the value in the sample data sequence.
[0081] The filtering module 303 is configured to filter the data output by the amplitude extraction module 302 to remove noise and discrete sequences. For example, Hampel filtering can be used to remove outlier sequences in the data; wavelet filtering can be used to remove noise in the data.
[0082] Difference processing module 304 is configured to perform a differential operation on the data output by filtering module 303, thereby obtaining the CSI data changes caused by changes in human movement during sleep. The differential data sequence is then used as the input sequence for residual neural network 306, enabling residual neural network 306 to obtain a more accurate and reasonable training data sequence. Through the processing of difference processing module 304, the influence of differences in CSI signal values collected in different environments on the final training results can be suppressed.
[0083] The action label module 305 is configured to input the action annotation information corresponding to the output sample data sequence (such as at least one of the annotation information reflecting the turning over action of the human body during sleep, the annotation information of the leg stretching action, the annotation information of the hand stretching action, the annotation information of the head movement, the annotation information of the foot movement and the annotation information of the hand movement) into the residual neural network 306, so that the training results obtained by training the network model of the residual neural network 30 (such as the model parameters of the network model obtained by training, etc.) are more accurate and can better reflect the action representation information.
[0084] FIG4 is a flow chart of a data acquisition method for training a residual neural network according to an embodiment of the present application. During the data acquisition process, a signal transmitting device and a signal receiving device need to establish a short-range communication connection (e.g., a WiFi connection) throughout the entire process, and neither device can enter a dormant state. There can be one or more signal receiving devices.
[0085] The data collection method includes but is not limited to the following steps S401 to S405.
[0086] In step S401, sampling parameters are set.
[0087] The sampling parameters may include at least one of the number of sampling packets, sampling duration, sampling interval duration, sampling frequency, and the network address of the signal receiving device. The number of sampling packets is determined based on a preset sampling duration and a transmission frequency of the channel state data.
[0088] For example, if the preset sampling frequency is set to f, and the preset sampling duration T is the time length of a set of sleep turning monitoring sample data (such as CSI signal data and its corresponding label information, etc.), then the number of sampling packets = T / f.
[0089] In step S402, channel state data collection begins.
[0090] For example, the channel state data may be CSI data and / or CFR data, etc.
[0091] In step S403 , the sample human body changes its motion.
[0092] The sample human body may perform any one or more of the following actions: turning over, leg stretching, hand stretching, head movement, foot movement, and hand movement.
[0093] If the sample human body is performing a turning action, the action annotation information corresponding to the data collected during the sample human body turning action is the turning action annotation information, and the sample human body in the sensing area performs at least one turning action.
[0094] Similarly, if the sample human body is performing a leg stretching action, the action annotation information corresponding to the data collected during the sample human body performing the leg stretching action is the leg stretching action annotation information, and the sample human body within the perception area performs the leg stretching action at least once.
[0095] For example, the action annotation information can be represented using a one-hot encoding method (also known as a one-bit encoding method), in which a K-bit state register is used to encode K states, each state has its own independent register bit, and at any time, only one of the bits is valid, and K is an integer greater than or equal to 1. For example, the labels [0,0,0,0,0,1] are set to represent a turning action, [0,0,0,0,1,0] are set to represent a leg stretching action, [0,0,0,1,0,0] are set to represent a hand stretching action, [0,0,1,0,0,0] are set to represent a head movement, [0,1,0,0,0,0] are set to represent a foot movement, and [1,0,0,0,0,0] are set to represent a hand movement.
[0096] In step S404, it is determined whether the collection of the sampled data is completed.
[0097] If it is determined that the collection of the sampled data is completed, step S405 is executed; otherwise, the process returns to step S402 to continue collecting data.
[0098] In step S405, the collected data is output.
[0099] In order to ensure that the actions performed by various groups during sleep can be correctly monitored and identified, the action data of various major groups (such as teenagers, adult males, adult females, the elderly, etc.) needs to be collected during the data collection stage.
[0100] After it is determined that the collection of the sampled data is completed, the collected data may be output to a specific processing device (eg, a computer, etc.).
[0101] It should be noted that during the data collection process, action labels need to be made for the collected data. For example, if the action of the sample human body corresponding to the collected data sequence is a turning action, then the action labeling information corresponding to the turning action needs to be added to the data sequence; similarly, if the action of the sample human body corresponding to the collected data sequence is a leg stretching action, then the action labeling information corresponding to the leg stretching action needs to be added to the data sequence, and so on. This allows the residual neural network to be trained based on different action labeling information and its corresponding data sequences, obtaining model parameters corresponding to different actions and improving the recognition accuracy of different actions.
[0102] Figure 5 is a schematic diagram of a process for training a residual neural network according to an embodiment of the present application. As shown in Figure 5 , the method for training a residual neural network includes but is not limited to the following steps S501 to S506 .
[0103] In step S501, the data file is parsed.
[0104] The data file includes data on multiple subcarriers. The maximum number of subcarriers can be 256. Processing data on multiple carriers requires a long time for the target neural network. Therefore, the standard deviation of multiple training data sequences transmitted on each subcarrier can be calculated separately. For example, the standard deviation can be calculated using formula (2). Then, the standard deviations of multiple training data sequences transmitted on multiple subcarriers are sorted.
[0105] in, represents the mean value of the data sequence; σ represents the standard deviation; y represents the data in the data sequence to be trained.
[0106] It should be noted that the standard deviation can characterize the degree of fluctuation of the data sequence to be trained; thus, subcarriers with more drastic fluctuations are selected as target subcarriers for processing.
[0107] For example, the calculated standard deviations of the multiple training data sequences transmitted on the 256 subcarriers are [σ0, σ1, ..., σ 255 ], where, σ0>σ1>σ2…>σ 255 , the subcarrier data corresponding to the first 30 standard deviations can be selected as training data.
[0108] In step S502, amplitude extraction is performed on the training data sequence to obtain multiple amplitude data sequences.
[0109] The dimensions of the parsed data may include: the number of transmitting antennas of the signal transmitting device, the number of receiving antennas of the signal receiving device, the number of subcarriers, and the number of sampling packets. The number of subcarriers is determined by the short-range communication protocol (e.g., WiFi communication protocol) used between the signal transmitting device and the signal receiving device. Furthermore, the parsed data is a complex number (e.g., represented by a + bi, where a and b are both real numbers).
[0110] For example, a signal transmitting device and a signal receiving device communicate over short distances via at least one pair of transceiver antennas, each pair of which carries multiple subcarriers. For each pair of transceiver antennas, the following operations are performed: channel state data transmitted on multiple subcarriers is collected based on the number of sampling packets; amplitude data from the channel state data transmitted on the multiple subcarriers is extracted, and multiple amplitude data sequences are constructed based on the amplitude data in the channel state data transmitted on the multiple subcarriers.
[0111] The amplitude value A of the amplitude data (a+bi) can be expressed as: Then the amplitude data sequence can be represented as {A0, A1, ..., A n}, n is an integer greater than or equal to 1.
[0112] In step S503, outlier data sequences are filtered out from the multiple amplitude data sequences based on the Hampel filtering method to obtain multiple first filtered data sequences.
[0113] First, a window sequence is constructed based on the window coefficient and a preset number of adjacent data in the extracted amplitude data sequence; the mean corresponding to the window sequence and the standard deviation corresponding to the window sequence are determined according to the individual data in the window sequence and the number of data in the window sequence; and the preset filtering range is determined according to the mean corresponding to the window sequence, the standard deviation corresponding to the window sequence and the preset coefficient.
[0114] For example, extract the i-th data in the amplitude data sequence and the x data to the left and right of the i-th data (a total of 2x+1 data) to form a window sequence, where i is an integer greater than or equal to 1 and less than m, m represents the maximum number of sequences included in the amplitude data sequence, and x represents the window coefficient (e.g., the window coefficient can be set to 2, 3, etc.). In this window sequence, calculate the mean μ of the 2x+1 data and the standard deviation σ of the 2x+1 data; and determine the preset filtering range as [μ-α*σ,μ+α*σ], where α represents the preset coefficient (e.g., the preset coefficient can be set to 2, 3, etc.).
[0115] Based on a preset filtering range of [μ-α*σ, μ+α*σ], multiple first data in each amplitude data sequence are screened separately. When it is determined that a certain first data is not within the preset filtering range, the first data is determined to be an outlier data in the amplitude data sequence; outlier data sequences including at least one discrete data in each amplitude data sequence are cleared separately, and then the mean μ is used to replace the above outlier data, thereby obtaining multiple first filtered data sequences.
[0116] In step S504, noise sequences in the plurality of first filtered data sequences are filtered out based on a wavelet filtering method to obtain a plurality of second filtered data sequences.
[0117] In which, according to a preset wavelet basis function (such as Haar function, Daubechies function, Biorthogonal function, etc.), each first filtered data sequence is subjected to wavelet decomposition to obtain a multi-layer decomposition sequence (such as a 3-layer decomposition sequence or a 5-layer decomposition sequence, etc.) corresponding to each first filtered data sequence; wherein each layer of decomposition sequence includes at least two sub-data sequences and wavelet coefficients corresponding to the sub-data sequences.
[0118] Based on a preset coefficient threshold (for example, setting the threshold to 0.04, etc.), the sub-data sequences in the multi-layer decomposition sequence corresponding to each first filtered data sequence are screened to obtain multiple filtered data sequences (for example, the sub-data sequences corresponding to the wavelet coefficients greater than the preset coefficient threshold are retained, and the sub-data sequences less than or equal to the preset coefficient threshold are deleted, thereby obtaining multiple filtered data sequences); the multiple filtered data sequences are reconstructed to obtain multiple second filtered data sequences.
[0119] In step S505, differential processing is performed on the plurality of second filtered data sequences respectively to obtain a plurality of data sequences to be trained.
[0120] For example, for the form [x0, x1, ..., x m-2 , x m-1], and perform differential processing using formula (3) to obtain multiple training data sequences [y0, y1, ..., y m-2 ].
[0121] Wherein, d represents a differential coefficient. For example, d may be a coefficient determined by normalization or calibration. The value of d may be set to 1, 0.3, etc.; m represents the number of sampling packets.
[0122] It should be noted that, since the data dimensions of the multiple data sequences to be trained are different from those of the input sequence of the target neural network, the data dimensions of the multiple data sequences to be trained need to be adjusted. For example, the data dimensions of the multiple data sequences to be trained are adjusted to [(m-1)*30*1] (30 is the number of data selected in step S501), so that the data dimensions of the adjusted multiple data sequences to be trained are the same as the data dimensions of the input sequence of the target neural network.
[0123] In step S506, the residual neural network is trained based on multiple data sequences to be trained to obtain a trained network model.
[0124] It should be noted that the data processing process is the same in the training process of the neural network model and the process of using the trained neural network model (i.e., the target neural network) to identify the human body, and both include data sampling, amplitude extraction, filtering processing, and differential processing.
[0125] FIG6 is a block diagram of a motion recognition system according to an embodiment of the present invention. As shown in FIG6 , the motion recognition system includes but is not limited to the following devices: a signal transmitting device 610 and a signal receiving device 620 .
[0126] The signal transmitting device 610 includes a short-range communication chip 611 , a processor chip 612 , a storage module 613 and an output module 614 , which are communicatively connected; the signal receiving device 620 includes a short-range communication chip 621 .
[0127] The sensing area 630 is constructed by communicating and interacting with the short-range communication chip 611 in the signal transmitting device 610 and the short-range communication chip 621 in the signal receiving device 620 .
[0128] The output module 614 is configured to output the perception results to a display screen or other interface to facilitate transmission of the perception results. The signal receiving device 620 can be a mobile phone or other device containing a short-range communication chip.
[0129] The signal transmitting device 610 (or the signal receiving device 620 ) can be any one of the following devices: a wireless network card, a router, a CPE, a mobile phone, and other devices containing a short-range communication chip.
[0130] When both the signal transmitting device 610 and the signal receiving device 620 communicate using short-range communication signals, their short-range communication mode can be set to AP-STA mode. Furthermore, the signal transmitting device and the signal receiving device must always establish a short-range communication connection and continuously transmit data. To achieve better motion recognition results, the short-range communication signal used can be a signal transmitted over a 5GHz frequency channel, such as a Wi-Fi signal.
[0131] Figure 7 is a schematic diagram of a process for recognizing human body movements according to an embodiment of the present application. The method for recognizing human body movements can be applied to a signal transmitting device 610, for example. As shown in Figure 7, the method includes but is not limited to the following steps S701 to S707.
[0132] In step S701, sampling parameters are set.
[0133] In step S702, channel state data collection begins.
[0134] In step S703 , the human body changes its motion.
[0135] For example, the human body changes its movements during sleep, such as turning over during sleep, leg stretching during sleep, hand stretching during sleep, head movement during sleep, foot movement during sleep, and hand movement during sleep.
[0136] In step S704 , it is determined whether the sampling period has arrived.
[0137] If it is determined that the sampling period has been reached, step S705 is executed; if it is determined that the sampling period has not been reached, step S702 is continued to be executed.
[0138] In step S705, the collected data is analyzed.
[0139] The dimensions of the parsed data may include: the number of transmitting antennas of the signal transmitting device, the number of receiving antennas of the signal receiving device, the number of subcarriers, and the number of sampling packets. The number of subcarriers is determined by the short-range communication protocol (e.g., WiFi communication protocol) used between the signal transmitting device and the signal receiving device. Furthermore, the parsed data is a complex number (e.g., the parsed data is represented by: a+bi, where a and b are both real numbers).
[0140] For example, a signal transmitting device and a signal receiving device communicate over short distances via at least one pair of transceiver antennas, each pair of which carries multiple subcarriers. For each pair of transceiver antennas, the following operations are performed: channel state data transmitted on multiple subcarriers is collected based on the number of sampling packets; amplitude data from the channel state data transmitted on the multiple subcarriers is extracted, and multiple amplitude data sequences are constructed based on the amplitude data in the channel state data transmitted on the multiple subcarriers.
[0141] The amplitude value A of the amplitude data (a+bi) can be expressed as: Then the amplitude data sequence can be represented as {A0, A1, ..., A n}, n is an integer greater than or equal to 1.
[0142] Furthermore, based on the Hampel filtering method, outlier data sequences are filtered out from the multiple amplitude data sequences to obtain multiple first filtered data sequences. First, a window sequence is constructed based on a window coefficient and a preset number of adjacent data in the extracted amplitude data sequence. Based on the individual data in the window sequence and the number of data in the window sequence, the mean value and the standard deviation corresponding to the window sequence are determined. Based on the mean value and the standard deviation corresponding to the window sequence and the preset coefficient, a preset filtering range is determined.
[0143] For example, the i-th data in the amplitude data sequence and the x data to the left and right of the i-th data are extracted respectively, thereby obtaining a window sequence including 2x+1 data, where i is an integer greater than or equal to 1 and less than m, m represents the maximum number of sequences included in the amplitude data sequence, and x represents the window coefficient (e.g., the window coefficient can be set to 2, 3, etc.). In this window sequence, the mean μ of the 2x+1 data and the standard deviation σ of the 2x+1 data are calculated; and the preset filtering range is determined to be [μ-α*σ,μ+α*σ], where α represents the preset coefficient (e.g., the preset coefficient can be set to 2, 3, etc.).
[0144] Based on a preset filtering range of [μ-α*σ, μ+α*σ], multiple first data in each amplitude data sequence are screened separately. When it is determined that a certain first data is not within the preset filtering range, the first data is determined to be an outlier data in the amplitude data sequence; outlier data sequences including at least one discrete data in each amplitude data sequence are cleared separately, and then the mean μ is used to replace the above outlier data, thereby obtaining multiple first filtered data sequences.
[0145] Then, based on a preset wavelet basis function (such as Haar function, Daubechies function, Biorthogonal function, etc.), each first filtered data sequence is subjected to wavelet decomposition to obtain a multi-layer decomposition sequence (such as a 3-layer decomposition sequence or a 5-layer decomposition sequence, etc.) corresponding to each first filtered data sequence; wherein each layer of decomposition sequence includes at least two sub-data sequences and wavelet coefficients corresponding to the sub-data sequences.
[0146] Based on a preset coefficient threshold (for example, setting the threshold to 0.04, etc.), the sub-data sequences in the multi-layer decomposition sequence corresponding to each first filtered data sequence are screened to obtain multiple filtered data sequences (for example, the sub-data sequences corresponding to the wavelet coefficients greater than the preset coefficient threshold are retained, and the sub-data sequences less than or equal to the preset coefficient threshold are deleted, thereby obtaining multiple filtered data sequences); the multiple filtered data sequences are reconstructed to obtain multiple second filtered data sequences.
[0147] Then, differential processing is performed on the multiple second filtered data sequences to obtain multiple target data sequences.
[0148] For example, for the form [x0, x1, ..., x n-2 , x n-1 ], and perform differential processing using the above formula (3) to obtain multiple target data sequences [y0, y1, ..., y n-2 ].
[0149] It should be noted that since the data dimensions of the multiple target data sequences are different from the input sequence of the target neural network, it is also necessary to adjust the data dimensions of the multiple target data sequences. For example, the data dimensions of the multiple target data sequences are adjusted to [(n-1)*30*1] so that the data dimensions of the adjusted multiple target data sequences are the same as the data dimensions of the input sequence of the target neural network. Where n represents the number of sampling packets.
[0150] In step S706, the adjusted multiple target data sequences are input into the target neural network for recognition to obtain recognition results.
[0151] Among them, in order to improve the data processing efficiency, in an exemplary embodiment, a 10-layer res-net network can be used as the target neural network. In actual use, the level of the target neural network can be set according to actual needs. It is not limited to the res-net network structure. Other unspecified network structures are also within the scope of protection of this application and will not be repeated here.
[0152] Among them, the recognition result can be a label representing the action type, for example, setting the label [0,0,0,0,0,1] to represent a turning action, [0,0,0,0,1,0] to represent a leg stretching action, [0,0,0,1,0,0] to represent a hand stretching action, [0,0,1,0,0,0] to represent a head movement, [0,1,0,0,0,0] to represent a foot movement, and [1,0,0,0,0,0] to represent a hand movement, etc.
[0153] Furthermore, the recognition result can be displayed on a display screen, or transmitted to other devices for use in upper-layer applications on other devices.
[0154] In step S707, it is determined whether the wireless sensing is completed.
[0155] If it is determined that the wireless sensing is completed, the process ends; if it is determined that the wireless sensing is not completed, step S702 is continued.
[0156] In this embodiment, based on a short-range communication protocol, a sensing area for sensing human movements is constructed between a signal transmitting device and a signal receiving device. Without exposing the user's privacy, the collected short-range communication data within the sensing area is processed to obtain multiple amplitude data sequences, making the acquired amplitude data sequences more complete and accurate. Furthermore, data preprocessing is performed on the multiple amplitude data sequences based on a differential method, so that the obtained multiple target data sequences can better reflect the characteristics of human movements and match the input sequence of the target neural network. The multiple target data sequences are then input into the target neural network for recognition, determining human movements and improving the accuracy of human movement recognition. In addition, there is no need to worry about the device falling off. Compared with the technical solution of using wearable devices to monitor movements during sleep, more accurate movement recognition can be performed.
[0157] The human motion monitoring method in this application can quantify the number of turning movements of a human body during sleep, which can not only be used as one of the data for evaluating personal sleep quality, but also as important reference data for diagnosing certain diseases; it can also be used to monitor the turning movements of the elderly and / or infants during sleep, and then report the monitoring results to the guardian to prevent the human body from falling off the bed, etc.
[0158] The following is a detailed description of the device according to the embodiment of the present application in conjunction with the accompanying drawings. Figure 8 shows a block diagram of the composition of the motion recognition device provided by the embodiment of the present application. As shown in Figure 8, the motion recognition device 800 includes but is not limited to the following modules.
[0159] The collection module 801 is configured to collect short-range communication data within the sensing area.
[0160] The data processing module 802 is configured to divide the short-distance communication data into multiple amplitude data sequences according to the data amplitude.
[0161] The extraction module 803 is configured to extract multiple target data sequences from multiple amplitude data sequences according to the amplitude fluctuation degree.
[0162] The recognition module 804 is configured to monitor human body motion based on multiple target data sequences.
[0163] It should be noted that the motion recognition device 800 in this embodiment can implement any human motion monitoring method in the embodiments of the present application.
[0164] According to the device of the embodiment of the present application, the short-range communication data within the sensing area is collected by the acquisition module, and the data within the sensing area can be obtained without exposing the user's privacy; the short-range communication data is divided into multiple amplitude data sequences according to the data amplitude to obtain multiple amplitude data sequences that can reflect the amplitude information; multiple target data sequences are extracted from the multiple amplitude data sequences according to the degree of amplitude fluctuation, so that the obtained multiple target data sequences can better reflect the characteristics of human body movements; and then human body movements are monitored based on the multiple target data sequences to improve the accuracy of human body movement recognition.
[0165] It should be understood that the present application is not limited to the specific configurations and processes described in the above embodiments and illustrated in the figures. For the sake of convenience and brevity, detailed descriptions of known methods are omitted here. The specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0166] FIG9 shows a structural diagram of an exemplary hardware architecture of a computing device capable of implementing the human motion monitoring method and apparatus according to an embodiment of the present application.
[0167] As shown in Figure 9, computing device 900 includes an input device 901, an input interface 902, a central processing unit 903, a memory 904, an output interface 905, and an output device 906. The input interface 902, the central processing unit 903, the memory 904, and the output interface 905 are interconnected via a bus 907. The input device 901 and the output device 906 are connected to the bus 907 via the input interface 902 and the output interface 905, respectively, and are further connected to other components of the computing device 900.
[0168] Specifically, the input device 901 receives input information from the outside and transmits the input information to the central processing unit 903 through the input interface 902; the central processing unit 903 processes the input information based on the computer-executable instructions stored in the memory 904 to generate output information, temporarily or permanently stores the output information in the memory 904, and then transmits the output information to the output device 906 through the output interface 905; the output device 906 outputs the output information to the outside of the computing device 900 for user use.
[0169] In one embodiment, the computing device shown in Figure 9 can be implemented as an electronic device, which may include: a memory configured to store a program; and a processor configured to run the program stored in the memory to execute the human motion monitoring method described in the above embodiment.
[0170] In one embodiment, the computing device shown in FIG9 can be implemented as a system, which may include: a memory configured to store a program; and a processor configured to run the program stored in the memory to execute the human motion monitoring method described in the above embodiment.
[0171] In some embodiments, the computing device may be implemented as the signal transmitting device and / or the signal receiving device.
[0172] The foregoing is merely an exemplary embodiment of the present application and is not intended to limit the scope of protection of the present application. Generally speaking, various embodiments of the present application can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device, although the present application is not limited thereto.
[0173] Embodiments of the present application may be implemented by executing computer program instructions by a processor of a motion recognition device, for example, in a processor entity, or by hardware, or by a combination of software and hardware. The computer program instructions may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages.
[0174] The block diagram of any logical flow in the accompanying drawings of the present application can represent program steps, or can represent interconnected logical circuits, modules and functions, or can represent a combination of program steps and logical circuits, modules and functions. The computer program can be stored on a memory. The memory can have any type suitable for the local technical environment and can be implemented using any suitable data storage technology, such as but not limited to read-only memory (ROM), random access memory (RAM), optical storage device and system (digital versatile disc DVD or CD optical disc) etc. The computer readable medium may include non-transient storage media. The processor can be any type suitable for the local technical environment, such as but not limited to a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (FPGA) and a processor based on a multi-core processor architecture.
[0175] The above description of exemplary embodiments of the present application has been provided by way of exemplary and non-limiting examples. However, various modifications and adaptations of the above embodiments will be apparent to those skilled in the art, when considered in conjunction with the accompanying drawings and the appended claims, without departing from the scope of the present application. Therefore, the proper scope of the present application will be determined by reference to the appended claims.
Claims
1. A human motion monitoring method, comprising: Collect short-range communication data within the sensing area; Dividing the short-distance communication data into a plurality of amplitude data sequences according to the data amplitude; Extracting multiple target data sequences from the multiple amplitude data sequences according to the amplitude fluctuation degree; The human body motion is monitored based on the multiple target data sequences.
2. The method according to claim 1, wherein: The sensing area is an area for sensing the human body motion, which is constructed based on the short-distance communication data interacting between the signal transmitting device and the signal receiving device.
3. The method according to claim 2, wherein: The method of dividing the short-distance communication data into a plurality of amplitude data sequences according to the data amplitude includes: According to the number of sampling packets, channel state data transmitted on multiple subcarriers carried by each pair of transceiver antennas are collected, wherein the number of sampling packets is a number determined based on a preset sampling duration and a preset sampling frequency, and each pair of transceiver antennas includes a transmitting antenna on the signal transmitting device and a receiving antenna on the signal receiving device; The amplitude data in the channel state data transmitted on the multiple subcarriers are respectively extracted, and the multiple amplitude data sequences are constructed based on the amplitude data in the channel state data transmitted on the multiple subcarriers, wherein the number of the amplitude data sequences is the same as the number of the subcarriers.
4. The method according to claim 1, wherein: The step of extracting a plurality of target data sequences from the plurality of amplitude data sequences according to the amplitude fluctuation degree comprises: Filter outlier data sequences from the plurality of amplitude data sequences based on a preset filtering range to obtain a plurality of first filtered data sequences; Filter out noise sequences in the plurality of first filtered data sequences based on a preset filtering method to obtain a plurality of second filtered data sequences; The plurality of second filtered data sequences are differentially processed to obtain a plurality of target data sequences.
5. The method according to claim 4, wherein: The step of filtering out outlier data sequences from the plurality of amplitude data sequences based on a preset filtering range to obtain a plurality of first filtered data sequences comprises: Based on a preset filtering range, respectively filter the plurality of first data in each of the amplitude data sequences; In a case where it is determined that the first data is not within the preset filtering range, determining that the first data is outlier data in the amplitude data sequence; Outlier data sequences including at least one outlier data in each of the amplitude data sequences are removed respectively to obtain a plurality of the first filtered data sequences.
6. The method according to claim 4, wherein: Before filtering the plurality of first data in each of the amplitude data sequences based on the preset filtering range, the human motion monitoring method further includes: Constructing a window sequence based on a preset window coefficient and a preset number of adjacent data extracted from the amplitude data sequence; Determine a mean value corresponding to the window sequence and a standard deviation corresponding to the window sequence according to each data in the window sequence and the amount of data in the window sequence; The preset filtering range is determined according to a mean value corresponding to the window sequence, a standard deviation corresponding to the window sequence, and a preset coefficient.
7. The method according to claim 4, wherein: The filtering out noise sequences in the plurality of first filtered data sequences based on a preset filtering method to obtain a plurality of second filtered data sequences includes: Based on a preset wavelet basis function, performing wavelet decomposition on each of the first filtered data sequences to obtain a multi-layer decomposition sequence corresponding to each of the first filtered data sequences, wherein each layer of the decomposition sequence includes at least two sub-data sequences and wavelet coefficients corresponding to the sub-data sequences; Based on a preset coefficient threshold, sub-data sequences in the multi-layer decomposition sequence corresponding to each first filtered data sequence are screened to obtain a plurality of screened data sequences; The multiple filtered data sequences are reconstructed to obtain multiple second filtered data sequences.
8. The method according to claim 4, wherein: Each of the second filtered data sequences includes m second filtered data, where m is the number of sampling packets; The performing differential processing on the plurality of second filtered data sequences respectively to obtain the plurality of target data sequences comprises: For each of the plurality of second data in the second filtered data sequence, determine the i-1th data in the target data sequence corresponding to the second filtered data sequence according to a preset differential coefficient and a difference between the i-th data and the i-1th data, where i is an integer greater than or equal to 1 and less than or equal to m; Repeat the above steps until i is equal to m, and obtain multiple target data sequences; The number of the target data sequences is the same as the number of subcarriers carried by each pair of transceiver antennas, the number of data in the target data sequence is m-1, and each pair of transceiver antennas includes a transmitting antenna on a signal transmitting device and a receiving antenna on a signal receiving device.
9. The method according to claim 8, wherein: The determining of the human body's motion based on the multiple target data sequences comprises: Determining an input sequence of a target neural network based on a target data sequence corresponding to the plurality of subcarriers; Inputting the input sequence of the target neural network into the target neural network for recognition to determine the movement of the human body; The target neural network is a network obtained by training the residual neural network with a preset training set, wherein the training set includes a sample data sequence and action labeling information corresponding to the sample data sequence.
10. The method according to claim 9, wherein: The step of determining an input sequence of a target neural network based on a target data sequence corresponding to the plurality of subcarriers comprises: respectively determining standard deviations of target data sequences corresponding to the plurality of subcarriers; Sorting standard deviations of target data sequences corresponding to the plurality of subcarriers to obtain a sorting result; Based on the sorting result, extract a preset number of the target data sequences as input sequences of the target neural network, and adjust the data dimensions of the preset number of target data sequences so that the data dimensions of the preset number of target data sequences are the same as the data dimensions required by the target neural network; The preset number is smaller than the maximum number of subcarriers.
11. The method according to claim 2, wherein: The sensing area is a rectangular area. Among them, the length of the rectangular area is the connection distance between the signal transmitting device and the signal receiving device, the width of the rectangular area is the length determined based on the wavelength of the collected short-distance communication signal, the preset adjustment coefficient and the connection distance between the signal transmitting device and the signal receiving device, the signal transmitting device and the signal receiving device are on a horizontal plane at the same height, and the height difference between the horizontal plane and the horizontal level of the human body is within a preset difference range.
12. An electronic device comprising: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a human motion monitoring method as described in any one of 1 to 11.
13. A readable storage medium, wherein: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the human body motion monitoring method as described in any one of 1 to 11 is implemented.
Citation Information
Patent Citations
Human body behavior identification method based on Wi-Fi channel state information (CSI)
CN111262637A
Human body action detection method and device and computer readable storage medium
CN113033318A
Indoor behavior real-time identification method and system based on WiFi channel state information
CN113723221A
Methods and systems for wireless crowd counting
US20160315682A1
Cited By
Object sensing method and object sensing device
CN120415606A
