Method for identifying intelligent shelf interaction behavior based on wireless Wi-Fi

Abstract

The present invention discloses an intelligent shelf interaction behavior recognition method based on wireless Wi-Fi, which mainly includes: according to the Wi-Fi CSI received by multiple Wi-Fi receivers, using the PCA method to preliminarily estimate the action amplitude of shoppers between shelves, and dividing the Wi-Fi CSI into large-amplitude action signal segments and small-amplitude action signal segments; using the large-amplitude action signal segments to coarsely perceive the walking actions of shoppers and identify the positions of shoppers; using the small-amplitude action signal segments and the obtained shopper position information to finely perceive the interaction actions between shoppers and shelves, so as to extract the three-dimensional relative direction features of the interaction actions between shoppers and shelves; for the three-dimensional relative direction features of the interaction actions between shoppers and shelves, using the convolutional long short-term memory network ConvLSTM and the deep adversarial neural network DANN, and combining the shopper position information, to identify the area where the shopper interacts with the shelf.

Description

Technical Field

[0001] The present invention relates to the fields of wireless sensing and behavior recognition, and particularly to a method for identifying intelligent shelf interaction behaviors using commercial wireless Wi-Fi. Background Art

[0002] In recent years, with the development and progress of mobile communication related technologies, the mobile Internet has gradually made a qualitative leap from providing only single voice, SMS, and data services to the Internet of Everything. The next-generation mobile communication network (6G) is considered to have a new ability of integrated sensing and communication (ISAC) different from traditional communication functions. At the same time, among various integrated sensing and communication solutions, the solution with the fastest commercialization ability and great development prospects is to endow the existing Wi-Fi, 4G / 5G networks with sensing capabilities, so as to sense people and the surrounding environment, combine the sensing network and the communication network to form a new type of intelligent 6G network.

[0003] With the development of the retail industry, driven by data, new retail with experience as the core has developed rapidly since its proposal. As a typical representative of it, unmanned stores have gradually become one of the development directions of future offline retail. Unmanned stores have achieved a closed-loop interaction with consumers, collected interaction data in the real world, which can help brands and retailers provide personalized marketing information at the correct customer touchpoints and provide a better consumer experience. Brands can make quick decisions based on sales activities and consumer demands, use methods such as wireless sensing and machine learning to identify customers' behaviors, infer their intentions, and understand their preferences. Stores can optimize the placement of goods and obtain higher profits.

[0004] [References]

[0005] [1]Multiple emitter location and signal parameter estimation, IEEE transactions on antennas and propagation, Ap-34(3):276-280, Mar. 1986.

[0006] [2]Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting, NIPS’15, P802-810.

[0007] [3]Unsupervised Domain Adaptation by Backpropagation,ICML'15,P1180-1189. Summary of the invention

[0008] In view of the above-mentioned prior art, the present invention proposes a smart shelf interactive behavior recognition method based on wireless Wi-Fi, which uses commercial wireless Wi-Fi to perform smart shelf interactive behavior recognition.

[0009] In order to solve the above technical problems, the present invention proposes a method for identifying interactive behaviors of smart shelves based on wireless Wi-Fi, which is characterized by comprising the following steps:

[0010] Step 1: Based on the Wi-Fi CSI received by multiple Wi-Fi receivers, the PCA method is used to preliminarily estimate the amplitude of shoppers' movements between shelves, and the Wi-Fi CSI is divided into large-amplitude movement signal segments and small-amplitude movement signal segments;

[0011] Step 2: Use the large-amplitude motion signal segments to perform coarse-grained perception of the shopper's walking motion and identify the shopper's location;

[0012] Step 3: Use the small-amplitude motion signal segments and the obtained shopper location information to perform fine-grained perception of the shopper's interaction with the shelf, thereby extracting the three-dimensional relative direction features of the shopper's interaction with the shelf;

[0013] Step 4: Based on the three-dimensional relative direction characteristics of the shoppers' interaction with the shelves, the convolutional long short-term memory network ConvLSTM and the deep adversarial neural network DANN are used, combined with the shoppers' location information to identify the area where the shoppers interact with the shelves.

[0014] Furthermore, the method for identifying interactive behaviors of smart shelves based on wireless Wi-Fi of the present invention includes:

[0015] The specific steps of step one are as follows:

[0016] 1-1) Processing of Wi-Fi CSI data, including: for Wi-Fi CSI data, first using a Hampel filter to remove the static component of each subcarrier; then using a Savitzky-Golay filter to remove noise;

[0017] 1-2) Perform principal component analysis on the processed Wi-Fi CSI data and calculate the variance of each principal component. The variance of the i-th principal component is recorded as Var i ;

[0018] 1-3) Take time windows on the continuous signal. In each time window, calculate the variance var respectively. i , and for each time window, mark the serial number i of the maximum principal component that satisfies var i > Var i . Denote i as the PCA penetration layer number.

[0019] 1-4) Denote the time windows with 3 < PCA penetration layer number < 5 as small-amplitude time windows, and the time windows with PCA penetration layer number > 5 as large-amplitude time windows.

[0020] 1-5) Segment the signal segments composed of continuous small-amplitude time windows into small-amplitude motion signal segments, and segment the signal segments composed of continuous large-amplitude time windows into large-amplitude motion signal segments.

[0021] The specific steps of Step Two are as follows:

[0022] 2-1) Take the sampling of several packets taken on the Wi-Fi receiving end antenna array as the input, and run the MUSIC algorithm to obtain the arrival angle and relative arrival time of the coarse-grained motion signal segment.

[0023] 2-2) According to the angles of the shopper's body relative to each receiving end sensed by using the MUSIC algorithm on multiple Wi-Fi receiving ends, and at the same time use the positions of the Wi-Fi receiving ends to calculate the angle between the line connecting the shopper's body at different positions and the Wi-Fi receiving end antenna array. Define this angle as the true arrival angle. Use Equation (1) to take the position with the minimum sum of the squares of the errors between the true angle and the estimated angle as the target position.

[0024]

[0025] In Equation (1), P is the estimated target position of the shopper's body, R is the number of Wi-Fi receiving ends, is the true arrival angle of the shopper's body at position p, and θ i is the arrival angle of the shopper's body estimated by the MUSIC algorithm.

[0026] The specific steps of Step Three are as follows:

[0027] 3-1) Establish a three-dimensional space coordinate system with the origin at the position h / 2 above the bottom surface perpendicular to point P. The position of the shopper's body is P, and the height of the shelf is h. Define: the direction parallel to the shelf and the ground is the x-axis, the direction perpendicular to the shelf and parallel to the ground is the y-axis, and the direction perpendicular to the ground is the z-axis. Set the origin of this three-dimensional space coordinate system at the position h / 2 above point P along the z-axis.

[0028] 3-2) In the above three-dimensional space coordinate system, the relationship between the walking speed and direction of a human body is as follows: The walking direction of the human body is:

[0029]

[0030] In formula (2) is the vector expression form of the direction, and d x , d y , d z are the components of the direction on the x-axis, y-axis, and z-axis respectively; then the speed in the direction is expressed as:

[0031]

[0032] In formula (3), v is the walking speed of the human body along the walking direction; is the vector expression form of the speed, and v x , v y , v z are the components of the speed on the x-axis, y-axis, and z-axis respectively;

[0033] 3-3) On multiple Wi-Fi receivers, use the short-time Fourier transform to transfer the fine-grained action signal segment to the frequency domain, and extract the Doppler frequency shift DFS of the signal. The relationship between the human body movement and DFS is expressed as follows:

[0034]

[0035] In formula (4), represents the influence of the human body walking speed on DFS on the m-th receiver, which is obtained through the position of the receiver in the above three-dimensional space coordinate system; the position of the transmitter is The position of the m-th receiver is Then:

[0036]

[0037]

[0038]

[0039] In formulas (5), (6) and (7), λ is the wavelength of the Wi-Fi CSI;

[0040] 3-4) For each direction, estimate the speed in that direction:

[0041]

[0042] In formula (8), V is the direction The speed estimation value on D i is the DFS estimation value calculated by using the short-time Fourier transform on the i-th receiving end;

[0043] 3 - 5) For each axis direction, k components are taken for the positive and negative directions respectively, so there are a total of K = 2k + 1 direction components. For three axes, there are a total of K×K×K directions to be estimated. Since the direction (0, 0, 0) cannot be estimated, actually K×K×K - 1 directions are estimated and stored in a three-dimensional matrix of K×K×K according to the spatial relationship. This three-dimensional matrix is the BDP;

[0044] 3 - 6) Calculate the BDP for each time window pane and connect them in time to obtain the BDP sequence.

[0045] The specific steps of Step Four are as follows:

[0046] 4 - 1) Take the projection of each BDP in the BDP sequence in each three-dimensional direction, including: for each K×K matrix in each direction, there are K values. Take the maximum value of the K estimated values at the corresponding positions to obtain the matrix projection of K×K;

[0047] 4 - 2) Connect the matrix projections of K×K in the three directions according to the time series to obtain the projection sequence. Use the convolutional long short-term memory network ConvLSTM to extract the spatio-temporal feature maps of K×K respectively, and connect the spatio-temporal feature maps in the three directions into a feature of K×K×3;

[0048] 4 - 3) According to the three-dimensional space coordinate system described above, divide the human body movement direction into: left, middle, right; front, middle, back; up, middle, down in three dimensions; after permutation and combination, remove the three directions of middle - upper - middle, middle - middle - middle and middle - lower - middle that coincide with the human body, and 24 directions are retained, including: left - front - up, left - front - middle, left - front - down, middle - front - up, middle - front - middle, middle - front - down, right - front - up, right - front - middle, right - front - down, left - middle - up, left - middle - middle, left - middle - down, right - middle - up, right - middle - middle, right - middle - down, left - back - up, left - back - middle, left - back - down, middle - back - up, middle - back - middle, middle - back - down, right - back - up, right - back - middle, right - back - down;

[0049] 4 - 4) Classify the spatio-temporal features of K×K×3 by using the deep adversarial neural network DANN to obtain the judgment of the action in the above 24 directions in the three-dimensional space under different environments;

[0050] 4 - 5) Combine the classification results of the directions and the position P of the human body to finally obtain the area where the shopper interacts with the shelf.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] (1) Combines two-granularity recognition methods. Because signal segmentation for action perception granularity is added, it combines two-granularity recognition methods and can flexibly process signal segments in reality.

[0053] (2) Direction judgment in three-dimensional space. The direction judgment is realized in the three-dimensional space of reality, enhancing the recognition and perception ability of real actions.

[0054] (3) Reduces the environmental dependence of signals. A coordinate system is established with the human body as the center, and the cross-domain model of DANN is used to enable the system to have cross-domain capabilities and work effectively in various human positions and environments. Description of the Drawings

[0055] Figure 1 is the overall flowchart of the intelligent shelf interaction behavior recognition method based on wireless Wi-Fi proposed by the present invention;

[0056] Figure 2 is an example of the Wi-Fi transceiver deployment and the establishment of a coordinate system with the human body as the center in the present invention. Detailed Embodiment

[0057] In terms of the hardware used to implement the method of the present invention, four ThinkPad T series laptops are used, and each of these four computers is equipped with an Intel 5300 network card. One of them uses the network card to connect an external antenna as the transmitter, while the other three use the network card to connect three external antennas as receivers. As for the software, the operating systems of the four laptops are all Ubuntu 12.04. To collect CSI data, the four laptops are installed with a CSI data collection tool Linux 8002.11n CSI Tool implemented based on the 802.11n protocol. The four laptops all operate in Monitor mode among the 161 channels in the 5GHz band. In this mode, each pair of transceiver antennas can obtain 30 orthogonal frequency division multiplexing subcarriers. Since there are three receiving antennas at the receiving end, 1×3×30 subcarrier data can be obtained. The sampling frequency used by the system is 1000Hz. When the packet loss effect is not considered, the receiving end will receive a 90×1000 data every 1s. As Figure 2 Deployed in the manner shown

[0058] The present invention will be further described below in conjunction with the drawings and specific embodiments, but the following embodiments are by no means any limitation to the present invention.

[0059] As Figure 1 shown, the present invention proposes an intelligent shelf interaction behavior recognition method based on wireless Wi-Fi, including the following steps:

[0060] Step 1: Through multiple (more than three in the present invention, such as Figure 2 shown, in this example, 1 transmitter Tx is deployed, orthogonally arranged with three receivers Rx1, Rx2, and Rx3, and a coordinate system O-xyz is established) Wi-Fi receivers receive Wi-Fi CSI (Channel State Information). According to the received Wi-Fi CSI, the PCA method is used to preliminarily estimate the amplitude of the shopper's movement between the shelves, and the Wi-Fi CSI is segmented into a large-amplitude motion signal segment and a small-amplitude motion signal segment. The specific steps are as follows:

[0061] 1-1) Processing of Wi-Fi CSI data, including: for Wi-Fi CSI data, first use the Hampel filter to remove the static components of each subcarrier; then use the Savitzky-Golay filter to further remove other noises;

[0062] 1-2) For the processed Wi-Fi CSI data, use Figure 2 to segment the Tx-Rx1 link, perform principal component analysis on the data on Rx1, calculate the variances of each principal component, and denote the variance of the i-th principal component as Var i ;

[0063] 1-3) Take time windows on the continuous signal. In each time window, calculate the variance var i respectively, and for each time window, mark the serial number i of the maximum principal component that satisfies var i > Var i . Denote i as the PCA penetration layer number;

[0064] 1-4) Denote the time windows with the PCA penetration layer number greater than 3 and less than 5 as fine-grained time windows, and denote the time windows with the PCA penetration layer number greater than 5 as coarse-grained time windows;

[0065] 1-5) Segment the signal segment composed of continuous small-amplitude time windows into small-amplitude motion signal segments, and segment the signal segment composed of continuous large-amplitude time windows into large-amplitude motion signal segments.

[0066] Step 2: For the large-amplitude walking action, use the large-amplitude motion signal segment to coarsely perceive the shopper's walking action, realize the tracking of the human movement trajectory, and identify the location of the shopper. The specific steps are as follows:

[0067] 2-1) Taking the sampling of several packets on the Wi-Fi receiving end antenna array as input, run the MUSIC algorithm to obtain the arrival angle and relative arrival time of the coarse-grained action signal segment. Among them, the MUSIC algorithm is the Multiple Signal Classification (MUSIC) method described in Document [1].

[0068] 2-2) According to Figure 2 the angles of the shopper's body relative to each receiving end sensed by using the MUSIC algorithm on two Wi-Fi receiving ends Rx1 and Rx2, and at the same time, using the positions of the Wi-Fi receiving ends to calculate the angles between the lines connecting the shopper's body at different positions and the Wi-Fi receiving end antenna array, define this angle as the true arrival angle, and use Equation (1) to take the position with the minimum sum of the squares of the errors between the true angle and the estimated angle as the target position

[0069]

[0070] In Equation (1), P is the estimated target position of the shopper's body, R is the number of Wi-Fi receiving ends, is the true arrival angle of the shopper's body at position p, and θ i is the arrival angle of the shopper's body estimated by the MUSIC algorithm.

[0071] Step 3: For the shelf interaction actions with small amplitudes, use the small amplitude action signal segments to identify the action directions in the three-dimensional space of the human body, and combine the obtained shopper's location information to perform fine-grained perception of the shopper's interaction actions with the shelf, so as to extract the three-dimensional relative direction feature BDP (Body-coordinate Direction Profile) of the shopper's interaction actions with the shelf; the specific steps are as follows:

[0072] 3-1) As Figure 2 shown, point P is the position of the human body. Establish a three-dimensional space coordinate system with the origin at a height of h / 2 perpendicular to the bottom surface of point P. As Figure 2 shown, in this example, the directions of each coordinate axis are the same as O-xyz. Establish a three-dimensional space coordinate system O'-x'y'z'. The position of the shopper's body is P, and the height of the shelf is h. Then, set the origin of the coordinate system at a position with a height of h / 2 at position P to establish a three-dimensional space coordinate system. In this coordinate system, the direction parallel to the shelf and the ground is the x-axis, the direction perpendicular to the shelf and parallel to the ground is the y-axis, and the direction perpendicular to the ground is the z-axis.

[0073] 3-2) In the above three-dimensional space coordinate system, the relationship between the walking speed and direction of the human body is as follows:

[0074] The walking direction of the human body is:

[0075]

[0076] In Equation (2) is the vector representation of the direction, d x , d y , d z are the components of the direction on the x-axis, y-axis, and z-axis respectively;

[0077] Then the velocity in the direction is expressed as:

[0078]

[0079] In Equation (3), v is the walking speed of the human body along the walking direction; is the vector representation of the velocity, v x , v y , v z are the components of the velocity on the x-axis, y-axis, and z-axis respectively.

[0080] 3-3) On the Wi-Fi receivers Rx1, Rx2, and Rx3 in Figure 2 , use the short-time Fourier transform to transfer the fine-grained action signal segment to the frequency domain, and extract the Doppler frequency shift (DFS) of the signal. The relationship between human movement and DFS can be expressed as follows:

[0081]

[0082] In Equation (4), represents the influence of the human walking speed on DFS at the m-th receiver, obtained from the position of the receiver in the above three-dimensional space coordinate system; in the three-dimensional space coordinate system, the position of the transmitter is the position of the m-th receiver is Then each coefficient can be calculated by the following formula:

[0083]

[0084]

[0085]

[0086] In Equations (5), (6), and (7), λ is the wavelength of the Wi-Fi CSI.

[0087] 3-4) For each direction, estimate the velocity in that direction:

[0088]

[0089] In formula (8), V is the velocity estimation value in the direction and D i is the DFS estimation value calculated by using the short-time Fourier transform on the i-th receiving end;

[0090] 3-5) For each axis direction, k components are taken for the positive and negative directions respectively, so there are a total of K = 2k + 1 direction components. For three axes, a total of K×K×K directions need to be estimated. Since the direction (0, 0, 0) cannot be estimated, actually K×K×K - 1 directions are estimated and stored in a three-dimensional matrix of K×K×K according to the spatial relationship. This three-dimensional matrix is the BDP;

[0091] 3-6) Calculate the BDP for each time window pane and connect them in time, then a BDP sequence can be obtained.

[0092] Step Four: For the three-dimensional relative direction features of the shopper's interaction actions with the shelf, use the Convolutional Long Short Term Memory Network (ConvLSTM) and the Deep Adversarial Neural Network (DANN), and combine the location information of the shopper to identify the area where the shopper interacts with the shelf. The specific steps are as follows:

[0093] 4-1) Take the projection of each BDP in the BDP sequence in each three-dimensional direction, including: for each K×K matrix in each direction, there are K values, and take the maximum value of the K estimated values at the corresponding positions to obtain the K×K matrix projection;

[0094] 4-2) Connect the K×K matrix projections in the three directions according to the time series respectively to obtain a projection sequence, and use the Convolutional Long Short Term Memory Network (ConvLSTM) to extract the K×K spatio-temporal feature maps respectively. Connect the spatio-temporal feature maps in the three directions to form a feature of K×K×3; among them, ConvLSTM is the machine learning model in reference [2].

[0095] 4-3) According to the described three-dimensional space coordinate system, divide the human body movement direction into: left, middle, right, front, middle, back, up, middle, down in three dimensions; after permutation and combination, remove the three directions of middle-upper-middle, middle-middle-middle and middle-lower-middle that coincide with the human body, then there are a total of 24 directions that need to be concerned about, including: left-front-up, left-front-middle, left-front-down, middle-front-up, middle-front-middle, middle-front-down, right-front-up, right-front-middle, right-front-down, left-middle-up, left-middle-middle, left-middle-down, right-middle-up, right-middle-middle, right-middle-down, left-back-up, left-back-middle, left-back-down, middle-back-up, middle-back-middle, middle-back-down, right-back-up, right-back-middle, right-back-down.

[0096] 4-4) For the spatio-temporal features of K×K×3, use the deep adversarial neural network DANN for classification to obtain the judgment of the above 24 directions of the action in the three-dimensional space in different environments; among them, DANN is the cross-domain learning model in reference [3], which can maintain a high classification accuracy in different environments.

[0097] 4-5) Combine the classification results of the directions and the position P of the human body to finally obtain the area where the shopper interacts with the shelf.

[0098] Although the present invention has been described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many variations without departing from the purpose of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for identifying intelligent shelf interaction behaviors based on wireless Wi-Fi, characterized in that, It includes the following steps: Step 1: According to the Wi-Fi CSI received by multiple Wi-Fi receivers, the PCA method is used to preliminarily estimate the amplitude of the shopper's actions between the shelves, and the Wi-Fi CSI is segmented into a large-amplitude action signal segment and a small-amplitude action signal segment; Step 2: The large-amplitude action signal segment is used to coarsely perceive the shopper's walking action and identify the shopper's location; Step 3: The small-amplitude action signal segment and the obtained shopper's location information are used to finely perceive the shopper's interaction action with the shelf, so as to extract the three-dimensional relative direction features of the shopper's interaction action with the shelf; Step 4: For the three-dimensional relative direction features of the shopper's interaction action with the shelf, the convolutional long short-term memory network ConvLSTM and the deep adversarial neural network DANN are used, and combined with the shopper's location information, the area where the shopper interacts with the shelf is identified.

2. The method for identifying intelligent shelf interaction behaviors based on wireless Wi-Fi according to claim 1, wherein, The specific steps of Step 1 are as follows: 1-1) Processing of Wi-Fi CSI data, including: for Wi-Fi CSI data, first use the Hampel filter to remove the static components of each subcarrier; then use the Savitzky-Golay filter to remove noise; 1-2) Perform principal component analysis on the processed Wi-Fi CSI data, calculate the variances of each principal component, and denote the variance of the i-th principal component as Var i ; 1-3) Take a time window pane on the continuous signal. In each time window pane, calculate the variance var respectively i , and for each time window pane, mark the serial number i of the maximum principal component that satisfies var i > Var i . Denote i as the number of PCA penetration layers; 1-4) Denote the time window pane with 3 < PCA penetration layer number < 5 as the small-amplitude time window, and the time window pane with PCA penetration layer number > 5 as the large-amplitude time window; 1-5) Segment the signal segment composed of continuous small-amplitude time windows into small-amplitude action signal segments, and segment the signal segment composed of continuous large-amplitude time windows into large-amplitude action signal segments.

3. The method for identifying the interactive behavior of an intelligent shelf based on wireless Wi-Fi according to claim 2, wherein, The specific steps of Step 2 are as follows: 2-1) Take the sampling of several packets taken on the Wi-Fi receiver antenna array as the input, and run the MUSIC algorithm to obtain the arrival angle and relative arrival time of the coarse-grained action signal segment; 2-2) According to the angles of the shopper's body relative to each receiver sensed by the MUSIC algorithm on multiple Wi-Fi receivers, and at the same time use the Wi-Fi receiver positions to calculate the angle between the line connecting the shopper's body at different positions and the Wi-Fi receiver antenna array, and define this angle as the true arrival angle. Use Equation (1) to take the position with the minimum sum of the squares of the errors between the true angle and the estimated angle as the target position In formula (1), P is the estimated position of the shopper's human body target, and R is the number of Wi-Fi receivers. is the true angle of arrival when the shopper's human body is at position p, and θ i is the angle of arrival of the shopper's human body estimated by the MUSIC algorithm.

4. The method for identifying the interactive behavior of an intelligent shelf based on wireless Wi-Fi according to claim 3, wherein, The specific steps of Step 3 are as follows: 3-1) Establish a three-dimensional space coordinate system with the origin at a height of h / 2 perpendicular to the bottom surface at point P. The position of the shopper's body is P, and the height of the shelf is h. Define: the direction parallel to the shelf and the ground is the x-axis, the direction perpendicular to the shelf and parallel to the ground is the y-axis, and the direction perpendicular to the ground is the z-axis. Set the origin of this three-dimensional space coordinate system at a height of h / 2 along the z-axis at point P; 3-2) In the above three-dimensional space coordinate system, the relationship between the human walking speed and direction is as follows: The human walking direction is: In formula (2) is the vector expression form of the direction, and d x , d y , d z are the components of the direction on the x-axis, y-axis, and z-axis, respectively; Then the direction The speed in is expressed as: In Equation (3), v is the walking speed of the human body along the walking direction; is the vector expression form of the speed, v x , v y , v z are the components of the speed on the x-axis, y-axis, and z-axis, respectively; 3-3) On multiple Wi-Fi receivers, use the short-time Fourier transform to transfer the fine-grained action signal segment to the frequency domain, and extract the Doppler frequency shift DFS of the signal. The relationship between the human action and the DFS is expressed as follows: In formula (4), represents the influence of the human walking speed on the DFS, which is obtained from the position of the receiving end in the above three-dimensional space coordinate system; The position of the sending end is The position of the m-th receiving end is Then: In formulas (5), (6) and (7), λ is the wavelength of Wi-Fi CSI; 3-4) For each direction, estimate the speed in that direction: In Equation (8), V is the estimated value of the velocity in the direction and D i is the DFS estimated value calculated by using the short-time Fourier transform on the i-th receiving end; 3-5) For each coordinate axis direction, take k components for the positive and negative directions respectively, so there are a total of K = 2k + 1 direction components. For three coordinate axes, a total of K×K×K directions need to be estimated. Since the direction (0, 0, 0) cannot be estimated, actually K×K×K - 1 directions are estimated and stored in a three-dimensional matrix of K×K×K according to the spatial relationship. This three-dimensional matrix is the BDP; 3-6) Calculate the BDP for each time window pane and connect them in time to obtain the BDP sequence.

5. The method for identifying intelligent shelf interaction behaviors based on wireless Wi-Fi according to claim 4, characterized in that, The specific steps of Step Four are as follows: 4-1) Take the projection of each BDP in the BDP sequence in each three-dimensional direction, including: for each K×K matrix in each direction, there are K values. Take the maximum value of the K estimated values at the corresponding positions to obtain the matrix projection of K×K; 4-2) Connect the K×K matrix projections in the three directions according to the time series respectively to obtain the projection sequence. Use the convolutional long short-term memory network ConvLSTM to extract the K×K spatio-temporal feature maps respectively, and connect the spatio-temporal feature maps in the three directions to form a feature of K×K×3; 4-3) According to the three-dimensional space coordinate system described above, divide the human body movement directions into: left, middle, right, front, middle, back, up, middle, down in three dimensions; after permutation and combination, remove the three directions of middle-upper-middle, middle-middle-middle and middle-lower-middle that coincide with the human body, and 24 directions are retained, including: left-front-up, left-front-middle, left-front-down, middle-front-up, middle-front-middle, middle-front-down, right-front-up, right-front-middle, right-front-down, left-middle-up, left-middle-middle, left-middle-down, right-middle-up, right-middle-middle, right-middle-down, left-back-up, left-back-middle, left-back-down, middle-back-up, middle-back-middle, middle-back-down, right-back-up, right-back-middle, right-back-down; 4-4) Use the deep adversarial neural network DANN to classify the K×K×3 spatio-temporal features to obtain the judgment of the action in the above 24 directions in the three-dimensional space under different environments; 4-5) Combine the classification results of the directions and the position P of the human body to finally obtain the area where the shopper interacts with the shelf.

Images