Low-sampling WiFi perception enhancement model training method, enhancement method, action recognition method and device
By building a low-sampled WiFi perception enhancement model, using the generative adversarial network to expand the data of undersampled channel state information, the integration of perception and communication caused by high sampling frequency under WiFi perception is solved, and the perception ability recovery and enhancement under undersampling conditions is achieved.
Patent Information
- Application Number
- CN202211042407.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-29
- Publication Date
- 2025-08-26
- Estimated Expiration
- 2042-08-29
AI Technical Summary
In the prior art, WiFi perception methods require a very high sampling frequency, which makes it impossible to integrate perception and communication, and the perception ability is impaired under undersampling conditions.
By building a low-sampled WiFi perception enhancement model, the data augmentation of undersampled channel state information is expanded by using the generative adversarial network, and combined with reconstruction loss, central feature loss and semantic loss, the generator is trained to generate complete images to enhance WiFi perception capabilities.
Recover and enhance WiFi perception capabilities under undersampling conditions, achieve a balance between perception and communication, and improve the perception capabilities and efficiency of WiFi devices.
Smart Images
Figure CN115457653B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of WiFi perception technology, and in particular to a low-sampling WiFi perception enhancement model training method, an enhancement method, and a motion recognition method and device. Background Art
[0002] In recent years, with the increasing number of WiFi hotspots and the widespread application of WiFi in sensing technology, human behavior sensing technology based on WiFi signals has attracted widespread attention. Significant breakthroughs have been achieved in fields such as human-computer interaction, smart homes, and surveillance and security. WiFi sensing based on CSI (Channel State Information) is particularly representative. CSI is physical layer information that can be extracted from WiFi data packets and reflects the channel state. Therefore, the motion state of an object can be inferred from CSI, enabling target perception. Compared to traditional sensing, WiFi-based sensing offers broad development prospects due to its advantages such as ubiquity, non-line-of-sight (NLOS) capability, privacy and security, and strong scalability. However, this sensing method requires a very high sampling frequency (typically above 200Hz), which undoubtedly compromises WiFi's basic communication capabilities, turning WiFi devices into dedicated sensing devices. This makes it impossible to truly achieve integrated sensing and communication, severely hindering the deployment and widespread use of WiFi applications. Furthermore, if the sampling frequency is too low, sensing capabilities are inevitably significantly impaired.
[0003] Therefore, there is an urgent need for a method to enhance WiFi perception capabilities under under-sampling conditions. Summary of the Invention
[0004] In view of this, the embodiments of the present invention provide a low-sampling WiFi perception enhancement model training method, enhancement method, action recognition method and device to eliminate or improve one or more defects existing in the existing technology and solve the problem that WiFi perception communication integration cannot be achieved under the existing technology.
[0005] In one aspect, the present invention provides a low-sampling WiFi perception enhancement model training method, the method comprising the following steps:
[0006] Acquire a sampling data set, the sampling data set including multiple data strips, each data strip including high sampling channel state information of a first sampling frequency and corresponding low sampling channel state information of a second sampling frequency, obtained by WiFi from sensing a target object; add an action tag of the target object to each data strip, the action tag including a movement direction and / or body posture of the target object; the first sampling frequency is greater than the second sampling frequency;
[0007] Converting the low-sampling channel state information in each data strip into a corresponding low-sampling image according to a preset method, inserting blank sampling frames between the image content of each sampling point in the low-sampling image so that the original low-sampling image reaches the first sampling frequency after the blank sampling frames are inserted, thereby forming a corresponding mask image; converting the high-sampling channel state information in each data strip into a corresponding high-sampling image according to the preset method; and constructing a training sample set using the mask image, the high-sampling image, and the action marker as samples;
[0008] Obtaining an initial generative adversarial network model, the initial generative adversarial network model including a generator and a discriminator; the initial generative adversarial network model inputs the mask image in a single sample into the generator to generate a completed image; inputs the high-sampled image in the single sample into the discriminator to train the discrimination capability; and inputs the completed image into the discriminator to determine the authenticity of the completed image;
[0009] The initial generative adversarial network model is trained using the training sample set to construct a reconstruction loss, a center feature loss, and a semantic loss between the completed image and the high-sampled image. A joint loss is constructed based on the generative adversarial loss, the reconstruction loss, the center feature loss, and the semantic loss of the initial generative adversarial network model. The parameters of the initial generative adversarial network model are iterated using the joint loss, and the generator in the finally trained generative adversarial network model is used as the low-sampling WiFi perception enhancement model.
[0010] In some embodiments of the present invention, converting the downsampled channel state information in each data strip into a corresponding downsampled image according to a preset method includes:
[0011] The amplitude corresponding to the specified carrier at the specified sampling point of a single antenna is converted into a color pixel block, wherein the data information corresponding to each antenna in the low-sampling channel state information is constructed into one or more channels of the low-sampling image. In the channel corresponding to each antenna, the length of the low-sampling image is constructed using the data information corresponding to each sampling point in the low-sampling channel state information, and the width of the low-sampling image is constructed using the data information corresponding to each carrier frequency in the low-sampling channel state information.
[0012] In some embodiments of the present invention, the initial generative adversarial network model adopts an Edge Connect model.
[0013] In some embodiments of the present invention, the step of constructing the semantic loss includes:
[0014] Obtaining a pre-trained first classifier, where the first classifier is obtained by training a preset neural network using the high-sampled images in the training sample set as input and the action labels as output;
[0015] Obtaining a pre-trained second classifier, where the second classifier is obtained by training a K-nearest neighbor classifier using the high-sampled images in the training sample set as input and the action labels as output;
[0016] The completed image corresponding to each sample is input into the first classifier to obtain a first classification result, and the completed image corresponding to each sample is input into the second classifier to obtain a second classification result. The semantic loss is constructed according to the first classification result, the second classification result and the true value of the action label corresponding to the completed image.
[0017] In some embodiments of the present invention, the completed image corresponding to each sample is input into the first classifier to obtain a first classification result, and a first classification loss between the first classification result and the true value of the action label is constructed, and the calculation formula is:
[0018]
[0019] in, represents the classification loss of the first classifier; p i represents the probability of the i-th action classification; m represents the total number of action classifications; p j represents the probability of the jth action classification;
[0020] The completed image corresponding to each sample is input into the second classifier to obtain the second classification result, and the second classification loss is constructed. The calculation formula is:
[0021]
[0022] in, represents the classification loss of the second classifier; λ represents the hyperparameter of the penalty when the second classifier misclassifies;
[0023] The calculation formula of the semantic loss is:
[0024]
[0025] in, Represents semantic loss; A weight factor representing the classification loss of the first classifier; A weight factor representing the classification loss of the second classifier; represents the classification loss of the first classifier; represents the classification loss of the second classifier.
[0026] In some embodiments of the present invention, the function calculation formula of the center feature loss is:
[0027]
[0028] in, represents the center feature loss; m represents the total number of action categories; x i Representing the feature of the i-th type of action corresponding to the completed image obtained by the first classifier; It represents the average value of the features of each i-type action obtained by training the first classifier using the high-sampled images in the training sample set.
[0029] In some embodiments of the present invention, the function calculation formula of the joint loss is:
[0030]
[0031] in, Indicates joint loss; represents the weight factor of the reconstruction loss; adv represents the weight factor of the generated adversarial loss; cen represents the weight factor of the central feature loss; sem A weight factor representing the semantic loss; represents the reconstruction loss; represents the generative adversarial loss; represents the central feature loss; represents the semantic loss;
[0032] and The calculation formula is:
[0033]
[0034]
[0035] Wherein, X represents the value of the upsampled image; D(x) represents the value returned by inputting the upsampled image x into the discriminator D; G(z) represents the value of the complement image generated by inputting the mask image z into the generator G.
[0036] On the other hand, the present invention also provides a low-sampling WiFi perception enhancement method, which includes the following steps:
[0037] Obtaining low sampling channel state information to be processed;
[0038] converting the downsampling channel state information into a corresponding downsampling image according to a preset method, inserting blank sampling frames between image contents of each sampling point in the downsampling image so that the original downsampling image reaches a target sampling frequency after the blank sampling frames are inserted, thereby forming a corresponding mask image;
[0039] The mask image is input into the low-sampling WiFi perception enhancement model in any of the low-sampling WiFi perception enhancement model training methods mentioned above to obtain a completed image corresponding to the low-sampling channel state information.
[0040] On the other hand, the present invention also provides a method for sensing target object motion behavior based on low-sampling WiFi, the method comprising the following steps:
[0041] Obtain low-sampling channel state information obtained by WiFi perception of the target object, and use the low-sampling WiFi perception enhancement method mentioned above to obtain a completed image corresponding to the low-sampling channel state information;
[0042] The completed image is input into a preset classifier to obtain the action behavior of the target object, wherein the preset classifier is obtained by training a preset model using a real high-sampled image set as input and action labels as output.
[0043] On the other hand, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any one of the methods mentioned above when executed by a processor.
[0044] The beneficial effects of the present invention are at least:
[0045] The present invention provides a low-sampling Wi-Fi perception enhancement model training method, enhancement method, action recognition method, and device. By downsampling continuous high-sampling channel state information to obtain multiple low-sampling channel state information, a sampling dataset is constructed to form a training sample set, meeting the training and testing requirements of the present invention. The collected low-sampling channel state information and high-sampling channel state information are converted into corresponding low-sampling images and high-sampling images, respectively. Blank sampling frames are inserted between the image content of each sampling point in the low-sampling image so that the original low-sampling image reaches the target high sampling frequency after the blank sampling frames are inserted, forming a corresponding mask image. The mask image, high-sampling image, and action markers are used as samples to construct a training sample set. The action markers contain the motion information of the target object, which is a unique action feature for Wi-Fi radio frequency signal perception. The under-sampling channel state information data is augmented using image completion technology based on a generative adversarial network. During model training, a generative adversarial loss, reconstruction loss, center feature loss, and semantic loss are constructed between the completed image and the corresponding high-sampling image. Model parameters are iterated to ultimately obtain a low-sampling Wi-Fi perception enhancement model, achieving the restoration and enhancement of Wi-Fi perception capabilities under under-sampling conditions.
[0046] Furthermore, when constructing the semantic loss, two classifiers are introduced to classify the actions of the completed image. At the same time, the extraction of WiFi radio frequency signal features is strengthened, and a corresponding classification loss is constructed. This forces the completed image to be more similar to the original action category of the target object in the action label through classification results, thereby enhancing the model's training effect on the specificity of perception ability.
[0047] Additional advantages, objects, and features of the present invention will be set forth in part in the following description and will become apparent to those skilled in the art upon examination of the following or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained by the structures particularly pointed out in the description and drawings.
[0048] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. In the drawings:
[0050] Figure 1 Schematic diagram of the steps of a low-sampling WiFi perception enhancement model training method in one embodiment of the present invention.
[0051] Figure 2FIG. 4 is a schematic diagram of a method for converting channel state information to a color image according to an embodiment of the present invention.
[0052] Figure 3 Schematic diagram of a method for completing a low-sampled image based on a low-sampled WiFi perception enhancement model in one embodiment of the present invention.
[0053] Figure 4 Schematic diagram of the training structure and training method of the low-sampling WiFi perception enhancement model in one embodiment of the present invention.
[0054] Figure 5 Schematic diagram of a low-sampling WiFi perception enhancement method based on a 25 Hz low-sampling WiFi perception enhancement model, a 50 Hz low-sampling WiFi perception enhancement model, and a 100 Hz low-sampling WiFi perception enhancement model in one embodiment of the present invention. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.
[0056] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.
[0057] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.
[0058] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.
[0059] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
[0060] It should be emphasized here that the step marks mentioned below do not limit the order of the steps, but it should be understood that the steps can be executed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be executed simultaneously.
[0061] In order to solve the problem that WiFi perception communication integration cannot be achieved under the existing technology, the present invention provides a low-sampling WiFi perception enhancement model training method, such as Figure 1As shown, the method includes the following steps S101 to S104:
[0062] Step S101: Acquire a sampling data set, wherein the sampling data set includes multiple data strips, each data strip includes high sampling channel state information of a first sampling frequency and corresponding low sampling channel state information of a second sampling frequency obtained by WiFi sensing a target object; add an action tag of the target object to each data strip, wherein the action tag includes the movement direction and / or body posture of the target object; the first sampling frequency is greater than the second sampling frequency.
[0063] Step S102: Convert the low-sampling channel state information in each data strip into a corresponding low-sampling image according to a preset method, insert blank sampling frames between the image content of each sampling point in the low-sampling image, so that the original low-sampling image reaches the first sampling frequency after the blank sampling frames are inserted, forming a corresponding mask image; convert the high-sampling channel state information in each data strip into a corresponding high-sampling image according to a preset method; use the mask image, high-sampling image and action marker as samples to construct a training sample set.
[0064] Step S103: Obtain an initial generative adversarial network model, which includes a generator and a discriminator. The initial generative adversarial network model inputs the mask image from a single sample into the generator to generate a completed image; inputs the upsampled image from the single sample into the discriminator to train its discrimination capability; and inputs the completed image into the discriminator to determine whether the completed image is genuine.
[0065] Step S104: The initial generative adversarial network model is trained using the training sample set to construct a reconstruction loss, a central feature loss, and a semantic loss between the completed image and the high-sampled image. A joint loss is constructed based on the generative adversarial loss, reconstruction loss, central feature loss, and semantic loss of the initial generative adversarial network model. The parameters of the initial generative adversarial network model are iterated using the joint loss. The generator in the finally trained generative adversarial network model is used as the low-sampling WiFi perception enhancement model.
[0066] In step S101, traditional image completion datasets typically use a variety of randomly generated mask images and perform extensive brute-force training using these datasets. Unlike traditional image completion tasks, the channel state information data obtained through WiFi sensing required by the present invention is readily available in large quantities. Channel state information data requires manual collection, and the sampling frequency in reality is continuous. Obtaining data samples for training at all sampling frequencies is extremely costly and time-consuming. On the other hand, the model used in the present invention is a generative adversarial network model, and training requires a large amount of data.
[0067] In light of the above issues, the present invention considers that although WiFi sampling frequency is continuous, WiFi signals, unlike image data, do not frequently jump. Instead, they change continuously and have strong internal connections. For example, downsampling at 20Hz can produce 10Hz. Therefore, the present invention downsamples the continuous high-sampling channel state information at a first sampling frequency to obtain a plurality of discrete low-sampling channel state information at a second sampling frequency. The first sampling frequency is greater than the second sampling frequency, and the values of the first and second sampling frequencies can be adjusted according to actual needs.
[0068] For example, the first sampling frequency is 250 Hz, and the discrete second sampling frequencies are 25 Hz, 50 Hz, and 100 Hz, respectively. For any other second sampling frequencies, a rate adaptation method can be adopted to classify them as close to the preset 25 Hz, 50 Hz, and 100 Hz, wherein the rate adaptation can adopt a downsampling or interpolation method.
[0069] Specifically, a sampling frequency of 28 Hz can be downsampled to 25 Hz, and a sampling frequency of 90 Hz can be interpolated to 100 Hz.
[0070] The high sampling channel state information of the first sampling frequency and the low sampling channel state information of the second sampling frequency obtained by downsampling are used to construct a sampling data set, eliminating the need for manual data collection and greatly reducing the workload of data collection.
[0071] In step S102, the downsampled channel state information in each data strip is converted into a corresponding downsampled image according to a preset method, wherein the preset method in some embodiments is as follows: Figure 2 Shown, including:
[0072] The amplitude corresponding to a specified carrier at a specified sampling point for a single antenna is converted into a color pixel block. The data information corresponding to each antenna in the downsampling channel state information is constructed into one or more channels of a downsampling image. In each channel corresponding to an antenna, the length of the downsampling image is constructed using the data information corresponding to each sampling point in the downsampling channel state information, and the width of the downsampling image is constructed using the data information corresponding to each carrier frequency in the downsampling channel state information. Specifically, the time corresponding to the downsampling channel state information is the length of the downsampling image; the carrier corresponding to the downsampling channel state information is the width of the downsampling image; and the antenna corresponding to the downsampling channel state information is the channel of the downsampling image.
[0073] The resulting downsampled image is evenly distributed in the time domain according to the second sampling frequency of the downsampled channel state information. Specifically, blank sampling frames are inserted between the image content of each sampling point in the downsampled image, so that the original downsampled image reaches the first sampling frequency after the blank sampling frames are inserted. The evenly distributed downsampled image is converted into a mask image. The mask image consists of only black and white colors and is used to inform the generative adversarial network which parts need to be completed, where black represents the parts that do not need to be completed, and white represents the parts that need to be completed.
[0074] For example, Figure 3 As shown, taking the case where 50 Hz low-sampling channel state information is restored to 250 Hz high-sampling channel state information within 2 seconds as an example, the second sampling frequency of the low-sampling channel state information corresponds to 50 Hz, and the first sampling frequency corresponds to 250 Hz.
[0075] The original low-sampling channel state information is converted according to Figure 3 The method shown is transformed into a low-sampled image, which is as follows Figure 3 As shown in (1), according to the second sampling frequency of 50 Hz and the sampling time of 2 s, it can be seen that the low-sampling image has a total of 100 sampling frames. Blank sampling frames are inserted between the image contents of each sampling point in the low-sampling image, as shown in FIG. Figure 3 (2) is shown in order to make the original low-sampling image reach the first sampling frequency after inserting the blank sampling frame. The first sampling frequency is 250Hz, so 400 blank sampling frames need to be inserted to generate the corresponding mask image as shown in Figure 3 (3) shown.
[0076] according to Figure 3 The method shown converts the high-sampled channel state information in each data strip into a corresponding high-sampled image. The mask image, high-sampled image and action label are used as samples to construct a training sample set.
[0077] Among them, action tags are motion features unique to WiFi radio frequency signal perception, containing motion information of the perceived object. Action tags are added to each sample. In subsequent model training, the action of the completed image is forced to be as similar as possible to the action category recorded by the action tags, thereby optimizing the model's completion effect.
[0078] In some embodiments, the actions are labeled as gestures, including push, pull, up, down, left, and right.
[0079] In some embodiments, the action markers are body postures of daily actions, including walking, squatting, sitting, lying down and falling.
[0080] In some embodiments, the action mark is identity authentication, including walking information of different persons, specifically, the gait of person 1, the gait of person 2, the gait of person 3, the gait of person 4, and the gait of person 5.
[0081] In step S103, the present invention is based on image completion technology, so an initial generative adversarial network model is selected as a basic model and trained. The initial generative adversarial network model includes a generator and a discriminator.
[0082] In some embodiments, the initial generative adversarial network model uses the Edge Connect model. The Edge Connect model uses edge information in the structural information of the image to achieve image restoration. The model includes a pair of generators and discriminators. The generator consists of two parts: an encoding module and a decoding module. In the encoding module, downsampling is performed twice, followed by 8 residual blocks. In the decoding module, the size is adjusted to the input size, and a dilated convolution with a dilation rate of 8 is used in the residual layer to replace the traditional convolution operation to increase the receptive field of the subsequent layers.
[0083] The initial generative adversarial network model inputs the mask image in a single sample into the generator to generate a completed image; inputs the high-sampled image in a single sample into the discriminator to train the discrimination ability; and inputs the completed image into the discriminator to determine the authenticity of the completed image.
[0084] In step S104, the training sample set is used to train the initial generative adversarial network model. The initial generative adversarial network model is achieved by training two neural networks, wherein the two neural network models are the generator and the discriminator. During the training process, the discriminator will try its best to improve its ability to identify true data and false data, while the generator will try its best to generate true data. The discriminator uses both true and false data for training to update the weights, while the generator can only use the complete model (freeze the discriminator, input false data, but mark it as true) to feed back the discriminator's error to the front to achieve weight update.
[0085] Specifically, the high-sampled channel state information in each training sample is converted into a high-sampled image (real data) and fed into the discriminator, training the discriminator's ability to distinguish between true and false images. The low-sampled channel state information in each training sample is converted into a corresponding mask image and fed into the generator. The generator continuously completes the damaged mask image to generate a completed image (fake data), which is then fed into the discriminator to determine authenticity. The two continuously learn the distribution characteristics and information of the image through mutual competition. Ultimately, the completed image generated by the generator is so realistic that it is indistinguishable from the real thing, thus achieving data augmentation. That is, the low-sampled channel state information is converted into high-sampled channel state information after data augmentation.
[0086] In conventional training of image completion based on generative adversarial network models, in order to make the completed image appear softer to the human eye, high-frequency details are overly smoothed. However, these details often contain extremely important motion information for the perception of WiFi radio frequency signals. Therefore, Figure 4 As shown, the present invention reconstructs the loss function of the model, constructs the reconstruction loss, center feature loss and semantic loss between the completed image and the high-sampled image, strengthens the extraction of WiFi radio frequency signal features, and enhances the training effect of the model specificity.
[0087] In some embodiments, the function calculation formula of the reconstruction loss is shown in formula (1):
[0088]
[0089] in, Represents the reconstruction loss; X represents the value of the high-sampled image; G(z) represents the value of the complement image generated by inputting the mask image z into the generator G.
[0090] As mentioned above, in the initial generative adversarial network model training, to avoid over-smoothing of high-frequency details, the present invention removes the style loss and perceptual loss that cause over-smoothing. In addition, for motion perception of WiFi radio frequency signals, a central feature loss is introduced. The central feature loss function can be used to make data features of the same action category closer and data features of different action categories farther apart.
[0091] In some embodiments, the function calculation formula of the center feature loss is shown in formula (2):
[0092]
[0093] in, represents the center feature loss; m represents the total number of action categories; x i Representing the feature of the i-th type of action corresponding to the completed image obtained by the first classifier; It represents the average value of the features of each i-type action obtained by training the first classifier using the high-sampled images in the training sample set.
[0094] In some embodiments, constructing the semantic loss includes the following steps:
[0095] A first classifier obtained through pre-training is obtained, wherein the first classifier is obtained by training a preset neural network using high-sampled images in a training sample set as input and action labels as output.
[0096] A pre-trained second classifier is obtained, wherein the second classifier is obtained by training a K-nearest neighbor classifier using a high-sampled image in a training sample set as an input and an action label as an output.
[0097] The completed image corresponding to each sample is input into the first classifier to obtain a first classification result, and the completed image corresponding to each sample is input into the second classifier to obtain a second classification result. The semantic loss is constructed according to the first classification result, the second classification result and the true value of the action label corresponding to the completed image.
[0098] Semantic loss is used to evaluate the overall quality of the completed image. The first and second classifiers are trained using a large number of high-sampled channel state information datasets. Therefore, they can evaluate the quality of the completed image and have the ability to classify actions.
[0099] In some embodiments, the first classifier employs a two-stream convolutional enhanced Transformer (THAT) model.
[0100] In some embodiments, the second type of classifier can also be trained using a classifier model such as a random forest.
[0101] The completed image is input into the first classifier. The first classifier trained based on the preset neural network obtains the probability of the completed image being judged as each action category through the Softmax layer, and obtains the final first classification result based on the probability value, and at the same time obtains the characteristics of the action corresponding to the completed image.
[0102] In some embodiments, the completed image corresponding to each sample is input into the first classifier to obtain a first classification result, and a first classification loss between the first classification result and the true value of the action label is constructed, that is, the distance between the first classification result and the true value of the action label corresponding to the completed image is calculated, as shown in formula (3):
[0103]
[0104] in, represents the classification loss of the first classifier; m represents the total number of action classifications; p i represents the probability of the i-th action classification; p j represents the probability of the jth action classification.
[0105] The completed image is fed into the second classifier. The second classifier, trained based on the K-nearest neighbor classifier, cannot obtain the probability of each action category. It can only obtain the final classification result, i.e., the second classification result. If the second classifier misclassifies, its parameters are penalized. The K-nearest neighbor classifier is a KNN (K-Nearest Neighbor) classifier.
[0106] In some embodiments, the completed image corresponding to each sample is input into the second classifier to obtain a second classification result, and a second classification loss is constructed. When the second classifier makes a classification error, the parameters of the second classifier are penalized, as shown in formula (4):
[0107]
[0108] in, represents the classification loss of the second classifier; λ represents the hyperparameter of the penalty when the second classifier misclassifies.
[0109] In some embodiments, the function calculation formula of semantic loss is shown in formula (5):
[0110]
[0111] in, Represents semantic loss; Represents the weight factor of the classification loss of the first classifier; Represents the weight factor of the classification loss of the second classifier; represents the classification loss of the first classifier; represents the classification loss of the second classifier.
[0112] The joint loss is constructed based on the generation adversarial loss, reconstruction loss, central feature loss and semantic loss of the initial generation adversarial network model, where the function calculation formula of the generation adversarial loss is shown in formula (6):
[0113]
[0114] Wherein, D(x) represents the value returned by inputting the high-sampled image x into the discriminator D; G(z) represents the value of the complement image generated by inputting the mask image z into the generator G.
[0115] Therefore, the function calculation formula of the joint loss is shown in formula (7):
[0116]
[0117] in, Indicates joint loss; Represents the weight factor of reconstruction loss; λ adv represents the weight factor for generating adversarial loss; cen Represents the weight factor of the central feature loss; λ sem Represents the weight factor of semantic loss; represents the reconstruction loss; represents the generative adversarial loss; represents the center feature loss; Represents semantic loss.
[0118] The joint loss is used to iterate the parameters of the initial generative adversarial network model, and the generator in the final trained generative adversarial network model is used as the low-sampling WiFi perception enhancement model.
[0119] For example, as mentioned in step S101, sampling data sets are constructed with a first sampling frequency of 250 Hz and a second sampling frequency of 25 Hz; a first sampling frequency of 250 Hz and a second sampling frequency of 50 Hz; and a first sampling frequency of 250 Hz and a second sampling frequency of 100 Hz, respectively. Three corresponding training sample sets are constructed according to the method of step S102. The initial generative adversarial network model is trained using the three training sample sets, respectively, and finally a 25 Hz low-sampling WiFi perception enhancement model, a 50 Hz low-sampling WiFi perception enhancement model, and a 100 Hz low-sampling WiFi perception enhancement model are obtained.
[0120] For example, continue as Figure 3 As shown above, we have obtained Figure 3 (3) shows the mask image, and the mask image is input into the trained low-sampling WiFi perception enhancement model to generate Figure 3 The completed image shown in (4) can be seen intuitively. After the low-sampling WiFi perception enhancement model completes the expanded data, a 250Hz high-sampling image is obtained, which has better perception ability, and the completed image is consistent with the Figure 3 The images composed of the real data shown in (5) are very similar. Therefore, the low-sampling WiFi perception enhancement model trained by the present invention can enhance the perception ability of WiFi under under-sampling conditions.
[0121] The present invention also provides a low-sampling WiFi perception enhancement method, which includes the following steps S201 to S203:
[0122] Step S201: Acquire low sampling channel state information to be processed.
[0123] Step S202: Convert the low-sampling channel state information into a corresponding low-sampling image according to a preset method, insert blank sampling frames between the image content of each sampling point in the low-sampling image, so that the original low-sampling image reaches the target sampling frequency after the blank sampling frames are inserted, and form a corresponding mask image.
[0124] Step S203: inputting the mask image into a low-sampling WiFi perception enhancement model in any of the low-sampling WiFi perception enhancement model training methods above to obtain a completed image corresponding to the low-sampling channel state information.
[0125] In some embodiments, the preset method in step S202 is as follows: Figure 2 As shown, the low-sampling channel state information is converted into a color low-sampling image, wherein the time of the low-sampling channel state information corresponds to the length of the low-sampling image; the carrier of the low-sampling channel state information corresponds to the width of the low-sampling image; and the antenna of the low-sampling channel state information corresponds to the channel of the low-sampling image.
[0126] In step S104 , a 25 Hz low-sampling WiFi perception enhancement model, a 50 Hz low-sampling WiFi perception enhancement model, and a 100 Hz low-sampling WiFi perception enhancement model are obtained, and the corresponding low-sampling WiFi perception enhancement model is selected according to the sampling frequency.
[0127] For example, the sampling frequency of the low sampling channel state information in step S201 is 28 Hz. Figure 5 As shown, the low-sampling channel state information corresponds to the original data in the figure. The signal with a sampling frequency of 28 Hz is rate-adapted, that is, downsampled to 25 Hz. According to step S202, a mask image corresponding to the low-sampling channel state information with a sampling frequency of 25 Hz is obtained. The mask image is input into the 25 Hz low-sampling WiFi perception enhancement model, and finally a completed image with a sampling frequency of 250 Hz is obtained.
[0128] The present invention also provides a method for sensing target object motion behavior based on low-sampling WiFi, the method comprising the following steps S301-S302:
[0129] Step S301: obtaining low-sampling channel state information obtained by WiFi sensing the target object, and using the low-sampling WiFi perception enhancement method mentioned above to obtain a completed image corresponding to the low-sampling channel state information.
[0130] Step S302: Input the completed image into a preset classifier to obtain the target object's action behavior. The preset classifier is trained using a real high-sampled image set as input and action labels as output to obtain a preset model.
[0131] In step S302, each high-sampled image in the high-sampled image set is labeled with a real-world action tag representing the target object. Exemplarily, the action tag can be a hand gesture or a body posture associated with everyday actions. Hand gestures include pushing, pulling, up, down, left, and right; and body postures associated with everyday actions include walking, squatting, sitting, lying down, and falling. The preset model can be any classification model, exemplarily including a neural network model, a K-nearest neighbor classifier, or a random forest model.
[0132] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of a low-sampling WiFi perception enhancement model training method, a low-sampling WiFi perception enhancement method, and a method for perceiving target object motion behavior based on low-sampling WiFi are implemented.
[0133] Corresponding to the above method, the present invention also provides a device, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the device implements the steps of the method described above.
[0134] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.
[0135] In summary, the present invention provides a low-sampling WiFi perception enhancement model training method, enhancement method, action recognition method, and device. By downsampling continuous high-sampling channel state information to obtain multiple low-sampling channel state information, a sampling dataset is constructed, which serves as the basis for the training sample set and meets the model training and testing requirements of the present invention. The collected low-sampling channel state information and high-sampling channel state information are converted into corresponding low-sampling images and high-sampling images, respectively. Blank sampling frames are inserted between the image content of each sampling point in the low-sampling image so that the original low-sampling image reaches the target high sampling frequency after the blank sampling frames are inserted, forming a corresponding mask image. The mask image, high-sampling image, and action markers are used as samples to construct a training sample set. The action markers contain the motion information of the target object, which is a unique action feature for WiFi radio frequency signal perception. The under-sampling channel state information data is expanded using image completion technology based on a generative adversarial network. During model training, a generative adversarial loss, reconstruction loss, center feature loss, and semantic loss are constructed between the completed image and the corresponding high-sampling image. The model parameters are iterated to ultimately obtain a low-sampling WiFi perception enhancement model, which achieves the restoration and enhancement of WiFi perception capabilities under under-sampling conditions.
[0136] Furthermore, when constructing the semantic loss, two classifiers are introduced to classify the actions of the completed image. At the same time, the extraction of WiFi radio frequency signal features is strengthened, and a corresponding classification loss is constructed. This forces the completed image to be more similar to the original action category of the target object in the action label through classification results, thereby enhancing the model's training effect on the specificity of perception ability.
[0137] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.
[0138] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.
[0139] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.
[0140] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.
Claims
1. A low-sampling WiFi perception enhancement model training method, characterized in that: The method comprises the following steps: Acquire a sampling data set, the sampling data set including multiple data strips, each data strip including high sampling channel state information of a first sampling frequency and corresponding low sampling channel state information of a second sampling frequency, obtained by WiFi from sensing a target object; add an action tag of the target object to each data strip, the action tag including a movement direction and body posture of the target object; the first sampling frequency is greater than the second sampling frequency; The low-sampling channel state information in each data strip is converted into a corresponding low-sampling image according to a preset method, including converting the amplitude corresponding to the specified carrier at the specified sampling point of a single antenna into a color pixel block, wherein the data information corresponding to each antenna in the low-sampling channel state information is constructed into one or more channels of the low-sampling image, and in the channel corresponding to each antenna, the length of the low-sampling image is constructed using the data information corresponding to each sampling point in the low-sampling channel state information, and the width of the low-sampling image is constructed using the data information corresponding to each carrier frequency in the low-sampling channel state information; blank sampling frames are inserted between the image content of each sampling point in the low-sampling image so that the original low-sampling image reaches the first sampling frequency after the blank sampling frames are inserted, thereby forming a corresponding mask image; the high-sampling channel state information in each data strip is converted into a corresponding high-sampling image according to the preset method; and the mask image, the high-sampling image, and the action marker are used as samples to construct a training sample set; Obtaining an initial generative adversarial network model, the initial generative adversarial network model including a generator and a discriminator; the initial generative adversarial network model inputs the mask image in a single sample into the generator to generate a completed image; inputs the high-sampled image in the single sample into the discriminator to train the discrimination capability; and inputs the completed image into the discriminator to determine the authenticity of the completed image; The initial generative adversarial network model is trained using the training sample set to construct a reconstruction loss, a center feature loss, and a semantic loss between the completed image and the high-sampled image. A joint loss is constructed based on the generative adversarial loss, the reconstruction loss, the center feature loss, and the semantic loss of the initial generative adversarial network model. The parameters of the initial generative adversarial network model are iterated using the joint loss, and the generator in the finally trained generative adversarial network model is used as the low-sampling WiFi perception enhancement model.
2. The low-sampling WiFi perception enhancement model training method according to claim 1, characterized in that: The initial generative adversarial network model adopts the Edge Connect model.
3. The low-sampling WiFi perception enhancement model training method according to claim 1, characterized in that: The steps of constructing the semantic loss include: Obtaining a pre-trained first classifier, where the first classifier is obtained by training a preset neural network using the high-sampled images in the training sample set as input and the action labels as output; Obtaining a pre-trained second classifier, where the second classifier is obtained by training a K-nearest neighbor classifier using the high-sampled images in the training sample set as input and the action labels as output; The completed image corresponding to each sample is input into the first classifier to obtain a first classification result, and the completed image corresponding to each sample is input into the second classifier to obtain a second classification result. The semantic loss is constructed according to the first classification result, the second classification result and the true value of the action label corresponding to the completed image.
4. The low-sampling WiFi perception enhancement model training method according to claim 3 is characterized in that: The completed image corresponding to each sample is input into the first classifier to obtain a first classification result, and a first classification loss is constructed between the first classification result and the true value of the action label. The calculation formula is: in, represents the classification loss of the first classifier; m represents the total number of action classifications; p i represents the probability of the i-th action classification; p j represents the probability of the jth action classification; The completed image corresponding to each sample is input into the second classifier to obtain the second classification result, and the second classification loss is constructed. The calculation formula is: in, represents the classification loss of the second classifier; λ represents the hyperparameter of the penalty when the second classifier misclassifies; The calculation formula of the semantic loss is: in, Represents semantic loss; A weight factor representing the classification loss of the first classifier; A weight factor representing the classification loss of the second classifier; represents the classification loss of the first classifier; represents the classification loss of the second classifier.
5. The low-sampling WiFi perception enhancement model training method according to claim 4 is characterized in that: The calculation formula of the center feature loss is: in, represents the center feature loss; m represents the total number of action categories; x i Representing the feature of the i-th type of action corresponding to the completed image obtained by the first classifier; It represents the average value of the features of each i-type action obtained by training the first classifier using the high-sampled images in the training sample set.
6. The low-sampling WiFi perception enhancement model training method according to claim 5, characterized in that: The calculation formula of the joint loss is: in, Indicates joint loss; represents the weight factor of the reconstruction loss; adv represents the weight factor of the generated adversarial loss; cen Represents the weight factor of the central feature loss; λ sem A weight factor representing the semantic loss; represents the reconstruction loss; represents the generative adversarial loss; represents the central feature loss; represents the semantic loss; and The calculation formula is: Wherein, X represents the value of the upsampled image; D(x) represents the value returned by inputting the upsampled image x into the discriminator D; G(z) represents the value of the complement image generated by inputting the mask image z into the generator G.
7. A low-sampling WiFi perception enhancement method, characterized in that: The method comprises the following steps: Obtaining low sampling channel state information to be processed; converting the downsampling channel state information into a corresponding downsampling image according to a preset method, inserting blank sampling frames between image contents of each sampling point in the downsampling image so that the original downsampling image reaches a target sampling frequency after the blank sampling frames are inserted, thereby forming a corresponding mask image; The mask image is input into the low-sampling WiFi perception enhancement model in the low-sampling WiFi perception enhancement model training method according to any one of claims 1 to 6 to obtain a completed image corresponding to the low-sampling channel state information.
8. A method for sensing target object motion behavior based on low-sampling WiFi, characterized in that: The method comprises the following steps: Acquire low-sampling channel state information obtained by WiFi sensing the target object, and use the low-sampling WiFi perception enhancement method according to claim 7 to obtain a completed image corresponding to the low-sampling channel state information; The completed image is input into a preset classifier to obtain the action behavior of the target object; wherein, the preset classifier is obtained by training a preset model using a real high-sampling image set as input and action labels as output.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.