A dual-task WiFi sensing method based on wavelet fusion and gradient surgery

Abstract

The application discloses a dual-task WiFi sensing method based on wavelet fusion and gradient operation, and comprises the following steps: acquiring WiFi channel state information data, and pre-processing the WiFi channel state information data; decomposing amplitude information and phase information of the pre-processed WiFi channel state information data into low-frequency approximate subbands and high-frequency detail subbands, obtaining characteristic representations of different scales, and forming multi-scale frequency domain characteristics; performing feature fusion on the low-frequency approximate subbands and the high-frequency detail subbands, reconstructing the fused subbands, and constructing joint characteristic representations of the amplitude and the phase; pre-setting a gesture and identity joint recognition model, training the model; inputting the fused data obtained in step C into the gesture and identity joint recognition model, and outputting user information and gesture information recognized. The application can improve the stability and accuracy of joint recognition of identity recognition and gesture recognition.

Description

Technical Field

[0001] This invention relates to the fields of signal processing and wireless sensing technology, and in particular to a dual-task WiFi sensing method based on wavelet fusion and gradient surgery. Background Technology

[0002] Gesture recognition, as an intuitive and natural human-computer interaction method, is crucial for enhancing the user experience. Compared to traditional visual methods that are susceptible to environmental and privacy constraints, WiFi wireless sensing based on Channel State Information (CSI) offers advantages such as contactless operation and low deployment costs, making it an effective alternative for robust gesture recognition. However, in most real-world scenarios, simple gesture recognition is insufficient to prevent misuse and security risks arising from unverified user identities. Therefore, jointly and collaboratively sensing gestures and identity recognition is of great significance for improving system security and enabling personalized services.

[0003] Despite the clear need for joint perception, achieving high-precision gesture and identity recognition simultaneously in a recognition system remains a multifaceted challenge. From a signal feature perspective, the large-scale energy distribution characteristics inherent in amplitude are well-suited to the long-term stability required for identity recognition, while the fine-grained micro-motion information contained in phase can accurately capture the instantaneous changes in dynamic gestures. Therefore, a single feature cannot simultaneously meet the discrimination requirements of different tasks. From a task collaboration perspective, recognition tasks exhibit fundamental differences in feature focus due to varying optimization objectives. This difference leads to imbalances and negative transfer effects between generation tasks, causing strong tasks to suppress weak tasks. Consequently, multi-task collaboration struggles to simultaneously address the recognition needs of different tasks, hindering the coordinated optimization of overall performance. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a dual-task WiFi sensing method based on wavelet fusion and gradient surgery, which can overcome the shortcomings of the prior art and improve the stability and accuracy of joint recognition of identity recognition and gesture recognition.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows.

[0006] A dual-task WiFi sensing method based on wavelet fusion and gradient surgery includes the following steps: A. Obtain WiFi channel status information data, preprocess the WiFi channel status information data, and convert the preprocessed data into an RGB pseudo image using a preset mapping rule; B. Perform discrete wavelet transform on the amplitude and phase information of the preprocessed WiFi channel state information data to decompose them into low-frequency approximate sub-bands and high-frequency detail sub-bands. Decompose the low-frequency approximate sub-bands and high-frequency detail sub-bands to obtain feature representations at different scales and form multi-scale frequency domain features. C. Perform feature fusion on the low-frequency approximate subband and the high-frequency detail subband, and reconstruct the fused subband by inverse discrete wavelet transform to construct a joint feature representation of amplitude and phase; D. Pre-set a gesture and identity joint recognition model, train the model, and the gesture and identity joint recognition model is a two-branch two-stream decoupled topological neural network model; E. Use the fused data obtained in step C to create a joint recognition model for input gestures and identity, and output the recognized user information and gesture information.

[0007] Preferably, in step A, the preprocessing includes, Data continuity is achieved by using an interpolation filling algorithm, outlier data is removed by using a Hampel filter, high-frequency noise is removed by using a low-pass filter, phase entanglement is eliminated by using an unwarp algorithm, and phase offset is calculated and removed by linear fitting.

[0008] Preferably, in step B, Symlets wavelets are used as the wavelet basis pair amplitude feature. Phase characteristics A two-dimensional discrete wavelet transform decomposition is performed, mapping the signal to a multi-scale sub-band space. Through the wavelet decomposition process, the original features are divided into four functional sub-bands, including a low-frequency approximation sub-band and three high-frequency detail sub-bands. The low-frequency approximation sub-band LL reflects the overall structural information and steady-state topological profile of the signal, while the high-frequency detail sub-bands HL, LH, and HH correspond to the local variation features of the signal in the vertical, horizontal, and diagonal directions, respectively. , ,in, This shows the subset of the bands from which DWT decomposition was performed. and They respectively represent through the and The sub-band obtained by performing DWT.

[0009] Preferably, in step C, the low-frequency approximate sub-band contains overall signal structure information and environmental stability components, corresponding to identity-related steady-state characteristics, and a non-equal-weighted averaging strategy is used to achieve fusion. The high-frequency detail subband contains gesture dynamic information, which is fused using the rule of taking the largest absolute value. Feature recombination is performed on each subband after fusion. The four subbands after fusion Feature Reconstruction Using Inverse Discrete Wavelet Transform And mapped to the YCbCr color space, where, This represents the low-frequency sub-segment corresponding to the amplitude characteristics. This represents the low-frequency sub-segment corresponding to the phase characteristics. This represents the fused low-frequency sub-segment. This represents the high-frequency sub-segment whose amplitude characteristics are in the vertical direction. This represents the high-frequency sub-segment of phase characteristics in the vertical direction. This represents the high-frequency subband obtained in the vertical direction after fusion.

[0010] Preferably, in step D, the gesture and identity joint recognition model uses the ConvNeXt-Tiny network as a shared backbone network with a dual-branch structure to form a shared feature extraction module. The Stage 1-3 network structure is retained to perform general feature representation learning on CSI pseudo-images and extract the general spatiotemporal texture information contained in the fused features. The ConvNeXt-Tiny network adopts a large convolutional kernel deep convolution structure, a staged feature extraction mechanism, and a depth-separable convolution structure. Information fusion is achieved by performing convolution operations in the channel dimension and combining them with pointwise convolution. The single-task classification head of the ConvNeXt-Tiny network is removed. Independent identity recognition branch modules and gesture recognition branch modules are set in parallel at the output of the shared backbone of the ConvNeXt-Tiny network to form a dual-task feature extraction framework.

[0011] As a preferred approach, the gesture recognition branch is used for dynamic transient feature modeling. It employs a multi-layer convolutional structure with in-depth and temporal enhancement to extract dynamic gesture features. High-level semantic features are extracted through multiple convolutional layers, and feature normalization and dimensionality reduction are achieved by combining batch normalization layers and pooling layers. Two additional convolutional blocks are added to enhance feature representation capabilities, and the last-level convolutional kernel is set to an asymmetric structure to expand the receptive field in the temporal dimension, adapting to the extraction requirements of rapidly changing gesture features. After feature extraction, the feature map is spatially compressed through a global pooling layer, and then mapped sequentially through a flattening layer and a fully connected layer. Dropout regularization is introduced to reduce the risk of overfitting, and finally, the probability distribution of gesture categories is output.

[0012] As a preferred approach, the identity recognition branch is used for lightweight modeling of static steady-state features. It employs a lightweight convolutional structure to extract identity discrimination features. By extracting discrimination features related to user identity through a single convolutional block, it combines batch normalization layers and Dropout mechanisms to achieve feature stabilization and regularization. After spatial dimension compression is performed by a global average pooling layer, feature mapping is completed through a fully connected layer, and the user identity classification result is output. This approach reduces the number of parameters while ensuring feature discrimination capability and avoids overfitting to static identity features.

[0013] As a preferred approach, during the model training phase, classification loss functions are constructed separately for the gesture recognition task and the identity recognition task. , ,in, Represents a real identity tag. Indicates a real gesture label, Indicates predicted identity tags, The predicted gesture label is represented; the fused CSI pseudo-image is input into the model, and the prediction results of the identity recognition task and gesture recognition task are obtained through forward propagation. The loss function of each task is calculated according to the prediction results and the corresponding real labels. After obtaining the loss of each task, the gradient vector corresponding to each task in the joint recognition model is calculated through the backpropagation algorithm. The gradient vector is used to characterize the direction of influence of the loss function of each task on the model parameters.

[0014] As a preferred approach, a conditional gradient projection correction mechanism is constructed during the model training phase to share the parameter gradients of the backbone network. The gradients of each task-specific branch are calculated and updated independently without cross-branch interference. The gradient projection correction is performed after the backpropagation of each batch is completed and before the optimizer parameters are updated. Calculate the dot product between gradients of different tasks based on the gradient vectors of each task to determine if gradient conflicts exist; let the gradient of the gesture recognition task on the shared backbone network parameters be... The gradient of the identity recognition task on the shared backbone network parameters is: When the dot product of the two is less than zero This indicates a conflict in the gradient update directions of the two tasks. After detecting this conflict, a gradient projection correction operation is performed on the current task gradient, projecting the gradient of the gesture recognition task onto the normal plane of the identity recognition task gradient, thus eliminating the conflicting components between them. The gradient after projection correction is represented as follows: ; The model parameters are updated based on the gradient correction results.

[0015] The beneficial effects of adopting the above technical solution are as follows: This invention improves the ability of CSI data to express complex environmental changes by constructing a feature representation that fuses amplitude and phase information. Based on this, a gradient projection-based conflict resolution mechanism is introduced to correct gradient conflicts between different tasks during multi-task learning, thereby avoiding negative interference caused by inconsistent gradient directions of different tasks during shared parameter updates. Related methods eliminate conflicting gradient components through projection, enabling the model to simultaneously consider the learning objectives of multiple tasks during optimization, thus improving the stability and accuracy of joint identity recognition and gesture recognition, and to a certain extent reducing the occurrence of negative transfer during multi-task training, thereby improving the overall optimization effect. Attached Figure Description

[0016] Figure 1 This is a flowchart of the dual-task WiFi sensing method based on wavelet fusion and gradient surgery of the present invention; Figure 2 This is a framework diagram of the amplitude and phase feature fusion method of the present invention; Figure 3 This is a schematic diagram of the dual-branch neural network structure of the present invention. Detailed Implementation

[0017] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0018] like Figure 1 As shown, a dual-task WiFi sensing method based on wavelet fusion and gradient surgery includes the following steps: S1. To address the technical problems of existing CSI data acquisition, such as data loss, outlier interference, high-frequency noise superposition, and phase information distortion due to noise, making it difficult to directly use for subsequent processing, this embodiment provides a CSI data processing method. Specifically, after collecting the user's raw CSI data, an interpolation filling algorithm is used to solve the data loss problem caused by device sampling errors, ensuring data continuity; outlier data is removed using a Hampel filter to avoid interference with subsequent processing; given that the frequency of human activity CSI signals is concentrated below 50Hz, a low-pass filter is used to filter out high-frequency noise, retain the effective signal, and separate the amplitude information and coarse phase information.

[0019] Meanwhile, to address the issues of phase information being susceptible to noise interference, phase entanglement, equipment calibration deviations, and distortion caused by inherent channel offsets, unwarping is performed to eliminate phase entanglement. Then, the phase offset is calculated and removed through linear fitting to obtain clean and accurate phase information. Finally, the amplitude and phase information are converted into RGB pseudo-images through preset mapping rules, realizing the conversion of CSI data from the signal domain to the image domain, so as to facilitate the image domain representation of subsequent deep learning model processing.

[0020] Through the above processing steps, the signal timing continuity is restored, outliers and high-frequency noise components are effectively suppressed, and the consistency and stability of the phase sequence after unwrapping and phase correction are significantly improved. This embodiment significantly improves the usability and discriminability of amplitude and phase signals while ensuring data integrity.

[0021] However, relying solely on the simple mapping in S1 is insufficient to fully exploit the deep complementary characteristics of amplitude and phase in the frequency domain, and the basic pseudo-image still contains a certain degree of frequency band aliasing noise. To address this issue, this invention, based on the data cleaning in S1, further designs and implements step S2: instead of directly using the basic pseudo-image, it returns to the pure amplitude and phase sequences already separated in S1, performing multi-scale frequency domain deep decomposition and fusion to construct a higher-order feature representation with stronger discriminative power, exhibiting superior feature expression performance compared to the basic pseudo-image. For example... Figure 2 As shown, this amplitude and phase feature fusion method mainly includes the following steps: S2. For the preprocessed CSI data, extract its amplitude and phase information respectively, and perform a two-dimensional discrete wavelet transform (DWT) to achieve multi-scale sub-band decomposition of amplitude and phase features. To address the difficulty of balancing detail preservation and robustness in traditional wavelet bases, this invention optimizes and improves the wavelet basis functions: Existing conventional methods mostly employ the Daubechies (Db) wavelet series. However, these wavelets exhibit poor symmetry and significant nonlinear phase characteristics. When processing non-stationary signals like CSI phase signals, which are highly sensitive to time and phase, they are prone to phase distortion and time-domain shifts, leading to blurred gesture micro-motion features and distorted identity background features. To address this issue, this invention optimizes the wavelet basis selection, ultimately adopting the Symlets wavelet as a dedicated wavelet basis. The Symlets wavelet is an improved version of the Db series, possessing approximately linear phase and better symmetry. Under the same vanishing moment, it can effectively reduce group delay bias and phase distortion during signal decomposition and reconstruction. When using sym8 to decompose CSI signals, it can improve the signal-to-noise ratio of reconstructed gesture edge abrupt changes and transient features. Without destroying the overall amplitude contour, it maximizes the preservation of micro-Doppler information contained in the phase, ensuring the fidelity and stability of signal decomposition.

[0022] First, regarding amplitude characteristics Phase characteristics A two-dimensional discrete wavelet transform (DWT) decomposition is performed, mapping the signal to a multi-scale sub-band space. Through the wavelet decomposition process, the original features are divided into four functional sub-bands: a low-frequency approximation sub-band and three high-frequency detail sub-bands. The low-frequency approximation sub-band (LL) primarily reflects the overall structural information and steady-state topological contour of the signal, while the high-frequency detail sub-bands HL, LH, and HH correspond to the local variation features of the signal in the vertical, horizontal, and diagonal directions, respectively. These sub-bands can accurately characterize the edge detail changes of the CSI signal caused by human activity disturbances, achieving comprehensive extraction of multi-scale features. This is the core manifestation of the DWT method in feature mining. The formula for the above decomposition process is defined as follows: ,

[0023] in, This shows the subset of the bands from which DWT decomposition was performed. and They respectively represent through the and The sub-band obtained by performing DWT.

[0024] The customized decomposition method based on the improved sym8 wavelet basis can effectively suppress the phase distortion and feature distortion caused by the traditional Db wavelet, avoid mutual interference and feature cancellation between amplitude and phase information during the decomposition process, and preserve the complementary characteristics between amplitude and phase to the maximum extent. This provides a high-quality, high-discrimination multi-scale feature foundation for subsequent frequency band adaptive fusion and dual-task recognition.

[0025] S3. At the feature fusion level, existing methods mostly employ simple arithmetic averages or single maximum value rules, failing to consider the heterogeneous distribution characteristics of identity features and gesture features: identity discrimination information is mainly concentrated in low-frequency amplitude components, while gesture dynamic information is mainly manifested in high-frequency phase abrupt changes. Simple fusion easily leads to high-energy low-frequency signals drowning out weak high-frequency signals, causing feature cancellation and loss of complementary information. Therefore, this invention proposes a frequency-band differentiated adaptive fusion algorithm, performing targeted fusion based on sub-band functions and feature distributions. For the low-frequency sub-band, which mainly contains overall signal structure information and environmental stability components, corresponding to identity-related steady-state features, this invention employs a non-equal-weighted average strategy to achieve fusion. The fusion process is represented as follows:

[0026] in, This represents the low-frequency sub-segment corresponding to the amplitude characteristics. This represents the low-frequency sub-segment corresponding to the phase characteristics. This represents the fused low-frequency sub-band. For the high-frequency sub-band, this embodiment uses the rule of taking the largest absolute value for fusion to ensure that abrupt energy is preserved and to weaken feature neutralization in subsequent fusion processes, thereby improving recognition performance and enhancing the ability to detect changes in human activity in WiFi sensing dual-task scenarios. Taking the vertical high-frequency sub-band HL as an example, its fusion process is as follows:

[0027] in, This represents the high-frequency sub-segment whose amplitude characteristics are in the vertical direction. This represents the high-frequency sub-segment of phase characteristics in the vertical direction. This represents the high-frequency sub-band in the vertical direction obtained after fusion. Through the above fusion strategy, the four sub-bands obtained from amplitude and phase features at various scales can be uniformly fused to form a new set of fused sub-bands, achieving effective integration of multi-source information and constructing a richer and more discriminative feature representation. Subsequently, the features of each fused sub-band are reorganized, and the fusion result is as follows:

[0028] Finally, the four subbands after fusion will be Feature reconstruction was performed using the Inverse Discrete Wavelet Transform (IDWT) and mapped to the YCbCr color space.

[0029]

[0030] The customized multi-scale fusion strategy proposed in this embodiment fundamentally overcomes the technical defects commonly found in traditional fusion algorithms, such as feature cancellation and loss of information due to amplitude complementarity. By accurately dividing and targeting multi-scale sub-bands, this scheme effectively prevents high-frequency fine-grained action features from being submerged by low-frequency background components, while avoiding the distortion of stable environmental features during the fusion process, thus preserving the complementary characteristics between amplitude and phase to the maximum extent.

[0031] The enhanced pseudo-image generated by this process not only fully inherits the temporal continuity of stage S1, but also deeply embeds the frequency domain complementary features extracted in stages S2 and S3. Given the highly heterogeneous feature information contained in this input data, traditional single-task networks struggle to effectively extract features without causing feature confusion. Therefore, step S4 specifically designs a dual-branch feature extraction network structure to adapt to and fully exploit the expressive potential of such enhanced features.

[0032] S4. In this embodiment, the amplitude-phase fusion features obtained in step S3 are used as model input. After the fusion features are processed into RGB pseudo-image representation, they are input into the joint recognition model constructed in this invention to complete deep feature extraction and multi-task classification reasoning. See also Figure 3 To achieve collaborative execution of user identity recognition and gesture recognition, this invention constructs a dual-branch, dual-stream decoupled topological neural network structure. The network structure includes a shared feature extraction module, an identity recognition branch module, and a gesture recognition branch module, which are used to achieve feature decoupling and joint optimization of heterogeneous tasks.

[0033] This invention employs the ConvNeXt-Tiny network as a shared backbone network with a dual-branch structure, forming a shared feature extraction module. Retaining its Stage 1-3 network structure, it is used to perform general feature representation learning on CSI pseudo-images, extracting the general spatiotemporal texture information contained in the fused features. The ConvNeXt-Tiny network uses a large-kernel deep convolution structure and a staged feature extraction mechanism, enabling global modeling of input features within a large receptive field, thus improving the model's ability to express complex spatial structure information. Simultaneously, it employs a depthwise separable convolution structure, achieving information fusion by performing convolution operations along the channel dimension and combining them with pointwise convolution. This reduces model computational complexity while maintaining feature expressive power, providing a shared feature foundation for dual-task recognition.

[0034] To address the inherent limitations of the standard ConvNeXt-Tiny network, which is designed solely for single-task image classification and employs only a single-task classification head at its terminals, the standard network uses a single optimization path. When applied to dual-task scenarios, this can easily lead to gradient competition, causing static identity features to suppress dynamic gesture features, thus failing to simultaneously adapt to the optimization requirements of both static steady-state features and dynamic transient features. Therefore, this invention structurally improves the standard ConvNeXt-Tiny network. It retains its backbone network as a shared feature extraction module, removes the original single-task classification head, and sets up independent identity recognition and gesture recognition branch modules in parallel at the output of the shared backbone, forming a dual-task feature extraction framework. This enables the reuse of shared features and the independent modeling and joint optimization of task-specific features.

[0035] The gesture recognition branch is used for dynamic transient feature modeling. It adopts a multi-layer convolutional structure with in-depth and temporal enhancement to achieve dynamic gesture feature extraction: high-level semantic features are extracted through multiple convolutional layers, and feature normalization and dimensionality reduction are completed by combining batch normalization layers and pooling layers. Two additional convolutional blocks are added to enhance feature expression capabilities, and the last-level convolutional kernel is set to an asymmetric structure to expand the receptive field in the temporal dimension and adapt to the extraction requirements of rapidly changing gesture features. After feature extraction, the feature map is spatially compressed through a global pooling layer, and then the feature is mapped through a flattening layer and a fully connected layer. Dropout regularization is introduced to reduce the risk of overfitting, and finally the gesture category probability distribution is output.

[0036] The identity recognition branch is used for lightweight modeling of static steady-state features. It adopts a lightweight convolutional structure to extract identity discrimination features: the discrimination features related to user identity are extracted through a single convolutional block, and the feature stabilization and regularization are achieved by combining batch normalization layer and Dropout mechanism. After spatial dimension compression is performed by global average pooling layer, the feature mapping is completed through fully connected layer and the user identity classification result is output. While ensuring the feature discrimination ability, the number of parameters is reduced, and overfitting to static identity features is avoided.

[0037] This invention achieves physical isolation and feature extraction of two task branches through a dual-stream feature extraction topology, solving the technical problem that a single model cannot simultaneously optimize static steady-state features and dynamic transient features, and avoiding feature confusion and gradient competition between tasks. Based on the feature representation capabilities of the complex ConvNeXt-Tiny network, it improves the accuracy and model stability of dual-task joint recognition by designing differentiated branches to match the feature modeling requirements of identity recognition and gesture recognition respectively.

[0038] During the model training phase, classification loss functions are constructed for both gesture recognition and identity recognition tasks. In this embodiment, the cross-entropy loss function is used to measure the difference between the model's prediction results and the true labels. The calculation process is as follows: ,

[0039] in, Represents a real identity tag. Indicates a real gesture label, Indicates predicted identity tags, Gesture labels indicating predictions.

[0040] During model training, the fused CSI pseudo-images are input into the joint recognition model, and prediction results for identity recognition and gesture recognition tasks are obtained through forward propagation. Subsequently, the loss function for each task is calculated based on the prediction results and the corresponding ground truth labels. After obtaining the loss for each task, the gradient vector corresponding to each task in the joint recognition model is calculated using the backpropagation algorithm. The gradient vector is used to characterize the direction of influence of the loss function of each task on the model parameters.

[0041] To address the gradient direction conflict problem that inevitably arises during training in a dual-branch shared backbone architecture, this invention further constructs a gradient projection correction mechanism in step S5, thereby forming a complete technical closed loop from "feature enhancement construction" to "structural decoupling and adaptation" to "optimization and conflict elimination," ultimately significantly improving the accuracy and robustness of dual-task recognition.

[0042] S5. To address the gradient conflict problem in multi-task learning, this invention constructs a gradient projection correction machine during model training. By dynamically projecting and correcting the gradients of shared parameters, the update direction of model parameters is adjusted, thereby alleviating the gradient interference problem between different tasks and improving the stability and convergence performance of multi-task joint training.

[0043] To address the issues of gradient conflicts and negative transfer that easily arise when sharing backbone network parameters in multi-task learning, this invention constructs a conditional gradient projection correction mechanism during the model training phase. By dynamically detecting and projecting the gradients of shared parameters, the mechanism explicitly constrains the parameter update direction, thereby eliminating gradient interference between tasks and improving the convergence stability and recognition performance of dual-task joint training. Traditional multi-task learning typically involves directly weighting and summing gradients. When the angle between the gradients of the gesture task and the identity task is greater than 90°, the gradient dot product is negative, leading to significant gradient conflicts and negative transfer in optimization. This results in the shared backbone being dominated by the strong task (identity), suppressing the update path of the weak task (gesture), and making it difficult for the model to converge collaboratively.

[0044] To address the aforementioned shortcomings, this invention implements a conditional gradient projection correction strategy with strictly limited scope: this correction operation only applies to the parameter gradients of the shared backbone network; the gradients of each task-specific branch are calculated and updated independently, without cross-branch interference, ensuring independent optimization of task-specific features. In terms of implementation, gradient projection correction is performed after backpropagation of each batch and before the optimizer parameters are updated.

[0045] Specifically, based on the gradient vectors of each task obtained in step S4, the dot product between the gradients of different tasks is calculated to determine whether gradient conflicts exist. Let the gradient of the gesture recognition task on the shared backbone network parameters be... The gradient of the identity recognition task on the shared backbone network parameters is: Then, by calculating the dot product of the two:

[0046] When the calculation result is less than zero, it indicates that there is a conflict in the gradient update directions of the two tasks, that is:

[0047] After detecting the aforementioned conflict conditions, this invention performs a gradient projection correction operation on the current task gradient, projecting the gradient of the gesture recognition task onto the normal plane of the identity recognition task gradient, thereby eliminating the conflicting components between the two. The gradient after projection correction is expressed as:

[0048] The second term represents the projection component of the current gesture recognition task gradient onto the identity recognition task gradient direction. By subtracting this projection component from the current gesture recognition task gradient, the conflicting parts of the gesture recognition task gradient with the identity recognition task gradient can be explicitly removed. After the gradient projection operation, the inner product of the corrected gesture recognition task gradient vector and the identity recognition task gradient vector is zero, thus ensuring that the updates of shared parameters in the current training iteration do not cancel each other out on the optimization objectives of the two tasks.

[0049] By constructing the gradient projection correction mechanism described above, this invention can proactively resolve gradient conflicts of shared parameters from an optimization perspective, avoiding negative impacts of the optimization process of a single task on another task, thereby improving the stability and recognition performance of multi-task joint learning, and further enhancing the overall generalization ability of the model in WiFi-sensing human activity recognition and identity recognition tasks.

[0050] S6. Update the parameters of the joint recognition model based on the gradient correction results to obtain the optimized joint recognition model. The gradient correction process eliminates gradient conflicts between different tasks, making the model parameter update direction more consistent, thereby improving the stability of the joint recognition model during multi-task training. The joint recognition model takes amplitude-phase fusion data as input and outputs the recognized user information and gesture information.

[0051] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dual-task WiFi sensing method based on wavelet fusion and gradient surgery, characterized in that... Includes the following steps: A. Obtain WiFi channel status information data, preprocess the WiFi channel status information data, and convert the preprocessed data into an RGB pseudo image using a preset mapping rule; B. Perform discrete wavelet transform on the amplitude and phase information of the preprocessed WiFi channel state information data to decompose them into low-frequency approximate sub-bands and high-frequency detail sub-bands. Decompose the low-frequency approximate sub-bands and high-frequency detail sub-bands to obtain feature representations at different scales and form multi-scale frequency domain features. C. Perform feature fusion on the low-frequency approximate subband and the high-frequency detail subband, and reconstruct the fused subband by inverse discrete wavelet transform to construct a joint feature representation of amplitude and phase; D. Pre-set a gesture and identity joint recognition model, train the model, and the gesture and identity joint recognition model is a two-branch two-stream decoupled topological neural network model; E. Use the fused data obtained in step C to create a joint recognition model for input gestures and identity, and output the recognized user information and gesture information.

2. The dual-task WiFi sensing method based on wavelet fusion and gradient surgery according to claim 1, characterized in that: In step A, the preprocessing includes, Data continuity is achieved by using an interpolation filling algorithm, outlier data is removed by using a Hampel filter, high-frequency noise is removed by using a low-pass filter, phase entanglement is eliminated by using an unwarp algorithm, and phase offset is calculated and removed by linear fitting.

3. The dual-task WiFi sensing method based on wavelet fusion and gradient surgery according to claim 1, characterized in that: In step B, Symlets wavelets are used as wavelet basis pairs for amplitude features. Phase characteristics A two-dimensional discrete wavelet transform decomposition is performed, mapping the signal to a multi-scale sub-band space. Through the wavelet decomposition process, the original features are divided into four functional sub-bands, including a low-frequency approximation sub-band and three high-frequency detail sub-bands. The low-frequency approximation sub-band LL reflects the overall structural information and steady-state topological profile of the signal, while the high-frequency detail sub-bands HL, LH, and HH correspond to the local variation features of the signal in the vertical, horizontal, and diagonal directions, respectively. , ,in, This shows the subset of the bands from which DWT decomposition was performed. and They respectively represent through the and The sub-band obtained by performing DWT.

4. The dual-task WiFi sensing method based on wavelet fusion and gradient surgery according to claim 3, characterized in that: In step C, the low-frequency approximate sub-band contains information about the overall signal structure and environmental stability components, corresponding to identity-related steady-state characteristics. A non-equal-weighted averaging strategy is used to achieve fusion. The high-frequency detail subband contains gesture dynamic information, which is fused using the rule of taking the largest absolute value. Feature recombination is performed on each subband after fusion. The four subbands after fusion Feature Reconstruction Using Inverse Discrete Wavelet Transform And mapped to the YCbCr color space, where, This represents the low-frequency sub-segment corresponding to the amplitude characteristics. This represents the low-frequency sub-segment corresponding to the phase characteristics. This represents the fused low-frequency sub-segment. This represents the high-frequency sub-segment whose amplitude characteristics are in the vertical direction. This represents the high-frequency sub-segment of phase characteristics in the vertical direction. This represents the high-frequency subband obtained in the vertical direction after fusion.

5. The dual-task WiFi sensing method based on wavelet fusion and gradient surgery according to claim 1, characterized in that: In step D, the gesture and identity joint recognition model uses the ConvNeXt-Tiny network as a shared backbone network with a dual-branch structure to form a shared feature extraction module. The Stage 1-3 network structure is retained to perform general feature representation learning on CSI pseudo-images and extract the general spatiotemporal texture information contained in the fused features. The ConvNeXt-Tiny network adopts a large convolutional kernel deep convolution structure, a staged feature extraction mechanism, and a depth-separable convolution structure. Information fusion is achieved by performing convolution operations in the channel dimension and combining them with pointwise convolution. The single-task classification head of the ConvNeXt-Tiny network is removed. Independent identity recognition branch modules and gesture recognition branch modules are set in parallel at the output of the shared backbone of the ConvNeXt-Tiny network to form a dual-task feature extraction framework.

6. The dual-task WiFi sensing method based on wavelet fusion and gradient surgery according to claim 5, characterized in that: The gesture recognition branch is used for dynamic transient feature modeling. It adopts a multi-layer convolutional structure with in-depth and temporal enhancement to achieve dynamic gesture feature extraction. High-level semantic features are extracted through multi-layer convolutional layers. Batch normalization and pooling layers are combined to complete feature normalization and dimensionality reduction. Two additional convolutional blocks are added to enhance feature expression capabilities. The last-level convolutional kernel is set to an asymmetric structure to expand the receptive field in the temporal dimension and adapt to the extraction requirements of fast-changing gesture features. After feature extraction, the feature map is spatially compressed through a global pooling layer, and then mapped through a flattening layer and a fully connected layer. Dropout regularization is introduced to reduce the risk of overfitting, and finally the probability distribution of gesture categories is output.

7. The dual-task WiFi sensing method based on wavelet fusion and gradient surgery according to claim 5, characterized in that: The identity recognition branch is used for lightweight modeling of static steady-state features. It adopts a lightweight convolutional structure to extract identity discrimination features. It extracts discrimination features related to user identity through a single convolutional block, and combines batch normalization layer and Dropout mechanism to achieve feature stabilization and regularization. After performing spatial dimension compression through global average pooling layer, it completes feature mapping through fully connected layer and outputs user identity classification results. This reduces the number of parameters while ensuring feature discrimination ability and avoids overfitting to static identity features.

8. The dual-task WiFi sensing method based on wavelet fusion and gradient surgery according to claim 7, characterized in that: During the model training phase, classification loss functions are constructed separately for the gesture recognition task and the identity recognition task. , ,in, Represents a real identity tag. Indicates a real gesture label, Indicates predicted identity tags, The predicted gesture label is represented; the fused CSI pseudo-image is input into the model, and the prediction results of the identity recognition task and gesture recognition task are obtained through forward propagation. The loss function of each task is calculated according to the prediction results and the corresponding real labels. After obtaining the loss of each task, the gradient vector corresponding to each task in the joint recognition model is calculated through the backpropagation algorithm. The gradient vector is used to characterize the direction of influence of the loss function of each task on the model parameters.

9. The dual-task WiFi sensing method based on wavelet fusion and gradient surgery according to claim 8, characterized in that: During the model training phase, a conditional gradient projection correction mechanism is constructed to share the parameter gradients of the backbone network. The gradients of each task-specific branch are calculated and updated independently without cross-branch interference. The gradient projection correction is performed after the backpropagation of each batch is completed and before the optimizer parameters are updated. Calculate the dot product between gradients of different tasks based on the gradient vectors of each task to determine if gradient conflicts exist; let the gradient of the gesture recognition task on the shared backbone network parameters be... The gradient of the identity recognition task on the shared backbone network parameters is: When the dot product of the two is less than zero This indicates a conflict in the gradient update directions of the two tasks. After detecting this conflict, a gradient projection correction operation is performed on the current task gradient, projecting the gradient of the gesture recognition task onto the normal plane of the identity recognition task gradient, thus eliminating the conflicting components between them. The gradient after projection correction is represented as follows: ; The model parameters are updated based on the gradient correction results.

Images