An mmwave-based breast cancer postoperative rehabilitation exercise recognition system and method based on adversarial learning

Abstract

The application discloses a kind of millimeter wave breast cancer postoperative rehabilitation movement identification system and method based on adversarial learning, and relates to the technical field of rehabilitation movement identification.The system includes sequentially connected data acquisition module, data preprocessing module, micro-doppler frequency shift enhancement module and rehabilitation movement identification module.The application extracts domain-invariant rehabilitation movement features through cross-domain adversarial training and learning to enhance the robustness and generalizability of the system, and can help nursing staff to master the recovery progress of breast cancer patients after surgery.

Description

Technical Field

[0001] This invention relates to the field of rehabilitation exercise recognition technology, and more specifically to a millimeter-wave postoperative rehabilitation exercise recognition system and method based on adversarial learning. Background Technology

[0002] Breast cancer is one of the most prevalent cancers worldwide, posing a serious threat to the lives and health of numerous women. Surgery, as the primary treatment for breast cancer, involves the removal of axillary lymph nodes. However, this reduces upper limb flexibility. Studies have shown that appropriate and effective postoperative rehabilitation exercises for breast cancer patients are beneficial for the recovery of upper limb function. However, global shortages of healthcare personnel and high medical service costs mean that many patients lack professional supervision in hospital wards or home bedrooms, making it difficult for them to consistently adhere to rehabilitation exercise plans and forcing them to extend their recovery period. Therefore, developing an efficient and reliable method for identifying postoperative rehabilitation exercises for breast cancer patients is of great significance. This method can help caregivers monitor patients' current recovery progress and thus establish future rehabilitation plans.

[0003] Traditional human motion recognition (HAR) methods based on wearable sensors, vision, and sound signals can be used for rehabilitation movement recognition, but their application is limited by shortcomings in user experience, sensing environment, and privacy. For example, wearable sensor-based methods require patients to wear contact sensor devices continuously, which can reduce user comfort and cause negative emotions. Vision-based methods not only require good lighting conditions but also pose a risk of exposing the patient's facial privacy, which is often unacceptable to female patients. In contrast to traditional methods, there has been widespread interest in researching HAR solutions based on wireless devices because wireless signals (such as WiFi, millimeter waves, and RFID) have strong advantages in user experience, sensing environment, and privacy. However, the limited bandwidth of WiFi devices makes it difficult to provide sufficient spatial resolution, resulting in significant limitations for fine-grained HAR. RFID sensing systems require the deployment of large antennas, reducing the deployment flexibility of such solutions in different scenarios. Due to the fine-grained sensing capabilities and ease of deployment of millimeter-wave radar devices, research on millimeter-wave-based HAR methods has become a more promising research direction.

[0004] Existing millimeter-wave-based HAR systems primarily utilize point clouds or feature maps. However, the sparsity of point cloud data imposes stringent requirements on radar hardware. In contrast, feature map-based methods leverage information such as micro-Doppler maps (MDM) for HAR, reducing radar hardware requirements and making them more suitable for real-world scenarios. Specifically, addressing the challenge of recognizing movements in post-mastectomy rehabilitation nursing care performed solely by the upper limbs, previous work utilized range-time maps (RTM) and MDM to effectively recognize different rehabilitation nursing actions. However, previous research revealed that rehabilitation movement features extracted from millimeter-wave signals contain parameters related to the scene and body posture (e.g., standing, sitting, and orientation), meaning the system can only maintain good recognition performance within the training data scenario.

[0005] On the one hand, the millimeter-wave signals collected for rehabilitation exercises include not only reflections from the limbs but also reflections from the environment. On the other hand, the collected millimeter-wave rehabilitation exercise characteristics will also differ when the same user is standing, sitting, or in different body postures. Therefore, in millimeter-wave-based research on rehabilitation exercise recognition after breast cancer surgery, systems trained in specific environments and fixed body postures will experience reduced performance when performing cross-domain rehabilitation exercise recognition (i.e., the same person in different environments or with changes in body posture). These reasons make it difficult for millimeter-wave-based rehabilitation exercise recognition systems for breast cancer surgery to be widely applied in situations involving changes in environment and body posture, and they cannot achieve the performance of systems based on wearable sensors or vision systems.

[0006] Therefore, proposing a millimeter-wave postoperative rehabilitation motion recognition system and method based on adversarial learning to solve the difficulties of existing technologies is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a millimeter-wave postoperative rehabilitation exercise recognition system and method for breast cancer based on adversarial learning. By learning and extracting domain-invariant rehabilitation exercise features through cross-domain adversarial training, the robustness and generalization of the system are enhanced, which can help nursing staff to grasp the postoperative recovery progress of breast cancer patients.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A millimeter-wave postoperative rehabilitation exercise recognition system for breast cancer patients based on adversarial learning includes a data acquisition module, a data preprocessing module, a micro-Doppler frequency shift enhancement module, and a rehabilitation exercise recognition module connected in sequence; wherein, The data acquisition module is used to collect the echo signals reflected by millimeter-wave radar during human rehabilitation exercises; The data preprocessing module is used to preprocess the collected echo signals to generate rehabilitation exercise data; The Micro-Doppler Frequency Shift Enhancement Module is used to enhance rehabilitation exercise data to obtain potential variation patterns of MDFS for different rehabilitation exercises and generate fine-grained feature maps. The rehabilitation exercise recognition module constructs a DCLD-Net network and uses domain adaptive adversarial learning to extract features related to rehabilitation exercises, thereby achieving cross-domain rehabilitation exercise recognition.

[0009] Optionally, the data preprocessing module performs clutter removal and rehabilitation exercise extraction preprocessing on the collected echo signals; Static clutter cancellation is performed on the collected echo signal using a Butterworth high-pass filter. The transfer function of the Butterworth high-pass filter is:

[0010] in, and These represent the frequency and cutoff frequency of the Butterworth filter, respectively. The order of the filter; Extracting rehabilitation movements and obtaining MDFS information of rehabilitation movements using short-time Fourier transform, specifically: This involves processing the radar echo signal after clutter removal. A one-dimensional fast Fourier transform is performed in the fast time dimension to obtain a distance-time two-dimensional matrix; the data between the maximum and minimum distance dimensions of the target unit are accumulated and added together, and the accumulated distance dimension data is subjected to an STFT operation to finally obtain rehabilitation exercises for different rehabilitation movements. For the input information, the STFT calculation process is as follows:

[0011] in, It is a window function. Indicates from The slow time series extracted from a certain distance cell, where t represents the time offset factor of the window function.

[0012] Optionally, the micro-Doppler frequency shift enhancement module adopts a rehabilitation motion enhancement algorithm based on an autoencoder network, following an encoder-decoder framework, including an input module, a noise generation module, an encoder module, a decoder module, and an output module connected in sequence. The noise generation module assumes that the input rehabilitation exercise data is The module output is ,in, This refers to rehabilitation exercises that address noise pollution. Indicates Gaussian noise. This represents the standard deviation of Gaussian noise; The encoder module consists of three 2D convolutional layers and three 2D max pooling layers. Through the encoder module, Abstracted into latent feature space , representing different characteristics of rehabilitation exercise data, namely, ,in, These are weight parameters; The decoder module consists of three upsampling layers and three two-dimensional convolutional layers. It utilizes the decoder module to extract features from the feature space. The rehabilitation exercise data input in the middle is reconstructed and represented as ,in, This represents the reconstructed rehabilitation exercise data. These are weight parameters; Also includes loss function This is used to measure the initial rehabilitation exercise data input. and post-reconstruction rehabilitation exercise data The difference between them, by minimizing prompt and Minimize the differences between features to ensure the feature space MDFS characteristics representing different rehabilitation exercises, loss function Specifically:

[0013] in, Indicates batch size; Achieved through backpropagation during training. Minimize the training process, which is performed only once. After training, the test data is input into the trained autoencoder network to enhance rehabilitation exercises, i.e., extract the latent features of MDFS. .

[0014] Optionally, the DCLD-Net network constructed in the rehabilitation movement recognition module includes: a feature extractor, a rehabilitation movement classifier, and a domain discriminator; The feature extractor aims to learn domain-invariant features from enhanced rehabilitation movements. It utilizes a Deep-3DCNN-xLSTM network as the feature extractor. The Deep-3DCNN consists of three convolutional layers of different sizes: 32... 3 3.64 3 3, 128 3 3. To extract spatial features of rehabilitation exercises, the xLSTM is composed of three residual blocks: mLSTM, sLSTM, and mLSTM stacked together. This is used to obtain the global temporal features of the rehabilitation exercises. Finally, the feature output by the feature extractor is defined as... ; The rehabilitation exercise classifier consists of three fully connected layers, and features are extracted using a feature extractor. Afterwards, Feed to the rehabilitation exercise classifier ; The loss function for the rehabilitation exercise classifier is defined as follows: it uses cross-entropy to calculate the difference between the predicted rehabilitation exercise category and the true category.

[0015] in, and Let these represent the true category and the predicted category of the rehabilitation action for the i-th sample, respectively. Indicates the number of samples in the source domain. Indicates the number of categories of rehabilitation exercises; Domain discriminator It consists of three fully connected layers, setting the target domain data label to 0 and the source domain data label to 1 to distinguish whether the data comes from or does not come from the same domain; The loss function of the domain discriminant is defined using the cross-entropy function:

[0016] in, and They represent the first The true domain category and the predicted domain category of each sample Indicates the number of samples. The number of categories representing the domain, .

[0017] Optionally, during the DCLD-Net network training phase, a feature extractor can be used to extract features from the input data. The data are fed into the rehabilitation exercise classifier. Domain discriminant In China, rehabilitation exercise classifier Domain discriminant distinguishes between different types of rehabilitation exercises in different domains. The system infers the domain origin of the input data, and the adversarial learning strategy reverses the loss through gradients during backpropagation. The feature extractor is trained to ignore domain-dependent features, enabling the DCLD-Net network to learn domain-invariant rehabilitation motion features. The parameters of the feature extractor, rehabilitation exercise classifier, and domain discriminator are respectively represented as follows:

[0018] in, These represent the optimal parameters for the feature extractor, rehabilitation exercise classifier, and domain discriminator, respectively. DCLD-Net overall loss function Represented as:

[0019] in, This represents a classifier for balance rehabilitation exercises. Domain discriminant Hyperparameters.

[0020] A millimeter-wave post-mastectomy rehabilitation exercise recognition method based on adversarial learning, comprising the following steps, implementing any of the above-mentioned millimeter-wave post-mastectomy rehabilitation exercise recognition systems: Collect echo signals reflected by millimeter-wave radar during human rehabilitation exercises; The collected echo signals are preprocessed to generate rehabilitation exercise data; Enhance rehabilitation exercise data to obtain potential variation patterns of MDFS for different rehabilitation exercises and generate fine-grained feature maps; A DCLD-Net network is constructed, and domain adaptive adversarial learning is used to extract features related to rehabilitation movements, thereby achieving cross-domain rehabilitation movement recognition.

[0021] As can be seen from the above technical solution, compared with the prior art, the present invention provides a millimeter-wave postoperative breast cancer rehabilitation exercise recognition system and method based on adversarial learning, which has the following beneficial effects: (1) This invention learns and extracts domain-invariant rehabilitation movement features through cross-domain adversarial training to enhance the robustness and generalization of the system, which can help nursing staff grasp the postoperative recovery progress of breast cancer patients. (2) The present invention is based on a robust and generalized rehabilitation motion recognition system based on millimeter waves, namely mm-CSRAR. It uses commercial millimeter wave radar to acquire the raw data of rehabilitation motion and performs noise reduction processing on it; then, an autoencoder network is used to enhance the extracted MDM; finally, adversarial learning is used to achieve cross-domain rehabilitation motion recognition; through extensive experiments on different indoor scenes and different user body postures, it is proved that the mm-CSRAR system has good cross-domain performance, which can help breast cancer patients recover, reduce the workload of nursing staff, and reduce the cost of medical services. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1 A block diagram of a millimeter-wave postoperative breast cancer rehabilitation motion recognition system based on adversarial learning provided by the present invention; Figure 2 Comparison diagrams of wrist rotation MDM clutter removal before and after provided by the present invention, wherein (a) is the original MDM and (b) is the MDM after clutter removal; Figure 3 The present invention provides a structural diagram of the micro-Doppler frequency shift enhancement module; Figure 4 The DCLD-Net network structure diagram provided by this invention; Figure 5 The following is a diagram of the Deep-3DCNN architecture provided by the present invention, wherein (a) is a schematic diagram of 3DCNN and (b) is an architecture diagram of Deep-3DCNN; Figure 6 The diagram shows the xLSTM residual structure provided by this invention. Figure 7 This invention provides schematic diagrams of different data acquisition scenarios; Figure 8 This invention provides a schematic diagram of the collected rehabilitation exercise movements; Figure 9 This is a schematic diagram illustrating the average accuracy of rehabilitation movement recognition under all body postures in various scenarios provided by the present invention. Figure 10 This is a schematic diagram illustrating the average accuracy of rehabilitation movement recognition under different body postures provided by the present invention; where 10a represents standing, 10b represents sitting, 10c represents left side posture, and 10d represents right side posture. Figure 11 This is a schematic diagram of the ablation experiment results provided by the present invention; Figure 12 This is a schematic diagram illustrating the performance of the system provided by the present invention under new environments and body postures. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] See Figure 1 As shown, this invention discloses a millimeter-wave postoperative rehabilitation exercise recognition system for breast cancer based on adversarial learning, comprising a data acquisition module, a data preprocessing module, a micro-Doppler frequency shift enhancement module, and a rehabilitation exercise recognition module connected in sequence; wherein, The data acquisition module is used to collect the echo signals reflected by millimeter-wave radar during human rehabilitation exercises; The data preprocessing module is used to preprocess the collected echo signals to generate rehabilitation exercise data; The Micro-Doppler Frequency Shift Enhancement Module is used to enhance rehabilitation exercise data to obtain potential variation patterns of MDFS for different rehabilitation exercises and generate fine-grained feature maps. The rehabilitation exercise recognition module constructs a DCLD-Net network and uses domain adaptive adversarial learning to extract features related to rehabilitation exercises, thereby achieving cross-domain rehabilitation exercise recognition.

[0026] Specifically, the data acquisition module utilizes the TI AWR1843 radar to capture echo data from different rehabilitation exercises. The AWR1843 radar is equipped with three transmitting antennas and four receiving antennas. During data acquisition, the radar's operating mode is set to one antenna transmitting signals and four antennas receiving signals. The AWR1843 is a frequency modulated continuous wave (FMCW) millimeter-wave radar that achieves target detection by transmitting and receiving chirp signals. Assuming the distance between the radar and the human target is *d*, the transmitted chirp signal is reflected by the human body, and the intermediate frequency (IF) signal obtained after mixing is represented as follows:

[0027] in, This indicates the amplitude change of the radar received signal; It is the frequency of the IF signal, i.e. B is the signal bandwidth, T is the period of the chirp signal, and c is the speed of light; It is the phase of the IF signal, i.e. , It is the wavelength of the signal; In practical applications, the acquired IF signal is:

[0028] in, and These represent the sampling sequences for fast and slow times, respectively. Indicates the number of chirp signals transmitted. m This indicates the number of samples for each chirp signal; and These represent the sampling intervals for fast and slow time, respectively; a frame of radar data acquired can be represented as... m The three-dimensional data block is l, where l represents the number of virtual antennas; then, Fast Fourier Transform (FFT) is performed in the three dimensions of fast time, slow time and channel respectively, which can obtain the target's distance, velocity and angle information, and generate feature maps such as RTM and MDM.

[0029] Furthermore, the data preprocessing module performs clutter removal and rehabilitation exercise extraction preprocessing on the collected echo signals; Static clutter cancellation is performed on the collected echo signal using a Butterworth high-pass filter. The transfer function of the Butterworth high-pass filter is:

[0030] in, and These represent the frequency and cutoff frequency of the Butterworth filter, respectively. The order of the filter; Extracting rehabilitation movements and obtaining MDFS information of rehabilitation movements using short-time Fourier transform, specifically: This involves processing the radar echo signal after clutter removal. A one-dimensional fast Fourier transform is performed in the fast time dimension to obtain a distance-time two-dimensional matrix; the data between the maximum and minimum distance dimensions of the target unit are accumulated and added together, and the accumulated distance dimension data is subjected to an STFT operation to finally obtain rehabilitation exercises for different rehabilitation movements. For the input information, the STFT calculation process is as follows:

[0031] in, It is a window function. Indicates from The slow time series extracted from a certain distance cell, where t represents the time offset factor of the window function.

[0032] Specifically, Figure 2The image shows a comparison of the MDM before and after clutter removal during wrist rotation. (a) is the original MDM, and (b) is the MDM after clutter removal. It is clear that before the filtering operation, the MDM has a large amount of static clutter and background noise. After filtering, the noise in the MDM is effectively filtered out, better preserving the key information of the MDFS changes in rehabilitation exercise.

[0033] Furthermore, the micro-Doppler frequency shift enhancement module adopts a rehabilitation motion enhancement algorithm based on an autoencoder network, following an encoder-decoder framework, which includes an input module, a noise generation module, an encoder module, a decoder module, and an output module connected in sequence. The noise generation module assumes that the input rehabilitation exercise data is The module output is ,in, This refers to rehabilitation exercises that address noise pollution. Indicates Gaussian noise. This represents the standard deviation of Gaussian noise; The encoder module consists of three 2D convolutional layers and three 2D max pooling layers. Through the encoder module, Abstracted into latent feature space , representing different characteristics of rehabilitation exercise data, namely, ,in, These are weight parameters; The decoder module consists of three upsampling layers and three two-dimensional convolutional layers. It utilizes the decoder module to extract features from the feature space. The rehabilitation exercise data input in the middle is reconstructed and represented as ,in, This represents the reconstructed rehabilitation exercise data. These are weight parameters; Also includes loss function This is used to measure the initial rehabilitation exercise data input. and post-reconstruction rehabilitation exercise data The difference between them, by minimizing prompt and Minimize the differences between features to ensure the feature space MDFS characteristics representing different rehabilitation exercises, loss function Specifically:

[0034] in, Indicates batch size; Achieved through backpropagation during training. Minimize the training process, which is performed only once. After training, the test data is input into the trained autoencoder network to enhance rehabilitation exercises, i.e., extract the latent features of MDFS. .

[0035] Specifically, such as Figure 3 As shown, the micro-Doppler frequency shift enhancement module includes an input module, a noise generation module, an encoder module, a decoder module, and an output module connected in sequence. Noise Generation Module: To further reduce environmental and device noise interference, a noise generation module was constructed. This module helps the autoencoder network acquire more robust features less affected by noise. In this module, environmental and device noise is simulated as additive Gaussian noise, assuming the input MDM data is... The module output is ,in Indicates an MDM subjected to noise. Indicates Gaussian noise. This represents the standard deviation of Gaussian noise; Encoder Module: The encoder consists of three 2D convolutional layers and three 2D max-pooling layers, where the kernel size of the three Conv2D layers is 4. 4. The output channels are 32, 16, and 8 respectively, the activation function is ReLU, and the size of each of the three MaxPooling2D channels is set to 2. 2. Through the encoder module, Abstracted into latent feature space It is represented as different characteristics of MDM data, and the above process can be represented as follows: ,in, These are weight parameters; Decoder Module: The decoder consists of three upsampling layers and three 2D convolutional layers, with the size of each of the three upsampling layers set to 2. 2. The kernel size of all three Conv2D convolutions is 4. 4. The output channels are 8, 16, and 32 respectively, with ReLU activation function. The decoder module is used to extract the data from the feature space. The process of reconstructing the input MDM data is represented as follows: ,in, This represents the reconstructed MDM data. These are weight parameters.

[0036] Furthermore, the DCLD-Net network constructed in the rehabilitation exercise recognition module includes: a feature extractor, a rehabilitation exercise classifier, and a domain discriminator; The feature extractor aims to learn domain-invariant features from enhanced rehabilitation movements. It utilizes a Deep-3DCNN-xLSTM network as the feature extractor. The Deep-3DCNN consists of three convolutional layers of different sizes: 32... 3 3, 64 3 3, 128 3 3. To extract spatial features of rehabilitation exercises, the xLSTM is composed of three stacked residual blocks: mLSTM, sLSTM, and mLSTM. This is used to obtain global temporal features of the rehabilitation exercises. Finally, the feature extractor outputs the following feature definition: ; The rehabilitation exercise classifier consists of three fully connected layers, and features are extracted using a feature extractor. Afterwards, Feed to the rehabilitation exercise classifier ; The loss function for the rehabilitation exercise classifier is defined as follows: it uses cross-entropy to calculate the difference between the predicted rehabilitation exercise category and the true category.

[0037] in, and Let these represent the true category and the predicted category of the rehabilitation action for the i-th sample, respectively. Indicates the number of samples in the source domain. Indicates the number of categories of rehabilitation exercises; Domain discriminator It consists of three fully connected layers, setting the target domain data label to 0 and the source domain data label to 1 to distinguish whether the data comes from or does not come from the same domain; The loss function of the domain discriminant is defined using the cross-entropy function:

[0038] in, and They represent the first The true domain category and the predicted domain category of each sample Indicates the number of samples. The number of categories representing the domain, .

[0039] Specifically, such as Figure 4 As shown, a specialized domain-adaptive network, DCLD-Net, was developed, which consists of three key components: a feature extractor, a rehabilitation motion classifier, and a domain discriminator, as detailed below: Feature Extractor: The feature extractor aims to learn domain-invariant features from the enhanced MDM. Considering that the MDM contains spatiotemporal information of rehabilitation movements, a Deep-3DCNN-xLSTM network is used as the feature extractor. The input data size is defined as T (number of frames) × C (channels) × H (height) × W (width), where the H and W dimensions contain spatial features of rehabilitation movements, the T dimension contains temporal features, and C is 1. Figure 5 As shown, Deep-3DCNN consists of three convolutional layers of different sizes, namely 32... 3 3.64 3 3, 128 3 3. They were used to extract the spatial features of rehabilitation exercises; such as Figure 6 As shown, xLSTM is composed of three residual blocks (i.e., mLSTM, sLSTM, and mLSTM) stacked together. It is used to obtain global temporal features of rehabilitation movements. Finally, the features output by the feature extractor are defined as follows: ; Rehabilitation exercise classifier: extracting features using a feature extractor Afterwards, Feed to the rehabilitation exercise classifier It consists of three fully connected (FC) layers, each with a hidden size of 512. The first two FC layers use the ReLU activation function, and the last FC layer uses the softmax function to calculate the predicted rehabilitation exercise category. Simultaneously, the loss function for the rehabilitation exercise classifier is defined, which uses cross-entropy to calculate the difference between the predicted and true rehabilitation exercise categories, given by the following formula:

[0040] in, and Let these represent the true category and the predicted category of the rehabilitation action for the i-th sample, respectively. Indicates the number of samples in the source domain. Indicates the number of categories of rehabilitation exercises; Domain Discriminator: Constructing a Domain Discriminator Its structure is the same as that of the rehabilitation exercise classifier. The domains represent different environments and user body postures from which the data originates. This complexity increases the difficulty of obtaining specific domain labels for the data. Therefore, these specific domain labels are ignored; only the target domain data label is set to 0, and the source domain data label is set to 1, to distinguish whether the data comes from or does not come from the same domain. Finally, the cross-entropy function is also used to define the loss function of the domain discriminator.

[0041] in, and Let these represent the true domain class and the predicted domain class of the i-th sample, respectively. Indicates the number of samples. Indicates the number of categories in the domain. .

[0042] Furthermore, during the DCLD-Net network training phase, a feature extractor is used to extract features from the input data. The data are fed into the rehabilitation exercise classifier. Domain discriminant In China, rehabilitation exercise classifier Domain discriminant distinguishes between different types of rehabilitation exercises in different domains. The system infers the domain origin of the input data, and the adversarial learning strategy reverses the loss through gradients during backpropagation. The feature extractor is trained to ignore domain-dependent features, enabling the DCLD-Net network to learn domain-invariant rehabilitation motion features. The parameters of the feature extractor, rehabilitation exercise classifier, and domain discriminator are respectively represented as follows:

[0043] in, These represent the optimal parameters for the feature extractor, rehabilitation exercise classifier, and domain discriminator, respectively. DCLD-Net overall loss function Represented as:

[0044] in, This represents a classifier for balance rehabilitation exercises. Domain discriminant Hyperparameters.

[0045] In one specific embodiment, the hardware implementation, experimental setup, performance evaluation, and analysis of the mm-CSRAR system (rehabilitation motion recognition system) are included.

[0046] 1. Hardware and Software

[0047] Hardware: The TI AWR1843 FMCW radar is used, which achieves millimeter-wave data acquisition by connecting to the DCA1000EVM data acquisition card. The specific parameters of the radar configuration when acquiring millimeter-wave signals for rehabilitation exercises are shown in Table 1.

[0048] Table 1 TI AWR1843 FMCW Parameter Settings

[0049] Software: The raw radar data was preprocessed using PyCharm 2024. Simultaneously, the autoencoder network model and the DCLD-Net model were trained on a PC equipped with an RTX 3090 Ti graphics card and an Intel Core i9-13900K CPU. All models were optimized using the Adam optimizer with an initial learning rate of 0.001. The autoencoder network model was trained with a batch size of 30 for 30 epochs. The DCLD-Net model was trained with a batch size of 30 for 200 epochs.

[0050] 2. Experimental Setup

[0051] Rehabilitation exercise data (MDM data) were collected from 20 female subjects aged 24 to 52 years and with heights between 1.55m and 1.75m to evaluate the mm-CSRAR system. Figure 7 As shown, data were collected from each subject in four different indoor scenarios. The same experimental setup was maintained in each scenario. Specifically, the radar device was fixed on a tripod at a height of approximately 1.3m above the ground, and the subject was positioned approximately 1.5m directly in front of the radar.

[0052] During data collection in each scenario, subjects were instructed to perform the same rehabilitation exercise in four common body postures (standing, sitting, turning to the left, and turning to the right). The turning posture involved the subject standing with a 45° semi-turn to face the radar. Five samples were collected for each rehabilitation exercise, with each sample containing 200 frames of radar data and nine rehabilitation movements being recorded, such as... Figure 8 As shown in the figure. Ultimately, 20 subjects collected a total of 14,400 samples across four scenarios. 70% of this sample data was used for training, and the remaining 30% was used for testing.

[0053] 3. Overall performance

[0054] First, assess the overall performance of the mm-CSRAR system. Figure 9 The average accuracy of rehabilitation movement recognition for subjects in different body postures within each scenario is presented. It can be seen that the mm-CSRAR system performs excellently in various environments. The average accuracy rates in the four environments (S1: entrance hallway environment; S2: central living room environment; S3: ward environment; S4: living room sofa environment) are 98.6%, 97.8%, 97%, and 98.3%, respectively, with very small performance differences and an overall accuracy of 97.9%. These results demonstrate that the mm-CSRAR system possesses cross-domain rehabilitation movement recognition capabilities.

[0055] Next, a confusion matrix is ​​used to more intuitively demonstrate system performance. Taking the S3 ward environment as an example, Figure 10 The results of recognizing rehabilitation movements in different body postures of the subjects in the S3 ward environment were presented. 10a was standing, 10b was sitting, 10c was left-side turning, and 10d was right-side turning. Specifically, the mm-CSRAR system maintained an average accuracy of over 97% across all body postures. Furthermore, the accuracy of recognizing any single category of rehabilitation movement was over 96%, reaching a maximum of 100%. This further validates the cross-domain capability of the mm-CSRAR system.

[0056] 4. Ablation test

[0057] To investigate the effectiveness of the micro-Doppler frequency shift enhancement module in the mm-CSRAR system, ablation experiments were conducted. Specifically, two variants were designed: 1) the system uses an autoencoder network (AN) and 2) the system disables the autoencoder network (w / oAN). For convenience, ablation experiments were only performed on the datasets S1 (entrance hallway environment) and S2 (living room central environment). Figure 11 Specific experimental results are presented. It can be observed that without AN (Autoencoder Network), the system's performance in recognizing rehabilitation movements in different scenarios significantly decreases, with the overall average accuracy dropping from 97.9% to 88.1%. When AN is added to the system, the accuracy of cross-domain rehabilitation movement recognition significantly increases. These results verify that the designed autoencoder network can enhance the MDM (Multi-Device Model) of rehabilitation movements and effectively extract the latent features of different MDFS (Multi-Device Models of Rehabilitation Movements). Therefore, the feature enhancement module of the mm-CSRAR system is crucial for improving the performance of cross-domain rehabilitation movement recognition.

[0058] 5. System Performance Analysis

[0059] Influence of Environment and Body Posture: To investigate the effects of environment and body posture on the mm-CSRAR system, rehabilitation movement data were collected from five subjects in different body postures in a new room and input into the mm-CSRAR system for recognition. Notably, these subjects were asked to incorporate two new body postures: random slight left turn and slight right turn. Unlike the previous setup, this slight turn involved a half-turn angle of less than 45°. For convenience, the different body postures are abbreviated as follows: Standing - St, Sitting - Si, Left turn - LS, Right turn - RS, Slight left turn - SLS, Slight right turn - SRS. Figure 12The results of recognizing rehabilitation movements in subjects under new conditions are presented. It can be observed that, under these new conditions, for subjects with known body postures, the system achieves an average accuracy rate of over 96% in recognizing rehabilitation movements. Even with unknown body postures, the system maintains an accuracy rate of over 90%. This demonstrates that the mm-CSRAR system can learn domain-invariant features of rehabilitation movements, thus exhibiting good cross-domain performance.

[0060] A millimeter-wave post-mastectomy rehabilitation exercise recognition method based on adversarial learning, comprising the following steps, implementing any of the above-mentioned millimeter-wave post-mastectomy rehabilitation exercise recognition systems: Collect echo signals reflected by millimeter-wave radar during human rehabilitation exercises; The collected echo signals are preprocessed to generate rehabilitation exercise data; Enhance rehabilitation exercise data to obtain potential variation patterns of MDFS for different rehabilitation exercises and generate fine-grained feature maps; A DCLD-Net network is constructed, and domain adaptive adversarial learning is used to extract features related to rehabilitation movements, thereby achieving cross-domain rehabilitation movement recognition.

[0061] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Regarding the methods disclosed in the embodiments, since they correspond to the systems disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the system section description.

[0062] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A millimeter-wave post-mastectomy breast cancer rehabilitation motion recognition system based on adversarial learning, characterized in that, It includes a data acquisition module, a data preprocessing module, a micro-Doppler frequency shift enhancement module, and a rehabilitation motion recognition module connected in sequence; among them, The data acquisition module is used to collect the echo signals reflected by millimeter-wave radar during human rehabilitation exercises; The data preprocessing module is used to preprocess the collected echo signals to generate rehabilitation exercise data; The Micro-Doppler Frequency Shift Enhancement Module is used to enhance rehabilitation exercise data to obtain potential variation patterns of MDFS for different rehabilitation exercises and generate fine-grained feature maps. The rehabilitation exercise recognition module constructs a DCLD-Net network and uses domain adaptive adversarial learning to extract features related to rehabilitation exercises, thereby achieving cross-domain rehabilitation exercise recognition.

2. The millimeter-wave post-mastectomy breast cancer rehabilitation motion recognition system based on adversarial learning according to claim 1, characterized in that, The data preprocessing module performs clutter removal and rehabilitation exercise extraction preprocessing on the collected echo signals; Static clutter cancellation is performed on the collected echo signal using a Butterworth high-pass filter. The transfer function of the Butterworth high-pass filter is: in, and These represent the frequency and cutoff frequency of the Butterworth filter, respectively. The order of the filter; Extracting rehabilitation movements and obtaining MDFS information of rehabilitation movements using short-time Fourier transform, specifically: This involves processing the radar echo signal after clutter removal. A one-dimensional fast Fourier transform is performed in the fast time dimension to obtain a distance-time two-dimensional matrix; the data between the maximum and minimum distance dimensions of the target unit are accumulated and added together, and the accumulated distance dimension data is subjected to an STFT operation to finally obtain rehabilitation exercises for different rehabilitation movements. For the input information, the STFT calculation process is as follows: in, It is a window function. Indicates from The slow time series extracted from a certain distance cell, t This represents the time offset factor of the window function.

3. The millimeter-wave post-mastectomy breast cancer rehabilitation motion recognition system based on adversarial learning according to claim 1, characterized in that, The micro-Doppler frequency shift enhancement module adopts a rehabilitation motion enhancement algorithm based on an autoencoder network, following an encoder-decoder framework, which includes an input module, a noise generation module, an encoder module, a decoder module, and an output module connected in sequence. The noise generation module assumes that the input rehabilitation exercise data is The module output is ,in, This refers to rehabilitation exercises that address noise pollution. Indicates Gaussian noise. This represents the standard deviation of Gaussian noise; The encoder module consists of three 2D convolutional layers and three 2D max pooling layers. Through the encoder module, Abstracted into latent feature space , representing different characteristics of rehabilitation exercise data, namely, ,in, These are weight parameters; The decoder module consists of three upsampling layers and three two-dimensional convolutional layers. It utilizes the decoder module to extract features from the feature space. The rehabilitation exercise data input in the middle is reconstructed and represented as ,in, This represents the reconstructed rehabilitation exercise data. These are weight parameters; Also includes loss function This is used to measure the initial rehabilitation exercise data input. and post-reconstruction rehabilitation exercise data The difference between them, by minimizing prompt and Minimize the differences between features to ensure the feature space MDFS characteristics representing different rehabilitation exercises, loss function Specifically: in, Indicates batch size; Achieved through backpropagation during training. Minimize the training process, which is performed only once. After training, the test data is input into the trained autoencoder network to enhance rehabilitation exercises, i.e., extract the latent features of MDFS. .

4. The millimeter-wave post-mastectomy rehabilitation motion recognition system based on adversarial learning according to claim 1, characterized in that, The DCLD-Net network constructed in the rehabilitation exercise recognition module includes: a feature extractor, a rehabilitation exercise classifier, and a domain discriminator; The feature extractor aims to learn domain-invariant features from enhanced rehabilitation movements. It utilizes a Deep-3DCNN-xLSTM network as the feature extractor. The Deep-3DCNN consists of three convolutional layers of different sizes: 32... 3 3.64 3 3, 128 3 3. To extract spatial features of rehabilitation exercises, the xLSTM is composed of three stacked residual blocks: mLSTM, sLSTM, and mLSTM. This is used to obtain global temporal features of the rehabilitation exercises. Finally, the feature extractor outputs the following feature definition: ; The rehabilitation exercise classifier consists of three fully connected layers, and features are extracted using a feature extractor. Afterwards, Feed to the rehabilitation exercise classifier ; The loss function for the rehabilitation exercise classifier is defined as follows: it uses cross-entropy to calculate the difference between the predicted rehabilitation exercise category and the true category. in, and Let these represent the true category and the predicted category of the rehabilitation action for the i-th sample, respectively. Indicates the number of samples in the source domain. Indicates the number of categories of rehabilitation exercises; Domain discriminator It consists of three fully connected layers, setting the target domain data label to 0 and the source domain data label to 1 to distinguish whether the data comes from or does not come from the same domain; The loss function of the domain discriminant is defined using the cross-entropy function: in, and They represent the first The true domain category and the predicted domain category of each sample. Indicates the number of samples. The number of categories representing the domain, .

5. The millimeter-wave post-mastectomy breast cancer rehabilitation motion recognition system based on adversarial learning according to claim 1, characterized in that, During the training phase of the DCLD-Net network, a feature extractor is used to extract features from the input data. The data are fed into the rehabilitation exercise classifier. Domain discriminant In China, rehabilitation exercise classifier Domain discriminant distinguishes between different types of rehabilitation exercises in different domains. The system infers the domain origin of the input data, and the adversarial learning strategy reverses the loss through gradients during backpropagation. The feature extractor is trained to ignore domain-dependent features, enabling the DCLD-Net network to learn domain-invariant rehabilitation movement features. The parameters of the feature extractor, rehabilitation exercise classifier, and domain discriminator are respectively represented as follows: in, These represent the optimal parameters for the feature extractor, rehabilitation exercise classifier, and domain discriminator, respectively. DCLD-Net overall loss function Represented as: in, This represents a classifier for balance rehabilitation exercises. Domain discriminant Hyperparameters.

6. A millimeter-wave post-mastectomy rehabilitation exercise recognition method based on adversarial learning, characterized in that, The millimeter-wave postoperative rehabilitation motion recognition system for breast cancer based on adversarial learning, as described in any one of claims 1-5, comprises the following steps: Collect echo signals reflected by millimeter-wave radar during human rehabilitation exercises; The collected echo signals are preprocessed to generate rehabilitation exercise data; Enhance rehabilitation exercise data to obtain potential variation patterns of MDFS for different rehabilitation exercises and generate fine-grained feature maps; A DCLD-Net network is constructed, and domain adaptive adversarial learning is used to extract features related to rehabilitation movements, thereby achieving cross-domain rehabilitation movement recognition.

Images