Feature interaction rebalance-based gait analysis disease diagnosis method and system

Abstract

The invention belongs to the technical field of disease diagnosis based on gait analysis, and discloses a gait analysis disease diagnosis method and system based on feature interaction rebalance, and the method comprises the steps: carrying out the feature extraction of gait data, obtaining a local feature and a global feature, carrying out the cross enhancement of the local feature and the global feature, and carrying out the splicing operation of a result. Performing classification prediction by using a splicing result to obtain a classification result, calculating cross entropy loss between the classification result and a real label, calculating prototype cross entropy loss between the weighted local feature and a prototype vector as well as between the weighted global feature and the prototype vector, and calculating similarity loss between a similarity matrix and a transpose matrix thereof and the real label; and performing weighted combination on the loss to obtain a total loss function of the model, and training the model. According to the method, the intra-joint coupling relation and the inter-joint coupling relation are learned at the same time, cross-domain feature distribution differences are reduced by aligning cross-domain features, meanwhile, the fitting speed of a cross-domain extractor is balanced, and effective and objective gait-based reference is provided for disease diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of disease diagnosis based on gait analysis, and in particular to a gait analysis disease diagnosis method and system based on feature interaction rebalancing. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Gait analysis is an important auxiliary method for effectively diagnosing diseases with abnormal gait characteristics, which are often misdiagnosed in clinical practice. Given the complexity of multivariate gait time series, deep learning methods have been widely used in disease diagnosis using gait signal analysis. Although current deep learning methods have greatly improved the accuracy of gait analysis tasks compared to traditional methods, there are still some challenges to overcome. Challenge 1: Complex coupling relationships between and within joints (e.g. Figure 1 Challenge 2: In the process of multi-feature fusion, due to the different fitting speeds of each feature extraction process, they will interfere with each other (such as Figure 2 As shown in Figure 1). For Challenge 1, existing graph neural networks (GNNs) consider coupling across joints, but they face a key shortcoming: they ignore the coupling features within a single joint, which is crucial for disease differentiation. For Challenge 2, the state-of-the-art Prototype Modality Rebalancing (PMR) algorithm accelerates the slower fitting process while evaluating the fit of the classifier by comparing the distance between features and prototypes in real time. However, the performance of PMR suffers from a limitation: it lacks the interaction between cross-domain features, resulting in a distribution gap. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a gait analysis disease diagnosis method and system based on feature interaction rebalancing, which simultaneously learns the intra-joint and inter-joint coupling relationships, reduces the cross-domain feature distribution differences by aligning cross-domain features, and balances the fitting speed of the cross-domain extractor, providing an effective and objective gait-based reference for disease diagnosis.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a gait analysis disease diagnosis method based on feature interaction rebalancing, comprising the following steps:

[0007] Obtain gait data samples, divide them into multiple batches, and perform feature extraction on the gait data samples of each batch to obtain local features and global features;

[0008] Perform cross-enhancement operations on local features and global features to obtain weighted local features and weighted global features;

[0009] Perform concatenation operation on weighted local features and weighted global features to obtain fused features, use the fused features to perform classification prediction, obtain classification results, calculate the cross entropy loss between the classification results and the true labels, obtain prototype vectors of weighted local features and weighted global features respectively, calculate the prototype cross entropy loss between weighted local features and weighted global features and prototype vectors respectively, calculate the similarity matrix of weighted local features and weighted global features of each batch, and calculate the similarity loss between the similarity matrix and its transposed matrix and the true labels respectively;

[0010] The above losses are weighted and combined to obtain the total loss function of the model, and the model is trained.

[0011] As an optional implementation, the local feature is the coupling within a joint, that is, the relationship between three degrees of freedom within a joint, and the global feature is the coupling between multiple joints.

[0012] As an optional implementation, a cross-enhancement operation is performed on the local features and the global features, specifically:

[0013] For local features and global features, one of the feature sequences is used to provide the query sequence, and the other feature sequence provides the key and value sequence. By calculating the similarity between the Query sequence and the Key sequence, the attention distribution between different sequences is obtained. The attention distribution is used to perform weighted summation on the value sequence to obtain weighted local features and weighted global features.

[0014] As an optional implementation, a local two-dimensional residual module is used to capture the coupling within a joint, and a global dynamic graph learning module is used to capture the coupling between joints.

[0015] As an optional implementation, the prototype cross entropy loss can evaluate the imbalance between two feature extractors, and the similarity loss enables the model to learn feature representations that reflect similarity relationships in the data.

[0016] As an optional implementation, the total loss function is:

[0017]

[0018] in, is the cross entropy loss, and are the prototype cross entropy losses of the two feature extractors, is the similarity loss, α is a hyperparameter that controls the equilibrium fitting speed, ε is a hyperparameter for similarity modulation, and β and γ are coefficients.

[0019] In a second aspect, the present invention provides a gait analysis disease diagnosis system based on feature interaction rebalancing, comprising:

[0020] The feature extraction module is configured to: obtain gait data samples, divide them into multiple batches, and perform feature extraction on the gait data samples of each batch to obtain local features and global features;

[0021] The cross enhancement module is configured to: perform a cross enhancement operation on the local features and the global features to obtain weighted local features and weighted global features;

[0022] The loss calculation module is configured to: perform a splicing operation on the weighted local features and the weighted global features to obtain fused features, use the fused features to perform classification prediction, obtain classification results, calculate the cross entropy loss between the classification results and the true labels, respectively obtain the prototype vectors of the weighted local features and the weighted global features, respectively calculate the prototype cross entropy loss between the weighted local features and the weighted global features and the prototype vectors, calculate the similarity matrix of the weighted local features and the weighted global features of each batch, and respectively calculate the similarity loss between the similarity matrix and its transposed matrix and the true labels;

[0023] The model training module is configured to: perform weighted combination of the above losses to obtain the total loss function of the model and train the model.

[0024] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is performed.

[0025] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0026] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] The present disclosure proposes a gait analysis disease diagnosis method and system based on feature interaction rebalancing, and systematically studies the challenges of gait-based disease diagnosis: complex coupling relationships and cross-domain fusion interference. A global-local interactive rebalancing gait network (GLIR-GaitNet) is proposed to solve these problems. Among them, it includes two modules: "global-local joint coupling feature extractor (GL-JCFE)" and "prototype interactive rebalancing (PIR)". The GL-JCFE module integrates a dynamic graph learning module and a two-dimensional residual module. The dynamic graph learning module is used to extract global coupling features across joints, and the two-dimensional residual module is used to extract local coupling features between the three degrees of freedom in the joint. The combination of the two is intended to capture the comprehensive coupling across joints and within joints. It solves the problem that complex coupling between and within joints is difficult to extract. In addition, the GL-JCFE module also introduces MCE to assign appropriate weights to cross-domain features, enhance the complementarity of cross-domain features, and reduce the feature redundancy generated by two parallel networks.

[0029] The introduction of cross-domain similarity supervision in the PIR module helps to learn shared feature representations, thereby reducing information conflicts and noise interference, and ultimately improving the model's ability to understand and interpret data. It reduces the difference in feature distribution and balances the convergence speed of the cross-domain feature extraction network, thereby promoting feature integration. It solves the problem that the existing gait-based disease diagnosis methods interfere with each other during the multi-feature fusion process due to the different fitting speeds of each feature extraction process. Experiments based on self-collected cervical spondylotic myelopathy (CSM) and Parkinson's syndrome (PS) datasets show that our method significantly improves disease diagnosis capabilities.

[0030] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0032] Figure 1 A schematic diagram of Challenge 1 in the background technology of the present invention;

[0033] Figure 2 A schematic diagram of Challenge 2 in the background technology of the present invention;

[0034] Figure 3 A framework diagram of the GL-JCFE module provided in Example 1 of the present invention;

[0035] Figure 4 A framework diagram of a PIR module provided in Embodiment 1 of the present invention;

[0036] Figure 5 The overall framework diagram of the gait analysis disease diagnosis method based on feature interaction rebalancing provided in Example 1 of the present invention;

[0037] Figure 6 Confusion matrix, scatter plot and ROC curve for sample classification of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0042] Example 1

[0043] like Figure 5 As shown, this embodiment provides a gait analysis disease diagnosis method based on feature interaction rebalancing, comprising the following steps:

[0044] S1 obtains gait data samples, divides them into multiple batches, and extracts features from each batch of gait data samples to obtain local features and global features;

[0045] S2 performs cross-enhancement operations on local features and global features to obtain weighted local features and weighted global features;

[0046] S3 performs concatenation operation on weighted local features and weighted global features to obtain fused features, uses the fused features to perform classification prediction, obtains classification results, calculates the cross entropy loss between the classification results and the true labels, obtains prototype vectors of weighted local features and weighted global features respectively, calculates the prototype cross entropy loss between the weighted local features and weighted global features and the prototype vector respectively, calculates the similarity matrix of the weighted local features and weighted global features of each batch, and calculates the similarity loss between the similarity matrix and its transposed matrix and the true labels respectively;

[0047] S4 performs weighted combination of the above losses to obtain the total loss function of the model and trains the model.

[0048] The local feature is the coupling within the joint, that is, the relationship between the three degrees of freedom within a joint, and the global feature is the coupling between multiple joints.

[0049] The Global-Local Interaction Rebalancing Gait Network (GLIR-GaitNet) disclosed in this paper is the first gait network that simultaneously learns the intra-joint and inter-joint coupling relationships. Figure 3 as shown) and PIR module (as Figure 4 As shown in Figure 2 ). The GL-JCFE module consists of three components: a local 2D residual module for capturing local features within the three degrees of freedom of the same joint, a global dynamic graph learning module for extracting global features between joints, and an MCE module for enhancing the complementarity between these two types of features. The PIR module enhances the interaction between global and local feature information by combining SIM loss, thereby solving the imbalance problem in the multi-feature fusion process. Finally, the fused multi-features are input into the fully connected layer for triple classification to obtain the classification results.

[0050] In the feature extraction part, the proposed GL-JCFE module can capture the comprehensive coupling within and between joints. In this disclosure, local features and global features refer to the features learned from the intra-joint and inter-joint relations, respectively. Specifically, the relationship between the three degrees of freedom within a joint is regarded as a local feature, while the coupling between multiple joints is regarded as a global feature.

[0051] The local 2D residual module is used to capture the intra-joint coupling as a local feature. A single set of gait time series consists of three degrees of freedom within the same joint and is considered as a 2D matrix. The intra-joint coupling features are obtained by using a 2D convolution kernel. The sliding operation of this 2D convolution kernel on the matrix can be interpreted as multiple small filters to generate new feature maps. These feature maps capture the dependencies between the three degrees of freedom.

[0052] The global dynamic graph learning module captures the coupling between joints as global features. The dependencies between 18 dimensions of multiple joints can be modeled as a graph, and the existing dynamic graph network is used to extract hidden spatiotemporal dependencies. To establish the initial relationship, the graph construction and learning module generates a set of graph adjacency matrices, each corresponding to a different time slot. The elements of the graph adjacency matrix can be learned through an iterative process. The core modules of the dynamic graph neural network include dynamic graph transformation and dynamic graph isomorphic network. Dynamic graph transformation is used for dynamic association between graphs at different times, and dynamic graph isomorphic network is used to capture spatiotemporal dependencies.

[0053] The multi-feature cross-enhancement module aims to enhance and integrate information from different feature extractors. In a multi-branch feature extraction network, features from different extractors are highly complementary. What should be enhanced is complementary (irrelevant) information, not redundant (relevant) features.

[0054] Inspired by the cross-attention mechanism, two branches with different parameters are used to extract features from two extractors. In the traditional self-attention mechanism, there is only one input sequence, which contains three representations: query, key and value. In contrast, in the cross-attention mechanism, for the two feature sequences and Provide a query sequence, and Provide a key and value sequence. By calculating the similarity between the query sequence and the key sequence, the attention distribution between different sequences is obtained. Then the attention distribution is used to perform a weighted summation on the value sequence to obtain the representation of the output sequence. The formula of the cross enhancement module is as follows:

[0055]

[0056] in, and Represents the input of the MCE module, and represents different representations of the input, is a learnable matrix, norm(·) represents the linear norm operation, and d is the dimension of the input vector.

[0057] In the multi-feature fusion process, a PIR strategy is established to reduce the interference between the two feature extraction processes and enhance the consistency of feature distribution. It accelerates the slower fitting feature extraction process by promoting the utilization of features and reduces the distribution gap by performing alignment operations on features from the same sample.

[0058] First, the concept of "prototype" of the two features is defined as "representative embedding of class instances". By using prototypes to build non-parametric classifiers, the performance of each fitting process is evaluated by measuring the distance between each sample feature and its respective prototype feature. The process with lower fitting performance is accelerated by assigning a prototype cross entropy (PCE) loss coefficient. In addition, for each batch of generated sample features, the cross-domain similarity loss (SIM loss) between them is calculated and added to the back-propagation process. This minimizes the SIM loss, increases the consistency of feature distribution, and reduces information conflict during the fusion process.

[0059] For the input f0 and f1 of the two feature extraction networks (global feature extraction network and local feature extraction network respectively), the dataset is represented as It consists of instances and their corresponding labels (x, y), where And y = {0,1,2}. The total number of categories in the classification task is 3, and the goal is to train the model with data that starts from x.

[0060] For data We can generate the representation we will and Represented as a subset of data for each category. The prototype is the centroid of each category data:

[0061]

[0062] The prototypes are used to produce a distribution over classes for the input data x based on the softmax over the distances to the prototypes in the embedding space of each modality:

[0063]

[0064] Where d(.,.) is the distance function, i.e. the Euclidean distance in this paper. The imbalance ratio is defined as:

[0065]

[0066] in, and is the batch data of training step time t, ρ t It can be used to truly evaluate the imbalance between the two feature extractors. In order to reduce the interference between the two extractors during the melting process, PCE loss is introduced independently to balance the fitting speed:

[0067]

[0068] For each batch, the weighted features obtained by the two extractors and They are represented by F1 and F2 respectively. The local features can be represented by {v0,v1,...,v K}, where K+1 is the total number of samples in the batch. Similarly, the global feature is represented as {t0,t1,...,t K} time. The similarity loss between them in two different directions is defined as:

[0069]

[0070] Taking the first equation as an example, τ is a hyperparameter, are all local features. When is a positive global sample feature, It is a negative global sample feature that does not match the local feature. where f v (·) and f t (·) are two projection heads that map the representations to the space where the cross entropy loss is applied. In summary, we define the SIM loss as:

[0071]

[0072] The loss during backpropagation is a weighted combination of CE loss, PCE loss, and SIM loss:

[0073]

[0074] in, is the cross entropy loss, and are the prototype cross entropy losses of the two feature extractors, is the similarity loss, α is a hyperparameter that controls the equilibrium fitting speed, ε is a hyperparameter for similarity modulation, and β and γ are coefficients.

[0075] Using ρ t To dynamically assess the imbalance, the learning speed of each modality can be adjusted by simply adjusting the coefficients β and γ:

[0076]

[0077] Among them, clip(a,b,c) is a truncation function that limits b to between a and c. In this way, the feature extractors with slower learning can be conveniently utilized, while the better extractors keep the original learning strategy, which alleviates the imbalance phenomenon.

[0078] Our dataset includes healthy subjects (HC), CSM patients, and PS patients. The data of subjects with problems such as insufficient length and poor quality were excluded, resulting in 51 CSM samples, 33 HC samples, and 49 PS samples, as shown in Table 1.

[0079] The inertial motion acquisition system is used to capture the motion process and output kinematic data from 11 sensors fixed on various parts of the subject's body, focusing on the data from 6 sensors at the lower limb joints. Each data series includes three directions: sagittal (X), coronal (Y) and horizontal (Z). Visual 3D software (version 2021, C-motion Inc, USA) is used to model the human body and convert the kinematic data captured by the sensors into joint parameter time series data.

[0080] Table 1: Dataset category name and sample number

[0081]

[0082]

[0083] Two time series data of joint angle and joint angular velocity were selected for the experiment. Based on three-minute walking data, the length of a single time series is 1202, the dimension is 17 (six sensors with three degrees of freedom), and there are 2 samples for each subject. In order to effectively combine the speed and acceleration data, the data is horizontally spliced ​​to form a length of 2404 as the total input of the network. The proposed framework is implemented using the PyTorch toolbox. Model training is carried out in an environment equipped with an NVIDIA RTX3090 GPU. The batch size is set to 8 and the training is carried out on 150 epochs. The SGD optimizer with a momentum of 0.9 is used, starting from an initial learning rate of 0.001 and reduced to 0.0001 after 70 epochs.

[0084] In order to comprehensively evaluate the proposed model, two key indicators are used: ACC (accuracy) and AUC (area under the curve). ACC measures the overall correctness of the model's predictions. It is calculated as follows:

[0085]

[0086] Where TP, TN, FP, and FN are true positive, true negative, false positive, and false negative, respectively. AUC measures the ability of the model to distinguish between classes. It is the area under the ROC (receiver operating characteristic) curve, which plots the true positive rate (sensitivity) versus the false positive rate (1-specificity) at various threshold settings. The higher the AUC, the better the model performance.

[0087] To demonstrate the feasibility of the proposed model, it is compared with several traditional and cutting-edge multivariate time series classification methods: CNN-LSTM, Deep 1D-Convnet, ALSTM-FCN, TodyNet, and ResNet. The classification results of these algorithms on our own collected datasets are shown in Table 2. For fair comparison, the training and test samples of all compared algorithms are the same. Experimental results show that our algorithm outperforms all state-of-the-art methods by a significant margin in all indicators, including ACC and AUC. It achieves an overall ACC of 91.45%. Specifically, the predicted ACC of PS is significantly high at 94.34%, while that of CSM is relatively low. Our method improves the average ACC by 2.27% and the average AUC by 3.27%. These improvements are attributed to our comprehensive extraction of coupling relations while ensuring the minimization of interference and consistency of features during fusion. Figure 6 ROC curves, scatter plots, and confusion matrices for each class are shown, highlighting the superior performance of our algorithm in gait diagnosis.

[0088] Table 2: Quantitative comparison of different methods on self-collected datasets. ACC (%) and AUC (%) are shown, and the best results are in bold

[0089]

[0090] In order to analyze the contribution of each component in the proposed model to the classification performance, experiments are conducted by selectively removing parts of the model. Tables 3 and 4 give the experimental results, respectively, where √ indicates adding a module.

[0091] Table 3: Ablation study of MCE strategy

[0092]

[0093] The MCE module plays a vital role in the feature fusion network. The experimental results in Table 3 highlight the effectiveness of the MCE module in enhancing and integrating information from different feature extractors. The results show that the MCE strategy improves the ACC and AUC by about 1.51% and 3.67%, respectively. Reasonable weight allocation makes the features of different extractors highly complementary and reduces the interference of redundant features on classification.

[0094] Table 4: Ablation study of PIR strategy

[0095]

[0096] The simulation results verify that integrating the PIR strategy improves the classification performance of the model, as shown in the results in Table 4. In addition, to provide additional evidence for the effectiveness of the PIR strategy, a comparative analysis of the PMR method is performed. Our results show that the PIR strategy can improve the ACC and AUC by approximately 0.51% and 0.67%, respectively. This improvement stems from the PIR strategy, which overcomes the limitation of inconsistent cross-domain feature distribution observed in the PMR method by introducing cross-domain similarity loss supervision.

[0097] This paper systematically studies the challenges of gait-based disease diagnosis and addresses them using GLIR-GaitNet, which includes a GL-JCFE module for feature extraction and a PIR module for feature fusion. For Challenge 1, which involves complex coupling relationships within and between joints, the GL-JCFE module integrates a local 2D residual module for intra-joint coupling and a dynamic graph learning module for global relationships between joints. For Challenge 2, multi-feature fusion is hindered by process interference and inconsistent feature expression. The PIR module enhances cross-domain similarity supervision and balances feature extraction speed, improving fusion performance. Experimental results show the superiority of our method on self-collected datasets on triple classification tasks (CSM, PS, and HC). The model has a prediction accuracy of 91.45% and 93.48% for ACC and AUC, respectively, which is 2.77% and 3.27% higher than existing models.

[0098] Example 2

[0099] This embodiment provides a gait analysis disease diagnosis system based on feature interaction rebalancing, including:

[0100] The feature extraction module is configured to: obtain gait data samples, divide them into multiple batches, and perform feature extraction on the gait data samples of each batch to obtain local features and global features;

[0101] The cross enhancement module is configured to: perform a cross enhancement operation on the local features and the global features to obtain weighted local features and weighted global features;

[0102] The loss calculation module is configured to: perform a splicing operation on the weighted local features and the weighted global features to obtain fused features, use the fused features to perform classification prediction, obtain classification results, calculate the cross entropy loss between the classification results and the true labels, respectively obtain the prototype vectors of the weighted local features and the weighted global features, respectively calculate the prototype cross entropy loss between the weighted local features and the weighted global features and the prototype vectors, calculate the similarity matrix of the weighted local features and the weighted global features of each batch, and respectively calculate the similarity loss between the similarity matrix and its transposed matrix and the true labels;

[0103] The model training module is configured to: perform weighted combination of the above losses to obtain the total loss function of the model and train the model.

[0104] It should be noted that the above modules correspond to the steps described in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.

[0105] In further embodiments, there is also provided:

[0106] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, it will not be described in detail here.

[0107] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0108] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0109] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in Example 1 is completed.

[0110] The method in Example 1 can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it is not described in detail here.

[0111] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in embodiment 1 is implemented.

[0112] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer executable instructions, such as instructions included in a program module, which are executed in a device on a real or virtual processor of the target to perform the process / method as described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules can be combined or divided between program modules as needed. Machine executable instructions for program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0113] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the computer or other programmable data processing device, causes the function / operation specified in the flow chart and / or block diagram to be implemented. The program code can be executed completely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer or completely on a remote computer or server.

[0114] In the context of the present invention, computer program codes or related data may be carried by any appropriate carrier to enable a device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, and the like. Examples of signals may include electrical, optical, radio, acoustic or other forms of propagation signals, such as carrier waves, infrared signals, and the like.

[0115] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0116] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A gait analysis disease diagnosis method based on feature interaction rebalancing, characterized in that: The following steps are involved: Obtain gait data samples, divide them into multiple batches, and perform feature extraction on the gait data samples of each batch to obtain local features and global features; Perform cross-enhancement operations on local features and global features to obtain weighted local features and weighted global features; Perform concatenation operation on weighted local features and weighted global features to obtain fused features, use the fused features to perform classification prediction, obtain classification results, calculate the cross entropy loss between the classification results and the true labels, obtain prototype vectors of weighted local features and weighted global features respectively, calculate the prototype cross entropy loss between weighted local features and weighted global features and prototype vectors respectively, calculate the similarity matrix of weighted local features and weighted global features of each batch, and calculate the similarity loss between the similarity matrix and its transposed matrix and the true labels respectively; The above losses are weighted and combined to obtain the total loss function of the model, and the model is trained.

2. The gait analysis disease diagnosis method based on feature interaction rebalancing according to claim 1, characterized in that: The local feature is the coupling within the joint, that is, the relationship between the three degrees of freedom within a joint, and the global feature is the coupling between multiple joints.

3. The gait analysis disease diagnosis method based on feature interaction rebalancing according to claim 1, characterized in that: Perform cross-enhancement operations on local features and global features, specifically: For local features and global features, one of the feature sequences is used to provide the query sequence, and the other feature sequence provides the key and value sequence. By calculating the similarity between the Query sequence and the Key sequence, the attention distribution between different sequences is obtained. The attention distribution is used to perform weighted summation on the value sequence to obtain weighted local features and weighted global features.

4. The gait analysis disease diagnosis method based on feature interaction rebalancing as claimed in claim 2, characterized in that: A local 2D residual module is used to capture the coupling within a joint, and a global dynamic graph learning module is used to capture the coupling between joints.

5. The gait analysis disease diagnosis method based on feature interaction rebalancing according to claim 1, characterized in that: The prototype cross entropy loss can evaluate the imbalance between two feature extractors, and the similarity loss enables the model to learn feature representations that reflect similarity relationships in the data.

6. The gait analysis disease diagnosis method based on feature interaction rebalancing as claimed in claim 4, characterized in that: The total loss function is: in, is the cross entropy loss, and are the prototype cross entropy losses of the two feature extractors, is the similarity loss, α is a hyperparameter that controls the equilibrium fitting speed, ε is a hyperparameter for similarity modulation, and β and γ are coefficients.

7. A gait analysis disease diagnosis system based on feature interaction rebalancing, characterized in that: include: The feature extraction module is configured to: obtain gait data samples, divide them into multiple batches, and perform feature extraction on the gait data samples of each batch to obtain local features and global features; The cross enhancement module is configured to: perform a cross enhancement operation on the local features and the global features to obtain weighted local features and weighted global features; The loss calculation module is configured to: perform a splicing operation on the weighted local features and the weighted global features to obtain fused features, use the fused features to perform classification prediction, obtain classification results, calculate the cross entropy loss between the classification results and the true labels, respectively obtain the prototype vectors of the weighted local features and the weighted global features, respectively calculate the prototype cross entropy loss between the weighted local features and the weighted global features and the prototype vectors, calculate the similarity matrix of the weighted local features and the weighted global features of each batch, and respectively calculate the similarity loss between the similarity matrix and its transposed matrix and the true labels; The model training module is configured to: perform weighted combination of the above losses to obtain the total loss function of the model and train the model.

8. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images