Model adaptation method, electronic device, and computer-readable storage medium

Abstract

The application relates to a model self-adaptive method, an electronic device and a computer readable storage medium, comprising a first data input first processing, which is used for generating a dynamically weighted feature representation; the feature representation is input into a second processing to generate a prediction output of the first data; based on the prediction output, an entropy minimization loss is calculated, and at least part of parameters of the model are fine-tuned by using the loss to improve the adaptability of the model to the first data; the application has the beneficial effects that the model is fine-tuned based on the entropy minimization loss of the model prediction output, the application can improve the performance of the model on new data distribution without relying on unlabeled data; the dynamically weighted feature representation is generated through the first processing, so that the model can flexibly adjust the internal feature extraction strategy thereof according to different input data, and the model can better capture and adapt to different source data.

Description

Technical Field

[0001] This invention relates to the fields of computer science and biomedical engineering, specifically to model adaptation methods, electronic devices, and computer-readable storage media. Background Technology

[0002] Intelligent diagnosis of cardiovascular diseases often relies on electrocardiogram (ECG) analysis. However, data collected from different medical institutions exhibit significant distributional differences, leading to decreased generalization performance of models trained on a single dataset when transferred to new data sources. Existing methods typically require fine-tuning with a large amount of labeled data in the target domain, which limits the practical application of models in scenarios where labeled data is scarce. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method that eliminates the need for extensive target domain annotation data for judgment.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a model adaptation method, comprising a model pre-trained with source domain data; The model includes a first processing and a second processing that sequentially process the first data; The first data is unlabeled; The preprocessed first data is input into the first processing step to generate a dynamically weighted feature representation; The feature representation is input into the second processing step. Based on the model's predicted output of the first data, the entropy of the model's output probability distribution is calculated, and the entropy is used to fine-tune at least some of the model's parameters to improve the model's adaptability to the target domain data.

[0005] This invention pre-trains a model on source domain data and then adaptively fine-tunes it in the target domain using only unlabeled first data. It introduces an optimization strategy based on the entropy of the model's output probability distribution, enabling the model to effectively adapt to new data distributions without requiring a large number of labeled samples in the target domain. Specifically, the pre-processed first data is input into a first processing step to generate dynamically weighted feature representations, which are then input into a second processing step to obtain the predicted output. The entropy of the probability distribution is calculated based on this output, and minimizing the entropy constrains model parameter updates, making the model's output more definite in the target domain and ensuring the discrimination boundary is far from high-density sample regions. This information-theory-driven adaptive mechanism alleviates the performance degradation caused by inconsistent data distributions across different medical institutions, improving cross-domain generalization ability. Furthermore, it reduces reliance on manual labeling of the target domain, offering a technological advantage for rapid deployment and implementation in scenarios with scarce labeling.

[0006] A further optimization of the present invention is that the first process includes shared experts and specialized experts; The feature representation is formed by fusing the output of the shared expert with the weighted output of the two selected specialists.

[0007] By simultaneously setting up shared experts and multiple specialized experts in the first processing step, and using the output of the shared expert and the weighted output of the two selected specialized experts to jointly construct the feature representation, the shared expert learns common features across medical institutions and data acquisition devices, ensuring the stability and transferability of the feature space. Simultaneously, the specialized experts focus on the differential patterns under different data distributions, suppressing noise interference from irrelevant experts and reducing the risk of overfitting by fusing only the weighted output of the two most relevant specialized experts. Thus, under the premise of controllable parameter scale, the ability of the feature representation to represent patterns specific to the target domain is enhanced, significantly improving the model's adaptability and generalization performance to heterogeneous target domain data.

[0008] A further optimization of the present invention is that the first process includes a routing network; The routing network dynamically selects two specialized experts and assigns weights based on the first data.

[0009] By introducing a routing network in the first processing step, two specialized experts are dynamically selected and assigned corresponding weights based on the input features of the first data. This allows the expert selection process to be automatically completed data-driven, without relying on explicit domain labels or manual data subset partitioning. This routing mechanism can adaptively activate the specialized experts that best match the current sample for unlabeled data of different sources and qualities. On the one hand, it avoids the redundant overhead caused by all experts participating in the calculation simultaneously, reducing inference costs. On the other hand, it improves the sensitivity of feature extraction to local distribution differences and fine-grained patterns, thereby further enhancing the model's expressive flexibility and computational efficiency in the target domain.

[0010] A further optimization of the present invention is that, during the fine-tuning process, the parameters of the routing network are optimized using entropy minimization loss, so that the routing network adapts to the distribution of the first data.

[0011] During fine-tuning, entropy minimization loss is used to optimize the routing network parameters, enabling the network to gradually adapt to the true distribution structure of the target domain, relying only on unlabeled initial data. By minimizing the entropy of the model's output probability distribution, the routing network is encouraged to allocate more samples to expert combinations with more certain predictions. This leads to more consistent expert selection and clearer decision boundaries in high-density sample areas, thereby reducing the randomness and instability of routing decisions. This information-theory-driven routing adaptation mechanism improves the model's ability to identify cluster structures and enhances decision reliability in the target domain, achieving higher-quality unsupervised model adaptation.

[0012] A further optimization of the present invention involves fine-tuning at least some parameters of the model, including fine-tuning the parameters of the second processing and the routing network parameters in the first processing.

[0013] By simultaneously fine-tuning the parameters of the second processing and the routing network parameters in the first processing, this invention achieves integrated adaptive optimization of the feature extraction and decision-making stages: updating the parameters of the second processing enables the classifier or high-level decision module to reshape the discrimination boundary according to the feature distribution of the target domain, avoiding performance degradation caused by simply using the source domain boundary; updating the routing network parameters ensures a more reasonable division of labor among different specialists in the target domain, preventing some experts from being idle or falsely activated for extended periods. The synergistic effect of these two processes significantly reduces the risk of negative transfer during cross-domain transfer, making the overall model's prediction performance in the target domain more stable and robust.

[0014] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the model adaptation method as described above.

[0015] By integrating memory and a processor into an electronic device, and pre-loading a computer program implementing the aforementioned model adaptation method in the memory, the electronic device can automatically complete the loading of the source domain pre-trained model, the input and preprocessing of unlabeled data in the target domain, adaptive fine-tuning based on entropy minimization, and the inference output of the updated model when the processor executes the program. This design eliminates the need for manual execution of complex retraining processes, enabling deployed intelligent diagnostic systems to automatically adapt and update according to the data distribution of different medical institutions and different acquisition environments. This shortens the debugging cycle after model deployment, reduces system maintenance costs, and improves the stability and practicality of the model in long-term operation in real clinical environments.

[0016] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model adaptation method as described above.

[0017] By storing the computer program implementing the aforementioned model adaptation method in a computer-readable storage medium, the program can be flexibly loaded and executed on different types of processors and computing platforms. This enables a unified implementation of unsupervised model adaptation capabilities based on entropy minimization and routing networks across various scenarios, including cloud servers, local workstations, and edge terminals. This approach facilitates standardized encapsulation and cross-platform migration of algorithm modules, ensuring consistency of adaptation strategies and diagnostic logic across different deployment environments. Furthermore, it allows for rapid iteration and promotion of the invention's technology by updating the program version in the storage medium, thereby improving engineering implementation efficiency and industrial application value.

[0018] The beneficial effects of this invention are: 1. By fine-tuning the loss based on minimizing the entropy of the model's predicted output, this invention can improve the model's performance on new data distributions without relying on unlabeled data; 2. The first processing generates a dynamically weighted feature representation, enabling the model to flexibly adjust its internal feature extraction strategy for different input data, thus better capturing and adapting to data from different sources. Attached Figure Description

[0019] Figure 1 This is a flowchart of the present invention; Figure 2 This is a model architecture diagram of the present invention; Figure 3 This is a diagram of the internal structure of the Transformer-MoE of the present invention; Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the embodiments shown in the accompanying drawings.

[0021] Reference Figure 1-3 As shown in this embodiment; Step 1: Data Preprocessing Butterworth filters are used to remove signals above 50Hz and below 0.05Hz, respectively, to remove baseline drift, electromyography interference, and power frequency interference.

[0022] Step 2: Model Building Step 2.1: Based on the 1D ViT architecture, design and implement the replacement construction of the feedforward network to the MoE structure. The modified MoE module introduces two types of expert nodes: one type is the shared expert (using a one-dimensional convolutional structure), which is mainly responsible for extracting the common features shared by different ECG samples; the other type is the specialized expert, which focuses on modeling the personalized fine features in specific local segments, thereby taking into account both representation ability and capacity model.

[0023] We use a 1D version of ViT as the base model. Given an original ECG sequence X of length L, we divide it into... There are 3 non-overlapping sub-patches, each of length P. Each patch... After linear projection and addition of position encoding: ; Where W is the learnable projection matrix and b is the bias term. This is the fixed learnable position code for the i-th segment.

[0024] The embeddings of all patches are concatenated to form the input sequence of the Transformer: ; Each Transformer module extracts local and global dependencies through a multi-head self-attention mechanism: ; To enhance representation capabilities, we replace the feedforward network in the Transformer with a hybrid expert module. The MoE contains E expert nodes, including a shared expert. It consists of two one-dimensional convolutional layers and is used to extract shared features across samples: ; It also includes E-1 specialized experts. It is used to model personalized, fine-grained features.

[0025] The routing network generates sparse activation weights based on the input embedding h, and performs Top-K selection on K experts: ; ; Where G is a learnable gated matrix. Let j be the activation weight of the j-th expert. The selected set of Top-K expert indexes.

[0026] The final output is a weighted combination of shared experts and specialized experts: ; To encourage balanced participation from all experts, we introduce a load balancing loss: ; ; Step 2.2: To further optimize the MoE routing strategy, we introduce SCE to measure the entropy and determinism of the expert selection distribution and the determinism of the classifier head output distribution. By adjusting the weight of this loss term, we can flexibly explore whether the routing network should favor a few high-confidence experts or activate more diverse experts in cross-domain scenarios to seek the optimal domain generalization strategy. For a probability vector p, SCE is defined as follows: ; Step 2.3: In the training and transfer phase, this project innovatively proposes to simultaneously optimize the SCE of MoE route output and classification head output during route training and transfer, effectively improving the model's adaptability and stability. Regarding the training and validation strategy, this project adopts full training within a single source domain (single dataset). During training, only the training and validation sets of the source domain are used, without setting a separate source domain test set. For other datasets, only the unlabeled test set is retained. After training in the source domain, the model is transferred to each target domain. The classification head is reinitialized for different datasets with varying numbers of classes. Fine-tuning is then performed on the unlabeled test set, combining the SCE of the classification head output and the loss from the MoE route output, to verify the cross-domain generalization effect and practicality of the proposed method under conditions of insufficient labeled data.

[0027] We simultaneously analyze the MoE routing output g and the classification header output. Introducing SCE, the total loss is as follows: ; in and This is the adjustment coefficient.

[0028] Step 3: Obtain the offline model: A usable model is obtained by training on the source dataset and fine-tuning it on the test set of the target dataset.

[0029] Step 4: Online model testing: The model obtained in step 3 is used to test the test set of the target dataset to obtain diagnostic results.

[0030] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A model adaptation method, characterized in that, This includes models pre-trained using source domain data; The model includes a first processing and a second processing; The first data is unlabeled; The preprocessed first data is input into the first processing step to generate a dynamically weighted feature representation; The feature representation is input into the second process to generate a prediction output for the first data; Based on the predicted output, the entropy of the model output probability distribution is calculated, and the entropy is used to fine-tune at least some of the model's parameters to improve the model's adaptability to the first data.

2. The model adaptation method according to claim 1, characterized in that, The first process includes shared experts and specialized experts; The feature representation is formed by fusing the output of the shared expert with the weighted output of the two selected specialists.

3. The model adaptation method according to claim 2, characterized in that, The first process includes a routing network; The routing network dynamically selects two specialized experts and assigns weights based on the first data.

4. The model adaptation method according to claim 3, characterized in that, During fine-tuning, the entropy minimization loss is used to optimize the parameters of the routing network so that the routing network adapts to the distribution of the initial data.

5. The model adaptation method according to claim 3, characterized in that, At least some parameters of the model are fine-tuned, including parameters of the second processing and routing network parameters in the first processing.

6. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor implements the model adaptation method as described in any one of claims 1-6 when executing a computer program.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the model adaptation method as described in any one of claims 1-6.

Images