Realistic test-oriented adaptive medical image classification method, system and device

Abstract

The application provides a medical image classification method, system and device adaptive to real-time test, relates to the technical field of medical image classification, takes domain and class as two independent dimensions, creates a generated medical image test set of multiple scenes based on original medical images by using a Markov state transition matrix, each scene being a combination of balance parameters and time correlation parameters of the domain and class respectively; images in the generated medical image test set are input into a target medical image classification model in turn to obtain classification results of the generated medical images, and the target medical image classification model is obtained by replacing a normalization layer in a pre-trained original medical image classification model with a balance normalization layer. The medical image classification method adaptive to real-time test is comprehensive and has a wide application range, and the most suitable method can be selected for a specific scene.

Description

Technical Field

[0001] This application relates to the field of medical image classification technology, and in particular to adaptive medical image classification methods, systems and devices for real-world testing. Background Technology

[0002] Deep learning has achieved remarkable success in various tasks, but when deployed in real-world tasks, the performance of the model often degrades due to domain shifting of the test data.

[0003] Current approaches employ test-time adaptive methods to mitigate this issue. The core of this approach is to recalibrate the batch normalization (BN) layer of the model using domain-shifted normalization statistics to address the mismatch between the BN layer statistics and the target domain. However, this method assumes that the test data are independent and homo-domain distributed (i.id), re-estimating the normalization statistics based on the current test batch. In real-world testing, scenarios are often more complex, and these assumptions may not hold true. Furthermore, domain shift issues in test data can be more severe during actual testing. Therefore, more robust adaptive methods are needed for real-world testing.

[0004] To apply test time adaptation methods to more realistic scenarios, methods such as continuous domain transition, mixed domain, time-dependent, and imbalanced class distribution have been proposed. However, these methods have only been evaluated in specific scenarios and lack a unified and comprehensive performance evaluation benchmark.

[0005] Therefore, there is an urgent need for a comprehensive and widely adaptable unified test time adaptive method. Summary of the Invention

[0006] This application provides an adaptive medical image classification method, system, and device for real-world testing to address the aforementioned problems.

[0007] In a first aspect, this application provides a medical image classification method that is adaptive for real-world testing, the method comprising:

[0008] Using domain and class as two independent dimensions, based on original medical images, a variety of generated medical image test sets are created using Markov state transition matrices. Each scenario is a combination of the balance parameters and temporal correlation parameters of the domain and class respectively. The balance parameters of the domain and class respectively include: balanced and unbalanced; the temporal correlation parameters of the domain and class respectively include: independent, non-independent, and continuous.

[0009] The images in the generated medical image test set are sequentially input into the target medical image classification model to obtain the classification result of the generated medical images. The target medical image classification model is obtained by replacing the normalization layer in the pre-trained original medical image classification model with a balanced normalization layer. The balanced normalization layer is used to perform domain prediction on the generated medical images, and the domain prediction result is used to obtain the classification result of the generated medical images.

[0010] In an optional embodiment of this application, the target medical image classification model further includes a classification module, and the method further includes:

[0011] The classification module obtains a first classification probability based on the features of the currently generated medical image and the features of the previously generated medical image.

[0012] Using the linear classifier of the pre-trained original medical image classification model, a second classification probability is obtained based on the features of the currently generated medical image;

[0013] When the first classification probability is greater than the second classification probability, the classification result of the currently generated medical image is determined based on the first classification probability;

[0014] When the first classification probability is not greater than the second classification probability, the classification result of the currently generated medical image is determined by the second classification probability.

[0015] In one optional embodiment of this application, the classification result of each generated medical image is obtained by using the target medical image classification model through three forward propagations, the three forward propagation processes being as follows:

[0016] Perform the first forward propagation on the generated medical image: In the absence of prior domain and class information, perform the first forward propagation using global statistics to obtain the initial pseudo-class label of the generated medical image;

[0017] A second forward propagation is performed on the generated medical image: with the initial pseudo-class label as the available class label, a second forward propagation is performed using class statistics and global statistics, and the class statistics and global statistics are updated. In addition, the generated medical image is domain predicted using the m-th balanced normalization layer to obtain pseudo-domain labels.

[0018] Perform a third forward propagation on the generated medical images: With pseudo-domain labels obtained, perform a third forward propagation using domain class statistics and domain statistics to obtain the classification results of the generated medical images, and update the domain class statistics and domain statistics.

[0019] The updated class statistics, updated global statistics, updated domain class statistics, and updated domain statistics are used to classify the next generated medical image.

[0020] In an optional embodiment of this application, the time-related parameters of the domain and class are set according to the following steps:

[0021] When it is necessary to set the time dependency parameters of the domain and class independently, the time dependency parameters α of the domain and class should be set separately. i Set to 1 / n, where i ranges from 1 to n, and n represents the total number of categories for each domain or class in the medical image.

[0022] When it is necessary to set the time dependency parameters of the domain and class to be continuous, the time dependency parameters α of the domain and class should be set accordingly. i Set to 1;

[0023] When it is necessary to set the time dependency parameters of the domain and class to be non-independent, the time dependency parameters α of the domain and class should be set accordingly. i Set it to a value greater than 1 / n and less than 1.

[0024] In an optional embodiment of this application, the balance parameters of the domain and the class are set according to the following steps:

[0025] If it is necessary to set the balance parameter of the domain and the class to be balanced, set the balance parameter β of the domain and the class to 1.

[0026] If it is necessary to set the balance parameter of the domain and class to be unbalanced, set the balance parameter β of the domain and class to a value greater than 1.

[0027] In an optional embodiment of this application, the Markov state transition matrix is ​​defined as follows:

[0028]

[0029] In the formula, k represents the number of states in a medical image, which is the product of the total number of domains and classes; P ij It refers to the probability of transitioning from state i to state j, where i ≠ j. The state is a combination of any domain and a class, and the values ​​of i and j range from 1 to k.

[0030] In one optional embodiment of this application, using domain and class as two independent dimensions, a test set of generated medical images for various scenarios is created based on the original medical images using Markov state transition matrices, including:

[0031] Select an original medical image from the original medical image set and determine the initial state S0;

[0032] Sampling is performed according to the transition probabilities in the Markov state transition matrix to obtain the next state S1;

[0033] Using the next state S1 as the initial state, sampling is performed according to the transition probabilities in the Markov state transition matrix until data of a preset length is obtained, which serves as a test set for generating medical images for various scenarios.

[0034] A second aspect of this application provides an adaptive medical image classification system for real-world testing, the system comprising:

[0035] The data generation module is used to create a test set of generated medical images for various scenarios based on the original medical images, with domain and class as two independent dimensions and Markov state transition matrix. Each scenario is a combination of the balance parameters and temporal correlation parameters of the domain and class respectively. The balance parameters of the domain and class respectively include: balanced and unbalanced; the temporal correlation parameters of the domain and class respectively include: independent, non-independent, and continuous.

[0036] The classification module is used to sequentially input the images from the generated medical image test set into the target medical image classification model to obtain the classification result of the generated medical images. The target medical image classification model is obtained by replacing the normalization layer in the pre-trained original medical image classification model with a balanced normalization layer. The balanced normalization layer is used to perform domain prediction on the generated medical images, and the domain prediction result is used to obtain the classification result of the generated medical images.

[0037] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the adaptive medical image classification method for real-world testing as described in any of the first aspects above.

[0038] In a fourth aspect of this application, a computer-readable storage medium is provided, on which a computer program / instruction is stored, which, when executed by a processor, implements the adaptive medical image classification method for real-world testing as described in any of the first aspects above.

[0039] This application offers the following advantages: It provides an adaptive medical image classification method, system, and device for real-world testing. Using domain and class as two independent dimensions, it creates multiple scenarios of generated medical image test sets based on original medical images using Markov state transition matrices. Each scenario is a combination of balance parameters and temporal correlation parameters for both the domain and class. The balance parameters for each domain and class include: balanced and imbalanced; the temporal correlation parameters for each domain and class include: independent, not independent, and continuous. The images in the generated medical image test sets are sequentially input into a target medical image classification model to obtain the classification results. The target medical image classification model is obtained by replacing the normalization layer in a pre-trained original medical image classification model with a balanced normalization layer. The balanced normalization layer is used for domain prediction of the generated medical images, and the domain prediction results are used to obtain the classification results of the generated medical images. The adaptive medical image classification method proposed in this application is comprehensive and widely applicable, and the most suitable method can be selected for specific scenarios. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the steps of an adaptive medical image classification method for real-world testing, as provided in an embodiment of this application.

[0042] Figure 2 This is a schematic diagram of a process for creating a medical image test set based on Markov state transition matrix for various scenarios, as provided in this application.

[0043] Figure 3 This is an architecture diagram of a medical image classification system that adapts to real-world testing, as provided in an embodiment of this application.

[0044] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] To make the test-time adaptive method applicable to a wider range of scenarios and more robust, this application proposes a medical image classification method adapted for real-world test times. (See attached document.) Figure 1 , Figure 1 This is a flowchart illustrating the steps of an adaptive medical image classification method for real-world testing, as provided in this application embodiment. The method includes the following steps:

[0047] Step 101: Using domain and class as two independent dimensions, based on the original medical images, create a test set of generated medical images for various scenarios using Markov state transition matrices. Each scenario is a combination of the balance parameters and temporal correlation parameters of the domain and class respectively. The balance parameters of the domain and class respectively include: balanced, unbalanced; the temporal correlation parameters of the domain and class respectively include: independent, not independent, continuous.

[0048] Step 102: Input the images in the generated medical image test set into the target medical image classification model in sequence to obtain the classification result of the generated medical images. The target medical image classification model is obtained by replacing the normalization layer in the pre-trained original medical image classification model with a balanced normalization layer. The balanced normalization layer is used to perform domain prediction on the generated medical images, and the domain prediction result is used to obtain the classification result of the generated medical images.

[0049] In specific implementation step 101, using domain and class as two independent dimensions, based on the original medical images, a test set of generated medical images for various scenarios is created using Markov state transition matrices. (See [link / reference]). Figure 2 , Figure 2 This is a schematic diagram of a process for creating a medical image test set based on a Markov state transition matrix, which is provided in this application.

[0050] In this application, medical imaging refers to the acquisition of images of internal human structures and organs through specific imaging equipment and methods, such as X-rays, ultrasound, CT (computed tomography), and MRI (magnetic resonance imaging), to understand the anatomical structure, physiological function, and pathological changes of the human body. Regarding medical imaging, the imaging equipment used to acquire medical images may experience certain effects over time, which will be reflected in the medical images. Here, the domain is considered to be the noise added to the medical images due to the impact on the imaging equipment. Since medical imaging includes images of internal human structures and organs, which can be used to understand the anatomical structure, physiological function, and pathological changes of the human body, the class here refers to the category to which the medical image belongs to a certain internal structure, organ, or lesion of the human body.

[0051] Each of the above scenarios is a combination of the balance parameters and temporal dependence parameters of the domain and class. The balance parameters of the domain and class include: balanced and unbalanced; the temporal dependence parameters of the domain and class include: independent, non-independent, and continuous.

[0052] In a given scenario, class imbalance refers to the different proportions of each domain or class. For example, if the medical images in training set 1 belong to the category of disease, class imbalance means that in this training set, medical images of disease A account for 90%, medical images of disease B account for 4%, medical images of disease C account for 3%, and medical images of disease D account for 3%. In training set 1, class imbalance occurs. If medical images of diseases A, B, C, and D each account for 25% in training set 1, then class imbalance occurs in training set 1.

[0053] Domain imbalance refers to the presence of multiple different types of noise in medical images within training set 1, such as noise a, noise b, and noise c. If the proportions of noise a, noise b, and noise c in the medical images are equal, then the domain is balanced in training set 1. If the proportions of noise a, noise b, and noise c in the medical images are different, then the domain is imbalanced in training set 1. (See also...) Figure 2 In a bar chart, where each bar represents the proportion of a domain in a medical image, the proportion of each domain differs in an unbalanced bar chart, while the proportion of each domain is the same in a balanced bar chart. Similarly, when each bar represents the proportion of a class in a medical image, the interpretation of the class balance parameter is similar to that of the domain imbalance parameter.

[0054] During the actual testing process, each medical image or each batch of medical images was tested sequentially, as described in the following reference. Figure 2 , Figure 2 Each circle represents a medical image, and different patterns within the circles indicate different classes or domains. If adjacent medical images have no relation to each other in their domains, the temporal correlation parameter of the domain is independent (iid); if adjacent medical images have a high probability of sharing the same domain, the temporal correlation parameter of the domain is non-independent (Non-iid); if the tested medical images consistently belong to the same domain (represented by a bar pattern), the temporal correlation parameter of the domain is continuous. Similarly, the interpretation of the temporal correlation parameter of a class is the same as that of a domain. Independence and continuity are extreme cases in the testing process.

[0055] In one optional embodiment of this application, a medical image test set can be created by selecting the respective balanced correlation parameter and temporal correlation parameter based on the domain or class dimension, and combining them with the Markov state transition matrix. The temporal correlation parameters of the domain and class are set according to the following steps: when it is necessary to set the temporal correlation parameters of the domain and class independently, the temporal correlation parameter α of the domain and class is... i Set to 1 / n, where i ranges from 1 to n, and n represents the total number of categories for each domain or class in the medical image; when it is necessary to set the temporal correlation parameters of each domain and class to be continuous, set the temporal correlation parameter α of each domain and class to 1 / n. i Set to 1; when it is necessary to set the time dependency parameters of the domain and class to be non-independent, set the time dependency parameters α of the domain and class to 1. i Set to a value greater than 1 / n and less than 1. This applies when the total number of categories in the domain is n. D When setting the temporal correlation parameter of a domain, the temporal correlation parameter α of the domain... i Set to 1 / n D The value of i ranges from 1 to n D When the total number of categories is n C When setting the time dependence parameter of a class, the time dependence parameter α of the domain is... i Set to 1 / n C The value of i ranges from 1 to n C .

[0056] In one optional embodiment of this application, the balance parameters of the domain and class are set according to the following steps: when it is necessary to set the balance parameters of the domain and class to be balanced, the balance parameter β of the domain and class is set to 1; when it is necessary to set the balance parameters of the domain and class to be unbalanced, the balance parameter β of the domain and class is set to a value greater than 1, or β can be set to a value that satisfies the following formula:

[0057]

[0058] In the formula, α1 is the time correlation parameter when i takes the value of 1.

[0059] In one optional embodiment of this application, using domain and class as two independent dimensions, a test set of generated medical images for various scenarios is created based on the original medical images using a Markov state transition matrix. Specifically, this includes: selecting an original medical image from the original medical image set and determining an initial state S0; sampling according to the transition probabilities in the Markov state transition matrix to obtain the next state S1; using the next state S1 as the initial state, sampling according to the transition probabilities in the Markov state transition matrix until a preset length of data is obtained, serving as the test set of generated medical images for various scenarios. The preset length refers to the amount of data required during actual testing. That is, after setting the time correlation parameters and balance parameters for each domain and class, the Markov state transition matrix is ​​determined using the time correlation parameters and balance parameters set for each domain and class.

[0060] In an optional embodiment of this application, the Markov state transition matrix is ​​defined as follows:

[0061]

[0062] In the formula, k represents the number of states in a medical image, which is the product of the total number of domains and classes; P ij This refers to the probability of transitioning from state i to state j, where i ≠ j. A state is any combination of a domain and a class. For example, if the total number of domain categories is 15 and the total number of class categories is 10, the number of states is 150. Initially, when k takes the value corresponding to i, let α... i =P ii The values ​​of i and j range from 1 to k. In the Markov state transition matrix, the probability of transitioning from any state to any other state is uniform.

[0063] Time correlation parameters based on domain D and balance parameter β D The Markov state transition matrix P of the domain can be constructed. D Time correlation parameters based on class C and balance parameter β D The Markov state transition matrix P of the class can be constructed. C The total number of categories in the domain is n D The total number of categories is n C The Markov state transition matrix P of the domain D The size is n D *n D The Markov state transition matrix P of the class C The size is n C *n C The Markov state transition matrix P of the domain.D Markov state transition matrix P of class C After performing the Kronecker product, we obtain the total Markov state transition matrix P, which contains the domain and class, and has a size of n. D *n C *n D *n C The number of states is n D *n C Then, based on the total Markov state transition matrix P containing the domain and class, sampling is performed, and the sampled medical images are input into the target medical image classification model for final classification.

[0064] This application uses Markov state transition matrices to construct test data for various scenarios. During construction, the domains and classes are standardized, statistical data for each class within each domain is calculated, and then averaged across classes to obtain balanced domain statistical data, thereby mitigating the impact of imbalance on domain statistics.

[0065] In specific implementation step 102, the images in the generated medical image test set generated according to the Markov state transition matrix are sequentially input into the target medical image classification model to obtain the classification result of the generated medical images. The aforementioned target medical image classification model is obtained by replacing the normalization layer in the pre-trained original medical image classification model with a balanced normalization layer. The replaced balanced normalization layer is used to perform domain prediction on the generated medical images, and the domain prediction result is used to obtain the classification result of the generated medical images.

[0066] In the category prediction process, selecting the corresponding statistical data based on the domain of the current data effectively solves the problem of domain independence and homo-domain distribution of test data. The Kullback-Leibler (KL) divergence between the standardized statistics μ and σ can effectively indicate changes in domain distribution. The domain of the test data is determined using the KL divergence between the statistics of the generated medical images for each test and the statistics of each domain. If there are 15 domains, then 15 KL divergence results are calculated, and the result with the smallest KL divergence is the predicted domain.

[0067] In one optional embodiment of this application, the classification result of each generated medical image is obtained by using the target medical image classification model through three forward propagations, the three forward propagation processes being as follows:

[0068] Perform a first forward propagation on the generated medical image: In the absence of prior domain and class information, perform a first forward propagation using global statistics to obtain the initial pseudo-class label of the generated medical image x. The above global statistics (μ) g, σ g ) represents the mean and variance of the feature maps of all generated medical images.

[0069] A second forward propagation is performed on the generated medical image: If the initial pseudo-class label is the available class label, the generated medical image x and the initial pseudo-class label are... A second forward propagation is performed using class statistics and global statistics, and these statistics are updated. Furthermore, the generated medical image is used to predict the domain using the m-th balanced normalization layer (Layer m) to obtain pseudo-domain labels. The above statistical information (μ) k , σ k ) refers to the mean and variance of the feature map of the class. When using the m-th balanced normalization layer for domain prediction, statistical information (μ) about the domain is used. d , σ d This involves generating medical images and performing domain prediction. For example, a test set of generated medical images might have 10 classes, with 10 sets of class statistics, representing the mean and variance of the feature maps for each class. Similarly, a test set of generated medical images might have 15 domains, with 15 sets of domain statistics, representing the mean and variance of the feature maps for each domain. Likewise, a test set of domain-class statistics might have 150 sets, representing the mean and variance of the feature maps for each domain and class.

[0070] Perform a third forward propagation on the generated medical image: Given the pseudo-domain labels, generate the medical image x and the initial pseudo-class labels. and pseudo-domain tags Using domain class statistics (μ dk , σ dk ), and domain statistics (μ d , σ d The third forward propagation is performed to obtain the classification results of the generated medical images. It also updates the domain class statistics and domain statistics, which are used to classify a certain internal structure, organ or lesion in the generated medical image.

[0071] Finally, based on the updated class statistics, updated global statistics, updated domain class statistics, and updated domain statistics, the next generated medical image is classified.

[0072] To address the potential temporal correlation of classes, this application proposes an effective and efficient method called COFA based on the correlation characteristics of features referenced in the test data, without requiring any modification to the model parameters. However, directly using COFA may lead to performance degradation in i.e., ...

[0073] In this application, COFA is applied to the balance normalization layer in the first and third forward propagation, and the results of the balance normalization layer are further filtered to improve the accuracy of medical image classification results.

[0074] In one optional embodiment of this application, the target medical image classification model further includes a classification module, the core of which is to further filter the results of the balanced normalization layer using COFA. The method also includes using the classification module to determine the classification result of the currently generated medical image. Specifically, based on the features of the currently generated medical image and the features of the previously generated medical image, the classification module obtains a first classification probability; then, using the linear classifier of the pre-trained original medical image classification model, a second classification probability is obtained based on the features of the currently generated medical image; the classification result of the currently generated medical image is determined based on the first and second classification probabilities. When the first classification probability is greater than the second classification probability, the classification result of the currently generated medical image is determined by the category corresponding to the first classification probability; when the first classification probability is not greater than the second classification probability, the classification result of the currently generated medical image is determined by the category corresponding to the second classification probability.

[0075] The above first-class probability can be expressed as:

[0076]

[0077] The probability of the second classification can be expressed as:

[0078] The classification result of the current medical image, determined based on the first classification probability and the second classification probability, can be expressed as:

[0079]

[0080] Among them, z i and z i-1 Let represent the features of the i-th and (i-1)-th generated medical images, respectively. ω and b represent the weights and biases of the linear classifier of the pre-trained original medical image classification model, respectively. COFA refers to relevant feature adaptation, and T represents transpose.

[0081] This application provides an adaptive medical image classification method for real-world testing. It uses domain and class as two independent dimensions, and based on original medical images, creates a test set of generated medical images for various scenarios using Markov state transition matrices. Each scenario is a combination of balance parameters and temporal correlation parameters for both the domain and class. The balance parameters for each domain and class include: balanced and imbalanced; the temporal correlation parameters for each domain and class include: independent, not independent, and continuous. Images from the generated medical image test set are sequentially input into a target medical image classification model to obtain the classification results. The target medical image classification model is obtained by replacing the normalization layer in a pre-trained original medical image classification model with a balanced normalization layer. The balanced normalization layer is used for domain prediction of the generated medical images, and the domain prediction results are used to obtain the classification results. The adaptive medical image classification method for real-world testing proposed in this application is comprehensive and widely applicable, and can select the most suitable method for specific scenarios.

[0082] In a second aspect of this application, an adaptive medical image classification system for real-world testing is provided, see [link to relevant documentation]. Figure 3 , Figure 3 This is an architecture diagram of an adaptive medical image classification system for real-world testing, provided in an embodiment of this application. The system includes:

[0083] The data generation module 301 is used to create a test set of generated medical images for various scenarios based on the original medical images, with domain and class as two independent dimensions and Markov state transition matrix. Each scenario is a combination of the balance parameters and temporal correlation parameters of the domain and class respectively. The balance parameters of the domain and class respectively include: balanced and unbalanced; the temporal correlation parameters of the domain and class respectively include: independent, non-independent, and continuous.

[0084] The classification module 302 is used to sequentially input the images in the generated medical image test set into the target medical image classification model to obtain the classification result of the generated medical image. The target medical image classification model is obtained by replacing the normalization layer in the pre-trained original medical image classification model with a balanced normalization layer. The balanced normalization layer is used to perform domain prediction on the generated medical image, and the domain prediction result is used to obtain the classification result of the generated medical image.

[0085] Optionally, the target medical image classification model further includes a classification module, and the system further includes:

[0086] The first classification module is used to obtain a first classification probability based on the features of the currently generated medical image and the features of the previously generated medical image.

[0087] The second classification module is used to obtain the second classification probability based on the features of the currently generated medical image by utilizing the linear classifier of the pre-trained original medical image classification model.

[0088] The first classification result determination module is used to determine the classification result of the currently generated medical image based on the first classification probability when the first classification probability is greater than the second classification probability.

[0089] The second classification result determination module is used to determine the classification result of the currently generated medical image based on the second classification probability when the first classification probability is not greater than the second classification probability.

[0090] Optionally, the classification result of each generated medical image in the classification module is obtained by using the target medical image classification model through three forward propagations. The classification module further includes:

[0091] The first forward propagation submodule is used to perform the first forward propagation on the generated medical image: in the absence of prior domain and class information, the first forward propagation is performed using global statistics to obtain the initial pseudo-class label of the generated medical image;

[0092] The second forward propagation submodule is used to perform a second forward propagation on the generated medical image: with the initial pseudo-class label as the available class label, the second forward propagation is performed using class statistics and global statistics, and the class statistics and global statistics are updated; and the generated medical image is used to perform domain prediction using the m-th balanced normalization layer to obtain pseudo-domain labels.

[0093] The third forward propagation submodule is used to perform a third forward propagation on the generated medical image: with the pseudo-domain labels obtained, the third forward propagation is performed using the domain class statistics and domain statistics to obtain the classification result of the generated medical image, and the domain class statistics and domain statistics are updated.

[0094] The update submodule is used to classify the updated class statistics, global statistics, domain class statistics, and domain statistics for the next generated medical image.

[0095] Optionally, the time-related parameters of the domains and classes in the generation module are set through the following sub-modules:

[0096] The independent setting submodule is used to set the time dependency parameters α of the domain and class independently when it is necessary to set them separately. i Set to 1 / n, where i ranges from 1 to n, and n represents the total number of categories for each domain or class in the medical image.

[0097] The continuous setting submodule is used to set the time dependency parameters α of the domain and class consecutively when it is necessary to set the time dependency parameters of the domain and class consecutively. i Set to 1;

[0098] The "Do Not Set Independently" submodule is used when it is necessary to set the time dependency parameters α of the domain and class to be independent. i Set it to a value greater than 1 / n and less than 1.

[0099] Optionally, the balance parameters of the domains and classes in the generation module are set through the following sub-modules:

[0100] The Balance Setting submodule is used to set the balance parameter β of the domain and class to 1 when it is necessary to set the balance parameter of the domain and class to balance.

[0101] The unbalanced setting submodule is used to set the balance parameter β of the domain and class to a value greater than 1 when it is necessary to set the balance parameter of the domain and class to be unbalanced.

[0102] Optionally, the Markov state transition matrix in the generation module is defined as follows:

[0103]

[0104] In the formula, k represents the number of states in a medical image, which is the product of the total number of domains and classes; P ij It refers to the probability of transitioning from state i to state j, where i ≠ j. The state is a combination of any domain and a class, and the values ​​of i and j range from 1 to k.

[0105] Optionally, the generation module includes:

[0106] The initial state determination submodule is used to select an original medical image from the original medical image set and determine the initial state S0;

[0107] The next state determination submodule is used to sample according to the transition probabilities in the Markov state transition matrix to obtain the next state S1;

[0108] The medical image test set generation submodule is used to take the next state S1 as the initial state, sample according to the transition probabilities in the Markov state transition matrix, until data of a preset length is obtained, which serves as the generated medical image test set for various scenarios.

[0109] Based on the same concept, this application discloses an electronic device in a third aspect. Figure 4A schematic diagram of an electronic device disclosed in an embodiment of this application is shown, such as... Figure 4 As shown, the electronic device 100 includes a memory 110 and a processor 120. The memory of the electronic device is not less than 12G, and the processor has a main frequency of not less than 2.4GHz. The memory 110 and the processor 120 are connected via a bus for communication. The memory 110 stores a computer program that can run on the processor 120 to implement an adaptive medical image classification method for real-world testing disclosed in the embodiments of this application.

[0110] Based on the same concept, this application discloses a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements an adaptive medical image classification method for real-world testing disclosed in this application.

[0111] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0112] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0113] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0115] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0116] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0117] The above provides a detailed description of an adaptive medical image classification method, system, and device for real-world testing provided by this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A medical image classification method that adapts to real-world testing, characterized in that, The method includes: Using domain and class as two independent dimensions, based on original medical images, a test set of generated medical images for various scenarios is created using Markov state transition matrices. Each scenario is a combination of the balance parameters and temporal correlation parameters of the domain and class respectively. The balance parameters of the domain and class respectively include: balanced and unbalanced; the temporal correlation parameters of the domain and class respectively include: independent, non-independent, and continuous; domain imbalance refers to the presence of multiple different noises in the medical image. The images in the generated medical image test set are sequentially input into the target medical image classification model to obtain the classification result of the generated medical images. The target medical image classification model is obtained by replacing the normalization layer in the pre-trained original medical image classification model with a balanced normalization layer. The balanced normalization layer is used to perform domain prediction on the generated medical images, and the domain prediction result is used to obtain the classification result of the generated medical images. Generative medical image test sets for various scenarios were created using Markov state transition matrices, including: Based on the domain's time dependence parameters and balance parameters, the Markov state transition matrix of the domain is constructed. Based on the class's time-dependent parameters and balance parameters, the Markov state transition matrix of the class is constructed. Perform the Kronecker product on the Markov state transition matrix of the domain and the Markov state transition matrix of the class to obtain the total Markov state transition matrix containing both the domain and the class.

2. The adaptive medical image classification method for real-world testing according to claim 1, characterized in that, The target medical image classification model further includes a classification module, and the method further includes: The classification module obtains a first classification probability based on the features of the currently generated medical image and the features of the previously generated medical image. Using the linear classifier of the pre-trained original medical image classification model, a second classification probability is obtained based on the features of the currently generated medical image; When the first classification probability is greater than the second classification probability, the classification result of the currently generated medical image is determined based on the first classification probability; When the first classification probability is not greater than the second classification probability, the classification result of the currently generated medical image is determined by the second classification probability.

3. The adaptive medical image classification method for real-world testing according to claim 1, characterized in that, The classification result of each generated medical image is obtained by using the target medical image classification model through three forward propagations. The three forward propagation processes are as follows: Perform the first forward propagation on the generated medical image: In the absence of prior domain and class information, perform the first forward propagation using global statistics to obtain the initial pseudo-class label of the generated medical image; A second forward propagation is performed on the generated medical image: with the initial pseudo-class label as the available class label, a second forward propagation is performed using class statistics and global statistics, and the class statistics and global statistics are updated. In addition, the generated medical image is domain predicted using the m-th balanced normalization layer to obtain pseudo-domain labels. Perform a third forward propagation on the generated medical images: With pseudo-domain labels obtained, perform a third forward propagation using domain class statistics and domain statistics to obtain the classification results of the generated medical images, and update the domain class statistics and domain statistics. The updated class statistics, updated global statistics, updated domain class statistics, and updated domain statistics are used to classify the next generated medical image.

4. The adaptive medical image classification method for real-world testing according to claim 1, characterized in that, The time-related parameters for the respective domains and classes are set according to the following steps: When it is necessary to set the time dependency parameters of the domain and class independently, the time dependency parameters of the domain and class should be set independently. Set to 1 / n, where i ranges from 1 to n, and n represents the total number of categories for each domain or class in the medical image. When it is necessary to set the time dependency parameters of both the domain and the class to be continuous, the time dependency parameters of both the domain and the class should be set to continuous. Set to 1; When it is necessary to set the time dependency parameters of the domain and class to be non-independent, the time dependency parameters of the domain and class should be set accordingly. Set it to a value greater than 1 / n and less than 1.

5. The adaptive medical image classification method for real-world testing according to claim 1, characterized in that, The balance parameters for the domains and classes are set according to the following steps: If it is necessary to set the balance parameter of the domain and the class to be balanced, set the balance parameter β of the domain and the class to 1. If it is necessary to set the balance parameter of the domain and class to be unbalanced, set the balance parameter β of the domain and class to a value greater than 1.

6. The adaptive medical image classification method for real-world testing according to claim 1, characterized in that, The Markov state transition matrix is ​​defined as follows: ， In the formula, k represents the number of states of a medical image, which is the product of the total number of domains and classes. P ij This refers to the state i Transition to state j The probability, and i ≠ j The state is any combination of a domain and a class. i and j The value range is from 1 to k.

7. The adaptive medical image classification method for real-world testing according to claim 6, characterized in that, Using domain and class as two independent dimensions, and based on original medical images, a test set of generated medical images for various scenarios is created using Markov state transition matrices, including: Select an original medical image from the original medical image set and determine the initial state S0; Sampling is performed according to the transition probabilities in the Markov state transition matrix to obtain the next state S1; Using the next state S1 as the initial state, sampling is performed according to the transition probabilities in the Markov state transition matrix until data of a preset length is obtained, which serves as a test set for generating medical images for various scenarios.

8. A medical image classification system that adapts to real-world testing, characterized in that, The system includes: The data generation module is used to create a test set of generated medical images for various scenarios based on original medical images, using domain and class as two independent dimensions and Markov state transition matrices. Each scenario is a combination of balance parameters and temporal correlation parameters for both the domain and class. The balance parameters for each domain and class include: balanced and imbalanced; the temporal correlation parameters for each domain and class include: independent, not independent, and continuous. Specifically, it is used to construct the Markov state transition matrix for the domain based on the temporal correlation parameters and balance parameters; and to construct the Markov state transition matrix for the class based on the temporal correlation parameters and balance parameters. The Markov state transition matrices for the domain and the class are then processed by Kronecker product to obtain the total Markov state transition matrix containing both the domain and class. Domain imbalance refers to the presence of multiple different types of noise in the medical images. The classification module is used to sequentially input the images from the generated medical image test set into the target medical image classification model to obtain the classification result of the generated medical images. The target medical image classification model is obtained by replacing the normalization layer in the pre-trained original medical image classification model with a balanced normalization layer. The balanced normalization layer is used to perform domain prediction on the generated medical images, and the domain prediction result is used to obtain the classification result of the generated medical images.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the adaptive medical image classification method for real-world testing as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program / instruction that, when executed by a processor, implements the adaptive medical image classification method for real-world testing as described in any one of claims 1-7.

Images