A medical ultrasound image recognition method based on generated domain alignment
By using generative domain alignment techniques, medical ultrasound image samples are aligned using conditional diffusion models and unconditional diffusion models, which solves the problem of performance degradation of deep neural networks on non-source domain data and achieves accurate prediction and classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2024-05-31
- Publication Date
- 2026-06-19
AI Technical Summary
Deep neural networks exhibit significant performance degradation on target domain data that are not in the source domain, necessitating the development of effective adaptive techniques to ensure consistency between the model and the data domain.
Labeled generative medical ultrasound image samples are created using a conditional diffusion model, and then aligned to the generative domain using an unconditional diffusion model of the generative domain. The source domain model is then fine-tuned, and finally the target medical ultrasound image is aligned to the generative domain for prediction.
This approach aligns the source domain model and target data in the generation domain, transforming cross-domain tasks into intra-domain prediction tasks and improving the accuracy of prediction and classification.
Smart Images
Figure CN118506099B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image classification technology, and in particular to a medical ultrasound image recognition method based on generator domain alignment. Background Technology
[0002] In related technologies, deep neural networks have become leaders in various visual recognition tasks. However, despite training deep neural networks on extensive datasets from the source domain, they often exhibit significant performance degradation when tested on target domain data that is not from the source domain. Therefore, effective adaptive techniques are needed to ensure consistency between the model and the data domain, enabling source domain models to predict target data from non-source domains. Summary of the Invention
[0003] This invention provides a medical ultrasound image recognition method and system based on generator domain alignment for accurate prediction of target data.
[0004] The first aspect of this invention provides a medical ultrasound image recognition method based on generator domain alignment, the method comprising:
[0005] A conditional diffusion model is used to create labeled generated medical ultrasound image samples for fine-tuning, wherein the labels are used to characterize the category corresponding to the generated medical ultrasound image sample.
[0006] Each labeled generated medical ultrasound image sample is aligned to the generated domain using the unconditional diffusion model of the generated domain, resulting in a set of generated domain medical ultrasound images.
[0007] The source domain model was fine-tuned using a set of medical ultrasound images from the generative domain to obtain the generative domain model.
[0008] The target medical ultrasound image is aligned to the generator domain using the unconditional diffusion model of the generator domain to obtain the generator domain target medical ultrasound image.
[0009] The target medical ultrasound image is input into the generator domain model to obtain the first predicted classification result of the target medical ultrasound image.
[0010] Optionally, each labeled generated medical ultrasound image sample is aligned to the generated domain using an unconditional diffusion model of the generated domain, resulting in a set of generated domain medical ultrasound images, including:
[0011] Noise is added and denoised for each labeled generated medical ultrasound image sample using the unconditional diffusion model of the generation domain, so that each generated medical ultrasound image sample is aligned to the generation domain, resulting in a set of generated medical ultrasound images.
[0012] The target medical ultrasound image is aligned to the generation domain using the unconditional diffusion model of the generation domain to obtain the target generation domain medical ultrasound image, including:
[0013] Noise is added to and denoised using the unconditional diffusion model of the generator domain to align the target medical ultrasound image to the generator domain, thus obtaining the generator domain target medical ultrasound image.
[0014] Optionally, each labeled generated medical ultrasound image sample is aligned to the generation domain using an unconditional diffusion model of the generation domain, including:
[0015] Standard Gaussian noise is added to the generated medical ultrasound image samples generated by the conditional diffusion model to generate noisy medical ultrasound images.
[0016] The noisy medical ultrasound image is restored to a clean medical ultrasound image of the generated domain using the unconditional diffusion model of the generated domain.
[0017] Optionally, labeled generative medical ultrasound image samples for fine-tuning are created using a conditional diffusion model, including:
[0018] Based on the K categories corresponding to the K class labels, t-step back diffusion processing is performed through the conditional diffusion model to uniformly generate N medical ultrasound image samples for each category from random Gaussian noise;
[0019] Inputting the target medical ultrasound image into the generator domain model yields the first predicted classification result of the target medical ultrasound image, including:
[0020] Input the target medical ultrasound image into the generator domain model to obtain the probability that the target medical ultrasound image belongs to each of the K categories.
[0021] Optionally, the method further includes:
[0022] Input the target medical ultrasound image into the source domain model to obtain the second predicted classification result of the target medical ultrasound image;
[0023] Based on the first and second prediction classification results, the final prediction classification result of the target medical ultrasound image is obtained.
[0024] A second aspect of this invention provides a medical ultrasound image recognition system based on generator domain alignment, the system comprising:
[0025] The sample creation module is used to create labeled generated medical ultrasound image samples for fine-tuning using a conditional diffusion model, wherein the labels are used to characterize the category corresponding to the generated medical ultrasound image sample.
[0026] The sample alignment module is used to align each labeled generated medical ultrasound image sample to the generated domain using the unconditional diffusion model of the generated domain, thereby obtaining a set of generated domain medical ultrasound images.
[0027] The fine-tuning module is used to fine-tune the source domain model using the generative domain medical ultrasound image set to obtain the generative domain model;
[0028] The target image alignment module is used to align the target medical ultrasound image to the generation domain using the unconditional diffusion model of the generation domain, so as to obtain the target medical ultrasound image in the generation domain.
[0029] The first prediction module is used to input the target medical ultrasound image from the generator domain into the generator domain model to obtain the first prediction classification result of the target medical ultrasound image.
[0030] Optionally, the sample alignment module is specifically used for:
[0031] Noise is added and denoised for each labeled generated medical ultrasound image sample using the unconditional diffusion model of the generation domain, so that each generated medical ultrasound image sample is aligned to the generation domain, resulting in a set of generated medical ultrasound images.
[0032] The target image alignment module is specifically used for:
[0033] Noise is added to and denoised using the unconditional diffusion model of the generator domain to align the target medical ultrasound image to the generator domain, thus obtaining the generator domain target medical ultrasound image.
[0034] Optionally, the sample alignment module is specifically used for:
[0035] Standard Gaussian noise is added to the generated medical ultrasound image samples generated by the conditional diffusion model to generate noisy medical ultrasound images.
[0036] The noisy medical ultrasound image is restored to a clean medical ultrasound image of the generated domain using the unconditional diffusion model of the generated domain.
[0037] Optionally, the sample creation module is specifically used for:
[0038] Based on the K categories corresponding to the K class labels, t-step back diffusion processing is performed through the conditional diffusion model to uniformly generate N medical ultrasound image samples for each category from random Gaussian noise;
[0039] The first prediction module is specifically used for:
[0040] Input the target medical ultrasound image into the generator domain model to obtain the probability that the target medical ultrasound image belongs to each of the K categories.
[0041] Optionally, the system further includes:
[0042] The second prediction module is used to input the target medical ultrasound image into the source domain model to obtain the second prediction classification result of the target medical ultrasound image.
[0043] The classification result determination module is used to obtain the final predicted classification result of the target medical ultrasound image based on the first predicted classification result and the second predicted classification result.
[0044] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executed, implements the medical ultrasound image recognition method based on generator domain alignment as described in the first aspect of the present invention.
[0045] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the medical ultrasound image recognition method based on generator domain alignment as described in the first aspect of the present invention.
[0046] In this embodiment of the invention, each labeled generated medical ultrasound image sample is aligned to the generated domain using an unconditional diffusion model of the generated domain, resulting in a generated domain medical ultrasound image set. This process mitigates the potential domain gap between the conditional and unconditional models. Based on this generated domain medical ultrasound image set, the source domain model is fine-tuned to align it to the generated domain, resulting in a generated domain model. This process alleviates the gap between the domain of the sample images created by the conditional diffusion model and the source domain model. Finally, during the prediction process, the target medical ultrasound image is aligned to the generated domain using the unconditional diffusion model of the generated domain, resulting in a generated domain target medical ultrasound image. Then, based on the generated domain model aligned to the generated domain, superior domain alignment is achieved for the generated domain target medical ultrasound image, transforming the cross-domain task into an intra-domain prediction task. The domains of the source domain model and the target data are aligned to the same generated domain, thereby obtaining accurate prediction and classification results. Attached Figure Description
[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0048] Figure 1 This is a flowchart illustrating the steps of the medical ultrasound image recognition method based on generator domain alignment provided in this embodiment of the invention.
[0049] Figure 2 This is a flowchart illustrating the medical ultrasound image recognition method based on generator domain alignment proposed in this embodiment of the invention.
[0050] Figure 3 This is a structural block diagram of a medical ultrasound image recognition system based on generator domain alignment provided in an embodiment of the present invention. Detailed Implementation
[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0052] Test-time adaptation (TTA) is an emerging research area that aims to address the domain offset problem in source domain model evaluation on target data with unknown offsets. Generally, TTA frameworks can be divided into two main types: 1) source-to-target domain model adaptation frameworks, which iteratively adjust the weights of the source domain model to better match the target data distribution; and 2) target-to-source domain data adaptation frameworks, which project the target data back into its source domain to achieve predictions within the source domain. Traditional TTA methods typically employ a source-to-target model adaptation framework. This method continuously updates the weights of the source domain model by processing batches of target data streams. However, due to the lack of labeling in the target data, this adaptation process either relies on batch statistics for updating the model or on additional unsupervised or self-supervised auxiliary tasks. Furthermore, batches of target data may not accurately represent the true distribution of the target domain, especially when the batches are small or exhibit class imbalance. Therefore, traditional model adaptation-based TTA methods are highly sensitive to the quantity and order of the target data streams.
[0053] In related technologies, the excellent image generation capabilities of diffusion models have inspired a diffusion-driven TTA method, which employs a target data-to-source domain model framework. Utilizing an unconditional diffusion model pre-trained in the source domain, each target image is projected back into the source domain, allowing the source domain model to make predictions without weight adaptation. Specifically, this technique first uses a diffusion model to clean the adversarial target data, then employs a forward diffusion process to diffuse the target data by adding a small amount of noise, and finally uses a backward diffusion process to recover the clean image, projecting each target image back into the source domain.
[0054] However, while diffusion-driven TTA methods aim to project target data back to the source domain, the projected target data remains confined to the generator domain of the unconditional diffusion model. Since data from the generator domain is ultimately processed by the source domain model, this domain misalignment limits the model's final predictive performance. To address this issue, this invention proposes Synthetic-Domain Alignment (SDA), which aligns the domains of both the source domain model and the target data with the generator domain of the diffusion model. The proposed SDA framework comprises two phases.
[0055] In the first stage, this invention proposes a supervised data fine-tuning method to adapt the source domain model to the generation domain of an unconditional diffusion model. Specifically, a conditional diffusion model is first used to generate samples with domain-agnostic labels as conditions, thereby creating a labeled generation dataset. Subsequently, an unconditional diffusion model is used to add noise to each sample and then denoise it. This process mitigates the potential domain gap between the conditional and unconditional models. Finally, a sufficiently large generation domain dataset is obtained, and the source domain model is fine-tuned based on this dataset. The fine-tuned model has a high discriminative ability for the generation domain data.
[0056] In the second stage, embodiments of the present invention align the target data to the generation domain of the unconditional diffusion model. Therefore, the SDA framework proposed in embodiments of the present invention can align both the source domain model and the target data domain to the same generation domain, effectively transforming cross-domain TTA tasks into intra-domain prediction tasks.
[0057] In this embodiment of the invention, the main contribution of SDA is to propose a general TTA framework from the perspective of generator domain alignment. Therefore, this framework does not depend on specific source domain model fine-tuning techniques or diffusion-driven target data adaptation methods.
[0058] This invention also provides a specific application method for applying the SDA framework to medical ultrasound image recognition. Specifically, this invention proposes a medical ultrasound image recognition method based on generator domain alignment, such as... Figure 1 The diagram illustrates a flowchart of a medical ultrasound image recognition method based on generator domain alignment provided in an embodiment of the present invention. The method includes the following steps:
[0059] S101, Using a conditional diffusion model, labeled generated medical ultrasound image samples are created for fine-tuning, wherein the labels are used to characterize the category corresponding to the generated medical ultrasound image samples.
[0060] S102, using the unconditional diffusion model of the generation domain, each labeled generated medical ultrasound image sample is aligned to the generation domain to obtain the generated domain medical ultrasound image set.
[0061] S103, the source domain model is fine-tuned using the generated domain medical ultrasound image set to obtain the generated domain model;
[0062] S104, The target medical ultrasound image is aligned to the generation domain using the unconditional diffusion model of the generation domain to obtain the target medical ultrasound image in the generation domain.
[0063] S105, input the target medical ultrasound image into the generator domain model to obtain the first predicted classification result of the target medical ultrasound image.
[0064] In this embodiment of the invention, a medical ultrasound image recognition method under the SDA framework is implemented based on the above steps S101-105. Specifically, Figure 2 This diagram illustrates the flowchart of the medical ultrasound image recognition method based on generator domain alignment proposed in an embodiment of the present invention. Figure 2 As shown, the method includes two stages. In the first stage, the source domain model is aligned to the generator domain, which is specifically implemented by steps S101 to S103. In the second stage, the target medical ultrasound image is aligned to the generator domain, which is specifically implemented by step S104.
[0065] Specifically, in this embodiment of the invention, the source domain model, the target medical ultrasound image, and the generator domain are aligned. In steps S101 to S103, labeled generator medical ultrasound image samples are first generated using a label-based conditional diffusion model. Then, the generated medical ultrasound image samples are aligned to the generator domain using an unconditional diffusion model. The source domain model is then fine-tuned using the aligned generator domain medical ultrasound image set to obtain a generator domain model adapted to the generator domain. In step S104, the target medical ultrasound image is projected into the generator domain using unconditional diffusion. Finally, in step S105, the generator domain model of the generator domain processes the projected generator domain target medical ultrasound image to generate an accurate prediction result.
[0066] In this embodiment of the invention, the source domain model is a model f with parameters θ, trained in a classic visual recognition setting. θ This model is based on specific source domain data. The training obtained above, for the input Model f θ Generate conditional output distribution and accurately predicted labels However, when the target domain data is offset When conducting the evaluation, the source domain model f θ Significant performance degradation is typically observed due to domain migration.
[0067] In this embodiment of the invention, each labeled generated medical ultrasound image sample is aligned to the generated domain using an unconditional diffusion model of the generated domain, resulting in a generated domain medical ultrasound image set. This process mitigates the potential domain gap between the conditional and unconditional models. Based on this generated domain medical ultrasound image set, the source domain model is fine-tuned to align it to the generated domain, resulting in a generated domain model. This process alleviates the gap between the domain of the sample images created by the conditional diffusion model and the source domain model. Finally, during the prediction process, the target medical ultrasound image is aligned to the generated domain using the unconditional diffusion model of the generated domain, resulting in a generated domain target medical ultrasound image. Then, based on the generated domain model aligned to the generated domain, superior domain alignment is achieved for the generated domain target medical ultrasound image, transforming the cross-domain task into an intra-domain prediction task. The domains of the source domain model and the target data are aligned to the same generated domain, thereby obtaining accurate prediction and classification results.
[0068] Based on the medical ultrasound image recognition method based on generator domain alignment provided in the above embodiments, in an optional embodiment, step S102 specifically includes: adding noise and denoising to each labeled generated medical ultrasound image sample using the unconditional diffusion model of the generator domain, so as to align each generated medical ultrasound image sample to the generator domain and obtain a generator domain medical ultrasound image set.
[0069] Step S104 specifically includes: adding noise and denoising the target medical ultrasound image using the unconditional diffusion model of the generation domain, so as to align the target medical ultrasound image to the generation domain and obtain the target medical ultrasound image in the generation domain.
[0070] In this embodiment of the invention, the generated domain, constructed by adding and removing noise from data generated through an unconditional diffusion model, is used as an intermediate domain. This intermediate domain is used to adapt the source domain model to the target medical ultrasound image. The domains of the source domain model and the target medical ultrasound image are aligned with the same generated domain. Because the domain gap is mitigated, the prediction accuracy of the source domain model on the target medical ultrasound image can be enhanced.
[0071] Based on the medical ultrasound image recognition method based on generator domain alignment provided in the above embodiments, in an optional embodiment, step S101 specifically includes:
[0072] Based on the K categories corresponding to the K class labels, a t-step back diffusion process is performed using a conditional diffusion model to uniformly generate N generated medical ultrasound image samples for each category from random Gaussian noise.
[0073] In this embodiment of the invention, the conditional diffusion model can be any mature model with image generation function in related technologies. Specifically, a conditional diffusion model with parameter η can be used. The T-step backdiffusion process shown in formula (4) above uniformly generates each of the K categories y from random Gaussian noise. i N samples. Specifically, the generative capability of the conditional diffusion model allows for the construction of arbitrarily large labeled generative domain datasets. No manual data collection is required.
[0074] Specifically, increasing the number of images helps the source domain model capture the generated domain more accurately during fine-tuning, thereby improving performance. Based on the balance between performance improvement and the time cost of generating images, this embodiment of the invention proposes using 50,000 generated medical ultrasound image samples as the default setting.
[0075] In this embodiment, step S105 includes: inputting the target medical ultrasound image of the generation domain into the generation domain model to obtain the probability of the target medical ultrasound image belonging to each of the K categories.
[0076] In this embodiment of the invention, K class labels can be determined based on multiple categories that the target medical ultrasound image to be predicted may correspond to (e.g., disease category represented by ultrasound image, organ category represented by ultrasound image, etc.). Based on these K class labels, for each category y i Generate N corresponding labeled medical ultrasound image samples.
[0077] Based on these labeled generated medical ultrasound image samples aligned to the generation domain, a generated domain medical ultrasound image set is obtained. The source domain model is then fine-tuned to obtain the generation domain model. This generation domain model can predict the probability of an image belonging to each of K categories for a given target medical ultrasound image. This allows for accurate prediction of the image's class label.
[0078] Based on the medical ultrasound image recognition method based on generator domain alignment provided in the above embodiments, in an optional embodiment, step S102 specifically includes: adding standard Gaussian noise to the generated medical ultrasound image sample generated by the conditional diffusion model to generate a noisy medical ultrasound image; and restoring the noisy medical ultrasound image to a clean medical ultrasound image of the generator domain through the unconditional diffusion model of the generator domain.
[0079] Step S104 specifically includes: adding standard Gaussian noise to the target medical ultrasound image to generate a noisy target medical ultrasound image; and restoring the noisy target medical ultrasound image to a clean target medical ultrasound image in the generation domain using the unconditional diffusion model of the generation domain.
[0080] Specifically, the diffusion model involves a forward process that converts an image into noise and a backward process that converts noise back into an image. Specifically, the forward process is a Markov chain that gradually converts random Gaussian noise ∈t ~N(0, I) are added to the image sampled from the true data distribution x0~p(x0), requiring a total of T steps. In step t, the noisy data x... t The calculation is as follows:
[0081]
[0082] Where, β t ∈(0,1) is the preset diffusion rate at step t, and α is set. t =1-β t ∈~N(0,I) we can obtain:
[0083]
[0084] When T becomes large enough, α t It tends towards zero. Therefore, x T It is close to ∈ ~N(0, I).
[0085] Given noise data x t As input, along with the time step t and the optional condition y of the conditional diffusion model, the diffusion model ξ is trained. A common training objective used during the training process is:
[0086]
[0087] After training is complete, the diffusion model can achieve diffusion from x through a back-diffusion process. T The noise is gradually removed to obtain a denoised image sequence.
[0088]
[0089] Where σ t For the posterior noise variance, from Sure.
[0090] In this embodiment of the invention, an unconditional diffusion model of the generative domain is used to align the generative medical ultrasound image sample set generated by the conditional diffusion model. In this alignment process, standard Gaussian noise is first added according to a specific time step t. Based on formula (2), a noisy process is created through forward diffusion. Subsequently, based on formula (4), a reverse diffusion process is used to... Restored to a clean state The unconditional diffusion model aligns the generated medical ultrasound image sample set used to fine-tune the source domain model and the target medical ultrasound image used for prediction to the same generating domain, thereby aligning the fine-tuned generating domain model and the target medical ultrasound image to the generating domain.
[0091] In an optional embodiment of the medical ultrasound image recognition method based on generator domain alignment provided in the above embodiments, the method further includes the following steps:
[0092] S106, Input the target medical ultrasound image into the source domain model to obtain the second predicted classification result of the target medical ultrasound image.
[0093] S107, Based on the first prediction classification result and the second prediction classification result, the final prediction classification result of the target medical ultrasound image is obtained.
[0094] In this embodiment of the invention, a source domain model and a generator domain model can be stored. The target medical ultrasound image is predicted based on the source domain model, and the target medical ultrasound image in the generator domain is predicted based on the generator domain model. Based on the two prediction results, the final predicted classification result of the target medical ultrasound image is obtained, which can be specifically represented as follows:
[0095]
[0096] Where, p θ (·) represents the prediction result obtained by predicting the target medical ultrasound image based on the source domain model, p θ′ (·) represents the prediction result obtained by predicting the target medical ultrasound image in the generative domain based on the generative domain model.
[0097] Based on the same inventive concept, embodiments of the present invention also provide a medical ultrasound image recognition system based on generator domain alignment, the system comprising:
[0098] The sample creation module 301 is used to create labeled generated medical ultrasound image samples for fine-tuning using a conditional diffusion model, wherein the labels are used to characterize the category corresponding to the generated medical ultrasound image sample.
[0099] The sample alignment module 302 is used to align each labeled generated medical ultrasound image sample to the generation domain using the unconditional diffusion model of the generation domain, so as to obtain the generation domain medical ultrasound image set.
[0100] The fine-tuning module 303 is used to fine-tune the source domain model using the generated domain medical ultrasound image set to obtain the generated domain model;
[0101] The target image alignment module 304 is used to align the target medical ultrasound image to the generation domain using the unconditional diffusion model of the generation domain, so as to obtain the target medical ultrasound image in the generation domain.
[0102] The first prediction module 305 is used to input the target medical ultrasound image of the generation domain into the generation domain model to obtain the first prediction classification result of the target medical ultrasound image.
[0103] Optionally, the sample alignment module 302 is specifically used for:
[0104] Noise is added and denoised for each labeled generated medical ultrasound image sample using the unconditional diffusion model of the generation domain, so that each generated medical ultrasound image sample is aligned to the generation domain, resulting in a set of generated medical ultrasound images.
[0105] The target image alignment module 304 is specifically used for:
[0106] Noise is added to and denoised using the unconditional diffusion model of the generator domain to align the target medical ultrasound image to the generator domain, thus obtaining the generator domain target medical ultrasound image.
[0107] Optionally, the sample alignment module 302 is specifically used for:
[0108] Standard Gaussian noise is added to the generated medical ultrasound image samples generated by the conditional diffusion model to generate noisy medical ultrasound images.
[0109] The noisy medical ultrasound image is restored to a clean medical ultrasound image of the generated domain using the unconditional diffusion model of the generated domain.
[0110] Optionally, the sample creation module 301 is specifically used for:
[0111] Based on the K categories corresponding to the K class labels, t-step back diffusion processing is performed through the conditional diffusion model to uniformly generate N medical ultrasound image samples for each category from random Gaussian noise;
[0112] The first prediction module 305 is specifically used for:
[0113] Input the target medical ultrasound image into the generator domain model to obtain the probability that the target medical ultrasound image belongs to each of the K categories.
[0114] Optionally, the system further includes:
[0115] The second prediction module is used to input the target medical ultrasound image into the source domain model to obtain the second prediction classification result of the target medical ultrasound image.
[0116] The classification result determination module is used to obtain the final predicted classification result of the target medical ultrasound image based on the first predicted classification result and the second predicted classification result.
[0117] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the robust image semantic communication method with a multi-scale visual transformer as described in any of the above embodiments.
[0118] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the robust image semantic communication method with a multi-scale visual transformer described in any of the above embodiments.
[0119] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0120] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0121] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (apparatus), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0123] These computer program instructions can also be loaded onto a computer or other programmable terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0124] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.
[0125] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0126] The above provides a detailed description of a medical ultrasound image recognition method and system based on generator domain alignment provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A medical ultrasound image recognition method based on generated domain alignment, characterized in that, The method includes: A conditional diffusion model is used to create labeled generated medical ultrasound image samples for fine-tuning, wherein the labels are used to characterize the category corresponding to the generated medical ultrasound image sample. The unconditional diffusion model of the generation domain is used to align each labeled generated medical ultrasound image sample to the generation domain to obtain a set of generated medical ultrasound images. This includes adding noise and denoising each labeled generated medical ultrasound image sample using the unconditional diffusion model of the generation domain to align each generated medical ultrasound image sample to the generation domain to obtain a set of generated medical ultrasound images. The source domain model was fine-tuned using a set of medical ultrasound images from the generative domain to obtain the generative domain model. Aligning the target medical ultrasound image to the generation domain using the unconditional diffusion model of the generation domain to obtain the target medical ultrasound image in the generation domain includes: adding noise and denoising the target medical ultrasound image using the unconditional diffusion model of the generation domain to align the target medical ultrasound image to the generation domain to obtain the target medical ultrasound image in the generation domain. Input the target medical ultrasound image into the generator domain model to obtain the first predicted classification result of the target medical ultrasound image. Each labeled generated medical ultrasound image sample is aligned to the generation domain using an unconditional diffusion model of the generation domain, specifically including: Standard Gaussian noise is added to the generated medical ultrasound image samples generated by the conditional diffusion model to generate noisy medical ultrasound images. The noisy medical ultrasound image is restored to a clean medical ultrasound image of the generated domain using the unconditional diffusion model of the generated domain. 2.The medical ultrasound image recognition method based on generation domain alignment according to claim 1, characterized in that, Labeled generative medical ultrasound image samples for fine-tuning are created using a conditional diffusion model, including: Based on the K categories corresponding to the K class labels, t-step back diffusion processing is performed through the conditional diffusion model to uniformly generate N medical ultrasound image samples for each category from random Gaussian noise; Inputting the target medical ultrasound image into the generator domain model yields the first predicted classification result of the target medical ultrasound image, including: Input the target medical ultrasound image into the generator domain model to obtain the probability that the target medical ultrasound image belongs to each of the K categories.
3. The method of claim 1 or 2, wherein the method further comprises: The method further includes: Input the target medical ultrasound image into the source domain model to obtain the second predicted classification result of the target medical ultrasound image; Based on the first and second prediction classification results, the final prediction classification result of the target medical ultrasound image is obtained.
4. A medical ultrasound image recognition system based on generative domain alignment, characterized by, The system includes: The sample creation module is used to create labeled generated medical ultrasound image samples for fine-tuning using a conditional diffusion model, wherein the labels are used to characterize the category corresponding to the generated medical ultrasound image sample. The sample alignment module is used to align each labeled generated medical ultrasound image sample to the generation domain using the unconditional diffusion model of the generation domain, thereby obtaining a set of generated medical ultrasound images. Specifically, it is used to add noise and denoise each labeled generated medical ultrasound image sample using the unconditional diffusion model of the generation domain, so as to align each generated medical ultrasound image sample to the generation domain, thereby obtaining a set of generated medical ultrasound images. The fine-tuning module is used to fine-tune the source domain model using the generative domain medical ultrasound image set to obtain the generative domain model; The target image alignment module is used to align the target medical ultrasound image to the generation domain using the unconditional diffusion model of the generation domain, so as to obtain the target medical ultrasound image in the generation domain; specifically, it is used to add noise and remove noise to the target medical ultrasound image using the unconditional diffusion model of the generation domain, so as to align the target medical ultrasound image to the generation domain, so as to obtain the target medical ultrasound image in the generation domain. The prediction module is used to input the target medical ultrasound image from the generator domain into the generator domain model to obtain the first predicted classification result of the target medical ultrasound image. The sample alignment module is also specifically used for: Standard Gaussian noise is added to the generated medical ultrasound image samples generated by the conditional diffusion model to generate noisy medical ultrasound images. The noisy medical ultrasound image is restored to a clean medical ultrasound image of the generated domain using the unconditional diffusion model of the generated domain.
5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the medical ultrasound image recognition method based on generator domain alignment as described in any one of claims 1-3.
6. A computer-readable storage medium having stored thereon a computer program, characterized in that, When executed by a processor, the computer program implements the medical ultrasound image recognition method based on generator domain alignment as described in any one of claims 1-3.