Medical image processing method, model training method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a medical image processing method, a model training method, an electronic device and a storage medium. The medical image processing method comprises: predicting an sub-optimal noise center based on a source modality medical image and a reference target modality medical image corresponding to the source modality medical image; projecting the sub-optimal noise center to a predetermined hypersphere to obtain a region center of a sub-optimal noise region; screening noise in a predetermined noise image based on the region center to obtain guide noise; and performing image diffusion generation processing on the source modality medical image based on the guide noise to obtain a target modality medical image. The present scheme can improve the processing effect of cross-modality generation of medical images.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a medical image processing method, a model training method, an electronic device, a computer storage medium, and a computer program product. Background Technology

[0002] Medical image cross-modal generation is a computer vision technology that can convert medical images from one modality into medical images from another to aid in clinical diagnosis. As an example, medical image cross-modal generation can convert plain CT (Computed Tomography) images (i.e., CT images taken without contrast agents) into enhanced CT images (i.e., CT images enhanced by injecting contrast agents into blood vessels). Enhanced CT can be understood exemplarily as CTPA (CT Pulmonary Angiography).

[0003] Currently, one approach to cross-modal generation of medical images is through a diffusion model. This model starts with randomly sampled guiding noise and then gradually transforms the source modality image into the target modality image through a multi-step iterative denoising process. However, the diffusion model in this field relies on the initial randomly sampled guiding noise for cross-modal image generation. This approach results in low-quality target modality medical images, impacting the reliability and clinical application value of cross-modal medical image generation. Summary of the Invention

[0004] In view of this, embodiments of this application provide a medical image processing and model training scheme to at least partially solve the above problems.

[0005] According to a first aspect of the embodiments of this application, a medical image processing method is provided, comprising: predicting a suboptimal noise center based on a source modal medical image and a reference target modal medical image corresponding to the source modal medical image; projecting the suboptimal noise center onto a predetermined hypersphere to obtain a region center of a suboptimal noise region; filtering noise in a predetermined noise image based on the region center to obtain guiding noise; and performing image diffusion generation processing on the source modal medical image based on the guiding noise to obtain a target modal medical image.

[0006] According to a second aspect of the embodiments of this application, a model training method is provided, comprising: acquiring training samples, wherein the training samples include source modality medical image samples and target modality medical image samples corresponding to the source modality medical image samples; inputting the training samples into a suboptimal noise predictor to perform cascaded quantization feature extraction processing based on the training samples to generate predicted noise samples; inputting the predicted noise samples and the training samples into a diffusion bridge model to predict and generate a target modality medical prediction image based on the source modality medical image samples and the predicted noise samples in the training samples; and training the suboptimal noise predictor based on the difference between the target modality medical prediction image and the target modality medical image samples.

[0007] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory is used to store a computer program; and the processor is used to execute the aforementioned medical image processing method or model training method by running the computer program stored in the memory.

[0008] According to a fourth aspect of the embodiments of this application, a computer storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the medical image processing method or model training method as described above.

[0009] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the medical image processing method or model training method as described above.

[0010] The medical image processing scheme provided in this application can predict the suboptimal noise center and project it onto a predetermined hypersphere to determine the regional center of the suboptimal noise region. Then, based on the regional center, the noise in the candidate noise is screened to obtain guiding noise. Based on the guiding noise, the source modality medical image is processed to generate an image diffusion model to obtain a target modality medical image. Therefore, on the one hand, this scheme introduces a suboptimal noise center and a hyperspherical projection mechanism to model and locate noise regions that may lead to a decrease in generation quality in a high-dimensional noise space. This region is actively avoided through a screening process, ensuring that the guiding noise used for image diffusion generation falls within a noise range more conducive to high-quality generation. This effectively reduces generation instability caused by the randomness of the initial guiding noise, improving the consistency and reliability of cross-modal generation results for medical images. On the other hand, the hyperspherical projection mechanism used in this scheme utilizes the property that high-dimensional random Gaussian noise is geometrically concentrated near a hypersphere. It projects the suboptimal noise center onto a unified predetermined hypersphere, ensuring geometric consistency in the similarity measurement between candidate noise and the region center. This ensures the effectiveness and robustness of the noise screening strategy in high-dimensional space, making the selection of guiding noise more precise. The resulting target modality medical image exhibits more stable and accurate presentation of key pathological structures and anatomical details. Therefore, the technical solution provided in this application can effectively improve the processing effect of cross-modal generation of medical images and enhance image generation quality. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0012] Figure 1 This is a schematic diagram of a medical image processing system according to an embodiment of this application.

[0013] Figure 2 This is a flowchart of the steps of a medical image processing method according to an embodiment of this application.

[0014] Figure 3 This is a flowchart illustrating an example of a process for predicting a suboptimal noise center according to an embodiment of this application.

[0015] Figure 4 This is a flowchart illustrating an example of predicting a suboptimal noise center using a suboptimal noise predictor according to an embodiment of this application.

[0016] Figure 5 This is a flowchart illustrating an example of filtering to obtain guiding noise according to an embodiment of this application.

[0017] Figure 6 This is a flowchart of the steps of a model training method according to an embodiment of this application.

[0018] Figure 7A This is a flowchart illustrating an example of obtaining a predicted noise sample using a suboptimal noise predictor according to an embodiment of this application.

[0019] Figure 7B This is a schematic diagram of a diffusion bridge model example.

[0020] Figure 8 This is a flowchart illustrating the steps of generating a target modality medical prediction image according to an embodiment of this application.

[0021] Figure 9 This is a flowchart illustrating the steps of training a suboptimal noise predictor based on differences, according to an embodiment of this application.

[0022] Figure 10 A flowchart illustrating an example of a model training scenario according to an embodiment of this application is shown.

[0023] Figure 11 A schematic diagram illustrating the implementation process of an example of a medical image processing scenario according to an embodiment of this application is shown.

[0024] Figure 12 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0026] Medical image cross-modal generation is a computer vision technique that can convert medical images from one modality into medical images from another to aid in clinical diagnosis. As an example, medical image cross-modal generation can convert plain CT images into enhanced CT images. Enhanced CT images can be understood through the example of CTPA.

[0027] For example, pulmonary embolism is a serious and life-threatening disease, making early diagnosis and treatment crucial. Taking the diagnosis of pulmonary embolism as an example, pulmonary embolism presents with nonspecific symptoms, and contrast-enhanced CT scans are an effective clinical diagnostic tool with high sensitivity and specificity. However, contrast-enhanced CT requires the injection of contrast agents, which can cause adverse consequences such as allergic reactions and kidney damage, and it is difficult to use around the clock in resource-limited areas. In contrast, plain CT scans are more convenient and widely available, but because the contrast difference between the embolic area and surrounding pulmonary vessels in plain CT images is minimal, radiologists are prone to false negatives when using plain CT to diagnose pulmonary embolism. Therefore, computer technology for cross-modal generation of medical images to automatically convert plain CT images into contrast-enhanced CT images can effectively assist in clinical diagnosis.

[0028] Currently, one approach to cross-modal generation of medical images is through a diffusion model. This model starts with randomly sampled guiding noise and then gradually transforms the source modality image into the target modality image through a multi-step iterative denoising process. However, in performing cross-modal image generation tasks, the diffusion model relies on the initial randomly sampled guiding noise. This approach often results in low-quality target modality medical images, affecting the reliability and clinical application value of cross-modal medical image generation. For example, this approach struggles to determine the quality of the initial guiding noise, typically treating it as a purely random variable and relying on random sampling. However, its random sampling strategy lacks a systematic understanding of "bad" noise, leading to a significant probability of sampling such noise and resulting in severe instability in the cross-modal generation of medical images. This makes the model unreliable in clinical applications, as it cannot guarantee the consistent synthesis of correct pathological features each time.

[0029] In complex source modal medical images, the signal-to-noise ratio of key features such as lesions is extremely low. Small random perturbations of guiding noise are gradually amplified during the multi-step iterative generation process, eventually causing the generation path to deviate from the trajectory. This results in the same source modal medical image being generated in different generation stages, producing target modal medical images of drastically different quality and with inconsistent key pathological features, which seriously reduces the reliability and clinical application value of cross-modal medical image generation.

[0030] In view of this, this application provides some new medical image processing schemes and model training schemes to improve the stability and reliability of cross-modal generation results of medical images and improve the processing effect of cross-modal generation of medical images.

[0031] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0032] Figure 1An exemplary system applicable to embodiments of this application is shown. For example... Figure 1 As shown, the system 100 may include a cloud server 102, a communication network 104, and / or one or more user devices 106. Figure 1 The example in the text uses multiple user devices 106, which can be used to receive access from a first user to services provided by a second user. For example, a user can access a service platform through user device 106 to access services provided by a second user through the service platform. Optionally, the service platform can be located on a cloud server 102.

[0033] The cloud server 102 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, the cloud server 102 can perform any suitable function.

[0034] For example, when the cloud server 102 implements the scheme of the embodiments of this application, in some embodiments, the cloud server 102 can be used to execute a medical image processing method. As an optional example, in some embodiments, the cloud server 102 can first predict a suboptimal noise center based on the source modal medical image and the reference target modal medical image corresponding to the source modal medical image; then, the suboptimal noise center can be projected onto a predetermined hypersphere to obtain the region center of the suboptimal noise region; then, based on the region center, the noise in the predetermined noise image is filtered to obtain guiding noise; then, based on the guiding noise, the source modal medical image is subjected to image diffusion generation processing to obtain the target modal medical image. Furthermore, the cloud server 102 can also feed back the generated target modal medical image to the user device 106.

[0035] To facilitate the implementation of the above-described medical image processing method, in some embodiments, the cloud server 102 can also be used to execute a model training method. As an optional example, in some embodiments, the cloud server 102 can first acquire training samples, which include source modality medical image samples and target modality medical image samples corresponding to the source modality medical image samples; then, the training samples are input into a suboptimal noise predictor to perform cascaded quantization feature extraction based on the training samples, generating predicted noise samples; then, the predicted noise samples and the training samples are input into a diffusion bridge model to predict and generate a target modality medical image prediction based on the source modality medical image samples and the predicted noise samples in the training samples; finally, the suboptimal noise predictor is trained based on the difference between the target modality medical image prediction image and the target modality medical image samples. The model trained in this way can be effectively used to subsequently implement the above-described medical image processing method.

[0036] In some embodiments, the communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The user equipment 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the cloud server 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between the user equipment 106 and the cloud server 102, such as a network link, a dial-up link, a wireless link, a hardwired link, any other suitable communication link, or any suitable combination of such links.

[0037] Optionally, user equipment 106 may include any one or more user devices suitable for displaying information, interacting with users, etc. In some embodiments, user equipment 106 receives a trigger operation when a first user accesses a service provided by a second user, and generates trigger information based on the trigger operation and sends it to cloud server 102. In some embodiments, user equipment 106 may include any suitable type of device. For example, in some embodiments, user equipment 106 may include mobile devices, tablet computers, laptop computers, desktop computers, and / or any other suitable type of user device.

[0038] Based on the above system, this application provides a medical image processing scheme and a model training scheme, which will be described below through several embodiments.

[0039] Figure 2 This is a flowchart of the steps of a medical image processing method according to an embodiment of this application, referring to... Figure 2 As shown, the medical image processing method includes the following steps: S202: Based on the source modal medical image and the corresponding reference target modal medical image, the suboptimal noise center is predicted.

[0040] In this embodiment, the source modal medical image can refer to a medical image that serves as the starting point for cross-modal generation. That is, the source modal medical image is the medical image that needs to be transformed across modalities. Before executing the medical image processing method, the source modal medical image can be obtained through methods such as medical image capture.

[0041] The source modality can be any suitable medical image modality. As some examples, the source modality medical image can be a plain CT image. As other examples, the source modality medical image can be an MRI (T1-weighted Magnetic Resonance Imaging) image. Appropriate source modalities of medical images can be selected for processing as needed to flexibly meet the requirements of cross-modal conversion.

[0042] It should be noted that the source modal medical image can be a single slice image or include multiple slice images; this application does not limit this in the embodiments.

[0043] Unlike conventional modal conversion (e.g., text to image, image to video, text to audio, audio to text, etc.), in the medical field, medical image modal conversion can transform images from one medical imaging modality (e.g., CT, MRI, ultrasound, etc.) into another modality to express information that is difficult to present in one modality (e.g., soft tissue contrast, metabolic activity, hemodynamics, etc.). Examples include the conversion from MRI to CT images, from ultrasound to PET (Positron Emission Tomography) images, from plain CT to enhanced CT images, and from T1-weighted MRI to T1-contrast-enhanced MRI (T1CE) images.

[0044] Based on this, the target modal medical image in the embodiments of this application can be a target modal medical image obtained by processing the source modal medical image and that is different from the source modality.

[0045] The reference target modality medical image can be a target modality reference medical image predicted in advance based on the source modality medical image. For example, the reference target modality medical image can be predicted from the source modality medical image using a trained diffusion model (such as a diffusion bridge model). It should be noted that although both the reference target modality medical image and the target modality medical image are target modalities, the reference target modality medical image is not directly used as the final target modality medical image output. Instead, it serves as an auxiliary input for predicting suboptimal noise centers, providing corresponding target domain reference information for subsequent noise quality assessment.

[0046] In this embodiment, the source modal medical image and the reference target modal medical image can be used as an input pair for suboptimal noise center prediction. The input pair is then input into the suboptimal noise predictor to predict the suboptimal noise center.

[0047] The suboptimal noise center characterizes the central location of noise distribution in a high-dimensional noise space that easily leads to poor-quality results in the image diffusion generation process. In the diffusion-based image generation process, the selection of initial guiding noise significantly impacts the quality and features of the final generated image. Empirical studies and related research in the embodiments of this application show that "good" noise capable of generating high-quality images is sparse throughout the noise space, while "bad" noise leading to generation failure tends to cluster in certain specific, structured regions, and the center of these clustered regions is the suboptimal noise center. In the embodiments of this application, by accurately predicting the suboptimal noise center, the region where "bad noise" clusters can be actively avoided during subsequent sampling of guiding noise. A "no-go zone" can be dynamically defined on a preset hypersphere, efficiently rejecting noise that may lead to generation failure, thereby constraining the cross-modal generation process of medical images within a higher-quality region. Clearly, such a guided noise sampling strategy is far more efficient and robust than traditional random sampling or expensive real-time optimization.

[0048] For ease of explanation, the source modal medical image will be denoted as... The reference target modal medical image is denoted as The target modal medical image is denoted as .

[0049] In some alternative embodiments, refer to Figure 3 The flowchart shown illustrates that step S202, based on the source modality medical image and the corresponding reference target modality medical image, predicts the suboptimal noise center, which may include the following steps: S2022: Input the source modal medical image into the trained diffusion model to perform single-step image diffusion generation processing to obtain the reference target modal medical image.

[0050] In this embodiment, the trained diffusion model has the ability to convert source modality medical images into target modality medical images. In one example, the diffusion model can be a diffusion bridge model. Diffusion bridge models learn a direct stochastic process from source modality images to target modality images, thereby achieving image-to-image conversion and generation tasks. They exhibit superior performance in tasks such as cross-modal generation of medical images. The appropriate type of diffusion bridge model can be selected as needed. For example, DDBM (Denoising Diffusion Bridge Models) and DBIM (Diffusion Bridge Implicit Models) can be used.

[0051] In step S2022, the source modality medical image can be processed using a diffusion model to generate a reference target modality medical image through a single-step image diffusion generation process, without performing a complete multi-step image diffusion generation process. It should be understood that although the reference target modality medical image obtained through this single-step image diffusion generation has slightly lower image quality compared to multi-step generation, it is still sufficient to provide reasonable target domain reference information for predicting suboptimal noise centers. Therefore, through the above-described single-step image diffusion generation process, the computational efficiency of the reference target modality medical image can be effectively improved while meeting the subsequent requirements for predicting suboptimal noise centers.

[0052] S2024: Input the source modal medical image and the reference target modal medical image into the trained suboptimal noise predictor for processing, and predict the suboptimal noise center.

[0053] In this embodiment, the trained suboptimal noise predictor can predict the corresponding suboptimal noise center based on a given input pair consisting of a source modality medical image and a reference target modality image. Optionally, the suboptimal noise predictor can be a model built on an encoder-decoder architecture, which can be trained using a large number of training samples. Based on learning from a large number of training samples, the distribution of suboptimal noise in a high-dimensional space can be parameterized, thereby accurately predicting the corresponding suboptimal noise center for the input data during the inference stage.

[0054] It should be understood that, in the embodiments of this application, through the optional implementation of the above steps S2022~S2024, the source modal medical image is processed by single-step image diffusion generation through a diffusion model to obtain a reference target modal medical image. Then, the source modal medical image and the reference target modal medical image are input into the suboptimal noise predictor for processing. Based on the ability to accurately predict the suboptimal noise center, the computational efficiency is further improved effectively.

[0055] The specific model structure of the suboptimal noise predictor is not limited in the embodiments of this application. Optionally, the suboptimal noise predictor may include an encoder, a cascaded quantizer, and a decoder. Step S2024 described above can be implemented in any suitable manner. In some optional embodiments, refer to... Figure 4 In the flowchart, step S2024 may include the following S2024A~S2024C, specifically: S2024A: Input the stitched source modal medical image and the reference target modal medical image into the encoder to obtain the feature map.

[0056] The encoder can perform feature extraction, mapping the input image to a high-dimensional latent feature map. Optionally, the encoder can adopt the network structure of the downsampling path of the U-Net network, or it can be implemented using other network structures as needed.

[0057] In one example, the U-Net network employs a symmetrical encoder-decoder structure. The encoder forms the downsampling path of the U-Net network, progressively extracting deep semantic features of the image through operations such as convolution and pooling, while simultaneously reducing the feature map size. The decoder forms the upsampling path of the U-Net network, progressively restoring the spatial resolution of the feature map through upsampling operations (such as deconvolution). Skip connections can be implemented between the encoder and decoder, concatenating feature maps from corresponding layers in the encoder and decoder to fuse low-level spatial details and high-level semantic information, thereby improving image processing accuracy. This is suitable for achieving high-precision processing of edges and small target regions.

[0058] In this embodiment of the application, the source modal medical image can be... and reference target modal medical images Perform splicing, and then display the spliced ​​result. The input is fed into the encoder, where it undergoes layer-by-layer downsampling and feature extraction to ultimately obtain a feature map that reflects the semantic features of the input image pair (i.e., the source modality medical image and the reference target modality medical image).

[0059] The encoder, which uses a suboptimal noise predictor, encodes the image information of the source and target modes, enabling the resulting feature maps to capture the relationship between the two modes and the image generation characteristics corresponding to the current input, thus providing high-quality feature representations for subsequent quantization processing.

[0060] S2024B: The feature map is quantized by a cascaded quantizer to obtain quantized features.

[0061] In this embodiment of the application, the cascaded quantizer may include a multi-stage quantizer, wherein the first-stage quantizer is used to quantize the feature map, and the (i+1)th-stage quantizer is used to quantize the quantization residual output by the i-th-stage quantizer, where i ≥ 1 and is an integer.

[0062] Quantization can be seen as the process of mapping continuous high-dimensional feature vectors to a finite discrete codebook. In Vector Quantization (VQ), the codebook consists of several discrete code vectors. The quantization process replaces the input feature vector with the closest code vector in the codebook, thereby achieving a discretized representation of the continuous feature space.

[0063] This application innovatively designs the aforementioned cascaded quantizer, which can be used to learn and parameterize the distribution of suboptimal noise. It employs a multi-stage quantizer structure. In this cascaded quantizer, the first-stage quantizer quantizes the feature map output by the encoder, the second-stage quantizer quantizes the quantization residual output by the first-stage quantizer, the third-stage quantizer quantizes the quantization residual output by the second-stage quantizer, and so on, with each subsequent stage quantizing the quantization residual of the previous stage. Through this cascaded quantizer structure, the feature map achieves progressive residual quantization. The cascaded quantizer can learn richer and more hierarchical discrete feature representations, effectively alleviating the "codebook collapse" problem common in traditional single-layer vector quantization techniques. This allows the suboptimal noise predictor to more accurately parameterize complex suboptimal noise distributions.

[0064] It should be understood that the embodiments of this application do not limit the specific number of quantizers included in the cascaded quantizer. The number of quantizers can be flexibly set as needed, and no restriction is imposed here.

[0065] S2024C: Suboptimal noise is predicted based on quantization features through the decoder and the jump connection between the decoder and the encoder, and the suboptimal noise center is determined based on the suboptimal noise.

[0066] The decoder can perform feature extraction, sampling the quantized features of the input layer by layer to ultimately reconstruct suboptimal noise. The suboptimal noise can be an image of the same size as the input image. Optionally, the decoder can employ a network structure with the upsampling path of a U-Net network, or other network structures as needed. During reconstruction, the decoder uses skip connections with the encoder to obtain high-resolution features from each layer of the encoder's output and fuses them with the sampled features. This effectively restores spatial details while preserving high-level semantic information, improving the accuracy and resolution of suboptimal noise prediction.

[0067] Subsequently, the corresponding suboptimal noise center can be determined based on the suboptimal noise. For example, statistical analysis can be performed on the suboptimal noise output by the decoder (e.g., by methods including but not limited to taking its spatial mean or performing overall normalization) to determine the suboptimal noise center, thereby obtaining the center vector representing the suboptimal noise aggregation region for use in the subsequent projection step of the predetermined hypersphere. The specific method for determining the suboptimal noise center can be flexibly selected according to actual needs, and this application embodiment does not limit it in this way.

[0068] Based on this, in the optional implementations of steps S2022A to S2024C above, the suboptimal noise predictor in this embodiment, including the encoder, cascaded quantizer, and decoder, and its corresponding complete processing flow, can efficiently extract information related to the suboptimal noise distribution from the joint features of the stitched source modality medical image and the reference target modality medical image, and parameterize the distribution in a discretized and hierarchical manner. Finally, the suboptimal noise center can be accurately predicted based on the predicted suboptimal noise. Furthermore, in this embodiment, by stitching the source modality medical image and the reference target modality medical image together and inputting them into the encoder, the suboptimal noise predictor can simultaneously perceive the image features of both modalities, thereby learning the corresponding suboptimal noise distribution for a specific input image pair, improving the personalization and accuracy of the prediction. Furthermore, the cascaded quantizer in this embodiment adopts a multi-layer quantizer structure and discretizes the features by means of progressive residual quantization. It can learn richer and more hierarchical discrete feature representations, which can effectively alleviate the "codebook collapse" problem commonly found in traditional single-layer vector quantization techniques. This allows the suboptimal noise predictor to more accurately parameterize complex suboptimal noise distributions, thereby providing reliable prior information for subsequent noise screening.

[0069] S204: Project the suboptimal noise center onto a predetermined hypersphere to obtain the region center of the suboptimal noise region.

[0070] A hypersphere is the set of all points in a high-dimensional Euclidean space that are equidistant from a central point (i.e., their radius). In high-dimensional space, sample points randomly sampled based on a standard Gaussian distribution tend to have their norm (i.e., their distance from the origin) concentrated around a specific value with high probability. This specific value is approximately equal to the square root of the spatial dimension. This phenomenon is known as the "hypersphere concentration" property of the high-dimensional Gaussian distribution. Based on this, random Gaussian noise samples in high-dimensional space almost all concentrate around a radius of approximately... ( D On a hypersphere (with dimensions 1-3), the hypersphere is in a hypersphere.

[0071] Therefore, projecting the suboptimal noise center onto the predetermined hypersphere allows the suboptimal noise center to be on the same geometric scale as the noise to be screened, making the suboptimal noise center compatible with the concentrated distribution properties of the noise on the predetermined hypersphere. This ensures the rationality and robustness of subsequent noise screening, and also guarantees the effectiveness and comparability of the similarity metrics (such as cosine similarity) used in subsequent noise screening.

[0072] The center of the suboptimal noise region can be characterized as the geometric center of the suboptimal noise accumulation region on the predetermined hypersphere. Subsequently, the noise can be screened using this center as a reference to actively avoid the suboptimal noise accumulation region and reject the sampling of "bad noise" as much as possible, thereby improving the screening effect.

[0073] In some optional embodiments, step S204 may include: normalizing the suboptimal noise center to a predetermined hypersphere radius, and projecting it onto the predetermined hypersphere corresponding to the hypersphere radius to obtain the region center of the suboptimal noise region.

[0074] Based on this, the suboptimal noise center can be accurately and effectively projected onto a predetermined hypersphere corresponding to the hypersphere radius, and the regional center of the suboptimal noise region can be accurately obtained.

[0075] The predetermined hypersphere radius can be a pre-defined hypersphere radius based on the dimensions of the noise space. This hypersphere radius is typically related to the square root of the noise vector dimension. Appropriate hypersphere radii can be set as needed; there is no single limitation.

[0076] The above normalization can be achieved in any suitable way. For example, alternatively, the vector of the suboptimal noise center can be divided by its own L2 norm to obtain a unit vector, and then the unit vector can be multiplied by the predetermined hypersphere radius to normalize the suboptimal noise center to the predetermined hypersphere radius, so as to project the suboptimal noise center onto the predetermined hypersphere, and thus obtain the vector of the region center of the normalized suboptimal noise region.

[0077] S206: Based on the region center, filter the noise in the predetermined noise image to obtain the guiding noise.

[0078] In this embodiment, a predetermined noise image can be obtained, and guiding noise can be obtained by filtering noise from the predetermined noise image based on the region center of the suboptimal noise region. The predetermined noise image can be a pure noise image obtained in any way.

[0079] The specific implementation of step S206 is not limited in the embodiments of this application. In some optional embodiments, refer to Figure 5In the flowchart shown, step S206 can be achieved by filtering the guiding noise through the following steps S2062~S2066, specifically: S2062: Sample the predetermined noisy image to obtain candidate noise.

[0080] Candidate noise is random noise that needs further screening; its suitability as guiding noise can be evaluated through subsequent steps. Optionally, it can be based on a standard Gaussian distribution. Candidate noise is obtained by randomly sampling from a predetermined noisy image. Optionally, the candidate noise can be the same size as the source modality medical image.

[0081] S2064: Based on the similarity between the region center and the candidate noise, determine whether to reject the sampled candidate noise.

[0082] In this embodiment, any type of similarity metric can be used to calculate the similarity between the region center and the candidate noise. For example, cosine similarity can be used. Alternatively, in other embodiments, Jaccard similarity coefficient or distance-based similarity metrics can be used, and the appropriate metric can be selected as needed.

[0083] Optionally, in step S2064, a rejection sampling mechanism can be used to evaluate candidate noise. For example, a rejection sampling condition can be set, and based on this condition, candidate noise that meets the condition is rejected, while candidate noise that does not meet the condition is accepted. In this embodiment, the rejection sampling is determined based on the similarity between the region center of the suboptimal noise region and the candidate noise.

[0084] In some optional embodiments, step S2064 may include: determining whether the similarity between the region center and the candidate noise meets a preset rejection rate, and determining whether to reject the sampled candidate noise based on the determined result.

[0085] Therefore, it is possible to accurately determine whether to reject the sampled candidate noise, so as to select the guiding noise based on the determination result for use in the cross-modal conversion generation of the source modality medical image to the target modality medical image in subsequent steps.

[0086] For example, in some embodiments, a Percentage of Rejection (PoR) can be preset, which represents the proportion of the worst noise region that is to be avoided. This can be set as needed, for example, it can be set to 30% in one example.

[0087] For example, with a rejection rate (PoR) of 30%, candidate noise with the highest similarity (e.g., cosine similarity) to the region center (top 30%) will be considered to fall into the suboptimal noise region and rejected. Optionally, a similarity threshold can be pre-calculated based on the rejection rate. When the similarity between the candidate noise and the region center is higher than this threshold, the candidate noise is considered to fall into the suboptimal noise region (which can be understood as the "worst" region defined by PoR) and rejected. When the similarity between the candidate noise and the region center is lower than this threshold, the similarity is low enough that the candidate noise is outside the suboptimal noise region and within the safe region, thus the candidate noise can be accepted.

[0088] S2066: If not, then the candidate noise is determined as pilot noise.

[0089] When it is determined that the candidate noise obtained from sampling is not rejected (i.e., the candidate noise is accepted), the candidate noise can be identified as pilot noise for use in the cross-modal conversion generation of the source modality medical image to the target modality medical image in subsequent steps.

[0090] In some alternative embodiments, if so, the candidate noise is rejected and the predetermined noise image is resampled to obtain new candidate noise (that is, return to step S2062 and repeat) until a candidate noise that can be identified as guiding noise is obtained.

[0091] Based on this, the optional implementation of steps S2062~S2066 in the embodiments of this application can, on the one hand, transform the selection of candidate noise in the preset noise image from a completely random process into a purposeful screening process by introducing a noise screening mechanism based on the similarity between the region center and the candidate noise. This actively avoids the identified suboptimal noise regions, ensuring that the final screened guiding noise falls within a noise range more conducive to the generation of high-quality images. Therefore, it can fundamentally reduce the instability of the cross-modal generation effect of medical images introduced by the randomness of the initial guiding noise, thereby effectively improving the consistency and reliability of the cross-modal generation results of medical images. On the other hand, the embodiments of this application adopt a rejection sampling strategy based on hyperspheres. When determining the candidate noise that is not rejected based on the similarity between the region center and the candidate noise, the candidate noise is then identified as guiding noise. This fully utilizes the concentrated properties of high-dimensional Gaussian noise in geometric distribution. Noise screening can be efficiently completed through simple similarity comparison without the need for complex real-time optimization calculations on the candidate noise. This allows the entire inference process to achieve effective noise quality control while maintaining low computational overhead, demonstrating good practicality.

[0092] S208: Based on the guiding noise, perform image diffusion generation processing on the source modality medical image to obtain the target modality medical image.

[0093] Optionally, in the embodiments of this application, after the guiding noise is obtained through screening, the guiding noise can be input into a diffusion model such as a diffusion bridge model, and the source modal medical image can be processed by the diffusion model to generate an image diffusion model, thereby obtaining the target modal medical image.

[0094] Based on this, in the embodiments of this application, the medical image processing scheme in steps S202 to S208 above can predict the suboptimal noise center and project it onto a predetermined hypersphere to determine the region center of the suboptimal noise region. Then, based on the region center, the noise in the candidate noise is screened to obtain the guiding noise. Based on the guiding noise, the source modality medical image is processed by image diffusion generation to obtain the target modality medical image. Therefore, on the one hand, this scheme introduces a suboptimal noise center and a hyperspherical projection mechanism to model and locate noise regions that may lead to a decrease in generation quality in a high-dimensional noise space. This region is actively avoided through a screening process, ensuring that the guiding noise used for image diffusion generation falls within a noise range more conducive to high-quality generation. This effectively reduces generation instability caused by the randomness of the initial guiding noise, improving the consistency and reliability of cross-modal generation results for medical images. On the other hand, the hyperspherical projection mechanism used in this scheme utilizes the property that high-dimensional random Gaussian noise is geometrically concentrated near a hypersphere. It projects the suboptimal noise center onto a unified predetermined hypersphere, ensuring geometric consistency in the similarity measurement between candidate noise and the region center. This ensures the effectiveness and robustness of the noise screening strategy in high-dimensional space, making the selection of guiding noise more precise. The resulting target modality medical image exhibits more stable and accurate presentation of key pathological structures and anatomical details. Therefore, the technical solution provided in this application can effectively improve the processing effect of cross-modal generation of medical images and enhance image generation quality.

[0095] In some alternative embodiments, when performing image diffusion generation processing using a diffusion model, the source modality medical image can be gradually transformed into the target modality medical image by starting with guiding noise and using the denoising process of the diffusion model.

[0096] Optionally, the diffusion model used can be a diffusion bridge model. That is, based on the guiding noise, the source modality medical image can be processed by diffusion generation through the diffusion bridge model to obtain the target modality medical image.

[0097] It should be understood that by performing image diffusion generation processing through the diffusion bridge model, the structural and semantic information carried by the source modality medical image can be fully utilized, and key anatomical structural information can be effectively preserved in the process of generating the target modality medical image, thereby further improving the medical credibility and clinical practical value of the generated target modality medical image.

[0098] It should be noted that the appropriate type of diffusion bridge model can be selected as needed, such as the aforementioned DDBM, DBIM, etc., and this application embodiment does not impose any restrictions on this.

[0099] In some optional embodiments, step S208 may include: performing multi-step image diffusion generation processing on the source modal medical image according to the diffusion step based on the guiding noise to obtain the target modal medical image.

[0100] In the embodiments of this application, the image diffusion generation process of the source modal medical image based on the guiding noise can be completed in a multi-step iterative manner. Through the complete multi-step image diffusion generation process, the current noisy state of the source modal medical image can be estimated to gradually approximate until the target modal medical image is obtained.

[0101] A diffusion step refers to each discrete time step in the inverse denoising process of a diffusion model. For example, in the diffusion generation process (inverse denoising process) of a diffusion bridge model, multiple diffusion denoising steps can be performed based on the guiding noise to reconstruct and generate the target modality medical image.

[0102] It is understood that, based on the guiding noise, the source modal medical image is processed in a multi-step image diffusion generation process according to the diffusion step, so that the processed target modal medical image can have richer details and higher fidelity, and has high stability.

[0103] It should be noted that the training process of the suboptimal noise predictor described in the embodiments of this application can be understood by referring to the model training method in the second aspect below.

[0104] Figure 6 This is a flowchart illustrating the steps of a model training method according to an embodiment of this application, with reference to... Figure 6 As shown, the model training method includes the following steps: S602: Obtain training samples.

[0105] In this embodiment, the training samples include source modal medical image samples and target modal medical image samples corresponding to the source modal medical image samples. In actual training, a large number of training samples can be used to train the suboptimal noise predictor multiple times to ensure it meets the usage requirements.

[0106] The source modality medical image samples can be understood by referring to source modality medical images, and historical source modality medical images can be used as source modality medical image samples. The target modality medical image samples can be real target modality medical image samples, or corresponding historical medical images can be used as target modality medical image samples. The target modality medical image samples can provide supervision signals for the training of the suboptimal noise predictor during the model training phase.

[0107] It is understandable that source modality medical image samples and target modality medical image samples are used as paired training sample data. In the same pair, medical images of the same patient collected in the same time period in two modalities (i.e., source modality and target modality, such as plain CT and CTPA) can be used to ensure that the two have a high degree of correspondence in anatomical structure. Such training samples can train a better suboptimal noise predictor.

[0108] Training samples can be obtained by reading from a pre-built training dataset or by dynamically loading them during training; this application embodiment does not impose any restrictions on this. The type, quantity, and source of training samples can be set as needed; there is no single limitation.

[0109] Optionally, the diffusion model can be pre-trained using a large number of training samples (including source modality medical image samples and target modality medical image samples corresponding to the source modality medical image samples). When the diffusion model uses a diffusion bridge model, the diffusion bridge model can be pre-trained using a large number of training samples. After training, it is combined with a suboptimal noise predictor for overall training, during which the parameters of the diffusion bridge model remain unchanged.

[0110] S604: Input the training samples into the suboptimal noise predictor to perform cascaded quantization feature extraction based on the training samples and generate predicted noise samples.

[0111] In this embodiment, the predicted noise sample is the noise output predicted by the suboptimal noise predictor for the input training samples during the training phase. During the model inference phase, the suboptimal noise predictor can predict suboptimal noise to obtain the suboptimal noise center. During the model training phase, the learning objective of the suboptimal noise predictor is to predict the noise that causes the diffusion bridge model to generate poor-quality images, i.e., suboptimal noise. Therefore, the predicted noise sample can be considered as the suboptimal noise predicted by the suboptimal noise predictor during the training phase. In the following text, the predicted noise sample can also be referred to as... .

[0112] The embodiments of this application do not limit the specific model structure of the suboptimal noise predictor. Optionally, the suboptimal noise predictor may include an encoder, a cascaded quantizer, and a decoder. Optionally, refer to Figure 7AThe flowchart shown includes steps S6042-S6046, specifically: S6042: Input the stitched source modality medical image samples and target modality medical image samples into the encoder to obtain feature map samples.

[0113] During the model training phase, the encoder's input consists of stitched source modality medical image samples and target modality medical image samples. Optionally, unlike the inference phase which uses a reference target modality image (e.g., the result of a single-step image diffusion generation process), the model training phase uses real target modality medical image samples as part of the input. This provides accurate supervision signals for training the suboptimal noise predictor, helping it learn the correspondence between the source and target modality images and their impact on suboptimal noise.

[0114] Optionally, the encoder adopts the downsampling path network structure of the U-Net network. By performing layer-by-layer downsampling and feature extraction on the stitched source modality medical image samples and target modality medical image samples, the corresponding feature map samples can be obtained.

[0115] S6044: The feature map samples are quantized by a cascaded quantizer to obtain quantized feature samples.

[0116] In this embodiment, the first-level quantizer is used to quantize the feature map samples, and the (i+1)th-level quantizer is used to quantize the quantization residual output by the i-th-level quantizer, where i ≥ 1 and is an integer.

[0117] During the model training phase, a cascaded quantizer is used to perform vector quantization on the feature map samples and the quantization residuals output by the previous quantizer, and the quantized feature samples are obtained from the output of the last quantizer.

[0118] Optionally, when the cascaded quantizers quantize the feature map samples, the codebooks of the cascaded quantizers can be trained and updated. For example, in a round of model training, before quantizing the feature map samples, the total number of cascaded quantizers and the number of codewords for each cascaded quantizer can be determined, and initial values ​​can be assigned to the codebooks of all cascaded quantizers. Then, the feature map samples are quantized, and the codebooks of each cascaded quantizer are trained level by level. The first-level quantizer to the last-level quantizer can be trained sequentially. Each level uses its input data to repeatedly adjust the codewords in the codebook so that the codewords can match the input features and minimize the quantization error. After the codebook of the current level quantizer is trained, the remaining error after quantization is passed to the next level quantizer as the input for codebook training. After the codebooks of all levels of quantizers are trained, the codewords of the codebooks of all levels of quantizers are fixed, and the training of the current round is completed. It should be understood that by continuously updating the codebook during training, the cascaded quantizer can gradually learn a discrete codebook representation that can effectively characterize the suboptimal noise distribution, providing a foundation for the suboptimal noise predictor to accurately predict noise samples.

[0119] S6046: Predicted noise samples are obtained based on quantized feature samples through the decoder and the skip connection between the decoder and the encoder.

[0120] During the model training phase, the input to the decoder is the quantized feature sample output by the cascaded quantizer. By sampling the input quantized feature sample layer by layer, the predicted noise sample is finally reconstructed so that it can be input into the diffusion bridge model for further processing.

[0121] Optionally, the decoder can adopt the network structure of the upsampling path of the U-Net network, or it can adopt other network structures as needed. During the reconstruction process, the decoder obtains high-resolution features of the encoder's output layers through skip connections with the encoder, and fuses them with the sampled features. This effectively restores spatial details while preserving high-level semantic information, thereby improving the accuracy and resolution of the predicted noise samples.

[0122] Based on this, in the optional implementations of steps S6042 to S6046 above, the suboptimal noise predictor in this embodiment, including the encoder, cascaded quantizer, and decoder, and its corresponding complete processing flow, can efficiently extract information related to the suboptimal noise distribution from the joint features of the stitched source modality medical image samples and target modality medical image samples, and parameterize the distribution in a discretized and hierarchical manner. Ultimately, it can accurately predict the predicted noise samples for subsequent training. Furthermore, in this embodiment, by stitching the source modality medical image samples and the target modality medical image samples together and inputting them into the encoder, the suboptimal noise predictor can simultaneously perceive the image features of both modalities. This allows the subsequently trained suboptimal noise predictor to learn the corresponding suboptimal noise distribution for specific input image pairs during the inference phase, improving the personalization and accuracy of the prediction. Furthermore, the cascaded quantizer in this embodiment adopts a multi-layer quantizer structure and discretizes the features by means of progressive residual quantization. It can learn richer and more hierarchical discrete feature representations, which can effectively alleviate the "codebook collapse" problem commonly found in traditional single-layer vector quantization techniques. This allows the suboptimal noise predictor to more accurately parameterize complex suboptimal noise distributions. As a result, the suboptimal noise predictor trained subsequently can provide reliable prior information for noise screening in the subsequent inference stage.

[0123] S606: Input the predicted noise samples and training samples into the diffusion bridge model to predict and generate the target modality medical prediction image based on the source modality medical image samples and predicted noise samples in the training samples.

[0124] In this embodiment, the target modality medical prediction image is a prediction image output by the diffusion bridge model after image diffusion generation processing, using predicted noise samples as guide noise and source modality medical image samples as source images. By generating the predicted target modality medical prediction image, its difference from the real target modality medical image sample can be compared to determine the difference between the two, so as to further train the suboptimal noise predictor.

[0125] For example, refer to Figure 7B A schematic diagram of a diffusion bridge model example from related technologies is shown. (Refer to...) Figure 7B As shown, the diffusion bridge model can include an encoder and a decoder, and the input image can be I. A The output image can be I B Input image I A Feature map z0 is extracted through the encoder. Feature map z0 is then progressively noise-added through a forward diffusion process (i.e., forward noise addition) using a diffusion bridge model (e.g., z0 is progressively noise-added to z1, z2, ..., z...). T-2 zT-1 Finally, the feature map z is generated through diffusion. T (Noisy image), feature map z T (Noisy image) is gradually denoised through the reverse diffusion process of the diffusion bridge model (i.e., reverse denoising) (e.g., z). T Stepwise denoising to z T-1 z T-2 (z1, ..., z2, z1), and finally the denoised result is reconstructed into the output image I by the decoder. B This allows for the image diffusion generation process of the diffusion bridge model to be completed. (The above...) Figure 7B This is merely an example and is not intended to limit any aspect of the embodiments described in this application.

[0126] In some optional embodiments, step S606 may include: based on the predicted noise samples, performing single-step image diffusion generation processing on the source modal medical image samples through a diffusion bridge model to predict and generate the target modal medical prediction image.

[0127] During the model training phase, the predicted noise samples obtained by the suboptimal noise predictor can be input into the diffusion bridge model. The diffusion bridge model performs single-step image diffusion generation processing to predict and generate the target modality medical prediction image.

[0128] The above implementation method of this application adopts a single-step inversion training strategy, which only performs single-step image diffusion generation processing of the diffusion bridge model to efficiently generate target modality medical prediction images. While meeting the requirements of subsequent model training, it can effectively improve computational efficiency and reduce computational costs.

[0129] In some alternative embodiments, refer to Figure 8 The flowchart shown illustrates how the target modality medical prediction image can be generated through the following steps S6062~S6064, specifically: S6062: The intermediate noise sample is obtained by single-step sampling of the predicted noise sample through the diffusion bridge model.

[0130] In predicting noise samples After inputting into the diffusion bridge model, only the first step of sampling (i.e., single-step sampling) can be performed to obtain intermediate state noise samples, which can be denoted as follows: .

[0131] S6064: Based on intermediate state noise samples and source modal medical image samples, input to the diffusion bridge model for single-step image diffusion generation processing to predict and generate target modal medical prediction images.

[0132] intermediate state noise samples Heyuan Modal Medical Image Samples The data is input into the data predictor of the diffusion bridge model, which is a component of the diffusion model and can directly predict a noise-free, clear image from any noisy state. Furthermore, by implementing a single-step image diffusion generation process through the data predictor of the diffusion bridge model, a target modality medical prediction image can be predicted and generated.

[0133] It should be understood that, in the optional embodiments of steps S6062 to S6064 above, the diffusion bridge model only performs single-step sampling on the predicted noise sample to obtain the intermediate state noise sample. Then, the intermediate state noise sample and the source modality medical image sample are input into the diffusion bridge model for single-step image diffusion generation processing, thereby efficiently predicting and generating the target modality medical prediction image. While meeting the requirements of subsequent model training, this process does not require a complete diffusion generation process with multiple iterations, thus greatly improving computational efficiency and reducing computational cost.

[0134] S608: The suboptimal noise predictor is trained based on the difference between the target modality medical prediction image and the target modality medical image sample.

[0135] In this embodiment, the training loss can be determined by comparing the difference between the target modality medical prediction image generated by the predicted noise sample and the real target modality medical image sample, so as to update and adjust the parameters of the suboptimal noise predictor based on the loss, thereby realizing the training of the suboptimal noise predictor.

[0136] The model training method provided in this embodiment adopts a reverse training strategy. That is, the training objective of the suboptimal noise predictor in this embodiment is not to maximize the generation quality, but to learn to predict noise that can degrade the generation quality (i.e., suboptimal noise). By using a large number of training samples and training the suboptimal noise predictor using the above model training method, the suboptimal noise predictor can gradually learn the suboptimal noise distribution pattern for different source modal images, providing reliable prior guidance for noise screening during the inference stage after training.

[0137] Based on this, in the model training method of steps S602 to S608 above, the acquired training samples are input into the suboptimal noise predictor for cascaded quantization feature extraction to generate predicted noise samples. These samples are then combined with a diffusion bridge model to predict and generate the target modality medical prediction image. Based on the difference between the target modality medical prediction image and the actual target modality medical image samples, the suboptimal noise predictor is trained. This allows the suboptimal noise predictor to learn the suboptimal noise distribution pattern for the source modality medical image, thus providing reliable prior guidance for noise screening during the inference stage after training. This helps to obtain better guiding noise, effectively reducing the generation instability problem introduced by the randomness of the initial guiding noise, and improving the consistency and reliability of cross-modal generation results of medical images. Therefore, the suboptimal noise predictor trained using the model training scheme in this embodiment can effectively assist in improving the processing effect of cross-modal generation of medical images during the inference stage.

[0138] The specific implementation of step S608 is not limited in the embodiments of this application. In some optional embodiments, refer to Figure 9 The flowchart shown illustrates how the target modality medical prediction image can be generated through the following steps S6082~S6084, specifically: S6082: Based on the target modality medical prediction image and the target modality medical image sample, obtain the first loss value through the first loss function.

[0139] In this embodiment, the first loss function is a loss function used to measure the structural similarity index (SSIM) between the target modality medical prediction image and the target modality medical image sample. SSIM is a similarity metric that comprehensively considers image brightness, contrast, and structural information. It can reflect the quality of the image at the visual perception level and is suitable for evaluating the degree of preservation of key anatomical structures and pathological features in medical images.

[0140] For example, the first loss function can be implemented in the following form:

[0141] in, This is the first loss value; This represents the source modality of medical image samples. Let G represent the predicted noise sample, and G represent the processing of the diffusion bridge model. This refers to the target modality medical prediction image; This represents a target modality medical image sample.

[0142] First loss value The loss value is calculated using the first loss function. Based on the first loss function in the above formula, the target modality medical prediction image is... and target modal medical image samples By performing calculations, the corresponding first loss value can be obtained. .

[0143] S6084: Train the suboptimal noise predictor based on the first loss value.

[0144] In this embodiment of the application, the parameters of the suboptimal noise predictor can be updated based on the first loss value, thereby enabling the training of the suboptimal noise predictor.

[0145] Since the reverse training strategy is used in this embodiment, the optimization direction of the suboptimal noise predictor is to minimize the first loss value, that is, the target modality medical prediction image. Medical image samples of the target modality The structural similarity between them (SSIM) is minimized, so that iterative training makes the noise predicted by the suboptimal noise predictor get closer and closer to the true distribution of suboptimal noise.

[0146] Based on this, through the optional implementation of steps S6082~S6084 above, on the one hand, since the reduction of structural similarity means the loss or distortion of key anatomical structures and pathological features, which corresponds precisely to the core manifestation of suboptimal noise causing the failure of cross-modal image generation, the embodiment of this application adopts the structural similarity loss function as the first loss function, so that the training objective of the suboptimal noise predictor is highly consistent with the actual diagnostic needs of medical images. The suboptimal noise predictor is trained by the first loss value of such a first loss function, so that the trained suboptimal noise predictor can more accurately predict suboptimal noise, which is conducive to assisting in the screening of better guiding noise, so as to effectively assist in improving the processing effect of cross-modal generation of medical images in the inference stage. On the other hand, the above optional implementation is combined with the single-step inversion training strategy (i.e., single-step image diffusion processing to predict the target modality medical prediction image), and the first loss value of the first loss function is calculated by using the target modality medical prediction image and the target modality medical image sample with low cost, so that the model training can maintain the effective training effect while greatly improving the efficiency of iterative calculation during training and effectively reducing the computational cost.

[0147] In some optional embodiments, the model training method further includes: obtaining a second loss value based on the quantized feature samples output by the cascaded quantizer and the feature map samples output by the encoder, using a second loss function, wherein the second loss function is used for codebook training of the cascaded quantizer. Optionally, in step S6084 above, the suboptimal noise predictor can be trained based on the first and second loss values.

[0148] The second loss function is used for training the codebook of the cascaded quantizer in the suboptimal noise predictor. The loss corresponding to the second loss function can be the vector quantization loss of the cascaded quantizer codebook. The second loss value is obtained by calculating the difference between the quantized feature samples output by the cascaded quantizer and the feature map samples output by the encoder using the second loss function. It can be used to guide the parameter updates of the codebook at each stage of the cascaded quantizer, enabling the codebook to gradually cover the key feature regions of the suboptimal noise distribution and improve the quality of the quantized representation.

[0149] You can choose an appropriate type of second loss function as needed. For example, the second loss function can include, but is not limited to, the mean squared error loss function, the average error loss function, the cosine similarity loss function, and so on.

[0150] In one example, the second loss function can be implemented as follows:

[0151] in, This represents the second loss value. The total number of quantizer stages in the cascaded quantizer is given by sg[.], which represents the stopping gradient operator. This represents the input of the i-th level quantizer. This represents the output of the i-th quantizer, where β is a preset coefficient.

[0152] The above formula satisfies:

[0153]

[0154]

[0155] in, This indicates the quantization process performed by the i-th quantizer. This represents the input to the first-level quantizer (i.e., the feature map sample). ).

[0156] In this embodiment of the application, in order to improve the training effect of the suboptimal noise predictor, the first loss value of the first loss function and the second loss value of the second loss function can be combined to calculate the total loss, and the parameters of the suboptimal noise predictor can be updated and adjusted based on the total loss, thereby realizing the training of the suboptimal noise predictor.

[0157] The total loss can be calculated in any way. In some optional embodiments, the first loss value and the second loss value can be weighted and summed according to predetermined weight information to obtain the total loss.

[0158] For example, the total loss can be calculated using the following formula:

[0159] in, Indicates the total loss. This represents the first loss value. Indicates the second loss value This represents the first weighting coefficient corresponding to the first loss value. This represents the second weighting coefficient corresponding to the second loss value. First weighting coefficient. Second weight system It can be set according to actual needs.

[0160] In this embodiment, a first loss value and a second loss value are jointly used to train and optimize the suboptimal noise predictor, allowing the predictor to consider multiple optimization objectives during the overall training process. Firstly, it ensures that the predicted noise samples lead to the diffusion bridge model generating lower-quality images, thus accurately capturing the suboptimal noise distribution. Secondly, it effectively trains the codebook of the cascaded quantizer, enabling accurate discretization and parameterization of the suboptimal noise distribution. These optimization objectives work together to ensure the training quality and model stability of the suboptimal noise predictor.

[0161] It should be understood that, in the above optional embodiments, introducing a second loss function to specifically train the codebook of the cascaded quantizer can effectively avoid the codebook collapse problem common in vector quantization training, ensuring the diversity and effectiveness of quantization expression, and thus providing stable and rich discrete feature support for the suboptimal noise predictor. Furthermore, training and optimizing the suboptimal noise predictor using the first and second loss values ​​enables the trained suboptimal noise predictor to possess good noise prediction capabilities and good internal quantization expression capabilities, thereby improving the overall training effect and generalization ability of the suboptimal noise predictor. This further enables the trained suboptimal noise predictor to more accurately and stably predict suboptimal noise for the input image in the subsequent inference stage, thus providing more reliable prior guidance for noise selection during the subsequent inference stage after training. This helps to obtain better guiding noise, effectively reducing the generation instability problem introduced by the randomness of the initial guiding noise, and improving the consistency and reliability of cross-modal generation results of medical images. Therefore, the suboptimal noise predictor trained in the above optional method can more effectively assist in improving the processing effect of cross-modal generation of medical images during the inference stage.

[0162] It is understood that the foregoing description of the medical image processing method and model training method is only an exemplary description of the embodiments of this application and is not intended to limit the embodiments of this application.

[0163] Below, refer to Figure 10 and Figure 11 The following is a further illustrative description of the medical image processing scheme of the embodiments of this application.

[0164] Figure 10 A flowchart illustrating an example of a model training scenario according to an embodiment of this application is shown. This optional scenario can be understood by substituting it for the cross-modal transformation of clinical CT medical images. Figure 10As shown, taking pulmonary embolism CT images as an example, clinical CT images of pulmonary embolism can be collected, including plain CT images of the source modality and enhanced CT images of the target modality (such as CTPA). These CT images can be used as training data. The training data can be used in the model training phase. First, the training data can be cleaned using standard methods and paired image registration can be performed. Multiple pairs of training samples can be obtained from the training data (e.g., pairs of source modality medical image samples (such as plain CT images) and their corresponding target modality medical image samples (such as enhanced CT images)). Then, the training samples can be preprocessed using standard methods (e.g., including but not limited to converting 3D images to 2D). The preprocessed training samples can be used to train the constructed suboptimal noise predictor. The training samples can also be used to pre-train the diffusion bridge model. The predicted noise samples obtained during the training of the suboptimal noise predictor are input into the pre-trained diffusion bridge model for single-step image diffusion generation processing to obtain the target modality medical prediction image. The suboptimal noise predictor can be iteratively trained based on the difference between the target modality medical prediction image output by the diffusion bridge model and the target modality medical image samples in the training samples, until the suboptimal noise predictor is trained. It should be understood that... Figure 10 The examples shown are not intended to limit any aspects of the embodiments in this application.

[0165] Figure 11 A schematic diagram illustrating the implementation process of an example of a medical image processing scenario according to an embodiment of this application is shown. Figure 11 As shown, a pair of source modal medical images and a reference target modal medical image of a brain region are displayed (obtained by single-step image diffusion generation of the source modal medical images using a diffusion bridge model). The concatenated images are then input into a suboptimal noise predictor. The concatenated result is processed by the encoder of the suboptimal noise predictor to obtain a feature map. The feature map is then input into a cascaded quantizer of the suboptimal noise predictor to obtain quantized features. The cascaded quantizer consists of multiple quantizers: the first-level quantizer quantizes the feature map, and the (i+1)th-level quantizer quantizes the quantization residual output by the first-level quantizer (i≥1 and is an integer). The quantized features are input into a decoder, and through the decoder and the jump connection between the decoder and encoder, suboptimal noise is output based on the quantized features. And based on suboptimal noise Determine the suboptimal noise center. Then project the suboptimal noise center onto a predetermined hypersphere to obtain the region center of the suboptimal noise region. Further, implement a hypersphere rejection sampling strategy (refer to the previous explanation) to filter noise in the predetermined noise image based on the region center to obtain guiding noise. For example, it can be based on a standard Gaussian distribution. Candidate noise is obtained by randomly sampling from a predetermined noisy image. The similarity between the region center and the candidate noise is then used to determine whether to reject the sampled candidate noise. If not, the candidate noise is identified as guiding noise. The guiding noise is then input into a diffusion bridge model for multi-step image diffusion generation, ultimately yielding the target modality medical image. This effectively achieves cross-modal conversion generation from the source modality medical image to the target modality medical image. In this example, the generated target modality medical image can better assist in brain medical diagnosis. It should be understood that... Figure 11 The examples shown are not intended to limit any aspects of the embodiments in this application.

[0166] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store a computer program; and the processor is used to execute the aforementioned medical image processing method or model training method by running the computer program stored in the memory.

[0167] Figure 12 A structural block diagram of an optional electronic device according to an embodiment of this application is shown. This application does not limit the specific implementation of the electronic device 1000; however, as an example, reference is made to... Figure 12 As shown, the electronic device 1000 provided in this application embodiment includes: a processor 1002, a communications interface 1004, a memory 1006, and a communication bus 1008. Wherein: The processor 1002, communication interface 1004, and memory 1006 communicate with each other via communication bus 1008.

[0168] Communication interface 1004 is used to communicate with other electronic devices or servers.

[0169] The processor 1002 is used to execute the computer program 1010, specifically to execute the relevant steps in any of the aforementioned medical image processing method or model training method embodiments.

[0170] Specifically, computer program 1010 may include program code that includes computer operation instructions.

[0171] The processor 1002 may be a CPU, a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0172] Memory 1006 is used to store computer program 1010. Memory 1006 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0173] Specifically, computer program 1010 can be used to cause processor 1002 to execute the medical image processing method or model training method in any of the foregoing embodiments.

[0174] The specific implementation of each step in computer program 1010 can be found in the corresponding steps and units described in any of the foregoing embodiments of medical image processing methods or model training methods, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0175] The electronic device 1000 in this application embodiment has been described in detail in the foregoing embodiments of the medical image processing method and model training method. Therefore, its related content and beneficial effects can be understood by referring to the above method embodiments, and will not be repeated here.

[0176] According to a fourth aspect of the embodiments of this application, the embodiments of this application also provide a computer storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the medical image processing method or model training method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk, etc.

[0177] According to a fifth aspect of the embodiments of this application, the embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the medical image processing method or model training method as described in any of the embodiments of the plurality of methods described above.

[0178] The electronic device 1000 / computer storage medium / computer program product embodiment in this application has been described in detail in the foregoing medical image processing method embodiment. Therefore, its related content and beneficial effects can be understood by referring to the above method embodiment, and will not be repeated here.

[0179] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0180] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0181] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0182] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for specific applications, but such implementations should not be considered beyond the scope of the embodiments of this application.

[0183] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". It should be noted that the concepts of "first", "second", etc., mentioned in the embodiments of this application are only used to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies. It should be noted that the modifications of "a" and "a plurality" mentioned in the embodiments of this application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0184] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A medical image processing method, comprising: Based on the source modal medical image and the corresponding reference target modal medical image, a suboptimal noise center is predicted; The suboptimal noise center is projected onto a predetermined hypersphere to obtain the region center of the suboptimal noise region. Based on the center of the region, noise in the predetermined noise image is filtered to obtain guiding noise; Based on the guiding noise, the source modal medical image is subjected to image diffusion generation processing to obtain the target modal medical image.

2. The method according to claim 1, wherein, The step of filtering noise in a predetermined noise image based on the center of the region to obtain guiding noise includes: The predetermined noise image is sampled to obtain candidate noise; Based on the similarity between the region center and the candidate noise, determine whether to reject the sampled candidate noise; If not, the candidate noise is identified as the guiding noise.

3. The method according to claim 2, wherein, The step of determining whether to reject the sampled candidate noise based on the similarity between the region center and the candidate noise includes: Determine whether the similarity between the region center and the candidate noise meets a preset rejection rate, and based on the determined result, determine whether to reject the sampled candidate noise.

4. The method according to any one of claims 1-3, wherein, The suboptimal noise center is predicted based on the source modal medical image and the corresponding reference target modal medical image, including: The source modal medical image is input into the trained diffusion model for single-step image diffusion generation to obtain the reference target modal medical image. The source modal medical image and the reference target modal medical image are input into the trained suboptimal noise predictor for processing to predict the suboptimal noise center.

5. The method according to claim 4, wherein, The suboptimal noise predictor includes an encoder, a cascaded quantizer, and a decoder; The step of inputting the source modality medical image and the reference target modality medical image into a trained suboptimal noise predictor for processing, and predicting the suboptimal noise center, includes: The stitched source modal medical image and the reference target modal medical image are input into the encoder to obtain a feature map; The feature map is quantized by the cascaded quantizer to obtain quantized features. The cascaded quantizer includes multiple quantizers. The first-level quantizer is used to quantize the feature map, and the (i+1)th-level quantizer is used to quantize the quantization residual output by the i-th-level quantizer, where i ≥ 1 and is an integer. The suboptimal noise is predicted based on the quantization features through the decoder and the skip connection between the decoder and the encoder, and the suboptimal noise center is determined based on the suboptimal noise.

6. The method according to any one of claims 1-3, wherein, The step of projecting the suboptimal noise center onto a predetermined hypersphere to obtain the region center of the suboptimal noise region includes: The suboptimal noise center is normalized to a predetermined hypersphere radius and then projected onto the predetermined hypersphere corresponding to the hypersphere radius to obtain the region center of the suboptimal noise region.

7. The method according to any one of claims 1-3, wherein, The step of performing image diffusion generation processing on the source modality medical image based on the guiding noise to obtain the target modality medical image includes: Based on the guiding noise, the source modal medical image is subjected to multi-step image diffusion generation processing according to the diffusion step to obtain the target modal medical image.

8. The method according to any one of claims 1-3, wherein, The step of performing image diffusion generation processing on the source modality medical image based on the guiding noise to obtain the target modality medical image includes: Based on the guiding noise, the source modal medical image is processed by image diffusion generation using a diffusion bridge model to obtain the target modal medical image.

9. A model training method, comprising: Acquire training samples, wherein the training samples include source modal medical image samples and target modal medical image samples corresponding to the source modal medical image samples; The training samples are input into the suboptimal noise predictor to perform cascaded quantization feature extraction based on the training samples, thereby generating predicted noise samples. The predicted noise samples and the training samples are input into the diffusion bridge model to predict and generate a target modality medical prediction image based on the source modality medical image samples and the predicted noise samples in the training samples. The suboptimal noise predictor is trained based on the difference between the target modality medical prediction image and the target modality medical image sample.

10. The method according to claim 9, wherein, The suboptimal noise predictor includes an encoder, a cascaded quantizer, and a decoder; The step of inputting the training samples into a suboptimal noise predictor to perform cascaded quantization feature extraction based on the training samples and generate predicted noise samples includes: The stitched source modality medical image sample and the target modality medical image sample are input into the encoder to obtain feature map samples; The feature map sample is quantized by the cascaded quantizer to obtain quantized feature samples. The first-level quantizer is used to quantize the feature map sample, and the (i+1)th-level quantizer is used to quantize the quantization residual output by the i-th-level quantizer, where i ≥ 1 and is an integer. Predicted noise samples are obtained based on the quantized feature samples through the decoder and the skip connection between the decoder and the encoder.

11. The method according to claim 10, wherein, The step of inputting the predicted noise sample and the training sample into the diffusion bridge model to predict and generate a target modality medical prediction image based on the source modality medical image sample and the predicted noise sample in the training sample includes: Based on the predicted noise samples, the source modality medical image samples are processed by single-step image diffusion generation through the diffusion bridge model to predict and generate the target modality medical prediction image.

12. The method according to claim 11, wherein, The step of performing single-step image diffusion generation processing on the source modality medical image samples based on the predicted noise samples and through the diffusion bridge model to predict and generate the target modality medical prediction image includes: The intermediate state noise sample is obtained by performing single-step sampling on the predicted noise sample using the diffusion bridge model. Based on the intermediate noise samples and the source modal medical image samples, the samples are input into the diffusion bridge model for single-step image diffusion generation processing to predict and generate the target modal medical prediction image.

13. The method according to claim 11, wherein, The step of training the suboptimal noise predictor based on the difference between the target modality medical prediction image and the target modality medical image samples includes: Based on the target modality medical prediction image and the target modality medical image sample, a first loss value is obtained through a first loss function, wherein the first loss function is a loss function used to measure the structural similarity between the target modality medical prediction image and the target modality medical image sample; The suboptimal noise predictor is trained based on the first loss value.

14. The method according to claim 13, wherein, The method further includes: obtaining a second loss value based on the quantized feature samples output by the cascaded quantizer and the feature map samples output by the encoder through a second loss function, wherein the second loss function is used for codebook training of the cascaded quantizer; The step of training the suboptimal noise predictor based on the first loss value includes: The suboptimal noise predictor is trained based on the first loss value and the second loss value.

15. An electronic device comprising: The processor, the communication interface, the memory, and the communication bus are provided, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The memory is used to store computer programs; The processor is configured to perform the method of any one of claims 1-14 by running the computer program stored in the memory.

16. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-14.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method as described in any one of claims 1-14.

Images