Image conversion model training method and device, image conversion method and device and electronic equipment
By training an image conversion model using a generative adversarial network model based on contrastive learning, the problem of converting frozen pathology slides to paraffin-embedded pathology slides was solved, improving image clarity and diagnostic accuracy, and supporting more precise diagnostic and treatment decisions.
Patent Information
- Application Number
- CN202410622598.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-20
- Publication Date
- 2026-02-13
AI Technical Summary
Existing technologies are insufficient to effectively convert frozen pathology slide images into paraffin-embedded pathology slide images, leading to inconsistent diagnostic results and affecting personalized and precise diagnosis and treatment.
A generative adversarial network model based on contrastive learning is adopted. By using a generator and a discriminator, combined with a dual attention module, and through consistency loss, noise contrast estimation loss and generative adversarial loss, an image conversion model is trained to realize the conversion of frozen pathological sections to paraffin-style sections.
It improves image clarity and quality, highlights image details, enhances the ability to identify lesion features, provides more accurate diagnostic evidence, reduces interference from subjective factors, and improves the accuracy and reliability of diagnosis.
Smart Images
Figure CN121527558A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image translation, and in particular to an image conversion model training method, an image conversion method, an apparatus, and an electronic device. Background Technology
[0002] Histopathological images are considered the gold standard for cancer diagnosis. Due to limitations in slide preparation and speed, frozen section pathology is the only source of pathological information during surgery; however, significant inconsistencies exist between its diagnostic results and those of conventional paraffin-embedded sections. Currently, the diagnostic accuracy of frozen section pathology often falls short of clinical needs, potentially leading to a reduction in the level of individualized precision medicine.
[0003] To improve clinical diagnosis and treatment, researchers are actively exploring the use of artificial intelligence to assist in the diagnosis of frozen pathology slides, thereby enhancing their accuracy and reliability. By training deep learning models, the ability to identify tissue structures and cellular features can be improved, enabling the analysis and diagnosis of a large number of slide images in a short time. However, these algorithms often only provide diagnostic solutions based on intraoperative frozen pathology images, lacking consideration of the correlation between frozen and paraffin-embedded pathology images, and failing to comprehensively consider pathological information during the decision-making process for accurate diagnosis.
[0004] Besides leveraging artificial intelligence to improve the diagnostic results of frozen pathology slides, image translation has become a current research hotspot. Image translation aims to convert images from a source domain to a target domain. In the field of medical imaging, this technology is widely used for image quality enhancement and image style transfer. Combining image translation technology with pathological image diagnosis can provide doctors with more comprehensive and intuitive image information, helping them to better understand pathological features. Therefore, further research is necessary to explore and establish a correlation model between frozen pathology images and conventional paraffin images to provide patients with more accurate and personalized diagnostic and treatment plans. Summary of the Invention
[0005] The technical problem to be solved by this disclosure is to overcome the deficiencies in the prior art and provide a training method, image conversion method, device and electronic equipment for an image conversion model that can convert frozen pathological sections to paraffin-style sections.
[0006] This disclosure solves the above-mentioned technical problems through the following technical solution:
[0007] An image conversion model training method, comprising,
[0008] Acquire sample images, which include a first sample image and a corresponding second sample image;
[0009] The first sample image is preprocessed;
[0010] The preprocessed first sample image is used as the input sample to the image conversion model, and the second sample image is used as the output sample of the image conversion model to train the image conversion model;
[0011] The image conversion model is a generative adversarial network based on contrastive learning, which includes a generator and a discriminator. The generator includes a dual attention module.
[0012] Preferably, the preprocessing of the first sample image includes,
[0013] Extract the foreground tissue pathology region from the first sample image;
[0014] The foreground tissue pathology region is divided into mutually independent image blocks.
[0015] Preferably, the step of using the preprocessed first sample image as an input sample to the image conversion model, and using the second sample image as the output sample of the image conversion model to train the image conversion model, specifically includes:
[0016] Obtain the consistency loss between the input samples and the output samples of the generator;
[0017] Obtain noise contrast estimation loss;
[0018] Obtain the generative adversarial loss of the discriminator;
[0019] The training is constrained by a total loss obtained by weighting and summing the consistency loss, the noise contrast estimation loss, and the generative adversarial loss.
[0020] Preferably, one of the dual attention modules is a spatial attention module and the other is a channel attention module, and the discriminator adopts the patchGAN model.
[0021] Another aspect of this disclosure provides an image conversion method, including,
[0022] Acquire frozen pathological section images of the target tissue;
[0023] Preprocess frozen pathological section images of the target tissue;
[0024] The preprocessed frozen pathological section image of the target tissue is input into an image conversion model to perform image conversion and obtain a paraffin-style pathological section image of the target tissue.
[0025] The image conversion model is trained according to the training method described in any one of the above claims.
[0026] Preferably, the method further includes the following steps:
[0027] After obtaining the paraffin-effect pathological section image of the target tissue, the paraffin-effect pathological section image is input into an image classification model for classification.
[0028] Another aspect of this disclosure provides an image conversion model training apparatus, comprising,
[0029] The acquisition module is used to acquire sample images, which include a first sample image and a corresponding second sample image;
[0030] The preprocessing module is used to preprocess the first sample image;
[0031] The training module is used to input the preprocessed first sample image as an input sample into the image conversion model, and to train the image conversion model by using the second sample image as the output sample of the image conversion model.
[0032] The image conversion model is a generative adversarial network based on contrastive learning, which includes a generator and a discriminator. The generator includes a dual attention module.
[0033] In another aspect of this disclosure, an image conversion apparatus is provided, comprising,
[0034] The target image acquisition module is used to acquire frozen pathological section images of the target tissue;
[0035] The target image preprocessing module is used to preprocess frozen pathological section images of the target tissue;
[0036] The conversion module is used to input the preprocessed frozen pathological section image of the target tissue into the image conversion model to convert the image and obtain the paraffin-style pathological section image of the target tissue.
[0037] The image conversion model is trained using the aforementioned training device.
[0038] In another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that the processor executes the computer program as the image conversion model training method described in any one of the preceding claims or implements the image conversion method described in any one of the preceding claims.
[0039] In another aspect, this disclosure provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the image conversion model training method described in any one of the above descriptions or implements the image conversion method described in any one of the above descriptions.
[0040] In another aspect of this disclosure, a computer program product is provided, comprising a computer program, characterized in that, when the computer program is executed by a processor, it implements the image conversion model training method or the image conversion method described in any one of the above descriptions.
[0041] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.
[0042] The positive advancements of this disclosure are as follows: The training method and apparatus for the image conversion model of this disclosure, through the synergistic effect of the generator, discriminator, adversarial loss, and contrastive loss, utilizes the maximization of mutual information of local images to optimize the target loss function. Simultaneously, a dual attention module is introduced into the generator to focus on details within the image, enabling the model to learn the potential correlations between different image domains and achieve effective image conversion. This not only preserves the key information of frozen pathology images but also endows the images with a clearer paraffin style, making image details more prominent and providing more easily diagnosed information for subsequent classification, thereby further improving the accuracy of image conversion. Furthermore, the image conversion method and apparatus based on this image conversion model effectively solve the problem of blurring in intraoperative frozen pathology images, improving image clarity and quality. It enables better attention to image details, more accurate identification of lesion features, and thus more accurate classification of the converted image, providing doctors with a more reliable basis for surgical approach selection. By providing clear and accurate image conversion results, it reduces the interference of subjective factors, improving the accuracy and reliability of decision-making. Attached Figure Description
[0043] Figure 1 This is a flowchart illustrating an image conversion model training method provided in Embodiment 1 of this disclosure;
[0044] Figure 2 This is a schematic diagram of the framework of the image conversion model provided in Embodiment 1 of this disclosure;
[0045] Figure 3 This is a schematic diagram of the generator framework in the image conversion model provided in Embodiment 1 of this disclosure;
[0046] Figure 4 This is a schematic flowchart of an image conversion method provided in Embodiment 2 of this disclosure;
[0047] Figure 5This is a schematic diagram of the frame of the image conversion model training device provided in Embodiment 3 of this disclosure;
[0048] Figure 6 This is a schematic diagram of the frame of the image conversion apparatus provided in Embodiment 4 of this disclosure;
[0049] Figure 7 This is a schematic diagram of the framework of the electronic device provided in Embodiment 5 of this disclosure. Detailed Implementation
[0050] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.
[0051] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.
[0052] In this embodiment of the disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good morals.
[0053] Example 1
[0054] Figure 1 This is a flowchart illustrating a training method for an image model conversion model, provided as an exemplary embodiment of the present disclosure. It includes the following steps:
[0055] S110, acquire sample images, the sample images include a first sample image and a corresponding second sample image, the first sample image is a frozen pathological slide image, and the second sample image is a paraffin-embedded pathological slide image;
[0056] S120, preprocess the first sample image;
[0057] S130, the preprocessed first sample image is used as an input sample to the image conversion model, and the second sample image is used as an output sample to train the image conversion model.
[0058] In step S110, the sample images obtained are frozen pathological slide images of existing medical records and corresponding postoperative paraffin-style pathological slide images. Based on such sample images, an image conversion model is trained so that the image conversion model can convert frozen pathological slide images into corresponding paraffin-style pathological slide images.
[0059] The image conversion model is a generative adversarial network based on contrastive learning, comprising a generator and a discriminator, wherein the generator includes a dual attention module. Figure 2 The diagram shown illustrates the framework of this image conversion model. The dual attention modules consist of a spatial attention module and a channel attention module, as shown below. Figure 3 As shown, the spatial attention module can accurately focus on the features at each location by selectively aggregating and highlighting these features by calculating a weighted sum of features at all locations. The channel attention module integrates the feature correlations between all channel mappings, thereby selectively emphasizing those channel mappings that are interdependent. To further improve the accuracy and richness of feature representation, the outputs of the spatial attention module and the channel attention module are fused together.
[0060] The discriminator employs the patchGAN model. First, the image is divided into equal patches, and each patch is judged to be either real or fake. Finally, the results are averaged. The discriminator uses style loss constraints to ensure that the generated images are visually realistic enough.
[0061] In step S120, the first sample image is preprocessed, including but not limited to operations such as extracting tissue regions and removing image noise, to ensure the quality of the processed image data. Extracting tissue regions is used to highlight the tissue structures of interest in the image, separating them from the background and avoiding the processing of irrelevant areas, thereby improving the efficiency and accuracy of subsequent analysis. Removing image noise can effectively reduce the noise level in the image by applying filtering techniques and image enhancement methods.
[0062] Specifically, preprocessing the first sample image includes:
[0063] S121, Extract the foreground tissue pathology region from the first sample image;
[0064] S122, the foreground tissue pathology region is divided into mutually independent image blocks.
[0065] First, the OTU algorithm is used to extract the foreground tissue region from the first sample image, eliminating a large amount of redundant background. In the first sample image, blank background areas typically exhibit low saturation and high RGB values. The final image mask is obtained by calculating the RGB and saturation channels of the image separately. Then, the union of the high-saturation and low-RGB regions is taken to obtain the foreground region. After morphological operations to fill in blanks and holes, the final generated tissue region foreground is obtained. For WSI (Whole Slide Image), extracting the tissue region of the pathological slide reduces many unnecessary calculations. Subsequently, to input the first sample image into the image conversion model, a "sliding window method" is used to divide the foreground tissue pathological region into non-overlapping image blocks of 512×512×3 pixels. Unlike ordinary images, slide images have billions of pixels and cannot be directly input into the network for training; they must be divided into small image blocks for feature extraction.
[0066] In step S130, the step of using the preprocessed first sample image as an input sample to the image conversion model, and using the second sample image as the output sample of the image conversion model to train the image conversion model, specifically includes:
[0067] S131, Obtain the consistency loss between the input samples and the output samples of the generator;
[0068] By applying a proportional regularization factor to the content between the generated image and the original image at the pixel level, the generator effectively prevents the introduction of a large amount of information irrelevant to the original image during the generation process, thus avoiding the introduction of potentially misleading information. This loss is calculated using the L1 regularization function and can be defined as follows:
[0069] L SC =E xX [||G(x)-x||1];
[0070] Where G(x) represents the generated image, and x represents the image in the source domain, i.e., the first sample image.
[0071] S132, obtain the noise contrast estimation loss; this helps ensure that the generated content remains coherent across different image domains to align with contextual information. Select a query from image x; the same location in other images is a positive sample, and other locations are negative samples.
[0072] Assuming there are N-1 negative samples, the noise-contrast estimation loss function is L. NCE It can be defined in the following form:
[0073]
[0074] Where sim(q,p) + ) represents the query sample q and the positive sample p. + The similarity between the samples is calculated by measuring the cosine distance. N refers to the number of samples, and τ is the temperature coefficient, preferably 0.07.
[0075] S133, Obtain the generative adversarial loss of the discriminator;
[0076] Generative adversarial loss aims to encourage the generator to produce samples that can fool the discriminator by minimizing the difference in probability distributions between real and synthetic samples. Specifically, generative adversarial loss can be defined as follows:
[0077] L GAN (G,D,X,Y)=E y～Y [logD(y)]+E x～X [1-logD(G(x))];
[0078] Where G is the generator output, D is the second sample, E is the expectation, x is the source domain image, and y is the target domain image.
[0079] S134, the consistency loss, the noise contrast estimation loss, and the generative adversarial loss are weighted and summed to obtain a total loss to constrain the training.
[0080] The total loss L is obtained by weighting the consistency loss, noise contrast estimation loss, and generative adversarial loss mentioned above. T It can be defined in the following form:
[0081] L T =λ1L sc +λ2L NCE +λ3L GAN
[0082] Where λ1, λ2, and λ3 are all hyperparameters, preferably set to 0.3, 1.0, and 1.0 respectively in this embodiment. These three loss functions work together to play a crucial role in ensuring image quality and realism. These three losses guide the model to learn the mapping relationship between the source domain (frozen pathology slide images) and the target domain (paraffin-style pathology slide images) through constraints from different perspectives, ensuring the visual and semantic accuracy of the generated images and achieving image transformation from the source domain to the target domain.
[0083] The image conversion model training method in this embodiment uses acquired frozen pathology slide images and corresponding postoperative paraffin-effected pathology slide image data to train the frozen-paraffin image conversion model. Through the synergistic effect of the generator, discriminator, adversarial loss, and contrastive loss, the target loss function is optimized by maximizing the mutual information of local images. Simultaneously, a dual attention module is introduced into the generator to focus on details within the image, enabling the model to learn the potential correlations between different image domains and achieve effective image conversion. This not only preserves the key information of the frozen pathology images but also endows the images with a clearer paraffin style, making image details more prominent and providing more easily diagnosed information for subsequent classification, thereby further improving the accuracy of image conversion.
[0084] Example 2
[0085] like Figure 4 The diagram shown is a flowchart of the image conversion method provided in Embodiment 2, which specifically includes:
[0086] S210, acquire frozen pathological section images of the target tissue;
[0087] S220, Preprocess the frozen pathological section images of the target tissue;
[0088] S230, the preprocessed frozen pathological section image of the target tissue is input into the image conversion model to perform image conversion to obtain the paraffin-style pathological section image of the target tissue;
[0089] The image conversion model is trained using the image conversion model training method described in Embodiment 1 above.
[0090] The preprocessing step in step S220 is the same as the preprocessing method in S120 in Example 1, so as to maintain the consistency of the image brick-and-mortar model input.
[0091] After obtaining the paraffin-embedded pathological section image of the target tissue, the method further includes the following steps:
[0092] S240, The paraffin-style pathological slide image is input into an image classification model for classification.
[0093] The image classification model is used to classify input paraffin-embedded pathological slide images, with the classification result being either high-risk or low-risk. This classification result is then used to assist in guiding diagnosis. The image classification model can include convolutional neural networks, multi-instance learning networks, transformer networks, or other algorithms. Conventional algorithms can be used to train and optimize the image classification model. The trained surgical method classification model is then evaluated and optimized, and cross-validation and validation set testing are performed to ensure the model's generalization ability and stability. During optimization, the model structure is adjusted, including increasing or decreasing layers and adjusting the number of neurons, to improve the model's complexity and representational ability. Furthermore, hyperparameter tuning, such as learning rate and batch size, is performed to find the optimal training configuration. Parameter adjustments are made based on the model's performance on the validation set. If the loss function stops decreasing prematurely over a period of time, it indicates that the model is overfitting. Reducing the number of layers and neurons is considered to lower the model complexity. If the loss function continues to decrease but the convergence speed is particularly slow, increasing the model complexity and the number of neurons is considered to deepen the network model.
[0094] The image conversion method in this embodiment first utilizes a frozen-paraffin pathological section image conversion model to effectively solve the problem of blurred intraoperative frozen images, improving image clarity and quality. By improving image clarity and quality, and paying better attention to image details, lesion features can be identified more accurately, and the converted images can be classified more precisely, thus providing doctors with a more reliable basis for selecting surgical procedures. By providing clear and accurate image conversion results, the interference of subjective factors is reduced, improving the accuracy and reliability of decision-making.
[0095] Example 3
[0096] Corresponding to Embodiment 1 of the aforementioned image conversion model training method, this disclosure also provides embodiments of an image conversion model training apparatus, such as... Figure 5 As shown, including,
[0097] The acquisition module 101 is used to acquire sample images, the sample images including a first sample image and a corresponding second sample image, the first sample image being a frozen pathological slide image and the second sample image being a paraffin-embedded pathological slide image;
[0098] Preprocessing module 102 is used to preprocess the first sample image;
[0099] Training module 103 is used to input the preprocessed first sample image as an input sample into the image conversion model, and to train the image conversion model by using the second sample image as an output sample of the image conversion model.
[0100] The image conversion model is a generative adversarial network based on contrastive learning, comprising a generator and a discriminator. The generator includes a dual attention module. The discriminator uses a patchGAN model. First, the image is divided into equal parts, and each part is judged as either real or fake. Finally, the results are averaged. The discriminator uses style loss constraints to ensure that the generated image is visually realistic enough.
[0101] The preprocessing module 102 performs preprocessing on the first sample image, including but not limited to operations such as extracting tissue regions and removing image noise, to ensure the quality of the processed image data. Extracting tissue regions highlights the tissue structures of interest in the image, separating them from the background and avoiding the processing of irrelevant areas, thereby improving the efficiency and accuracy of subsequent analysis. Removing image noise can effectively reduce the noise level in the image by applying filtering techniques and image enhancement methods.
[0102] Specifically, the preprocessing module 102 includes,
[0103] Extraction module 1021 is used to extract the foreground tissue pathological region in the first sample image;
[0104] The segmentation module 1022 is used to divide the foreground tissue pathological region into mutually independent image blocks.
[0105] First, the OTU algorithm is used to extract the foreground tissue region from the first sample image, eliminating a large amount of redundant background. In the first sample image, blank background areas typically exhibit low saturation and high RGB values. The final image mask is obtained by calculating the RGB and saturation channels of the image separately. Then, the union of the high-saturation and low-RGB regions is taken to obtain the foreground region of the image. After morphological operations to fill in blanks and holes, the final generated tissue region foreground is obtained. For WSI (Whole Slide Image), extracting the tissue region of the pathological slide can reduce many unnecessary calculations. Subsequently, to input the first sample image into the image conversion model, the "sliding window method" can be used to divide the foreground tissue pathological region into non-overlapping image blocks of 512×512×3.
[0106] The training module 103 specifically includes,
[0107] The first acquisition module 1031 is used to acquire the consistency loss between the input samples and the output samples of the generator;
[0108] The second acquisition module 1032 is used to acquire the noise contrast estimation loss;
[0109] The third acquisition module 1033 is used to acquire the generative adversarial loss of the discriminator;
[0110] The weighting module 1034 is used to weight and sum the consistency loss, the noise contrast estimation loss, and the generative adversarial loss to obtain a total loss that constrains the training. These three loss functions work together to play a crucial role in ensuring image quality and realism. These three losses guide the model to learn the mapping relationship between the source domain (frozen pathology slide images) and the target domain (paraffin-style pathology slide images) through constraints from different perspectives, ensuring the visual and semantic accuracy of the generated images and achieving image transformation from the source domain to the target domain.
[0111] The image conversion model training device in this embodiment uses acquired frozen pathology slide images and corresponding postoperative paraffin-effected pathology slide image data to train a frozen-paraffin image conversion model. Through the synergistic effect of a generator, discriminator, adversarial loss, and contrastive loss, the target loss function is optimized by maximizing the mutual information of local images. Simultaneously, a dual attention module is introduced into the generator to focus on details within the image, enabling the model to learn the potential correlations between different image domains and achieve effective image conversion. This not only preserves the key information of the frozen pathology images but also endows the images with a clearer paraffin style, making image details more prominent and providing more easily diagnosed information for subsequent classification, thereby further improving the accuracy of image conversion.
[0112] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.
[0113] Example 4
[0114] like Figure 6 The diagram shown is a schematic representation of the frame of an image conversion device. Specifically, it includes:
[0115] The target image acquisition module 201 is used to acquire frozen pathological section images of the target tissue.
[0116] Target image preprocessing module 202 is used to preprocess frozen pathological section images of the target tissue;
[0117] The conversion module 203 is used to input the preprocessed frozen pathological section image of the target tissue into the image conversion model to perform image conversion and obtain the paraffin-style pathological section image of the target tissue;
[0118] The image conversion model is trained using the training device described in Embodiment 3 above.
[0119] The preprocessing steps in the target image preprocessing module 202 are the same as those in S120 of Embodiment 1, in order to maintain the consistency of the image conversion model input.
[0120] After obtaining the paraffin-embedded pathological section image of the target tissue, the device further includes:
[0121] The classification module 204 is used to input the paraffin-style pathological slide image into the image classification model for classification.
[0122] The image classification model is used to classify input paraffin-embedded pathological slide images, with the classification result being either high-risk or low-risk. This classification result is then used to assist in guiding diagnosis. The image classification model can include convolutional neural networks, multi-instance learning networks, transformer networks, or other algorithms. Conventional algorithms can be used to train and optimize the image classification model. The trained surgical method classification model is then evaluated and optimized, and cross-validation and validation set testing are performed to ensure the model's generalization ability and stability. During optimization, the model structure is adjusted, including increasing or decreasing layers and adjusting the number of neurons, to improve the model's complexity and representational ability. Furthermore, hyperparameter tuning, such as learning rate and batch size, is performed to find the optimal training configuration. Parameter adjustments are made based on the model's performance on the validation set. If the loss function stops decreasing prematurely over a period of time, it indicates that the model is overfitting. Reducing the number of layers and neurons is considered to lower the model complexity. If the loss function continues to decrease but the convergence speed is particularly slow, increasing the model complexity and the number of neurons is considered to deepen the network model.
[0123] The image conversion device in this embodiment utilizes a frozen-paraffin pathological section image conversion model to effectively solve the problem of blurred frozen images during surgery, improving image clarity and quality. By improving image clarity and quality, and paying better attention to image details, lesion features can be identified more accurately, and the converted images can be classified more precisely, thus providing doctors with a more reliable basis for selecting surgical methods. By providing clear and accurate image conversion results, the interference of subjective factors is reduced, improving the accuracy and reliability of decision-making.
[0124] Example 5
[0125] Figure 7 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the training method or image conversion method of the image conversion model described in any of the above embodiments. Figure 7 The electronic device 50 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.
[0126] like Figure 7 As shown, the electronic device 50 can be manifested in the form of a general-purpose computing device, such as a server device. The components of the electronic device 50 may include, but are not limited to: at least one processor 51, at least one memory 52, and a bus 53 connecting different system components (including memory 52 and processor 51).
[0127] Bus 53 includes a data bus, an address bus, and a control bus.
[0128] The memory 52 may include volatile memory, such as random access memory (RAM) 521 and / or cache memory 522, and may further include read-only memory (ROM) 523.
[0129] The memory 52 may also include a program tool 525 (or utility) having a set (at least one) program module 524, such program module 524 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.
[0130] The processor 51 executes various functional applications and data processing by running computer programs stored in the memory 52, such as the training method for the image conversion model or the image conversion method provided in any of the above embodiments.
[0131] Electronic device 50 can also communicate with one or more external devices 54 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 55. Furthermore, electronic device 50 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 56. As shown, network adapter 56 communicates with other modules of electronic device 50 via bus 53. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 50, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.
[0132] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.
[0133] Example 6
[0134] This disclosure also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the training method or image conversion method of the image conversion model described in any of the above embodiments.
[0135] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.
[0136] Example 7
[0137] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the training method or image conversion method of the image conversion model described in any of the above embodiments.
[0138] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.
[0139] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.
Claims
1. A method for training an image conversion model, characterized in that, include, Acquire sample images, which include a first sample image and a corresponding second sample image; The first sample image is preprocessed; The preprocessed first sample image is used as the input sample to the image conversion model, and the second sample image is used as the output sample of the image conversion model to train the image conversion model; The image conversion model is a generative adversarial network based on contrastive learning, which includes a generator and a discriminator. The generator includes a dual attention module.
2. The image conversion model training method as described in claim 1, characterized in that, The preprocessing of the first sample image includes, Extract the foreground tissue pathology region from the first sample image; The foreground tissue pathology region is divided into mutually independent image blocks.
3. The image conversion model training method as described in claim 1, characterized in that, The step of using the preprocessed first sample image as input to the image conversion model and the second sample image as output to train the image conversion model specifically includes: Obtain the consistency loss between the input samples and the output samples of the generator; Obtain noise contrast estimation loss; Obtain the generative adversarial loss of the discriminator; The training is constrained by a total loss obtained by weighting and summing the consistency loss, the noise contrast estimation loss, and the generative adversarial loss.
4. The image conversion model training method as described in claim 1, characterized in that, The dual attention module consists of a spatial attention module and a channel attention module, and the discriminator uses the patchGAN model.
5. An image conversion method, characterized in that, include, Acquire frozen pathological section images of the target tissue; Preprocess frozen pathological section images of the target tissue; The preprocessed frozen pathological section image of the target tissue is input into an image conversion model to perform image conversion and obtain a paraffin-style pathological section image of the target tissue. The image conversion model is trained according to the training method described in any one of claims 1-4.
6. The image conversion method as described in claim 5, characterized in that, It also includes the following steps, After obtaining the paraffin-effect pathological section image of the target tissue, the paraffin-effect pathological section image is input into an image classification model for classification.
7. An image conversion model training device, characterized in that, include, The acquisition module is used to acquire sample images, which include a first sample image and a corresponding second sample image; The preprocessing module is used to preprocess the first sample image; The training module is used to input the preprocessed first sample image as an input sample into the image conversion model, and to train the image conversion model by using the second sample image as the output sample of the image conversion model. The image conversion model is a generative adversarial network based on contrastive learning, which includes a generator and a discriminator. The generator includes a dual attention module.
8. An image conversion device, characterized in that, include, The target image acquisition module is used to acquire frozen pathological section images of the target tissue; The target image preprocessing module is used to preprocess frozen pathological section images of the target tissue; The conversion module is used to input the preprocessed frozen pathological section image of the target tissue into the image conversion model to convert the image and obtain the paraffin-style pathological section image of the target tissue. The image conversion model is trained using the training device described in claim 7.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the image conversion model training method according to any one of claims 1 to 4 or the image conversion method according to claim 5 or 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image conversion model training method as described in any one of claims 1 to 4 or the image conversion method as described in claim 5 or 6.