Unpaired pathological image to pseudo-dic image conversion method and application thereof

Abstract

The application provides a non-paired pathological image to pseudo DIC image conversion method and application thereof, and belongs to the technical field of medical image processing and computer vision. Technical scheme points are as follows: a pathological image conversion network comprising a generator, a pre-trained topological feature extractor and a topological consistency discriminator is constructed; topological features of images in a feature space are extracted by using the topological feature extractor, and topological consistency constraints are established to retain pathological structure semantics; the topological consistency discriminator is used to establish topological adversarial constraints, so that the topological feature distribution of generated images approximates to real DIC images; and the network is subjected to adversarial training in combination with image domain discrimination constraints, so that a structure-faithful pseudo DIC image generation model is obtained. The application is mainly used for enhancement and visualization of digital pathological images, and can assist in pathological diagnosis and quantitative analysis.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision technology, and in particular to a method for converting unpaired pathological images to pseudo-DIC images and its application. Background Technology

[0002] Differential interferometry (DIC) microscopy can produce high-contrast images with a three-dimensional relief effect based on the sample's refractive index gradient and thickness difference without staining, making it particularly suitable for observing fine structures such as cell membranes and nuclear membranes. However, DIC microscopes are expensive and complex to operate, and a large number of archived images in clinical pathology departments are standard bright-field stained images.

[0003] To obtain visual information similar to DIC from bright-field images, existing technologies mainly include the following approaches:

[0004] Firstly, physical model-based methods deduce phase information from bright-field images by simulating the DIC optical path, but these methods are computationally complex, sensitive to model parameters, and difficult to adapt to different tissue types and staining differences. Secondly, the end-to-end method based on paired deep learning uses bright-field and real DIC images of the same sample to train convolutional neural networks or generative adversarial networks. However, this method is highly dependent on large-scale, high-quality paired data, which is extremely difficult and costly to obtain in clinical practice. Third, the image style transfer method based on unpaired deep learning uses a recurrent consistency generative adversarial network framework, which can perform inter-domain transformation without pairing data. However, when the general framework is applied to pathological images, it is easy to cause distortion of key pathological topology, such as changes in cell nucleus shape, blurred or broken boundaries. Fourth, methods based on simple image filters use directional gradient or embossing filters to simulate shadow effects, but the generated images are coarse and cannot truly reflect optical thickness and interference effects, thus having limited diagnostic value.

[0005] Therefore, there is an urgent need for a method for converting unpaired pathological images to pseudo-DIC images and its application, in order to solve the problems existing in the current technology. Summary of the Invention

[0006] This invention provides a method for converting unpaired pathological images to pseudo-DIC images and its application. It addresses the problem that existing unpaired image conversion technologies cannot ensure that key pathological topological structures such as cell morphology and tissue boundaries are not distorted during image style conversion when paired data guidance is lacking.

[0007] The core technology of this invention is to introduce a pre-trained topological feature extractor and a topological consistency discriminator with fixed parameters into an unpaired image conversion network. By simultaneously constructing topological consistency loss and topological adversarial loss in the high-level semantic feature space, explicit structural semantic constraints are imposed on the image generation process.

[0008] In a first aspect, the present invention provides a method for converting unpaired pathological images to pseudo-DIC images, the method comprising the following steps: Obtain unpaired source domain image sets and target domain image sets, where the source domain images are bright-field pathological images and the target domain images are true differential interferometry (DIC) images; A pathological image conversion network is constructed, which includes a generator, an image domain discriminator, a pre-trained topological feature extractor, and a topological consistency discriminator. The generator is used to convert the input bright-field pathological image into a pseudo-DIC image; A topological feature extractor is used to extract the topological features of bright-field pathological images and pseudo-DIC images in the feature space, and topological consistency constraints are established based on the semantic correlation of the topological features in the feature space. A topology consistency discriminator is used to distinguish the topology features of fake DIC images from those of real DIC images, and topology adversarial constraints are established. Based on topological consistency constraints, topological adversarial constraints, and image domain discrimination constraints established by the image domain discriminator, a total loss function is constructed by weighted summation, and the pathological image conversion network is adversarially trained. The trained generator is used to convert the bright-field pathological image to be converted, and output a pseudo-DIC image with preserved structure.

[0009] Furthermore, the topological feature extractor is a semantic segmentation network pre-trained on a pathological annotation dataset. The semantic segmentation network maintains fixed parameters during the training of the pathological image conversion network. The pre-training task of the semantic segmentation network includes at least one of cell nucleus segmentation, tissue region segmentation, and cell membrane segmentation.

[0010] Furthermore, the topology feature extractor adopts a U-Net network structure, which includes an encoder and a decoder; the extraction of topology features includes: obtaining the feature map output of a specific upsampling layer in the decoder as the topology feature.

[0011] Furthermore, topological consistency constraints are achieved through topological consistency loss, which is used to constrain the minimization of the distance in the feature space output by the topological feature extractor before and after the same bright-field pathological image is transformed by the generator, so as to preserve the pathological topological semantics of the original image, including the cell nucleus outline and tissue region distribution.

[0012] Furthermore, the topological adversarial constraint is achieved through the topological adversarial loss, which is obtained through an adversarial game between the generator and the topological consistency discriminator. This loss is used to make the topological feature distribution of the pseudo-DIC image generated by the generator approximate the topological feature distribution of the real DIC image.

[0013] Furthermore, the topology consistency loss adopts the L1 norm loss function; the topology adversarial loss adopts the least squares generative adversarial loss function; the topology consistency discriminator is a fully convolutional downsampling classification network, whose input is the multi-channel feature map output by the topology feature extractor, and whose output is a scalar value representing the true and false probabilities.

[0014] Furthermore, the pathological image conversion network also includes an inverse generator for reconstructing the generated pseudo-DIC image into a bright-field image to establish a cycle consistency constraint; the image domain discriminator includes a first discriminator for distinguishing between real DIC images and generated pseudo-DIC images, and a second discriminator for distinguishing between real bright-field images and reconstructed bright-field images; the total loss function also includes adversarial loss and cycle consistency loss.

[0015] Secondly, the present invention provides an apparatus for converting unpaired pathological images to pseudo-DIC images, comprising: The data acquisition module is used to acquire the set of unpaired bright-field pathological images and the set of true differential interference (DIC) images. The model building module is used to construct a pathological image conversion network that includes a generator, an image domain discriminator, a pre-trained topological feature extractor, and a topological consistency discriminator. The image conversion module is used to convert bright-field pathological images into pseudo-DIC images using a generator; The topology constraint module is used to obtain the topology consistency loss mapped to the feature space using the topology feature extractor, and to obtain the topology adversarial loss at the feature level using the topology consistency discriminator. The training application module is used to construct a total loss function based on topological consistency loss, topological adversarial loss, and image domain discrimination constraints established by the image domain discriminator, and to perform adversarial training on the generator; and to use the trained generator to output pseudo-DIC images with structural fidelity to brightfield pathological images.

[0016] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the above-described method for converting unpaired pathological images to pseudo-DIC images.

[0017] Fourthly, the present invention provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the above-described method for converting unpaired pathological images to pseudo-DIC images.

[0018] The main contributions and innovations of this invention are as follows: 1. Overcoming the bottleneck of paired data acquisition: This invention adopts an unpaired image generation framework, which does not require providing a pair of bright field and real differential interference difference images from the same sample with strict pixel-level alignment during the training phase. End-to-end training can be completed using only two non-homogeneous image sets, which greatly reduces the implementation cost and data dependence of the algorithm.

[0019] 2. Achieving high-fidelity conversion of pathological structures: By introducing a pre-trained semantic segmentation network as a feature extractor and applying topological consistency constraints at the feature level, this invention transforms the prior knowledge of "preserving key pathological structures" into a computable loss gradient, effectively avoiding problems such as cell nucleus fusion, breakage, and morphological changes that occur during the generation process, and ensuring the high reliability of the generated image in terms of morphological features.

[0020] 3. Improve visual rendering quality and downstream task performance: Under the combined effect of topological adversarial loss and cycle consistency loss, this invention not only enables the generated image to highly reproduce the optical imaging effect of a real microscope in terms of stereoscopic sense and contrast, but also provides higher quality feature input for subsequent computer-aided diagnostic systems such as cell segmentation and lesion classification because it retains accurate tissue semantics.

[0021] 4. Excellent model generalization ability: The topological constraint mechanism in this invention enables the model to focus on learning the essential mapping between pathological topology and specific optical rendering styles, rather than simple pixel-level interpolation, so that it has excellent robustness and adaptability for bright field images generated by different tissue types, staining differences and scanning instruments.

[0022] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is an architecture diagram of a non-paired pathological image to pseudo-DIC image conversion system according to an embodiment of the present invention; Figure 2This is a comparison diagram of the generation effects of the method of the present invention according to an embodiment of the present invention and a baseline method (such as standard CycleGAN); Figure 3 This is a feature visualization diagram of the topology feature extractor T according to an embodiment of the present invention; Figure 4 This is a flowchart of a method for converting unpaired pathological images to pseudo-DIC images according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0025] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0026] This invention provides a method for converting unpaired pathological images to pseudo-DIC images by incorporating topological constraints. The overall concept is as follows: based on an unpaired image conversion framework, a pre-trained, parameter-fixed topological feature extractor is introduced, along with a topological consistency discriminator, forming a topological constraint module. Here, "topological features" refer to morphological structural features reflecting the biological characteristics of pathological tissues, specifically including the outline, size, and shape of cell nuclei, as well as the spatial distribution and connectivity between cells and tissue regions. This topological constraint module, by applying topological consistency and topological adversarial constraints in a high-level semantic feature space, guides the generator to strictly preserve key pathological topological structures in the original image, including cell nucleus outlines and tissue region distribution, while altering the image rendering style. This achieves the generation of structurally accurate pseudo-DIC images without requiring paired training data.

[0027] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0028] Example 1 See Figure 1 This is a schematic diagram of the overall system architecture provided in this embodiment. The diagram illustrates the data flow and loss calculation relationship between the bright-field image domain (source domain A) and the DIC image domain (target domain B).

[0029] like Figure 4 As shown in the figure, this embodiment provides a method for converting unpaired pathological images to pseudo-DIC images with topological constraints, which specifically includes the following steps: Step S1: Data preparation.

[0030] Two unpaired image domains are collected: source domain A and target domain B. Source domain A contains a large number of bright-field pathological images, specifically scanned images of pathological sections stained with hematoxylin and eosin; target domain B contains a large number of true differential interference contrast (DIC) pathological images. The images in the two image sets do not need to come from the same sample, i.e., there is no pixel-level pairing relationship.

[0031] In this embodiment, the number of images in target domain B is more than 3000 to ensure coverage of various illumination changes, cell densities, and tissue types in DIC images. Preferably, the images in source domain A and target domain B are taken from the same tissue or organ type, such as both being lung tissue or both being breast tissue, or lesion types with similar histological morphology, such as both being adenocarcinoma tissue. Although the two sets of images are not paired at the pixel level, they contain the same category of pathological structures at the semantic level, such as cell nuclei, glandular ducts, and stroma. This helps the topology feature extractor to effectively extract and constrain cross-modal structural consistency.

[0032] Step S2: Construct the network model.

[0033] A pathological image transformation network was constructed, incorporating a ternary training paradigm of "generation-discrimination-topological constraint". Specific components include: Generator G and inverse generator F: Generator G is responsible for converting the brightfield pathological image of source domain A into a pseudo-DIC image; inverse generator F is responsible for reconstructing the generated pseudo-DIC image back into a brightfield image, used to establish cycle consistency constraints. Both generators G and F adopt a ResNet structure, consisting of downsampling convolutional layers, 9 residual blocks, and upsampling deconvolutional layers.

[0034] Specifically, the downsampling part consists of three convolutional layers with 64, 128, and 256 channels respectively, a 3×3 kernel size, and a stride of 2; each residual block consists of two 3×3 convolutional layers, maintaining 256 channels; the upsampling part consists of three deconvolutional layers with 128, 64, and 3 channels respectively, a 3×3 kernel size, and a stride of 2. The use of residual blocks helps to preserve the content structure information of the input image while changing the image style.

[0035] Image domain discriminators Dy and Dx: The first discriminator Dy distinguishes between real DIC images in the target domain B and pseudo DIC images generated by generator G; the second discriminator Dx distinguishes between real bright-field pathological images and bright-field images reconstructed by generator F. Both discriminators employ a PatchGAN structure, using a multi-layer convolutional network to determine the authenticity of locally overlapping regions in the input image, outputting a two-dimensional discrimination matrix. Each matrix element corresponds to the probability of authenticity for a local patch in the input image, thereby enhancing the ability to discriminate the realism of local textures.

[0036] The topology feature extractor T employs a pre-trained U-Net semantic segmentation network on the publicly available MoNuSeg dataset. This U-Net network contains a symmetric encoder-decoder architecture. The encoder consists of four downsampling blocks, each containing two 3×3 convolutional layers and one 2×2 max-pooling layer, with channel numbers of 64, 128, 256, and 512 respectively. The decoder consists of four upsampling blocks, each containing one 2×2 upsampling layer and two 3×3 convolutional layers, with channel numbers of 512, 256, 128, and 64 respectively. The pre-training task is cell nucleus semantic segmentation, and the dataset contains over 2000 cell nucleus annotations from seven different organs, including the liver, kidney, prostate, and breast. After pre-training, all network parameters of the topology feature extractor T are frozen to remain unchanged during subsequent adversarial training, serving only as a fixed feature space mapping function. In this embodiment, the topological feature extractor T extracts the output feature map of the second upsampled block in the decoder. This feature map has 128 channels, which is downsampled by 4 times relative to the input image, and is rich in high-level semantic information such as the shape, boundary and spatial distribution of cell nuclei.

[0037] The topology consistency discriminator Dt is designed as a fully convolutional downsampling classification network, consisting of four convolutional modules. Each module contains a 3×3 convolutional layer (stride of 2), a batch normalization layer, and a LeakyReLU activation function, with channel numbers of 128, 256, 512, and 1 respectively. The last layer outputs a scalar value representing the probability that the input feature map belongs to the topological features of the real DIC image. Dt does not contain upsampling layers or a segmentation head; its task is to distinguish topological features extracted from real DIC images from those extracted from generated pseudo-DIC images.

[0038] Step S3: Define the loss function.

[0039] Total loss function It is obtained by weighted summation of image domain adversarial loss, cycle consistency loss, topological consistency loss, and topological adversarial loss, and its mathematical expression is:

[0040] The following sections provide a detailed explanation of each loss item.

[0041] Image domain adversarial loss It consists of two parts: the adversarial loss between generator G and discriminator Dy, and the adversarial loss between generator F and discriminator Dx. This embodiment uses the least squares generative adversarial loss form:

[0042]

[0043]

[0044] Where x represents a bright-field pathological image from source domain A, y represents a true DIC image from target domain B, and E represents the expectation.

[0045] Cycle consistency loss This is used to ensure that the image can be reconstructed from the original image after cross-domain transformation. This embodiment uses L1 norm loss, the mathematical expression of which is:

[0046] This loss function ensures that the generator does not arbitrarily change the image content, and is a fundamental constraint for achieving unpaired image transformation.

[0047] Topology consistency loss This is used to minimize the distance in the feature space output by the topological feature extractor T before and after the same bright-field pathological image is transformed by the generator G. In this embodiment, The L1 norm loss function is used, and its mathematical expression is:

[0048] in, This represents the multi-channel feature map tensor output by the topological feature extractor. This represents the summation of the absolute values ​​of all elements of the feature map tensor. By minimizing this loss, the generator G is forced to preserve the pathological topological semantics of the original image, including cell nucleus outlines, shapes, sizes, and tissue region distribution, while changing the image rendering style.

[0049] Topological adversarial loss This is obtained through an adversarial game between the generator G and the topological consistency discriminator Dt. This embodiment employs a least-squares generative adversarial loss, the mathematical expression of which is:

[0050] in, Let T(x) be the scalar probability value output by the topology consistency discriminator. The constant 1 represents the soft label of the true distribution, indicating that the probability of prompting the discriminator Dt to classify it as true is maximized. The goal of this loss function is to make the topological feature distribution of the pseudo-DIC image generated by the generator G approximate the topological feature distribution of the true DIC image, that is, to make Dt unable to distinguish between T(G(x)) and T(y).

[0051] The optimization objective of the topology consistency discriminator Dt adopts an independent loss function. Its mathematical expression is:

[0052] That is, the goal of Dt is to determine the topological feature T(y) from the real DIC image as true (output approaches 1) and the topological feature T(G(x)) from the generated image as false (output approaches 0).

[0053] The weights for each loss are set as follows: This corresponds to image domain adversarial loss; This corresponds to the cycle consistency loss; This corresponds to a loss of topological content consistency. This corresponds to the topological style adversarial loss. The above weight values ​​are merely preferred examples; those skilled in the art can adjust the weight ratios according to actual application scenarios. and Comparison and Higher values ​​are chosen to prioritize the fidelity of both image content and pathological structure. Typical adjustable ranges for each weight are: , , , .

[0054] Step S4: Model training.

[0055] The network constructed in step S2 is trained end-to-end using the unpaired image data obtained in step S1.

[0056] The training process employs an alternating optimization strategy: in each iteration, the parameters of generators G and F are first fixed, while the parameters of discriminators Dy, Dx, and Dt are optimized; then, the parameters of all discriminators are fixed, while the parameters of generators G and F are optimized. The optimizer used is the Adam optimizer, with momentum parameters... , The initial learning rate was set to 0.0002 and linearly decayed to zero over the last 100 epochs of training. The batch size was set to 1. Training was performed for 200 epochs on a dataset containing thousands of bright-field images of colon adenocarcinoma and thousands of DIC images of cells from independent sources.

[0057] The criterion for convergence is: the total loss function. The value fluctuates by less than 5% of its average value over 20 consecutive periods. Throughout the training process, the parameters of the topological feature extractor T remain fixed, do not participate in gradient updates, and only serve as a feature space mapping function to provide topological semantic constraints.

[0058] Step S5: Image conversion.

[0059] After training, the bright-field pathological image to be converted is input into the trained generator G, which outputs a high-quality pseudo-DIC image with preserved structure. For high-resolution whole-slice bright-field pathological images, a sliding window inference strategy is adopted during the image conversion process: the whole-slice image is divided into multiple overlapping local image blocks according to a set step size (e.g., 256 pixels), and these local pseudo-DIC image blocks are sequentially input into the trained generator G to obtain the local pseudo-DIC image blocks; then, an edge overlap fusion algorithm (e.g., Gaussian weighted fusion) is used to stitch the local pseudo-DIC image blocks into a complete whole-slice pseudo-DIC image to eliminate artifact boundaries at the image block stitching points.

[0060] The pseudo-DIC image visually exhibits DIC imaging features with a three-dimensional relief effect and high contrast, while also being highly consistent with the original bright-field image in key pathological structures such as cell nucleus morphology, cell boundaries, and glandular structures.

[0061] See Figure 2 This figure compares the generation results of the method in this embodiment with the standard CycleGAN method. The left side of the figure shows the input bright-field pathological image, the middle side shows the pseudo-DIC image output by the standard CycleGAN method, and the right side shows the pseudo-DIC image output by the method in this embodiment. As can be seen from the areas marked by arrows, the image generated by the standard CycleGAN method suffers from altered cell nucleus shapes and blurred boundaries, while the image generated by the method in this embodiment maintains a high degree of consistency with the original bright-field image in key structures such as the cell nucleus, effectively avoiding nuclear fusion or morphological distortion.

[0062] See Figure 3 The figure shows the feature visualization of the topological feature extractor T. It illustrates the feature activation maps of T on the bright-field image, the generated pseudo-DIC image, and the real DIC image. It can be clearly seen that the pseudo-DIC image generated by the method in this embodiment maintains the structural correlation with the original bright-field image in the topological feature space, and its feature presentation style is close to that of the real DIC image, intuitively demonstrating the synergistic effect of topological consistency loss and topological adversarial loss.

[0063] The beneficial effects of this embodiment include: no paired training data is required, and training can be performed with only two unpaired image sets, which significantly reduces the application threshold and cost of the method; the generated pseudo-DIC images closely approximate the real DIC imaging effect in terms of texture, contrast and three-dimensionality; due to the high structural fidelity, the generated images can be used to assist in downstream quantitative analysis tasks such as cell segmentation and classification; and it has a strong generalization ability to different tissue types, staining differences and scanning instruments.

[0064] Example 2 This embodiment optimizes the prior knowledge for segmentation of multiple types of tissues based on Embodiment 1.

[0065] The topological feature extractor T is a model pre-trained on tissue region segmentation datasets such as GlaS or BCSS. This model can distinguish multiple tissue types, including epithelium, stroma, necrosis, and lymphocyte infiltration. The pre-training task is semantic segmentation of multiple tissue regions.

[0066] When extracting topological features, feature maps of specific upsampling layers in the U-Net decoder are obtained. Specifically, the output feature map of the second upsampling block in the decoder is extracted. This feature map is downsampled by a factor of 4 relative to the input image, i.e., a feature layer with a stride of 4 and 128 channels. This layer's features can simultaneously capture cellular and tissue-level semantic information.

[0067] This embodiment, based on the loss function in Embodiment 1, modifies the topology consistency loss. To extend the algorithm, in addition to the L1 norm loss of the feature map, a structural similarity loss based on the predicted segmentation map is added. Specifically, the predicted segmentation map is obtained by performing an argmax operation on the feature map output by T. The Dice loss between the original brightfield image x and the predicted segmentation map of the generated pseudo-DIC image G(x) is calculated, and its mathematical expression is:

[0068] The extended topology consistency loss is:

[0069] in The weight for the newly added loss term typically ranges from 1 to 3.

[0070] Regarding training details, for whole-section pathology images, 512×512 pixel image patches were randomly cropped from the images for training. Bright-field and DIC image patches of mixed tissue types were used for training, including breast cancer and prostate cancer.

[0071] The pseudo-DIC images generated in this embodiment not only clearly distinguish cellular structures but also produce discriminative contrast between different tissue regions. For example, between dense tumor epithelial regions and loose matrix regions, a differentiated three-dimensional relief effect similar to that observed under a real DIC microscope can be presented, with a stronger overall sense of topological hierarchy, further enhancing the auxiliary diagnostic value of the images in histopathological evaluation.

[0072] Example 3 Based on the above method embodiments, this embodiment provides a system for converting unpaired pathological images to pseudo-DIC images by incorporating topological constraints. The system includes: The data acquisition module is used to acquire the set of unpaired bright-field pathological images and the set of true differential interference (DIC) images. The model building module is used to construct a pathological image conversion network that includes a generator, an inverse generator, an image domain discriminator, a pre-trained topological feature extractor, and a topological consistency discriminator. The image conversion module is used to convert bright-field pathological images into pseudo-DIC images using a generator; The topology constraint module is used to obtain the topology consistency loss mapped to the feature space using the topology feature extractor, and to obtain the topology adversarial loss at the feature level using the topology consistency discriminator. The training application module is used to perform adversarial training on the generator based on the total loss function, and to use the trained generator to output pseudo-DIC images with structural fidelity to brightfield pathological images.

[0073] Example 4 This embodiment also provides an electronic device, see reference. Figure 5 It includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0074] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0075] Memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to a data processing device. In a particular embodiment, memory 404 is non-volatile memory. In a particular embodiment, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0076] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.

[0077] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the non-paired pathological image to pseudo-DIC image conversion methods in the above embodiments.

[0078] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.

[0079] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0080] Input / output device 408 is used to input or output information.

[0081] Example 5 This embodiment also provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the method for converting unpaired pathological images to pseudo-DIC images according to Embodiment 1.

[0082] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0083] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0084] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 1 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0085] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0086] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims

1. A method for converting unpaired pathological images to pseudo-DIC images by incorporating topological constraints, characterized in that, Includes the following steps: Obtain an unpaired set of source domain images and a set of target domain images, wherein the source domain images are bright-field pathological images and the target domain images are true differential interference (DIC) images; A pathological image conversion network is constructed, which includes a generator, an image domain discriminator, a pre-trained topological feature extractor, and a topological consistency discriminator. The generator is used to convert the input bright-field pathological image into a pseudo-DIC image; The topological features of the bright-field pathological image and the pseudo-DIC image in the feature space are extracted using the topological feature extractor, and topological consistency constraints are established based on the semantic relevance of the topological features in the feature space. The topological consistency discriminator is used to distinguish the topological features of the fake DIC image from those of the real DIC image, and topological adversarial constraints are established. Based on the topological consistency constraint, the topological adversarial constraint, and the image domain discrimination constraint established by the image domain discriminator, a total loss function is constructed by weighted summation, and the pathological image conversion network is subjected to adversarial training. The trained generator is used to convert the bright-field pathological image to be converted, and output a structure-preserving pseudo-DIC image.

2. The method as described in claim 1, characterized in that, The topological feature extractor is a semantic segmentation network pre-trained on a pathological annotation dataset. The semantic segmentation network maintains fixed parameters during the training of the pathological image conversion network. The pre-training task of the semantic segmentation network includes at least one of cell nucleus segmentation, tissue region segmentation, and cell membrane segmentation.

3. The method as described in claim 2, characterized in that, The topology feature extractor adopts a U-Net network structure, which includes an encoder and a decoder; extracting the topology features includes: obtaining the feature map output by a specific upsampling layer in the decoder as the topology feature.

4. The method as described in claim 1, characterized in that, The topological consistency constraint is achieved through topological consistency loss, which is used to minimize the distance between the same bright-field pathological image before and after transformation by the generator in the feature space output by the topological feature extractor, so as to preserve the pathological topological semantics of the original image, including cell nucleus contours and tissue region distribution.

5. The method as described in claim 4, characterized in that, The topological adversarial constraint is achieved through topological adversarial loss, which is obtained through adversarial game between the generator and the topological consistency discriminator, and is used to make the topological feature distribution of the pseudo-DIC image generated by the generator approximate the topological feature distribution of the real DIC image.

6. The method as described in claim 5, characterized in that, The topology consistency loss adopts the L1 norm loss function; the topology adversarial loss adopts the least squares generative adversarial loss function; the topology consistency discriminator is a fully convolutional downsampling classification network, whose input is the multi-channel feature map output by the topology feature extractor, and whose output is a scalar value representing the probability of being true or false.

7. The method as described in claim 1, characterized in that, The pathological image conversion network also includes an inverse generator for reconstructing the generated pseudo-DIC image into a bright-field image to establish a cycle consistency constraint; the image domain discriminator includes a first discriminator for distinguishing between real DIC images and generated pseudo-DIC images, and a second discriminator for distinguishing between real bright-field images and reconstructed bright-field images; the total loss function also includes adversarial loss and cycle consistency loss.

8. A system for converting unpaired pathological images to pseudo-DIC images by incorporating topological constraints, characterized in that, include: The data acquisition module is used to acquire the set of unpaired bright-field pathological images and the set of true differential interference (DIC) images. The model building module is used to construct a pathological image conversion network that includes a generator, an image domain discriminator, a pre-trained topological feature extractor, and a topological consistency discriminator. The image conversion module is used to convert bright-field pathological images into pseudo-DIC images using the generator; The topology constraint module is used to obtain the topology consistency loss mapped to the feature space using the topology feature extractor, and to obtain the topology adversarial loss at the feature level using the topology consistency discriminator. The training application module is used to construct a total loss function by weighted summation based on the topology consistency loss, the topology adversarial loss, and the image domain discrimination constraints established by the image domain discriminator, and to perform adversarial training on the generator. The trained generator then outputs a pseudo-DIC image that has structural fidelity to the bright-field pathological image.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling a process to execute the process, the process including the method according to any one of claims 1 to 7.

Images