Intelligent pathological section analysis method and system based on image processing technology

Abstract

The application discloses a pathological section intelligent analysis method and system based on image processing technology. By introducing a content-style decoupling autoencoder, the method can accurately separate the content features related to diagnosis and the style features irrelevant to diagnosis from pathological images, thereby ensuring that subsequent intelligent analysis and classification interpretation only rely on pathological tissue morphology information essential to disease state, avoiding model coupling with image staining style, and realizing robustness to style changes. By adopting the content-style decoupling strategy, the AI misjudgment and missed report problems caused by staining variation are effectively overcome, so that the trained model can be stably and reliably applied to pathological section data from different institutions, thereby significantly improving the generalization ability and diagnostic accuracy of the pathological section intelligent analysis model when facing different staining style images.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent analysis, and more specifically, to a pathological section intelligent analysis method and system based on image processing technology. BACKGROUND

[0002] Pathological diagnosis is crucial in modern medicine, especially in the accurate diagnosis and grading of diseases such as tumors. However, the traditional manual reading process under a microscope requires high professional experience and concentration of pathologists, and is of high intensity and long duration. With the growing demand for medical treatment, the relative scarcity of pathologists has led to great pressure on diagnosis efficiency, and the subjectivity of manual judgment may also affect the consistency of diagnosis.

[0003] To address the above challenges, with the breakthrough of deep learning, especially convolutional neural networks (CNN) in image recognition, applying deep learning models to digital pathology image analysis has become mainstream. These models, trained on a large amount of labeled data, can effectively learn pathological features and achieve high-precision classification and detection on specific data sets. However, although existing intelligent analysis solutions perform well in controlled environments, they generally face the core technical bottleneck of staining generalization in actual clinical applications. During the preparation of pathological sections, different institutions, different laboratories, or even the same laboratory in different batches of operations may cause significant differences in staining style due to differences in staining reagents, parameters, and tissue processing. Existing deep learning models, when trained, often not only learn the content features of pathological tissues and cells, but also inadvertently couple or the style features (such as color, brightness) of the data. This coupling of features and style makes the model sensitive to staining style. When the model is trained in one institution (source domain), if it is used in other institutions with different staining styles (target domain), its diagnostic performance will drop sharply, causing a large number of false negatives and false positives. This is because the model may mistakenly consider a specific staining style as a diagnostic basis. Once the color does not match, even if the content features are correct, it may not be recognized, which seriously hinders the clinical promotion of AI pathological diagnosis technology.

[0004] Therefore, an optimized pathological section intelligent analysis method based on image processing technology is expected. SUMMARY

[0005] To solve the above technical problems, the present application is proposed. Embodiments of the present application provide a pathological section intelligent analysis method and system based on image processing technology, which introduces a content-style decoupling autoencoder to accurately separate the content features related to diagnosis and the style features irrelevant from the pathological images, thereby ensuring that the subsequent intelligent analysis and classification interpretation only rely on the pathological tissue morphology information essential to the disease state, avoiding the model coupling with the staining style of the images, and realizing the robustness to style changes. By adopting the content-style decoupling strategy, the AI misjudgment and missed report problems caused by staining variation are effectively overcome, so that the trained model can be stably and reliably applied to pathological section data from different institutions, thereby significantly improving the generalization ability and diagnostic accuracy of the pathological section intelligent analysis model when facing different staining style images.

[0006] According to one aspect of the present application, a pathological section intelligent analysis method based on image processing technology is provided, which comprises:

[0007] obtaining a global visual pathological image;

[0008] performing full-section preprocessing and image block extraction on the global visual pathological image to obtain a pathological original image block set;

[0009] inputting the pathological original image block set into a trained content-style decoupling autoencoder to obtain a content encoding vector set;

[0010] inputting the content encoding vector set into a classifier network to obtain an image block classification result set;

[0011] performing result aggregation and full-section heat map generation on the image block classification result set to obtain a final pathological analysis heat map.

[0012] According to another aspect of the present application, a pathological section intelligent analysis system based on image processing technology is provided, which comprises:

[0013] an image acquisition module for acquiring a global visual pathological image;

[0014] a pathological original image block extraction module for performing full-section preprocessing and image block extraction on the global visual pathological image to obtain a pathological original image block set;

[0015] a content-style decoupling autoencoding module for inputting the pathological original image block set into a trained content-style decoupling autoencoder to obtain a content encoding vector set;

[0016] an image block classification module for inputting the content encoding vector set into a classifier network to obtain an image block classification result set;

[0017] a pathology analysis module configured to aggregate results of the set of image block classification results and generate a whole-slide heat map to obtain a final pathology analysis heat map.

[0018] Compared with the prior art, the pathology slice intelligent analysis method and system based on image processing technology provided by the application can accurately separate the content features related to diagnosis and the style features irrelevant to diagnosis from the pathological image by introducing a content-style decoupling autoencoder, so as to ensure that the subsequent intelligent analysis and classification interpretation only depend on the pathological tissue morphology information essential to the disease state, avoid the model coupling with the staining style of the image, and realize the robustness to the style change. By adopting the content-style decoupling strategy, the AI misjudgment and missed report problems caused by staining variation are effectively overcome, so that the trained model can be stably and reliably applied to the pathological slice data from different institutions, thereby significantly improving the generalization ability and diagnostic accuracy of the pathological slice intelligent analysis model when facing different staining style images. BRIEF DESCRIPTION OF DRAWINGS

[0019] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application taken in conjunction with the accompanying drawings. The accompanying drawings are provided to aid in the understanding of the present application and constitute a part of the specification, together with the written description. The accompanying drawings are not intended to limit the scope of the present application, but to explain the present application together with the written description. In the drawings, the same reference numerals refer to the same components or steps throughout.

[0020] Figure 1 a flowchart of the pathology slice intelligent analysis method based on image processing technology according to the embodiments of the present application;

[0021] Figure 2 a data flow schematic diagram of the pathology slice intelligent analysis method based on image processing technology according to the embodiments of the present application;

[0022] Figure 3 a block diagram of the pathology slice intelligent analysis system based on image processing technology according to the embodiments of the present application. DETAILED DESCRIPTION

[0023] Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0024] As shown in the present application and claims, unless the context clearly indicates otherwise, the words "a," "an," "the," and / or "said" are not limited in scope to the singular, but rather include the plural. Generally, the terms "comprises" and "comprising" are not used in an exclusive sense, but rather are used in an inclusive sense to mean that steps and elements that are included in the described embodiments are not necessarily exhaustive of all steps and elements that can be employed in accordance with the systems and methods described herein.

[0025] While the present application makes various references to certain modules in the system according to embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or server. The modules are merely illustrative, and different aspects of the systems and methods can use different modules.

[0026] Flowcharts are used in the present application to illustrate operations performed by the system according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in the exact order. Rather, various steps can be processed in reverse order, or at the same time, as desired. Other operations can also be added to, or removed from, these processes.

[0027] In the following, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It is obvious that the described embodiments are only a part of the embodiments of the present application, and are not the whole of the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.

[0028] In the technical solution of the present application, an intelligent pathological section analysis method based on image processing technology is proposed. Figure 1 A flowchart of the intelligent pathological section analysis method based on image processing technology according to embodiments of the present application. Figure 2 A system architecture diagram of the intelligent pathological section analysis method based on image processing technology according to embodiments of the present application. As shown in Figure 1 and Figure 2 As shown in the intelligent pathological section analysis method based on image processing technology according to embodiments of the present application, the method comprises the following steps: S1, obtaining a global visual pathology image; S2, performing full-section preprocessing and image block extraction on the global visual pathology image to obtain a set of pathological original image blocks; S3, inputting the set of pathological original image blocks into a trained content-style decoupling autoencoder to obtain a set of content encoding vectors; S4, inputting the set of content encoding vectors into a classifier network to obtain a set of image block classification results; S5, performing result aggregation and full-section heat map generation on the set of image block classification results to obtain a final pathological analysis heat map.

[0029] In particular, the S1 obtains a global visual pathology image. The global visual pathology image refers to a high-resolution digital image obtained from a pathology section by digital technology, covering the entire section area. The pathology section refers to a biological tissue sample (such as a biopsy, surgical resection specimen, etc.) prepared through a series of complex preparation processes in pathological diagnosis practice, including fixation, dehydration, transparency, paraffin embedding, ultrathin sectioning, and staining (such as hematoxylin-eosin H&E staining commonly used in pathological diagnosis), etc., and finally made into a sample placed on a glass slide for morphological observation under a microscope. That is, in the technical solution of the present application, the physical pathology section containing rich biomedical diagnostic information can be efficiently and accurately converted into computer-recognizable and processable image data stream by using advanced digital technology, thereby laying a solid foundation for subsequent intelligent analysis.

[0030] In specific implementation, a traditional pathological glass section is converted into a high-resolution digital image file by a special digital scanning device, such as a whole section scanner. Such a scanner can scan the entire pathology section layer by layer, point by point, or by splicing multiple high-power field regions, thereby generating a super-large size image file containing the entire section area and having multi-scale observation capability. These generated digital images not only completely retain the visual features of the original section, such as tissue morphology, cell structure, and staining information, but also support lossless zooming in, zooming out, panning, etc. on a computer, and can be stored in SVS, NDPI, WSI, etc. various industry standard formats, so as to facilitate subsequent automatic processing and analysis by computer programs.

[0031] In particular, the S2 performs whole section preprocessing and image block extraction on the global visual pathology image to obtain a set of pathological original image blocks. It can be understood that the global visual pathology image often contains a large number of blank background areas (such as the gap of the glass slide), which do not carry any pathological diagnostic information. If directly processed, it will not only increase redundant calculation, but also interfere with model learning and reduce the ability of the model to focus on pathological features. Therefore, in the technical solution of the present application, the global visual pathology image is preprocessed and image block extraction is performed to remove these irrelevant areas and focus on the truly diagnostic tissue part.

[0032] wherein the whole-slide pre-processing refers to a series of preparatory work on the whole global visual pathology image before image patch extraction, mainly including but not limited to tissue region identification (such as removing background by threshold segmentation) and possible data format conversion, preliminary quality check, the purpose of which is to remove redundant information and focus on meaningful pathological regions; and the image patch extraction refers to the process of dividing a large-size global pathology image into a plurality of small image regions (i.e. "image patches" or "patches") of fixed size. This is to adapt to the input size limitation of the deep learning model and facilitate the analysis of local pathological features.

[0033] In specific implementation, first, the global visual pathology image is subjected to threshold segmentation to obtain a tissue region and a blank background region; that is, by threshold segmentation, the region in the image that actually contains biological tissue is distinguished from the background region (such as glass) that does not contain diagnostic information. In an example, this can be achieved by analyzing the color and brightness information of the image. For example, for H&E (hematoxylin-eosin) stained pathology images, the color of the tissue usually falls within a specific color space range (such as magenta, blue, purple, etc.), while the background is mostly white or light blue. A variety of image processing techniques can be used for threshold segmentation: one common method is to perform global thresholding based on the intensity or color components of the pixels, for example, the histogram of the image in a specific color channel (such as the S channel or V channel in the HSV color space, or the R, G, B components in the RGB space) can be calculated, and then Otsu's method is applied to automatically determine the optimal segmentation threshold to divide the pixels into two categories of tissue and background; a more complex adaptive threshold segmentation method can also be used, which dynamically adjusts the threshold according to the brightness distribution of the local region of the image to adapt to the uneven illumination that may exist in the image. For a more robust method, color space conversion (such as from RGB to Lab or HSV) can be used, and then thresholding can be performed on a specific color component (such as the b component in the Lab space or the S component in the HSV space), as these components may have a higher degree of distinction between tissue and background. In addition, morphological operations (such as opening operation, closing operation) can also be applied after threshold segmentation to eliminate small noise points or fill small holes inside the tissue region, so as to obtain a smoother and more accurate tissue mask. Finally, the result of threshold segmentation is a binary mask image, in which the tissue region is marked as 1 and the blank background region is marked as 0.

[0034] Further, the tissue regions are divided into image patches to obtain the set of pathological raw image patches. That is, after the tissue regions in the image are determined, these regions of interest are subdivided into fixed-size image patches. Specifically, this can be done on a predefined grid or by using a sliding window. Specifically, a fixed image patch size (e.g., 256x256 pixels or 512x512 pixels) is first set, which is determined according to the input requirements of the selected deep learning model and the calculation resource constraints. Then, the algorithm traverses the previously obtained tissue region mask. Only when the center point or a large part of the region (e.g., more than 50% or 70% of the pixels) of a potential image patch falls within the tissue region, the image patch will be extracted. The extraction of the image patch can be non-overlapping (i.e., each block does not intersect with each other), or overlapping, which can help capture subtle features at the boundaries of the image patch and increase the amount of data. For each extracted image patch, it is saved as an independent sub-image to obtain the set of pathological raw image patches. It is worth mentioning that during the extraction process, image patches can also be extracted at different resolution levels (e.g., from different pyramid layers of the original scanned image) to support multi-scale analysis.

[0035] In particular, S3, the set of pathological raw image patches is input into the trained content-style decoupled autoencoder to obtain a set of content encoding vectors. It should be understood that in the analysis of pathological sections, traditional deep learning models not only learn the content features of the pathological tissue cells, but also inadvertently couple or learn the style features (such as color, brightness, and saturation) of the data. This coupling of content and style makes the model extremely sensitive to staining style. When the model is trained in one institution (source domain), if it is used in other institutions (target domain) with different staining styles, its diagnostic performance will drop sharply, resulting in a large number of false negatives and false positives. This seriously hinders the clinical promotion of AI pathological diagnosis technology. Therefore, in the technical solution of the present application, the content and style information are explicitly modeled as independent variables through decoupling technology. In this way, the subsequent classifier network can only learn and judge based on pure content features related to the disease state, thereby showing stronger robustness and generalization ability to images with different staining styles, ensuring the consistency and accuracy of diagnosis.

[0036] The content-style decoupled autoencoder includes an encoder network and a decoder network.

[0037] In implementation, the set of pathological original image blocks is input into the trained content-style decoupling autoencoder. In this process, each image block is first processed by the encoder network to encode the input image block into two independent latent vectors: one representing the content encoding vector and the other representing the style encoding vector. Since the autoencoder has been specially trained to achieve content-style decoupling, it means that the content encoding vector output from the encoder will only contain information about the morphology, cell tissue structure, etc. in the image that is truly related to pathological diagnosis, and is irrelevant to the visual style such as staining depth, hue, brightness, etc. The style encoding vector, on the other hand, captures these visual style information. Finally, for each image block in the set of pathological original image blocks, a corresponding content encoding vector is obtained, and a set of content encoding vectors is obtained.

[0038] In particular, in pathological section analysis, the staining style of sections of different medical institutions (such as H&E staining differences) will interfere with the extraction of pathological features. The core goal of the content-style decoupling autoencoder is to separate the pathological content features (such as cell morphology, tissue structure) in the image from the non-pathological style features (such as staining depth, brightness contrast). By training the autoencoder, it can be ensured that the subsequent classifier network only performs pathological classification based on the content encoding vector, eliminating misjudgment caused by style differences and improving the model generalization ability. In the technical solution of the present application, the training process of the content-style decoupling autoencoder includes:

[0039] First, a batch of training original image blocks and their corresponding batch domain labels are obtained. That is, at the beginning of model training, in order to enable the model to understand and distinguish different staining styles, a large number of pathological section image blocks from different sources (i.e. different staining styles or different laboratories) need to be prepared. These image blocks are reasonably grouped into small batches to facilitate efficient use of computing resources and stability of the training process. The key is that each image block in each batch should be pre-assigned a clear domain label, for example, if the image block comes from a certain institution or follows a certain staining batch, it will be assigned a label representing the source (for example, different domains are represented by one-hot encoding). These batch domain labels are not the pathological diagnosis information of the image blocks themselves, but the identification of the non-diagnostic style source, which is the key supervision information for guiding style decoupling, especially for calculating style adversarial loss, in the training process. Here, the batch of training original image blocks refers to a group of image samples processed for each model parameter update, and the batch domain label is the source information or non-pathological attribute identification of the staining style attached to these samples.

[0040] Next, the training batch of raw image patches are inputted into the encoder network to obtain a batch of content encoding vectors and a batch of style encoding vectors. The encoder network is a core component of the content-style decoupling autoencoder, which receives the inputted raw image patches and maps them into two independent latent spaces, generating content encoding vectors representing image content information and style encoding vectors representing image style information, respectively. This step is the basis of decoupling, which reduces the dimensionality of the original high-dimensional image data and abstracts it, providing a concise and meaningful intermediate representation for subsequent decoupling reconstruction and classification tasks.

[0041] Then, the ith content encoding vector and the ith style encoding vector corresponding to the ith training raw image patch are extracted from the batch of content encoding vectors and the batch of style encoding vectors. Since the encoding process in the previous step is performed on the entire batch, it is often necessary to operate on individual image samples (i.e., the "ith image patch") within the batch during subsequent loss function calculations. Therefore, in this step, the ith content encoding vector and the ith style encoding vector corresponding to the ith training raw image patch are extracted from the batch of content encoding vectors and the batch of style encoding vectors to accurately locate and extract the content encoding vector and the style encoding vector corresponding to the specific image patch currently being processed. Here, the ith training raw image patch is any selected sample in the batch, and the corresponding ith content encoding vector and ith style encoding vector are the latent representations of the content and style specific to this sample.

[0042] Then, the jth style encoding vector corresponding to the jth training raw image patch is randomly selected from the batch of style encoding vectors. It should be understood that in the content and style decoupling training strategy, the key is to force the model to learn the independence of content and style. Traditional autoencoders may only learn how to reconstruct the original image, but this does not guarantee that the content and style are truly separated. Therefore, in this step, by introducing a cross-reconstruction mechanism, the model is required to combine the content (ith content encoding vector) of one image patch with the style (jth style encoding vector) of another randomly selected image patch and generate a new image. Here, by randomly selecting the jth style encoding vector, the model must have the ability to combine the content with any style it may encounter, thereby verifying the independence of its content encoding and specific style.

[0043] Further, after fusing the ith content encoding vector and the ith style encoding vector into a self-reconstruction latent encoding vector, the self-reconstruction latent encoding vector is input into a decoder network to obtain a self-reconstruction loss. Specifically, the ith content encoding vector and the ith style encoding vector are information-intensive interacted to obtain a self-reconstruction latent encoding vector; the self-reconstruction latent encoding vector is input into the decoder network to obtain a self-reconstruction ith original image block; and an L2 distance between the self-reconstruction ith original image block and the ith training original image block is calculated as the self-reconstruction loss. In the technical solution of the present application, the content encoding and the corresponding style encoding are combined into a self-reconstruction latent encoding vector, which is input into the decoder network for image reconstruction, and then the pixel-level difference (usually L2 distance, i.e. mean square error) between the reconstructed image and the original image is calculated, so that the model is motivated to learn an accurate enough encoding representation to achieve high-fidelity image reconstruction. This loss ensures that the model does not lose any important information.

[0044] Specifically, in the technical solution of the present application, the self-reconstruction loss is calculated according to the following distance formula:

[0045] ,

[0046] wherein, represents the ith training original image block, represents a self-reconstruction ith original image block, represents a self-reconstruction loss.

[0047] Further, after fusing the ith content encoding vector and the jth style encoding vector into a cross-reconstruction latent encoding vector, the cross-reconstruction latent encoding vector is input into a decoder network to obtain a cross-reconstruction loss. Specifically, the cross-reconstruction latent encoding vector is input into the decoder network to obtain a cross-reconstruction ith original image block; and an L2 distance between the cross-reconstruction ith original image block and the ith training original image block is calculated as the cross-reconstruction loss. This loss is an important part of realizing content-style decoupling. It forces the model to learn that even if the content (ith content encoding vector) of the current image block is combined with a randomly selected style (jth style encoding vector) from a different image block, the reconstructed image should still be consistent with the original image content in terms of content. This means that the model must separate the content features that are truly independent of style. By calculating the L2 distance between the reconstructed image with "consistent content and different style" and the original image, the model is guided to learn a content encoding that can accurately express the pathological morphology of the original image when facing different styles.

[0048] Specifically, in the technical solution of the present application, the cross-reconstruction loss is calculated according to the following distance formula:

[0049] ,

[0050] wherein, denotes cross-reconstruction of the i-th original image block, denotes cross-reconstruction loss.

[0051] Further, based on the i-th style encoding vector and the domain label of the i-th training original image block, a style adversarial loss is calculated. Specifically, the i-th style encoding vector is input into a style discriminator network to obtain an inference domain label probability vector; and a cross-entropy between the inference domain label probability vector and the domain label of the i-th training original image block is calculated as the style adversarial loss. This loss adopts the principle of adversarial learning, aiming to ensure that the style encoding vector does not contain any information that can be used to identify its original domain (i.e., the source of the staining style). Through the style discriminator network, the domain label to which the input style encoding vector belongs is predicted. Meanwhile, the encoder network generates, in the training process, those style encoding vectors that make the style discriminator unable to accurately identify their original domain. In this way of adversarial game, the style encoding vector is finally domain-invariant, i.e., cannot leak any information about its source. By calculating the cross-entropy between the inference domain label probability vector predicted by the discriminator and the true domain label, and taking it as a loss function to optimize the encoder (to generate style encodings that are more difficult to be discriminated) and the discriminator (to make it more accurate in discrimination), the purity of the content encoding is finally ensured, completely eliminating the interference of the staining style.

[0052] Specifically, after the i-th style encoding vector is extracted, it is input into a style discriminator network. This discriminator network is usually a feedforward neural network, and the input layer thereof is designed to match the dimension of the style encoding vector, for example, if the style encoding is 64-dimensional, the input layer of the discriminator also contains 64 neurons. This input vector will be subjected to nonlinear transformation and feature learning through several hidden layers of the discriminator network. These hidden layers are usually composed of fully connected layers and activation functions (such as ReLU). After multiple layers of abstraction, the output layer of the discriminator outputs a vector consistent with the preset number of domains (for example, if there are data from 3 different institutions, the output layer contains 3 neurons). This output layer is usually followed by a softmax activation function. The softmax function converts the original numerical values (logits) of the output into a probability distribution vector, where each element represents the probability that the input style encoding vector belongs to a certain specific domain, and the sum of all elements is 1, i.e., an inference domain label probability vector

[0053] More specifically, in the technical solution of the present application, the style adversarial loss is calculated as follows:

[0054] ,

[0055] wherein, denotes the total number of domain labels, denotes the i-th feature value in the k-th position of the one-hot encoding vector of denotes a logarithm operation, denotes a style adversarial loss.

[0056] Finally, the content-style decoupled autoencoder is trained based on the self-reconstruction loss, cross-reconstruction loss and style adversarial loss, and through the back propagation of gradient descent. Specifically, the weighted sum of the self-reconstruction loss, cross-reconstruction loss and style adversarial loss is calculated as the total loss value; the parameters of the encoder network and the decoder network are updated based on the back propagation of gradient descent based on the total loss value. That is, the aforementioned three loss functions are weighted and summed according to the preset weight to obtain the total loss value; this value represents the comprehensive performance of the model on all training targets.

[0057] Specifically, in the technical solution of the present application, the total loss value is calculated by the following weighted formula:

[0058] ,

[0059] wherein, , and denote preset weight coefficients, denotes the total loss value.

[0060] Then, the parameters of the encoder network and the decoder network (and the style discriminator network) constituting the content-style decoupled autoencoder are iteratively updated using the back propagation of gradient descent, in order to minimize the total loss value. This iteration process continues until the performance of the model on the validation set is stable or reaches the expected target, and finally generates an autoencoder model that can effectively decouple content and style. This process ensures that the trained model can effectively extract pure content features related to diagnosis, thereby adapting to different staining styles of pathological sections.

[0061] In particular, the S4 inputs the content encoding vector set into a classifier network to obtain a set of image block classification results. It should be understood that in traditional pathological diagnosis, a pathologist needs to observe the slice image under a microscope by naked eye, and judge whether there is a lesion, the type and level of the lesion according to the morphological characteristics of cells and tissues. This process is not only time-consuming and labor-intensive, but also easily affected by the experience, fatigue degree and subjective judgment of the doctor. Through the previous step, the staining style information irrelevant to diagnosis has been stripped from the original image block, and the content encoding vector containing only pathological content features has been obtained. These vectors are highly abstract and condensed pathological feature representations. Therefore, in the technical solution of the present application, the content encoding vector set is further input into the classifier network, so as to accurately classify each image block. For example, to judge whether an image block is normal tissue, benign lesion, malignant tumor, inflammation, etc. This classification based on decoupled features can effectively avoid misdiagnosis and missed diagnosis caused by staining generalization in traditional models, significantly improve the accuracy and reliability of diagnosis, and realize high-throughput analysis of a large number of pathological images.

[0062] In specific implementation, first, the content encoding vector set is input into a classifier network, which is usually a deep neural network, and the dimension of the input layer of the classifier network matches the dimension of the content encoding vector. In the classification process, the content encoding vector is forward propagated through several layers of the classifier network (for example, fully connected layer, batch normalization layer, activation function layer, Dropout layer, etc.). These layers will gradually perform nonlinear transformation and abstraction on the features in the content encoding vector, and extract higher-level and more discriminative patterns. Finally, the output layer of the network is usually a fully connected layer with a softmax activation function, and the number of neurons is equal to the number of preset pathological categories (for example, normal, benign, malignant). The softmax function will convert the original Logits value of the output into a probability distribution, indicating the possibility of the image block belonging to each category. The classifier network will perform the same inference process for each content encoding vector in the content encoding vector set and generate the corresponding classification probability. Finally, all image blocks are collected according to the predicted categories (usually the category with the highest probability) obtained by probability to obtain a set of image block classification results.

[0063] In particular, the S5, the result aggregation of the set of image block classification results and the whole slice heat map generation are performed to obtain the final pathological analysis heat map. It should be understood that although the previous step has accurately classified each image block, the classification result of a single image block is local, and provides fragmented information for the pathologist to macroscopically evaluate the lesion distribution, size and infiltration degree of the whole slice. The pathologist usually needs to observe the distribution of the whole lesion area in the tissue, and the isolated image block classification result is difficult to provide such an overall view. The whole slice heat map can superimpose the classification results of discrete and large number of image blocks on the original global pathological image in a visual and intuitive manner. Through color coding, the heat map can clearly indicate the distribution, density and boundary of different pathological regions, so that the pathologist can identify suspicious or lesion regions at a glance, thereby greatly improving the reading efficiency and reducing the workload of manual screening. Such macroscopic visual presentation is more conducive to the doctor to quickly locate the key area, review and diagnose, and makes up for the deficiency of AI focusing only on the microcosmic part. It is worth mentioning that the heat map can also be used as a qualitative and quantitative combination tool, with different colors representing different confidence levels or classification categories, providing additional decision basis for doctors. It improves the explainability and clinical usability of AI-assisted diagnosis, and truly integrates the analysis capability of AI into the daily work of pathologists.

[0064] In specific implementation, first, in the result aggregation stage of the set of image block classification results, the purpose is to organize and integrate the classification information of each image block contained in the set of image block classification results. During image block extraction, each image block is not only given a classification result, but also retains its spatial position information in the original global visual pathology image (for example, through the coordinates x, y of its upper left corner or index). The result aggregation utilizes the spatial position information to map and align the classification results of all image blocks belonging to the same global visual pathology image according to their actual positions on the whole large slice. If there is overlap during image block extraction, the classification results of the overlapping regions need to be considered for weighted averaging or voting during aggregation to eliminate the bias caused by redundant information and ensure that the classification result of each region is accurate and unique. For example, for a classification task involving multiple categories (such as normal, benign and malignant), each image block will have a probability distribution of these categories. During aggregation, the category with the highest prediction probability of each image block can be simply selected, or the probabilities of the overlapping regions can be averaged to obtain a smoother classification confidence map.

[0065] Further, in the full-slice heat map generation stage, the purpose is to draw the aggregated classification results in a visualized form on the corresponding positions of the global visual pathology image. The specific operation is usually as follows: first, a blank transparent canvas with the same size as the original global visual pathology image or scaled down is created; then, the classification results (or the confidence of the category to which it belongs) of each aggregated image block are traversed. According to the accurate coordinates and size of each image block in the original image, the corresponding color or transparency value is drawn to the corresponding area of the heat map canvas; then, according to the preset color mapping scheme, different classification categories (for example, normal, benign, and malignant) or different confidence ranges are mapped to specific colors and saturation. For example, the normal area may be displayed as transparent or light blue, the benign lesion may be displayed as yellow, and the malignant tumor area may be displayed as red or dark red. The deeper the color, the higher the classification confidence or the more serious the lesion degree. If the classification result is a probability value, the color can be interpolated on a gradient color spectrum; finally, the drawn heat map is overlaid on the original global visual pathology image as a transparent layer with a certain transparency. In this way, the pathologist can not only see the original tissue morphology, but also clearly identify the lesion area and its distribution recognized by AI through the superimposed heat map. The transparency of the heat map can be adjusted so that the doctor can observe the underlying tissue details more clearly.

[0066] In summary, the pathological slice intelligent analysis method based on image processing technology according to the embodiments of the present application is illustrated, which introduces a content-style decoupling autoencoder to accurately separate the content features related to diagnosis and the style features irrelevant from the pathology image, thereby ensuring that the subsequent intelligent analysis and classification interpretation only rely on the pathological tissue morphology information essential to the disease state, avoiding the model coupling with the staining style of the image, and realizing the robustness to style changes. By adopting the content-style decoupling strategy, the AI misjudgment and false negative problems caused by staining variation are effectively overcome, so that the trained model can be stably and reliably applied to pathological slice data from different institutions, thereby significantly improving the generalization ability and diagnostic accuracy of the pathological slice intelligent analysis model when facing different staining style images.

[0067] Further, a pathological slice intelligent analysis system based on image processing technology is also provided.

[0068] Figure 3 A block diagram of the pathological slice intelligent analysis system based on image processing technology according to the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the system includes a content-style decoupling autoencoder 100, a feature extractor 200, a feature classifier 300, and a heat map generator 400. Figure 3As shown, the pathological section intelligent analysis system based on image processing technology 300 according to the embodiment of the present application comprises: an image acquisition module 310, configured to acquire a global visual pathology image; a pathological original image block extraction module 320, configured to perform full-section preprocessing and image block extraction on the global visual pathology image to obtain a pathological original image block set; a content-style decoupling auto-encoding module 330, configured to input the pathological original image block set into a trained content-style decoupling auto-encoder to obtain a content encoding vector set; an image block classification module 340, configured to input the content encoding vector set into a classifier network to obtain an image block classification result set; and a pathological analysis module 350, configured to perform result aggregation and full-section heat map generation on the image block classification result set to obtain a final pathological analysis heat map.

[0069] As described above, the pathological section intelligent analysis system based on image processing technology 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with pathological section intelligent analysis algorithm based on image processing technology, etc. In a possible implementation manner, the pathological section intelligent analysis system based on image processing technology 300 according to the embodiment of the present application can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the pathological section intelligent analysis system based on image processing technology 300 can be a software module in the operating system of the wireless terminal, or can be an application program developed for the wireless terminal; of course, the pathological section intelligent analysis system based on image processing technology 300 can also be one of the many hardware modules of the wireless terminal.

[0070] Alternatively, in another example, the pathological section intelligent analysis system based on image processing technology 300 and the wireless terminal can also be separate devices, and the pathological section intelligent analysis system based on image processing technology 300 can be connected to the wireless terminal through a wired and / or wireless network, and transmit interactive information in an agreed data format.

[0071] The above has described the embodiments of the present disclosure, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles, practical application, or improvement of the technology in the market of the embodiments, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. An intelligent pathological section analysis method based on image processing technology, characterized in that, The method comprises: obtaining a global visual pathology image; performing whole-slide preprocessing and image block extraction on the global visual pathology image to obtain a set of pathological original image blocks; inputting the set of pathological original image blocks into a trained content-style decoupling autoencoder to obtain a set of content encoding vectors; inputting the set of content encoding vectors into a classifier network to obtain a set of image block classification results; performing result aggregation and whole-slide heat map generation on the set of image block classification results to obtain a final pathological analysis heat map; wherein the content-style decoupling autoencoder comprises an encoder network and a decoder network; wherein the training process of the content-style decoupling autoencoder comprises: obtaining a batch of training original image blocks and their corresponding batch of domain labels; inputting the batch of training original image blocks into the encoder network to obtain a batch of content encoding vectors and a batch of style encoding vectors; extracting an i-th content encoding vector and an i-th style encoding vector corresponding to an i-th training original image block from the batch of content encoding vectors and the batch of style encoding vectors; randomly selecting an j-th style encoding vector corresponding to an j-th training original image block from the batch of style encoding vectors; after fusing the i-th content encoding vector and the i-th style encoding vector into a self-reconstruction latent encoding vector, inputting the self-reconstruction latent encoding vector into the decoder network to obtain a self-reconstruction loss; after fusing the i-th content encoding vector and the j-th style encoding vector into a cross-reconstruction latent encoding vector, inputting the cross-reconstruction latent encoding vector into the decoder network to obtain a cross-reconstruction loss; calculating a style adversarial loss based on the i-th style encoding vector and the domain label of the i-th training original image block; training the content-style decoupling autoencoder based on the self-reconstruction loss, the cross-reconstruction loss and the style adversarial loss through back propagation of gradient descent.

2. The method of claim 1, wherein the image processing technique-based intelligent analysis of pathological sections is characterized by, performing whole-slide preprocessing and image block extraction on the global visual pathology image to obtain a set of pathological original image blocks, comprising: performing threshold segmentation on the global visual pathology image to obtain a tissue region and a blank background region; performing image block division on the tissue region to obtain the set of pathological original image blocks. 3.The image processing technology-based intelligent pathological section analysis method according to claim 1, characterized in that, after fusing the i-th content encoding vector and the i-th style encoding vector into a self-reconstruction latent encoding vector, inputting the self-reconstruction latent encoding vector into the decoder network to obtain a self-reconstruction loss, comprising: performing information-intensive interaction on the i-th content encoding vector and the i-th style encoding vector to obtain a self-reconstruction latent encoding vector; inputting the self-reconstruction latent encoding vector into the decoder network to obtain a self-reconstruction i-th original image block; calculating an L2 distance between the self-reconstruction i-th original image block and the i-th training original image block as the self-reconstruction loss. 4.The image processing technology-based intelligent pathological section analysis method according to claim 1, characterized in that, after fusing the i-th content encoding vector and the j-th style encoding vector into a cross-reconstruction latent encoding vector, inputting the cross-reconstruction latent encoding vector into the decoder network to obtain a cross-reconstruction loss, comprising: inputting the cross-reconstruction latent encoding vector into the decoder network to obtain a cross-reconstruction i-th original image block; calculating an L2 distance between the cross-reconstruction i-th original image block and the i-th training original image block as the cross-reconstruction loss. 5.The image processing technology-based intelligent pathological section analysis method according to claim 1, characterized in that, calculating a style adversarial loss based on the i-th style encoding vector and the domain label of the i-th training original image block, including: inputting the i-th style encoding vector into a style discriminator network to obtain an inference domain label probability vector; calculating a cross-entropy between the inference domain label probability vector and the domain label of the i-th training original image block as the style adversarial loss. 6.The image processing technology-based intelligent pathological section analysis method according to claim 1, characterized in that, training the content-style decoupled auto-encoder based on the self-reconstruction loss, the cross-reconstruction loss and the style adversarial loss through back propagation of gradient descent, including: calculating a weighted sum of the self-reconstruction loss, the cross-reconstruction loss and the style adversarial loss as a total loss value; updating parameters of the encoder network and the decoder network based on the total loss value through back propagation of gradient descent.

7. An intelligent pathological section analysis system based on image processing technology, characterized in that, including: an image acquisition module configured to acquire a global visual pathology image; a pathology original image block extraction module configured to perform full slice preprocessing and image block extraction on the global visual pathology image to obtain a pathology original image block set; a content-style decoupled auto-encoding module configured to input the pathology original image block set into a trained content-style decoupled auto-encoder to obtain a content encoding vector set; an image block classification module configured to input the content encoding vector set into a classifier network to obtain an image block classification result set; a pathology analysis module configured to perform result aggregation and full slice heat map generation on the image block classification result set to obtain a final pathology analysis heat map; wherein the content-style decoupled auto-encoder includes an encoder network and a decoder network; wherein the training process of the content-style decoupled auto-encoder includes: acquiring a batch of training original image blocks and their corresponding batch of domain labels; inputting the training batch of original image blocks into the encoder network to obtain a batch of content encoding vectors and a batch of style encoding vectors; extracting an i-th content encoding vector corresponding to an i-th training original image block from the batch of content encoding vectors and the batch of style encoding vectors; randomly selecting a j-th style encoding vector corresponding to a j-th training original image block from the batch of style encoding vectors; after fusing the i-th content encoding vector and the i-th style encoding vector into a self-reconstruction latent encoding vector, inputting the self-reconstruction latent encoding vector into the decoder network to obtain a self-reconstruction loss; after fusing the i-th content encoding vector and the j-th style encoding vector into a cross-reconstruction latent encoding vector, inputting the cross-reconstruction latent encoding vector into the decoder network to obtain a cross-reconstruction loss; calculating a style adversarial loss based on the i-th style encoding vector and the domain label of the i-th training original image block; training the content-style decoupled auto-encoder based on the self-reconstruction loss, the cross-reconstruction loss and the style adversarial loss through back propagation of gradient descent.

Images