Pathological image imaging quality assessment method

Through the frequency domain feature and feature distillation method based on deep neural network, the problem of low pathological image quality caused by out-of-focus blurring of optical microscopy imaging is solved, and the accuracy of image evaluation and diagnostic reliability are improved.

CN119048446AActive Publication Date: 2024-11-29JIANGSU MICROCONTROL BIOTECHNOLOGY CO LTD

Patent Information

Application Number
CN202411087165.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-29
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

Out-of-focus blur in optical microscopy imaging results in low quality of pathological images, affecting the accuracy of diagnosis and the effectiveness of computer-assisted diagnostic systems.

Method used

Using a pathological image imaging quality evaluation method combining frequency domain features and feature distillation, a deep neural network-based model is constructed, and clear and blurred image pairs are trained to optimize the model to improve the accuracy of image quality evaluation.

Benefits of technology

It effectively improves the accuracy and stability of the evaluation of the quality of pathological image imaging, reduces diagnostic errors, and enhances the reliability of lesion judgment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048446A_ABST
    Figure CN119048446A_ABST
Patent Text Reader

Abstract

The invention discloses a pathological image imaging quality evaluation method, which belongs to the field of medical image quality control, and comprises the following steps of: constructing a pathological image defocus data set by utilizing acquired pathological images shot by an optical microscope on a plurality of defocus planes; preprocessing the out-of-focus pathological image data set; establishing a pathological image quality evaluation network based on frequency domain features; performing optimization training by using a combined loss function to obtain an optimized pathological image quality evaluation deep neural network model based on frequency domain features; and outputting a pathological image quality grade by using the optimized pathological image quality evaluation network. According to the method, the end-to-end deep learning model is adopted, the frequency convolution module is provided, the frequency characteristics of the pathological image are extracted, the pathological image quality evaluation performance is improved, the robustness is good, and the adaptability is high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical image quality control, and in particular to a method for evaluating the imaging quality of pathological images. Background Art

[0002] Optical microscopy imaging provides visualization of tissues at the cellular level and can intuitively display information such as tissue structure and physiological status. Therefore, it has gradually become an important means for biomedical research, clinical diagnosis, and prognosis of diseases such as cancer. Although optical microscopy has made significant contributions to pathology, a basic prerequisite is to capture high-quality focused images. However, since the magnification of cell pathology imaging microscopes is usually very high, the depth of field of the microscope is usually very small (less than 1 micron). Out-of-focus blur is the main reason for the low quality of microscope imaging. This blur may cause doctors to make errors in their judgment of lesions, affecting the accuracy of diagnosis and the effectiveness of treatment. In addition, computer-aided diagnosis (CAD) systems used for microscopy imaging are also affected by low-quality samples. Therefore, the evaluation of the imaging quality of pathological images is of great significance to biomedical research, clinical diagnosis, and prognosis of diseases such as cancer. Summary of the invention

[0003] The main purpose of the present invention is to provide a pathological image imaging quality assessment method combining frequency domain features and feature distillation, so as to avoid the problem that the image quality affects the auxiliary diagnosis results in the computer-aided diagnosis system of medical pathological images.

[0004] A method for evaluating the imaging quality of a pathological image comprises the following steps:

[0005] S1, build a pathological image dataset;

[0006] S2, preprocessing the pathological image dataset;

[0007] S3. Construct a deep neural network model for pathological image quality assessment based on frequency domain features;

[0008] S4, training and optimizing the constructed deep neural network model through the preprocessed pathological image dataset;

[0009] S5. Input the pathological image to be judged into the optimized pathological image imaging quality assessment deep neural network model to obtain the imaging quality assessment level of the pathological image.

[0010] The method for constructing the pathological image dataset in S1 is as follows:

[0011] Obtain sample pathological images taken by an optical microscope at multiple defocus planes, obtain a focus stack about the sample, and select clear pathological images and blurred pathological images as training sample pairs. All training samples constitute a pathological image dataset.

[0012] The training sample pairs are randomly sampled in pairs. During sampling, a clear pathological image and another image in the same sample focus stack are selected to form a training sample pair.

[0013] The screening method of the clear pathological image and the fuzzy pathological image is as follows:

[0014] The image with a defocus distance of 0 was selected as a clear pathological image, and the image with a defocus distance greater than 4 μm was selected as a blurred pathological image.

[0015] The steps of preprocessing the pathological image dataset in S2 are as follows:

[0016] S21, dividing the pathological image dataset into a training set and a test set according to a ratio;

[0017] S22, cutting the training sample pairs in the training set into regions of size 512×512 with a step size of 256;

[0018] S23, randomly select a 256×256 area from each area as a training sample;

[0019] S24. All training samples construct a training set, and the obtained training set is:

[0020] {data(x c,h,w ,y c,h,w ),x∈[1,X],[1,Y],c=3,h=256,w=256}

[0021] Among them, data(x c,h,w ,y c,h,w ) is the input fuzzy pathology image x and clear pathology image y, c represents the number of channels of the image, h represents the height of the image, w represents the width of the image, X represents the number of fuzzy pathology images, and Y represents the number of clear pathology images.

[0022] The deep neural network model training and optimization method in S4 is as follows:

[0023] The training sample pairs of the training set are input into the deep neural network model, and the backbone network of the deep neural network model is used to train the pathological image I 1 and pathological images I 2 For feature extraction, the distillation network of the deep neural network model is based on one of the pathological images I 1As input, feature extraction is performed, and after fusing the intermediate layer features extracted by the backbone network, another defocus distance image I is generated. 1 ′, in order to simulate the generation process of defocused images;

[0024] Then by creating another input image I 2 And generate image I 1 ′ loss function is used to optimize the deep neural network model parameters.

[0025] The backbone network of the deep neural network model includes four layers of ResBlock. Assume that the feature map size extracted by the third layer of ResBlock in the backbone network is R C,H,W , where C represents the number of channels, H and W represent the height and width of the feature map, and the image I 1 and image I 2 The third layer feature maps of the 2D image are spliced ​​together to obtain a feature map size of R. C*2,H,W , the defocus kernel of the paired images in the same sample is estimated through two layers of 1*1 convolution, the mutual defocus distance features between the two images are extracted, and the feature map is passed to the distillation network. The distillation network takes one of the images as input and first extracts features through the multi-layer encoder of the distillation network. After each layer of encoder, the height and width of the feature map are halved through a maximum pooling. The size of the feature map obtained after the last layer of encoder is also R C*2,H,W , and then the feature map and the features extracted by the backbone network Figure 1 And input to the adaptive instance normalization AdaIn module for feature fusion, and use the mutual defocus distance feature to adjust the image I 1 The characteristics of image I 2 The statistical properties of are calculated as follows:

[0026]

[0027] Among them, α represents the input image I 1 The feature map of nean represents the average value of α, α std represents the standard deviation of α, β mean represents the average value of the input mutual defocus distance feature β, β std Represents the standard deviation of the input mutual defocus distance feature β.

[0028] The complementary information of the features extracted by the distillation network encoder and the features obtained from the backbone network ResNet is used to learn the implicit defocus distance information;

[0029] In the decoder stage, the feature map is gradually restored to the original image resolution and finally added to the original image to obtain the final output image I 1 ′.

[0030] The method to optimize the parameters of the deep neural network model is as follows:

[0031] The combined loss function L is constructed by the probability distribution loss function and the distillation loss function total , where the probability distribution loss function uses the KL divergence loss function L dis , the distillation loss function includes the pixel loss function L pixel And the Fourier loss function L FFT , the specific formula is as follows:

[0032] L total =αL dis +βL pixel +γL FFT

[0033] Among them, α, β, and γ are weight coefficients respectively.

[0034] The KL divergence loss function L dis The calculation method is as follows:

[0035] By calculating the difference between the true probability distribution and the predicted probability distribution and summing the differences in all dimensions, we get the KL divergence loss function L dis , the specific calculation formula is as follows:

[0036]

[0037] Among them, d p is the defocus distance, To predict the defocus distance value, M is the vector dimension, p(·) is the probability value, and the KL divergence loss function is used to make the probability distributions of the two closer;

[0038] The calculation method of the distillation loss function is as follows:

[0039] Suppose the output of image A after distillation network is f(x i ), the expected value is the paired image y i The difference at each pixel, and the absolute value of the difference of all pixels is summed up, and then divided by the total number of pixels to obtain the pixel loss function L pixel , the specific calculation formula is as follows:

[0040]

[0041] Where n represents the total number of pixels;

[0042] f(x i ) and y i After fast Fourier transform to the frequency domain, calculate f(x i ) and yi The difference of each pixel in the frequency domain is calculated, and the absolute value of the difference of all pixels is taken and summed, and then divided by the total number of pixels to obtain the Fourier loss. The loss calculation formula in the frequency domain is as follows:

[0043]

[0044] Wherein, F(·) represents fast Fourier transform.

[0045] In S5, the pathological image to be judged is input into the optimized pathological image imaging quality assessment deep neural network model to obtain the defocus distance, and the image quality assessment grade is obtained through the label distribution function.

[0046] The label distribution function is established as follows:

[0047] The scanning range is evenly divided into N intervals with the preset scanning step length as the interval, and N+1 anchor points are obtained. For any defocus distance, the following label probability distribution is obtained:

[0048]

[0049] Among them, d p is the defocus distance, φ(·) is the Euclidean distance metric, p(·) is the probability value, and M is the dimension of the transformed vector. Each dimension of the transformed vector represents the probability of the true label at that point. The closer the true label is to this anchor point, the greater the probability value is, and vice versa. The sum of all probabilities of the vector is 1.

[0050] After the input pathological image is labeled and distributed through the model, each dimension of the inferred probability distribution vector is multiplied by the corresponding preset anchor point value and summed up to convert it into the defocus distance value inferred by the model.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] By utilizing pathological images captured by an optical microscope at multiple defocus planes and screening out clear and blurred image pairs, a dataset specifically for pathological image deblurring was constructed, ensuring the pertinence and practicality of the dataset and providing a high-quality data foundation for subsequent model training.

[0053] Combined with frequency domain features, a deep neural network model for pathological image imaging quality assessment based on frequency domain features was established. Frequency domain features can reflect the essential information of the image more deeply. Therefore, this method can more accurately identify and process the imaging quality grading problems of pathological images and improve the effect of pathological image imaging quality assessment.

[0054] The Adam algorithm is used to optimize the training process and construct a combined loss function, so that the model can continuously optimize and improve the performance of pathological image quality assessment during the training process. This method can effectively improve the accuracy and stability of the model.

[0055] The present invention adopts an end-to-end deep learning model to extract the frequency characteristics of the image, improve the determination of image imaging quality, has strong robustness and good adaptability, and is helpful for the subsequent analysis and diagnosis of pathological images. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a method flow chart of an embodiment of the present invention;

[0057] Figure 2 is a fuzzy pathological image according to an embodiment of the present invention;

[0058] Figure 3 is a clear pathological image of an embodiment of the present invention;

[0059] Figure 4 It is a comparison of test result indicators of the embodiments of the present invention;

[0060] Figure 5 is a schematic diagram of the structure of a deep neural network model according to an embodiment of the present invention;

[0061] Figure 6 It is a schematic diagram of the Conditional UNet network structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0062] A specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation.

[0063] like Figures 1 to 6 As shown, a method for evaluating the imaging quality of a pathological image provided by an embodiment of the present invention comprises the following steps:

[0064] S1, build a pathological image dataset;

[0065] S2, preprocessing the pathological image dataset;

[0066] S3. Construct a deep neural network model for pathological image quality assessment based on frequency domain features;

[0067] S4, training and optimizing the constructed deep neural network model through the preprocessed pathological image dataset;

[0068] S5. Input the pathological image to be judged into the optimized pathological image imaging quality assessment deep neural network model to obtain the imaging quality assessment level of the pathological image.

[0069] The method for constructing the pathological image dataset in S1 is as follows:

[0070] The following operations are performed on multiple samples: pathological images of the samples taken by an optical microscope at multiple defocus planes are obtained to obtain a focus stack of the sample, from which clear pathological images and blurred pathological images are screened out as training sample pairs, and all training samples constitute a pathological image dataset.

[0071] The training sample pairs are randomly sampled in pairs. When sampling, a clear pathology image and another image in the same sample focus stack are selected to form a training sample pair. When paired sampling, a clear pathology image and a blurred pathology image are randomly selected, and the clear pathology image and the blurred pathology image are not fixedly selected as the training sample pair. When this pair of samples is simultaneously input into the deep neural network model for training, the problems of model overfitting and poor generalization ability can be avoided.

[0072] Among them, the screening method of clear pathological images and fuzzy pathological images is as follows:

[0073] The image with a defocus distance of 0 was selected as a clear pathological image, and the image with a defocus distance greater than 4 microns was selected as a blurred pathological image. The blurred pathological image was moved from a defocus distance of -28 microns to a defocus distance of +28 microns in steps of 4 microns to classify the blurred pathological images of different degrees;

[0074] Then perform the following preprocessing:

[0075] S21, dividing the pathological image dataset into a training set and a test set according to a ratio;

[0076] S22, cutting the training sample pairs in the training set into regions of size 512×512 with a step size of 256;

[0077] S23, randomly select a 256×256 area from each area as a training sample;

[0078] S24. All training samples construct a training set, and the obtained training set is:

[0079] {data(x c,h,w ,y c,h,w ),x∈[1,X],[1,Y],c=3,h=256,w=256}

[0080] Among them, data(x c,h,w ,y c,h,w ) is the input fuzzy pathology image x and clear pathology image y, c represents the number of channels of the image, h represents the height of the image, w represents the width of the image, X represents the number of fuzzy pathology images, and Y represents the number of clear pathology images.

[0081] The division ratio of this embodiment is 4:1;

[0082] The collection of the pathological image data set of this embodiment is specifically as follows:

[0083] A Nikon Eclipse motorized microscope with a 0.75NA, 20X objective was used. The training samples were 35 research-grade human pathology slides stained with hematoxylin and eosin (Omano OMSK-HP50). The images were acquired with a 5-megapixel color camera (Pointgrey BFS-U3-5155C-C) with a pixel size of 3.45 μm. Three different illumination conditions were tested during the autofocus process: (1) incoherent Kolner illumination condition, (2) dual-plane wave partially coherent illumination, and (3) single-plane wave partially coherent illumination. During the acquisition process, z-stacks were acquired by moving the sample to 41 different defocus positions (from -10 μm to +10 μm with a step size of 0.5 μm).

[0084] Each z-stack image of size 2448×2048 was segmented into regions of size 224×224 after 4-to-1 downsampling, and finally 128699 regions were generated.

[0085] The test set is divided into two groups. The first group includes 697 images processed with the same staining scheme as the training set, referred to as "same scheme". 1312 images are processed with different staining schemes at different locations, referred to as "different scheme". For all cases of "same scheme" and "different scheme", each case contains 41 images taken at different focal lengths.

[0086] In S3, the deep neural network model for pathological image quality assessment based on frequency domain features includes a backbone network and a distillation network. The backbone network is retained during both training and testing, and the other is a distillation network that only works during training.

[0087] The deep neural network model training method in S4 is as follows:

[0088] The training sample pairs of the training set are input into the deep neural network model, and the backbone network (ResNet) of the deep neural network model is used to train the pathological image I 1 and pathological images I 2 For feature extraction, the distillation network (Conditional UNet) uses one of the pathological images I 1 As input, feature extraction is performed, and after fusing the intermediate layer features extracted by the backbone network, another defocus distance image I is generated. 1 ′, in order to simulate the generation process of defocused images;

[0089] Specifically, the backbone network ResNet includes four layers of Reslock. Assume that the feature map size extracted by the third layer ResBlock of the backbone network is R C,H,W , where C represents the number of channels, H and W represent the height and width of the feature map, and the image I 1 and image I 2 The third layer feature maps of the 2D image are spliced ​​together to obtain a feature map size of R. C*2,H,W , after two layers of 1*1 convolution, the defocus kernel of the paired images in the same sample is estimated, the mutual defocus distance feature between the two images is extracted, and the feature map is passed to the distillation network (Conditional UNet), which only accepts one of the images I 1 or I 2 As input (in this embodiment, I 1 For example), firstly, the encoder Embedding is used to extract features. After each layer of encoder, the feature map is halved in height and width through a maximum pooling. The feature map size obtained after the last layer of encoder is also R C*2,H,W , after the last encoder layer, it will not go through the maximum pooling again, so the sizes of the two feature maps are aligned. Then, the features extracted by the Conditional UNet encoder and the third layer of the backbone network ResNet are input into the decoder layer of the Conditional UNet for feature fusion. By using the complementary information of the features extracted by the Conditional UNet encoder and the features obtained from the backbone network ResNet, the implicit defocus distance information is learned.

[0090] The specific fusion method is as follows:

[0091] The feature map and the features extracted by the backbone network Figure 1 And input to the adaptive instance normalization AdaIn module for feature fusion, using the mutual defocus distance feature to adjust I 1 characteristics, making it have I 2 The statistical properties of are calculated as follows:

[0092]

[0093] Among them, α represents the input image I 1 The feature map of mean represents the average value of α, α std represents the standard deviation of α, β mean represents the average value of the input mutual defocus distance feature β, β std Represents the standard deviation of the input mutual defocus distance feature β.

[0094] In the decoder stage, the feature map is gradually restored to the original image resolution size and finally combined with the original image I 1 Add together to get the final output image I 1 ′, through another image I 2 And generate image I 1 ′’s pixel loss and the Fourier loss converted to the frequency domain are used to optimize the parameters of the deep neural network model, as follows:

[0095] Construct a combined loss function. The loss function is divided into two parts, probability distribution loss and distillation loss. The probability distribution loss uses the KL divergence loss function, which calculates the difference between the true probability distribution and the predicted probability distribution and sums the differences in all dimensions to obtain the KL divergence loss L. dis , the specific calculation formula is as follows:

[0096]

[0097] Among them, d p is the defocus distance, To predict the defocus distance value, M is the vector dimension, p(·) is the probability value, and the KL divergence loss is used to make the probability distributions of the two closer;

[0098] The distillation loss is divided into two parts. One is the pixel loss. Assume that the output of image A after passing through the conditional UNet network is f(x i ), the expected value is the paired image y i The difference at each pixel, and the absolute value of the difference of all pixels is summed up, and then divided by the total number of pixels to get the pixel loss L pixel , the specific calculation formula is as follows:

[0099]

[0100] Among them, n represents the total number of pixels, and the other part is the Fourier loss. We will i ) and y i After fast Fourier transform to frequency domain, the difference of each pixel in the frequency domain is calculated, and the absolute value of the difference of all pixels is taken and summed, and then divided by the total number of pixels to obtain the Fourier loss. The loss calculation formula in the frequency domain is as follows:

[0101]

[0102] Wherein, F(·) represents fast Fourier transform.

[0103] During the training process, the parameters of the backbone network are optimized by supervising the loss of the synthetic image and the real contrast image in the spatial domain and frequency domain. The total loss function of pathological image focal length estimation includes probability distribution loss, pixel loss and Fourier loss. The total loss function calculation formula is as follows:

[0104] L total =αL dis +βL pixel +γL FFT

[0105] Among them, α, β, and γ are weight coefficients respectively;

[0106] The network model code is implemented based on Pytorch1.12, and 1 NVIDIA GeForce RTX 3090 GPU is used for model training. First, 30 rounds of pre-training are performed on the Incoherent dataset with a batch size of 8 and an initial learning rate of 1e-5. Then, 40 rounds of training are performed using the own dataset. For model optimization, the Adam optimizer is used. If the loss does not decrease for 10 consecutive rounds during training, it will be reduced to 0.1 times the original value. Using the randomly rotated data augmentation strategy, the input image is cropped to a size of 224*224 and sent to the model training.

[0107] In step S5, the image of the test set is input into the deep neural network model optimized in step S4 to obtain the defocus distance, and the image quality assessment level is obtained through the label distribution function;

[0108] The label distribution function is established as follows:

[0109] The scanning range is evenly divided into N intervals with the preset scanning step length as the interval, and N+1 anchor points are obtained. For any defocus distance, the following label probability distribution is obtained:

[0110]

[0111] Among them, d p is the defocus distance, φ(·) is the Euclidean distance metric, p(·) is the probability value, and M is the converted vector dimension. Each dimension of the converted vector represents the probability of the true label at that point. The closer the true label is to the anchor point, the greater the probability value, and vice versa. The sum of all vector probabilities is 1. After the input image is calculated through the model to obtain this probability distribution, each dimension of the inferred probability distribution vector is multiplied by the corresponding preset anchor point value and summed up to convert it into the defocus distance value inferred by the model.

[0112] The present invention purifies the features learned by the backbone network through feature distillation, and passes the defocus distance features learned by the backbone network to the distillation network. After the distillation network fuses the image features and the defocus distance features, it converts the input image into a paired image and calculates their pixel losses in the spatial domain and the frequency domain, providing more accurate and detailed defocus distance features, and the average absolute error is reduced from 6.86 microns to 6.41 microns; in addition, through model pre-training, the model's feature extraction capability for sparsely distributed data sets is enhanced, and the performance of the model is significantly improved, and the average absolute error of defocus distance determination is reduced from 6.41 microns to 1.45 microns.

[0113] Compared with the SOTA model KDAF network, this method has a smaller mean absolute error and better performance.

[0114] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit and essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0115] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for evaluating the imaging quality of pathological images. It is characterized in that The steps include: S1, build a pathological image dataset; S2, preprocessing the pathological image dataset; S3. Construct a deep neural network model for pathological image quality assessment based on frequency domain features; S4, training and optimizing the constructed deep neural network model through the preprocessed pathological image dataset; S5. Input the pathological image to be judged into the optimized pathological image imaging quality assessment deep neural network model to obtain the imaging quality assessment level of the pathological image.

2. A pathological image quality assessment method as claimed in claim 1, It is characterized in that The method for constructing the pathological image dataset in S1 is as follows: Obtain sample pathological images taken by an optical microscope at multiple defocus planes, obtain a focus stack about the sample, and select clear pathological images and blurred pathological images as training sample pairs. All training samples constitute a pathological image dataset. The training sample pairs are randomly sampled in pairs. During sampling, a clear pathological image and another image in the same sample focus stack are selected to form a training sample pair.

3. A method for evaluating the imaging quality of a pathological image as claimed in claim 2, It is characterized in that The screening method of the clear pathological image and the fuzzy pathological image is as follows: The image with a defocus distance of 0 was selected as a clear pathological image, and the image with a defocus distance greater than 4 μm was selected as a blurred pathological image.

4. A method for evaluating the imaging quality of a pathological image as claimed in claim 2, It is characterized in that The steps of preprocessing the pathological image dataset in S2 are as follows: S21, dividing the pathological image dataset into a training set and a test set according to a ratio; S22, cutting the training sample pairs in the training set into regions of size 512×512 with a step size of 256; S23, randomly select a 256×256 area from each area as a training sample; S24. All training samples construct a training set, and the obtained training set is: {data(x c,h,w ,y c,h,w ),x∈[1,X],[1,T],c=3,h=256,w=256} Among them, data(x c,h,w ,y c,h,w ) is the input fuzzy pathology image x and clear pathology image y, c represents the number of channels of the image, h represents the height of the image, w represents the width of the image, X represents the number of fuzzy pathology images, and Y represents the number of clear pathology images.

5. A method for evaluating the imaging quality of a pathological image as claimed in claim 1, It is characterized in that The deep neural network model training and optimization method in S4 is as follows: The training sample pairs of the training set are input into the deep neural network model, and the backbone network of the deep neural network model is used to train the pathological image I 1 and pathological images I 2 For feature extraction, the distillation network of the deep neural network model is based on one of the pathological images I 1 As input, feature extraction is performed, and after fusing the intermediate layer features extracted by the backbone network, an image I′ of another defocus distance is generated. 1 , in order to simulate the generation process of defocused images; Then by creating another input image I 2 and generate image I′ 1 The loss function is used to optimize the deep neural network model parameters.

6. A method for evaluating the imaging quality of a pathological image as claimed in claim 5, It is characterized in that The backbone network of the deep neural network model includes four layers of ResBlock. Assume that the feature map size extracted by the third layer of ResBlock in the backbone network is R C ,H,W , where C represents the number of channels, H and W represent the height and width of the feature map, and the image I 1 and image I 2 The third layer feature maps of the 2D image are spliced ​​together to obtain a feature map size of R. C*2,H,W , the defocus kernel of the paired images in the same sample is estimated through two layers of 1*1 convolution, the mutual defocus distance features between the two images are extracted, and the feature map is passed to the distillation network. The distillation network takes one of the images as input and first extracts features through the multi-layer encoder of the distillation network. After each layer of encoder, the height and width of the feature map are halved through a maximum pooling. The size of the feature map obtained after the last layer of encoder is also R C *2,H,W Then, the feature map and the feature map extracted by the backbone network are input into the adaptive instance normalization AdaIn module for feature fusion, and the mutual defocus distance feature is used to adjust the image I 1 The characteristics of image I 2 The statistical properties of are calculated as follows: Among them, α represents the input image I 1 The feature map of mean represents the average value of α, α std represents the standard deviation of α, β mean represents the average value of the input mutual defocus distance feature β, β Std Represents the standard deviation of the input mutual defocus distance feature β. The complementary information of the features extracted by the distillation network encoder and the features obtained from the backbone network ResNet is used to learn the implicit defocus distance information; In the decoder stage, the feature map is gradually restored to the original image resolution and finally added to the original image to obtain the final output image I 1 ′.

7. A method for evaluating the imaging quality of a pathological image as claimed in claim 5, It is characterized in that The method to optimize the parameters of the deep neural network model is as follows: The combined loss function L is constructed by the probability distribution loss function and the distillation loss function total , where the probability distribution loss function uses the KL divergence loss function L diS , the distillation loss function includes the pixel loss function L pixel And the Fourier loss function L FFT , the specific formula is as follows: L total =αL dis +βL pixel +γL FFT Among them, α, β, and γ are weight coefficients respectively.

8. A method for evaluating the imaging quality of a pathological image as claimed in claim 7, It is characterized in that The KL divergence loss function L dis The calculation method is as follows: By calculating the difference between the true probability distribution and the predicted probability distribution and summing the differences in all dimensions, we get the KL divergence loss function L dis , the specific calculation formula is as follows: Among them, d p is the defocus distance, To predict the defocus distance value, M is the vector dimension, p(·) is the probability value, and the KL divergence loss function is used to make the probability distributions of the two closer; The calculation method of the distillation loss function is as follows: Suppose the output of image A after distillation network is f(x i ), the expected value is the paired image y i The difference at each pixel, and the absolute value of the difference of all pixels is summed up, and then divided by the total number of pixels to obtain the pixel loss function L pixel , the specific calculation formula is as follows: Where n represents the total number of pixels; f(x i ) and y i After fast Fourier transform to frequency domain, calculate f(x i ) and y i The difference of each pixel in the frequency domain is calculated, and the absolute value of the difference of all pixels is taken and summed, and then divided by the total number of pixels to obtain the Fourier loss. The loss calculation formula in the frequency domain is as follows: Wherein, F(·) represents fast Fourier transform.

9. A pathological image quality assessment method as claimed in claim 1, It is characterized in that In S5, the pathological image to be judged is input into the optimized pathological image imaging quality assessment deep neural network model to obtain the defocus distance, and the image quality assessment grade is obtained through the label distribution function.

10. A method for evaluating the imaging quality of a pathological image as claimed in claim 9, It is characterized in that The label distribution function is established as follows: The scanning range is evenly divided into N intervals with the preset scanning step length as the interval, and N+1 anchor points are obtained. For any defocus distance, the following label probability distribution is obtained: Among them, d p is the defocus distance, φ(·) is the Euclidean distance metric, p(·) is the probability value, and M is the dimension of the transformed vector. Each dimension of the transformed vector represents the probability of the true label at that point. The closer the true label is to this anchor point, the greater the probability value is, and vice versa. The sum of all probabilities of the vector is 1. After the input pathological image is labeled and distributed through the model, each dimension of the inferred probability distribution vector is multiplied by the corresponding preset anchor point value and summed up to convert it into the defocus distance value inferred by the model.

Citation Information

Patent Citations

  • Microscopic image blurring kernel extraction and out-of-focus restoration method based on deep convolutional network

    CN116051411A

  • Ultra-high-definition video fuzzy quality evaluation method based on deep learning

    CN116258669A

  • Microscope image definition evaluation method based on improved MobileViT network

    CN116612092A

  • Pathological image deblurring method based on frequency domain characteristics

    CN118261823A

  • Image focusing evaluation algorithm based on convolutional neural network model

    CN118365622A

Cited By

  • Embryo image automatic focusing method and device based on paired comparison learning, and medium

    CN119722658A

  • Embryo image automatic focusing method, device and medium based on paired comparison learning

    CN119722658B

  • Pathological image preprocessing and feature extraction method

    CN119964154A

  • A method for pathological image preprocessing and feature extraction

    CN119964154B

  • Oral medical image precision optimization method based on multi-modal data

    CN120823471A