An adaptive pathological detection method and system for sparse lesions

Abstract

The application discloses a kind of self-adapting pathological detection methods and systems for sparse lesion, belong to medical image processing and artificial intelligence technical field.The method includes: obtaining the pathological image to be detected and pre-processing;Enhancement processing is carried out using pre-trained frequency domain-space domain joint enhancement model, wherein the frequency domain submodule is enhanced by Haar wavelet decomposition and point spread function estimation, and the spatial domain submodule is locally contrast enhanced for G, B channel, and adaptively fused;The enhanced image is input into the pre-trained detection model, which is based on the bidirectional multi-level local feature aggregation Transformer network, extracts multi-scale features and generates adversarial masks through local discriminator to modulate self-attention weight, and finally outputs lesion location and class.The application effectively solves the problems of sample scarcity, image blur and cross-scale feature fusion difficulty in sparse lesion detection, significantly improves the detection accuracy and robustness.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and artificial intelligence technology, specifically relating to an adaptive pathological detection method and system for sparse lesions. Background Technology

[0002] Metabolic steatohepatitis (MASH) is a prevalent chronic liver disease worldwide, and hepatocellular ballooning degeneration (HB) is a key indicator for disease grading in its pathological diagnosis. However, HB lesions exhibit sparseness (positive samples usually account for less than 1%) and morphological polymorphism (cell swelling, vacuolation, irregular shape) in liver biopsy sections. Traditional manual microscopic interpretation suffers from significant problems such as strong subjectivity and poor consistency (Kappa value is often below 0.5).

[0003] Deep learning-based detection methods are limited by: ① Extreme scarcity of positive samples, conventional enhancement cannot generate new biologically accurate samples; ② HB recognition requires simultaneous reliance on local cytoplasmic texture and global hepatic cord structure, and existing models lack multi-scale fusion capabilities; ③ Local blurring of pathological images (due to scanning focus drift) leads to missed detections.

[0004] Therefore, a detection method that can simultaneously address sparse data, fuzzy interference, and cross-scale feature aggregation is needed. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive pathological detection method and system for sparse lesions. Through an integrated generation-enhancement-detection framework, based on an improved generative adversarial network and a dual-path feature aggregation network, it achieves lesion detection in pathological images, improving the recall and robustness of sparse, blurred, and morphologically variable pathological lesions (such as hepatic ballooning cells). This invention is particularly applicable to the automatic identification and localization of sparse and morphologically variable lesions such as hepatic ballooning (HB) cells in pathological images of metabolic-associated steatohepatitis (MASH).

[0006] To achieve the above objectives, the following technical solution is adopted:

[0007] In a first aspect, the present invention provides an adaptive pathological detection method for sparse lesions, comprising the following steps:

[0008] S1: Acquire the pathological image to be detected, and preprocess the pathological image to obtain the preprocessed pathological image;

[0009] S2: The pre-processed pathological image is enhanced using a pre-trained frequency domain-spatial domain joint enhancement model to obtain an enhanced pathological image; the frequency domain-spatial domain joint enhancement model includes a frequency domain sub-module, a spatial domain sub-module, and an adaptive fusion module, and the adaptive fusion module fuses the enhancement results of the frequency domain sub-module and the spatial domain sub-module through learnable weights;

[0010] S3: Input the enhanced pathological image into the pre-trained detection model for lesion detection. The detection model is constructed based on a bidirectional multi-level local feature aggregation Transformer network, uses the Swing Transformer to extract multi-scale features, and generates an adversarial mask through a local discriminator to modulate the self-attention weights, and finally outputs the location and category information of the lesion.

[0011] Furthermore, in step S1, the preprocessing includes: dividing the pathological image to be detected into multiple pathological image blocks of preset size, and performing pixel value normalization processing on each pathological image block to obtain a normalized pathological image.

[0012] Furthermore, in step S2:

[0013] The frequency domain submodule is used to perform Haar wavelet decomposition on the preprocessed pathological image to obtain low-frequency and high-frequency components. It uses a pre-trained point spread function estimation operator to estimate the fuzzy kernel parameters from the high-frequency components, and performs Wiener filtering enhancement on the high-frequency components. Then, it performs inverse wavelet transform on the enhanced high-frequency components and the low-frequency components to obtain the frequency domain enhanced image.

[0014] The spatial domain submodule is used to calculate the local mean and local variance of each pixel's neighborhood for the G and B channels of the preprocessed pathological image, and to enhance the pixel values ​​using a nonlinear mapping that includes a hyperbolic tangent function and a bias term, to obtain a spatial domain enhanced image.

[0015] The adaptive fusion module is used to perform weighted fusion of the frequency domain enhanced image and the spatial domain enhanced image using learnable fusion weights to obtain the enhanced pathological image.

[0016] Furthermore, in step S3, the detection model includes a bidirectional multi-level local feature aggregation module; the bidirectional multi-level local feature aggregation module uses two bidirectional paths, one from top to bottom and the other from bottom to top, to fuse multi-scale feature maps. At each fusion node, feature maps from different directions are aligned by bilinear interpolation and then spliced ​​together, and an adversarial mask is generated through the local discriminator.

[0017] The adversarial mask is obtained by applying a Sigmoid mapping to the difference between the fused class score and the non-fused class score output by the local discriminator; the adversarial mask is then bilinearly interpolated, added to the local window self-attention score, and multiplied by a learnable scaling factor to modulate the attention weights.

[0018] Furthermore, the local discriminator is a lightweight convolutional neural network, whose inputs are a fused feature map and an upsampled feature map, and whose outputs are a fused class score and a non-fused class score;

[0019] The adversarial mask is represented as:

[0020]

[0021] in, and The scores for classifying the fused feature map and the upsampled feature map as fused are respectively. For the Sigmoid function, For adversarial masking.

[0022] Furthermore, the detection model employs the local window self-attention mechanism of the Swin Transformer, and modulates the attention weights of the adversarial mask with additive bias. Specifically, the adversarial mask is bilinearly interpolated to the same spatial size as the local window, and the corresponding mask value for each query position is broadcast to the bias of all key positions of that query pair. The modulated attention weight matrix is ​​expressed as follows:

[0023]

[0024] in, and These are the query matrix and key matrix within the local window, respectively. For the dimensions of query and key, The adversarial mask, after bilinear interpolation, corresponds to the mask value at the i-th query position within the local window. A learnable scaling factor. This indicates the outer product expansion operation. Let M be a row vector of all 1s, and M be the side length of each local window; This is the attention weight matrix after additive bias modulation within the local window.

[0025] Furthermore, the frequency domain-spatial domain joint enhancement model and the detection model are trained using a phased strategy, which includes:

[0026] Synthetic lesion image samples were generated using a sparse lesion augmentation model based on generative adversarial networks and bidirectional cyclic consistency constraints to expand the training sample set;

[0027] Fix the point spread function estimation operator in the frequency domain-spatial domain joint enhancement model, freeze the sparse lesion augmentation model, and pre-train the detection model using the original samples and the synthetic lesion image samples;

[0028] Unfreeze the learnable fusion weights in the frequency domain-spatial domain joint enhancement model and some layers of the detection model, and perform end-to-end fine-tuning.

[0029] Furthermore, the sparse lesion augmentation model includes a generator, an encoder, and a discriminator;

[0030] The process of generating synthetic lesion images by the sparse lesion augmentation model includes: generating a corresponding synthetic background image from the real lesion image through image restoration; mapping the real lesion image to the latent space and sampling the latent variables through an encoder; and then using the synthetic background image and latent variables as inputs to synthesize new lesion image patches by a generator. The loss functions used to train the sparse lesion augmentation model include adversarial loss, bidirectional cyclic consistency loss, and KL divergence loss.

[0031] Furthermore, the phased strategy also includes: compressing the end-to-end fine-tuned model into a lightweight student model through knowledge distillation;

[0032] The knowledge distillation uses an end-to-end fine-tuned model as the teacher model and a lighter-weight SwinTransformer as the student model. The distillation loss function covers both the classification and regression branches, including the hard loss of the student model on the true labels, the KL divergence loss of the classification branch, and the Euclidean distance loss between the bounding box parameters of the regression branch.

[0033] Secondly, the present invention also provides an adaptive pathological detection system for sparse lesions, comprising:

[0034] The preprocessing module is used to acquire the pathological image to be detected, preprocess the pathological image, and output the preprocessed pathological image.

[0035] The frequency domain-spatial domain joint enhancement module is used to enhance the preprocessed pathological image using a pre-trained enhancement model and output the enhanced pathological image. The frequency domain-spatial domain joint enhancement module includes a frequency domain sub-module, a spatial domain sub-module, and an adaptive fusion module. The adaptive fusion module fuses the enhancement results of the frequency domain sub-module and the spatial domain sub-module through learnable weights.

[0036] The lesion detection module is used to input the enhanced pathological image into a pre-trained detection model for lesion detection and output the location and category information of the lesion. The detection model is constructed based on a bidirectional multi-level local feature aggregation Transformer network, including a Swing Transformer backbone network, a bidirectional multi-level local feature aggregation module with a local discriminator, and a detection head.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] (1) By using the sparse lesion augmentation module based on bidirectional cyclic consistency constraints, only a small number of real seed samples are needed to generate high-fidelity synthetic lesion images, which effectively solves the problem of extremely scarce positive samples in sparse lesion detection and significantly improves the detection recall rate.

[0039] (2) By using the frequency domain-spatial domain joint enhancement module, high-frequency details and local contrast are enhanced while preserving the original structural information, which improves the robustness of the blurred image caused by the scanning focus drift and reduces the false negative rate.

[0040] (3) Through the bidirectional multi-level local feature aggregation module and the local adversarial attention mechanism, the effective fusion of cross-scale features was achieved, and the model was guided to focus on the discriminative region, thereby improving the detection accuracy of atypical lesions.

[0041] (4) A lightweight student model is obtained through knowledge distillation, which greatly improves the reasoning speed while maintaining high detection accuracy and is suitable for real-time processing scenarios of pathological auxiliary screening.

[0042] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0043] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of the invention. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0044] Figure 1 This is a flowchart of the steps of an adaptive pathological detection method for sparse lesions according to an embodiment of the present invention;

[0045] Figure 2 It is an overall process framework diagram;

[0046] Figure 3 This is a schematic diagram of a bidirectional multi-level local feature aggregation network structure according to an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the sparse lesion augmentation model according to an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of a module of an adaptive pathological detection system for sparse lesions according to an embodiment of the present invention;

[0049] Figure 6 This is an example diagram of the simulation results of the present invention in the task of detecting hepatic ballooning degeneration cells. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0052] Example 1:

[0053] Figure 1 This is a flowchart of the steps of an adaptive pathological detection method for sparse lesions according to Embodiment 1 of the present invention; Figure 2 This is a diagram of the overall process framework. For example... Figure 1 and Figure 2 As shown, an adaptive pathological detection method for sparse lesions includes the following steps:

[0054] S1: Acquire the pathological image to be detected, and preprocess the pathological image to obtain the preprocessed pathological image;

[0055] Furthermore, in step S1, the preprocessing includes: dividing the pathological image to be detected into multiple pathological image blocks of preset sizes, and performing pixel value normalization processing on each pathological image block to obtain a normalized pathological image. Its pixel values ​​are normalized to .

[0056] S2: The pre-processed pathological image is enhanced using a pre-trained frequency domain-spatial domain joint enhancement model to obtain an enhanced pathological image; the frequency domain-spatial domain joint enhancement model includes a frequency domain sub-module, a spatial domain sub-module, and an adaptive fusion module, and the adaptive fusion module fuses the enhancement results of the frequency domain sub-module and the spatial domain sub-module through learnable weights;

[0057] Step S2 is used to achieve adaptive deblurring and feature enhancement of pathological images through a pre-trained frequency domain-spatial domain joint enhancement model.

[0058] The pre-trained frequency-space domain joint augmentation model contains two types of parameters: (1) point spread function (PSF) estimator (Pre-trained on the synthetic defocused blur dataset and then frozen, not participating in subsequent fine-tuning); (2) Fusion weights (Learning available, will be unfrozen and updated in Phase 3). The goal of this module is to enhance high-frequency details and preserve structure in the input image, rather than to completely restore the original sharp image.

[0059] S2.1 Frequency Domain Submodule:

[0060] Frequency domain submodule: used for normalizing the preprocessed pathological images. Haar wavelet decomposition is performed to obtain low-frequency and high-frequency components. A pre-trained point spread function estimation operator is used to estimate the blur kernel parameters from the high-frequency components. Wiener filtering is then applied to enhance the high-frequency components. Finally, inverse wavelet transform is performed between the enhanced high-frequency components and the low-frequency components to obtain the frequency-domain enhanced image. The details are as follows:

[0061] Normalized pathological images Haar wavelet decomposition was performed to obtain the low-frequency components. and high frequency components Design a learnable point spread function estimator. —This operator consists of 3 layers of convolution (Conv-BN-LeakyReLU), with high-frequency components as input and a blur kernel parameter matrix (a separable kernel of size 5×5) as output. A blind deblurring strategy is employed, and the training data uses synthetic defocused blur pairs (applying random Gaussian kernels to clear pathological image patches, with Gaussian kernel standard deviation...). (simulating scan focus drift) by minimizing the reconstructed image With clear images L1 loss in Haar wavelet high-frequency subbands train :

[0062]

[0063] in, These represent the three high-frequency subband extraction operators of Haar wavelet decomposition. The code is frozen after training. During inference, the estimated fuzzy kernel parameters are used to analyze the high-frequency components. Perform Wiener filtering (e.g., using Wiener filtering or an approximate variant) to enhance high-frequency details, then combine with the unprocessed low-frequency components. Perform inverse wavelet transform to obtain Low-frequency components retain the original structural information, avoiding over-processing that could lead to background distortion.

[0064] S2.2, Spatial Domain Submodule (Local Contrast Adaptive Enhancement Operator):

[0065] The spatial domain submodule is used for normalized pathological images obtained after preprocessing. The G and B channels are used to calculate the local mean and local variance of each pixel's neighborhood. A nonlinear mapping involving a hyperbolic tangent function and a bias term is then applied to enhance the pixel values, resulting in a spatially enhanced image. Specifically:

[0066] According to the training set statistics in this embodiment, the G and B channels are more sensitive to the contrast of the HB boundary (statistical method, such as calculating the mean difference in contrast between the lesion boundary and the background area under different channels). Therefore, for normalized pathological images... Calculate the local mean of the G and B channels. (Window size 7×7) and local variance Enhanced formula :

[0067]

[0068] in: Represents pixel coordinates; For G and B channels in coordinates The average value at; It is the hyperbolic tangent function; This means truncating the input value to the range [0,1].

[0069] This bias term Used to suppress areas that are too dark or too bright.

[0070] When the enhancement result causes local pixel values ​​to exceed the range, a clamp operation is used to truncate them to avoid color anomalies or noise amplification caused by numerical overflow.

[0071] S2.3, Adaptive Fusion Module:

[0072] The adaptive fusion module is used to perform weighted fusion of the frequency domain enhanced image and the spatial domain enhanced image using learnable fusion weights to obtain the enhanced pathological image. Specifically:

[0073] Enhance the image in the frequency domain Spatial domain enhanced images Weighted fusion was performed to obtain the enhanced pathological image. Enhanced pathological image. Defined as:

[0074]

[0075] in, The fusion weights are learnable, with an initial value of 0.5, and are updated during fine-tuning in stage three.

[0076] S3: Input the enhanced pathological image into the pre-trained detection model for lesion detection. The detection model is constructed based on a bidirectional multi-level local feature aggregation Transformer network, uses the Swing Transformer to extract multi-scale features, and generates an adversarial mask through a local discriminator to modulate the self-attention weights, and finally outputs the location and category information of the lesion.

[0077] Step S3 is used to construct a Transformer detection network based on bidirectional multi-level local feature aggregation for lesion detection.

[0078] Furthermore, the detection model includes a bidirectional multi-level local feature aggregation module (BMFA). The BMFA employs two bidirectional paths—top-down and bottom-up—to fuse multi-scale feature maps. At each fusion node, feature maps from different directions are aligned using bilinear interpolation and then concatenated. An adversarial mask is generated through a local discriminator. The adversarial mask is obtained by applying a sigmoid mapping to the difference between the fused class score and the non-fused class score output by the local discriminator. After bilinear interpolation, the adversarial mask is added to the local window self-attention score and multiplied by a learnable scaling factor to modulate the attention weights. Figure 3 The diagram shown is a schematic of a bidirectional multi-level local feature aggregation network structure according to an embodiment of the present invention. Details are as follows:

[0079] The backbone uses the Swin-Tiny Transformer to output multi-scale feature maps. Innovative design of a bidirectional multi-level local feature aggregation module (BMFA):

[0080] S3.1, Bidirectional Path:

[0081] Both top-down (high-level semantics → low-level details) and bottom-up (low-level texture → high-level semantics) approaches are used, with each fusion node performing bilinear interpolation to align the feature maps before stitching them together.

[0082] S3.2, Local Adversarial Attention Mechanism:

[0083] The local discriminator is a lightweight convolutional neural network. Its inputs are a fused feature map and an upsampled feature map, and its outputs are the fused class score and the non-fused class score. Details are as follows:

[0084] Set a lightweight discriminator in each fusion node (Constitutes 3 convolutional layers, with 2 output channels), its function is to perform binary classification on the input feature map: outputting a fused feature score and a non-fused feature score. Specifically, under the same discriminator, it classifies the input feature map into two categories. and After two independent forward propagations, we obtain:

[0085]

[0086] in, The discriminator pairs the fused feature maps. Similarly, the scores for the two output channels (fused score and non-fused score) are also calculated. The discriminator is configured to upsample the feature map. The output scores for the two channels (fused score and non-fused score).

[0087] Adversarial mask Defined as:

[0088]

[0089] in, For the Sigmoid function, i.e. The difference is mapped to [0,1], representing the relative advantage of the fused feature over the simple upsampled feature at each position.

[0090] S3.3 Attention Weight Calculation:

[0091] Furthermore, the detection model employs the local window self-attention mechanism of the Swin Transformer, and modulates the attention weights using an additive bias to apply the adversarial mask. Specifically, the adversarial mask is bilinearly interpolated to the same spatial size as the local window, and the corresponding mask value for each query position is broadcast to the biases of all key positions in that query pair, resulting in the modulated attention weight matrix. The specific process is as follows:

[0092] An additive bias approach is adopted, adapted to the local window attention mechanism of the Swin Transformer. Let the input feature map size be... Divided into Each window contains [number] windows. One patch (default) For each window, calculate local self-attention. :

[0093]

[0094] in, Batch size; For the query / key dimension, It equals the feature dimension of each attention head, usually the number of channels divided by the number of heads; and These are the query matrix and key matrix within the local window, respectively; This is the Softmax activation function, applied to each row of the matrix.

[0095] Will Bilinear interpolation to the same spatial size for each window ,get .

[0096] The additive bias broadcast method is as follows: for each query position (Intra-window index), and its corresponding The value is broadcast to the bias of all key positions for that query, i.e.:

[0097]

[0098] in: This is the attention weight matrix after additive bias modulation within the local window; The mask value corresponding to the i-th query position within the local window after bilinear interpolation of the adversarial mask; To expand the outer product, it is actually implemented through dimension broadcasting; It is a row vector consisting entirely of 1s, with dimensions [missing information]. ; This is a learnable scaling factor, with an initial value of 1.0.

[0099] S3.4, Alternating adversarial training strategy:

[0100] Standard minimax adversarial training was used during the network training phase (see S4):

[0101] S3.4.1: Fixed Feature Fusion Module, Update To maximize differentiation and The ability. Loss function. :

[0102]

[0103] in: , .

[0104] The discriminator will Similarly, the probability of classifying it as a fusion class... The discriminator will The probability of classifying it as a fusion class. This loss forces .

[0105] S3.4.2: Fixed Update the feature fusion module to improve The probability of being classified as a fusion type is used to guide the fusion features to more closely approximate the ideal fusion distribution. A non-saturating loss form is employed. :

[0106]

[0107] Optimizing this loss makes That is, the fusion module hopes to generate It was classified as a fusion class, implementing adversarial game theory. Steps S3.4.1 and S3.4.2 are executed alternately once each in each batch. Total detection loss. This includes GIoU loss (GIoU stands for Generalized Intersection over Union), classification loss (cross entropy), and... The weights are 1:1:0.2 respectively.

[0108] S3.5, Detection Head:

[0109] Anchor-free methods are used to directly predict the lesion center point, size, and category confidence.

[0110] For example, the convolutional layer outputs three branches: center point heatmap, bounding box size regression value, and class confidence score.

[0111] S4. Staged Model Training and Lightweight Deployment:

[0112] The joint frequency-space domain enhancement model and detection model were trained using a phased strategy. The training process consisted of four phases, with the specific phased strategy including:

[0113] S4.1, Phase One (GAN Pre-training):

[0114] Synthetic lesion images are generated using a sparse lesion augmentation model based on generative adversarial networks and bidirectional cycle consistency constraints to expand the training sample set. For example... Figure 4 The diagram shown is a structural schematic of the sparse lesion augmentation model according to an embodiment of the present invention. The sparse lesion augmentation model includes a generator, an encoder, and a discriminator. The process of generating synthetic lesion images by the sparse lesion augmentation model includes: generating a corresponding synthetic background image from a real lesion image through image inpainting; mapping the real lesion image to the latent space and sampling latent variables through the encoder; and then using the synthetic background image and latent variables as inputs, the generator synthesizes new lesion image patches. The loss functions used to train the sparse lesion augmentation model include adversarial loss, bidirectional cycle consistency loss, and KL divergence loss.

[0115] Constructing a sparse lesion augmentation module based on bidirectional cyclic consistency constraints (independent pre-training), including: constructing a module containing a generator. encoder and discriminator The adversarial generative network is independently pre-trained for subsequent data augmentation. The model was then fully trained. Afterward, the entire sparse lesion augmentation model was frozen and used only for offline augmentation training. Details are as follows:

[0116] S4.1.1 Background Image Generation:

[0117] Starting with real lesion images (no more than 15 images in this embodiment), the pathologist first marks the bounding boxes of HB cells on the images. Using the bounding boxes as the initial mask region, multi-threshold Otsu segmentation is used to finely extract the lesion foreground, and then the boundaries are corrected by experts. The Fast Moving Model (FMM) combined with the Telea algorithm is used to fill and repair the mask region, generating an approximately paired background image without lesions after repair. If the peak signal-to-noise ratio (PSNR) of the repaired image in the lesion area is lower than the preset threshold (30dB in this embodiment, which is set according to preliminary experiments and can be adjusted according to the validation set in actual applications), or if obvious artifacts (such as edge breakage or abnormal texture) appear, the sample is removed and a new selection is made. The repaired background image is highly similar to the background texture and illumination distribution of the original lesion image, and only the lesion itself is removed.

[0118] S4.1.2, Encoding Mapping:

[0119] encoder Receive real lesion images ( (Represents a real pathological image block containing lesions), outputting latent variables. mean of the distribution Sum of logarithmic variance That is, the posterior distribution .in The mean is variance is The distribution is Gaussian. A KL divergence constraint is introduced to make the posterior distribution approximate the standard normal distribution. :

[0120]

[0121] in, For the dimension of the hidden space, The first The mean and variance of the dimension; This represents the KL divergence operator.

[0122] S4.1.3, Latent variable sampling:

[0123] Hidden coding With random noise ( To generate diverse latent variables by fusing noise vectors sampled from a standard normal distribution. ,in, This indicates element-wise multiplication.

[0124] S4.1.4, Generator:

[0125] generator Background image and latent variables As input, synthesize new lesion images. .

[0126] S4.1.5, Discriminator:

[0127] Discriminator Used to differentiate between real lesions and synthetic lesions, and to combat damage. :

[0128]

[0129] in, This represents the actual distribution of lesions. The background image distribution (uniformly sampled from the background image library). , This represents the expectation operator.

[0130] S4.1.6, Bidirectional Cyclic Consistency Loss (Using perceptual loss and spatially normalizing the perceptual loss term):

[0131]

[0132] in: The pre-trained VGG-16 feature extractor (outputting the 3rd pooling layer) has a feature map size of [size missing]. In this embodiment ( (Input image height and width). Denotes the L1 norm; denominator Used to normalize the magnitude of the perceptual loss; and As a hyperparameter, this embodiment takes... ; It follows a standard normal distribution. ; Output the mean vector of the latent variable distribution for the encoder (before reparameterization).

[0133] Total loss : ,in , To balance the hyperparameters of each loss term.

[0134] S4.1.7, Training:

[0135] Alternate updates A total of 200 rounds were run, with a batch size of 2 and a learning rate of 0.0002. After training, the dataset was frozen. The parameters are used only as augmentation tools to generate synthetic lesion samples to expand the training set.

[0136] S4.2, Phase Two (Detection Network Pre-training):

[0137] The point spread function estimation operator in the frequency-spatial domain joint enhancement model is fixed, and the sparse lesion augmentation model is frozen. The detection model is then pre-trained using the original samples and the synthetic lesion image samples. Specifically:

[0138] Point spread function estimation operator in a fixed pre-trained frequency-space domain joint augmentation model The sparse lesion augmentation model is frozen, and the S3 detection network (including the feature fusion module) is trained using original real samples plus synthetic lesion image samples generated by the sparse lesion augmentation model (ratio 1:2). Alternating training (100 rounds in total), learning rate The optimizer uses AdamW with weight decay. The batch size is 8. Input images are normalized using ImageNet statistics (mean vector). standard deviation vector ).

[0139] S4.3, Phase Three (End-to-End Fine-tuning):

[0140] Unfreeze the learnable fusion weights in the frequency-spatial domain joint enhancement model and some layers of the detection model, and perform end-to-end fine-tuning. Specifically:

[0141] The fusion weights of the frequency-spatial domain joint enhancement model in the three-stage unfreezing S2 phase. The detection head of the detection network and the two Transformer encoding layers adjacent to the output branch are trained for another 20 epochs, with the learning rate reduced to [value missing]. This stage Continue training using the alternating strategy.

[0142] S4.4, Phase Four (Knowledge Distillation):

[0143] Phase four uses knowledge distillation to compress the end-to-end fine-tuned model into a lightweight student model. This knowledge distillation uses the end-to-end fine-tuned model as the teacher model and the more lightweight Swin Transformer as the student model. The distillation loss function covers both the classification and regression branches, including the hard loss of the student model on the true labels, the KL divergence loss of the classification branch, and the Euclidean distance loss between the bounding box parameters of the regression branch. Specifically:

[0144] Phase Four uses the large model (the teacher) trained in Phase Three to guide the student model. The student model reduces the number of Swin-T layers from 4 to 2 and halves the number of channels. (Distillation loss) Simultaneously covers both classification and regression branches:

[0145]

[0146] in: The hard loss of the student model on the real labels (including GIoU loss and classification cross-entropy); For temperature parameters; Indicates KL divergence; These are the classification logic values ​​of the student and teacher detection heads, respectively (without Softmax). For bounding box regression parameters (center point coordinates) ,width ,high ); , This is the balance coefficient; This represents the L2 norm.

[0147] Distillation training was performed for 50 rounds, with only the student model parameters being updated.

[0148] Training and inference details: The non-maximum suppression (NMS) threshold was 0.5, and the detection confidence threshold was 0.3. All comparative experiments were conducted on the same dataset partition, with a fixed random seed (42), and repeated 3 times with the average value taken. The selection of each weight in the loss function was determined based on a grid search on the validation set; those skilled in the art can adjust the above weights appropriately according to different datasets and task requirements without departing from the spirit of this invention.

[0149] Example 2:

[0150] like Figure 5 The diagram shown is a modular schematic of an adaptive pathological detection system for sparse lesions according to an embodiment of the present invention. The system includes:

[0151] The preprocessing module 210 is used to acquire the pathological image to be detected, preprocess the pathological image, and output the preprocessed pathological image.

[0152] The frequency domain-spatial domain joint enhancement module 220 is used to enhance the preprocessed pathological image using a pre-trained enhancement model and output the enhanced pathological image. The frequency domain-spatial domain joint enhancement module includes a frequency domain sub-module, a spatial domain sub-module, and an adaptive fusion module. The adaptive fusion module fuses the enhancement results of the frequency domain sub-module and the spatial domain sub-module through learnable weights.

[0153] The lesion detection module 230 is used to input the enhanced pathological image into a pre-trained detection model for lesion detection and output the location and category information of the lesion. The detection model is constructed based on a bidirectional multi-level local feature aggregation Transformer network, including a Swing Transformer backbone network, a bidirectional multi-level local feature aggregation module with a local discriminator, and a detection head.

[0154] The adaptive pathological detection system for sparse lesions provided in this embodiment of the invention can execute the adaptive pathological detection method for sparse lesions provided in any of the above embodiments of the invention, and has the corresponding functions and beneficial effects of executing the method. For detailed process, please refer to the relevant operations of the method in the foregoing embodiments. It will not be repeated here.

[0155] Example 3:

[0156] The present invention will be described in detail using the detection of HB cells in MASH pathological images as an example.

[0157] (1) Dataset: 600 liver biopsy pathology images were collected, and HB cells (bounding boxes) were independently annotated by 3 pathologists. The training and test sets were divided at a ratio of 4:1. The total number of HB cells in the training set was about 1200, and the image patch size was 512×512.

[0158] (2): 15 image patches containing HB cells were randomly selected from the training set as real seed samples. Approximately paired background images were obtained following the aforementioned background image generation process. The Adam optimizer was used with a learning rate of 0.0002, a batch size of 2, and 200 training epochs. The synthetic image diversity index was LPIPS = 0.32 (LPIPS between real samples = 0.11), and the pathologist's visual Turing test recognition accuracy was 55%, indicating that the synthetic image distribution is close to that of the real image.

[0159] (3): PSF estimator A three-layer convolutional model is used, with a kernel size of 3×3 in each layer and 16, 32, and 16 channels respectively, outputting 5×5 separable blur kernel parameters. Pre-training is performed using a synthetic defocused blur dataset: 2000 images are randomly cropped from clear pathological images, and a random Gaussian kernel is applied (…). Blurry-sharp pairs were generated and trained for 50 epochs using wavelet high-frequency subband L1 loss before being frozen. The local statistical window size in spatial augmentation was 7×7. The average PSNR of high-frequency details in the augmented image was improved by 2.1 dB.

[0160] (4): Swing-Tiny backbone network input The output feature map scales are 128, 64, 32, and 16, respectively. Local window size. In the BMFA module It consists of three convolutional layers (2 output channels), for and Forward sequentially. Update in batches using an alternating strategy. The scaling factor can be learned once, once for both the fusion module and the training settings in this embodiment. The convergence occurred within the 0.6–1.4 range, with no significant degradation observed. The total training epochs (stages two and three) were 120, using the AdamW optimizer with an initial learning rate of [missing information]. Weight decay Batch size 8.

[0161] (5): Implemented in four stages. Distillation losses... After distillation, the student model has 8.7M parameters (compared to 28.3M for the teacher model), and an AP50 of 0.84 on the test set (compared to 0.86 for the teacher model), representing a 2.3x improvement in inference speed. CPU testing environment: Intel Xeon 4210 (2.2GHz, 8 cores). Inference time per 512×512 image patch was 31ms for the student model and 72ms for the teacher model (average of 1000 tests).

[0162] Simulation results: such as Figure 6As shown, in this example image, the model detected all labeled HB cells (8 in total, 100% recall), and the IoU between the output bounding box and the ground truth was greater than 0.75. Some cells were missed by traditional YOLOv8 due to slight swelling (6 were detected), but this method successfully detected them through adversarial attention. Performance on the complete test set (120 cases): AP50=0.86, YOLOv8 was 0.72, and the baseline Transformer without using the sparse lesion augmentation model and the frequency-spatial domain joint enhancement model was 0.68. Ablation experiments (same test set, repeated 3 times and averaged): AP50=0.71 after removing the sparse lesion augmentation model (no synthetic data); AP50=0.78 after removing the frequency-spatial domain joint enhancement model; AP50=0.80 after removing the adversarial attention of the detection model (using pure Softmax local window attention). This confirms that each module contributes independently.

[0163] Hardware environment: Training was performed on an NVIDIA RTX 3080Ti, and CPU inference tests were conducted on an Intel Xeon 4210. The results are shown above.

[0164] The above results are experimental results based on a specific dataset in this embodiment and do not limit the scope of protection of this invention.

[0165] In summary, this invention proposes an adaptive pathological detection method and system for sparse lesions. Addressing the challenge of detecting sparse, blurred, and morphologically variable lesions, such as hepatocyte ballooning degeneration, in pathological images of metabolic-related steatohepatitis, this invention constructs an integrated framework comprising a sparse lesion augmentation module, a frequency-spatial domain joint enhancement module, and a Transformer detection network based on bidirectional multi-level local feature aggregation. The problem of sample scarcity is solved by generating high-fidelity synthetic lesion image samples using a generative adversarial network based on bidirectional cyclic consistency constraints. The details and contrast of blurred images are improved through Haar wavelet domain high-frequency enhancement and adaptive local contrast enhancement of the G / B channels. Self-attention weights are modulated by adversarial masks generated by bidirectional multi-scale feature fusion and a local discriminator, guiding the model to focus on discriminative regions and improving detection accuracy. The lightweight model, after knowledge distillation, maintains high performance while meeting real-time processing requirements, achieving significant performance improvements in the HB cell detection task of MASH pathological images.

[0166] The above description is only a preferred embodiment of the present invention and is used only to explain the present invention. It is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0167] It should also be noted that, in the embodiments of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0168] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in the embodiments of this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown in this application, but is to be accorded the widest scope consistent with the principles and novel features disclosed in the embodiments of this application.

Claims

1. An adaptive pathological detection method for sparse lesions, characterized in that, Includes the following steps: S1: Acquire the pathological image to be detected, and preprocess the pathological image to obtain the preprocessed pathological image; S2: The pre-processed pathological image is enhanced using a pre-trained frequency domain-spatial domain joint enhancement model to obtain an enhanced pathological image; the frequency domain-spatial domain joint enhancement model includes a frequency domain sub-module, a spatial domain sub-module, and an adaptive fusion module, and the adaptive fusion module fuses the enhancement results of the frequency domain sub-module and the spatial domain sub-module through learnable weights; S3: Input the enhanced pathological image into the pre-trained detection model for lesion detection. The detection model is constructed based on a bidirectional multi-level local feature aggregation Transformer network, uses the Swing Transformer to extract multi-scale features, and generates an adversarial mask through a local discriminator to modulate the self-attention weights, and finally outputs the location and category information of the lesion. In step S2: The frequency domain submodule is used to perform Haar wavelet decomposition on the preprocessed pathological image to obtain low-frequency and high-frequency components. It uses a pre-trained point spread function estimation operator to estimate the fuzzy kernel parameters from the high-frequency components, and performs Wiener filtering enhancement on the high-frequency components. Then, it performs inverse wavelet transform on the enhanced high-frequency components and the low-frequency components to obtain the frequency domain enhanced image. The spatial domain submodule is used to calculate the local mean and local variance of each pixel's neighborhood for the G and B channels of the preprocessed pathological image, and to enhance the pixel values ​​using a nonlinear mapping that includes a hyperbolic tangent function and a bias term, to obtain a spatial domain enhanced image. The adaptive fusion module is used to perform weighted fusion of the frequency domain enhanced image and the spatial domain enhanced image using learnable fusion weights to obtain the enhanced pathological image. In step S3, the detection model includes a bidirectional multi-level local feature aggregation module; the bidirectional multi-level local feature aggregation module uses two bidirectional paths, one from top to bottom and the other from bottom to top, to fuse multi-scale feature maps. At each fusion node, feature maps from different directions are aligned by bilinear interpolation and then spliced ​​together, and an adversarial mask is generated through the local discriminator. The adversarial mask is obtained by applying a Sigmoid mapping to the difference between the fusion class score and the non-fusion class score output by the local discriminator; the adversarial mask is then bilinearly interpolated, added to the local window self-attention score, and multiplied by a learnable scaling factor to modulate the attention weights. The local discriminator is a lightweight convolutional neural network, whose inputs are a fused feature map and an upsampled feature map, and whose outputs are a fused class score and a non-fused class score. The adversarial mask is represented as: in, and The scores for classifying the fused feature map and the upsampled feature map as fused are respectively. For the Sigmoid function, For adversarial masking.

2. The method according to claim 1, characterized in that, In step S1, the preprocessing includes: dividing the pathological image to be detected into multiple pathological image blocks of preset size, and performing pixel value normalization processing on each pathological image block to obtain a normalized pathological image.

3. The method according to claim 1, characterized in that, The detection model employs the local window self-attention mechanism of the Swing Transformer and modulates the attention weights of the adversarial mask with additive bias. Specifically, the adversarial mask is bilinearly interpolated to the same spatial size as the local window, and the corresponding mask value for each query position is broadcast to the biases of all key positions in that query pair. The modulated attention weight matrix is ​​represented as follows: in, and These are the query matrix and key matrix within the local window, respectively. For the dimensions of query and key, The adversarial mask, after bilinear interpolation, corresponds to the mask value at the i-th query position within the local window. A learnable scaling factor. This indicates the outer product expansion operation. Let M be a row vector of all 1s, and M be the side length of each local window; This is the attention weight matrix after additive bias modulation within the local window.

4. The method according to claim 1, characterized in that, The frequency domain-spatial domain joint enhancement model and the detection model are trained using a phased strategy, which includes: Synthetic lesion image samples were generated using a sparse lesion augmentation model based on generative adversarial networks and bidirectional cyclic consistency constraints to expand the training sample set; Fix the point spread function estimation operator in the frequency domain-spatial domain joint enhancement model, freeze the sparse lesion augmentation model, and pre-train the detection model using the original samples and the synthetic lesion image samples; Unfreeze the learnable fusion weights in the frequency domain-spatial domain joint enhancement model and some layers of the detection model, and perform end-to-end fine-tuning.

5. The method according to claim 4, characterized in that, The sparse lesion augmentation model includes a generator, an encoder, and a discriminator; The process of generating synthetic lesion images by the sparse lesion augmentation model includes: generating a corresponding synthetic background image from the real lesion image through image restoration; mapping the real lesion image to the latent space and sampling the latent variables through an encoder; and then using the synthetic background image and latent variables as inputs to synthesize new lesion image patches by a generator. The loss functions used to train the sparse lesion augmentation model include adversarial loss, bidirectional cyclic consistency loss, and KL divergence loss.

6. The method according to claim 4, characterized in that, The phased strategy also includes: compressing the end-to-end fine-tuned model into a lightweight student model through knowledge distillation; The knowledge distillation uses an end-to-end fine-tuned model as the teacher model and a lighter-weight Swing Transformer as the student model. The distillation loss function covers both the classification and regression branches, including the hard loss of the student model on the true labels, the KL divergence loss of the classification branch, and the Euclidean distance loss between the bounding box parameters of the regression branch.

7. An adaptive pathological detection system for sparse lesions, characterized in that, For performing the method according to any one of claims 1-6, comprising: The preprocessing module is used to acquire the pathological image to be detected, preprocess the pathological image, and output the preprocessed pathological image. The frequency domain-spatial domain joint enhancement module is used to enhance the preprocessed pathological image using a pre-trained enhancement model and output the enhanced pathological image. The frequency domain-spatial domain joint enhancement module includes a frequency domain sub-module, a spatial domain sub-module, and an adaptive fusion module. The adaptive fusion module fuses the enhancement results of the frequency domain sub-module and the spatial domain sub-module through learnable weights. The lesion detection module is used to input the enhanced pathological image into a pre-trained detection model for lesion detection and output the location and category information of the lesion. The detection model is constructed based on a bidirectional multi-level local feature aggregation Transformer network, including a Swing Transformer backbone network, a bidirectional multi-level local feature aggregation module with a local discriminator, and a detection head.

Images