Helicobacter pylori fluorescence image fine-tuning detection method and device and readable storage medium thereof

Abstract

The application provides a helicobacter pylori fluorescence image fine-tuning detection method and device and a readable storage medium thereof, and belongs to the technical field of medical image processing and computer-aided diagnosis. The method aims to solve the problems that the prior art only relies on fluorescence signals, leading to high false positives, and the model has poor generalization to different data environments. The method comprises the following steps: outputting a detection box of a suspected target from a fluorescence image block by a detection module; determining a target analysis region containing the detection box and a surrounding tissue region, and inputting the analysis region into a classification module to distinguish true and false positives; when the classification result is confirmed to be wrong, using the target analysis region data corresponding to the error and combining a group of preset representative sample features to fine-tune the classifier in the classification module. The application significantly reduces the false positive rate by using the surrounding tissue information, and improves the generalization ability and scene adaptability of the model through an efficient fine-tuning mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical information processing, in particular to a computer-aided diagnosis method based on artificial intelligence. More specifically, the present application relates to a method and system for digital pathology image processing, especially an automatic detection and fine-tuning algorithm for Helicobacter pylori fluorescence images. BACKGROUND

[0002] Helicobacter pylori infection is a key factor leading to chronic gastritis, peptic ulcer, and even gastric cancer and other gastric diseases. Therefore, accurate and efficient detection of Helicobacter pylori is of great significance in clinical diagnosis.

[0003] Fluorescence scanning technology is one of the commonly used methods for detecting Helicobacter pylori. It uses fluorescent probes to specifically bind to target bacteria, has high sensitivity and specificity, and can clearly present the morphology of bacteria. However, when the number of bacteria in the sample is small or they are in an inactive state, the fluorescence signal may become weak, which increases the difficulty of manual interpretation and easily leads to missed detection (i.e. false negatives).

[0004] To improve detection efficiency and accuracy, artificial intelligence (AI) technology, especially image recognition algorithms based on deep learning, has been introduced into the automatic analysis of Helicobacter pylori fluorescence images.

[0005] However, existing AI detection algorithms still face two major technical challenges in practical application:

[0006] First, the false positive rate is high. Most existing algorithms only rely on the intensity of the fluorescence signal as the main basis for judgment. However, in pathological samples, fluorescent probes may non-specifically bind to certain non-target substances (such as gastric mucosal epithelial cells, neutrophils, or necrotic tissue), also producing fluorescence signals. If the algorithm ignores the surrounding tissue environment information of the target and only relies on the fluorescence signal for judgment, it is easy to misjudge these fluorescent interference substances as Helicobacter pylori, leading to false positives.

[0007] Second, the generalization ability is poor. The data characteristics of pathological images are affected by many factors, such as different batches of reagents, different film-making processes, and different types of scanning instruments. These factors will cause significant differences (i.e. domain shift) in image style, signal intensity, etc. between data obtained by different medical institutions at different times. When a model trained on a dataset (i.e. fixed weights and parameters) is deployed to a new data environment, it often performs significantly worse due to inconsistent data distribution, lacking sufficient generalization. SUMMARY

[0008] The embodiment of the present application provides a kind of H. pylori fluorescence image fine-tuning detection method, device and readable storage medium thereof, to the detection technology of existing H. pylori fluorescence image or only according to the judgment of fluorescence signal intensity Target is ignored Surrounding tissue false positive, Or because the detection condition difference leads to training data coverage not complete makes model generalization poor, Cannot guarantee detection accuracy and multi-scene adaptability simultaneously Problem such as.

[0009] The core technology of the present application is mainly through the three-level technical system of "detection module extracts features to locate H. pylori detection box + classification module based on enlarged detection box area (including target and surrounding tissue) to distinguish true and false positive + representative samples and classification error samples are used to fine-tune the classifier", Realize the accurate detection of H. pylori fluorescence image and multi-detection condition adaptation.

[0010] In the first aspect, the present application provides a kind of H. pylori fluorescence image fine-tuning detection method, the method comprises the following steps:

[0011] Obtain the fluorescence image block to be detected;

[0012] The fluorescence image block is input into the pre-trained detection module, and the detection module is used to process the block to output one or more detection boxes of suspected target area;

[0013] For detection box, determine target analysis area, and the target analysis area includes the suspected target area indicated by the detection box and the tissue area around it;

[0014] The image data corresponding to the target analysis area is input into the pre-trained classification module, and the classification module includes a classifier to output the classification result of the suspected target area, and the classification result indicates that it is true positive or false positive;

[0015] In response to the classification result being confirmed as error, the target analysis area data corresponding to the error classification and the preset representative sample feature are used to fine-tune the classifier in the classification module.

[0016] Further, the step of determining the target analysis area specifically includes:

[0017] Keep the center position of the detection box unchanged, enlarge its size by a preset ratio, and use the enlarged box area as the target analysis area.

[0018] Further, before the image data corresponding to the target analysis area is input into the classification module, further comprising:

[0019] The image data corresponding to the target analysis area is cut and scaled to a preset size.

[0020] Further, the detection module comprises:

[0021] a segmentation network based on an encoder-decoder architecture, the segmentation network being configured to generate a probability map indicating a probability of each pixel belonging to a bacillus; and

[0022] a post-processing unit configured to perform thresholding, morphological opening operation and contour extraction on the probability map to generate a bounding box.

[0023] Further, the classification module comprises:

[0024] at least one encoding module configured to extract a feature vector from image data of a target analysis region; the classifier is a fully connected layer configured to receive the feature vector and map it to a class space, and in the step of fine-tuning the classifier, the weights of the encoding module in the classification module remain unchanged.

[0025] Further, the pre-set representative sample features comprise:

[0026] a true positive clustering center feature obtained by clustering the true positive sample feature vectors of the classification module on the original training set; and

[0027] a false positive clustering center feature obtained by clustering the false positive sample feature vectors of the classification module on the original training set.

[0028] wherein the number of clustering clusters used for clustering is determined based on a channel weight vector learned by the classification module during the original training process, and the number of clustering clusters is related to the number of channels whose values in the channel weight vector are greater than a pre-set weight threshold.

[0029] Further, the loss function used for fine-tuning comprises:

[0030] a classification loss term on the target analysis region data corresponding to the misclassified sample and the representative sample features; and

[0031] a regularization term for constraining the difference between the weights of the fine-tuned classifier and the weights of the classifier before fine-tuning.

[0032] In a second aspect, the present application provides a fine-tuned detection device for fluorescence images of Helicobacter pylori, comprising:

[0033] an acquisition module configured to acquire a fluorescence image patch to be detected;

[0034] a detection module configured to receive the fluorescence image patch, process the patch, and output one or more bounding boxes of suspected target regions;

[0035] a region determination module configured to determine a target analysis region for the bounding box, the target analysis region including the suspected target region indicated by the bounding box and the tissue region around the suspected target region.

[0036] The classification module is configured to receive image data corresponding to the target analysis area, and the classification module includes a classifier to output a classification result of the suspected target area, and the classification result indicates whether it is a true positive or a false positive.

[0037] The fine-tuning module is configured to, in response to the classification result being confirmed as an error, fine-tune the classifier in the classification module by using target analysis area data corresponding to the error classification and a set of preset representative sample features.

[0038] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned fine-tuning detection method of the Helicobacter pylori fluorescence image.

[0039] In a fourth aspect, the present application provides a readable storage medium, the readable storage medium stores a computer program, and the computer program includes program code for controlling a process to execute the process, and the process includes the above-mentioned fine-tuning detection method of the Helicobacter pylori fluorescence image.

[0040] The main contributions and innovations of the present application are as follows:

[0041] 1. Significantly reduce the false positive rate and improve the detection accuracy. The present application adopts a two-stage strategy of "first detection and then classification". The detection module first quickly locates all the fluorescent suspected targets; then, the classification module does not analyze the target itself, but analyzes a "magnification frame" area containing the target and its surrounding tissue environment. This design enables the algorithm to utilize the surrounding tissue morphology information (such as whether the target is located in the gastric mucosa structure) for comprehensive identification, thereby effectively distinguishing between real Helicobacter pylori and non-target substances (such as necrotic tissue, epithelial cells) that produce interfering fluorescence, solving the misjudgment problem caused by ignoring environmental information in the prior art.

[0042] 2. Improve the generalization and scene adaptability of the model, and reduce the deployment cost. The present application designs an efficient classifier fine-tuning mechanism. After the initial training of the model, the "representative feature vectors" of the true / false positive samples are extracted and saved by a clustering algorithm. When the model makes errors in a new data environment (such as replacing the scanner), there is no need to retrain the entire large model, only the features of the new error samples are combined with the pre-stored representative features, and the lightweight fully connected layer (classifier) is fine-tuned. This mechanism can quickly correct errors to adapt to new data, and can also constrain the recognition ability of the model to old samples (avoid catastrophic forgetting) through representative samples and loss functions, achieving rapid and low-cost adaptation to different data environments.

[0043] The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the application will be apparent from the description of the preferred embodiments and from the drawings in which like reference numerals refer to like elements throughout. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0045] Figure 1 is a flow chart of a helicobacter pylori fluorescence image fine-tuning detection method according to an embodiment of the application;

[0046] Figure 2 is a structural schematic diagram of a detection module according to an embodiment of the application;

[0047] Figure 3 is a structural schematic diagram of a convolution module according to an embodiment of the application;

[0048] Figure 4 is a structural schematic diagram of a decoding module according to an embodiment of the application;

[0049] Figure 5 is a structural schematic diagram of a classification module according to an embodiment of the application;

[0050] Figure 6 is a structural schematic diagram of an encoding module according to an embodiment of the application;

[0051] Figure 7 is a detection result diagram according to an embodiment of the application;

[0052] Figure 8 is a classification result diagram according to an embodiment of the application;

[0053] Figure 9 is a structural schematic diagram of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0054] In order to make the technical problems solved by the application, the technical solutions and the beneficial effects more clear, the application will be further described in detail below with reference to the drawings and preferred embodiments. It should be understood that the preferred embodiments described herein are only used to explain the application, but not used to limit the protection scope of the application.

[0055] The embodiment of the application provides a helicobacter pylori fluorescence image detection method and system, and aims to solve the problems of high false positive rate caused by only relying on the fluorescence signal strength and poor generalization caused by data difference in the prior art.

[0056] Embodiment one

[0057] The present application aims to provide a Helicobacter pylori fluorescence image fine-tuning detection method, specifically, with reference to Figure 1 The core idea of the present application is to adopt a two-stage strategy of "first detection, then classification", supplemented by an efficient fine-tuning mechanism.

[0058] Detection stage: Use the detection module to quickly find all possible fluorescence targets from the global image.

[0059] Classification stage: For the detected targets, analyze a magnified area containing the surrounding tissue environment to accurately distinguish true positives from false positives.

[0060] Fine-tuning stage: When the model makes mistakes in a new environment (such as different reagents or instruments), use the pre-stored "representative sample features" and new error samples to fine-tune only the classifier, so that the model can quickly adapt to new data without retraining the entire model, achieving generalization.

[0061] The specific process of the method is described in detail below.

[0062] I. Training data preparation

[0063] In the model training stage, two sets of data need to be prepared first:

[0064] Detection module training data: Cut the original fluorescence scan slice image into a series of image blocks. Pathologists label these image blocks to generate corresponding segmentation masks (Mask) for training the detection module. For example:

[0065] Obtain clinically collected Helicobacter pylori fluorescence staining sections (sections labeled with standard fluorescence probes, meeting clinical detection specifications), and use a digital slice scanning device (such as a pathology slice scanner) to scan the sections into high-resolution digital images. Then, perform "image cutting processing": cut the image into 512x512x3 (widthxheightxchannel number, 3 channels corresponding to the RGB or specific fluorescence channel of the fluorescence image) image blocks with no overlap (or a pre-set overlap rate, such as 10%) to obtain the image blocks as the training data for the detection module.

[0066] Classification module training data: First, use the trained detection module to detect the above image blocks to obtain a series of detection boxes. Then, for each detection box, keep the center position unchanged and enlarge it by one time to obtain an "enlarged box". Cut the image from the original image block according to the position of the enlarged box, and label it as "true positive" or "false positive" by an expert to serve as the training data for the classification module. For example:

[0067] The detection module trained in the subsequent steps is used to detect the 512x512 pixel image block, and the initial detection box corresponding to the Helicobacter pylori is output (for example, an initial detection box with a size of 80x60 pixels and a center coordinate of (256, 256)). The center coordinate of the detection box is kept unchanged, and the length and width are both enlarged by 1 times (i.e., the size of the enlarged detection box is 160x120 pixels, and the center is still (256, 256)); based on the enlarged detection box, the corresponding region is cut on the original 512x512 pixel image block to obtain an image block containing Helicobacter pylori and surrounding gastric mucosa tissue, which is used as training data for the classification module.

[0068] II. Detection module

[0069] The task of the detection module is to segment and locate all suspected Helicobacter pylori target regions from the input fluorescence image block.

[0070] As shown in Figure 2 In a preferred embodiment, the detection module uses an encoder-decoder (Encoder-Decoder) architecture. It mainly consists of a feature extraction model, a decoding module and a 1x1 convolution layer.

[0071] Feature extraction model (encoder): responsible for extracting high-dimensional features from the input image block. This model can use a pre-trained model known in the art, such as the ResNet series, or a digital pathology base model (such as progiga-path), etc. Its output is usually a high-dimensional feature vector. For example:

[0072] The open source ResNet50 model is selected, which receives a 512x512x3 image block input, extracts high-dimensional features of the image through multiple convolution and pooling operations, and finally outputs an image feature vector with a dimension of [1, 1, 256] (256 is the feature dimension, which can be adjusted according to the model adaptability).

[0073] Decoding module (decoder): responsible for upsampling the high-dimensional features extracted by the encoder, gradually recovering the spatial information of the image, and achieving pixel-level accurate positioning. For example:

[0074] The decoding module is composed of four convolution modules and one up-sampling module. In this embodiment, the decoding module contains four convolution modules.

[0075] 1x1 convolution layer: acts on the output of the decoding module. It is equivalent to a pixel-level fully connected layer, which is used to map the channel number of the decoder output feature map to the final class number (for example, two classes of background and bacillus). Then, through the Softmax activation function, a probability map is generated, and the value of each pixel in the map represents the probability of belonging to bacillus. For example:

[0076] The [512, 512, 64] feature map output by the decoding module is converted into a feature map with a channel number equal to the class number (2 classes: H. pylori and background) through channel mapping, and finally a [512, 512, 1] feature map (single channel corresponds to the probability output of binary classification) is output.

[0077] As shown in Figure 4 , the decoding module is composed of several convolution modules and an up-sampling module. In this embodiment, the decoding module contains four convolution modules.

[0078] As shown in Figure 3 , the convolution module is the basic component unit of the encoder and decoder (such as the decoding module and the encoding module). It is composed of a convolution layer, a batch normalization (Batch Normalization) layer and a ReLU activation function in turn. The parameters of the convolution layer include convolution kernel size k, step size s, padding p, input channel number c_in and output channel number c_out. The batch normalization layer is used to accelerate training and improve stability. Batch normalization is to normalize each feature dimension separately, so that the mean is 0 and the variance is 1.

[0079] After the detection module outputs the probability map, the system performs post-processing operations:

[0080] 1. Threshold segmentation: Set a probability threshold (for example, 0.5), and pixels with a probability value greater than the threshold are judged to belong to bacillus.

[0081] 2. Morphological processing: Preferably, an opening operation is used to remove isolated small noise points (i.e. small area artifacts) that can exist on the probability map.

[0082] 3. Contour extraction: The contours of the connected regions are extracted from the processed binary image.

[0083] 4. Generate detection boxes: Calculate the horizontal bounding rectangle of each contour, which is the detection box output by the detection module, as shown in Figure 7 , it can be seen that there are multiple white detection boxes on the map.

[0084] When training the detection module, the weights of the feature extraction model can be fixed, and only the parameters of the decoding module and the 1x1 convolution layer are trained. Since the proportion of bacillus in the image is small, a mask segmentation loss L mask is preferably composed of binary cross-entropy loss (L BCE ) and Dice loss (L Dice ), where the weight hyperparameter a of the Dice loss is preferably 0.1. For example:

[0085]

[0086] where a is a hyperparameter, preferably 0.1. N represents the number of samples, y p and y gt represent the predicted class and label respectively, e represents a smoothing term, preferably 1x10 -6 .

[0087] Preferably, the training image blocks of the detection module are labeled by a pathologist, marking the area where the helicobacter pylori is located, generating a binary segmentation mask consistent with the size of the image block (the pixel value of the helicobacter pylori area is 1, and the background area is 0). When training, the pre-training weights of the ResNet50 model are fixed (only the feature extraction capability is used to avoid overfitting caused by retraining), and only the weights of the decoding module and the 1x1 convolution layer are updated, using the Adam optimizer, the learning rate is set to 1e-4, and the iteration is trained until the loss function converges.

[0088] III. Classification module

[0089] The task of the classification module is to perform secondary identification on the "suspected targets" output by the detection module, and to distinguish between true positives and false positives. Its core lies in the use of the tissue environment information around the target.

[0090] 3.1 Determination of target analysis area

[0091] When the detection module outputs a detection box, the system does not directly use the image in the detection box for classification. Instead, the following steps are performed to determine a target analysis area:

[0092] 1. Obtain the center point coordinate of the detection box.

[0093] 2. Keep the center point coordinate unchanged, and enlarge the length and width of the detection box by one time (i.e. enlarge by a preset ratio, which is preferably 2).

[0094] 3. The enlarged box region is defined as the target analysis region.

[0095] The target analysis region not only contains the suspected target in the detection box, but more importantly, it contains the tissue region around the target. This enables the classification module to learn the context information such as "whether the fluorescence signal appears in the typical gastric mucosa structure", thereby effectively excluding those interference objects (such as necrotic tissue) that also emit fluorescence but are in atypical positions, significantly reducing the false positive rate.

[0096] 3.2 Structure and training of the classification module

[0097] Before inputting into the classification module, the system first crops the image corresponding to the above-mentioned "target analysis region" from the original image block, and uniformly scales it to a preset size (e.g. 128x128 pixels).

[0098] As shown in FIG. 4, in a preferred embodiment, the classification module is composed of several (e.g. 4) encoding modules and a classifier (preferably a fully connected layer). Figure 5 As shown in FIG. 5, the encoding module is used to extract features from the input target analysis region image. Its structure can be stacked by one or more convolution modules, and can contain pooling layers to reduce the size of the feature map. For example:

[0099] Figure 6 A total of 4 encoding modules are provided, each of which is composed of two convolution modules in series of "convolution layer -> batch normalization layer -> ReLU activation function layer" (the convolution layer hyperparameters are consistent with those of the convolution module of the detection module). The encoding module receives the input of the scaled 128x128x3 image region (i.e. the region cropped in the classification module training data is scaled to 128x128 pixels), and through layer-by-layer convolution and downsampling, the spatial dimension is gradually compressed and the channel dimension is gradually increased, and finally a feature vector with a dimension of [1, 1, 256] is output (consistent with the output feature dimension of the detection module, for subsequent processing).

[0100] The encoding part of the classification module finally outputs a high-dimensional feature vector f. In order to improve the discrimination, this embodiment introduces a channel weight vector w (the initial value can be set to 0.5) for weighting each channel (representing a feature) of the feature vector f. The loss function during training is

[0101] The encoding part of the classification module finally outputs a high-dimensional feature vector f. In order to improve the discrimination, this embodiment introduces a channel weight vector w (the initial value can be set to 0.5) for weighting each channel (representing a feature) of the feature vector f. The loss function during training is ​The classifier is designed to minimize the original feature vector f and the weighted feature vector simultaneously Classification error after the classifier, where the balance hyperparameter β is preferably 0.5. For example:

[0102]

[0103] where F c represents the full connection layer, f represents the feature vector, and w is the channel weight vector. Preferably, the Adam optimizer is used, the learning rate is set to 1e-5, the weights of the encoding module, the channel weight unit and the full connection layer are trained as a whole, and the iteration is performed until the classification accuracy (using the pathological expert labeling result as the gold standard) is stable at more than 95%.

[0104] Preferably, a fixed weight of 0.5 is assigned to each channel of the feature vector at the initial stage (forming a channel weight vector of [1, 1, 256]); during the training process, the weight is dynamically adjusted according to the contribution of each channel feature to the classification result - the weight of the channel with high contribution to the differentiation of "Hp and non-Hp" (such as the channel reflecting the gastric mucosa structure and Hp morphology) is gradually increased to 0.7-0.9; the weight of the channel with low contribution is reduced to 0.1-0.3, so as to enhance the discriminability of key features.

[0105] where the full connection layer: contains two layers of full connection network, the first layer receives the "original feature vector" and the "channel weight weighted feature vector" as input, and maps it to a 128-dimensional intermediate feature space; the second layer maps the intermediate features to a 2-dimensional class space (corresponding to true positive and false positive), and outputs the classification result.

[0106] The detection frame output by the detection module is processed in the manner of "center invariant, length and width enlarged by 1 times", and the image region containing Hp and surrounding tissue is cut; after scaling the region to 128x128 pixels, it is input into the classification module; the encoding module extracts the feature vector, the channel weight unit weights the feature vector, and the full connection layer maps the two types of feature vectors to the class space, and finally outputs the "true positive" (Hp) or "false positive" (non-Hp, such as gastric mucosa epithelial cell fluorescence interference) judgment result corresponding to the detection frame, as shown in Figure 8 .

[0107] 4. Classifier fine-tuning mechanism

[0108] The application further provides a classifier fine-tuning mechanism for solving the generalization problem of the model in different data environments. The mechanism includes two stages of "preparation of representative sample features" and "execution of fine-tuning training".

[0109] 4.1 Preparation of representative sample features

[0110] To ensure that the subsequent classifier fine-tuning can still accurately classify old samples, representative samples need to be determined from the training data. After the classification module completes the initial training, the following steps are performed:

[0111] 1. Input all true positive samples in the training set into the encoder (i.e., the encoding module) of the classification module to extract their corresponding feature vectors.

[0112] 2. Use the K-Means clustering algorithm to cluster all true positive feature vectors.

[0113] 3. Store the resulting K T cluster centers (feature vectors) as "true positive representative sample features".

[0114] 4. Repeat steps 1-3 for all false positive samples to obtain K N "false positive representative sample features" and store them.

[0115] Each channel on a feature vector is a feature. If a feature vector's value on a channel is closer to H. pylori, the greater the probability that the feature vector represents H. pylori. This is similar to the clustering algorithm, where the closer to the cluster center, the greater the probability of being in the same class. Therefore, the channels of the feature vector can be considered as cluster centers. Channel weights with a value greater than 0.5 indicate that the channel is important for classification, so the number of cluster centers can be set to the number of channels with a value greater than 0.5 in the channel weight. Considering the possibility that there are few channels with a value greater than 0.5, or even equal to 0, set K T and K N to be greater than or equal to 5. The channel weights w and the representative sample feature vectors of K T true positives and K N false positives obtained in this step will be used for subsequent classifier fine-tuning.

[0116] That is, the number of K T and K N (i.e., the number of cluster clusters) can be set empirically, or preferably, determined based on the channel weight vector w learned from the initial training. For example, the number of channels with a value greater than 0.5 (i.e., important feature channels) in w is used as the number of cluster clusters. To ensure stability, a minimum value can be set, for example and .

[0117] 4.2 Execution of fine-tuning training

[0118] When the model is deployed in a new environment (such as a new hospital or a new batch of reagents) and classification errors occur (e.g., confirmed by manual review), the fine-tuning mechanism is triggered:

[0119] 1. Obtain the "target analysis area" image that causes the classification error.

[0120] 2. Input it into the encoding module of the classification module to extract its feature vector. For example: input the "128x128 pixel image area" of the classification error into the encoding module of the classification module to extract the error sample feature vector [1, 1, 256] (denoted as M, M is the number of classification error samples).

[0121] 3. Freeze the weights of the encoding module (i.e. keep them unchanged).

[0122] 4. Construct a fine-tuning dataset containing new error sample features and all pre-stored representative sample features (a total of ).

[0123] 5. Only use this fine-tuning dataset to retrain the classifier (i.e. fully connected layer) (fine-tune).

[0124] Loss function used in fine-tuning Contains two parts:

[0125] Classification loss term: Ensure that the fine-tuned classifier can correctly classify new error samples M and continue to correctly classify representative samples.

[0126] Regularization term: Restrict the weights of the fine-tuned classifier Should not deviate too far from the original weights .

[0127] This regularization term ensures that the model learns new knowledge (corrects errors) while not "forgetting" old knowledge (i.e. the original data distribution represented by representative samples), effectively avoiding "catastrophic forgetting" and ensuring the overall performance of the fine-tuned model.

[0128] For example:

[0129]

[0130] Where M is the new image feature data. By controlling the influence of new image features and representative features on fine-tuning through M / (K T +K N ), it is ensured that the fine-tuned model not only corrects the classification error but also guarantees the classification accuracy of old samples. Preferably, use the Adam optimizer with a learning rate of 1e-6, and iterate for 5-10 rounds (adjust according to loss convergence), to complete the fine-tuning of the classifier.

[0131] In summary, the application effectively reduces false positives through the cooperative work of the detection module and the classification module and the use of a target analysis region containing surrounding tissue information. Meanwhile, through a classifier fine-tuning mechanism based on representative samples, the application realizes rapid and low-cost adaptation to new data environments and significantly improves the generalization ability of the model.

[0132] Embodiment Two

[0133] Based on the same concept, the application further provides a H. pylori fluorescence image fine-tuning detection device, which comprises:

[0134] An acquisition module is configured to acquire a fluorescence image patch to be detected.

[0135] A detection module is configured to receive the fluorescence image patch, process the patch, and output one or more detection boxes of suspected target regions.

[0136] A region determination module is configured to determine, for the detection boxes, target analysis regions, which contain the suspected target regions indicated by the detection boxes and the tissue regions around the suspected target regions.

[0137] A classification module is configured to receive image data corresponding to the target analysis regions, and the classification module contains a classifier to output classification results of the suspected target regions, which indicate whether the suspected target regions are true positives or false positives.

[0138] A fine-tuning module is configured to, in response to the classification results being confirmed as incorrect, fine-tune the classifier in the classification module by using target analysis region data corresponding to the incorrect classification and a set of preset representative sample features.

[0139] Embodiment Three

[0140] The embodiment also provides an electronic device, which refers to Figure 9 and comprises a memory 404 and a processor 402, the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0141] Specifically, the above processor 402 can include a central processing unit (CPU) or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the application.

[0142] The memory 404 can include a mass storage that stores data or instructions. For example, and without limitation, the memory 404 can include a Hard Disk Drive (HDD), a floppy disk drive, a Solid State Drive (SSD), a flash drive, a Compact Disc Read Only Memory (CD-ROM), a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. The memory 404 can be removable and / or non-removable (or fixed) as appropriate. The memory 404 can be internal or external as appropriate. In particular embodiments, the memory 404 is a Non-Volatile memory. In particular embodiments, the memory 404 includes a Read-Only Memory (ROM) and a Random Access Memory (RAM). The ROM can be a mask-programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH memory, or a combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random Access Memory (FPMDRAM), an Extended Data Output Dynamic Random Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0143] The memory 404 can be used to store or buffer various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 402.

[0144] The processor 402 implements the Helicobacter pylori fluorescent image fine-tuning detection method in any of the above embodiments by reading and executing the computer program instructions stored in the memory 404.

[0145] Optionally, the electronic device described above can further include a transmission device 406 connected to the processor 402 and an input / output device 408 connected to the processor 402.

[0146] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network can include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (NIC) that can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module for communicating with the Internet in a wireless manner.

[0147] The input / output device 408 is used for inputting or outputting information.

[0148] Embodiment Four

[0149] The embodiment also provides a readable storage medium, and the readable storage medium stores a computer program. The computer program includes program codes for controlling a process to execute the process. The process includes the Helicobacter pylori fluorescent image fine-tuning detection method according to Embodiment One.

[0150] It should be noted that specific examples in the embodiment can refer to examples described in the above embodiments and optional implementation manners, which will not be described herein again.

[0151] Generally, various embodiments can be implemented in hardware or special-purpose circuitry, software, logic or any combination thereof. Some aspects of the application can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, microprocessor or other computing device, but the application is not limited thereto. Although various aspects of the application can be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein can be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0152] Embodiments of the application can be implemented by computer software executable by a data processor of the mobile device such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or program, also called program product, including software routines, applets and / or macros, can be stored in any apparatus-readable data storage medium and they include program instructions to implement certain tasks. The program product can include one or more computer-executable components such as by way of example scripts, programs, routines, objects, components, data structures, procedures, subroutines, etc., for implementing one or more embodiments of the application. The computer-executable components can be stored in computer-readable storage medium. The computer-readable storage medium can include one or more of a floppy disk, a ZIP® disk, a hard disk, a video tape, a pen drive, a memory stick, a system memory, a RAM, a ROM, a programmable ROM (EPROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a portable computer disk, a Figure 1 Any block in the logical flow of the method described herein, including those of the artificial neural network, can represent a program step, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software can be stored on such physical media as memory chips, or memory blocks implemented in the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD, etc. The physical media are non-transitory media.

[0153] Those skilled in the art should understand that all or some of the technical features in the above embodiments can make any combinations, and in order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

[0154] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as the limitation to the scope of the present application. It should be noted that for those skilled in the art, some modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for fine-tuning fluorescence images of Helicobacter pylori, characterized in that, Includes the following steps: Acquire the fluorescence image patch to be detected; The fluorescence image patch is input into a pre-trained detection module, which processes the patch to output one or more detection boxes for suspected target regions. For the detection frame, a target analysis region is determined, which includes the suspected target region indicated by the detection frame and the surrounding tissue region; The step of determining the target analysis area specifically includes: keeping the center position of the detection frame unchanged, enlarging its size by a preset ratio, and using the enlarged frame area as the target analysis area; The image data corresponding to the target analysis region is input into a pre-trained classification module, which includes a classifier to output the classification result of the suspected target region, and the classification result indicates whether it is a true positive or a false positive. In response to the classification result being confirmed as incorrect, the classifier in the classification module is fine-tuned and trained using the target analysis region data corresponding to the incorrect classification and the preset representative sample features. The preset representative sample features include: The true positive cluster center features are obtained by clustering the true positive sample feature vectors of the classification module on the original training set; and... The false positive cluster center features are obtained by clustering the false positive sample feature vectors of the classification module on the original training set. The number of clusters used in the clustering is determined based on a channel weight vector learned by the classification module during the original training process. The number of clusters is related to the number of channels in the channel weight vector whose values ​​are greater than a preset weight threshold. The loss function used in the fine-tuning training includes: The classification loss term on the target analysis region data corresponding to the misclassification and the representative sample features; and, A regularization term used to constrain the difference between the classifier weights after fine-tuning and the classifier weights before fine-tuning.

2. The method for fine-tuning the fluorescence image of Helicobacter pylori as described in claim 1, characterized in that, Before inputting the image data corresponding to the target analysis region into the classification module, the method further includes: The image data corresponding to the target analysis area is cropped and scaled to a preset size.

3. The method for fine-tuning the fluorescence image of Helicobacter pylori as described in claim 1, characterized in that, The detection module includes: A segmentation network based on an encoder-decoder architecture, wherein the segmentation network is used to generate a probability map indicating that each pixel belongs to a bacillus; and, The post-processing unit is used to perform thresholding, morphological opening operations, and contour extraction on the probability map to generate the detection box.

4. The method for fine-tuning the fluorescence image of Helicobacter pylori as described in claim 1, characterized in that, The classification module includes: At least one encoding module is used to extract feature vectors from image data of the target analysis region; the classifier is a fully connected layer used to receive the feature vectors and map them to a category space, and in the step of fine-tuning the classifier in the classification module, the weights of the encoding module in the classification module remain unchanged.

5. A device for fine-tuning fluorescence images of Helicobacter pylori, characterized in that, include: The acquisition module is used to acquire the fluorescence image patch to be detected; The detection module is configured to receive the fluorescence image patch, process the patch, and output a detection box for one or more suspected target areas; The region determination module is configured to determine a target analysis region for the detection frame, wherein the target analysis region includes the suspected target region indicated by the detection frame and the surrounding tissue region; The step of determining the target analysis area specifically includes: keeping the center position of the detection frame unchanged, enlarging its size by a preset ratio, and using the enlarged frame area as the target analysis area; A classification module is configured to receive image data corresponding to the target analysis region. The classification module includes a classifier to output the classification result of the suspected target region, and the classification result indicates whether it is a true positive or a false positive. The fine-tuning module is configured to, in response to the classification result being identified as incorrect, use the target analysis region data corresponding to the incorrect classification and a set of preset representative sample features to fine-tune and train the classifier in the classification module. The preset representative sample features include: The true positive cluster center features are obtained by clustering the true positive sample feature vectors of the classification module on the original training set; and... The false positive cluster center features are obtained by clustering the false positive sample feature vectors of the classification module on the original training set. The number of clusters used in the clustering is determined based on a channel weight vector learned by the classification module during the original training process. The number of clusters is related to the number of channels in the channel weight vector whose values ​​are greater than a preset weight threshold. The loss function used in the fine-tuning training includes: The classification loss term on the target analysis region data corresponding to the misclassification and the representative sample features; and, A regularization term used to constrain the difference between the classifier weights after fine-tuning and the classifier weights before fine-tuning.

6. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the Helicobacter pylori fluorescence image fine-tuning detection method according to any one of claims 1 to 4.

7. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling the process to execute the process, the process including the Helicobacter pylori fluorescence image fine-tuning detection method according to any one of claims 1 to 4.

Images