Helicobacter pylori pathological image optimization method based on dense region recognition

Abstract

The invention discloses a helicobacter pylori pathological image optimization method based on dense region recognition, and the method comprises the following steps: 1, obtaining a pathological image containing helicobacter pylori, and obtaining a first data set and a second data set; 2, training a first key point detection model based on the first data set; training a target detection model based on the second data set; training a second key point detection model based on the dense region image; step 3, acquiring a pathological image to be optimized; obtaining an initial detection result of the pathological image to be optimized; obtaining predicted key point coordinates corresponding to helicobacter pylori in the pathological image to be optimized; and carrying out mapping replacement to obtain an optimized pathological image. According to the method, missing detection or false detection caused by form difference of helicobacter pylori can be reduced, so that the accuracy of dense region identification is improved; the model has interpretability and helps doctors to understand decision basis of the model; and enhancement optimization based on the dense region frame can improve the image quality in a more targeted manner.

Description

Technical Field

[0001] The present invention belongs to the technical field of image enhancement, and particularly relates to a method for optimizing Helicobacter pylori pathological images based on dense region recognition. Background Art

[0002] In pathological image analysis, Helicobacter pylori images play a crucial role and are the core basis for disease diagnosis, mechanism research, treatment plan formulation, and prognosis assessment. By analyzing the morphology, structure, and distribution of Helicobacter pylori, pathologists can accurately judge the disease type, severity, and development stage. In the field of digital pathology, Helicobacter pylori images are the core data for training and testing artificial intelligence algorithms. By annotating and analyzing Helicobacter pylori, efficient automated diagnostic tools can be developed to improve the efficiency and accuracy of pathological analysis. In addition, Helicobacter pylori images are also important materials for pathology teaching, providing opportunities for medical students and pathologists to learn the microscopic characteristics of diseases and helping to improve diagnostic capabilities.

[0003] In pathological image analysis, it is of great significance to identify the densely distributed regions of Helicobacter pylori in Helicobacter pylori images and perform enhancement optimization based on this. Dense region recognition can accurately locate the distribution of Helicobacter pylori, avoid missing key regions, and thus improve the accuracy of diagnosis; dense region recognition helps to improve the efficiency of automated analysis, enabling the algorithm to quickly focus on key regions and reducing the computational burden; while enhancement optimization further improves the image quality and provides more reliable data support for automated feature extraction and classification. Dense region recognition lays the foundation for the quantity statistics and distribution research of Helicobacter pylori, and enhancement optimization ensures the accuracy of quantitative results.

[0004] However, at the present stage, there are still many challenges in identifying the densely distributed regions of Helicobacter pylori in Helicobacter pylori images.

[0005] Firstly, the insufficient accuracy of dense region recognition is a major problem. The morphology, size, and distribution of Helicobacter pylori may vary greatly, making it difficult for algorithms to accurately capture all target regions. For example, Helicobacter pylori may present irregular shapes, and the size and staining intensity vary greatly under different pathological conditions. This diversity makes it difficult for algorithms to design unified feature extraction rules, resulting in missed detections or false detections. In addition, the distribution of Helicobacter pylori may be highly dense and uneven, further increasing the difficulty of recognition. For example, in cancer pathological images, tumor cells may be closely aggregated, with a low contrast with normal cells or background tissues, and it is difficult for traditional algorithms to accurately distinguish the target regions.

[0006] Secondly, many deep learning-based algorithms lack interpretability and it is difficult to gain the trust of pathologists. In the identification of Helicobacter pylori dense distribution areas in pathological images, deep learning models may rely on some unintuitive or pathological knowledge-inconsistent features for prediction, resulting in results inconsistent with the professional judgment of pathologists. For example, the model may misidentify staining artifacts or background noise as Helicobacter pylori, or ignore some atypically shaped Helicobacter pylori. This lack of interpretability seriously reduces pathologists' trust in the algorithm, thereby affecting its acceptance in actual diagnosis. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an optimized method for Helicobacter pylori pathological images based on dense area recognition in view of the above-mentioned deficiencies in the prior art. The method has a simple structure and reasonable design. It trains a key point detection model based on labeled Helicobacter pylori to reduce missed or false detections caused by morphological differences of Helicobacter pylori; generates dense boxes for Helicobacter pylori dense distribution areas based on point annotation data, which has interpretability; uses the dense area box as the label of the pathological image to train an object detection model, obtains the predicted dense box of the Helicobacter pylori dense distribution area, accurately predicts the dense area range of Helicobacter pylori, intuitively displays the infected area, and enables doctors to comprehensively understand the condition.

[0008] To solve the above technical problems, the technical solution adopted by the present invention is: an optimized method for Helicobacter pylori pathological images based on dense area recognition, which is characterized by including the following steps:

[0009] Step 1. Obtain a data set:

[0010] Step 101. Obtain pathological images containing Helicobacter pylori. The basic information and the first annotation information of the pathological images form the first data set, and the first annotation information includes at least the true coordinates of each Helicobacter pylori.

[0011] Step 102. The basic information and the second annotation information of the pathological images form the second data set, and the second annotation information includes at least the dense area box of the Helicobacter pylori dense distribution area.

[0012] Step 2. Train the model:

[0013] Step 201. Train the first key point detection model based on the first data set, define the first key loss function, and obtain the trained first key point detection model.

[0014] Step 202. Train the object detection model based on the second data set, define the object loss function, and obtain the trained object detection model.

[0015] Step 203: Use the trained object detection model to obtain the predicted dense bounding boxes for the areas with dense distribution of Helicobacter pylori in the pathological image; crop and enlarge the predicted dense bounding boxes to obtain the dense area images.

[0016] Step 204: Train the second key point detection model based on the dense area images, define the second key loss function, and obtain the trained second key point detection model.

[0017] Step Three: Optimize the pathological image:

[0018] Step 301: Obtain the pathological image to be optimized.

[0019] Step 302: Use the trained first key point detection model to obtain the initial detection result of the pathological image to be optimized.

[0020] Step 303: The pathological image to be optimized sequentially passes through the trained object detection model, cropping, enlargement, and the trained second key point detection model to obtain the predicted key point coordinates corresponding to Helicobacter pylori in the dense area image.

[0021] Step 304: Map the predicted key point coordinates of the dense area image back to the corresponding positions of the pathological image to be optimized according to the original ratio, and replace the initial detection result of the pathological image to be optimized to obtain the optimized pathological image.

[0022] The above method for optimizing Helicobacter pylori pathological images based on dense area recognition is characterized in that: the first key point detection model uses the P2PNet model, and the P2PNet model has a backbone network, a feature pyramid network, a key point coordinate regression head, and a classification prediction head.

[0023] The above method for optimizing Helicobacter pylori pathological images based on dense area recognition is characterized in that: the first key loss function where L loc represents the regression loss function, represents the classification loss function, and L hp represents the difficult point reinforcement learning loss function; λ loc 、λ cls 、λ hp represent weights respectively.

[0024] The above method for optimizing Helicobacter pylori pathological images based on dense area recognition is characterized in that: the difficult point reinforcement learning loss function where K represents the number of difficult points selected during the training process, α represents the difficult point weight factor, and p i represents the true key point coordinates, represents the key point coordinates predicted by the model.

[0025] The above-mentioned Helicobacter pylori pathological image optimization method based on dense region recognition is characterized in that: represents the cross-entropy loss function in the classification loss function; L loc represents the Euclidean loss function in the regression loss function.

[0026] The above-mentioned Helicobacter pylori pathological image optimization method based on dense region recognition is characterized in that: in step 102, the dense region box of the densely distributed area of Helicobacter pylori is generated according to the first annotation information of the pathological image. The specific method is as follows:

[0027] Step 1021: Calculate the point matrix based on the point annotation data of the first annotation information; use the C matrix to record the connection quantity at each position in the point matrix;

[0028] Step 1022: Find the row with the largest connection number in the C matrix; find the minimum and maximum coordinates of all regions connected to this row, so as to obtain a merged dense region box d; expand and enlarge the dense region box d to generate a new dense region box; add the new dense region box to the region list D, and update the C matrix at the same time;

[0029] Step 1023: Repeat step 1022 to generate a new dense region box;

[0030] Step 1024: Screen and filter the dense region boxes. After m iterations, obtain the dense region box of the Helicobacter pylori distribution area.

[0031] The above-mentioned Helicobacter pylori pathological image optimization method based on dense region recognition is characterized in that: in step 202, the object detection model adopts the yolov7 model, and the object detection model has an input end, a backbone network, a neck network and a head network.

[0032] The above-mentioned Helicobacter pylori pathological image optimization method based on dense region recognition is characterized in that: in step 202, the object loss function where represents the coordinate regression loss function, represents the class confidence loss function, represents the object confidence loss function, λ coord 、λ obj 、λ conf represent weights respectively.

[0033] The above-mentioned Helicobacter pylori pathological image optimization method based on dense region recognition is characterized in that: represents the CIOU Loss loss function in the coordinate regression loss function; represents the binary cross-entropy loss function in the class confidence loss function, It represents the binary cross-entropy loss function in the target confidence loss function.

[0034] For the above-mentioned optimization method of Helicobacter pylori pathological images based on dense region recognition, it is characterized in that the specific method of "cropping and magnifying the predicted dense bounding boxes" is: magnifying the dense region image by two times using the bilinear interpolation or bicubic interpolation method.

[0035] The present invention has the following advantages compared with the prior art:

[0036] 1. The structure of the present invention is simple, reasonably designed, and convenient to implement and use.

[0037] 2. The present invention trains the first key point detection model based on the labeled Helicobacter pylori images. By identifying the key points of individual Helicobacter pylori, the morphological characteristics of Helicobacter pylori can be captured more accurately, reducing missed detections or false detections caused by sample morphological differences, thereby improving the accuracy of dense region recognition.

[0038] 3. The present invention generates dense region bounding boxes for the dense distribution regions of Helicobacter pylori based on point annotation data, making the dense region bounding boxes interpretable, capable of providing intuitive analysis basis for pathologists, and enhancing their trust in the algorithm output results.

[0039] 4. The present invention uses the dense region bounding boxes as the labels of the pathological images in the second data set to train the object detection model. When the pathological images to be optimized are input into the object detection model, the predicted dense bounding boxes of the dense distribution regions of Helicobacter pylori can be obtained, capable of more accurately predicting the positions and dense region ranges of Helicobacter pylori, reducing the situations of missed detections or false detections.

[0040] 5. The present invention maps the key point coordinates of the dense region image back to the corresponding positions of the pathological image to be optimized according to the original ratio, replacing the initial detection results of the pathological image to be optimized. The enhancement optimization based on the dense region bounding boxes can more specifically improve the image quality, making the details of Helicobacter pylori clearer and avoiding the loss of details or overprocessing caused by global enhancement.

[0041] 6. The present invention further constrains the first key point detection model by adding a difficult point reinforcement learning loss function to the loss function, which is used to increase the loss of Helicobacter pylori with relatively poor prediction results during the training process, so that the model focuses more attention on those Helicobacter pylori points with poor training effects and conducts reinforcement learning on them.

[0042] In summary, the structure of the present invention is simple and reasonably designed. By training the first key point detection model based on the labeled Helicobacter pylori images, the missed detection or misdetection caused by the morphological differences of the samples is reduced. The dense region boxes of the Helicobacter pylori distribution area are generated based on the point annotation data, which is interpretable. Using the dense region boxes as the labels of the pathological images in the second dataset to train the object detection model, the predicted dense boxes of the dense distribution area of Helicobacter pylori are obtained, accurately predicting the range of the dense area of Helicobacter pylori and reducing the situation of missed detection or misdetection.

[0043] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is the flowchart of the method of the present invention.

[0045] Figure 2 is the flowchart of the method for training the model of the present invention.

[0046] Figure 3 is the flowchart of the method for optimizing the pathological images of the present invention.

[0047] Figure 4 is the effect diagram of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments of the present invention.

[0049] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the drawings and embodiments.

[0050] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0051] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0052] For ease of description, spatial relative terms such as "above", "over", "on the upper surface", "above" etc. can be used here to describe the spatial positional relationship of a device or feature shown in the figure with other devices or features. It should be understood that spatial relative terms are intended to include different orientations in use or operation in addition to the orientation described in the figure of the device. For example, if the device in the figure is inverted, the device described as "above other devices or structures" or "over other devices or structures" will then be positioned as "below other devices or structures" or "under other devices or structures". Thus, the exemplary term "above" can include both the orientations of "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and corresponding interpretations are made for the spatial relative descriptions used here.

[0053] As Figure 1 、 Figure 2 and Figure 3 shown, a method for optimizing Helicobacter pylori pathological images based on dense region recognition of the present invention includes the following steps:

[0054] Step 1. Obtain a data set:

[0055] Step 101. Obtain pathological images containing Helicobacter pylori. The basic information and the first annotation information of the pathological images form the first data set, and the first annotation information includes at least the true coordinates of each Helicobacter pylori.

[0056] In this embodiment, the pathological images containing Helicobacter pylori refer to the pathological images of Helicobacter pylori stained by immunohistochemistry.

[0057] The pathological image optimization method of this application is applied to the pathological images of Helicobacter pylori, and can also be applied to common cell detection tasks, such as breast cancer mitotic cells.

[0058] In a possible embodiment, the basic information of the pathological image includes the size, resolution, and pixel value of the pathological image.

[0059] In a possible embodiment, the annotation information includes at least the true coordinates of each Helicobacter pylori. The central point coordinates of Helicobacter pylori are used as the annotation information. For approximately circular Helicobacter pylori cells, the central point coordinates can be determined by calculating the centroid of their pixel area. For Helicobacter pylori with irregular shapes, image analysis algorithms can also be used to find their approximate central positions.

[0060] In a possible embodiment, the annotation information can also include morphology, size, virulence factors, and genotyping. For example, the morphology of Helicobacter pylori includes spiral, S-shaped, and arc-shaped; the virulence factors of Helicobacter pylori include cytotoxin-associated gene A, vacuolating cytotoxin A, urease, and adhesin; genotyping includes genotyping based on the 16S rRNA gene sequence. According to the differences in the 16S rRNA gene sequence, Helicobacter pylori can be divided into multiple different subtypes, such as hpEurope and hpEastAsia. By annotating the morphology and size, the boundaries of individual Helicobacter pylori can be more clearly distinguished, thus accurately defining the scope of the dense area. By annotating the virulence factors and genotyping, the pathogenicity of the dense area can be predicted and the source can be traced, so as to deeply understand the infection mechanism and treatment effect.

[0061] Step 102: The basic information of the pathological image and the second annotation information form the second dataset. The second annotation information includes at least the dense area box of the densely distributed area of Helicobacter pylori. In a possible embodiment, some pathological images are selected from the first dataset to form the pathological images in the second dataset.

[0062] The dense area box of the densely distributed area of Helicobacter pylori in step 102 is generated according to the first annotation information of the pathological image. The specific method is as follows:

[0063] Step 1021: Calculate the point matrix based on the point annotation data of the first annotation information; use the C matrix to record the connection quantity at each position in the point matrix; in the first iteration process, since the true annotation of Helicobacter pylori in the image is presented in the form of point annotation, the point matrix is obtained by calculating the distance between points through the two-point distance formula to represent the distance situation between Helicobacter pylori points. In subsequent iteration processes, the point matrix is obtained by calculating the intersection over union (IoU) between dense area boxes to represent the overlapping degree between dense areas. Use the C matrix to record the connection quantity at each position in the point matrix. The specific operation is to assign 1 to the values in the point matrix that are higher than the threshold.

[0064] Step 1022: Find the row in the C matrix with the largest number of connections; find the minimum and maximum coordinates of all regions connected to this row, so as to obtain a merged dense region box d; expand the dense region box d to generate a new dense region box; add the new dense region box to the region list D, and update the C matrix at the same time.

[0065] Specifically, select the row with the largest number of connections in the C matrix, and find the minimum and maximum coordinates of all regions connected to the row with the largest number of connections to obtain a merged dense region box d. Expand the minimum and maximum coordinates of the dense region box d by σ pixels to magnify the box and generate a new dense region box. The newly generated dense region box is added to the region list D, and the C matrix is updated at the same time. Set the value of the corresponding row in the C matrix to zero, and select the next row with the largest number of connections.

[0066] Step 1023: Repeat Step 1022 to generate new dense region boxes;

[0067] Step 1024: Filter the dense region boxes. After m iterations, obtain the dense region boxes of the Helicobacter pylori distribution area. Filter all the dense region boxes obtained during this iterative process, and eliminate those dense region boxes with a small number of labeled points inside. This process is iteratively merged m times to obtain the final dense region boxes of the Helicobacter pylori distribution area in the pathological image.

[0068] The rules for point annotation are simple and clear. It is easier for different annotators to keep the annotation results of the same Helicobacter pylori consistent, reducing the subjective differences during the annotation process. Moreover, compared with directly annotating the dense region box, point annotation is easier to implement, significantly reducing the annotation time and labor costs. At the same time, point annotation is usually based on the significant features of Helicobacter pylori, such as the central coordinates. The dense region box generated based on point annotation is more in line with pathological knowledge, enhancing the interpretability of the model and being able to provide intuitive analysis basis for pathologists and enhancing their trust in the algorithm output results.

[0069] Step Two: Train the model:

[0070] Step 201: Train the first key point detection model based on the first data set, define the first key loss function, and obtain the trained first key point detection model.

[0071] In a possible embodiment, the data set in Step One includes at least 500 pathological images containing Helicobacter pylori. Randomly divide the data set in Step One according to the ratio of training set, validation set, and test set as 6∶2∶2. The training set is used to train the key point detection model.

[0072] In another possible embodiment, when the method of the present application is applied to the optimization of breast cancer mitotic cell pathological images, the dataset used can be the CBIS-DDSM dataset or the BreakHis dataset.

[0073] In a possible embodiment, the first key point detection model adopts the P2PNet model, and the P2PNet model has a backbone network, a feature pyramid network, a key point coordinate regression head, and a classification prediction head.

[0074] The backbone network adopts the VGG-16 network. The VGG-16 network extracts the feature maps corresponding to 5 groups of convolutional layers, providing comprehensive feature information for subsequent tasks. The feature pyramid network is used to fuse the multi-level features extracted by the backbone network to generate feature maps with rich semantic information and relatively high resolution; the key point coordinate regression head uses a regression network to predict the precise coordinates of Helicobacter pylori key points on the feature map; the classification prediction head uses three convolutional operations to predict the classification confidence scores of each detected Helicobacter pylori key point.

[0075] The first key loss function where L loc represents the regression loss function, represents the classification loss function, and L hp represents the hard point reinforcement learning loss function; λ loc represents the weight of the regression loss function, λ cls represents the weight of the classification loss function, and λ hp represents the weight of the hard point reinforcement learning loss function.

[0076] To further constrain the first key point detection model, a hard point reinforcement learning loss function L hp is added to the first key loss function. The loss function of the P2PNet model is the weighted sum of the regression loss function L loc , the classification loss function , and the hard point reinforcement learning loss function L hp . The weighted sum loss function can better adapt to the characteristics of medical images, balance the localization accuracy and classification confidence, thereby improving the overall performance.

[0077] The hard point reinforcement learning loss function where K represents the number of hard points selected during the training process, α represents the hard point weight factor, and p i represents the true key point coordinates, represents the key point coordinates predicted by the model. The hard point reinforcement learning loss function L hpUsed to increase the loss of Helicobacter pylori points with poor prediction results during the training process, so that the model focuses more attention on those Helicobacter pylori points with poor training effects and conducts reinforcement learning on them.

[0078] Specifically, difficult points are defined as those points with a large mean squared error value between the predicted points and the matching ground truth annotation points p i in the regression branch. In the model, the top K predicted points with the largest mean squared error values are selected, and a difficult point weight factor α is added to their regression loss function to strengthen the model's learning of Helicobacter pylori points during the training process, thereby achieving precise localization of Helicobacter pylori. In a possible embodiment, the value of K is set to the floor of the number of ground truth annotation points of Helicobacter pylori in each image divided by 10, and the difficult point weight factor α is set to 1.5.

[0079] L loc represents the Euclidean loss function in the regression loss function, ensuring that the model can accurately predict the positions of Helicobacter pylori key points. represents the cross-entropy loss function in the classification loss function, ensuring that the model can correctly judge the existence of key points.

[0080] Step 202: Train the object detection model based on the second dataset, define the object loss function, and obtain the trained object detection model.

[0081] The object detection model uses the yolov7 model, and the object detection model has an input end, a backbone network, a neck network, and a head network. The input end preprocesses and augments the input Helicobacter pylori pathological images to adapt to Helicobacter pylori pathological images in different situations; the backbone network adopts the CBS structure to extract features of different scales and levels of Helicobacter pylori pathological images, reducing the computational amount and improving the feature extraction ability through partial cross-stage connections; the neck network adopts the feature pyramid network structure or the path aggregation network structure to fuse multi-scale features and generate feature maps suitable for object detection, enabling the model to simultaneously focus on the local details and overall context information of Helicobacter pylori, and improving the detection accuracy of Helicobacter pylori dense regions with different sizes and distributions; the head network of the yolov7 model adopts a decoupled head design, and the head network predicts the bounding box coordinates of each anchor point through convolutional layers to obtain the bounding boxes and confidence levels of Helicobacter pylori dense regions. In Helicobacter pylori pathological images, in this way, the bounding boxes of Helicobacter pylori dense regions can be obtained, and at the same time, the confidence level corresponding to each bounding box can also be output.

[0082] In step 202, where represents the coordinate regression loss function, represents the class confidence loss function, Denotes the target confidence loss function, λ coord Denotes the weight of the coordinate regression loss function, λ obj Denotes the weight of the class confidence loss function, λ conf Denotes the weight of the target confidence loss function.

[0083] In a possible embodiment, Denotes the CIOU Loss function in the coordinate regression loss function; Denotes the binary cross-entropy loss function in the class confidence loss function, Denotes the binary cross-entropy loss function in the target confidence loss function.

[0084] Input the labeled pathological image into the yolov7 model, perform forward propagation, and calculate the predicted dense region boxes and class probabilities. Then, calculate the loss function based on the prediction results and the true labels; next, through the backpropagation algorithm, calculate the gradient of the loss function with respect to the network parameters, and use the optimizer to update the network parameters to minimize the loss function and obtain the trained object detection model.

[0085] Step 203: Obtain the predicted dense boxes of the Helicobacter pylori dense distribution area in the pathological image through the trained object detection model; crop and enlarge the predicted dense boxes to obtain the dense region image.

[0086] The specific method of "cropping and enlarging the predicted dense boxes" is: use bilinear interpolation or bicubic interpolation method to enlarge the dense region image by two times.

[0087] In a possible embodiment, a pathological image obtains a set of predicted dense boxes B, B = {b1,..., b i ,..., b r}, where b i represents the i-th predicted dense box, b i = (x i , y i , w i , h i ,), (x i , y i ) represents the upper left corner coordinates of the predicted dense box, w i represents the width of the predicted dense box, and h i represents the height of the predicted dense box.

[0088] Train an object detection model using the dense region bounding boxes as the labels of the second dataset, and predict the predicted dense bounding boxes of the densely distributed regions of Helicobacter pylori in the pathological images through the object detection model, enabling the object detection model to have the function of dense region recognition, reducing the annotation and calculation costs, and improving the accuracy and efficiency of pathological image analysis.

[0089] Step 204: Train a second key point detection model based on the dense region images, define a second key loss function, and obtain a trained second key point detection model.

[0090] When training the second key point detection model, the labels of the dense region images are the key point coordinates corresponding to Helicobacter pylori in these images. The model continuously compares its prediction results with these known key point coordinate labels, calculates the loss value through the second key loss function, and adjusts the parameters of the model according to the loss value, so as to gradually learn how to accurately output the key point coordinates corresponding to Helicobacter pylori from the dense region images.

[0091] It should be noted that the second key loss function is also composed of a weighted combination of a regression loss function, a classification loss function, and a difficult point reinforcement learning loss function.

[0092] Step Three: Optimize the pathological images:

[0093] Step 301: Obtain the pathological images to be optimized; the pathological images are pathological images containing Helicobacter pylori. In this embodiment, the pathological images containing Helicobacter pylori refer to the pathological images of Helicobacter pylori stained by immunohistochemistry. Optimizing the pathological images containing Helicobacter pylori is of great significance for the diagnosis of diseases such as gastritis and gastric ulcer.

[0094] In a possible embodiment, the pathological image optimization method of the present application can also be applied to common cell detection tasks, such as in breast cancer pathological images, optimizing breast cancer mitotic cells.

[0095] Step 302: Obtain the initial detection results of the pathological images to be optimized through the trained first key point detection model.

[0096] By identifying the key points and key point credibility of individual Helicobacter pylori, the morphological characteristics of Helicobacter pylori can be captured more accurately, reducing missed detections or false detections caused by sample morphological differences, thereby improving the accuracy of dense region recognition. Moreover, the key point-based recognition method can adapt to different staining methods, imaging devices, or sample types.

[0097] In a possible embodiment, the initial detection result includes the initial key point coordinates and the confidence of the initial key points. A confidence threshold is set to filter out the key points with a confidence higher than the confidence threshold; for the selected initial key points, the non-maximum suppression algorithm is used to remove redundant points, avoiding the same initial key point being detected multiple times, ensuring that only one optimal result is retained for each initial key point, and obtaining the deduplicated initial key point coordinates and their confidence scores.

[0098] Step 303: The pathological image to be optimized is successively passed through a trained object detection model, cropped, enlarged, and a second key point detection model to obtain the predicted key point coordinates corresponding to Helicobacter pylori in the dense region image.

[0099] The object detection model outputs the dense region image of the pathological image to be optimized; the dense region image of the pathological image to be optimized is cropped and enlarged, and then used as the input of the second key point detection model. The second key point detection model can further identify Helicobacter pylori in the enlarged dense region image, capture the morphological characteristics of Helicobacter pylori more accurately, obtain more subtle and key detail information of Helicobacter pylori, and output the predicted key point coordinates corresponding to Helicobacter pylori in the dense region image.

[0100] The predicted key point coordinates of Helicobacter pylori correspond to the significant features of Helicobacter pylori, have clear pathological significance, can intuitively reflect the position and morphological characteristics of Helicobacter pylori, help doctors understand the decision-making basis of the model, and at the same time deeply learn the association between Helicobacter pylori points in the dense region, thereby improving the recognition accuracy of Helicobacter pylori in the dense region.

[0101] Step 304: Map the key point coordinates of the dense region image back to the corresponding positions of the pathological image to be optimized according to the original ratio, and replace the initial detection result of the pathological image to be optimized to obtain the optimized pathological image.

[0102] Since there is a certain proportional relationship between the enlarged dense region image and the pathological image to be optimized, in order to accurately correspond the detection result obtained on the dense region image to the pathological image to be optimized, a mapping operation is required.

[0103] Assume the magnification factor is k, and the key point coordinates detected on the dense area image are (x', y'). Then the calculation formula for mapping them back to the coordinates (x, y) of the pathological image to be optimized is: x = x' / k, y = y' / k. For example, when the magnification factor is 2 and a key point coordinate detected on the dense area image is (200, 300), then the coordinates mapped back to the pathological image to be optimized are (100, 150). In the pathological image to be optimized, for the corresponding dense area covered by the previous initial detection result, replace it with the newly mapped detection result. The new detection result is based on magnification and more refined analysis, and can more accurately reflect the actual situation of Helicobacter pylori in this area.

[0104] Dense area recognition can accurately locate the infected area, visually display the infected area, enabling doctors to comprehensively understand the condition, while enhancement and optimization can more clearly show the bacterial morphology, avoid missed and misdiagnoses to improve the diagnostic accuracy, thus providing a more reliable basis for clinical diagnosis and treatment.

[0105] As Figure 4 shown, the first row is the pathological image to be optimized, the second row is the truly labeled pathological image, the third row is the pathological image before replacement, and the fourth row is the pathological image after replacement. In the replaced image, the detection position of Helicobacter pylori is closer to the true label. Compared with before replacement, it can more accurately locate Helicobacter pylori, reducing false detections and missed detections. Moreover, the details are presented more clearly. The replaced pathological image can more clearly present details such as the morphology of Helicobacter pylori, which helps for more accurate analysis and judgment.

[0106] For example, taking Figure 4 the first column of images as an example, in the pathological image of the second row in the first column, the green marked points within the dashed box, that is, Helicobacter pylori, are relatively dispersed and have a certain pattern in quantity. In the replaced pathological image of the fourth row in the first column, the quantity and distribution of the red marked points are closer to those in the second row; while in the pathological image before replacement of the third row in the first column, the quantity of the red marked points is large and the distribution is messy, with a large difference from the second row.

[0107] For example, taking Figure 4 the fourth column as an example, in the pathological image of the third row in the fourth column, the initial detection result detected 5 Helicobacter pylori points within the dense area. After magnification and detection by P2PNet and mapping back to the pathological image to be optimized, that is, Figure 4 the pathological image of the fourth row in the fourth column, 10 Helicobacter pylori points are detected in this area. At this time, replace the original detection result of 5 points with the new detection result of 10 points.

[0108] The above are only embodiments of the present invention and do not impose any limitations on the present invention. Any simple modifications, changes, and equivalent structural changes made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A Helicobacter pylori pathological image optimization method based on dense area recognition, characterized in that: The following steps are involved: Step 1: Get the dataset: Step 101: Acquire a pathological image containing Helicobacter pylori, wherein basic information of the pathological image and first annotation information constitute a first data set, and the first annotation information at least includes the real coordinates of each Helicobacter pylori; Step 102: The basic information of the pathological image and the second annotation information constitute a second data set, and the second annotation information at least includes a dense area frame of the densely distributed area of ​​Helicobacter pylori; Step 2: Training model: Step 201: training a first key point detection model based on a first data set, defining a first key loss function, and obtaining a trained first key point detection model; Step 202: training a target detection model based on the second data set, defining a target loss function, and obtaining a trained target detection model; Step 203: Obtain a predicted dense frame of the densely distributed area of ​​Helicobacter pylori in the pathological image through the trained target detection model; crop and enlarge the predicted dense frame to obtain a dense area image; Step 204: training a second key point detection model based on the dense area image, defining a second key loss function, and obtaining a trained second key point detection model; Step 3: Optimize pathological images: Step 301: Acquire a pathological image to be optimized; Step 302: Obtaining an initial detection result of the pathological image to be optimized through the trained first key point detection model; Step 303: The pathological image to be optimized is subjected to the target detection model, cropping, enlargement and the second key point detection model to obtain the predicted key point coordinates corresponding to Helicobacter pylori in the dense area image; Step 304: The predicted key point coordinates are mapped back to the pathological image to be optimized according to the original ratio, replacing the initial detection results of the pathological image to be optimized, and obtaining the optimized pathological image.

2. A Helicobacter pylori pathological image optimization method based on dense area recognition according to claim 1, characterized in that: The first key point detection model adopts the P2PNet model, which has a backbone network, a feature pyramid network, a key point coordinate regression head, and a classification prediction head.

3. A Helicobacter pylori pathological image optimization method based on dense area recognition according to claim 1 or 2, characterized in that: The first key loss function Where L loc represents the regression loss function, represents the classification loss function, L hp Represents the loss function of the difficult point reinforcement learning; λ loc , cls , hp Represent weights respectively.

4. A Helicobacter pylori pathological image optimization method based on dense area recognition according to claim 3, characterized in that: Difficulty point reinforcement learning loss function Where K represents the number of difficult points selected during training, α represents the difficult point weight factor, and p i represents the real key point coordinates, Represents the keypoint coordinates predicted by the model.

5. A Helicobacter pylori pathological image optimization method based on dense area recognition according to claim 3, characterized in that: Represents the cross entropy loss function in the classification loss function; L loc Represents the Euclidean loss function in the regression loss function.

6. A Helicobacter pylori pathological image optimization method based on dense area recognition according to claim 1, characterized in that: In step 102, the dense area frame of the densely distributed area of ​​Helicobacter pylori is generated according to the first annotation information of the pathological image, and the specific method is as follows: Step 1021, calculating a point matrix according to the point annotation data of the first annotation information; using a C matrix to record the number of connections at each position in the point matrix; Step 1022: find the row with the maximum number of connections in the C matrix; find the minimum and maximum coordinates of all regions connected to the row, thereby obtaining a merged dense region frame d; Expand and enlarge the dense region frame d to generate a new dense region frame; the new dense region frame is added to the region list D, and the C matrix is ​​updated at the same time; Step 1023, repeat step 1022 to generate a new dense area frame; Step 1024: Filter the dense area frame, and after m iterations, obtain the dense area frame of the Helicobacter pylori distribution area.

7. The method for optimizing Helicobacter pylori pathological images based on dense area recognition according to claim 1, characterized in that: In step 202, the target detection model adopts the yolov7 model, and the target detection model has an input end, a backbone network, a neck network and a head network.

8. A Helicobacter pylori pathological image optimization method based on dense area recognition according to claim 1 or 7, characterized in that: In step 202, the target loss function in represents the coordinate regression loss function, represents the category confidence loss function, represents the target confidence loss function, λ coord , obj , conf Represent weights respectively.

9. A Helicobacter pylori pathological image optimization method based on dense area recognition according to claim 8, characterized in that: Represents the CIOU Loss loss function in the coordinate regression loss function; represents the binary cross entropy loss function in the category confidence loss function, Represents the binary cross entropy loss function in the target confidence loss function.

10. The method for optimizing Helicobacter pylori pathological images based on dense area recognition according to claim 1, characterized in that: The specific method of "cropping and enlarging the predicted dense frame" is: using a bilinear interpolation method or a bicubic interpolation method to enlarge the dense area image by two times.

Images