Granuloma circular representation positioning method and system based on multi-scale feature aggregation

Abstract

The application discloses a granuloma circular representation positioning method and system based on multi-scale feature aggregation, first, collecting a histopathological image; then, performing preliminary feature encoding and extraction through a Patch Embedding module of a detection network; then, obtaining multi-scale aggregation feature output through a plurality of multi-scale attention feature extraction stages of the detection network; further, performing different scale feature fusion from top to bottom on the output features of each feature extraction stage through a feature fusion module FPN of the detection network; finally, positioning and detecting the granuloma in the histopathological image through three detection heads of the detection network; the application uses a Transformer structure based on multi-scale feature aggregation to extract image features, so that effective learning of a plurality of different scale features is realized, and the effect that both large targets and small targets can be effectively detected is achieved.

Description

Technical Field

[0001] This invention belongs to the field of deep learning and medical image recognition technology, and relates to a method and system for localizing circular representations of granulomas. Specifically, it relates to a method and system for localizing circular representations of granulomas based on multi-scale feature aggregation, which belongs to the application of deep learning in the medical field. Background Technology

[0002] Granulomas are well-defined nodular lesions formed by the localized infiltration and proliferation of macrophages and their evolved cells. Their essence is inflammation caused by delayed-type hypersensitivity reactions. Based on the different causes of granulomas, they can be classified into the following categories: infectious granulomas, caused by bacterial or fungal infections such as Mycobacterium tuberculosis and Mycobacterium leprae; foreign body granulomas, chronic inflammation caused by the long-term presence of foreign bodies such as surgical sutures, talcum powder, and artificial blood vessels; and sarcoidosis granulomas, non-necrotizing epithelial cell granulomas that occur during sarcoidosis. Taking the very common pulmonary granulomas as an example, microbial infectious granulomas are the most common. Granulomas cause granulomatous inflammation, and if not treated promptly, the infection can spread to the entire lung, causing pulmonary fibrosis, cavitation, and changes in lung structure, potentially affecting lung function, causing systemic inflammatory response syndrome, and even death. Furthermore, if malignant components are found in the granulomatous lesion, there is a possibility of cancerous transformation; without timely intervention and treatment, the consequences may be even more serious. Therefore, timely detection of granulomas allows patients to receive treatment earlier, preventing further spread of infection and reducing harm to the patient's body. Pathological examination is the gold standard for granuloma diagnosis in clinical practice and is also of great significance in differentiating between benign and malignant granulomas. Computer-aided pathological diagnostic methods can automatically locate granuloma areas, which can greatly reduce the workload of pathologists. At the same time, because computer-automated localization methods identify granulomas in a short time, this method can also buy patients more time, which has important practical significance.

[0003] Granulomas in histopathological sections exhibit relatively distinct and specific structures, often round or oval, making their features relatively easy to learn. However, automated granuloma localization methods, such as using existing single-stage object detection models like YOLO and two-stage object detection models like Faster R-CNN, or traditional texture-based manual feature extraction methods, still face several challenges. First, the shape and structural features of granulomas are quite pronounced, requiring more effective attention and utilization of these features. Second, different granulomas within the same histopathological section can exhibit significant size variations, further complicating the granuloma localization task. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for locating circular representations of granulomas based on multi-scale feature aggregation.

[0005] The technical solution adopted by the method of the present invention is: a method for localizing granuloma circles based on multi-scale feature aggregation, comprising the following steps:

[0006] Step 1: Acquire tissue pathology images;

[0007] Step 2: Input the acquired histopathological images into the detection network, and perform preliminary feature encoding and extraction through the PatchEmbedding module of the detection network;

[0008] The Patch Embedding module consists of several convolutions with strides, the number of which and the strides can be set as needed.

[0009] Step 3: Input the output of Step 2 into multiple consecutive multi-scale attention feature extraction stages of the detection network to obtain multi-scale feature output;

[0010] The multi-scale attention feature extraction stage includes several multi-scale attention feature aggregation modules (Attention Blocks) and Linear Embedding modules; the number of Attention Blocks is generally set according to the rule of first increasing and then decreasing, and is set as needed.

[0011] The Linear Embedding module adopts a similar structure to the Patch Embedding module, differing only in the number of convolutional layers and the downsampling rate, which can be set as needed.

[0012] The multi-scale attention feature aggregation module, Attention Block, has n key-value (KV) computation branches and one Q-computation branch. Each KV computation branch sequentially includes a downsampling convolutional layer, a fully connected layer, a KV splitting layer, a depthwise convolutional layer, and a residual enhancement layer. The Q-computation branch has one fully connected layer and one Q-splitting layer, with the Q-splitting layer producing n split outputs. i The outputs of the residual enhancement layer of each of the n KV computation branches are respectively processed by the self-attention computation layer. After the self-attention layer, the outputs of the n branches are merged by a concat layer. Then, a fully connected layer, a deep convolutional layer, a residual enhancement layer and a fully connected layer are set in sequence.

[0013] Step 4: Input the output of each stage in Step 3 into the Feature Fusion Module (FPN) of the detection network to perform top-down feature fusion at different scales on the output features of each feature extraction stage in Step 3.

[0014] Step 5: Input the fusion features output in Step 4 into the three detection heads of the detection network to locate and detect granulomas in the histopathological image;

[0015] Each of the three detection heads consists of two convolutional layers, one activation layer, and one batch normalization layer.

[0016] Preferably, in step 2, the Patch Embedding module generates an input sequence X of size C×H×W through convolution with stride, where H0 and W0 are the length and width of the input histopathological image, respectively.

[0017] Preferably, in step 3, self-attention calculation and feature aggregation are performed through the multi-scale attention feature aggregation module Attention Block;

[0018] The specific implementation includes the following steps:

[0019] Step 3.1: By setting n different downsampling rates r i The input sequence X is convolved with a kernel and stride of r. i The convolution operation yields X samples with different sampling rates. i At this time, X i The size is Where C, H, and W represent the number of convolution channels, the length of the image, and the width of the image, respectively;

[0020] Step 3.2: For After performing a full connection operation, it is split into two parts, denoted as K. i and V i ;

[0021] Step 3.3: Process the split V i Perform depthwise convolution and residual enhancement, i.e., V i " = V i +LC(V i );

[0022] Step 3.4: Perform a fully connected operation on the input sequence X to obtain Q in the self-attention calculation, and split Q into n Q's. i sequence;

[0023] Step 3.5: Using Q i K i V i "Perform self-attention calculations," Where d is the dimension and T represents the transpose;

[0024] Step 3.6: For n x i The sequences are concatenated to obtain the self-attention sequence output Y.

[0025] Step 3.7: Further supplement local information through two fully connected layers and depthwise convolution between the fully connected layers, i.e., Y′=FC(Y) and Z=FC(σ(Y′+LC(Y′))), where σ is the activation function and FC() is the fully connected operation.

[0026] Preferably, in step 4, the feature Z output from the deep feature extraction stage is used... i Sampling to the shallow feature extraction stage outputs feature Z i-1 After reaching the same size, with Z i-1 Addition achieves the fusion of features at different scales in the opposite direction.

[0027] Preferably, in step 5, the three detection heads are a center point detection head, a radius detection head, and a center point offset detection head;

[0028] The three detection heads all output feature maps of the same size. The only difference is the number of channels; R is the downsampling ratio of the output feature map compared to the network input image.

[0029] The center point detection head outputs a feature map with c channels. Each channel of the feature map represents the center point position of the current category object, specifically represented by the feature value at the center point. Eigenvalues ​​tending towards 1 rather than at the center point Approaching 0;

[0030] The radius detection head outputs a feature map. With 1 channel, each value in the output feature map represents the radius of the granuloma at that point.

[0031] The center point offset detection head outputs a feature map. The number of channels is 2, and the two channels represent the offset of the center point on the horizontal and vertical axes due to the non-integer quantization during sampling.

[0032] Preferably, the detection network is a trained detection network;

[0033] The training includes the following steps:

[0034] (1) Collect several histopathological sections, and have a professional pathologist mark the granuloma area with a circle to obtain the center coordinates and radius of each granuloma circle;

[0035] (2) Input the labeled histopathological images into the detection network for network training. The loss function is composed of the joint loss of the three detectors, i.e., L = L c +λ r L r +λ off L off ; where λ r and λ off It is a constant used to balance the losses of various items; L c L r L off These represent the center point loss function, radius loss function, and offset loss function, respectively.

[0036] For the granuloma center point p = (p in the input image) x p y The output corresponding to the center point detection head is... in Where R represents the radius of the circle; the center point detection head is constructed using Gaussian kernel transformation to output the corresponding true label, where the value at the feature map coordinate point (x, y) in category c is... The center point loss is calculated using focal loss, and the specific calculation formula is as follows:

[0037]

[0038] Where α and β are hyperparameters in focal loss, and N is the number of center points; The output of the center point detection head is the value at the coordinate point (x, y) of the c-th channel. The standard deviation is Gaussian.

[0039] Both radius loss and offset loss are calculated using L1 loss, and the specific calculation formulas are as follows:

[0040]

[0041]

[0042] in The radius prediction value at the center point pk This is the true value of the radius at this location. To output the center point of the feature map The predicted offset at that location;

[0043] The set of granuloma centroids detected by the centroid detector during the prediction process is as follows: n is the number of center points; then each granuloma target is represented as a center point. radius The circular area, in which and p, respectively, predicted by the offset detection head i The offset of the x and y coordinates at the point. p for radius prediction head prediction i The radius of the target at the point.

[0044] The technical solution adopted by the system of the present invention is: a granuloma circular representation and localization system based on multi-scale feature aggregation, comprising:

[0045] One or more processors;

[0046] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the method for localizing granuloma circles based on multi-scale feature aggregation.

[0047] The beneficial effects of this invention are as follows:

[0048] (1) The present invention locates the target by circular representation instead of using conventional rectangular bounding box representation. This representation method is more suitable for circular structural targets such as granulomas, thus making more effective use of the target's morphological features. Moreover, the circular representation has rotation invariance compared to the rectangular bounding box representation, and will not produce completely different rectangular bounding box annotations due to the rotation of the input image.

[0049] (2) This invention uses a Transformer structure based on multi-scale feature aggregation to extract image features, thereby achieving effective learning of features at various scales and enabling effective detection of both large and small targets. In terms of implementation, the network first introduces attention computation that fuses features at multiple scales in the Transformer module during the feature extraction stage. Then, it uses FPN to perform reverse feature fusion on features at different scales in each stage. Finally, it uses the target circle to represent the three required detection heads to determine the target center and radius, thereby locating the final target region. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a network structure diagram for an embodiment of the present invention;

[0052] Figure 2This is a structural diagram of the AttentionBlock module in an embodiment of the present invention;

[0053] Figure 3 This is a structural diagram of the detection head layer in an embodiment of the present invention;

[0054] Figure 4 This is an example image of granulomas in a lung tissue pathology section used in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Please see Figure 1 , Figure 2 and Figure 3 This embodiment provides a method for locating circular representations of granulomas based on multi-scale feature aggregation, including the following steps:

[0057] Step 1: Acquire tissue pathology images;

[0058] Step 2: Input the acquired histopathological images into the detection network, and perform preliminary feature encoding and extraction through the Patch Embedding module of the detection network;

[0059] In this embodiment, the Patch Embedding module generates an input sequence X of size C×H×W through three convolutions with strides, where H0 and W0 are the length and width of the input histopathological image, respectively.

[0060] Step 3: Input the output of Step 2 into multiple consecutive multi-scale attention feature extraction stages of the detection network to obtain multi-scale feature output;

[0061] Please see Figure 1Each multi-scale attention feature extraction stage (Stage1, Stage2, Stage3, Stage4) in this embodiment includes several multi-scale attention feature aggregation modules (Attention Blocks) and Linear Embedding modules. The number of Attention Blocks is generally set according to the rule of increasing first and then decreasing, and is set according to the specific needs. The initial and final stages are mainly used to change the feature map size (reduce the amount of computation), so the number of Attention Blocks required is small, while the intermediate stages achieve attention feature extraction by stacking multiple Attention Blocks.

[0062] In this embodiment, the Embedding module in Stage 1 is the Patch Embedding module in step 2; while the Linear Embedding in subsequent Stages is a convolutional layer with a downsampling rate of 2×2.

[0063] In this embodiment, the number of AttentionBlocks in each feature extraction stage (Stage1, Stage2, Stage3, Stage4) is 2, 4, 12, and 1, respectively.

[0064] The Linear Embedding module in this embodiment adopts a similar structure to the Patch Embedding module, differing only in the number of convolutional layers and the downsampling rate (the Patch Embedding downsampling rate is set to 4, and the Linear Embedding downsampling rate is 2). The Linear Embedding uses a smaller downsampling rate because the Patch Embedding module is located at the beginning of the network and needs to significantly reduce the feature map size to reduce the overall computational cost of the network.

[0065] Please see Figure 2 The multi-scale attention feature aggregation module (Attention Block) in this embodiment has two KV computation branches and one Q computation branch; each KV computation branch sequentially sets downsampling convolutional layers (r i ×r i The code consists of a covn layer, a fully connected linear layer, a KV split layer, a depthwise convolutional layer (Dwconv), and a residual enhancement layer. The Q-computation branch has one fully connected linear layer and one Q-split layer, with the Q-split layer producing two split outputs. The two split Q-values... iThe outputs of the residual enhancement layers of the two KV computation branches are respectively processed by a self-attention layer. After the self-attention layer, a concat layer is used to merge the outputs of the two branches. Then, a fully connected layer (Linear), a deep convolutional layer (Dwconv), a residual enhancement layer and a fully connected layer (Linear) are set sequentially after it.

[0066] In this embodiment, self-attention calculation and feature aggregation are performed through the multi-scale attention feature aggregation module Attention Block;

[0067] The specific implementation includes the following steps:

[0068] Step 3.1: By setting two different downsampling rates r i The input sequence X is convolved with a kernel and stride of r. i The convolution operation yields X samples with different sampling rates. i At this time, X i The size is Where C, H, and W represent the number of convolution channels, the length of the image, and the width of the image, respectively;

[0069] Step 3.2: For After performing a full connection operation, it is split into two parts, denoted as K. i and V i ;

[0070] Step 3.3: Process the split V i Perform depthwise convolution and residual enhancement, i.e., V i " = V i +LC(V i ); where LC(·) is grouped convolution, which achieves the goal of preserving positional information as much as possible while performing representation learning by performing convolution on the features of each channel separately;

[0071] Step 3.4: Perform a fully connected operation on the input sequence X to obtain Q in the self-attention calculation, and divide Q into two Q's. i sequence;

[0072] Step 3.5: Using Q i K i V i "Perform attention calculations," Where d is the dimension and T represents the transpose;

[0073] Step 3.6: For the two x's i The sequences are concatenated to obtain the self-attention sequence output Y.

[0074] Step 3.7: Further supplement local information through two fully connected layers and depthwise convolution between the fully connected layers, i.e., Y′=FC(Y) and Z=FC(σ(Y′+LC(Y′))), where σ is the activation function and FC() is the fully connected operation.

[0075] In the specific implementation process, the multi-scale features used by the Attention Blocks in the first three stages are sequence features with downsampling rates of 8 and 4, 4 and 2, and 2 and 1, respectively, while the last stage uses a single original scale feature for self-attention calculation.

[0076] In this embodiment, the image feature representation output in step 2 is input into several consecutive Stages to obtain the multi-scale aggregated feature output {Z1, Z2, Z3, Z4} based on the Transformer structure in each Stage;

[0077] Step 4: Input the feature fusion module FPN of the detection network into the output of each stage in Step 3, and perform top-down feature fusion of different scales on the output features of each feature extraction stage in Step 3;

[0078] This embodiment uses an FPN structure to perform top-down feature fusion of different scales on the output features {Z2, Z3, Z4} of each Stage to obtain a multi-scale fused feature F. Specifically, the fusion method is to start from feature Z4 and fuse features Z... i Sampling to shallow Stage output features Z i-1 After reaching the same size, with Z i-1 The addition yields the new feature Z after fusion. i-1 This operation is repeated until the features are added to feature Z2 to achieve reverse feature fusion at different scales. It should be noted that Z1 is not included because its feature map scale is large and requires a large amount of computing resources. Therefore, in the implementation process, some scale features can be selectively used to construct the fused features.

[0079] Step 5: Input the fusion features output in Step 4 into the three detection heads of the detection network to locate and detect granulomas in the histopathological image;

[0080] Please see Figure 3 In this embodiment, each of the three detection heads consists of two convolutional layers, one activation layer, and one batch normalization layer.

[0081] In this embodiment, the three detection heads all output feature maps of the same size. The only difference is the number of channels; R is the downsampling ratio of the output feature map compared to the network input image.

[0082] The center point detection head in this embodiment outputs a feature map with c channels (the number of object categories to be detected; in this application scenario, only granulomas need to be detected, so the center point detection head outputs 1 channel). Each channel of the feature map represents the center point position of the current category of object, specifically represented by the feature value at the center point. Eigenvalues ​​tending towards 1 rather than at the center point Approaching 0;

[0083] The radius detection head in this embodiment outputs a feature map. With 1 channel, each value in the output feature map represents the radius of the granuloma at that point (meaningful only in the granuloma region);

[0084] The center point offset detection head in this embodiment outputs a feature map. The number of channels is 2, and the two channels represent the offset of the center point on the horizontal and vertical axes due to the non-integer quantization during sampling.

[0085] The detection network in this embodiment is a pre-trained detection network; training includes the following steps:

[0086] (1) Collect several histopathological sections, and have a professional pathologist mark the granuloma area with a circle to obtain the center coordinates and radius of each granuloma circle;

[0087] Please see Figure 4 This is an example image of granulomas in a lung tissue pathology section used in the embodiment. In the image, a professional pathologist marked the granuloma areas with circles, obtaining the center coordinates and radius of each granuloma circle.

[0088] (2) Input the labeled histopathological images into the detection network for network training. The loss function is composed of the joint loss of the three detectors, i.e., L = L c +λ r L r +λ off L off ; where λ r and λ off It is a constant used to balance the losses of various items; L c L r L off These represent the center point loss function, radius loss function, and offset loss function, respectively.

[0089] In this embodiment, for the granuloma center point p = (p) in the input image... x p y The output corresponding to the center point detection head is... in Where R represents the radius of the circle; the center point detection head is constructed using Gaussian kernel transformation to output the corresponding true label, where the value at the feature map coordinate point (x, y) in category c is... The center point loss is calculated using focal loss, and the specific calculation formula is as follows:

[0090]

[0091] Where α and β are hyperparameters in focal loss, typically set to α = 2 and β = 4, and N is the number of center points; The output of the center point detection head is the value at the coordinate point (x, y) of the c-th channel. The standard deviation is Gaussian.

[0092] Both radius loss and offset loss are calculated using L1 loss, and the specific calculation formulas are as follows:

[0093]

[0094]

[0095] in Center point p k The predicted radius at that location This is the true value of the radius at this location. To output the center point of the feature map The predicted offset at that location;

[0096] The set of granuloma centroids detected by the centroid detector during the prediction process is as follows: n is the number of center points; then each granuloma target is represented as a center point. radius The circular area, in which and p, respectively, predicted by the offset detection head i The offset of the x and y coordinates at the point. p for radius prediction head prediction i The radius of the target at the point.

[0097] Granulomas are a common disease, and effective and automated granuloma localization techniques are crucial for reducing harm to patients. Addressing the two major challenges currently faced in automated granuloma localization, this invention proposes a circular representation-based granuloma localization method. This method uses a Transformer structure based on multi-scale feature aggregation to extract image features, achieving effective identification of both large and small targets. Furthermore, using circular representation to locate targets more effectively utilizes their morphological features, further improving the accuracy of the automated granuloma localization model.

[0098] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for locating granuloma circles based on multi-scale feature aggregation, characterized in that, Includes the following steps: Step 1: Acquire tissue pathology images; Step 2: Input the acquired histopathological images into the detection network, and perform preliminary feature encoding and extraction through the Patch Embedding module of the detection network; The Patch Embedding module consists of several convolutions with strides, the number of which and the strides can be set as needed. Step 3: Input the output of Step 2 into multiple consecutive multi-scale attention feature extraction stages of the detection network to obtain multi-scale feature output; The multi-scale attention feature extraction stage includes several multi-scale attention feature aggregation modules (AttentionBlock) and Linear Embedding modules; the number of Attention Blocks is generally set according to the rule of increasing first and then decreasing, and is set as needed. The Linear Embedding module adopts a similar structure to the Patch Embedding module, differing only in the number of convolutional layers and the downsampling rate, which can be set as needed. The multi-scale attention feature aggregation module, Attention Block, has n key-value (KV) computation branches and one Q-computation branch. Each KV computation branch sequentially includes a downsampling convolutional layer, a fully connected layer, a KV splitting layer, a depthwise convolutional layer, and a residual enhancement layer. The Q-computation branch has one fully connected layer and one Q-splitting layer, with the Q-splitting layer producing n split outputs. i The outputs of the residual enhancement layer of each of the n KV computation branches are respectively processed by the self-attention computation layer. After the self-attention layer, the outputs of the n branches are merged by a concat layer. Then, a fully connected layer, a deep convolutional layer, a residual enhancement layer and a fully connected layer are set in sequence. Step 4: Input the output of each stage in Step 3 into the Feature Fusion Module (FPN) of the detection network to perform top-down feature fusion at different scales on the output features of each feature extraction stage in Step 3. Step 5: Input the fusion features output in Step 4 into the three detection heads of the detection network to locate and detect granulomas in the histopathological image; Each of the three detection heads consists of two convolutional layers, one activation layer, and one batch normalization layer.

2. The method for locating granuloma circles based on multi-scale feature aggregation according to claim 1, characterized in that: In step 2, the Patch Embedding module generates an input sequence X of size C×H×W through convolution with stride, where H0 and W0 are the length and width of the input histopathological image, respectively.

3. The method for locating granuloma circles based on multi-scale feature aggregation according to claim 1, characterized in that: In step 3, self-attention calculation and feature aggregation are performed through the multi-scale attention feature aggregation module Attention Block; The specific implementation includes the following steps: Step 3.1: By setting n different downsampling rates r i The input sequence X is convolved with a kernel and stride of r. i The convolution operation yields X samples with different sampling rates. i At this time, X i The size is Where C, H, and W represent the number of convolution channels, the length of the image, and the width of the image, respectively; Step 3.2: For After performing a full connection operation, it is split into two parts, denoted as K. i and V i ; Step 3.3: Process the split V i Perform depthwise convolution and residual enhancement, i.e., V i " = V i +LC(V i ); Step 3.4: Perform a fully connected operation on the input sequence X to obtain Q in the self-attention calculation, and split Q into n Q's. i sequence; Step 3.5: Using Q i K i V i "Perform self-attention calculations," Where d is the dimension and T represents the transpose; Step 3.6: For n x i The sequences are concatenated to obtain the self-attention sequence output Y. Step 3.7: Further supplement local information through two fully connected layers and depthwise convolution between the fully connected layers, i.e., Y′=FC(Y) and Z=FC(σ(Y′+LC(Y′))), where σ is the activation function and FC() is the fully connected operation.

4. The method for locating granuloma circles based on multi-scale feature aggregation according to claim 1, characterized in that: In step 4, the feature Z output from the deep feature extraction stage is used... i Sampling to the shallow feature extraction stage outputs feature Z i-1 After reaching the same size, with Z i-1 Addition achieves the fusion of features at different scales in the opposite direction.

5. The method for locating granuloma circles based on multi-scale feature aggregation according to claim 1, characterized in that: In step 5, the three detection heads are the center point detection head, the radius detection head, and the center point offset detection head; The three detection heads all output feature maps of the same size. R represents the downsampling ratio of the output feature map compared to the network input image, differing only in the number of channels; The center point detection head outputs a feature map with c channels. Each channel of the feature map represents the center point position of the current category object, specifically represented by the feature value at the center point. Eigenvalues ​​tending towards 1 rather than at the center point Approaching 0; The radius detection head outputs a feature map. With 1 channel, each value in the output feature map represents the radius of the granuloma at that point. The center point offset detection head outputs a feature map. The number of channels is 2, and the two channels represent the offset of the center point on the horizontal and vertical axes due to the non-integer quantization during sampling.

6. The method for locating circular representations of granulomas based on multi-scale feature aggregation according to any one of claims 1-5, characterized in that: The detection network is a trained detection network; The training includes the following steps: (1) Collect several histopathological sections, and have a professional pathologist mark the granuloma area with a circle to obtain the center coordinates and radius of each granuloma circle; (2) Input the labeled histopathological images into the detection network for network training. The loss function is composed of the joint loss of the three detectors, i.e., L = L x +λ r L r +λ off L off ; where λ r and λ off It is a constant used to balance the losses of various items; L c L r L off These represent the center point loss function, radius loss function, and offset loss function, respectively. For the granuloma center point p = (p in the input image) x ,p y The output corresponding to the center point detection head is... in Where R represents the radius of the circle; the center point detection head is constructed using Gaussian kernel transformation to output the corresponding true label, where the value at the feature map coordinate point (x, y) in category c is... The center point loss is calculated using focal loss, and the specific calculation formula is as follows: Where α and β are hyperparameters in focal loss, and N is the number of center points; The output of the center point detection head is the value at the coordinate point (x, y) of the c-th channel. The standard deviation is Gaussian. Both radius loss and offset loss are calculated using L1 loss, and the specific calculation formulas are as follows: in Center point p k The predicted radius at that location This is the true value of the radius at this location. To output the center point of the feature map The predicted offset at that location; The set of granuloma centroids detected by the centroid detector during the prediction process is as follows: n is the number of center points; then each granuloma target is represented as a center point. radius The circular area, in which and p, respectively, predicted by the offset detection head i The offset of the x and y coordinates at the point. p for radius prediction head prediction i The radius of the target at the point.

7. A granuloma circular representation and localization system based on multi-scale feature aggregation, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method for locating granuloma circles based on multi-scale feature aggregation as described in any one of claims 1 to 6.

Images