Cell instance segmentation method and system based on YOLOv11 improvement

Abstract

The invention provides an improved cell instance segmentation method and system based on YOLOv11, and the method comprises the steps: 1, constructing an improved YOLOv11 network, extracting the multi-scale features of an image, and generating feature maps of different scales; 2, introducing a multi-scale channel attention module into the backbone network, and introducing a new attention module into the multi-scale channel attention module to iteratively optimize initial fusion; step 3, integrating hierarchical feature maps generated by the encoder; 4, adding a scale sequence feature fusion module in the neck network, adding a triple feature encoder module, and fusing feature maps of different scales; step 5, adding a channel and position attention mechanism module in the neck network; and step 6, outputting a final segmentation result. The method is specially designed to be used for cell instance segmentation, and the accuracy and speed of processing cell images are improved by combining space and scale features.

Description

Technical Field

[0001] The present invention relates to the technical field of deep neural network, and in particular to a cell instance segmentation method and system based on an improvement of YOLOv11. Background Art

[0002] Instance segmentation is an important branch of computer vision. It achieves accurate segmentation and recognition of objects by assigning independent pixel labels to each object in the image. As instance segmentation models have achieved outstanding results in tasks such as object detection and medical image analysis, they have attracted widespread attention in scientific research and industrial applications. Traditional instance segmentation methods often rely on manually designed feature extraction and segmentation algorithms, which require a lot of manpower, time and professional experience. In order to meet the needs of instance segmentation in complex scenarios, instance segmentation methods combined with deep learning technology have emerged.

[0003] Cell instance segmentation uses deep learning technology to automatically segment cells in microscopic images. This technology automatically extracts the features of cell regions and generates segmentation results by building an end-to-end neural network, enabling researchers to quickly and efficiently complete cell identification and statistical tasks. In recent years, cell instance segmentation has been widely used in the fields of tumor pathology diagnosis, drug sensitivity testing, and biological experimental analysis. It not only significantly improves segmentation accuracy, but also greatly reduces the time and labor costs of experimental analysis.

[0004] Cell instance segmentation is a complex task that requires the ability to simultaneously separate different cell regions in a dense background and annotate each cell at the pixel level. In the early days, segmentation methods based on traditional image processing, such as threshold segmentation, edge detection, and morphological operations, were the mainstream of cell segmentation. However, these methods are sensitive to noise and have difficulty processing cells with different shapes and blurred boundaries in complex samples.

[0005] With the development of deep learning, instance segmentation methods based on convolutional neural networks (CNNs) have begun to emerge in cell segmentation tasks. For example, classic networks such as Mask R-CNN and U-Net achieve accurate segmentation of single cells through multi-stage detection and segmentation. These methods make full use of the feature extraction capabilities of the network and greatly improve the robustness of segmentation. However, multi-stage methods usually require high computing resources and are limited in performance when processing real-time segmentation tasks.

[0006] In order to solve the efficiency problem of multi-stage methods, single-stage cell instance segmentation methods have gradually become a research hotspot. For example, the real-time segmentation method based on improved YOLO integrates detection and segmentation tasks into one network, which greatly improves the segmentation speed. At the same time, in order to further optimize the segmentation performance, researchers have introduced technologies such as multi-scale feature fusion, attention mechanism and self-supervised learning, which can better capture the morphological characteristics of cells and identify areas with blurred boundaries.

[0007] Therefore, it is necessary to continuously optimize cell instance segmentation technology. Summary of the invention

[0008] Purpose of the invention: The technical problem to be solved by the present invention is to provide a cell instance segmentation method based on an improvement of YOLOv11 in view of the deficiencies of the prior art, comprising the following steps: Step 1: Build an improved YOLOv11 network to extract multi-scale features of the image and generate feature maps of different scales; Step 2, by introducing a multi-scale channel attention module into the backbone network and introducing a new attention module into the multi-scale channel attention module to iteratively optimize the initial fusion, the quality of feature fusion and the ability to extract multi-scale features of the improved YOLOv11 network are enhanced; Step 3, by introducing a multi-level feature fusion module SDI in the neck network, the hierarchical feature map generated by the encoder is integrated to enhance the semantic information and detail information in the image; Step 4: Add a scale sequence feature fusion module to the neck network to enhance the ability of the convolutional neural network in multi-scale information extraction, add a triple feature encoder module, and fuse feature maps of different scales; Step 5, adding a channel and position attention mechanism module in the neck network, wherein the channel and position attention mechanism module includes a scale sequence feature fusion module and a triple feature encoder module to improve detection and segmentation performance; Step 6: Output the final segmentation result.

[0009] In step 1, the improved YOLOv11 network includes a backbone network, a neck network and a head network, which are used to extract multi-scale features and generate feature maps of different scales; The input image is first processed through two or more convolutional layers, gradually reducing the spatial resolution while increasing the number of channels, thereby extracting the low-level feature map P3, the middle-level feature map P4, and the high-level feature map P5; By introducing adaptive pooling, interpolation and feature fusion modules, feature maps of different scales are fused and weighted.

[0010] Step 2 includes: Step 2.1, introduce local and global context information through the multi-scale channel attention module (MS-CAM) in the backbone network. The multi-scale channel attention module uses a bottleneck structure and implements local feature extraction through point-wise convolution. The formula is: , in It indicates that the multi-scale channel attention module uses a bottleneck structure and achieves the result after local feature extraction through point convolution; Represents an intermediate feature map, containing channels, each channel has a size of The feature map of , Indicates the number of channels, and Represent the height and width of the feature map respectively; Represents batch normalization; ReLU activation function. These are two different point convolutions. The weight size is , The weight size is ; is the channel compression ratio; R represents the real number space; Step 2.2: Use Global Average Pooling to extract global features. The formula is: , in Represents the result of global average pooling to extract global features, The colon in : means selecting all elements in the first dimension (the number of channels), that is, selecting the values ​​of all channels. In this expression, set The first dimension represents the number of channels, then Indicates the position in space A vector consisting of the values ​​of all channels on ; Global average pooling is used to capture global semantic information and provide an overall context description for the feature map; Step 2.3, the extracted local features and global features are fused by element-by-element addition to generate multi-scale attention weights. The formula is: , in represents the attention weight, , σ represents the Sigmoid activation function, and the feature value of each position in the attention weight is given by and The values ​​of the corresponding positions in the are directly added, keeping the spatial resolution and number of channels unchanged, to form an intermediate feature; Indicates another group with The corresponding eigenvalues ​​at the same spatial position; The input features are weighted element-wise using the generated attention weights, as follows: , in represents the feature map after the attention mechanism weighting, ⊗ represents element-by-element multiplication, and the feature value of each position in the feature map after the attention mechanism weighting is given by and The values ​​at the corresponding positions in are directly multiplied to keep the spatial resolution and number of channels unchanged; Step 2.4: The intermediate feature map and Initial integration is done by simple addition, with the formula: , in Indicates that the two intermediate feature maps and Conduct initial integration; Then use the initial fused weights to generate the formula: , in represents the final attention map, Represents the result of local information processing on the integrated feature map. Represents the result of global information processing on the integrated feature map; Then use the attention weights to the intermediate feature maps and Perform weighted fusion, the formula is: , Where Z represents the final fusion feature map; Step 2.5, introduce a new attention module to iteratively optimize the initial fusion, the formula is: , in Represents the feature map and The attention map obtained by processing the added result; Each iterative fusion will use new dynamic weights to better balance the input features and , and obtain a fused feature map with more semantic consistency and feature expressiveness.

[0011] Step 3 includes: Step 3.1, extract low-level features from the encoder and advanced features ; The low-level features , extracted from the shallow layer of the encoder (close to the input end), contains rich edge, texture and other details, and has a high spatial resolution. Indicates the number of low-level feature channels; Advanced Features , extracted from the deep layers of the encoder (close to the output), containing abstract semantic information, such as object categories or global context, with low spatial resolution, The number of channels representing high-level features, are the height and width of the high-level features, respectively; Represents the downsampling factor, which is used to control the ratio of spatial size reduction; Step 3.2, due to The resolution is low and needs to be upsampled Advanced Features With low-level features Alignment: , in Represents the high-level feature map after upsampling operation alignment, adjusting the low-resolution high-level features to the same resolution as the low-level features for fusion; represents an upsampling operation (such as bilinear interpolation or deconvolution) that restores the spatial resolution of high-level features through interpolation or learning, Represents the upsampling scaling ratio, which increases the resolution of the high-level feature map from Restore to ; Step 3.3: Align the high-level feature maps and low-level features Perform element-by-element fusion to generate intermediate features, and generate fusion weights through a weight generation module: , in Represents the dynamically generated fusion weights, which are used to control the contribution of low-level and high-level features in the fusion. The output dimension is ,and and Consistency; Step 3.4, using fusion weights and low-level features , aligned high-level feature map Perform weighted fusion to obtain the enhanced feature map : , in Represents the fused enhanced feature map, which is used to contain both the detail information of low-level features and the semantic information of high-level features. The output dimension is , To compensate for the weight operation, used for balance , ensure that the sum of weight distribution is 1, and output the enhanced feature map : , The feature map Detailed information and semantic information , It is a descriptive name of the enhanced feature, used to indicate that the feature map has been fused.

[0012] Step 4 includes: Step 4.1, based on the low-level feature map P3, the middle-level feature map P4 and the high-level feature map P5, extract the multi-scale feature map set through the scale sequence feature fusion module and the triple feature encoder module , represents the set of multi-scale feature maps extracted by the scale sequence feature fusion module and the triple feature encoder module, represents the i-th multi-scale feature map, and n represents the total number of multi-scale feature maps; , and are the number of channels, height and width of the i-th multi-scale feature map respectively; Each multi-scale feature map With different resolutions and numbers of channels, they are extracted from different layers or modules of the network for subsequent fusion. Multi-scale feature maps reflect multi-level information from details to the global level. For example, shallow features contain more edge and texture information, while deep features capture more semantic information. This feature extraction mechanism provides the network with richer contextual information, especially in complex scenes or tasks with uneven target sizes. Step 4.2: Adjust the multi-scale feature maps to the same spatial resolution through upsampling: For the low-resolution feature map , using the interpolation method The feature map The resolution is uniformly adjusted to the highest resolution feature map Dimensions: , in Represents the feature map after the i-th multi-scale feature map is aligned, which is used to The feature map is resized to the same resolution as the first feature map; Respectively represent The height and width of each feature map are used to calculate the scaling ratio. The purpose of feature alignment is to ensure that feature maps of all scales have the same spatial size so that element-by-element fusion or weighted calculation can be performed. Through the alignment operation, the network can integrate the information of different feature maps at the same scale, avoiding the problem of inconsistent fusion due to resolution differences; Step 4.3: Extract the aligned feature map through global average pooling Global features , and then use the multi-layer perceptron MLP to calculate the attention weights of each scale ; Finally, the weights are normalized through the Softmax operation to ensure that the sum is 1: , in Indicates The weights of the features of each scale are calculated, and the contribution of each scale is dynamically adjusted through the attention mechanism; It is a global feature, and the global representation of the feature map is extracted by global average pooling; represents a multi-layer perceptron that extracts the relationship between scales through a small network; Softmax is a normalization function that maps weights to [0, 1] and satisfies =1, the role of the scale attention generation module is to dynamically adjust the contribution of each scale feature in the final fusion according to its importance. Through the attention mechanism, the network can more flexibly adapt to the needs of different tasks, highlight the features of key scales, and weaken redundant or irrelevant feature information; Step 4.4, use The weight of the scale feature For feature maps Weighted, then all feature maps are added element by element according to the weighted results to obtain the fused multi-scale features : , Used to integrate information of all scales. The purpose of scale feature fusion is to ensure that multi-scale information is fully utilized while dynamically controlling the contribution of each feature map through a weighting mechanism, thereby obtaining stronger feature expression capabilities. This fusion method can balance details and semantic information, making the network more robust when dealing with objects of different sizes or complex scenes; Step 4.5: Convolution operation is used to perform the fused multi-scale features. Further processing is performed to generate the final output features : , Represents the enhanced features of the final output. Through convolution operations, the network can further enhance the expressive power of important features while reducing feature dimensions and improving computational efficiency.

[0013] Step 5 includes: Step 5.1, extract three sets of features from the low-level feature map P3, the middle-level feature map P4, and the high-level feature map P5, representing different spatial resolutions and semantic information respectively: ;in Represents a set of three scale feature maps extracted from low-level feature maps, middle-level feature maps, and high-level feature maps. represents the features extracted from the low-level feature map, represents the features extracted from the middle-level feature map, Represents the features extracted from the middle-level feature map; Step 5.2: Upsampling the feature maps with different spatial resolutions and semantic information , aligned to the highest resolution: , The alignment operation ensures that feature maps can be fused pixel by pixel or element by element, avoiding computational problems caused by inconsistent resolutions; Step 5.3, apply a separate convolution operation to each aligned feature map to further compress and extract features: , in Represents the i-th feature after encoding. The feature information is further extracted through the convolution layer. The feature encoding module helps reduce redundant information and improve the expressiveness of the feature. Step 5.4, use the dynamically generated weights to perform weighted fusion on all encoded features to generate a comprehensive multi-scale feature representation: , , in Indicates The weights of the scale features after weighted fusion and normalization, Represents the fused multi-scale features, which are used to integrate information of different scales. represents the i-th feature after encoding, Represents the global pooling feature, relative to , The convolutional layer further extracts feature information, and the feature encoding module helps reduce redundant information while improving the expressiveness of features. The fusion module is used to extract the global representation of each feature map, and effectively integrates multi-scale information, enabling the network to focus on both global and local information. Step 5.5: Apply convolution operation to the fused multi-scale features to generate the final output feature map , for downstream tasks: ,

[0014] This step further extracts high-level features to improve the network's expressiveness and adaptability.

[0015] Step 6 includes: Step 6.1, extract initial features from the scale sequence feature fusion module (SSFF) and the triple feature encoder module (TFE) : , in Represents the features extracted by the scale sequence feature fusion module, represents the features extracted by the triple feature encoder module; By combining multi-scale features and multi-level features, the input features have rich information expression capabilities, covering details, semantics and global information; Step 6.2, generate the attention weight of each channel through global pooling and multi-layer perceptron MLP: , in Represents the channel attention weight, assigns weights to each channel, and highlights important features. Represents the initial features Perform global pooling operations to compress each channel of the feature map into a global description, highlight important feature channels through the channel attention mechanism, and weaken irrelevant or redundant information; Step 6.3, through convolution operation Generate attention weights for each spatial position : , The position attention mechanism helps the network focus on important areas or small objects, thereby improving the ability to capture local features; Step 6.4, combine the channel attention weight and spatial position attention weight to enhance the input features: , in Represents the enhanced feature map, combining channel and spatial attention to improve feature expression, while considering channel and spatial information, enhancing the feature expression ability of the network, making it better aware of both global and local features; Step 6.5, further process the enhanced features through convolution to generate the final output: , in Represents the enhanced features of the final output, which are used for downstream detection or segmentation tasks, The output features contain both detailed information and global context expression, and can be directly used for detection and segmentation tasks.

[0016] The present invention also provides a cell instance segmentation system based on YOLOv11 improvement implemented by the method, comprising a feature extraction module, a multi-scale feature fusion module, a semantic and detail enhancement module, a scale sequence and feature encoding module, a channel and space attention module and a result output module; The feature extraction module is used to construct an improved YOLOv11 network structure, extract multi-scale features of the image, generate feature maps of different scales, and provide rich feature information for subsequent modules; The multi-scale feature fusion module optimizes the feature fusion process by introducing an iterative attention feature mechanism and a multi-scale channel attention module, thereby improving the convolutional neural network's ability to express multi-scale information and thus enhancing the ability to capture details of the target area. The semantic and detail enhancement module uses the SDI module to integrate the hierarchical feature map generated by the encoder to enhance the semantic information and detail information in the image, especially the ability to accurately identify small defect areas; The scale sequence and feature encoding module improves the network's ability to capture multi-scale information by adding a scale sequence feature fusion module; at the same time, the triple feature encoder module deeply fuses feature maps of different scales to further enhance the ability to describe the details of complex defect areas; The channel and spatial attention module integrates the channel and position attention mechanism module, and is combined with the scale sequence feature fusion module and the triple feature encoder module to focus on key features related to information channels and spatial positions, significantly improving the detection and segmentation performance of small target defects and local complex areas; The result output module performs post-processing on the segmentation results, including classification of defect areas, boundary marking, confidence scoring, etc., and outputs them in the form of image overlay, statistical reports and visualization.

[0017] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the described method.

[0018] The present invention also provides a storage medium storing a computer program or instruction. When the computer program or instruction is run on a computer, the steps of the method described are executed.

[0019] The present invention introduces an iterative attention feature mechanism, a multi-level feature fusion module SDI, a scale sequence feature fusion module, a triple feature encoder module, and a channel and position attention mechanism module; the iterative attention feature mechanism improves feature fusion through a more sophisticated attention mechanism, thereby enhancing the convolutional neural network. It not only handles the feature fusion problem caused by scale and semantic inconsistency, but also introduces a multi-scale channel attention module, providing a unified and universal feature fusion solution. The multi-level feature fusion module SDI enhances the semantic information and detail information in the image by integrating the hierarchical feature maps generated by the encoder, effectively combining high-level semantic information with low-level detail information, thereby improving the model's ability to understand the image content. The scale sequence feature fusion module is used to enhance the ability of the neural network in extracting features of different scales, and can provide richer and more detailed feature representations, which helps to improve the performance of the model when processing objects of different sizes. This fusion strategy is particularly suitable for tasks that require accurate positioning and identification of multi-scale objects in images, such as cell segmentation or facial recognition. The triple feature encoder is an innovative feature fusion mechanism that combines feature maps of different scales to form a comprehensive feature representation to capture more contextual information and details. Such a design enables the network to process multi-scale information more effectively, which is particularly important for fine-grained tasks such as cell segmentation. The channel and position attention mechanism aims to improve the detection and segmentation capabilities of small objects with rich details by integrating scale sequence feature fusion and triple feature encoder modules, allowing the model to more accurately identify and locate small targets in images, thereby improving the performance of segmentation tasks; combined with these innovations, the present invention is specially designed for cell instance segmentation, and by combining spatial and scale features, it improves the accuracy and speed of processing cell images.

[0020] The beneficial effects of the present invention are as follows: the method of the present invention proposes an efficient instance segmentation scheme based on the improvement of the YOLOv11 model for the cell image instance segmentation task. The scheme significantly improves the segmentation performance of the model by introducing an iterative attention feature mechanism, a multi-level feature fusion module SDI, a scale sequence feature fusion module, a triple feature encoder module, and a channel and position attention mechanism module. The iterative attention feature mechanism improves feature fusion through a more sophisticated attention mechanism, better integrates features of different scales and semantic inconsistencies, and improves the cell segmentation accuracy by improving the feature fusion process. The multi-level feature fusion module SDI enhances the semantic information and detail information in the image by integrating the hierarchical feature maps generated by the encoder, making the segmentation effect more accurate while maintaining the computational and memory efficiency. The scale sequence feature fusion module can enhance the ability of the neural network in extracting features of different scales. By aggregating features from different layers of the network, the module can provide richer and more detailed feature representations, which helps to improve the performance of the model when processing objects of different sizes. The triple feature encoder is used to process input feature maps of different sizes. This module processes feature maps from different layers of the network through specific operations such as convolution, batch normalization, and activation functions, allowing the network to process multi-scale information more efficiently, which is particularly important for fine-grained tasks such as cell segmentation. It aims to improve the detection and segmentation capabilities of small objects with rich details by integrating scale sequence feature fusion and triple feature encoder modules, allowing the model to more accurately identify and locate small targets in the image, thereby improving the performance of segmentation tasks. Through these five major innovations, the method of the present invention not only improves the segmentation accuracy, but also maintains a high inference speed and has good real-time detection capabilities. It is especially suitable for cell instance segmentation of small targets and has high practical value and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and other advantages of the present invention will become more clear.

[0022] Figure 1 Flow chart of the method of the present invention.

[0023] Figure 2 This is the overall network structure diagram of the improved YOLOv11 in the present invention.

[0024] Figure 3 This is the network structure diagram of the iterative attention feature mechanism module in the present invention.

[0025] Figure 4 This is a network structure diagram of the multi-level feature fusion module in the present invention.

[0026] Figure 5This is a network structure diagram of the scale sequence feature fusion module in the present invention.

[0027] Figure 6 This is the network structure diagram of the triple feature encoder module in the present invention.

[0028] Figure 7 This is the network structure diagram of the channel and position attention mechanism module in the present invention.

[0029] Figure 8 This is a schematic diagram of comparing the average accuracy of each training with the original YOLOv11 model. DETAILED DESCRIPTION

[0030] like Figure 1 (Take the search space containing 6 connecting edges and 3 candidate operations as an example) and Figure 2 As shown, an embodiment of the present invention provides a cell instance segmentation method based on an improvement of YOLOv11, comprising the following steps: Step 1: Build an improved YOLOv11 network to extract multi-scale features of the image and generate feature maps of different scales. The improved YOLOv11 network includes a backbone network, a neck network, and a head network to accurately extract multi-scale features and generate feature maps of different scales. The input image is first processed through multiple convolutional layers to gradually reduce the spatial resolution while increasing the number of channels, thereby extracting low-level feature maps P3, middle-level feature maps P4, and high-level feature maps P5. The low-level feature maps mainly retain detail information, such as edges and textures, and are suitable for small object detection; the middle-level feature maps integrate spatial and semantic information and are suitable for medium-sized object detection; the high-level feature maps focus on extracting high-level semantic information, such as object category and location, and are suitable for large objects and complex scenes. By introducing adaptive pooling, interpolation, and feature fusion modules, feature maps of different scales are fused and weighted to enhance the accuracy and robustness of the network in multi-scale object detection and segmentation.

[0031] Step 2, such as Figure 3 As shown in the figure, iAFF is a dynamic feature fusion method that introduces a multi-scale channel attention mechanism and combines iterative optimization strategies to adjust and enhance the fusion features multiple times, making the final fusion result have higher expressive power. Step 2 includes: Step 2.1, introduce local and global context information through the multi-scale channel attention module (MS-CAM) to improve the expressiveness of the fusion feature. Use the bottleneck structure to achieve local feature extraction through point-wise convolution: in: , represents an intermediate feature map, including channels, each channel has a size of The feature map of represents the input features, Indicates the number of channels, and Represent the height and width of the feature map respectively; Represents batch normalization (BatchNormalization); represents the ReLU activation function; is a point convolution, whose weight size is: , ; : Channel compression ratio; Step 2.2, use Global Average Pooling to extract global features: , used to capture global semantic information and provide an overall context description for the feature map; Step 2.3, the extracted local features and global features are fused by element-by-element addition to generate multi-scale attention weights: , represents the attention weight, σ represents the Sigmoid activation function, ⊕ represents element-by-element addition, and the eigenvalue of each position is given by and The values ​​at the corresponding positions in the vector are directly added, keeping the spatial resolution and number of channels unchanged to form an intermediate feature; the generated attention weights are then used to weight the input features element by element: , where ⊗ represents element-by-element multiplication, represents element-by-element multiplication, and the eigenvalue at each position is given by and The values ​​at the corresponding positions in are directly multiplied to keep the spatial resolution and number of channels unchanged.

[0032] Step 2.4: transform the two input features and Initial integration is done by simple addition: , and then generate with the initial fused weights: , and then use the attention weights and Perform weighted fusion: , Z represents the final fusion feature map; Step 2.5, introduce a new attention module to iteratively optimize the initial fusion: Each iterative fusion will use new dynamic weights to better balance the input features and , and obtain a fused feature map with more semantic consistency and feature expression; Step 3, such as Figure 4 As shown in Figure 1, the SDI module is an innovative technology designed to enhance the semantic information and detail information of an image. The module integrates the multi-level feature maps generated by the encoder to effectively combine high-level semantic information with low-level detail information, thereby improving the model's ability to understand the image content; Step 3 includes: Step 3.1, extract multi-level features from the encoder: Low-level features: , extracted from the shallow layer of the encoder (close to the input end), contains rich edge, texture and other details, and has a high spatial resolution. Indicates the number of channels, They are the height and width of the feature map respectively; advanced features: , extracted from the deep layers of the encoder (close to the output), containing abstract semantic information, such as object categories or global context, with low spatial resolution, Indicates the number of channels, Reduced resolution for high-level features; Step 3.2, due to The resolution is low and needs to be upsampled. Alignment, , Represents the high-level feature map after upsampling operation alignment, adjusting the low-resolution high-level features to the same resolution as the low-level features for fusion; represents an upsampling operation (such as bilinear interpolation or deconvolution) that restores the spatial resolution of high-level features through interpolation or learning, Represents the upsampling scaling ratio, which increases the resolution of the high-level feature map from Restore to ; Step 3.3: Align the and Perform element-by-element fusion to generate intermediate features. Generate dynamic weights through a weight generation module: , Represents the dynamically generated fusion weights, which are used to control the contribution of low-level and high-level features in the fusion. The output dimension is ,and and Consistency; Step 3.4, using dynamic weights and , Perform weighted fusion: , Represents the fused enhanced feature map, which is used to contain both the detail information of low-level features and the semantic information of high-level features. The output dimension is , To compensate for the weight operation, used for balance , ensure that the sum of weight assignments is 1, and output the enhanced feature map: , the feature map also has detailed information and semantic information , It is a descriptive name of the enhanced feature, used to indicate that the feature map has been fused; Step 4, such as Figure 5 and Figure 6 As shown in the figure, the scale sequence feature fusion module is a key component designed to process multi-scale information. Its main function is to enhance the ability of neural networks in extracting features of different scales. By aggregating features from different layers of the network, this module is able to provide richer and more detailed feature representations, which helps improve the performance of the model when dealing with objects of different sizes. This fusion strategy is particularly suitable for tasks that require precise positioning and identification of multi-scale objects in images, such as cell segmentation or facial recognition. With this module, the model can more effectively identify and understand the details in the image, thereby improving the overall detection and recognition performance; In the present invention, the triple feature encoder processes feature maps from different layers of the network through specific operations such as convolution, batch normalization, and activation functions. Then, it uses upsampling and downsampling techniques to adjust the spatial size of the feature maps to ensure that the size of the feature maps is consistent. Finally, through a fusion operation, these feature maps of different scales are combined to form a comprehensive feature representation to capture more contextual information and details. Such a design enables the network to process multi-scale information more effectively, which is particularly important for fine-grained tasks such as cell segmentation; Step 4 includes: Step 4.1, add a scale sequence feature fusion module to enhance the network's ability to extract multi-scale information; Step 4.2, add a triple feature encoder module to fuse feature maps of different scales to increase detailed information; Step 4.1.1: Extract a set of multi-scale feature maps from different layers or specific modules of the network , each feature map With different resolutions and channel numbers, Represents a multi-scale feature set, extracted from different layers or modules of the network for subsequent fusion. The multi-scale feature map reflects multi-level information from details to the global level. For example, shallow features contain more edge and texture information, while deep features capture more semantic information. This feature extraction mechanism provides the network with richer contextual information, especially in complex scenes or tasks with uneven target sizes. Step 4.1.2, adjust the multi-scale feature maps to the same spatial resolution through upsampling. Specifically, for the low-resolution feature map , use the interpolation method to uniformly adjust its resolution to the highest resolution feature map Dimensions: , Represents the aligned feature map, which is used to The feature map is resized to the same resolution as the first feature map; Indicates The resolution of each feature map is used to calculate the scaling ratio. The purpose of feature alignment is to ensure that feature maps of all scales have the same spatial size so that element-by-element fusion or weighted calculation can be performed. Through the alignment operation, the network can integrate the information of different feature maps at the same scale, avoiding the problem of inconsistent fusion due to resolution differences; Step 4.1.3: Extract the aligned feature map by global average pooling Global features , and then use a multi-layer perceptron (MLP) to calculate the attention weights for each scale ; Finally, the weights are normalized through the Softmax operation to ensure that the sum is 1: , Indicates The weights of the features of each scale are calculated, and the contribution of each scale is dynamically adjusted through the attention mechanism; It is a global feature, and the global representation of the feature map is extracted by global average pooling; represents a multi-layer perceptron that extracts the relationship between scales through a small network; Softmax is a normalization function that maps weights to [0, 1] and satisfies =1, the role of the scale attention generation module is to dynamically adjust the contribution of each scale feature in the final fusion according to its importance. Through the attention mechanism, the network can more flexibly adapt to the needs of different tasks, highlight the features of key scales, and weaken redundant or irrelevant feature information; Step 4.1.4, use scaled attention weights For each aligned feature map Weighted, then all feature maps are added element by element according to the weighted results to obtain the fused multi-scale features : , Represents the fused multi-scale features, which are used to integrate information of all scales. The purpose of scale feature fusion is to ensure that multi-scale information is fully utilized while dynamically controlling the contribution of each feature map through a weighting mechanism, thereby obtaining stronger feature expression capabilities. This fusion method can balance details and semantic information, making the network more robust when dealing with objects of different sizes or complex scenes; Step 4.1.5: Convolution operation is used to perform the fused features Further processing is performed to generate the final output features : , The enhanced features representing the final output are used as the input for downstream tasks. The output feature generation module optimizes the feature representation after multi-scale fusion to make it more suitable for the input requirements of downstream tasks (such as segmentation, detection, etc.). Through convolution operations, the network can further enhance the expression of important features, while reducing feature dimensions and improving computational efficiency. Step 4.2 includes: Step 4.2.1, extract three sets of features from different layers in the network, representing different spatial resolutions and semantic information: ; Step 4.2.2, use upsampling operation on feature maps of different resolutions to align them with the highest resolution: ,The alignment operation ensures that feature maps can be fused pixel by pixel or element by element, avoiding ,computational problems caused by inconsistent resolutions; In step 4.2.3, a separate convolution operation is applied to each aligned feature map to further compress and extract features: , Represents the encoded features, and further extracts feature information through the convolution layer. The feature encoding module helps reduce redundant information and improves the expressiveness of the features; Step 4.2.4, use the dynamically generated weights to perform weighted fusion on all encoded features to generate a comprehensive multi-scale feature representation: , Represents the fused multi-scale features, which are used to integrate information of different scales. Represents the global pooling feature, which is used to extract the global representation of each feature map. The fusion module effectively integrates multi-scale information, enabling the network to focus on both global and local information; Step 4.2.5, apply convolution operation to the fused features to generate the final output feature map for downstream tasks: , further extract high-level features and improve the network’s expressiveness and adaptability; Step 5, such as Figure 7As shown in the figure, by integrating the channel and position attention mechanism modules with the SSFF and TFE modules, the network can more accurately capture the multi-scale features and detail information of the target. The channel attention mechanism focuses on the channel dimension of information, while the position attention mechanism focuses on the target area in the spatial dimension. The two complement each other and significantly improve the performance of the model in small object detection and segmentation tasks.

[0033] Step 5 includes: Step 5.1, extract initial features from the scale sequence feature fusion module (SSFF) and the triple feature encoder module (TFE): ,By combining multi-scale features and multi-level features, the input features have rich information expression capabilities, covering details, semantics and global information; Step 5.2, generate the attention weight of each channel through global pooling and multi-layer perceptron (MLP): , Represents the channel attention weight, assigns weights to each channel, and highlights important features. Represents a global pooling operation, which compresses each channel of the feature map into a global description, highlights important feature channels through the channel attention mechanism, and weakens irrelevant or redundant information; Step 5.3, generate the attention weights for each spatial position through convolution operation: ,The position attention mechanism helps the network focus on important areas or small objects, thereby improving the ability to capture local features; Step 5.4, combine the channel and position attention weights to enhance the input features: , Represents the enhanced feature map, combining channel and spatial attention to improve feature expression, while considering channel and spatial information, enhancing the feature expression ability of the network, making it better aware of both global and local features; Step 5.5, further process the enhanced features through convolution to generate the final output: , Represents the enhanced features of the final output, which are used for downstream detection or segmentation tasks, The output features contain both detailed information and global context expression, and can be directly used for detection and segmentation tasks.

[0034] Step 6, output the final detection result; Compared with the existing technology, the method of the present invention significantly improves the performance of cell instance segmentation, especially in terms of accuracy and multi-scale target adaptability. It can more accurately segment various targets in cells, such as fine structures, boundaries, shape changes, cracks, vacuoles, intracellular impurities, division states, fusion, cell membrane damage, etc., demonstrating extremely strong robustness and wide practicality. Under the same experimental conditions (model size is m, no pre-trained weights are used, and NVIDIA 4090 graphics card is used, etc. Among them, m: [0.50, 1.00, 512], m represents medium, 0.50 represents the depth scaling factor of the model is 0.50, which is used to control the number of layers of the network; 1.00 represents the width scaling factor of the model is 1.00, which is used to control the number of channels of each convolutional layer; 512 represents the maximum number of channels allowed for each convolution layer in the network.), with 100 epochs (number of times) training time, Box P (precision) increased from 78.7% to 82.8%, Box R (recall rate) increased from 64.2% to 69.8%, mAP50 increased from 73.5% to 81.5%, and mAP50-95 increased from 45.6% to 52.8%. Specifically, step 1, build an improved YOLOv11 network structure, integrate iterative attention feature mechanism, multi-level feature fusion module, scale sequence feature fusion module, triple feature encoder module, channel and position attention mechanism module. Step 2, repeat the training of the cell segmentation dataset 100 times, using the same experimental environment, including the same model scale m, no pre-training weights, and NVIDIA 4090 graphics card, to obtain the training results; During the training process, the present invention compares the average accuracy of each training with the original YOLOv11 model (using the same experimental environment), such as Figure 8 Compared with YOLOv11, which converges to a lower mAP50 value at the end of training (i.e., the lower convergence effect shown by the orange curve), the method of the present invention shows significant advantages. The improved model can maintain a stable optimization trend in the middle and late stages of training, and finally converge to a higher verification accuracy.

[0035] The present invention provides a cell instance segmentation method and system based on an improved YOLOv11. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A cell instance segmentation method based on YOLOv11 improvement, characterized in that: include: Step 1: Build an improved YOLOv11 network to extract multi-scale features of the image and generate feature maps of different scales; Step 2: introduce a multi-scale channel attention module into the backbone network, and introduce a new attention module into the multi-scale channel attention module to iteratively optimize the initial fusion; Step 3, by introducing a multi-level feature fusion module SDI in the neck network, the hierarchical feature map generated by the encoder is integrated to enhance the semantic information and detail information in the image; Step 4: Add a scale sequence feature fusion module to the neck network to enhance the ability of the convolutional neural network in multi-scale information extraction, add a triple feature encoder module, and fuse feature maps of different scales; Step 5, adding a channel and position attention mechanism module in the neck network, wherein the channel and position attention mechanism module includes a scale sequence feature fusion module and a triple feature encoder module to improve detection and segmentation performance; Step 6: Output the final segmentation result.

2. The method according to claim 1, characterized in that In step 1, the improved YOLOv11 network includes a backbone network, a neck network and a head network, which are used to extract multi-scale features and generate feature maps of different scales; The input image is first processed through two or more convolutional layers, gradually reducing the spatial resolution while increasing the number of channels, thereby extracting the low-level feature map P3, the middle-level feature map P4, and the high-level feature map P5; By introducing adaptive pooling, interpolation and feature fusion modules, feature maps of different scales are fused and weighted.

3. The method according to claim 2, characterized in that Step 2 includes: Step 2.1, introduce local and global context information through the multi-scale channel attention module in the backbone network. The multi-scale channel attention module uses a bottleneck structure and realizes local feature extraction through point convolution. The formula is: ， in It indicates that the multi-scale channel attention module uses a bottleneck structure and achieves the result after local feature extraction through point convolution; Represents an intermediate feature map, containing channels, each channel has a size of The feature map of , Indicates the number of channels, and Represent the height and width of the feature map respectively; represents batch normalization; ReLU activation function. These are two different point convolutions. The weight size is , The weight size is ; is the channel compression ratio; R represents the real number space; Step 2.2: Use Global Average Pooling to extract global features. The formula is: , in Represents the result of global average pooling to extract global features, The colon in : means selecting all elements in the first dimension, setting The first dimension represents the number of channels, then Indicates the position in space A vector consisting of the values ​​of all channels on ; Global average pooling is used to capture global semantic information and provide an overall context description for the feature map; Step 2.3, the extracted local features and global features are fused by element-by-element addition to generate multi-scale attention weights. The formula is: ， in represents the attention weight, ,σ represents the Sigmoid activation function; Indicates another group with The corresponding eigenvalues ​​at the same spatial position; The input features are weighted element-wise using the generated attention weights, as follows: ， in represents the feature map after weighting by the attention mechanism, and ⊗ represents element-by-element multiplication; Step 2.4: The intermediate feature map and Initial integration is done by simple addition, with the formula: ， in Indicates that the two intermediate feature maps and Conduct initial integration; Then use the initial fused weights to generate the formula: ， in represents the final attention map, Represents the result of local information processing on the integrated feature map. Represents the result of global information processing on the integrated feature map; Then use the attention weights to the intermediate feature maps and Perform weighted fusion, the formula is: ， Where Z represents the final fusion feature map; Step 2.5, introduce a new attention module to iteratively optimize the initial fusion, the formula is: ， in Represents the feature map and The attention map is obtained by processing the added result.

4. The method according to claim 3, characterized in that Step 3 includes: Step 3.1, extract low-level features from the encoder and advanced features ; The low-level features , Indicates the number of low-level feature channels; Advanced Features , The number of channels representing high-level features, are the height and width of the high-level features, respectively; represents the downsampling factor; Step 3.2, through upsampling operation Advanced Features With low-level features Alignment: ， in Represents the high-level feature map after upsampling alignment; represents the upsampling operation, Represents the upsampling scaling ratio, which increases the resolution of the high-level feature map from Restore to ; Step 3.3: Align the high-level feature maps and low-level features Perform element-by-element fusion to generate intermediate features, and generate fusion weights through a weight generation module: ， in represents the dynamically generated fusion weight; Step 3.4, using fusion weights and low-level features , aligned high-level feature map Perform weighted fusion to obtain the enhanced feature map : ， in represents the enhanced feature map after fusion, It is the weight compensation operation.

5. The method according to claim 4, characterized in that Step 4 includes: Step 4.1, based on the low-level feature map P3, the middle-level feature map P4 and the high-level feature map P5, extract the multi-scale feature map set through the scale sequence feature fusion module and the triple feature encoder module , represents the set of multi-scale feature maps extracted by the scale sequence feature fusion module and the triple feature encoder module, represents the i-th multi-scale feature map, and n represents the total number of multi-scale feature maps; , and are the number of channels, height and width of the i-th multi-scale feature map respectively; Step 4.2: Adjust the multi-scale feature maps to the same spatial resolution through upsampling: For the low-resolution feature map , using the interpolation method The feature map The resolution is uniformly adjusted to the highest resolution feature map Dimensions: ， in Represents the feature map after the i-th multi-scale feature map is aligned; Respectively represent The height and width of the feature map; Step 4.3: Extract the aligned feature map through global average pooling Global features , and then use the multi-layer perceptron MLP to calculate the attention weights of each scale ; Finally, the weights are normalized through the Softmax operation to ensure that the sum is 1: ， in Indicates The weight of each scale feature; It is a global feature; represents a multilayer perceptron; Softmax is a normalization function used to map weights to [0, 1] and satisfy =1; Step 4.4, use The weight of the scale feature For feature maps Weighted, then all feature maps are added element by element according to the weighted results to obtain the fused multi-scale features : ； Step 4.5: Convolution operation is used to perform the fused multi-scale features. Further processing is performed to generate the final output features : 。 6. The method according to claim 5, characterized in that Step 5 includes: Step 5.1, extract three sets of features from the low-level feature map P3, the middle-level feature map P4, and the high-level feature map P5, representing different spatial resolutions and semantic information respectively: ;in Represents a set of three scale feature maps extracted from low-level feature maps, middle-level feature maps, and high-level feature maps. represents the features extracted from the low-level feature map, represents the features extracted from the middle-level feature map, Represents the features extracted from the middle-level feature map; Step 5.2: Upsampling the feature maps with different spatial resolutions and semantic information , aligned to the highest resolution: ， Step 5.3, apply a separate convolution operation to each aligned feature map to further compress and extract features: ， in Represents the i-th feature after encoding; Step 5.4, use the dynamically generated weights to perform weighted fusion on all encoded features to generate a comprehensive multi-scale feature representation: ， ， in Indicates The weights of the scale features after weighted fusion and normalization, represents the fused multi-scale features, represents the i-th feature after encoding, Represents the global pooling feature; Step 5.5: Apply convolution operation to the fused multi-scale features to generate the final output feature map , for downstream tasks: 。 7. The method according to claim 6, characterized in that Step 6 includes: Step 6.1: Extract initial features from the scale sequence feature fusion module and the triple feature encoder module : ， in Represents the features extracted by the scale sequence feature fusion module, represents the features extracted by the triple feature encoder module; Step 6.2, generate the attention weight of each channel through global pooling and multi-layer perceptron MLP: , in represents the channel attention weight, Represents the initial features Perform global pooling operations; Step 6.3, through convolution operation Generate attention weights for each spatial position : ； Step 6.4, combine the channel attention weight and spatial position attention weight to enhance the input features: , in Represents the enhanced feature map; Step 6.5, further process the enhanced features through convolution to generate the final output: ， in Represents the enhanced features of the final output.

8. A cell instance segmentation system based on improved YOLOv11 implemented by the method according to any one of claims 1 to 7, characterized in that: It includes feature extraction module, multi-scale feature fusion module, semantic and detail enhancement module, scale sequence and feature encoding module, channel and spatial attention module and result output module; The feature extraction module is used to construct an improved YOLOv11 network structure, extract multi-scale features of the image, generate feature maps of different scales, and provide rich feature information for subsequent modules; The multi-scale feature fusion module optimizes the feature fusion process by introducing an iterative attention feature mechanism and a multi-scale channel attention module, thereby improving the convolutional neural network's ability to express multi-scale information and thus enhancing the ability to capture details of the target area. The semantic and detail enhancement module uses the SDI module to integrate the hierarchical feature map generated by the encoder to enhance the semantic information and detail information in the image, especially the ability to accurately identify small defect areas; The scale sequence and feature encoding module improves the network's ability to capture multi-scale information by adding a scale sequence feature fusion module; at the same time, the triple feature encoder module deeply fuses feature maps of different scales to further enhance the ability to describe the details of complex defect areas; The channel and spatial attention module integrates the channel and position attention mechanism module, and is combined with the scale sequence feature fusion module and the triple feature encoder module to focus on key features related to information channels and spatial positions, significantly improving the detection and segmentation performance of small target defects and local complex areas; The result output module performs post-processing on the segmentation results, including classification of defect areas, boundary marking, confidence scoring, etc., and outputs them in the form of image overlay, statistical reports and visualization.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: A computer program or instruction is stored, and when the computer program or instruction is run on a computer, the steps of the method according to any one of claims 1 to 7 are executed.

Images