A real-time instance segmentation method and system based on cross-stage feature information fusion

Abstract

This invention discloses a real-time instance segmentation method and system based on cross-stage feature information fusion. The method includes: improving the SOLOv2 model to reduce the computational cost of the backbone network and improve the segmentation of small targets and target occlusion, resulting in an instance segmentation model; wherein the instance segmentation model includes a cross-stage fusion backbone network based on positional attention, a bidirectional feature pyramid network, and a cross-stage mask feature fusion structure; training the instance segmentation model; and processing the image to be segmented using the trained instance segmentation model to obtain the corresponding instance segmentation result. This invention further improves the overall segmentation accuracy of the model by reducing the computational cost of the backbone network and improving the segmentation of small targets and target occlusion.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a real-time instance segmentation method and system that integrates cross-stage feature information. Background Technology

[0002] As an important component of the intelligent transportation field, instance segmentation of street scenes has broad application prospects in real-time traffic detection, traffic planning optimization, and autonomous driving.

[0003] Instance segmentation combines object detection and semantic segmentation, enabling the extraction of more accurate image target information. Street scene-based instance segmentation refers to the process of using computer vision technology to identify and distinguish various targets in a street scene, such as vehicles, road signs, buildings, and pedestrians, and accurately segment them.

[0004] Deep learning-based instance segmentation methods are mainly divided into two-stage, single-stage, and multi-stage methods. Two-stage and multi-stage instance segmentation methods are widely used due to their high segmentation accuracy, but these models struggle to handle real-time issues. Single-stage instance segmentation methods achieve a relative balance between segmentation accuracy and inference speed by performing detection and segmentation in parallel. Street scenes require high timeliness of system feedback, but most current instance segmentation algorithms cannot simultaneously meet the requirements of segmentation accuracy and real-time performance. Summary of the Invention

[0005] This invention provides a real-time instance segmentation method and system that integrates cross-stage feature information to solve the technical problem that existing instance segmentation algorithms cannot simultaneously meet the requirements of segmentation accuracy and real-time performance.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] On one hand, the present invention provides a real-time instance segmentation method based on cross-stage feature information fusion, the real-time instance segmentation method based on cross-stage feature information fusion includes:

[0008] The SOLOv2 model is improved to reduce the computational cost of the backbone network and improve the problems of missing small target segmentation and target occlusion, resulting in an instance segmentation model. The instance segmentation model includes a cross-stage fusion backbone network based on positional attention, a bidirectional feature pyramid network, and a cross-stage mask feature fusion structure.

[0009] The instance segmentation model is trained to obtain a trained instance segmentation model;

[0010] The trained instance segmentation model is used to process the image to be segmented to obtain the instance segmentation result.

[0011] Furthermore, the improvement to the SOLOv2 model includes:

[0012] The original backbone network ResNet of the SOLOv2 model is improved by adopting a cross-stage fusion structure, reducing the number of parameters in the convolutional layers, reusing feature maps, thereby reducing memory consumption and improving the training efficiency of the model.

[0013] The ReLU activation function is replaced with the Mish activation function;

[0014] The design incorporates positional attention between residual blocks to extract positional information from the underlying feature map.

[0015] Furthermore, the improvement to the SOLOv2 model also includes:

[0016] By introducing the fusion of low-level location information flow and high-level semantic information, a bidirectional feature pyramid network (TWFPN) is designed to supplement the lack of location information.

[0017] To address the issue of the original network FPN which scales up feature layers to the same size and then stacks them, we propose to upsample the high-level semantic layer P5 by 2 times and then concatenate it with the lower-level feature N4 in the channel dimension. We also enhance the semantic information by upsampling by 8 times across stages. We design a cross-stage mask feature fusion structure that incorporates global semantic information into the mask feature branch while considering computational cost, thereby improving the model's ability to detect and segment small targets.

[0018] Furthermore, the improvement to the SOLOv2 model also includes:

[0019] Design an adaptive minimum loss function matching method, including: firstly, in the function L... Mask An influence factor ε is introduced to adaptively adjust the weight information of the prediction mask, improving instance segmentation accuracy. Then, the sigmoid function is used to normalize the category and mask losses, and the two losses are combined through summation. Finally, the losses are compared, and the network is trained using the minimum loss. Wherein, L... Mask ε and Loss are defined as follows:

[0020]

[0021]

[0022] Loss = sigmoid(L Cate )+sigmoid(λL Mask )

[0023] Among them, L Cate This represents the category loss function, calculated using focus loss, where P is the set of positive samples, and N... posIt is the number of positive samples, m i Let g represent the prediction mask for the i-th feature point. i Let C represent the true mask of the i-th feature point, with λ set to 3. PX C PY C represents the x and y coordinates of the predicted mask center point. GX C GY The x and y coordinates represent the actual location of the center point of the annotation.

[0024] On the other hand, the present invention also provides a real-time instance segmentation system with cross-stage feature information fusion, the real-time instance segmentation system with cross-stage feature information fusion comprising:

[0025] The model improvement module is used to improve the SOLOv2 model to reduce the computational cost of the backbone network, improve the problems of missing segmentation of small targets and target occlusion, and obtain an instance segmentation model; wherein, the instance segmentation model includes a cross-stage fusion backbone network based on positional attention, a bidirectional feature pyramid network, and a cross-stage mask feature fusion structure;

[0026] The model training module is used to train the instance segmentation model obtained by the model improvement module to obtain a trained instance segmentation model.

[0027] The instance segmentation module is used to process the image to be segmented using the instance segmentation model trained by the model training module, and obtain the instance segmentation result.

[0028] Furthermore, the model improvement module is specifically used for:

[0029] The original backbone network ResNet of the SOLOv2 model is improved by adopting a cross-stage fusion structure, reducing the number of parameters in the convolutional layers, reusing feature maps, thereby reducing memory consumption and improving the training efficiency of the model.

[0030] The ReLU activation function is replaced with the Mish activation function;

[0031] The design incorporates positional attention between residual blocks to extract positional information from the underlying feature map.

[0032] Furthermore, the model improvement module is specifically used for:

[0033] By introducing the fusion of low-level location information flow and high-level semantic information, a bidirectional feature pyramid network (TWFPN) is designed to supplement the lack of location information.

[0034] To address the issue of the original network FPN which scales up feature layers to the same size and then stacks them, we propose to upsample the high-level semantic layer P5 by 2 times and then concatenate it with the lower-level feature N4 in the channel dimension. We also enhance the semantic information by upsampling by 8 times across stages. We design a cross-stage mask feature fusion structure that incorporates global semantic information into the mask feature branch while considering computational cost, thereby improving the model's ability to detect and segment small targets.

[0035] Furthermore, the model improvement module is specifically used for:

[0036] Design an adaptive minimum loss function matching method, including: firstly, in the function L... Mask An influence factor ε is introduced to adaptively adjust the weight information of the prediction mask, improving instance segmentation accuracy. Then, the sigmoid function is used to normalize the category and mask losses, and the two losses are combined through summation. Finally, the losses are compared, and the network is trained using the minimum loss. Wherein, L... Mask ε and Loss are defined as follows:

[0037]

[0038]

[0039] Loss = sigmoid(L Cate )+sigmoid(λL Mask )

[0040] Among them, L Cate This represents the category loss function, calculated using focus loss, where P is the set of positive samples, and N... pos It is the number of positive samples, m i Let g represent the prediction mask for the i-th feature point. i Let C represent the true mask of the i-th feature point, with λ set to 3. PX C PY C represents the x and y coordinates of the predicted mask center point. GX C GY The x and y coordinates represent the actual location of the center point of the annotation.

[0041] In another aspect, the present invention also provides an electronic device comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described method.

[0042] In another aspect, the present invention also provides a computer-readable storage medium storing at least one instruction that is loaded and executed by a processor to implement the above-described method.

[0043] The beneficial effects of the technical solution provided by this invention include at least the following:

[0044] 1. This invention employs a cross-stage backbone network based on positional attention. Compared with ResNet, a common feature extraction network for instance segmentation, the network structure of the cross-stage backbone network based on positional attention used in this invention is reduced, fully extracting positional features, which is beneficial to further improve the accuracy of instance segmentation.

[0045] 2. This invention improves the loss of location information by introducing the fusion of low-level location information flow and high-level semantic information to form a bidirectional pyramid; at the same time, it improves the problem of small targets being difficult to segment due to the elimination of bounding boxes by introducing cross-stage mask feature fusion.

[0046] 3. This invention employs a novel loss function. Addressing the issue of unreasonable positive and negative sample allocation in instance segmentation tasks, this invention proposes an adaptive minimum loss matching method, which can effectively mitigate the segmentation accuracy loss caused by target occlusion in the image, thereby further improving instance segmentation accuracy. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of the execution flow of the real-time instance segmentation method with cross-stage feature information fusion provided in this embodiment of the invention;

[0049] Figure 2 This is a structural diagram of the instance segmentation model provided in the embodiments of the present invention;

[0050] Figure 3 This is a cross-stage fusion network structure diagram provided in an embodiment of the present invention;

[0051] Figure 4 This is a structural diagram of the positional attention mechanism provided in an embodiment of the present invention;

[0052] Figure 5 This is a structural diagram of feature information fusion provided in the embodiments of the present invention;

[0053] Figure 6 This is a comparison chart of the results of seven instance segmentation experiments on the COCO dataset provided in this embodiment of the invention;

[0054] Figure 7 This is a comparison chart of the present invention with other models on the COCO dataset;

[0055] Figure 8 This is a comparison chart of the present invention with other models on the Cityscapes dataset. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0057] First Embodiment

[0058] To address the issue that most current instance segmentation algorithms cannot simultaneously meet the requirements of segmentation accuracy and real-time performance, this embodiment proposes a real-time instance segmentation method for street scenes in the field of intelligent transportation. This method combines attention mechanisms and feature information fusion, employing cross-stage fusion, feature concatenation, and the introduction of underlying location information. By reducing the computational load of the backbone network and improving the segmentation of small targets and addressing target occlusion issues, the overall segmentation accuracy of the model is further enhanced. This real-time instance segmentation method can be implemented by an electronic device, such as a terminal or a server. The execution flow of this real-time instance segmentation method is as follows: Figure 1 As shown, it includes the following steps:

[0059] S1, improve the SOLOv2 model to reduce the computational cost of the backbone network, improve the problem of missing segmentation of small targets and target occlusion, and obtain the instance segmentation model; the instance segmentation model includes a cross-stage fusion backbone network based on positional attention, a bidirectional feature pyramid network and a cross-stage mask feature fusion structure;

[0060] S2, train the instance segmentation model to obtain a trained instance segmentation model;

[0061] S3 uses the trained instance segmentation model to process the image to be segmented and obtain the segmentation result.

[0062] Specifically, the instance segmentation model in this embodiment is as follows: Figure 2 As shown, the improvements to the original network in this embodiment include: First, a cross-stage fusion backbone network based on positional attention is proposed, which improves the model accuracy and reduces the computational cost; then, a bidirectional feature pyramid network is fused to improve the loss of shallow positional information; at the same time, a cross-stage mask feature fusion is designed to solve the problem of missing segmentation of small targets; finally, an adaptive minimum loss matching method is proposed to improve the segmentation accuracy loss caused by target occlusion in the image.

[0063] The following is a detailed description of the method for improving the SOLOv2 model used in this embodiment.

[0064] 1. Feature Extraction Network

[0065] First, a cross-stage fusion structure is adopted to improve the original ResNet backbone network of the SOLOv2 model. This reduces the number of parameters in the convolutional layers and reuses feature maps, thereby reducing memory consumption and improving the model's training efficiency. Second, considering the advantages of the Mish activation function, such as training stability, non-monotonicity, no upper bound, and smoothness, the ReLU activation function is replaced. The improved network structure is as follows: Figure 3 As shown. Finally, to fully extract the positional information of the underlying feature map, a positional attention mechanism is designed to insert between residual blocks, with the module structure as follows. Figure 4 As shown.

[0066] 2. Feature Fusion Structure Design

[0067] First, by introducing a bottom-up information flow, a bidirectional feature pyramid network (TWFPN) was designed to supplement the missing positional information. Second, addressing the original FPN network's operation of scaling feature layers to the same size before stacking, the high-level semantic layer P5 was upsampled by 2x and concatenated with the lower-level feature N4 along the channel dimension. Furthermore, semantic information was enhanced through cross-stage 8x upsampling. A cross-stage mask information fusion structure was designed, incorporating global semantic information into the mask feature branch while considering computational complexity, thus improving the model's ability to detect and segment small targets. The feature fusion structure is as follows: Figure 5 As shown.

[0068] 3. Adaptive minimum loss function matching method

[0069] To address the occlusion issue in SOLOv2 caused by multiple targets falling into the same grid, the first step is to improve the function L... Mask An influence factor ε is introduced to adaptively adjust the weight information of the prediction mask, improving instance segmentation accuracy. Then, the sigmoid function is used to normalize the category and mask losses, and the two losses are combined through summation. Finally, the losses are compared, and the network is trained using the minimum loss.

[0070] Among them, L Mask ε and Loss are defined as follows:

[0071]

[0072]

[0073] Loss = sigmoid(L Cate )+sigmoid(λL Mask )

[0074] Among them, L CateThis represents the category loss function, calculated using focus loss, where P is the set of positive samples, and N... pos It is the number of positive samples, m i Let g represent the prediction mask for the i-th feature point. i Let C represent the true mask of the i-th feature point, with λ set to 3. PX C PY C represents the x and y coordinates of the predicted mask center point. GX C GY The x and y coordinates represent the actual location of the center point of the annotation.

[0075] The performance of the instance segmentation model in this embodiment will be verified below.

[0076] This embodiment selects two public instance segmentation datasets, MS COCO and Cityscapes, for model performance analysis. MS COCO is a large-scale multi-task computer vision dataset containing images and corresponding semantic annotations for various common scenes. Cityscapes is a large-scale urban street scene image dataset used for scene understanding research, annotated with high-quality, dense semantic and instance-level annotations. These two datasets are commonly used in instance segmentation tasks. This embodiment converts the Cityscapes dataset to the COCO dataset format and reduces the original categories in the Cityscapes dataset, retaining common targets in street scenes such as cars, pedestrians, trucks, buses, and riders.

[0077] For data preprocessing, data augmentation (random flipping, resizing, random cropping, etc.) and data normalization were employed, consistent with the original SOLOv2 algorithm. To reduce model computation, training and testing images were scaled to 1200*680 pixels. After multiple experiments, the model parameters were set as follows: batch size (batch_size) was 16, with 8 graphics cards × 2 images per card; SGD was selected as the optimizer, with an initial learning rate (base_lr) of 0.01 and momentum of 0.9; L2 regularization weight decay coefficient (weight_decay) was 0.0001; the number of training epochs was 40; and the learning rate decay rate (lr_decay) was 0.1, set at the 10th epoch of model training.

[0078] Figure 6This paper compares the segmentation results of the original SOLOv2 algorithm and the present invention on seven sets of instances on the COCO dataset. The top-bottom comparison of sets 1 and 2 shows that the original algorithm has shortcomings in segmenting boundary details, while the improved algorithm performs better in this area. The top-bottom comparison of set 3 and the left-right comparison of set 4 show that the original algorithm has label errors in distinguishing between birds and trucks, while the improved algorithm obtains the correct category labels. The comparison of sets 4, 5, 6, and 7 shows that in segmenting smaller targets, the original algorithm is prone to missing segments, while the algorithm of this invention handles the problem of missing segments of small targets such as people and cars in the image much better.

[0079] This invention evaluates the accuracy of different instance segmentation algorithms using Average Precision (AP) and the instance segmentation speed of different models using Frames Per Second (FPS). The performance of the real-time instance segmentation model with cross-stage feature information fusion proposed in this invention is compared and analyzed, and it is compared with models such as the two-stage instance segmentation method Mask R-CNN, the multi-stage instance segmentation method MaskFormer, and the single-stage instance segmentation method PolarMask. The comparison results on the two publicly available instance segmentation datasets mentioned above are as follows: Figure 7 , Figure 8 As shown in the figure, the instance segmentation model proposed in this invention has high segmentation accuracy and its inference speed meets real-time requirements.

[0080] In summary, this embodiment provides a real-time instance segmentation method based on cross-stage feature information fusion. The cross-stage backbone network based on positional attention, compared to the ResNet feature extraction network commonly used in instance segmentation, has a reduced network structure and fully extracts positional features, which is beneficial for further improving instance segmentation accuracy. By introducing the fusion of low-level positional information flow and high-level semantic information to form a bidirectional pyramid, the loss of positional information is mitigated. Simultaneously, cross-stage mask feature fusion is introduced to address the difficulty in segmenting small targets caused by bounding box removal. Furthermore, to address the unreasonable positive and negative sample allocation in instance segmentation tasks, this method proposes an adaptive minimum loss matching method, which can effectively improve the segmentation accuracy loss caused by target occlusion in the image, thereby further enhancing instance segmentation accuracy.

[0081] Second Embodiment

[0082] This embodiment provides a real-time instance segmentation system that integrates cross-stage feature information fusion, including:

[0083] The model improvement module is used to improve the SOLOv2 model to reduce the computational cost of the backbone network, improve the problems of missing segmentation of small targets and target occlusion, and obtain an instance segmentation model; wherein, the instance segmentation model includes a cross-stage fusion backbone network based on positional attention, a bidirectional feature pyramid network, and a cross-stage mask feature fusion structure;

[0084] The model training module is used to train the instance segmentation model obtained by the model improvement module to obtain a trained instance segmentation model.

[0085] The instance segmentation module is used to process the image to be segmented using the instance segmentation model trained by the model training module, and obtain the instance segmentation result.

[0086] The real-time instance segmentation system with cross-stage feature information fusion in this embodiment corresponds to the real-time instance segmentation method with cross-stage feature information fusion in the first embodiment described above. The functions implemented by each functional module in this real-time instance segmentation system with cross-stage feature information fusion correspond one-to-one with the process steps in the real-time instance segmentation method with cross-stage feature information fusion in the first embodiment. Therefore, it will not be described again here.

[0087] Third Embodiment

[0088] This embodiment provides an electronic device, which includes a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment.

[0089] The electronic device can vary considerably depending on its configuration or performance, and may include one or more processors (central processing units, CPUs) and one or more memories, wherein the memories store at least one instruction that is loaded by the processor and executed in accordance with the above method.

[0090] Fourth embodiment

[0091] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc. The instruction stored therein can be loaded and executed by a processor in a terminal.

[0092] Furthermore, it should be noted that the present invention can be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0093] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0095] It should also be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0096] Finally, it should be noted that the above description represents a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A real-time instance segmentation method involving cross-stage feature information fusion, characterized in that, include: The SOLOv2 model is improved to reduce the computational cost of the backbone network and improve the problems of missing small target segmentation and target occlusion, resulting in an instance segmentation model. The instance segmentation model includes a cross-stage fusion backbone network based on positional attention, a bidirectional feature pyramid network, and a cross-stage mask feature fusion structure. The instance segmentation model is trained to obtain a trained instance segmentation model; The trained instance segmentation model is used to process the image to be segmented to obtain the instance segmentation result; The improvements to the SOLOv2 model include: The original backbone network ResNet of the SOLOv2 model is improved by adopting a cross-stage fusion structure, reducing the number of parameters in the convolutional layers, reusing feature maps, thereby reducing memory consumption and improving the training efficiency of the model. The ReLU activation function is replaced with the Mish activation function; The design incorporates positional attention between residual blocks to extract positional information from the underlying feature map; By introducing the fusion of low-level location information flow and high-level semantic information, a bidirectional feature pyramid network (TWFPN) is designed to supplement the lack of location information. To address the issue of the original network FPN which scales up feature layers to the same size and then stacks them, we propose to upsample the high-level semantic layer P5 by 2 times and then concatenate it with the lower-level feature N4 in the channel dimension. We also enhance the semantic information by upsampling it by 8 times across stages. We design a cross-stage mask feature fusion structure that incorporates global semantic information into the mask feature branch while considering the computational cost, thereby improving the model's ability to detect and segment small targets. Design an adaptive minimum loss function matching method, including: firstly, in the function L... Mask An influence factor ε is introduced to adaptively adjust the weight information of the prediction mask, improving instance segmentation accuracy. Then, the sigmoid function is used to normalize the category and mask losses, and the two losses are combined through summation. Finally, the losses are compared, and the network is trained using the minimum loss. Wherein, L... Mask ε and Loss are defined as follows: Loss=sigmoid(L Cate )+sigmoid(λL Mask ) Among them, L Cate This represents the category loss function, calculated using focus loss, where P is the set of positive samples, and N... pos It is the number of positive samples, m i Let g represent the prediction mask for the i-th feature point. i Let C represent the true mask of the i-th feature point, with λ set to 3. PX C PY C represents the x and y coordinates of the predicted mask center point. GX C GY The x and y coordinates represent the actual location of the center point of the annotation.

2. A real-time instance segmentation system that fuses cross-stage feature information, characterized in that, include: The model improvement module is used to improve the SOLOv2 model to reduce the computational cost of the backbone network, improve the problems of missing segmentation of small targets and target occlusion, and obtain an instance segmentation model; wherein, the instance segmentation model includes a cross-stage fusion backbone network based on positional attention, a bidirectional feature pyramid network, and a cross-stage mask feature fusion structure; The model training module is used to train the instance segmentation model obtained by the model improvement module to obtain a trained instance segmentation model. The instance segmentation module is used to process the image to be segmented using the instance segmentation model trained by the model training module, and obtain the instance segmentation result. The model improvement module is specifically used for: The original backbone network ResNet of the SOLOv2 model is improved by adopting a cross-stage fusion structure, reducing the number of parameters in the convolutional layers, reusing feature maps, thereby reducing memory consumption and improving the training efficiency of the model. The ReLU activation function is replaced with the Mish activation function; The design incorporates positional attention between residual blocks to extract positional information from the underlying feature map; By introducing the fusion of low-level location information flow and high-level semantic information, a bidirectional feature pyramid network (TWFPN) is designed to supplement the lack of location information. To address the issue of the original network FPN which scales up feature layers to the same size and then stacks them, we propose to upsample the high-level semantic layer P5 by 2 times and then concatenate it with the lower-level feature N4 in the channel dimension. We also enhance the semantic information by upsampling it by 8 times across stages. We design a cross-stage mask feature fusion structure that incorporates global semantic information into the mask feature branch while considering the computational cost, thereby improving the model's ability to detect and segment small targets. Design an adaptive minimum loss function matching method, including: firstly, in the function L... Mask An influence factor ε is introduced to adaptively adjust the weight information of the prediction mask, improving instance segmentation accuracy. Then, the sigmoid function is used to normalize the category and mask losses, and the two losses are combined through summation. Finally, the losses are compared, and the network is trained using the minimum loss. Wherein, L... Mask ε and Loss are defined as follows: Loss=sigmoid(L Cate )+sigmoid(λL Mask ) Among them, L Cate This represents the category loss function, calculated using focus loss, where P is the set of positive samples, and N... pos It is the number of positive samples, m i Let g represent the prediction mask for the i-th feature point. i Let C represent the true mask of the i-th feature point, with λ set to 3. PX C PY C represents the x and y coordinates of the predicted mask center point. GX C GY The x and y coordinates represent the actual location of the center point of the annotation.

Images