High-precision bipartite image segmentation method and device based on prior mask

Abstract

The invention relates to the technical field of image processing and computer vision, in particular to a high-precision bipartite image segmentation method based on a prior mask. The method comprises the following steps: firstly, acquiring an RGB image to be segmented; then, the user forms a polygon covering the target object through dotting marking, a priori mask is generated, pixels in the polygon are set to be 1, and pixels outside the polygon are set to be 0; and finally, inputting the RGB image and the prior mask into a pre-trained bipartite image segmentation model to obtain a target foreground segmentation result. The bipartite image segmentation model comprises a prior information fusion stage, a target identification stage and a result refinement stage, the segmentation accuracy can be ensured, and the bipartite image segmentation model has excellent performance in detail processing and complex scenes. The invention also provides a high-precision bipartite image segmentation device based on the prior mask, which is used for executing the method to realize high-precision image segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and computer vision, and in particular to a high-precision binary image segmentation method and device based on a priori mask. Background Art

[0002] High-precision dichotomous image segmentation (DIS) refers to the technology of accurately detecting and segmenting target foreground objects in high-resolution images. It is widely used in fields such as image editing, 3D reconstruction, augmented reality, medical image processing, and robotic operation. With the development of modern industry and technology, especially in heavy industrial environments, the demand for high-resolution and refined image processing is increasing. This requires not only that the segmentation algorithm has higher accuracy, but also that it can cope with complex application scenarios such as dynamic environments, complex backgrounds, and multi-target scenes. However, although the widespread application of deep learning technology has promoted the development of image segmentation technology, high-precision dichotomous image segmentation still faces a series of technical challenges that need to be solved.

[0003] First, when dealing with complex backgrounds, the existing image segmentation methods often have a vague definition of the target object, resulting in unclear distinction between the foreground and background, especially when the background is complex or the edge details of the foreground object are complex, making it difficult to accurately locate the target object. Secondly, although the segmentation algorithm can achieve good segmentation results in some scenes, it is often difficult to retain the high-resolution details of the foreground object at the same time. In many high-precision applications, the lack of details will lead to inaccurate segmentation results, which in turn affects the performance and effect of downstream applications.

[0004] At present, most high-precision image segmentation methods can be divided into two categories: one is based on convolutional neural networks (CNNs), which gradually refine the segmentation results through a multi-stage architecture. High-level features are used to provide global semantic information, while low-level features help refine the boundaries of target objects. However, although this type of method can improve the segmentation accuracy to a certain extent, the multi-stage processing architecture often leads to slow inference speed and is difficult to meet the real-time requirements in practical applications. In addition, this method still has shortcomings when processing the edges of subtle objects, resulting in blurred boundaries. Another type of method is based on Transformer, which refines the feature layer through a cross-scale feedback mechanism to improve the performance of low-resolution segmentation. However, although the Transformer architecture can effectively capture long-distance dependencies, its ability to process the boundaries of foreground objects in the image is still insufficient, especially when dealing with complex high-precision tasks, the segmentation effect still needs to be improved.

[0005] The existing technology also has the problem of inconsistent understanding of the same task by the model, that is, different methods handle the same segmentation task differently, resulting in inconsistent segmentation results. This instability affects the overall performance and repeatability of the segmentation. In addition, due to the lack of effective processing of high precision and detail retention, some methods cause the performance of the model to drop significantly when facing complex scenes, especially in scenes with complex backgrounds or variable edge details of target objects. Therefore, the existing high-precision binary image segmentation methods still have much room for improvement in terms of accuracy, detail retention, edge processing, and inference speed.

[0006] In general, the main problems faced by high-precision binary image segmentation include: inaccurate positioning of target objects in complex backgrounds, difficulty in retaining high-resolution details at the same time, the inference speed of existing methods is difficult to meet real-time requirements, and the processing results of the model for the same task are inconsistent. These problems significantly limit the breadth and reliability of existing technologies in practical applications. There is an urgent need for a technical solution that can simultaneously improve segmentation accuracy, retain details and speed up inference speed to better meet the challenges of high-precision image segmentation. Summary of the invention

[0007] In view of the problems of inaccurate positioning of target objects and poor detail segmentation effect in the prior art, the present invention provides a high-precision binary image segmentation method and device based on a priori mask. By combining the user-annotated priori mask with the high-resolution RGB image and using a multi-stage fusion, decoding and refinement processing technical solution, the technical effect of accurately positioning the foreground object and retaining the high-resolution details is achieved.

[0008] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0009] A high-precision binary image segmentation method based on a priori mask comprises the following steps:

[0010] S1, obtain the RGB image to be segmented;

[0011] S2, add a polygon covering the desired object in each RGB image to generate a priori mask;

[0012] The generation of the priori mask includes: on the basis of the RGB image, forming a polygon by dotting and marking, setting the pixels inside the polygon to "1", and setting the pixels outside the polygon to "0", and outputting the priori mask;

[0013] S3, inputting the RGB image and the priori mask into a pre-trained binary image segmentation model to obtain a target foreground segmentation result;

[0014] The binary image segmentation model includes a multi-stage data processing system for generating segmentation results in combination with prior information. The system is composed of multiple data processing modules, each of which performs specific data operations to extract image features and generate segmentation results.

[0015] Preferably, the binary image segmentation model includes a prior information fusion stage, a target recognition stage and a result refinement stage; wherein: the prior information fusion stage fuses the prior information with the RGB image to obtain fused image features with prior auxiliary information; the fused image features are input into the scale aggregation decoder of the target recognition stage to obtain a preliminary segmentation prediction result; the preliminary segmentation prediction result is subdivided to obtain a target foreground segmentation result. The image segmentation is made more accurate by combining the prior information fusion stage and the scale aggregation decoder. The introduction of prior information improves the model's sensitivity to the position of the target object, and the segmentation effect is more accurate, especially for the processing of complex backgrounds. The result refinement stage further improves the detail retention of the segmentation results.

[0016] Preferably, the prior information fusion stage obtains the fusion image features of the RGB image and the prior information, comprising the following steps:

[0017] S201, performing a fusion pooling operation on the RGB image and the priori mask to obtain a feature vector with auxiliary position information;

[0018] S202: Input the feature vector into a priori information fusion module for updating, and obtain updated first data with priori auxiliary information.

[0019] In this way, by fusing the prior mask and RGB image into a pooling operation, the user-annotated information can be effectively used to locate the foreground object. The fused image features contain more spatial location information, and the model can handle segmentation tasks in complex scenes more accurately.

[0020] Preferably, the prior information fusion module calculates the first data by the following formula:

[0021]

[0022] F pfm =Conv[I rgb ,I m ,V]+I rgb ,

[0023] Among them, GAP represents the global average pooling operation, I rgb is the RGB image, I m is the prior mask, represents element-by-element multiplication, V is the feature vector with auxiliary position information, [I rgb ,I m ,V] represents I rgb ,I m , V is a channel series operation, Conv represents the convolution layer, F pfm The fusion image feature calculation method based on a specific formula can effectively combine RGB images and prior information, improving the model's ability to segment foreground and background details. The global average pooling operation further optimizes the aggregation of feature information and improves the robustness of the segmentation model.

[0024] Preferably, the target identification stage includes the following steps:

[0025] S301, performing image feature compression processing on the first data to obtain second data;

[0026] S302, inputting the first data and the second data into an encoder to obtain hierarchical features; wherein the first data is input into a ResNet encoder to obtain hierarchical features of the first data, represented as {R i}, subscript i∈{1, 2, 3, 4}; the second data is input into the Swin-Transformer encoder to obtain the hierarchical features of the second data, represented as {S i}, subscript i∈{1, 2, 3, 4};

[0027] S303, inputting feature pairs of adjacent levels in the second data into a neighbor aggregation module to obtain adjacent aggregation features;

[0028] S304: input the first data and the adjacent aggregated features into a scale aggregated decoder module to perform cross-scale aggregated decoding to obtain updated scale aggregated features;

[0029] S305 , generating a preliminary segmentation prediction result according to the scale aggregation feature; the preliminary segmentation prediction result includes preliminary prediction features.

[0030] Through the above multi-level feature compression and encoding steps, the preferred solution can effectively capture the details and global information in the image. After fusing high-resolution and low-resolution image features, the model performs better in edge processing of foreground objects and suppression of background interference, and the segmentation results are more accurate.

[0031] Preferably, the formula for cross-scale aggregation decoding performed by the scale aggregation decoder in step S304 is as follows:

[0032] G(S t ,S t-1 )=BConv([BConv(St ),S t-1 ])+Conv(S t )

[0033] f g =G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3)))

[0034] f d =G(BConv(R1+UP(R2)),UP(f g ))

[0035] C=P(UP(f d ))

[0036] Among them, BConv represents a processing module composed of a convolutional layer, a batch normalization layer, and a linear rectification function in sequence; G(·,·) represents a proximity aggregation operation module, UP represents an upsampling module, and P represents a prediction head; S t Indicates the second data of the current level, S t-1 Indicates the second data of the previous layer; G(S t ,S t-1 ) represents the current adjacent aggregation feature; f d represents the scale aggregation feature, and C is the preliminary segmentation prediction result. In this way, feature fusion is performed through the cross-scale aggregation decoder module, so that image features at different levels can fully interact with each other, improving the model's ability to segment multi-scale objects. The combination of convolution operation and upsampling module improves the performance of the model in high-precision image segmentation tasks, especially when processing edge details.

[0037] Preferably, the result refinement stage also includes refining the scale aggregation features, specifically including:

[0038] S401, refining the hierarchical features of the first data to obtain refined features of the first data; the refining is to gradually refine the high-level features to the low-level features;

[0039] S402, refining the refined features of the first data and the preliminary prediction features to obtain refined prediction features;

[0040] S403: Obtain target foreground segmentation results according to the refined prediction features.

[0041] This preferred solution further improves the ability to process the details of the target object by refining the high-resolution hierarchical features. The refinement module effectively corrects the initial predicted features, making the final segmentation result more accurate and retaining more details.

[0042] Preferably, the refining is performed by the following formula:

[0043]

[0044] Among them, the superscript i represents the number of layers in the hierarchical feature sequence, i∈{1, 2, 3}, and Concat represents concatenation in the channel dimension. Represents the different levels of refinement features after gradual refinement;

[0045] The formula for obtaining the target foreground segmentation result is as follows:

[0046]

[0047] in, and is the refined features of the first data of the first layer and the second layer, is the intermediate refined prediction feature, is the final refined prediction feature, and S is the target foreground segmentation result.

[0048] By refining, the details of the foreground objects can be further optimized during the segmentation prediction process. This preferred solution effectively integrates high-level and low-level features to ensure the accuracy and consistency of the segmentation results in complex backgrounds.

[0049] Preferably, the high-precision binary image segmentation model is trained using a structural learning loss function, and the structural learning loss function is:

[0050]

[0051] in, is the binomial cross entropy loss function, is the intersection-over-union loss function, and G represents the ground-truth map. By introducing the structural learning loss function, this preferred solution can not only optimize the accuracy of segmentation during training, but also improve the matching degree of foreground and background areas through the intersection-over-union loss function. This loss function design enables the model to show stronger generalization ability in a variety of scenarios.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The present invention provides a high-precision binary image segmentation method based on a priori mask, which improves the accuracy of target object positioning. Through the polygonal priori mask input by the user, the specific position of the foreground object can be clarified during the image segmentation process, thereby avoiding the model's vague definition of the target foreground and background. The user's prior annotation provides clear location information, which solves the problem of unclear positioning of the target foreground object in high-precision binary image segmentation through positioning, and can better utilize the performance of the model, so that it can pay more attention to the area containing the foreground target object, and can obtain a more accurate segmentation result. The situation of unclear positioning of the target object is reduced, especially in complex backgrounds.

[0054] Since the prior mask introduces annotations with location information, the model can focus more on the area containing the target object and reduce the misjudgment of irrelevant areas. This method can not only accurately segment the target object, but also show higher accuracy in edge details, thereby improving the overall segmentation quality. Users only need to perform simple dot annotation to generate a rough polygon mask, which is simpler and faster than the traditional manual annotation method, greatly reducing the cost and difficulty of manual annotation, while improving user acceptance.

[0055] The high-precision binary image segmentation method based on prior mask provided by the present invention does not rely on a single segmentation strategy, but integrates user prior information to enable the model to flexibly adapt to different scenarios and task requirements. Regardless of the complexity of the image segmentation task or the diversity of the target objects, this framework can achieve good segmentation effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of a high-precision binary image segmentation method based on a priori mask according to Embodiment 1 of the present invention;

[0057] Figure 2 A schematic diagram of the network structure of a high-precision binary image segmentation method based on a priori mask according to Embodiment 2 of the present invention;

[0058] Figure 3 A schematic diagram of a priori mask marked by a user in Example 2 of the present invention;

[0059] Figure 4 Embodiment 3 of the present invention is a result comparison diagram of a high-precision binary image segmentation method based on a priori mask and an existing method. DETAILED DESCRIPTION

[0060] The high-precision binary image segmentation method based on a priori mask provided by the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. However, this should not be understood as the scope of the above-mentioned subject matter of the present invention being limited to the following embodiments. All technologies implemented based on the content of the present invention belong to the scope of the present invention. Combined with the following description, the advantages and features of the present invention will become clearer. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0061] Example 1

[0062] This embodiment provides a high-precision binary image segmentation method based on a priori mask. The method steps are as follows: Figure 1 As shown. This method aims to improve the accuracy of binary image segmentation by introducing a priori masks, especially to achieve more accurate segmentation in foreground object positioning and detail processing under complex backgrounds. By combining the prior information annotated by the user with the deep learning ability of the model, the overall accuracy and application efficiency of the segmentation results are improved. Specifically, the following steps are included:

[0063] S1, obtain the RGB image to be segmented;

[0064] S2, add a polygon covering the desired object in each RGB image to generate a priori mask;

[0065] The generation of the priori mask includes: on the basis of the RGB image, forming a polygon by dotting and marking, setting the pixels inside the polygon to "1", and setting the pixels outside the polygon to "0", and outputting the priori mask;

[0066] S3, inputting the RGB image and the priori mask into a pre-trained binary image segmentation model to obtain a target foreground segmentation result;

[0067] The binary image segmentation model includes a multi-stage data processing system for generating segmentation results in combination with prior information. The system is composed of multiple data processing modules, each of which performs specific data operations to extract image features and generate segmentation results.

[0068] Preferably, the binary image segmentation model includes a prior information fusion stage, a target recognition stage and a result refinement stage; wherein: the prior information fusion stage fuses the prior information with the RGB image to obtain fused image features with prior auxiliary information; the fused image features are input into the scale aggregation decoder of the target recognition stage to obtain a preliminary segmentation prediction result; the preliminary segmentation prediction result is subdivided to obtain a target foreground segmentation result. The image segmentation is made more accurate by combining the prior information fusion stage and the scale aggregation decoder. The introduction of prior information improves the model's sensitivity to the position of the target object, and the segmentation effect is more accurate, especially for the processing of complex backgrounds. The result refinement stage further improves the detail retention of the segmentation results.

[0069] Preferably, the prior information fusion stage obtains the fusion image features of the RGB image and the prior information, comprising the following steps:

[0070] S201, performing a fusion pooling operation on the RGB image and the priori mask to obtain a feature vector with auxiliary position information;

[0071] S202: Input the feature vector into a priori information fusion module for updating, and obtain updated first data with priori auxiliary information.

[0072] In this way, by fusing the prior mask and RGB image into a pooling operation, the user-annotated information can be effectively used to locate the foreground object. The fused image features contain more spatial location information, and the model can handle segmentation tasks in complex scenes more accurately.

[0073] Preferably, the prior information fusion module calculates the first data by the following formula:

[0074]

[0075] F pfm =Conv[I rgb ,I m ,V]+I rgb ,

[0076] Among them, GAP represents the global average pooling operation, I rgb is the RGB image, I m is the prior mask, represents element-by-element multiplication, V is the feature vector with auxiliary position information, [I rgb ,I m ,V] represents I rgb ,I m , V is a channel series operation, Conv represents the convolution layer, F pfmThe fusion image feature calculation method based on a specific formula can effectively combine RGB images and prior information, improving the model's ability to segment foreground and background details. The global average pooling operation further optimizes the aggregation of feature information and improves the robustness of the segmentation model.

[0077] Preferably, the target identification stage includes the following steps:

[0078] S301, performing image feature compression processing on the first data to obtain second data;

[0079] S302, inputting the first data and the second data into an encoder to obtain hierarchical features; wherein the first data is input into a ResNet encoder to obtain hierarchical features of the first data, represented as {R i}, subscript i∈{1, 2, 3, 4}; the second data is input into the Swin-Transformer encoder to obtain the hierarchical features of the second data, represented as {S i}, subscript i∈{1, 2, 3, 4};

[0080] S303, inputting feature pairs of adjacent levels in the second data into a neighbor aggregation module to obtain adjacent aggregation features;

[0081] S304: input the first data and the adjacent aggregated features into a scale aggregated decoder module to perform cross-scale aggregated decoding to obtain updated scale aggregated features;

[0082] S305 , generating a preliminary segmentation prediction result according to the scale aggregation feature; the preliminary segmentation prediction result includes preliminary prediction features.

[0083] Through the above multi-level feature compression and encoding steps, the preferred solution can effectively capture the details and global information in the image. After fusing high-resolution and low-resolution image features, the model performs better in edge processing of foreground objects and suppression of background interference, and the segmentation results are more accurate.

[0084] Preferably, the formula for cross-scale aggregation decoding performed by the scale aggregation decoder in step S304 is as follows:

[0085] G(S t ,S t-1 )=BConv([BConv(S t ),S t-1 ])+Conv(S t )

[0086] f g=G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3)))

[0087] f d =G(BConv(R1+UP(R2)),UP(f g ))

[0088] C=P(UP(f d ))

[0089] Among them, BConv represents a processing module composed of a convolutional layer, a batch normalization layer, and a linear rectification function in sequence; G(·,·) represents a proximity aggregation operation module, UP represents an upsampling module, and P represents a prediction head; S t Indicates the second data of the current level, S t-1 Indicates the second data of the previous layer; G(S t ,S t-1 ) represents the current adjacent aggregation feature; f d represents the scale aggregation feature, and C is the preliminary segmentation prediction result. In this way, feature fusion is performed through the cross-scale aggregation decoder module, so that image features at different levels can fully interact with each other, improving the model's ability to segment multi-scale objects. The combination of convolution operation and upsampling module improves the performance of the model in high-precision image segmentation tasks, especially when processing edge details.

[0090] Preferably, the result refinement stage also includes refining the scale aggregation features, specifically including:

[0091] S401, refining the hierarchical features of the first data to obtain refined features of the first data; the refining is to gradually refine the high-level features to the low-level features;

[0092] S402, refining the refined features of the first data and the preliminary prediction features to obtain refined prediction features;

[0093] S403: Obtain target foreground segmentation results according to the refined prediction features.

[0094] This preferred solution further improves the ability to process the details of the target object by refining the high-resolution hierarchical features. The refinement module effectively corrects the initial predicted features, making the final segmentation result more accurate and retaining more details.

[0095] Preferably, the refining is performed by the following formula:

[0096]

[0097] Among them, the superscript i represents the number of layers in the hierarchical feature sequence, i∈{1, 2, 3}, and Concat represents concatenation in the channel dimension. Represents the different levels of refinement features after gradual refinement;

[0098] The formula for obtaining the target foreground segmentation result is as follows:

[0099]

[0100] in, and is the refined features of the first data of the first layer and the second layer, is the intermediate refined prediction feature, is the final refined prediction feature, and S is the target foreground segmentation result.

[0101] By refining, the details of the foreground objects can be further optimized during the segmentation prediction process. This preferred solution effectively integrates high-level and low-level features to ensure the accuracy and consistency of the segmentation results in complex backgrounds.

[0102] Preferably, the high-precision binary image segmentation model is trained using a structural learning loss function, and the structural learning loss function is:

[0103]

[0104] in, is the binomial cross entropy loss function, is the intersection-over-union loss function, and G represents the ground-truth map. By introducing the structural learning loss function, this preferred solution can not only optimize the accuracy of segmentation during training, but also improve the matching degree of foreground and background areas through the intersection-over-union loss function. This loss function design enables the model to show stronger generalization ability in a variety of scenarios.

[0105] Example 2

[0106] As a preferred embodiment of Embodiment 1, Embodiment 2 adds a melting pooling operation in the prior information fusion stage, and further refines the target recognition stage through multiple steps, providing a specific implementation method of a high-precision binary image segmentation method based on a priori mask. Specifically:

[0107] S1, obtain a high-resolution RGB image to be segmented;

[0108] S2, the user adds a polygon covering the required object in each image, and generates a priori mask based on such polygon with user prior information;

[0109] The generation of a user priori mask includes: the user simply marks the foreground target object based on the image, and the points are connected to form a polygon covering the required object. The pixels inside the polygon are explicitly set to "1", and the pixels outside the polygon are explicitly set to "0", and the user priori mask can be obtained. Annotating polygons is very convenient, that is, only a few clicks are required, so it has a high user acceptance.

[0110] S3. Input the RGB image and the prior mask into a pre-trained high-precision binary image segmentation model to obtain a high-precision target foreground segmentation result.

[0111] The high-precision binary image segmentation model includes: a prior information fusion stage, a target recognition stage, and a result refinement stage. The prior information fusion stage fuses the prior information with the RGB image, and then inputs it into the scale aggregation decoder of the target recognition stage, and finally refines the result to obtain an accurate foreground segmentation result.

[0112] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, the priori information fusion stage realizes the fusion of priori information and image through the following steps, including:

[0113] S201, performing a fusion pooling operation on the image and the priori mask to obtain a feature vector V with auxiliary position information;

[0114] S202: Input the feature vector into a priori information fusion module for updating, and obtain updated fused image features with priori auxiliary information.

[0115] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, in step 202, the RGB image and the a priori mask are fused using the following formula:

[0116]

[0117] F pfm =Conv[I rgb ,I m ,V]+I rgb ,

[0118] Among them, I rgb Represents RGB image; I m represents the prior mask, where the pixels inside the polygon in the prior mask are 1 and the pixels outside the polygon are 0; represents element-by-element multiplication, GAP represents global average pooling operation, [,] represents channel concatenation operation, Conv represents convolution layer, V represents feature vector with auxiliary position information, and F pfm To fuse image features.

[0119] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, the target recognition stage captures and recognizes the rough foreground target object through the following steps, including:

[0120] S301, compressing the scale of the high-resolution fused image features to obtain low-resolution fused image features.

[0121] S302, inputting two fused image features into an encoder to obtain a series of hierarchical feature pairs;

[0122] S303, taking the adjacent level feature pairs (S t ,S t-1 ) is input to the neighbor aggregation module to obtain the neighbor aggregation features;

[0123] S303, taking the high-resolution feature sequence pair and the obtained adjacent aggregated features, and inputting them into the scale aggregate decoder module for decoding to obtain preliminary prediction features;

[0124] S304: Generate a preliminary segmentation prediction map according to the decoded preliminary prediction features.

[0125] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, in step 301, the high-resolution fusion feature is input into the ResNet encoder to obtain a series of hierarchical feature pairs {R i}(i=1,2,3,4), the low-resolution fusion features are input into the Swin-Transformer encoder to obtain a series of hierarchical feature pairs {S i}(i=1,2,3,4).

[0126] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, the neighbor aggregation module uses the following formula to achieve the aggregation of adjacent hierarchical features:

[0127] G(S t ,S t-1 )=BConv([BConv(S t ),S t-1 ])+Conv(S t )

[0128] Among them, BConv represents a processing module composed of a convolutional layer, a batch normalization layer, and a linear rectification function in sequence. t-1 Represents the low-resolution fusion features of the current level, S t-1 Represents the fusion features of the previous layer. G(S t ,S t-1 ) represents the output of this neighbor aggregation module, that is, the current neighbor aggregation feature.

[0129] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, the scale aggregation decoder module performs cross-scale aggregation decoding on the received features by the following formula:

[0130] f g =G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3)))

[0131] f d =G(BConv(R1+UP(R2)),UP(f g ))

[0132] C=P(UP(f d ))

[0133] Among them, G(·,·) represents the neighbor aggregation operation module, UP represents the upsampling module, which is used to align the features of different levels in the scale dimension, and P represents the prediction head. g represents the intermediate decoding feature, and f d represents the preliminary prediction features after scale aggregation decoding, and C is the preliminary segmentation prediction map.

[0134] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, the result refinement stage refines the preliminary segmentation prediction obtained previously by the following steps, including

[0135] S401, performing cross-level refining processing on the above series of high-resolution hierarchical feature pairs, and gradually refining the high-level abstract features to the lower levels to obtain a series of high-resolution refined feature pairs;

[0136] S402: Input the high-resolution refined feature pair and the preliminary prediction features into a refinement module to obtain refined prediction features, and generate a high-precision segmentation map according to the refined prediction features.

[0137] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, in step 401, the following formula is used to refine the multiple high-resolution hierarchical features:

[0138]

[0139] The superscript i∈{1, 2, 3} represents the number of layers in the hierarchical feature sequence. Concat means concatenation in the channel dimension. Represents the different levels of refined features after gradual refinement.

[0140] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, the step 402 refines the received preliminary prediction feature f by the following formula: d use and Refine and refine to obtain refined prediction features and generate high-precision segmentation maps:

[0141]

[0142] Among them, P represents the prediction head and UP represents the upsampling module. and is the high-resolution refined features of the first and second layers above, f d It is the preliminary prediction feature after scale aggregation decoding. is the intermediate refined prediction feature, is the final refined prediction feature, and S generates a high-precision segmentation map for the refined prediction feature.

[0143] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, a structural learning loss function is used to train the high-precision binary image segmentation model to obtain the pre-trained high-precision binary image segmentation model.

[0144] Preferably, in the above-mentioned high-precision binary image segmentation method based on a priori mask, the joint learning loss function is:

[0145]

[0146] in, is the binomial cross entropy loss function, is the intersection-over-union loss function, S is the final high-precision segmentation map, and G represents the GT true value map.

[0147] Example 3

[0148] As a preferred embodiment of Example 2, this embodiment introduces a pyramid cross-scale decoding module to perform feature decoding, and also uses cross-level refinement processing in the result refinement stage, providing a better implementation method of a high-precision binary image segmentation method based on a priori mask.

[0149] In this embodiment, an image and a priori mask are input into a pre-trained high-precision segmentation model, and the final high-precision segmentation result is obtained from the decoder through forward propagation of the network. Among them, the high-resolution branch encoder is based on the common ResNet network structure, and the low-resolution branch is based on the common Swin-Transformer network structure. The method of this embodiment mainly includes the following steps:

[0150] S1. Obtain a high-resolution RGB image to be segmented and a user priori mask. The specific priori mask is generated by processing the polygon surrounded by the points marked by the user in the image, such as Figure 3 As shown, the inside of the polygon is set to "1" to represent the white foreground area containing the target object, and the outside of the polygon is set to "0" to represent the black background area. At the same time, both images are cropped to a size of 1024×1024.

[0151] S2. Input the RGB image and the user prior mask into the prior information fusion module and use the fusion operator to embed the prior position information into the image to obtain the fused image feature F. pfm The high-resolution fused image feature scale is compressed to obtain the low-resolution fused image feature. The high-resolution feature and the low-resolution feature are input to the ResNet encoder and the Swin-Transformer decoder to obtain a series of feature pairs {R i} and {S i}, with high-resolution features {R i} as an example, the feature resolutions are: 64x256x256, 128x128x128, 256x64x64, 512x32x32.

[0152] S3. Take the high-resolution feature sequence pair and the low-resolution feature sequence, and input them into the multi-scale aggregation decoder in the target recognition stage to obtain the decoded preliminary prediction features. Specifically, this embodiment introduces the pyramid cross-scale decoding module of the document "Xie C, Xia C, Ma M, et al. Pyramid Grafting Network for One-Stage High Resolution Saliency Detection[J]. 2022" to perform feature decoding, such as Figure 2 As shown, the specific steps are as follows:

[0153] S31, taking the adjacent level feature pairs in the low-resolution feature sequence pair (S t ,S t-1 ) is input into the neighboring aggregation module to obtain the adjacent aggregation features, and the following formula is used to realize the aggregation of adjacent hierarchical features:

[0154] G(S t ,S t-1 )=BConv([BConv(S t ),S t-1 ])+Conv(S t )

[0155] Among them, BConv represents a processing module composed of a convolutional layer, a batch normalization layer, and a linear rectification function in sequence.t Represents the low-resolution fusion features of the current level, S t-1 Represents the fusion features of the previous layer. G(S t ,S t-1 ) represents the output of this neighbor aggregation module, that is, the current neighbor aggregation feature.

[0156] S32, take the high-resolution feature sequence pair and the obtained adjacent aggregation feature, and input it into the scale aggregation decoder module to obtain the preliminary prediction feature f after multi-scale aggregation decoding d .

[0157] Specifically, the scale aggregation decoder module performs coarse prediction decoding on the received features through the following formula:

[0158] f g =G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3)))

[0159] f d =G(BConv(R1+UP(R2)),UP(f g ))

[0160] C=P(UP(f d ))

[0161] Among them, G(·,·) represents the neighbor aggregation operation module, UP represents the upsampling module, which is used to align the features of different levels in the scale dimension, and P represents the prediction head. g represents the intermediate decoding feature, and f d represents the scale aggregation feature, and C is the preliminary segmentation prediction map.

[0162] S4, high-resolution feature sequence {R i} Perform cross-level refinement and then combine with the preliminary prediction feature f described in S3 d All are input into the refinement module in the result refinement stage to obtain the refined prediction features, such as Figure 3 The specific steps are as follows:

[0163] S41, according to the following formula, the high-resolution hierarchical feature sequence {R i}Perform cross-level refining processing:

[0164]

[0165] The superscript i∈{1, 2, 3} represents the number of layers in the hierarchical feature sequence. Concat means concatenation in the channel dimension. Represents the different levels of refined features after gradual refinement.

[0166] S42, the preliminary prediction feature f d Refine according to the following formula:

[0167]

[0168] in, and is the high-resolution refined features of the first and second layers above, f d is the scale aggregation feature. is the intermediate refined prediction feature, is the final refined prediction feature.

[0169] S5, the final refined prediction features updated in S4 Input into the prediction head, after Sigmoid activation, the final detection result is obtained, such as Figure 2 shown.

[0170] S6. Before using the network, first train the high-precision binary image segmentation model, calculate the model training loss function and perform error back propagation and gradient descent optimization to optimize the network parameters in the prior information fusion stage, target recognition stage, and result refinement stage.

[0171] Specifically, the total training loss function of the model consists of two parts: the binomial cross entropy loss function and the intersection-over-union loss function, which are calculated using the following formula:

[0172]

[0173] in, is the binomial cross entropy loss function, is the intersection-over-union loss function, S is the final high-precision segmentation map, and G represents GT (true value map).

[0174] In a further embodiment of the present invention, we conducted a comparative experiment with three existing high-precision binary image segmentation methods (ISNet, RMFormer, HitNet). Figure 4As shown. In the experiment, the segmentation method of the present invention has shown significant advantages in a variety of scenarios. First, in terms of edge detail preservation of the target, the model of the present invention (Ours) can more accurately capture foreground objects of complex shapes, such as subtle features such as the antennae of instrument players and insects. In contrast, the segmentation results of other models such as ISNet and RMFormer have blurred edges and are prone to missing some details. Secondly, in high-contrast scenes (such as the iron tower), the model of the present invention can better distinguish the foreground and background, and the segmentation contours are clear, while other methods such as HitNet have the phenomenon of mis-segmentation when processing background noise. Therefore, the high-precision binary image segmentation model provided by the present invention is superior to existing methods in terms of accuracy and detail preservation, especially in detail processing and foreground segmentation of complex backgrounds. It shows higher accuracy.

[0175] Example 4

[0176] This embodiment provides a high-precision binary image segmentation device based on a priori mask, which is intended to perform image segmentation tasks to improve the accuracy and efficiency of image segmentation, especially foreground object segmentation under complex backgrounds. The device includes:

[0177] 1. Processor: At least one processor is responsible for performing various image processing tasks, including image preprocessing, prior mask generation, fusion feature extraction, target recognition and result refinement, etc. The processor can quickly process large-scale image data to ensure the performance of the system in high-precision image segmentation tasks.

[0178] 2. Memory: The device is equipped with a memory that is communicatively connected to the processor. The memory stores instructions executable by the processor, which are based on the high-precision binary image segmentation method based on a priori mask described in the present invention, and guide the processor to complete the entire process from inputting a high-resolution RGB image to outputting a high-precision segmentation result. The memory can be a random access memory (RAM) or a read-only memory (ROM), which is used to temporarily store intermediate calculation results or maintain pre-trained parameters of the segmentation model.

[0179] 3. Instruction set: The instructions stored in the memory include image processing algorithms for performing the following steps:

[0180] Get a high-resolution RGB image to be segmented;

[0181] Generate a priori mask input by the user and fuse the mask with the RGB image;

[0182] The fused image is input into a high-precision binary image segmentation model to sequentially complete prior information fusion, target recognition, and result refinement;

[0183] Output high-precision foreground segmentation results.

[0184] 4. Input / output interface: The device also includes an image data input and output interface for receiving high-resolution RGB images and outputting the processed results. The interface can be a wired or wireless interface, adapting to a variety of image sources and external devices.

[0185] Through this design, the device can efficiently execute a high-precision binary image segmentation method based on a priori mask, which is suitable for a variety of application scenarios, such as autonomous driving, medical image processing, and industrial image detection, and has significant advantages in segmentation tasks with complex backgrounds and high precision requirements.

[0186] The above is only a detailed description of the specific implementation of the present invention, rather than a limitation of the present invention. Various substitutions, modifications and improvements made by those skilled in the relevant art without departing from the principle and scope of the present invention should be included in the protection scope of the present invention.

[0187] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.

[0188] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0189] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A high-precision binary image segmentation method based on a priori mask, characterized in that: The following steps are involved: S1, obtain the RGB image to be segmented; S2, add a polygon covering the desired object in each RGB image to generate a priori mask; The generation of the priori mask includes: on the basis of the RGB image, forming a polygon by dotting and marking, setting the pixels inside the polygon to "1", and setting the pixels outside the polygon to "0", and outputting the priori mask; S3, inputting the RGB image and the priori mask into a pre-trained binary image segmentation model to obtain a target foreground segmentation result; The binary image segmentation model includes a multi-stage data processing system for generating segmentation results in combination with prior information. The system is composed of multiple data processing modules, each of which performs specific data operations to extract image features and generate segmentation results.

2. The high-precision binary image segmentation method based on a priori mask according to claim 1, characterized in that: The binary image segmentation model includes a priori information fusion stage, a target recognition stage and a result refinement stage; wherein: The prior information fusion stage fuses the prior information with the RGB image to obtain fused image features with prior auxiliary information; The fused image features are input into the scale aggregation decoder of the target recognition stage to obtain a preliminary segmentation prediction result; the preliminary segmentation prediction result is subdivided to obtain a target foreground segmentation result.

3. The high-precision binary image segmentation method based on a priori mask according to claim 2, characterized in that: The prior information fusion stage obtains the fusion image features of the RGB image and the prior information, including the following steps: S201, performing a fusion pooling operation on the RGB image and the priori mask to obtain a feature vector with auxiliary position information; S202: Input the feature vector into a priori information fusion module for updating, and obtain updated first data with priori auxiliary information.

4. The high-precision binary image segmentation method based on a priori mask according to claim 3, characterized in that: The prior information fusion module calculates the first data using the following formula: F pfm =Conv[I rgb ,I m ,V]+I rgb , Among them, GAP represents the global average pooling operation, I rgb is the RGB image, I m is the prior mask, represents element-by-element multiplication, V is the feature vector with auxiliary position information, [I rgb , I m , V] represents I rgb ,I m , V is a channel series operation, Conv represents the convolution layer, F pfm is the first data.

5. The high-precision binary image segmentation method based on a priori mask according to claim 4, characterized in that: The target identification phase includes the following steps: S301, performing image feature compression processing on the first data to obtain second data; S302, inputting the first data and the second data into an encoder to obtain hierarchical features; wherein the first data is input into a ResNet encoder to obtain hierarchical features of the first data, represented as {R i }, subscript i∈{1, 2, 3, 4}; the second data is input into the Swin-Transformer encoder to obtain the hierarchical features of the second data, represented as {S i }, subscript i∈{1, 2, 3, 4}; S303, inputting feature pairs of adjacent levels in the second data into a neighbor aggregation module to obtain adjacent aggregation features; S304: input the first data and the adjacent aggregated features into a scale aggregated decoder module to perform cross-scale aggregated decoding to obtain updated scale aggregated features; S305 , generating a preliminary segmentation prediction result according to the scale aggregation feature; the preliminary segmentation prediction result includes preliminary prediction features.

6. The high-precision binary image segmentation method based on a priori mask according to claim 5, characterized in that: The formula for cross-scale aggregation decoding performed by the scale aggregation decoder in step S304 is as follows: G(S t ,S t-1 )=BConv([BConv(S t ),S t-1 ])+Conv(S t ) f g =G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3))) f d =G(BConv(R1+UP(R2)),UP(f g )) C=P(UP(f d )) Among them, BConv represents a processing module composed of a convolutional layer, a batch normalization layer, and a linear rectification function in sequence; G(·, ·) represents a proximity aggregation operation module, UP represents an upsampling module, and P represents a prediction head; S t Indicates the second data of the current level, S t-1 Indicates the second data of the previous layer; G(S t , S t-1 ) represents the current adjacent aggregation feature; f d represents the scale aggregation feature, and C is the preliminary segmentation prediction result.

7. The high-precision binary image segmentation method based on a priori mask according to claim 6, characterized in that: The result refinement stage also includes refining the scale aggregation features, specifically including: S401, refining the hierarchical features of the first data to obtain refined features of the first data; the refining is to gradually refine the high-level features to the low-level features; S402, refining the refined features of the first data and the preliminary prediction features to obtain refined prediction features; S403: Obtain target foreground segmentation results according to the refined prediction features.

8. The high-precision binary image segmentation method based on a priori mask according to claim 7, characterized in that: The refinement is carried out by the following formula: Among them, i represents the number of layers in the hierarchical feature sequence, the superscript i∈{1, 2, 3}, and Concat represents concatenation in the channel dimension. Represents the different levels of refinement features after gradual refinement; The formula for obtaining the target foreground segmentation result is as follows: in, and is the refined features of the first data of the first layer and the second layer, is the intermediate refined prediction feature, is the final refined prediction feature, and S is the target foreground segmentation result.

9. A high-precision binary image segmentation method based on a priori mask according to any one of claims 1 to 8, characterized in that: The binary image segmentation model is trained using a structural learning loss function, where the structural learning loss function is: in, is the binomial cross entropy loss function, is the intersection-over-union loss function, and G represents the GT truth map.

10. A device for a high-precision binary image segmentation method based on a priori mask, characterized in that: It includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 9.

Images