A high-precision binary image segmentation method and device based on a prior mask

Abstract

The present application relates to the field of image processing and computer vision, and particularly relates to a high-precision binary image segmentation method based on prior mask. The method comprises the following steps: first, obtaining an RGB image to be segmented; then, a user forms a polygon covering a target object by dotting annotation, and generates a prior mask, wherein pixels inside the polygon are set to "1", and pixels outside the polygon are set to "0"; finally, inputting the RGB image and the prior mask into a pre-trained binary image segmentation model to obtain a target foreground segmentation result. The binary image segmentation model comprises a prior information fusion stage, a target recognition stage and a result refinement stage, which can ensure the accuracy of segmentation and perform excellently in detail processing and complex scenarios. The present application also provides a high-precision binary image segmentation device based on prior mask, which is used to execute the above method to realize high-precision image segmentation.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a high-precision binary image segmentation method and apparatus based on prior masks. Background Technology

[0002] High-precision binary image segmentation (DIS) refers to the technique of accurately detecting and segmenting foreground objects in high-resolution images. It is widely used in fields such as image editing, 3D reconstruction, augmented reality, medical image processing, and robotics. With the development of modern industry and technology, especially in heavy industrial environments, the demand for high-resolution, refined image processing is increasing. This requires segmentation algorithms not only to have higher accuracy but also to be able to handle complex application scenarios, such as dynamic environments, complex backgrounds, and multi-object scenes. However, despite the widespread application of deep learning technology driving the development of image segmentation techniques, high-precision binary image segmentation still faces a series of pressing technical challenges.

[0003] First, existing image segmentation methods often suffer from ambiguous target object definitions when dealing with complex backgrounds, leading to insufficient distinction between foreground and background. This is particularly problematic when the background is complex or the foreground object's edge details are intricate, making accurate target object localization difficult. Second, while segmentation algorithms can achieve good results in certain scenarios, they often struggle to preserve high-resolution details of foreground objects simultaneously. In many high-precision applications, the loss of detail results in inaccurate segmentation, consequently impacting the performance and effectiveness of downstream applications.

[0004] Currently, most high-precision image segmentation methods can be divided into two main categories: one is based on convolutional neural networks (CNNs), which refine the segmentation results step by step through a multi-stage architecture. High-level features are used to provide global semantic information, while low-level features help refine the boundaries of target objects. However, although this type of method can improve segmentation accuracy to some extent, the multi-stage processing architecture often leads to slow inference speed, making it difficult to meet the real-time requirements of practical applications. In addition, this method still has shortcomings in handling fine object edges, resulting in blurred boundaries. The other type of method is based on Transformers, which refines feature layers through cross-scale feedback mechanisms to improve the performance of low-resolution segmentation. However, although the Transformer architecture can effectively capture long-distance dependencies, its ability to handle the boundaries of foreground objects in the image is still insufficient, especially when dealing with complex high-precision tasks, and the segmentation effect still needs to be improved.

[0005] Existing technologies suffer from inconsistent model interpretation of the same task, meaning different methods handle the same segmentation task differently, leading to inconsistent segmentation results. This instability affects the overall performance and repeatability of segmentation. Furthermore, some methods lack effective handling of high precision and detail preservation, causing significant performance degradation in complex scenes, particularly in scenarios with intricate backgrounds or varied object edge details. Therefore, existing high-precision binary image segmentation methods still have considerable room for improvement in terms of accuracy, detail preservation, edge processing, and inference speed.

[0006] In summary, the main challenges of high-precision binary image segmentation include: inaccurate target object localization in complex backgrounds, difficulty in simultaneously preserving high-resolution details, the inference speed of existing methods failing to meet real-time requirements, and inconsistent model results for the same task. These problems significantly limit the versatility and reliability of existing technologies in practical applications, necessitating a technical solution that can simultaneously improve segmentation accuracy, preserve details, and accelerate inference speed to better address the challenges of high-precision image segmentation. Summary of the Invention

[0007] To address the problems of inaccurate target object localization and poor detail segmentation in existing technologies, this invention provides a high-precision binary image segmentation method and apparatus based on prior masks. By combining user-annotated prior masks with high-resolution RGB images and employing a multi-stage fusion, decoding, and thinning process, the method achieves accurate localization of foreground objects while preserving high-resolution details.

[0008] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0009] A high-precision binary image segmentation method based on prior masks includes the following steps:

[0010] S1. Obtain the RGB image to be segmented;

[0011] S2. Add a polygon that covers the desired object in each RGB image to generate a priori mask;

[0012] The generation of the prior mask includes: forming a polygon by marking dots on the RGB image, setting the pixels inside the polygon to "1" and the pixels outside the polygon to "0", and outputting the prior mask.

[0013] S3. Input the RGB image and the prior mask into the pre-trained binary image segmentation model to obtain the target foreground segmentation result;

[0014] The binary image segmentation model includes a multi-stage data processing system for generating segmentation results by combining prior information. The system consists of multiple data processing modules, each of which performs specific data operations to extract image features and generate segmentation results.

[0015] Preferably, the binary image segmentation model includes a prior information fusion stage, a target recognition stage, and a result refinement stage; wherein: the prior information fusion stage fuses prior information with the RGB image to obtain fused image features with prior auxiliary information; the fused image features are input into the scale aggregation decoder of the target recognition stage to obtain a preliminary segmentation prediction result; the preliminary segmentation prediction result is further subdivided to obtain the target foreground segmentation result. The combination of the prior information fusion stage and the scale aggregation decoder makes image segmentation more accurate. The introduction of prior information improves the model's sensitivity to the target object's location, resulting in more accurate segmentation, especially for complex backgrounds. The result refinement stage further improves the preservation of detail in the segmentation result.

[0016] Preferably, the prior information fusion stage for obtaining the fused image features of the RGB image and the prior information includes the following steps:

[0017] S201. Perform a fusion pooling operation on the RGB image and the prior mask to obtain a feature vector with auxiliary position information;

[0018] S202. Input the feature vector into the prior information fusion module for updating, and obtain the updated first data with prior auxiliary information.

[0019] In this way, by fusing and pooling the prior mask and the RGB image, user-annotated information can be effectively utilized for foreground object localization. The fused image features contain more spatial location information, enabling the model to handle segmentation tasks in complex scenes more accurately.

[0020] Preferably, the prior information fusion module calculates the first data using the following formula:

[0021]

[0022] F pfm =Conv[I rgb ,I m ,V]+I rgb ,

[0023] Where GAP represents global average pooling operation, I rgb For the RGB image, I m For the aforementioned prior mask, This indicates element-wise multiplication, where V is the feature vector with auxiliary positional information, [I rgb ,I m [,V] represents I rgb I m The concatenation operation of channels V, where Conv represents a convolutional layer, and F... pfm This serves as the primary data source. The image feature fusion calculation method based on a specific formula effectively combines RGB images and prior information, enhancing the model's ability to segment foreground and background details. Global average pooling further optimizes feature aggregation, improving the robustness of the segmentation model.

[0024] Preferably, the target identification stage comprises the following steps:

[0025] S301. Perform image feature compression processing on the first data to obtain the second data;

[0026] S302. Input the first data and the second data into the encoder to obtain hierarchical features; wherein, the first data is input into the ResNet encoder to obtain the hierarchical features of the first data, represented as {R i}, where i ∈ {1, 2, 3, 4}; the second data is input into the Swin-Transformer encoder to obtain the hierarchical features of the second data, represented as {S i}, where the index i ∈ {1, 2, 3, 4};

[0027] S303. Input the feature pairs of adjacent levels in the second data into the neighbor aggregation module to obtain the adjacent aggregation features;

[0028] S304. Input the first data and the adjacent aggregated features into the scale aggregation decoder module to perform cross-scale aggregation decoding and obtain the updated scale aggregation features.

[0029] S305. Generate preliminary segmentation prediction results based on the scale aggregation features; the preliminary segmentation prediction results include preliminary prediction features.

[0030] Through the multi-level feature compression and encoding steps described above, this preferred scheme can effectively capture details and global information in images. After fusing features from high-resolution and low-resolution images, the model performs better in edge processing of foreground objects and suppression of background interference, resulting in more accurate segmentation results.

[0031] Preferably, the formula for cross-scale aggregation decoding by the scale aggregation decoder in step S304 is as follows:

[0032] G(S t ,S t-1 ) = BConv([BConv(St ),S t-1 ])+Conv(S t )

[0033] f g =G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3)))

[0034] f d =G(BConv(R1+UP(R2)),UP(f g ))

[0035] C = P(UP(f) d ))

[0036] Where BConv represents the processing module consisting of a convolutional layer, a batch normalization layer, and a linear rectified function in sequence; G(·,·) represents the nearest neighbor aggregation operation module, UP represents the upsampling module, and P represents the prediction head; S t S represents the second data in the current level. t-1 G(S) represents the second data in the previous layer; t ,S t-1 ) represents the current neighboring aggregation feature; f d Let C represent the scale-aggregated features, and C be the initial segmentation prediction result. In this way, feature fusion through the cross-scale aggregation decoder module allows for full interaction of image features at different levels, improving the model's ability to segment multi-scale targets. The combination of convolutional operations and the upsampling module enhances the model's performance in high-precision image segmentation tasks, especially in handling edge details.

[0037] Preferably, the result refinement stage further includes refining and detailing the scale-aggregated features, specifically including:

[0038] S401. Refine and refine the hierarchical features of the first data to obtain the refined features of the first data; the refinement and refinement is to gradually refine the high-level features to the low-level features.

[0039] S402. Refine and refine the refined features of the first data and the preliminary prediction features to obtain refined prediction features;

[0040] S403. Obtain the target foreground segmentation result based on the refined prediction features.

[0041] This preferred approach further enhances the ability to process details of target objects by refining and detailing high-resolution hierarchical features. The refinement module effectively corrects the initial predicted features, resulting in a more accurate final segmentation result that retains more details.

[0042] Preferably, the refining and elaboration is achieved through the following formula:

[0043]

[0044] Where the superscript i represents the layer number in the hierarchical feature sequence, i∈{1,2,3}, and Concat indicates concatenation along the channel dimension. This represents the different levels of refinement features after progressive refinement.

[0045] The formula for obtaining the target foreground segmentation result is as follows:

[0046]

[0047] in, and These are the refined features of the first data in the first and second layers mentioned above. These are intermediate, refined predictive features. S represents the final refined prediction feature and the target foreground segmentation result.

[0048] By refining and detailing, the details of foreground objects can be further optimized during the segmentation prediction process. This optimized scheme effectively fuses high-level and low-level features, ensuring the accuracy and consistency of segmentation results in complex backgrounds.

[0049] Preferably, the high-precision binary image segmentation model is trained using a structure learning loss function, wherein the structure learning loss function is:

[0050]

[0051] in, The binary cross-entropy loss function is... Let G be the Cross-Union Ratio (CUNR) loss function, and G denote the ground truth graph of the ground truth. By introducing a structured learning loss function, this optimal approach not only optimizes segmentation accuracy during training but also improves the matching degree between foreground and background regions through the CUNR loss function. This loss function design enables the model to exhibit stronger generalization ability in various scenarios.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] This invention provides a high-precision binary image segmentation method based on prior masks, which improves the accuracy of target object localization. By using a user-input polygonal prior mask, the specific location of foreground objects can be clearly defined during image segmentation, thus avoiding ambiguity in the model's definition of the target foreground and background. The user's prior annotations provide clear location information, solving the problem of unclear target foreground object localization in high-precision binary image segmentation. This allows for better utilization of the model's performance, enabling it to focus more on regions containing foreground target objects, resulting in more accurate segmentation results. It reduces instances of unclear target object localization, especially in complex backgrounds.

[0054] Because the prior mask introduces annotations with location information, the model can focus more intently on regions containing the target object, reducing incorrect identification of irrelevant areas. This method not only accurately segments the target object but also exhibits higher precision in edge details, thereby improving the overall segmentation quality. Users only need to perform simple point annotations to generate a coarse polygon mask, which is simpler and faster than traditional manual annotation methods, greatly reducing the cost and difficulty of manual annotation and increasing user acceptance.

[0055] The high-precision binary image segmentation method based on prior masks provided in this invention does not rely on a single segmentation strategy. Instead, it integrates user prior information, enabling the model to flexibly adapt to different scenarios and task requirements. Regardless of the complexity of the image segmentation task or the diversity of target objects, this framework can achieve good segmentation results. Attached Figure Description

[0056] Figure 1 This is a flowchart of a high-precision binary image segmentation method based on a priori mask according to Embodiment 1 of the present invention;

[0057] Figure 2 This is a schematic diagram of the network structure of a high-precision binary image segmentation method based on prior masks according to Embodiment 2 of the present invention;

[0058] Figure 3 This is a schematic diagram of the user-annotated prior mask in Embodiment 2 of the present invention;

[0059] Figure 4 This is a comparison diagram of the results of a high-precision binary image segmentation method based on prior mask and existing methods in Embodiment 3 of the present invention. Detailed Implementation

[0060] The high-precision binary image segmentation method based on a priori masks provided by the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. However, this should not be construed as limiting the scope of the above-mentioned subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention. The advantages and features of the present invention will become clearer in conjunction with the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the purpose of illustrating the embodiments of the present invention.

[0061] Example 1

[0062] This embodiment provides a high-precision binary image segmentation method based on prior masks. The method steps are as follows: Figure 1 As shown. This method aims to improve the accuracy of binary image segmentation by introducing prior masks, especially in foreground object localization and detail processing in complex backgrounds. By combining user-annotated prior information with the model's deep learning capabilities, the overall accuracy and application efficiency of the segmentation results are improved. Specifically, it includes the following steps:

[0063] S1. Obtain the RGB image to be segmented;

[0064] S2. Add a polygon that covers the desired object in each RGB image to generate a priori mask;

[0065] The generation of the prior mask includes: forming a polygon by marking dots on the RGB image, setting the pixels inside the polygon to "1" and the pixels outside the polygon to "0", and outputting the prior mask.

[0066] S3. Input the RGB image and the prior mask into the pre-trained binary image segmentation model to obtain the target foreground segmentation result;

[0067] The binary image segmentation model includes a multi-stage data processing system for generating segmentation results by combining prior information. The system consists of multiple data processing modules, each of which performs specific data operations to extract image features and generate segmentation results.

[0068] Preferably, the binary image segmentation model includes a prior information fusion stage, a target recognition stage, and a result refinement stage; wherein: the prior information fusion stage fuses prior information with the RGB image to obtain fused image features with prior auxiliary information; the fused image features are input into the scale aggregation decoder of the target recognition stage to obtain a preliminary segmentation prediction result; the preliminary segmentation prediction result is further subdivided to obtain the target foreground segmentation result. The combination of the prior information fusion stage and the scale aggregation decoder makes image segmentation more accurate. The introduction of prior information improves the model's sensitivity to the target object's location, resulting in more accurate segmentation, especially for complex backgrounds. The result refinement stage further improves the preservation of detail in the segmentation result.

[0069] Preferably, the prior information fusion stage for obtaining the fused image features of the RGB image and the prior information includes the following steps:

[0070] S201. Perform a fusion pooling operation on the RGB image and the prior mask to obtain a feature vector with auxiliary position information;

[0071] S202. Input the feature vector into the prior information fusion module for updating, and obtain the updated first data with prior auxiliary information.

[0072] In this way, by fusing and pooling the prior mask and the RGB image, user-annotated information can be effectively utilized for foreground object localization. The fused image features contain more spatial location information, enabling the model to handle segmentation tasks in complex scenes more accurately.

[0073] Preferably, the prior information fusion module calculates the first data using the following formula:

[0074]

[0075] F pfm =Conv[I rgb ,I m ,V]+I rgb ,

[0076] Where GAP represents global average pooling operation, I rgb For the RGB image, I m For the aforementioned prior mask, This indicates element-wise multiplication, where V is the feature vector with auxiliary positional information, [I rgb ,I m [,V] represents I rgb I m The concatenation operation of channels V, where Conv represents a convolutional layer, and F... pfmThis serves as the primary data source. The image feature fusion calculation method based on a specific formula effectively combines RGB images and prior information, enhancing the model's ability to segment foreground and background details. Global average pooling further optimizes feature aggregation, improving the robustness of the segmentation model.

[0077] Preferably, the target identification stage comprises the following steps:

[0078] S301. Perform image feature compression processing on the first data to obtain the second data;

[0079] S302. Input the first data and the second data into the encoder to obtain hierarchical features; wherein, the first data is input into the ResNet encoder to obtain the hierarchical features of the first data, represented as {R i}, where i ∈ {1, 2, 3, 4}; the second data is input into the Swin-Transformer encoder to obtain the hierarchical features of the second data, represented as {S i}, where the index i ∈ {1, 2, 3, 4};

[0080] S303. Input the feature pairs of adjacent levels in the second data into the neighbor aggregation module to obtain the adjacent aggregation features;

[0081] S304. Input the first data and the adjacent aggregated features into the scale aggregation decoder module to perform cross-scale aggregation decoding and obtain the updated scale aggregation features.

[0082] S305. Generate preliminary segmentation prediction results based on the scale aggregation features; the preliminary segmentation prediction results include preliminary prediction features.

[0083] Through the multi-level feature compression and encoding steps described above, this preferred scheme can effectively capture details and global information in images. After fusing features from high-resolution and low-resolution images, the model performs better in edge processing of foreground objects and suppression of background interference, resulting in more accurate segmentation results.

[0084] Preferably, the formula for cross-scale aggregation decoding by the scale aggregation decoder in step S304 is as follows:

[0085] G(S t ,S t-1 ) = BConv([BConv(S t ),S t-1 ])+Conv(S t )

[0086] f g=G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3)))

[0087] f d =G(BConv(R1+UP(R2)),UP(f g ))

[0088] C = P(UP(f) d ))

[0089] Where BConv represents the processing module consisting of a convolutional layer, a batch normalization layer, and a linear rectified function in sequence; G(·,·) represents the nearest neighbor aggregation operation module, UP represents the upsampling module, and P represents the prediction head; S t S represents the second data in the current level. t-1 G(S) represents the second data in the previous layer; t ,S t-1 ) represents the current neighboring aggregation feature; f d Let C represent the scale-aggregated features, and C be the initial segmentation prediction result. In this way, feature fusion through the cross-scale aggregation decoder module allows for full interaction of image features at different levels, improving the model's ability to segment multi-scale targets. The combination of convolutional operations and the upsampling module enhances the model's performance in high-precision image segmentation tasks, especially in handling edge details.

[0090] Preferably, the result refinement stage further includes refining and detailing the scale-aggregated features, specifically including:

[0091] S401. Refine and refine the hierarchical features of the first data to obtain the refined features of the first data; the refinement and refinement is to gradually refine the high-level features to the low-level features.

[0092] S402. Refine and refine the refined features of the first data and the preliminary prediction features to obtain refined prediction features;

[0093] S403. Obtain the target foreground segmentation result based on the refined prediction features.

[0094] This preferred approach further enhances the ability to process details of target objects by refining and detailing high-resolution hierarchical features. The refinement module effectively corrects the initial predicted features, resulting in a more accurate final segmentation result that retains more details.

[0095] Preferably, the refining and elaboration is achieved through the following formula:

[0096]

[0097] Where the superscript i represents the layer number in the hierarchical feature sequence, i∈{1,2,3}, and Concat indicates concatenation along the channel dimension. This represents the different levels of refinement features after progressive refinement.

[0098] The formula for obtaining the target foreground segmentation result is as follows:

[0099]

[0100] in, and These are the refined features of the first data in the first and second layers mentioned above. These are intermediate, refined predictive features. S represents the final refined prediction feature and the target foreground segmentation result.

[0101] By refining and detailing, the details of foreground objects can be further optimized during the segmentation prediction process. This optimized scheme effectively fuses high-level and low-level features, ensuring the accuracy and consistency of segmentation results in complex backgrounds.

[0102] Preferably, the high-precision binary image segmentation model is trained using a structure learning loss function, wherein the structure learning loss function is:

[0103]

[0104] in, The binary cross-entropy loss function is... Let G be the Cross-Union Ratio (CUNR) loss function, and G denote the ground truth graph of the ground truth. By introducing a structured learning loss function, this optimal approach not only optimizes segmentation accuracy during training but also improves the matching degree between foreground and background regions through the CUNR loss function. This loss function design enables the model to exhibit stronger generalization ability in various scenarios.

[0105] Example 2

[0106] This second embodiment, as a preferred embodiment of the first embodiment, adds a melt pooling operation to the prior information fusion stage and refines the target recognition stage through multiple steps, providing a specific implementation of a high-precision binary image segmentation method based on prior masks. Specifically:

[0107] S1. Obtain the high-resolution RGB image to be segmented;

[0108] S2. The user adds a polygon that covers the desired object in each image, and generates a priori mask based on the polygon with the user's prior information.

[0109] The process of generating the user's prior mask involves the user marking simple dots on the image to represent the foreground target object. Connecting these dots creates a polygon that covers the desired object. Pixels inside the polygon are explicitly set to "1", and pixels outside the polygon are explicitly set to "0", thus obtaining the user's prior mask. Marking the polygon is very convenient, requiring only a few clicks, and therefore has high user acceptance.

[0110] S3. Input the RGB image and prior mask into the pre-trained high-precision binary image segmentation model to obtain the high-precision target foreground segmentation result.

[0111] The high-precision binary image segmentation model includes a prior information fusion stage, a target recognition stage, and a result refinement stage. The prior information fusion stage fuses prior information with the RGB image, then inputs it into the scale aggregation decoder of the target recognition stage, and finally refines the result to obtain an accurate foreground segmentation result.

[0112] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, the prior information fusion stage achieves the fusion of prior information and image through the following steps:

[0113] S201. Perform a fusion pooling operation on the image and the prior mask to obtain a feature vector V with auxiliary position information;

[0114] S202. The feature vector is input into the prior information fusion module for updating, to obtain the updated fused image features with prior auxiliary information.

[0115] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, in step 202, the RGB image and the prior mask are fused using the following formula:

[0116]

[0117] F pfm =Conv[I rgb ,I m ,V]+I rgb ,

[0118] Among them, I rgb Represents an RGB image; I m This represents the prior mask, where pixels inside the polygon in the prior mask are 1 and pixels outside the polygon are 0. F represents element-wise multiplication, GAP represents global average pooling, [,] represents channel concatenation, Conv represents a convolutional layer, V is a feature vector with auxiliary positional information, and F pfm To fuse image features.

[0119] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, the target recognition stage captures and identifies coarse foreground target objects through the following steps:

[0120] S301, the scale of the high-resolution fused image features is compressed to obtain low-resolution fused image features.

[0121] S302. Input two fused image features into the encoder to obtain a series of hierarchical feature pairs;

[0122] S303, Take the adjacent hierarchical feature pairs (S) from the low-resolution feature sequence pair. t ,S t-1 The input is fed into the neighbor aggregation module to obtain neighbor aggregation features;

[0123] S303. Take the high-resolution feature sequence pairs and the obtained adjacent aggregated features, and input them into the scale aggregation decoder module for decoding to obtain preliminary predicted features;

[0124] S304. Generate a preliminary segmentation prediction map based on the decoded preliminary prediction features.

[0125] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, in step 301, the high-resolution fused features are input into the ResNet encoder to obtain a series of hierarchical feature pairs {R i}(i=1,2,3,4), the low-resolution fused features are input into the Swin-Transformer encoder to obtain a series of hierarchical feature pairs {S i (i = 1, 2, 3, 4).

[0126] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, the neighbor aggregation module uses the following formula to aggregate adjacent hierarchical features:

[0127] G(S t ,S t-1 ) = BConv([BConv(S t ),S t-1 ])+Conv(S t )

[0128] Here, BConv represents a processing module consisting of a convolutional layer, a batch normalization layer, and a linear rectified function in sequence. t-1 S represents the low-resolution fusion feature at the current level. t-1 This represents the fusion feature of the previous layer. G(S) t ,S t-1 ) represents the output of the current neighbor aggregation module, i.e., the current neighbor aggregation feature.

[0129] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, the scale aggregation decoder module performs cross-scale aggregation decoding on the received features using the following formula:

[0130] f g =G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3)))

[0131] f d =G(BConv(R1+UP(R2)),UP(f g ))

[0132] C = P(UP(f) d ))

[0133] Where G(·,·) represents the nearest neighbor aggregation module, UP represents the upsampling module, which is used to align features at different levels in the scale dimension, and P represents the prediction head. g This represents intermediate decoding features, while f d C represents the initial predicted features after scale aggregation decoding, and C is the initial segmentation prediction map.

[0134] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, the result refinement stage refines the preliminary segmentation predictions obtained earlier through the following steps:

[0135] S401. Perform cross-level refinement on the above high-resolution series of hierarchical feature pairs, and gradually refine the high-level abstract features to the low level to obtain a series of high-resolution refined feature pairs.

[0136] S402. Input the high-resolution refined feature pair and the above preliminary prediction features into the refinement module to obtain refined prediction features, and generate a high-precision segmentation map based on the refined prediction features.

[0137] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, in step 401, the following formula is used to refine the multiple high-resolution hierarchical features:

[0138]

[0139] Here, the superscript i∈{1, 2, 3} represents the layer number in the hierarchical feature sequence. Concat indicates concatenation along the channel dimension. This indicates the different levels of refinement features after progressive refinement.

[0140] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, the refinement in step 402 is performed by refining the received preliminary prediction features f using the following formula. d use and Refinement and optimization are performed to obtain refined predictive features and generate high-precision segmentation maps:

[0141]

[0142] Where P represents the prediction head and UP represents the upsampling module. and For the high-resolution refinement features of the first and second layers mentioned above, f d These are the initial predicted features after scale aggregation decoding. These are intermediate, refined predictive features. For the final refined prediction features, S generates a high-precision segmentation map for the refined prediction features.

[0143] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, the high-precision binary image segmentation model is trained using a structure learning loss function to obtain the pre-trained high-precision binary image segmentation model.

[0144] Preferably, in the above-mentioned high-precision binary image segmentation method based on prior masks, the joint learning loss function is:

[0145]

[0146] in, The binary cross-entropy loss function is... Let S be the intersection-union loss function, S be the final high-precision segmentation map, and G be the ground truth map of the ground truth.

[0147] Example 3

[0148] As a preferred embodiment of Embodiment 2, this embodiment introduces a pyramid cross-scale decoding module for feature decoding and also uses cross-level refinement processing in the result refinement stage, providing a better implementation of a high-precision binary image segmentation method based on prior masks.

[0149] In this embodiment, the image and prior mask are input into a pre-trained high-precision segmentation model. The final high-precision segmentation result is obtained from the decoder through forward propagation of the network. The high-resolution branch encoder is based on the common ResNet network structure, and the low-resolution branch is based on the common Swin-Transformer network structure. The method of this embodiment mainly includes the following steps:

[0150] S1. Obtain the high-resolution RGB image to be segmented and the user's prior mask. The specific prior mask is generated by processing the polygon enclosed by the points marked by the user in the image, such as... Figure 3 As shown, the inside of the polygon is set to "1" to represent the white foreground area containing the target object, and the outside of the polygon is set to "0" to represent the black background area. Both images are cropped to 1024×1024 size.

[0151] S2. Input both the RGB image and the user's prior mask into the prior information fusion module, and then use the fusion operator to embed the prior position information into the image to obtain the fused image features F. pfm The high-resolution fused image features are then compressed to obtain low-resolution fused image features. These high-resolution and low-resolution features are then input into a ResNet encoder and a Swin-Transformer decoder to obtain a series of feature pairs {R}. i} and {S i}, using high-resolution features for {R i For example, the feature resolutions are: 64x256x256, 128x128x128, 256x64x64, and 512x32x32.

[0152] S3. Take high-resolution feature sequence pairs and low-resolution feature sequences, and input them into the multi-scale aggregation decoder in the target recognition stage to obtain the preliminary predicted features after decoding. Specifically, this embodiment introduces the pyramid cross-scale decoding module from the literature "Xie C, Xia C, Ma M, et al. Pyramid Grafting Network for One-Stage High Resolution Saliency Detection[J].2022" for feature decoding, such as... Figure 2 As shown, the specific steps are as follows:

[0153] S31. Take adjacent hierarchical feature pairs (S) from the low-resolution feature sequence pair. t ,S t-1 The input is fed into the neighbor aggregation module to obtain adjacent aggregated features. The aggregation of adjacent hierarchical features is achieved using the following formula:

[0154] G(S t ,S t-1 ) = BConv([BConv(S t ),S t-1 ])+Conv(S t )

[0155] Here, BConv represents a processing module consisting of a convolutional layer, a batch normalization layer, and a linear rectified function in sequence.t S represents the low-resolution fusion feature at the current level. t-1 This represents the fusion feature of the previous layer. G(S) t ,S t-1 ) represents the output of the current neighbor aggregation module, i.e., the current neighbor aggregation feature.

[0156] S32. Take the high-resolution feature sequence pairs and the obtained adjacent aggregated features, and input them into the scale aggregation decoder module to obtain the preliminary predicted features f after multi-scale aggregation decoding. d .

[0157] Specifically, the scale aggregation decoder module performs coarse prediction decoding on the received features using the following formula:

[0158] f g =G(BConv(G(S2,S1)+UP(G(R4,R3))),UP(G(S4,S3)))

[0159] f d =G(BConv(R1+UP(R2)),UP(f g ))

[0160] C = P(UP(f) d ))

[0161] Where G(·,·) represents the nearest neighbor aggregation operation module, UP represents the upsampling module, which is used to align features at different levels in the scale dimension, and P represents the prediction head. g This represents intermediate decoding features, while f d C represents the scale aggregation feature, and C is the preliminary segmentation prediction map.

[0162] S4, The high-resolution feature sequence {R} i Then perform cross-level refinement processing and combine it with the preliminary predicted features f described in S3. d All inputs are fed into the refinement module of the result refinement stage to obtain the refined prediction features, such as... Figure 3 As shown. The specific steps are as follows:

[0163] S41. Apply the following formula to the high-resolution hierarchical feature sequence {R}. i Perform cross-level refining:

[0164]

[0165] Here, the superscript i∈{1, 2, 3} represents the layer number in the hierarchical feature sequence. Concat indicates concatenation along the channel dimension. This indicates the different levels of refinement features after progressive refinement.

[0166] S42, Preliminary prediction feature f d Refine according to the following formula:

[0167]

[0168] in, and For the high-resolution refinement features of the first and second layers mentioned above, f d This is a scale-aggregated feature. These are intermediate, refined predictive features. This is for the final refined prediction features.

[0169] S5. The updated final refined prediction features described in S4. The input is fed into the prediction head, activated by a sigmoid function, to obtain the final detection result, such as... Figure 2 As shown.

[0170] S6. Before using the network, the high-precision binary image segmentation model is trained, the model training loss function is calculated, and error backpropagation and gradient descent optimization are performed to optimize the network parameters in the prior information fusion stage, the target recognition stage, and the result refinement stage.

[0171] Specifically, the overall training loss function of the model consists of two parts: the binomial cross-entropy loss function and the cross-union ratio (CUI) loss function, calculated using the following formula:

[0172]

[0173] in, The binary cross-entropy loss function is... Let S be the intersection-union loss function, S be the final high-precision segmentation map, and G be the ground truth map (GT).

[0174] In a further embodiment of the present invention, we conducted comparative experiments with three existing high-precision binary image segmentation methods (ISNet, RMFormer, and HitNet), such as... Figure 4As shown in the figure. In experiments, the segmentation method of this invention demonstrated significant advantages in various scenarios. First, in terms of preserving the edge details of the target, the model (Ours) of this invention can more accurately capture the subtle features of complex-shaped foreground objects, such as the antennae of musical instrument players and insects. In contrast, other models such as ISNet and RMFormer produce segmentation results with blurred edges and are prone to missing some details. Second, in high-contrast scenes (such as the Eiffel Tower), the model of this invention can better distinguish between the foreground and background, and the segmentation contours are clear, while other methods such as HitNet exhibit missegmentation when dealing with background noise. Therefore, the high-precision binary image segmentation model provided by this invention outperforms existing methods in both accuracy and detail preservation, especially in detail processing and foreground segmentation in complex backgrounds, where it demonstrates higher accuracy.

[0175] Example 4

[0176] This embodiment provides a high-precision binary image segmentation device based on prior masks, designed to perform image segmentation tasks to improve the accuracy and efficiency of image segmentation, particularly foreground object segmentation against complex backgrounds. The device includes:

[0177] 1. Processor: At least one processor is responsible for performing various image processing tasks, including image preprocessing, prior mask generation, fusion feature extraction, target recognition, and result refinement. The processor is capable of rapidly processing large-scale image data, ensuring the system's performance in high-precision image segmentation tasks.

[0178] 2. Memory: The device is equipped with a memory that is communicatively connected to the processor. The memory stores instructions executable by the processor, which, based on the high-precision binary image segmentation method using a priori masks described in this invention, guide the processor through the entire process from inputting a high-resolution RGB image to outputting a high-precision segmentation result. The memory can be random access memory (RAM) or read-only memory (ROM) for temporarily storing intermediate calculation results or maintaining pre-trained parameters of the segmentation model.

[0179] 3. Instruction Set: The instructions stored in memory include image processing algorithms used to perform the following steps:

[0180] Obtain the high-resolution RGB image to be segmented;

[0181] Generate a priori mask input by the user, and then fuse the mask with the RGB image;

[0182] The fused image is input into a high-precision binary image segmentation model, which sequentially performs prior information fusion, target recognition, and result refinement.

[0183] Outputs high-precision foreground segmentation results.

[0184] 4. Input / Output Interface: The device also includes an image data input and output interface for receiving high-resolution RGB images and outputting the processing results. This interface can be wired or wireless, adaptable to various image sources and external devices.

[0185] With this design, the device can efficiently execute the high-precision binary image segmentation method of prior mask, which is suitable for a variety of application scenarios, such as autonomous driving, medical image processing and industrial image detection, and has significant advantages in segmentation tasks with complex backgrounds and high precision requirements.

[0186] The above description is merely a detailed illustration of specific embodiments of the present invention and is not intended to limit the invention. Various substitutions, modifications, and improvements made by those skilled in the art without departing from the principles and scope of the present invention should be included within the protection scope of the present invention.

[0187] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0188] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0189] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A high-precision binary image segmentation method based on prior masks, characterized in that, Includes the following steps: S1. Obtain the RGB image to be segmented; S2. Add a polygon that covers the desired object in each RGB image to generate a priori mask; The generation of the prior mask includes: forming a polygon by marking dots on the RGB image, setting the pixels inside the polygon to "1" and the pixels outside the polygon to "0", and outputting the prior mask. S3. Input the RGB image and the prior mask into the pre-trained binary image segmentation model to obtain the target foreground segmentation result; The binary image segmentation model includes a multi-stage data processing system for generating segmentation results by combining prior information. The system consists of multiple data processing modules, each of which performs specific data operations to extract image features and generate segmentation results. The binary image segmentation model includes a prior information fusion stage, a target recognition stage, and a result refinement stage; wherein: The prior information fusion stage fuses prior information with the RGB image to obtain fused image features with prior auxiliary information; The fused image features are input into the scale aggregation decoder in the target recognition stage to obtain preliminary segmentation prediction results; the preliminary segmentation prediction results are then further subdivided to obtain target foreground segmentation results. The prior information fusion stage acquires the fused image features of the RGB image and the prior information, including the following steps: S201. Perform a fusion pooling operation on the RGB image and the prior mask to obtain a feature vector with auxiliary position information; S202. Input the feature vector into the prior information fusion module for updating, and obtain the updated first data with prior auxiliary information; The prior information fusion module calculates the first data using the following formula: Here, GAP represents the global average pooling operation. For the RGB image, For the aforementioned prior mask, This indicates element-wise multiplication. V The feature vector with auxiliary location information, express , , Channel serial operation, Indicates a convolutional layer. This is the first data; The target identification stage involves the following steps: S301. Perform image feature compression processing on the first data to obtain the second data; S302. Input the first data and the second data into the encoder to obtain hierarchical features; wherein, the first data is input into the ResNet encoder to obtain the hierarchical features of the first data, represented as { } , subscript i ∈{1, 2, 3, 4}; The second data is input into the Swin-Transformer encoder to obtain the hierarchical features of the second data, represented as { }, subscript i ∈{1, 2, 3, 4}; S303. Input the feature pairs of adjacent levels in the second data into the neighbor aggregation module to obtain the adjacent aggregation features; S304. Input the first data and the adjacent aggregated features into the scale aggregation decoder module to perform cross-scale aggregation decoding and obtain the updated scale aggregation features. S305. Generate preliminary segmentation prediction results based on the scale aggregation features; the preliminary segmentation prediction results include preliminary prediction features.

2. The high-precision binary image segmentation method based on prior masking according to claim 1, characterized in that, The formula for cross-scale aggregation decoding in step S304 is as follows: in, This represents a processing module consisting of a convolutional layer, a batch normalization layer, and a linear rectified function in sequence; G(·,·) represents the nearest neighbor aggregation operation module, UP represents the upsampling module, and P represents the prediction head; This represents the second data in the current level. This indicates the second data in the previous layer; Indicates the current adjacent aggregation feature; Indicates scale aggregation features. This is the preliminary segmentation prediction result.

3. The high-precision binary image segmentation method based on prior masks according to claim 2, characterized in that, The result refinement stage also includes refining the scale-aggregated features, specifically including: S401. Refine and refine the hierarchical features of the first data to obtain the refined features of the first data; the refinement and refinement is to gradually refine the high-level features to the low-level features. S402. Refine and refine the refined features of the first data and the preliminary prediction features to obtain refined prediction features; S403. Obtain the target foreground segmentation result based on the refined prediction features.

4. The high-precision binary image segmentation method based on prior masks according to claim 3, characterized in that, The refining and elaboration are achieved through the following formula: in, i The superscript indicates the layer number in the hierarchical feature sequence. i ∈{1, 2, 3}, This indicates concatenation along the channel dimension. This represents the different levels of refinement features after progressive refinement. The formula for obtaining the target foreground segmentation result is as follows: in, and These are the refined features of the first data in the first and second layers mentioned above. These are intermediate, refined predictive features. For the final refined prediction features, S The result is the segmentation of the target foreground.

5. A high-precision binary image segmentation method based on prior masks according to any one of claims 1 to 4, characterized in that, The binary image segmentation model is trained using a structure learning loss function, which is: = in, The binary cross-entropy loss function is... The intersection-union loss function is... G This represents the ground truth graph.

6. An apparatus for a high-precision binary image segmentation method based on prior masks, characterized in that, It includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method of any one of claims 1 to 4.

Images