Ultra-high-resolution original picture image layered segmentation method and device, medium and product

Abstract

The invention discloses a hierarchical segmentation method and device for an original image with extremely high resolution, a medium and a product, and relates to the field of image segmentation, and the method comprises the steps: obtaining a to-be-segmented original image with extremely high resolution; performing preliminary segmentation on the to-be-segmented original picture image by using the SAM model to obtain a preliminary segmentation result; the initial segmentation result contains a plurality of initial mask regions; performing global optimization, edge optimization and residual hole completion on the preliminary segmentation result by using a GrabCut algorithm to obtain an optimized segmentation result; and reconstructing the edge of the optimized segmentation result by using a Poisson equation to obtain a final segmentation result. According to the method, the image can be segmented into different particle sizes according to own requirements. In the segmentation process, automation is achieved to a great extent through the SAM model and the GrabCut algorithm, in addition, the final segmentation result edge is optimized, and the efficiency and accuracy of image segmentation are greatly improved.

Description

Technical Field

[0001] The present application relates to the field of image segmentation, and in particular to a method, device, medium and product for layered segmentation of extremely high-resolution original images. Background Art

[0002] In the process of making Live2D art materials, extremely high-resolution original images are the basis, and the original images need to be segmented into layers. Currently, manual segmentation is generally used. However, original images are often complex and rich in details. Manual segmentation is not only time-consuming but also prone to errors. Therefore, it is particularly important to use an advanced and highly automated layered segmentation tool. Summary of the invention

[0003] The purpose of this application is to provide a method, device, medium and product for hierarchical segmentation of extremely high-resolution original images to improve the efficiency and accuracy of image segmentation.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a method for hierarchical segmentation of an extremely high-resolution original image, comprising:

[0006] Obtaining an original image to be segmented; the resolution of the original image to be segmented is 7000×7000;

[0007] Performing preliminary segmentation on the original image to be segmented using the SAM model to obtain a preliminary segmentation result; the preliminary segmentation result contains a plurality of initial mask areas;

[0008] The GrabCut algorithm is used to perform global optimization, edge optimization, and residual hole filling on the preliminary segmentation result to obtain an optimized segmentation result;

[0009] The edges of the optimized segmentation result are reconstructed using the Poisson equation to obtain a final segmentation result.

[0010] Optionally, the SAM model is used to perform preliminary segmentation on the original image to be segmented to obtain a preliminary segmentation result, specifically including:

[0011] The original image to be segmented is input into the SAM model, and the hyperparameters are adjusted to obtain a preliminary segmentation result; the hyperparameters include points_per_side and stability_score_thresh.

[0012] Optionally, the GrabCut algorithm is used to perform global optimization, edge optimization, and residual hole filling on the preliminary segmentation result to obtain an optimized segmentation result, specifically including:

[0013] Using the GrabCut algorithm to globally optimize the preliminary segmentation result to obtain a globally optimized segmentation result; the globally optimized segmentation result includes a plurality of optimized mask regions;

[0014] Performing edge optimization on the globally optimized segmentation result using a GrabCut algorithm to obtain an edge-optimized segmentation result; the edge-optimized segmentation result includes a plurality of edge-optimized mask regions;

[0015] Using the GrabCut algorithm to complete the residual holes in the segmentation result after edge optimization, to obtain a segmentation result after the residual holes are completed;

[0016] The segmentation result after the residual hole filling is subjected to selected area segmentation and selected area cutting to obtain an optimized segmentation result.

[0017] Optionally, the preliminary segmentation result is globally optimized using a GrabCut algorithm to obtain a globally optimized segmentation result, specifically including:

[0018] Setting an initial label for each of the initial mask regions; the initial label is a negative number;

[0019] Performing global optimization on the corresponding initial mask region according to the initial label to obtain an optimized mask region; the optimized mask region includes one connected domain or multiple connected domains;

[0020] The optimized mask region containing a connected domain is saved as the globally optimized segmentation result;

[0021] For the optimized mask region containing multiple connected domains:

[0022] Divide the currently optimized mask area according to connected domains, and for any connected domain, determine whether the area of ​​the current connected domain is greater than 100 pixels to obtain a first determination result;

[0023] If the first judgment result is yes, the current connected domain is used as an optimized mask area and saved as a globally optimized segmentation result;

[0024] If the first judgment result is no, a merging operation is performed on the current connected domain to obtain a merged connected domain, and the merged connected domain is used as an optimized mask region and saved as a globally optimized segmentation result.

[0025] Optionally, a merging operation is performed on the current connected domain to obtain a merged connected domain, specifically including:

[0026] Perform expansion operation on the current connected domain to obtain the expanded connected domain;

[0027] If there is no mask region in the expanded connected domain, the current connected domain is merged into the mask region that is closest to the current connected domain to obtain a merged connected domain;

[0028] If there is only one mask region in the expanded connected domain, the current connected domain is merged into the corresponding mask region to obtain a merged connected domain;

[0029] If there are multiple mask regions in the expanded connected domain, the current connected domain is merged into the mask region with the largest number of pixels in the expanded connected domain to obtain a merged connected domain.

[0030] Optionally, edge optimization is performed on the globally optimized segmentation result using a GrabCut algorithm to obtain an edge-optimized segmentation result, specifically including:

[0031] Performing an erosion operation on the optimized mask region in the globally optimized segmentation result to obtain a first erosion mask region;

[0032] Performing an etching operation on the first etching mask region to obtain a second etching mask region;

[0033] Performing an expansion operation on the optimized mask region in the globally optimized segmentation result to obtain a first expanded mask region;

[0034] Performing an expansion operation on the first expansion mask region to obtain a second expansion mask region;

[0035] removing pixels within the second erosion mask region and outside the second dilation mask region;

[0036] The area outside the first corrosion mask area and within the first expansion mask area is optimized to obtain an edge-optimized mask area, and is saved as an edge-optimized segmentation result.

[0037] Optionally, using the GrabCut algorithm to fill in the residual holes on the segmentation result after edge optimization to obtain the segmentation result after the residual holes are filled, specifically including:

[0038] According to the edge-optimized segmentation result, the foreground area of ​​the original image to be segmented is divided into an area covered by only one edge-optimized mask area, an area covered by multiple edge-optimized mask areas, and an area not covered by the edge-optimized mask area;

[0039] For areas covered by multiple edge-optimized mask areas and areas not covered by edge-optimized mask areas:

[0040] Divide the current area into multiple connected domains;

[0041] If the area of ​​the connected domain is less than 50 pixels, the connected domain is expanded and merged into the mask area after edge optimization with the largest number of pixels in the corresponding expansion area;

[0042] If the area of ​​the connected domain is greater than or equal to 50 pixels, the connected domain is expanded and the pixel labels of the expanded area are set. The labels of different mask areas in the expanded area are set to 1-N, and the labels of the connected domains in the expanded area are set to 0; N is the total number of mask areas in the expanded area.

[0043] Optimize the connected domain in the dilated area after setting the label, obtain the optimized connected domain, and merge it into the corresponding mask area after edge optimization;

[0044] Using the transparency channel to check the area in the foreground area of ​​the original image to be segmented that is not covered by the mask area after edge optimization, to obtain an area checked based on the transparency channel;

[0045] The connected domain in the area checked based on the transparency channel is expanded, and the connected domain is merged into the mask area after edge optimization with the largest number of pixels in the corresponding expanded area, so as to complete the residual hole filling of the segmentation result after the edge optimization and obtain the segmentation result after residual hole filling.

[0046] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the extremely high-resolution original image hierarchical segmentation methods described above.

[0047] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for hierarchical segmentation of extremely high-resolution original images.

[0048] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned extremely high-resolution original image hierarchical segmentation methods.

[0049] According to the specific embodiments provided in this application, this application has the following technical effects:

[0050] The present application provides a method, device, medium and product for hierarchical segmentation of original images with extremely high resolution, to obtain the original images to be segmented with extremely high resolution; to use the SAM model to perform preliminary segmentation on the original images to be segmented, to obtain preliminary segmentation results; the preliminary segmentation results contain multiple initial mask areas; to use the GrabCut algorithm to perform global optimization, edge optimization and residual hole filling on the preliminary segmentation results, to obtain optimized segmentation results; to use the Poisson equation to reconstruct the edges of the optimized segmentation results, to obtain the final segmentation results. The present application can segment images into different granularity sizes according to its own needs. During the segmentation process, the present application uses the SAM model and the GrabCut algorithm to achieve a great degree of automation. In addition, the present application optimizes the edges of the final segmentation results, greatly improving the efficiency and accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A schematic diagram of a process for hierarchical segmentation of an extremely high-resolution original image provided by an embodiment of the present application;

[0053] Figure 2 This is a schematic diagram of the results of the initial automatic segmentation by the SAM model;

[0054] Figure 3 This is a schematic diagram of the results of using the GrabCut algorithm to globally optimize the segmentation results of the SAM model;

[0055] Figure 4 This is a schematic diagram of the result of using the GrabCut algorithm to further optimize the edge of each segmentation result;

[0056] Figure 5 This is a schematic diagram of the result of using the GrabCut algorithm to fill the residual hole according to the transparency channel;

[0057] Figure 6 This is a schematic diagram of the first result of "selection segmentation" using the GrabCut algorithm to introduce human controllability;

[0058] Figure 7 This is a schematic diagram of the second result of using the GrabCut algorithm to introduce "selection segmentation" in human controllability;

[0059] Figure 8This is a schematic diagram of the third result of using the GrabCut algorithm to introduce "selection segmentation" in human controllability;

[0060] Fig. 9 This is a schematic diagram of the first result of using the GrabCut algorithm to introduce "selection cutting" in human controllability;

[0061] Fig.10 This is a schematic diagram of the second result of using the GrabCut algorithm to introduce "selection cutting" in human controllability;

[0062] Fig.11 This is a schematic diagram of the third result of using the GrabCut algorithm to introduce "selection cutting" in human controllability;

[0063] Fig.12 is a schematic diagram of an image to be reconstructed by Poisson;

[0064] Fig.13 yes Fig.12 Schematic diagram of the eroded area that needs to be rebuilt;

[0065] Fig.14 yes Fig.12 Schematic diagram of the internal area remaining as boundary conditions after corrosion;

[0066] Fig.15 yes Fig.12 Schematic diagram of the result after Poisson reconstruction;

[0067] Fig.16 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0069] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0070] This application relates to a method for layering and segmenting an original image with an extremely high resolution of about 7000*7000. It can accurately segment the original image into layers, automatically identify different elements such as characters, backgrounds, clothing, etc., so as to quickly extract the required materials while retaining human controllability. This not only improves work efficiency, but also ensures the accuracy of the segmentation results, providing a reliable foundation for subsequent animation production. With such a tool, artists can focus on creativity and design, improve the quality and expressiveness of the overall work, and ultimately achieve a more vivid Live2D performance.

[0071] In an exemplary embodiment, Figure 1 As shown, a method for hierarchical segmentation of an extremely high-resolution original image is provided, comprising the following steps:

[0072] S1: Acquire an original image to be segmented; the resolution of the original image to be segmented is 7000×7000.

[0073] S2: Perform preliminary segmentation on the original image to be segmented using the SAM (Segment Anything Model) model to obtain a preliminary segmentation result; the preliminary segmentation result contains multiple initial mask areas.

[0074] In practical applications, the original image to be segmented (the original image to be segmented) is input into the SAM model, and the hyperparameters of SAM are adjusted to obtain the automatic segmentation result of the SAM model (i.e., the preliminary segmentation result), such as Figure 2 shown.

[0075] As an optional implementation, S2 specifically includes:

[0076] The original image to be segmented is input into the SAM model, and the hyperparameters are adjusted to obtain a preliminary segmentation result; the hyperparameters include points_per_side and stability_score_thresh.

[0077] In practical applications, the SAM model, which is open sourced by Facebook, is used to perform preliminary automatic segmentation on the input original image to be segmented. By deploying the SAM model locally, placing the original image to be segmented in the input file path, and calling the model initialization and model inference functions implemented in the open source code, the preliminary segmentation results of the SAM model can be obtained. It should be noted that during the calling process, some hyperparameter settings need to be adjusted, mainly the two hyperparameters points_per_side and stability_score_thresh, which can be set to suit the data to be segmented.

[0078] S3: Using the GrabCut algorithm, the preliminary segmentation result is globally optimized, edge optimized, and residual hole filled to obtain an optimized segmentation result.

[0079] In practical applications, the GrabCut algorithm is used to implement a pipeline for global optimization, edge optimization, and hole filling. Then the results of the SAM model segmentation are input into the pipeline to obtain the results of the above three steps of automatic segmentation, such as Figure 3-Figure 5 shown.

[0080] The GrabCut algorithm is used to implement the "selection segmentation" and "selection cutting" functions through code. The staff manually inputs the initial state to be set, and then optimizes it through the algorithm, such as Figure 6-Figure 8 The "district segmentation" and Figure 9-11 Implemented "selection cutting".

[0081] S4: Reconstructing the edge of the optimized segmentation result using the Poisson equation to obtain a final segmentation result.

[0082] In practical applications, the code is used to automatically smooth the edges of the segmentation results. The staff inputs the segmentation results into this module to quickly obtain segmentation results that can be directly used, such as Figure 12-Figure 15 shown.

[0083] As an optional implementation, S3 specifically includes:

[0084] S31: globally optimizing the preliminary segmentation result using the GrabCut algorithm to obtain a globally optimized segmentation result; the globally optimized segmentation result includes a plurality of optimized mask regions.

[0085] As an optional implementation, S31 specifically includes:

[0086] An initial label is set for each of the initial mask regions; the initial label is a negative number.

[0087] The corresponding initial mask region is globally optimized according to the initial label to obtain an optimized mask region; the optimized mask region includes one connected domain or multiple connected domains.

[0088] The optimized mask region containing a connected domain is saved as the globally optimized segmentation result;

[0089] For the optimized mask region containing multiple connected domains:

[0090] The currently optimized mask area is divided according to connected domains, and for any connected domain, it is determined whether the area of ​​the current connected domain is greater than 100 pixels to obtain a first determination result.

[0091] If the first judgment result is yes, the current connected domain is used as an optimized mask area and saved as a globally optimized segmentation result.

[0092] If the first judgment result is no, a merging operation is performed on the current connected domain to obtain a merged connected domain, and the merged connected domain is used as an optimized mask region and saved as a globally optimized segmentation result.

[0093] As an optional implementation mode, a merging operation is performed on the current connected domain to obtain a merged connected domain, specifically including:

[0094] Perform an expansion operation on the current connected domain to obtain the expanded connected domain.

[0095] If there is no mask region in the expanded connected domain, the current connected domain is merged into the mask region that is closest to the current connected domain to obtain a merged connected domain.

[0096] If there is only one mask region in the expanded connected domain, the current connected domain is merged into the corresponding mask region to obtain a merged connected domain.

[0097] If there are multiple mask regions in the expanded connected domain, the current connected domain is merged into the mask region with the largest number of pixels in the expanded connected domain to obtain a merged connected domain.

[0098] In practical applications, the initial state of the GrabCut algorithm is set so that it can be optimized according to the initial state. Specifically, after obtaining the preliminary segmentation results of the SAM model, different initial labels are set for different initial mask areas in the preliminary segmentation results. All labels are negative numbers, and the order of the absolute values ​​of the labels is gradually increased from 1 until it equals the total number of initial mask areas. The label of the background area is set to 0. Among them, the positive label represents that the final classification label of the pixel is the initial label and is not modified; 0 and negative labels represent that the final classification label of the pixel may be different from the initial label and may be modified during the optimization process. In the final output result, labels with the same absolute value represent the same category, and 0 represents the background.

[0099] After obtaining the optimized mask area, it will be found that some of the optimized mask areas have multiple connected domains. This situation will affect the subsequent optimization process, so the following operations are performed here: for each optimized mask area with multiple connected domains, separate it according to the connected domain. If the area of ​​the connected domain is greater than 100 pixels, the connected domain is retained as an optimized mask area; if the area of ​​the connected domain is less than or equal to 100 pixels, the connected domain is merged so that it is merged into a mask instead of being retained as a mask separately. The specific merging operation is as follows: dilate the connected domain. If there is no mask in the dilated area, find the mask closest to the connected domain according to the centroid of the mask and merge the connected domain into the mask; if there is only one mask in the dilated area, directly merge the connected domain into the mask; if there are multiple masks in the dilated area, find the mask with the largest number of pixels in the dilated area and merge the connected domain into the mask.

[0100] S32: performing edge optimization on the globally optimized segmentation result using the GrabCut algorithm to obtain an edge-optimized segmentation result; the edge-optimized segmentation result includes a plurality of edge-optimized mask regions.

[0101] As an optional implementation, S32 specifically includes:

[0102] An erosion operation is performed on the optimized mask region in the globally optimized segmentation result to obtain a first erosion mask region.

[0103] An etching operation is performed on the first etching mask region to obtain a second etching mask region.

[0104] An expansion operation is performed on the optimized mask region in the globally optimized segmentation result to obtain a first expanded mask region.

[0105] The first expansion mask region is expanded to obtain a second expansion mask region.

[0106] Pixels within the second erosion mask area and outside the second dilation mask area are removed.

[0107] The area outside the first corrosion mask area and within the first expansion mask area is optimized to obtain an edge-optimized mask area, and is saved as an edge-optimized segmentation result.

[0108] In practical applications, for the segmentation results after global optimization, each optimized mask area in the result is edge optimized. Specifically, two erosion operations and two dilation operations are performed on each optimized mask area, where erosion 2 is further erosion based on erosion 1, and dilation 2 is further dilation based on dilation 1. Then, the pixel label of the area between erosion 2 and erosion 1 is set to 1, the pixel label of the area between dilation 1 and dilation 2 is set to 2, and the pixel labels of all remaining areas are set to 0. Through optimization, only the pixel labels of the area between erosion 1 and dilation 1 are updated, so that the purpose of only optimizing the edges of each optimized mask area is achieved, while other areas of the optimized mask area are not modified. In the optimization process, the areas within the second erosion and outside the second dilation are removed, and the two annular areas from the second erosion to the first erosion and from the first dilation to the second dilation are used as the boundary conditions for optimization. This is done to reduce the number of pixels used as boundary conditions and speed up the optimization.

[0109] S33: Using the GrabCut algorithm to complete the residual holes in the segmentation result after edge optimization, to obtain a segmentation result after the residual holes are completed.

[0110] As an optional implementation, S33 specifically includes:

[0111] According to the edge-optimized segmentation result, the foreground area of ​​the original image to be segmented is divided into an area covered by only one edge-optimized mask area, an area covered by multiple edge-optimized mask areas, and an area not covered by the edge-optimized mask area.

[0112] For areas covered by multiple edge-optimized mask areas and areas not covered by edge-optimized mask areas:

[0113] Divide the current region into multiple connected domains.

[0114] If the area of ​​the connected domain is less than 50 pixels, the connected domain is expanded and merged into the edge-optimized mask area with the largest number of pixels in the corresponding expansion area.

[0115] If the area of ​​the connected domain is greater than or equal to 50 pixels, the connected domain is expanded and the pixel labels of the expanded area are set. The labels of different mask areas in the expanded area are set to 1-N, and the label of the connected domain in the expanded area is set to 0; N is the total number of mask areas in the expanded area.

[0116] The connected domain in the dilated area after setting the label is optimized to obtain the optimized connected domain, and merge it into the corresponding mask area after edge optimization.

[0117] After completing the above operations, do a final check for accuracy, as follows:

[0118] The transparency channel is used to check the area in the foreground area of ​​the original image to be segmented that is not covered by the mask area after edge optimization, and the area checked based on the transparency channel is obtained.

[0119] The connected domain in the area checked based on the transparency channel is expanded, and the connected domain is merged into the mask area after edge optimization with the largest number of pixels in the corresponding expanded area, so as to complete the residual hole filling of the segmentation result after the edge optimization and obtain the segmentation result after residual hole filling.

[0120] In practical applications, after obtaining the edge-optimized segmentation results, it is found that in the foreground area of ​​the original image, some areas are covered by multiple edge-optimized mask areas, and some areas are not covered by the edge-optimized mask areas. These two areas need to be processed. Specifically, first, according to the edge-optimized segmentation results, the areas covered by multiple masks and not covered by masks are obtained. Next, these areas are divided into different connected domains, and each connected domain is processed separately. The specific processing method is: if the area of ​​the connected domain is less than 50, it is expanded and the edge-optimized mask area with the largest number of pixels in the corresponding expanded area is found, and then the connected domain is merged into the edge-optimized mask area. If the area of ​​the connected domain is greater than or equal to 50, it is expanded and the pixel label of the corresponding expanded area is set. Specifically, in the corresponding expanded area, different edge-optimized mask areas are set to different positive labels, starting from 1 to the total number of edge-optimized mask areas in the corresponding expanded area. A positive number represents that the label of the pixel will not be modified during the optimization process. Then, the pixel label in the connected domain area is set to 0, which means that it can be modified. Next, the GrabCut algorithm is used to optimize the connected domain according to the set initial state so that it can be merged into different masks.

[0121] After different processing is performed according to the area of ​​the connected domain, the area in the foreground area that is not covered by the edge-optimized mask area is checked again according to the transparency channel. For each connected domain in the area, it is expanded, and the edge-optimized mask area with the largest number of pixels in the corresponding expanded area is found, and then the connected domain is merged into the edge-optimized mask area. In this way, the residual holes in all foreground areas are completed.

[0122] S34: performing selected area segmentation and selected area cutting on the segmentation result after the residual hole filling, to obtain an optimized segmentation result.

[0123] In practical applications, the GrabCut algorithm is used to introduce human controllability to achieve more precise control, including:

[0124] In the segmentation process, different initial states can be set by manually specifying the labels of some pixels, so that the GrabCut algorithm can be optimized according to different initial states, thereby obtaining different optimization results. This idea is used to introduce human controllability to achieve more precise control, mainly in two ways: "selection segmentation" and "selection cutting".

[0125] Selection segmentation: During the segmentation process, the segmentation mask is operated to artificially divide an area that you want to segment separately. First, set the pixel label in the area to -1, then dilate the area, set the label in the expanded area to 2, and then run the GrabCut algorithm to get the optimization result. In this result, some foreground areas will not be divided into the mask or the segmentation results are too fragmented, so the divided areas need to be post-processed. For the divided area, first find the largest connected domain with absolute values ​​of 1 and 2 for the pixel labels in the area, divide the remaining area into different connected domains, dilate each connected domain, and determine whether the mask with the largest number of pixels in the expanded area is 1 or 2, and then merge the connected domain into the mask with an absolute value of 1 or 2. The final result of the selection segmentation is the area with an absolute value of 1 for the pixel label.

[0126] Selection cutting: During the segmentation process, if you want to further cut a mask into several small masks, you can do it through selection cutting. Specifically, first select the mask to be further cut, then set the pixel labels of the area near the dividing line you want to cut to 0, and finally set the labels of all the remaining areas in the mask according to the connected domain. Specifically, set a positive label for each connected domain, and the label sequence starts from 1 to the total number of connected domains in the remaining area of ​​the mask. Through the GrabCut algorithm, the pixel labels at all the dividing lines of the mask will be optimized to make the segmentation boundary more accurate. Similarly, it is necessary to post-process the areas in the mask that are not divided into different connected domains or the areas where the segmentation results are too fragmented. Specifically, these incomplete or fragmented areas are expanded, and the connected domain labels with the largest number of pixels in the expanded area are extracted, and then the area is merged into the connected domain. Finally, each connected domain is a new smaller mask, and the absolute values ​​of all the pixel labels inside it are the same.

[0127] In practical applications, S4 uses the Poisson equation to reconstruct the edges of the segmentation results to enhance the usability of the segmentation results, including:

[0128] For the segmentation results (optimized segmentation results) after all the above processes, there is sometimes overflow on the edges. It is necessary to reconstruct the edges of the optimized segmentation results so that their colors are consistent with the colors of the closer internal areas and the transition between different colors is smooth. The specific method is: for each optimized segmentation result, first perform corrosion processing, and use the corroded internal area as the boundary condition to reconstruct the pixels of the corroded area. The reconstruction method is to establish the Poisson equation of RGB color values. For the pixels in the internal area, their color values ​​are the colors of the initial original picture input to the pipeline; for the pixels in the corroded area, the first-order gradient of their color values ​​and the color values ​​of the surrounding four-domain pixels is set to a constant, and the second-order gradient is set to 0, that is, four times the color value of the pixel is equal to the sum of the color values ​​of the four pixels in the four domains. If there are elements in the four domains that should not be considered, such as exceeding the boundary of the current mask, the corresponding number needs to be subtracted. For example, if there is a pixel in the four domains that exceeds the boundary of the current mask, only three pixels in the four domains need to be considered, then the established equation is corrected to three times the color value of the pixel is equal to the sum of the color values ​​of the three pixels that need to be considered in the four domains. By performing the above operations on each mask, a mask with smoothed edges can be obtained, thereby improving the usability of the mask.

[0129] This application aims to solve the problem that the traditional manual segmentation process of original images with extremely high resolution of about 7000*7000 is too slow. It proposes a highly automated segmentation tool, which greatly speeds up the segmentation process while retaining the accuracy of manual interactive segmentation. By using this method, workers can quickly segment images into different granularity sizes according to their needs. In addition, this method optimizes the edges of the final segmentation result, greatly improving the usability of the segmentation result.

[0130] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the above-mentioned extremely high-resolution original image hierarchical segmentation method is implemented.

[0131] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the above-mentioned extremely high-resolution original image hierarchical segmentation method.

[0132] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned extremely high-resolution original image hierarchical segmentation method when executed by a processor.

[0133] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.16 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for hierarchical segmentation of an extremely high-resolution original image is implemented.

[0134] Those skilled in the art will understand that Fig.16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0136] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0137] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0138] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for hierarchical segmentation of extremely high-resolution original images, characterized in that: include: Obtain the original image to be segmented; The resolution of the original image to be segmented is 7000×7000; Performing preliminary segmentation on the original image to be segmented using the SAM model to obtain a preliminary segmentation result; the preliminary segmentation result contains a plurality of initial mask areas; The GrabCut algorithm is used to perform global optimization, edge optimization, and residual hole filling on the preliminary segmentation result to obtain an optimized segmentation result; The edges of the optimized segmentation result are reconstructed using the Poisson equation to obtain a final segmentation result.

2. The method for segmenting original images with extremely high resolution according to claim 1, characterized in that: The SAM model is used to perform preliminary segmentation on the original image to be segmented to obtain a preliminary segmentation result, which specifically includes: The original image to be segmented is input into the SAM model, and the hyperparameters are adjusted to obtain a preliminary segmentation result; the hyperparameters include points_per_side and stability_score_thresh.

3. The method for segmenting original images with extremely high resolution according to claim 1, characterized in that: The GrabCut algorithm is used to perform global optimization, edge optimization, and residual hole filling on the preliminary segmentation result to obtain an optimized segmentation result, which specifically includes: Using the GrabCut algorithm to globally optimize the preliminary segmentation result to obtain a globally optimized segmentation result; the globally optimized segmentation result includes a plurality of optimized mask regions; Performing edge optimization on the globally optimized segmentation result using a GrabCut algorithm to obtain an edge-optimized segmentation result; the edge-optimized segmentation result includes a plurality of edge-optimized mask regions; Using the GrabCut algorithm to complete the residual holes in the segmentation result after edge optimization, to obtain a segmentation result after the residual holes are completed; The segmentation result after the residual hole filling is subjected to selected area segmentation and selected area cutting to obtain an optimized segmentation result.

4. The method for segmenting original images with extremely high resolution according to claim 3, characterized in that: The GrabCut algorithm is used to globally optimize the preliminary segmentation results to obtain globally optimized segmentation results, which specifically include: Setting an initial label for each of the initial mask regions; the initial label is a negative number; Performing global optimization on the corresponding initial mask region according to the initial label to obtain an optimized mask region; the optimized mask region includes one connected domain or multiple connected domains; The optimized mask region containing a connected domain is saved as the globally optimized segmentation result; For the optimized mask region containing multiple connected domains: Divide the currently optimized mask area according to connected domains, and for any connected domain, determine whether the area of ​​the current connected domain is greater than 100 pixels to obtain a first determination result; If the first judgment result is yes, the current connected domain is used as an optimized mask area and saved as a globally optimized segmentation result; If the first judgment result is no, a merging operation is performed on the current connected domain to obtain a merged connected domain, and the merged connected domain is used as an optimized mask region and saved as a globally optimized segmentation result.

5. The method for segmenting original images with extremely high resolution according to claim 4, characterized in that: The current connected domain is merged to obtain a merged connected domain, which specifically includes: Perform expansion operation on the current connected domain to obtain the expanded connected domain; If there is no mask region in the expanded connected domain, the current connected domain is merged into the mask region that is closest to the current connected domain to obtain a merged connected domain; If there is only one mask region in the expanded connected domain, the current connected domain is merged into the corresponding mask region to obtain a merged connected domain; If there are multiple mask regions in the expanded connected domain, the current connected domain is merged into the mask region with the largest number of pixels in the expanded connected domain to obtain a merged connected domain.

6. The method for segmenting original images with extremely high resolution according to claim 3, characterized in that: The GrabCut algorithm is used to perform edge optimization on the segmentation result after global optimization to obtain the segmentation result after edge optimization, which specifically includes: Performing an erosion operation on the optimized mask region in the globally optimized segmentation result to obtain a first erosion mask region; Performing an etching operation on the first etching mask region to obtain a second etching mask region; Performing an expansion operation on the optimized mask region in the globally optimized segmentation result to obtain a first expanded mask region; Performing an expansion operation on the first expansion mask region to obtain a second expansion mask region; removing pixels within the second erosion mask region and outside the second dilation mask region; The area outside the first corrosion mask area and within the first expansion mask area is optimized to obtain an edge-optimized mask area, and is saved as an edge-optimized segmentation result.

7. The method for segmenting original images with extremely high resolution according to claim 3, characterized in that: The GrabCut algorithm is used to fill the residual holes in the segmentation result after edge optimization to obtain the segmentation result after residual holes are filled, which specifically includes: According to the edge-optimized segmentation result, the foreground area of ​​the original image to be segmented is divided into an area covered by only one edge-optimized mask area, an area covered by multiple edge-optimized mask areas, and an area not covered by the edge-optimized mask area; For areas covered by multiple edge-optimized mask areas and areas not covered by edge-optimized mask areas: Divide the current area into multiple connected domains; If the area of ​​the connected domain is less than 50 pixels, the connected domain is expanded and merged into the mask area after edge optimization with the largest number of pixels in the corresponding expansion area; If the area of ​​the connected domain is greater than or equal to 50 pixels, the connected domain is expanded and the pixel labels of the expanded area are set. The labels of different mask areas in the expanded area are set to 1-N, and the labels of the connected domains in the expanded area are set to 0; N is the total number of mask areas in the expanded area. Optimize the connected domain in the dilated area after setting the label, obtain the optimized connected domain, and merge it into the corresponding mask area after edge optimization; Using the transparency channel to check the area in the foreground area of ​​the original image to be segmented that is not covered by the mask area after edge optimization, to obtain an area checked based on the transparency channel; The connected domain in the area checked based on the transparency channel is expanded, and the connected domain is merged into the mask area after edge optimization with the largest number of pixels in the corresponding expanded area, so as to complete the residual hole filling of the segmentation result after the edge optimization and obtain the segmentation result after residual hole filling.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the extremely high-resolution original image hierarchical segmentation method described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the extremely high-resolution original image hierarchical segmentation method described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the extremely high-resolution original image hierarchical segmentation method described in any one of claims 1 to 7 is implemented.

Images