A method, device, medium and product for layering and segmenting a high-resolution original picture image

Abstract

The application discloses a kind of extremely high resolution original drawing image layering segmentation method, equipment, medium and product, it is related to image segmentation field, the method includes obtaining extremely high resolution to be segmented original drawing image;Using SAM model to the original drawing image to be segmented is preliminarily segmented, and obtains preliminary segmentation result;Multiple initial mask regions are contained in preliminary segmentation result;Using GrabCut algorithm, the preliminary segmentation result is globally optimized, edge optimization and residual hole completion, and obtains the segmentation result after optimization;Using Poisson equation, the edge of the segmentation result after optimization is reconstructed, and obtains final segmentation result.The application can be segmented according to the needs of oneself to different granularity size.In the segmentation process, the application realizes maximum degree of automation using SAM model and GrabCut algorithm, in addition to this, the application optimizes the edge of last segmentation result, greatly improves the efficiency and accuracy of image segmentation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image segmentation, and in particular to a high-resolution original image layering and segmentation method, device, medium and product. BACKGROUND

[0002] In the process of making Live2D art materials, high-resolution original images are the basis, and the original images need to be layered and segmented. Currently, manual segmentation is generally used. However, the original image is often complex and detailed, and manual segmentation is not only time-consuming but also prone to errors. Therefore, it is particularly important to use an advanced highly automated layering and segmentation tool. SUMMARY

[0003] The purpose of the present application is to provide a high-resolution original image layering and segmentation method, device, medium and product to improve the efficiency and accuracy of image segmentation.

[0004] To achieve the above purpose, the present application provides the following solutions.

[0005] In a first aspect, the present application provides a high-resolution original image layering and segmentation method, comprising:

[0006] Obtaining an original image to be segmented; the resolution of the original image to be segmented is 7000x7000;

[0007] Using a SAM model to preliminarily segment the original image to be segmented to obtain a preliminary segmentation result; the preliminary segmentation result contains a plurality of initial mask regions;

[0008] Using a GrabCut algorithm to globally optimize, edge optimize and hole fill the preliminary segmentation result to obtain an optimized segmentation result;

[0009] Using a Poisson equation to reconstruct the edges of the optimized segmentation result to obtain a final segmentation result.

[0010] Optionally, using a SAM model to preliminarily segment the original image to be segmented to obtain a preliminary segmentation result, specifically comprising:

[0011] Inputting the original image to be segmented into the SAM model, adjusting hyperparameters to obtain a preliminary segmentation result; the hyperparameters include points_per_side and stability_score_thresh.

[0012] Optionally, using a GrabCut algorithm to globally optimize, edge optimize and hole fill the preliminary segmentation result to obtain an optimized segmentation result, specifically comprising:

[0013] perform edge optimization on the globally optimized segmentation result by using the GrabCut algorithm to obtain an edge-optimized segmentation result; the edge-optimized segmentation result comprises a plurality of edge-optimized mask regions;

[0014] perform edge optimization on the globally optimized segmentation result by using the GrabCut algorithm to obtain an edge-optimized segmentation result; the edge-optimized segmentation result comprises a plurality of edge-optimized mask regions;

[0015] perform hole completion on the edge-optimized segmentation result by using the GrabCut algorithm to obtain a hole-completed segmentation result;

[0016] perform region selection segmentation and region selection cutting on the hole-completed segmentation result to obtain an optimized segmentation result.

[0017] Optionally, the globally optimized segmentation result is obtained by performing global optimization on the preliminary segmentation result by using the GrabCut algorithm, and specifically comprises:

[0018] an initial label is set for each initial mask region; the initial label is a negative number;

[0019] an optimized mask region is obtained by performing global optimization on the corresponding initial mask region according to the initial label; the optimized mask region comprises one connected domain or a plurality of connected domains;

[0020] the optimized mask region comprising one connected domain is saved as the globally optimized segmentation result;

[0021] for the optimized mask region comprising a plurality of connected domains:

[0022] the current optimized mask region is divided according to the connected domains, and for any connected domain, it is determined whether the area of the current connected domain is greater than 100 pixels to obtain a first determination result;

[0023] if the first determination result is yes, the current connected domain is taken as an optimized mask region and saved as the globally optimized segmentation result;

[0024] if the first determination result is no, a merging operation is performed on the current connected domain to obtain a merged connected domain, and the merged connected domain is taken as an optimized mask region and saved as the globally optimized segmentation result.

[0025] Optionally, the merging operation is performed on the current connected domain to obtain a merged connected domain, and specifically comprises:

[0026] an inflation operation is performed on the current connected domain to obtain an inflated connected domain;

[0027] If there is no mask region in the dilated connected domain, the current connected domain is merged into the mask region closest to the current connected domain to obtain a merged connected domain;

[0028] If there is only one mask region in the dilated connected domain, the current connected domain is merged into the corresponding mask region to obtain a merged connected domain;

[0029] If there are multiple mask regions in the dilated connected domain, the current connected domain is merged into the mask region with the most pixels in the dilated connected domain to obtain a merged connected domain.

[0030] Optionally, an edge optimization is performed on the globally optimized segmentation result by using a GrabCut algorithm to obtain an edge-optimized segmentation result, and the edge optimization specifically includes:

[0031] An erosion operation is performed on the optimized mask region in the globally optimized segmentation result to obtain a first eroded mask region;

[0032] An erosion operation is performed on the first eroded mask region to obtain a second eroded mask region;

[0033] An inflation operation is performed on the optimized mask region in the globally optimized segmentation result to obtain a first inflated mask region;

[0034] An inflation operation is performed on the first inflated mask region to obtain a second inflated mask region;

[0035] Pixels inside the second eroded mask region and outside the second inflated mask region are removed;

[0036] Regions outside the first eroded mask region and inside the first inflated mask region are optimized to obtain an edge-optimized mask region, which is saved as an edge-optimized segmentation result.

[0037] Optionally, a hole filling is performed on the edge-optimized segmentation result by using the GrabCut algorithm to obtain a hole-filled segmentation result, and the hole filling specifically includes:

[0038] According to the edge-optimized segmentation result, the foreground region of the to-be-segmented original drawing image is divided into a region covered only by one edge-optimized mask region, a region covered by multiple edge-optimized mask regions, and a region not covered by an edge-optimized mask region;

[0039] For the region covered by multiple edge-optimized mask regions and the region not covered by an edge-optimized mask region:

[0040] The current region is divided into multiple connected domains;

[0041] If the area of the connected domain is less than 50 pixels, the connected domain is dilated, and the connected domain is merged into the edge-optimized mask region with the largest number of pixels in the corresponding dilated region;

[0042] If the area of the connected domain is greater than or equal to 50 pixels, the connected domain is dilated, and the pixel label of the dilated region is set, the labels of different mask regions in the dilated region are set to 1-N, and the label of the connected domain in the dilated region is set to 0; N is the total number of mask regions in the dilated region;

[0043] The connected domain in the dilated region with the set label is optimized to obtain an optimized connected domain, and is merged into the corresponding edge-optimized mask region;

[0044] The region based on the transparency channel inspection is obtained by checking the region in the foreground region of the original image to be segmented that is not covered by the edge-optimized mask region;

[0045] The connected domain in the region based on the transparency channel inspection is dilated, and the connected domain is merged into the edge-optimized mask region with the largest number of pixels in the corresponding dilated region, so as to complete the hole filling of the edge-optimized segmentation result, and obtain a hole-filled segmentation result.

[0046] In a second aspect, the present application provides a computer device, comprising: a memory, a processor to store a computer program on the memory and executable on the processor, and the processor executes the computer program to implement the high-resolution original image layered segmentation method in any one of the above.

[0047] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the high-resolution original image layered segmentation method in any one of the above.

[0048] In a fourth aspect, the present application provides a computer program product comprising a computer program, which is executed by a processor to implement the high-resolution original image layered segmentation method in any one of the above.

[0049] According to the specific embodiments provided by the present application, the present application has the following technical effects:

[0050] The application provides a high-resolution original drawing image layered segmentation method, device, medium and product, acquires a high-resolution original drawing image to be segmented; a SAM model is used to preliminarily segment the original drawing image to be segmented, to obtain a preliminary segmentation result; the preliminary segmentation result contains a plurality of initial mask regions; a GrabCut algorithm is used to globally optimize, edge optimize and hole fill the preliminary segmentation result, to obtain an optimized segmentation result; a Poisson equation is used to reconstruct the edges of the optimized segmentation result, to obtain a final segmentation result. The application can segment the image to different granularity sizes according to the needs. In the segmentation process, the application realizes a great degree of automation by using the SAM model and the GrabCut algorithm, in addition, the application optimizes the edges of the final segmentation result, greatly improving the efficiency and accuracy of image segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 A flowchart of a high-resolution original drawing image layered segmentation method provided by an embodiment of the present application is shown in the figure.

[0053] Figure 2 The figure is a result schematic diagram of a preliminary automatic segmentation by a SAM model.

[0054] Figure 3 The figure is a result schematic diagram of optimization of the segmentation result of the SAM model in the global by using a GrabCut algorithm.

[0055] Figure 4 The figure is a result schematic diagram of further optimization of the edges of each segmentation result by using a GrabCut algorithm.

[0056] Figure 5 The figure is a result schematic diagram of hole filling according to the transparency channel by using a GrabCut algorithm.

[0057] Figure 6 The figure is a first result schematic diagram of "selected area segmentation" in the controllability of human beings by using a GrabCut algorithm.

[0058] Figure 7 The figure is a second result schematic diagram of "selected area segmentation" in the controllability of human beings by using a GrabCut algorithm.

[0059] Figure 8is a third result schematic diagram of "selected area segmentation" introduced by using the GrabCut algorithm in human controllability;

[0060] Figure 9 is a first result schematic diagram of "selected area segmentation" introduced by using the GrabCut algorithm in human controllability;

[0061] Figure 10 is a second result schematic diagram of "selected area segmentation" introduced by using the GrabCut algorithm in human controllability;

[0062] Figure 11 is a third result schematic diagram of "selected area segmentation" introduced by using the GrabCut algorithm in human controllability;

[0063] Figure 12 is a schematic diagram of an image to be poisson reconstructed;

[0064] Figure 13 is Figure 12 a schematic diagram of a region to be reconstructed eroded away;

[0065] Figure 14 is Figure 12 a schematic diagram of an internal region remaining as a boundary condition after erosion;

[0066] Figure 15 is Figure 12 a result schematic diagram after poisson reconstruction;

[0067] Figure 16 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0069] The above purposes, features and advantages of the present application can be more obvious and easy to understand. The present application will be further described in detail below with reference to the drawings and specific embodiments.

[0070] The application relates to a method for layering and segmenting a high-resolution original drawing image with a resolution of about 7000*7000. The method can accurately layer and segment the original drawing image, automatically identify different elements such as characters, backgrounds, clothes and the like, thereby quickly extracting the required materials, and meanwhile, the method retains human controllability. The method not only improves work efficiency, but also guarantees the accuracy of the segmentation result, and provides a reliable basis for subsequent animation production. Through the tool, artists can focus on creativity and design, improve the quality and expressiveness of the whole work, and finally realize more vivid Live2D performance.

[0071] In an exemplary embodiment, as shown in Figure 1 A high-resolution original drawing image layering and segmenting method is provided, comprising the following steps:

[0072] S1: obtaining an original drawing image to be segmented; the resolution of the original drawing image to be segmented is 7000*7000.

[0073] S2: using a SAM (Segment Anything Model) model to preliminarily segment the original drawing image to be segmented, to obtain a preliminary segmentation result; the preliminary segmentation result contains a plurality of initial mask regions.

[0074] In actual application, the original drawing image to be segmented (the original drawing image to be segmented) is input into the SAM model, and the hyperparameters of the SAM are adjusted to obtain the automatic segmentation result of the SAM model (i.e., the preliminary segmentation result), as shown in Figure 2

[0075] As an optional implementation, S2 specifically comprises:

[0076] The original drawing image to be segmented is input into the SAM model, and the hyperparameters are adjusted to obtain the preliminary segmentation result; the hyperparameters include points_per_side and stability_score_thresh.

[0077] In actual application, the input original drawing image to be segmented is preliminarily automatically segmented by using the SAM model opened source by Facebook. By deploying the SAM model to the local, placing the original drawing image to be segmented into an input file path, and calling the model initialization and model inference functions realized in the open source code, the preliminary segmentation result of the SAM model can be obtained. It should be noted that in the calling process, the settings of part of the hyperparameters need to be adjusted, mainly the two hyperparameters of points_per_side and stability_score_thresh, and the settings are suitable for the data to be segmented.

[0078] ​S3: performing global optimization, edge optimization and hole filling on the preliminary segmentation result by using GrabCut algorithm to obtain an optimized segmentation result.

[0079] In practical application, a pipeline for global optimization, edge optimization and hole filling by using GrabCut algorithm is realized by code. Then the segmentation result of the SAM model is input into the pipeline to obtain the automatic segmentation result of the above three steps, as shown in Figures 3-5 .

[0080] The functions of "region selection segmentation" and "region selection cutting" realized by using GrabCut algorithm are realized by code. The staff manually inputs the initial state to be set, and then the algorithm is optimized, as shown in Figures 6-8 the "region selection segmentation" realized by Figures 9-11 the "region selection cutting" realized by.

[0081] S4: reconstructing the edge of the optimized segmentation result by using Poisson equation to obtain a final segmentation result.

[0082] In practical application, the automatic smoothing of the edge of the segmentation result is realized by code. The staff inputs the segmentation result into the module to quickly obtain the segmentation result which can be directly used, as shown in Figures 12-15 .

[0083] As an optional implementation manner, S3 specifically comprises:

[0084] S31: performing global optimization on the preliminary segmentation result by using GrabCut algorithm to obtain a globally optimized segmentation result; the globally optimized segmentation result comprises a plurality of optimized mask regions.

[0085] As an optional implementation manner, S31 specifically comprises:

[0086] An initial label is set for each initial mask region; the initial label is a negative number.

[0087] The corresponding initial mask region is globally optimized according to the initial label to obtain an optimized mask region; the optimized mask region contains one connected domain or a plurality of connected domains.

[0088] The optimized mask region containing one connected domain is saved as the globally optimized segmentation result;

[0089] For the optimized mask region containing a plurality of connected domains:

[0090] The current optimized mask region is divided according to the connected domain, and for any connected domain, it is judged whether the area of the current connected domain is greater than 100 pixels to obtain a first judgment result.

[0091] If the first determination result is yes, the current connected domain is taken as an optimized mask region, and is saved as a global optimized segmentation result.

[0092] If the first determination result is no, a merging operation is performed on the current connected domain to obtain a merged connected domain, and the merged connected domain is taken as an optimized mask region, and is saved as a global optimized segmentation result.

[0093] As an optional implementation, the merging operation is performed on the current connected domain to obtain a merged connected domain, and the merging operation specifically includes:

[0094] An inflation operation is performed on the current connected domain to obtain an inflated connected domain.

[0095] If the inflated connected domain has no mask region, the current connected domain is merged into a mask region closest to the current connected domain to obtain a merged connected domain.

[0096] If the inflated connected domain has only one mask region, the current connected domain is merged into the corresponding mask region to obtain a merged connected domain.

[0097] If the inflated connected domain has a plurality of mask regions, the current connected domain is merged into a mask region with the most pixels in the inflated connected domain to obtain a merged connected domain.

[0098] In actual application, an initial state of the GrabCut algorithm is set, and the GrabCut algorithm is optimized according to the initial state. Specifically, after obtaining the preliminary segmentation result of the SAM model, different initial mask regions in the preliminary segmentation result are set with different initial labels, all the labels are negative numbers, and the order of the absolute values of the labels gradually increases from 1 until the absolute values are equal to the total number of the initial mask regions. The label of the background region is set as 0. Among them, the positive label represents that the final classification label of the pixel is the initial label and no modification is performed; and the 0 and negative labels represent that the final classification label of the pixel can be different from the initial label and the final classification label can be modified in the optimization process. In the final output result, the labels with the same absolute value represent the same class, and 0 represents the background.

[0099] After the optimized mask region is obtained, it is found that some of the optimized mask regions have multiple connected domains, which will affect the subsequent optimization process, so the following operations are performed: for each optimized mask region with multiple connected domains, it is divided according to the connected domains, if the area of the connected domain is greater than 100 pixels, the connected domain is reserved as an optimized mask region; if the area of the connected domain is less than or equal to 100 pixels, the connected domain is merged to be merged into a mask instead of being reserved as a mask alone, and the specific merging operation is as follows: the connected domain is dilated, if there is no mask in the dilated region, the nearest mask to the connected domain is found according to the centroid of the mask, and the connected domain is merged into the mask; if there is only one mask in the dilated region, the connected domain is directly merged into the mask; if there are multiple masks in the dilated region, the mask with the most pixels in the dilated region is found, and the connected domain is merged into the mask.

[0100] S32: performing edge optimization on the globally optimized segmentation result by using a GrabCut algorithm to obtain an edge-optimized segmentation result; the edge-optimized segmentation result includes multiple edge-optimized mask regions.

[0101] As an optional implementation, S32 specifically includes:

[0102] Performing an erosion operation on the optimized mask region in the globally optimized segmentation result to obtain a first eroded mask region.

[0103] Performing an erosion operation on the first eroded mask region to obtain a second eroded mask region.

[0104] Performing a dilation operation on the optimized mask region in the globally optimized segmentation result to obtain a first dilated mask region.

[0105] Performing a dilation operation on the first dilated mask region to obtain a second dilated mask region.

[0106] Removing pixels within the second eroded mask region and outside the second dilated mask region.

[0107] Optimizing a region outside the first eroded mask region and inside the first dilated mask region to obtain an edge-optimized mask region, and saving it as an edge-optimized segmentation result.

[0108] In practical applications, for the globally optimized segmentation result, the edge of each optimized mask region in the result is optimized. Specifically, two erosion operations and two inflation operations are performed on each optimized mask region, wherein the second erosion is further erosion based on the first erosion, and the second inflation is further inflation based on the first inflation. Then, the pixel label of the region between the first erosion and the second erosion is set to 1, the pixel label of the region between the first inflation and the second inflation is set to 2, and the pixel label of the remaining region is set to 0. Through optimization, only the pixel label of the region between the first erosion and the first inflation is updated, so that the purpose of optimizing only the edge of each optimized mask region is achieved, and the other regions of the optimized mask region are not modified. In the optimization process, the region within the second erosion and the region outside the second inflation are removed, and the two annular regions between the second erosion and the first erosion and between the first inflation and the second inflation are used as the boundary conditions for optimization. This is done to reduce the number of pixels as boundary conditions and speed up the optimization.

[0109] S33: performing hole filling on the edge-optimized segmentation result by using a GrabCut algorithm to obtain a hole-filled segmentation result.

[0110] As an optional implementation, S33 specifically includes:

[0111] According to the edge-optimized segmentation result, the foreground region of the to-be-segmented original drawing image is divided into a region covered only by one edge-optimized mask region, a region covered by multiple edge-optimized mask regions, and a region not covered by an edge-optimized mask region.

[0112] For the region covered by multiple edge-optimized mask regions and the region not covered by an edge-optimized mask region:

[0113] The current region is divided into multiple connected domains.

[0114] If the area of the connected domain is less than 50 pixels, the connected domain is subjected to an inflation operation, and the connected domain is merged into the edge-optimized mask region with the most pixels in the corresponding inflation region.

[0115] If the area of the connected domain is greater than or equal to 50 pixels, the connected domain is subjected to an inflation operation, and the pixel label of the inflation region is set, the labels of different mask regions in the inflation region are set to 1-N, and the label of the connected domain in the inflation region is set to 0; N is the total number of mask regions in the inflation region.

[0116] The connected domain in the inflation region after the label setting is optimized to obtain an optimized connected domain, and is merged into the corresponding edge-optimized mask region.

[0117] After the above operations are performed, in order to make a final check for accuracy, the following is performed:

[0118] An area in the foreground region of the original image to be segmented that is not covered by the edge-optimized mask region is checked using the transparency channel to obtain a region based on the transparency channel check.

[0119] The connected domain in the region based on the transparency channel check is dilated, and the connected domain is merged into the edge-optimized mask region with the most pixels in the corresponding dilated region, so that the hole completion of the edge-optimized segmentation result is completed, and a hole-completed segmentation result is obtained.

[0120] In actual applications, after obtaining the edge-optimized segmentation result, it is found that in the foreground region of the original image, some regions are covered by multiple edge-optimized mask regions, and some regions are not covered by the edge-optimized mask region, and these two regions need to be processed. Specifically, first, according to the edge-optimized segmentation result, the regions covered by multiple masks and the regions not covered by the mask are obtained. Next, these regions are divided into different connected domains, and each connected domain is processed separately. The specific processing method is as follows: if the area of the connected domain is less than 50, the connected domain is dilated and the edge-optimized mask region with the most pixels in the corresponding dilated region is found, and then the connected domain is merged into the edge-optimized mask region. If the area of the connected domain is greater than or equal to 50, the connected domain is dilated and the pixel label of the corresponding dilated region is set. Specifically, in the corresponding dilated region, different edge-optimized mask regions are set to different positive labels, starting from 1 to the total number of edge-optimized mask regions in the corresponding dilated region, and the positive number represents that the label of the pixel will not be modified in the optimization process. Then, the pixel label in the connected domain region is set to 0, representing that it can be modified. Next, the GrabCut algorithm is used to optimize the connected domain according to the set initial state, so that it is merged into different masks.

[0121] After different processing according to the area of the connected domain, the area in the foreground region that is not covered by the edge-optimized mask region is checked again according to the transparency channel. For each connected domain in the region, it is dilated and the edge-optimized mask region with the most pixels in the corresponding dilated region is found, and then the connected domain is merged into the edge-optimized mask region. In this way, the holes in all foreground regions are completed.

[0122] S34: Region selection segmentation and region selection cutting are performed on the hole-completed segmentation result to obtain an optimized segmentation result.

[0123] In practical applications, the GrabCut algorithm is used to introduce human controllability to achieve more accurate control, which specifically includes:

[0124] During the segmentation process, different initial states can be set by manually assigning labels to some pixels, so that the GrabCut algorithm can be optimized according to different initial states to obtain different optimization results. Using this idea to introduce human controllability to achieve more accurate control, there are mainly two implementations: "selected area segmentation" and "selected area cutting".

[0125] Selected area segmentation: During the segmentation process, a region that is desired to be segmented separately is manually divided by operating on the segmentation mask. First, set the labels of the pixels in this region to -1, then perform an inflation operation on this region, set the labels in the inflated region to 2, and then run the GrabCut algorithm to obtain the optimization result. Some of the foreground regions in this result may not be divided into the mask or the segmentation result may be too fragmented, so post-processing is needed for the divided region. For the divided region, first find the largest connected domain of pixels with label absolute value of 1 and 2, divide the remaining region into different connected domains, and for each connected domain, perform inflation processing to determine whether the mask with the most pixels in the inflated region is 1 or 2, and then merge the connected domain to the mask with label absolute value of 1 or 2. The final result of selected area segmentation is the region with pixel label absolute value of 1.

[0126] Selected area cutting: During the segmentation process, if a mask is further cut into several small masks, selected area cutting can be used to achieve this. Specifically, first select the mask that needs to be further cut, then set the labels of the pixels in the region near the boundary that needs to be cut to 0, and finally set the labels of the remaining regions in this mask according to the connected domains. Specifically, for each connected domain, set a positive label with the label number starting from 1 until the total number of connected domains in the remaining region of the mask. Through the GrabCut algorithm, the pixel labels at all boundaries of the mask will be optimized to make the segmentation boundary more accurate. Similarly, the last step is to post-process the regions in this mask that are not divided into different connected domains or have too fragmented segmentation results. Specifically, perform inflation processing on these incomplete regions or fragmented regions, extract the connected domain label with the most pixels in the inflated region, and then merge the region to the connected domain. Finally, each connected domain is a new smaller mask, and all pixels inside have the same absolute value of the label.

[0127] In practical applications, S4 uses the Poisson equation to reconstruct the edge of the segmentation result to enhance the usability of the segmentation result, which specifically includes:

[0128] For the segmentation result processed by all the above processes (the optimized segmentation result), the edge sometimes has a color overflow phenomenon, and the edge of the optimized segmentation result needs to be reconstructed to make the color consistent with the color of the nearby internal region and the transition between different colors smooth. The specific method is: for each optimized segmentation result, first perform erosion processing, and use the internal region after erosion as the boundary condition to reconstruct the pixels of the eroded region. The reconstruction method is to establish a Poisson equation of RGB color value. For internal region pixels, the color value is the color of the initial original drawing input to the pipeline; for the pixels of the eroded region, the first order gradient of the color value of the surrounding four-domain pixels is set to a constant, and the second order gradient is set to 0, that is, four times the color value of the pixel equals the sum of the color values of the four pixels in the four domains. If there are elements in the four domains that should not be considered, such as exceeding the boundary of the current mask, the corresponding number needs to be subtracted. For example, if one pixel in the four domains exceeds the boundary of the current mask, only three pixels in the four domains need to be considered, and the equation established is modified to three times the color value of the pixel equals the sum of the color values of the three pixels in the four domains that need to be considered. By performing the above operation on each mask, a mask with a smoothed edge can be obtained, improving the usability of the mask.

[0129] The present application is directed to the problem that the traditional manual segmentation process of an original drawing image with an ultra-high resolution of about 7000*7000 is too slow, and proposes a segmentation tool with high automation, which greatly speeds up the segmentation process while retaining the accuracy of manual interactive segmentation. By using this method, the staff can quickly segment the image into different granularity sizes according to their own needs. In addition, this method optimizes the edge of the final segmentation result, greatly improving the usability of the segmentation result.

[0130] In an exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-mentioned ultra-high resolution original drawing image layered segmentation method.

[0131] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, which is executed by a processor to implement the above-mentioned ultra-high resolution original drawing image layered segmentation method.

[0132] In an exemplary embodiment, a computer program product is provided, including a computer program, which is executed by a processor to implement the above-mentioned ultra-high resolution original drawing image layered segmentation method.

[0133] In an exemplary embodiment, a computer device, which can be a server or a terminal, has an internal structure as shown in Figure 16 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement an extremely high resolution original picture image layering segmentation method.

[0134] Those skilled in the art can understand that Figure 16 The structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or less components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the relevant data need to comply with relevant regulations.

[0136] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0137] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0138] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for hierarchical segmentation of an extremely high resolution original image, characterized in that, The method comprises the following steps: acquiring a to-be-segmented original drawing image; the resolution of the to-be-segmented original drawing image is 7000*7000; performing preliminary segmentation on the to-be-segmented original drawing image by using a SAM model to obtain a preliminary segmentation result; the preliminary segmentation result contains a plurality of initial mask regions; performing global optimization, edge optimization and hole filling on the preliminary segmentation result by using a GrabCut algorithm to obtain an optimized segmentation result, specifically comprising: performing global optimization on the preliminary segmentation result by using the GrabCut algorithm to obtain a globally optimized segmentation result, specifically comprising: setting an initial label for each initial mask region; the initial label is a negative number; performing global optimization on the corresponding initial mask region according to the initial label to obtain an optimized mask region; the optimized mask region contains one connected domain or a plurality of connected domains; saving the optimized mask region containing one connected domain as a globally optimized segmentation result; the globally optimized segmentation result comprises a plurality of optimized mask regions; performing edge optimization on the globally optimized segmentation result by using the GrabCut algorithm to obtain an edge-optimized segmentation result; the edge-optimized segmentation result comprises a plurality of edge-optimized mask regions; performing hole filling on the edge-optimized segmentation result by using the GrabCut algorithm to obtain a hole-filled segmentation result, specifically comprising: according to the edge-optimized segmentation result, dividing the foreground region of the to-be-segmented original drawing image into a region covered only by one edge-optimized mask region, a region covered by a plurality of edge-optimized mask regions and a region not covered by an edge-optimized mask region; for the region covered by a plurality of edge-optimized mask regions and the region not covered by an edge-optimized mask region: dividing the current region into a plurality of connected domains; if the area of a connected domain is less than 50 pixels, performing an inflation operation on the connected domain, and merging the connected domain into the edge-optimized mask region with the most pixels in the corresponding inflation region; if the area of a connected domain is greater than or equal to 50 pixels, performing an inflation operation on the connected domain, and setting the pixel label of the inflation region; setting the labels of different mask regions in the inflation region to 1-N, and setting the label of the connected domain in the inflation region to 0; N is the total number of mask regions in the inflation region; optimizing the connected domain in the inflation region after setting the label to obtain an optimized connected domain, and merging it into the corresponding edge-optimized mask region; checking the region not covered by the edge-optimized mask region in the foreground region of the to-be-segmented original drawing image by using a transparency channel to obtain a region based on the transparency channel check; performing an inflation operation on the connected domain in the region based on the transparency channel check, and merging the connected domain into the edge-optimized mask region with the most pixels in the corresponding inflation region, thereby completing the hole filling on the edge-optimized segmentation result to obtain a hole-filled segmentation result; performing region selection segmentation and region selection cutting on the hole-filled segmentation result to obtain an optimized segmentation result; The edges of the optimized segmentation result are reconstructed by using a Poisson equation to obtain a final segmentation result.

2. The extreme high resolution draft image hierarchical segmentation method of claim 1, wherein, The SAM model is used for preliminary segmentation of the original image to be segmented to obtain a preliminary segmentation result, specifically including: The original image to be segmented is input into the SAM model, and the hyperparameters are adjusted to obtain a preliminary segmentation result; the hyperparameters include points_per_side and stability_score_thresh.

3. The extreme high resolution draft image hierarchical segmentation method of claim 1, wherein, The GrabCut algorithm is used for global optimization of the preliminary segmentation result to obtain a globally optimized segmentation result, specifically including: For the optimized mask region containing multiple connected domains: The current optimized mask region is divided according to the connected domains, and for any connected domain, it is determined whether the area of the current connected domain is greater than 100 pixels to obtain a first determination result; If the first determination result is yes, the current connected domain is taken as an optimized mask region and saved as a globally optimized segmentation result; If the first determination result is no, a merging operation is performed on the current connected domain to obtain a merged connected domain, and the merged connected domain is taken as an optimized mask region and saved as a globally optimized segmentation result.

4. The extremely high resolution draft image layering segmentation method of claim 3, wherein, The merging operation on the current connected domain to obtain a merged connected domain specifically includes: An inflation operation is performed on the current connected domain to obtain an inflated connected domain; If there is no mask region in the inflated connected domain, the current connected domain is merged into the mask region closest to the current connected domain to obtain a merged connected domain; If there is only one mask region in the inflated connected domain, the current connected domain is merged into the corresponding mask region to obtain a merged connected domain; If there are multiple mask regions in the inflated connected domain, the current connected domain is merged into the mask region with the most pixels in the inflated connected domain to obtain a merged connected domain.

5. The extreme high resolution draft image hierarchical segmentation method of claim 1, wherein, The GrabCut algorithm is used for edge optimization of the globally optimized segmentation result to obtain an edge-optimized segmentation result, specifically including: An erosion operation is performed on the optimized mask region in the globally optimized segmentation result to obtain a first eroded mask region; An erosion operation is performed on the first eroded mask region to obtain a second eroded mask region; An inflation operation is performed on the optimized mask region in the globally optimized segmentation result to obtain a first inflated mask region; An inflation operation is performed on the first inflated mask region to obtain a second inflated mask region; Pixels inside the second eroded mask region and outside the second inflated mask region are removed; Regions outside the first eroded mask region and inside the first inflated mask region are optimized to obtain an edge-optimized mask region, which is saved as an edge-optimized segmentation result.

6. A computer device comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that the processor executes the computer program to implement the high-resolution original image layering segmentation method of any one of claims 1-5.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the method of hierarchical segmentation of a very high resolution draft image according to any one of claims 1 to 5.

8. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method of hierarchical segmentation of a very high resolution draft image according to any one of claims 1 to 5.

Images