Image segmentation method and device

Abstract

This application provides an image segmentation method, which includes: obtaining a first segmentation mask in a target image. The first segmentation mask is a segmentation mask corresponding to a target region in the target image. Based on the first segmentation mask and the target image, the boundary of the target region is obtained. Based on the boundary and one or more preset vertex calculation methods, one or more contour vertex sets of the target region are obtained. Based on the one or more contour vertex sets, a target contour vertex set is obtained. Based on the target contour vertex set, a corrected target region is obtained. The technical solution of this application embodiment can make the shape contour of the target region more accurate, improve stability and robustness, and enhance the delivery effect of print advertisements.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image segmentation method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] With the rise of video, print advertising has taken on a new form: print ads can be embedded in videos.

[0003] When placing print ads within videos, accurately extracting the target area is crucial for ensuring the ads blend seamlessly with the environment. Target area extraction typically employs semantic segmentation techniques; however, the accuracy of the segmentation mask edges predicted by semantic segmentation is often low, resulting in print ads failing to achieve the desired performance.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0005] This application provides an image segmentation method, apparatus, computer device, and computer-readable storage medium to solve or alleviate one or more of the technical problems mentioned above.

[0006] One aspect of this application provides an image segmentation method, the method comprising:

[0007] Obtain the first segmentation mask in the target image; wherein, the first segmentation mask is the segmentation mask corresponding to the target region in the target image;

[0008] The boundary of the target region is obtained based on the first segmentation mask and the target image;

[0009] Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region;

[0010] Based on the one or more contour vertex sets, obtain the target contour vertex set;

[0011] Based on the set of vertexes of the target contour, obtain the corrected target region.

[0012] Optionally, obtaining the boundary of the target region based on the first segmentation mask and the target image includes:

[0013] Based on the first segmentation mask, obtain the first boundary set of the segmentation mask;

[0014] Based on the first boundary set, obtain the initial contour vertices;

[0015] If the initial contour vertices match the prior information of the target region shape, the first mask is used for boundary calculation.

[0016] Optionally, obtaining the boundary of the target region based on the first segmentation mask and the target image further includes:

[0017] If the initial contour vertices do not match the prior shape information of the target region, perform the following operations:

[0018] Boundary completion is performed based on the first boundary set to update the first segmentation mask;

[0019] The updated first segmentation mask is used for boundary calculation.

[0020] Optionally, obtaining the boundary of the target region based on the first segmentation mask and the target image includes:

[0021] Based on the first segmentation mask, a second segmentation mask is obtained, wherein the second segmentation mask is the segmentation mask of the target region boundary;

[0022] Based on the target image, obtain the first gradient map corresponding to the target image;

[0023] Based on the first gradient map and the second segmentation mask, a second gradient map is obtained, which is the gradient map of the boundary of the target region.

[0024] The boundary of the target region is obtained based on the second gradient map.

[0025] Optionally, obtaining the second segmentation mask based on the first segmentation mask includes:

[0026] According to the dilation and erosion algorithm, the first segmentation mask is dilated and eroded to obtain the dilated segmentation mask and the eroded segmentation mask.

[0027] The second segmentation mask is obtained based on the dilation segmentation mask and the erosion segmentation mask.

[0028] Optionally, obtaining a first gradient map corresponding to the target image based on the target image includes:

[0029] Based on the target image, obtain a first grayscale image corresponding to the target image;

[0030] An image enhancement operation is performed on the first grayscale image to obtain a second grayscale image;

[0031] The first gradient image is obtained based on the second grayscale image.

[0032] Optionally, obtaining the first gradient map based on the second grayscale image includes:

[0033] Obtain the X-direction gradient map of the first gradient map in the X direction;

[0034] Obtain the Y-direction gradient map of the first gradient map;

[0035] The first gradient map is obtained by merging and fusing the gradient map in the X direction and the gradient map in the Y direction.

[0036] Optionally, the one or more preset vertex calculation methods include a line detection algorithm;

[0037] Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region, including:

[0038] The boundary is subjected to line detection using a line detection algorithm to obtain a set of candidate lines;

[0039] The number of edges in the target region is obtained based on prior information about the shape of the target region.

[0040] Based on the number of edges, the candidate lines in the candidate line set are clustered to obtain multiple target line sets; wherein, the number of edges is the same as the number of clusters, and each cluster corresponds to one target line set;

[0041] Based on the multiple sets of target lines, multiple target lines are obtained, with each set of target lines corresponding to one target line;

[0042] Based on the plurality of target straight lines, obtain the first set of contour vertices.

[0043] Optionally, the one or more preset vertex calculation methods include contour extraction algorithms;

[0044] Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region, including:

[0045] Contour extraction is performed on the boundary to obtain the outer boundary corresponding to the boundary;

[0046] Polygon fitting is performed on the outer boundary to obtain the second contour vertex set.

[0047] Optionally, obtaining the target contour vertex set based on the one or more contour vertex sets includes:

[0048] If the second set of contour vertices matches the prior shape information of the target region, perform the following operations:

[0049] Set the weight of the first contour vertex set to the first weight, and set the weight of the second contour vertex set to the second weight;

[0050] The target contour vertex set is obtained by performing a weighted summation on the first contour vertex set and the second contour vertex set.

[0051] Optionally, it also includes:

[0052] If the first set of vertices does not match the prior shape information, perform the following operations:

[0053] The weight of the first contour vertex set is set as the third weight, and the weight of the second contour vertex set is set as the fourth weight, wherein the third weight is greater than the fourth weight;

[0054] The target contour vertex set is obtained by performing a weighted summation on the first contour vertex set and the second contour vertex set.

[0055] Another aspect of this application provides an image segmentation apparatus, the apparatus comprising:

[0056] The first acquisition module is used to acquire a first segmentation mask in the target image; wherein the first segmentation mask is a segmentation mask corresponding to the target region in the target image;

[0057] The second acquisition module is used to acquire the boundary of the target region based on the first segmentation mask and the target image;

[0058] The third acquisition module is used to acquire one or more sets of contour vertices of the target region based on the boundary and one or more preset vertex calculation methods;

[0059] The fourth acquisition module is used to acquire a target contour vertex set based on the one or more contour vertex sets;

[0060] The fifth acquisition module is used to acquire the corrected target region based on the target contour vertex set.

[0061] Another aspect of this application provides a computer device, including:

[0062] At least one processor; and

[0063] A memory that is communicatively connected to the at least one processor;

[0064] Wherein: the memory stores instructions that can be executed by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0065] Another aspect of this application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method described above.

[0066] The embodiments of this application employing the above-described technical solution may have the following advantages:

[0067] The boundary of the first segmentation mask of the target area is extracted. Then, based on the boundary of the target area and the corresponding vertex calculation method, the target contour vertex set is obtained. The shape contour of the target area is then corrected, making the shape contour of the target area more accurate, improving stability and robustness, and enhancing the effectiveness of print advertising. Attached Figure Description

[0068] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0069] Figure 1 This diagram illustrates a flowchart for inserting print ads into a video.

[0070] Figure 2A This schematically illustrates the target area with rough edges of the segmentation mask;

[0071] Figure 2B This schematically illustrates the target area with recessed edges of the segmentation mask;

[0072] Figure 2C This schematically illustrates the target area where the corners of the segmentation mask are missing;

[0073] Figure 3 This diagram schematically illustrates the operating environment of the image segmentation method according to Embodiment 1 of this application;

[0074] Figure 4 A flowchart illustrating an image segmentation method according to Embodiment 1 of this application is shown schematically.

[0075] Figure 5 Schematic illustration Figure 4 Flowchart of the sub-steps in step S402;

[0076] Figure 6 Schematic illustration Figure 4 Flowchart of the sub-steps in step S402;

[0077] Figure 7 Schematic illustration Figure 4 Flowchart of the sub-steps in step S402;

[0078] Figure 8 Schematic illustration Figure 7 Flowchart of the sub-steps in step S700;

[0079] Figure 9 Schematic illustration Figure 7 Flowchart of the sub-steps in step S702;

[0080] Figure 10 Schematic illustration Figure 9 Flowchart of the sub-steps in step S904;

[0081] Figure 11 Schematic illustration Figure 4 Flowchart of the sub-steps in step S404;

[0082] Figure 12 Schematic illustration Figure 4 Flowchart of the sub-steps in step S404;

[0083] Figure 13 Schematic illustration Figure 4 Flowchart of the sub-steps in step S406;

[0084] Figure 14 Schematic illustration Figure 4 Flowchart of the sub-steps in step S406;

[0085] Figure 15A and Figure 15B This illustration schematically shows a system architecture diagram of the image segmentation method according to an embodiment of this application;

[0086] Figure 16 This diagram illustrates the state changes of the target image during a hole-filling operation.

[0087] Figure 17 This illustration shows a set of application examples for obtaining boundaries based on the updated first segmentation mask;

[0088] Figure 18 The illustration shows a set of application examples of using line detection and the K-means algorithm to obtain boundary lines;

[0089] Figure 19 A block diagram of an image segmentation apparatus according to Embodiment 2 of this application is schematically shown; and

[0090] Figure 20 A schematic diagram of the hardware architecture of a computer device according to Embodiment 3 of this application is shown. Detailed Implementation

[0091] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0092] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0093] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0094] First, a definition of the terminology used in this application is provided:

[0095] Semantic segmentation is a deep learning algorithm that associates labels or categories with each pixel of an image. Semantic segmentation algorithms can be used to identify sets of pixels.

[0096] Segmentation mask: The prediction result of a semantic segmentation algorithm, which identifies the set of pixels that constitute the target object in an image. A segmentation mask is a binary image (pixel values ​​are 0 or 1).

[0097] Robustness refers to a system's ability to maintain stability and correctness when faced with uncertainty and unexpected events. Specifically, it refers to the system's capacity to correctly perform its tasks even when input data is erroneous or unexpected.

[0098] Polygon fitting algorithm: An algorithm in image processing that extracts the object enclosed by the smallest polygon in a binary image.

[0099] Perspective transformation: Perspective transformation is a transformation based on the collinearity of three points and the law of perspective rotation, which rotates the projection plane (perspective plane) around the trace line (perspective axis) by a certain angle while keeping the projected geometry unchanged on the projection plane. Essentially, perspective transformation projects an image from one geometric plane to another.

[0100] Contour extraction algorithm: An algorithm in image processing for extracting closed contours from binary images.

[0101] A convex hull is a convex polygon formed by connecting the outermost points, which can contain all the points in the point set.

[0102] Convex hull solving algorithm: This is a polygon fitting algorithm based on the convex hull. The convex hull solving algorithm determines the position and shape of the polygon by solving the convex hull of the point set.

[0103] Dilation and erosion algorithms: These are algorithms based on binary images. Dilation involves merging all background points in contact with an object into that object, causing the boundary to expand outwards. Dilation can be used to fill holes in an object. Erosion, on the other hand, eliminates boundary points, causing the boundary to shrink inwards. Erosion can be used to eliminate small and insignificant objects.

[0104] Image gradient: refers to the rate of change of a pixel in an image in both the X and Y directions (compared to neighboring pixels). The image gradient is a two-dimensional vector.

[0105] Sobel operator: Used in image processing to obtain the first-order gradient of an image. The Sobel operator can be used for edge detection; it is a discrete difference operator used to calculate the grayscale approximation of the image gradient function.

[0106] Edge detection is a detection tool in image processing, computer vision, and machine vision used for feature extraction and detection. It aims to detect edges or discontinuous regions with significant changes in brightness within a digital image. The purpose of edge detection is to identify points in a digital image where brightness variations are obvious.

[0107] Line detection algorithm: An algorithm used in image processing to detect regular straight lines.

[0108] K-means clustering algorithm: A data partitioning algorithm based on Euclidean distance, which can divide given data into several categories.

[0109] Image brightness refers to the size of an image pixel. The larger the pixel value, the brighter the image at that pixel. For grayscale images, each pixel has only one component, ranging from 0 to 255, where 0 represents black (the darkest) and 1 represents white (the brightest).

[0110] Image contrast: refers to the magnitude of the grayscale contrast in an image, that is, the ratio of maximum brightness to minimum brightness.

[0111] Color Image: Each image is composed of three grayscale images, representing the grayscale values ​​of the red, green, and blue channels, respectively, i.e., RGB.

[0112] Binary Image: In an image, the gray value of each pixel can only be 0 or 1. A binary image can be understood as a black and white image.

[0113] Grayscale image: Each pixel in an image can be represented by a grayscale value of 0-255.

[0114] Secondly, to facilitate understanding of the technical solutions provided in the embodiments of this application by those skilled in the art, the relevant technologies are described below:

[0115] With the rise of VR / AR applications and the popularity of video as an information carrier, the placement of print ads has also taken on new forms, namely, embedding AI technology into VR / AR or video content. Taking the placement of print ads within a video as an example, the process can be divided into three steps: "extracting the target area, distorting the ad material, and replacing the content in the target area with the material." Accurately extracting the target area is a crucial prerequisite for ensuring the ad's integration with its environment. Although print ads are typically placed on rectangular objects such as walls, billboards, and displays, based on the principle of perspective transformation, these objects cannot maintain their rectangular shape indefinitely when viewed from different angles and positions (e.g., ...). Figure 1 As shown in the middle image, the white area is not rectangular. Therefore, to effectively extract the target region, semantic segmentation technology is needed to obtain the segmentation mask of the target carrier, followed by image processing techniques such as contour extraction to obtain accurate vertices and boundaries. However, the segmentation mask predicted by semantic segmentation technology has low edge accuracy and its performance in video is unstable, often exhibiting errors such as "dents" and "incompleteness," specifically: ① When the edges of the segmentation mask are relatively rough (e.g., Figure 2A As shown), it is difficult to obtain the precise vertices of the target region. ② When the segmentation mask is recessed in the center of the boundary (as shown), Figure 2B As shown), during the polygon fitting process of the segmentation mask, the target region, which is originally a quadrilateral, will be incorrectly estimated as a concave polygon. ③ When the segmentation mask is incomplete near the vertices (such as... Figure 2C As shown, during the process of polygon fitting of the segmentation mask, the target region will be incorrectly estimated as a convex polygon.

[0116] In summary, the target region extraction schemes known to the inventors rely excessively on the accuracy of the segmentation mask. When the segmentation mask is not accurate enough (such as indentations or missing corners), it is prone to misestimating the shape of the target region, resulting in an inaccurate fitted target region and consequently affecting the effectiveness of print advertising. Therefore, the aforementioned target region extraction schemes are imprecise and lack sufficient stability and robustness.

[0117] Therefore, this application provides an image segmentation technology solution. In this technology solution, the boundary of the first segmentation mask of the target region is extracted, and then the target contour vertex set is obtained according to the boundary of the target region and the corresponding vertex calculation method. Then, the shape contour of the target region is corrected, thereby making the shape contour of the target region more accurate, improving stability and robustness, and improving the placement effect of flat advertising. The advantages of this technology solution are: (1) When the segmentation mask is basically regular (without holes, missing corners, etc.) but the edges are relatively rough, the segmentation mask can be fine-tuned to obtain a more accurate region shape contour. (2) When the segmentation mask has concavity or missing parts, it can be adaptively processed to obtain the corrected region shape contour. (3) It has a very low dependence on the original segmentation mask and has extremely high robustness and stability. See the following text for details.

[0118] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0119] Figure 3 The illustration shows an environmental application diagram of the method according to an embodiment of this application.

[0120] Computer devices 10000 include various types of electronic devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, single servers, or server clusters. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (e.g., Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows, Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices can include cellular phones, smartphones, tablets, etc. Client devices are capable of executing various applications, such as various Internet-related applications, communication applications (e.g., email applications), and short message service (SMS) applications, and can use various communication protocols.

[0121] Computer device 10000 may include input / output interfaces. Input interfaces may include a touchpad, touchscreen, mouse, keyboard, or other sensing elements. Input interfaces can be configured to receive user commands that enable computer device 10000 to perform various operations, such as acquiring a set of boundary / contour vertices of a target region, extracting the target region, etc. Output interfaces are used to output information to the user, such as video or image information.

[0122] Based on the operating environment described above, computer device 10000 may also have one or more applications installed, such as a billboard embedding program. This billboard embedding program is used to embed billboards into videos or images.

[0123] The technical solutions of this application will be described below using computer device 10000 as the implementation subject through multiple embodiments. It should be understood that these embodiments can be implemented in many different forms and should not be construed as being limited to the embodiments described herein.

[0124] Example 1

[0125] Figure 4 A flowchart illustrating an image segmentation method according to Embodiment 1 of this application is shown schematically.

[0126] like Figure 4 As shown, the image segmentation method may include steps S400 to S408, wherein:

[0127] Step S400: Obtain the first segmentation mask in the target image. The first segmentation mask is the segmentation mask corresponding to the target region in the target image.

[0128] Step S402: Obtain the boundary of the target region based on the first segmentation mask and the target image.

[0129] Step S404: Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region.

[0130] Step S406: Obtain the target contour vertex set based on the one or more contour vertex sets.

[0131] Step S408: Obtain the corrected target region based on the target contour vertex set.

[0132] In the technical solution of this application embodiment, after extracting the first segmentation mask of the target region from the target image, the boundary of the first segmentation mask of the target region is extracted. Then, based on the boundary of the target region and the corresponding vertex calculation method, the target contour vertex set is obtained, and the shape contour of the target region is corrected, thereby making the shape contour of the target region more accurate, improving stability and robustness, and improving the placement effect of print advertisements. Specifically, the technical solution of this application embodiment can correct the shape contour of the target region according to the contour vertices of the target region, so that even when the accuracy of the first segmentation mask is poor, the shape contour of the target region can still be accurately obtained, giving the technical solution of this application sufficient stability and robustness.

[0133] The following combination Figure 4 The steps in steps S400 to S408, as well as other optional steps, are described in detail.

[0134] Step S400 The first segmentation mask in the target image is obtained. Wherein, the first segmentation mask M... ori This is the segmentation mask for the target region in the target image.

[0135] The target image can be one or more video frame images from the video.

[0136] The target area can be a rectangular object such as a wall, billboard, or display screen in the video, or it can be other polygonal objects; there is no limitation on this.

[0137] The first segmentation mask is the set of pixels constituting the target region. In some embodiments, semantic segmentation algorithms (such as FCNN (fully connected convolutional neural network) or DeepLab v3 (a semantic segmentation network with dilated convolutions)) can be used to obtain the segmentation mask of the target region.

[0138] Step S402 The boundary of the target region is obtained based on the first segmentation mask and the target image.

[0139] The boundary of the target area can be obtained in a variety of ways.

[0140] For example, the boundary of the target region can be obtained using methods such as the region boundary tracking method and the target neighborhood point boundary tracking method. Taking the region boundary tracking method as an example, the specific steps are as follows: (1) Determine the starting search point of the boundary. (2) Determine appropriate boundary discrimination criteria and search criteria. The discrimination criteria are used to determine whether a point is a boundary point, and the search criteria are used to determine how to search for the next edge point. (3) Starting from the starting search point of the target region, and then searching for the next boundary point according to the corresponding criteria, until the search point coincides with the initial point.

[0141] However, the first segmentation mask may have concavities or missing corners, and the boundary obtained directly based on the above method may be inaccurate. Specifically: (1) When the first segmentation mask has concavities at the boundary, during the polygon fitting process of the first segmentation mask, the target area of ​​the polygon will be incorrectly estimated as a concave polygon, resulting in an error in the initial contour vertices, i.e., it does not match the actual target area; (2) When the first segmentation mask has missing corners at the boundary, during the polygon fitting process of the first segmentation mask, the target area of ​​the polygon will be incorrectly estimated as a convex polygon, resulting in an error in the initial contour vertices, i.e., it does not match the actual target area. In other words, concavities and missing corners in the first segmentation mask will cause a large deviation in the obtained boundary, such as incorrectly fitting a target area that is originally a quadrilateral as a convex polygon. To alleviate the above defects, the following optional embodiments are provided.

[0142] In optional embodiments, such as Figure 5 As shown, step S402 may include:

[0143] Step S500: Obtain the first boundary set of the segmentation mask based on the first segmentation mask.

[0144] Step S502: Obtain the initial contour vertices based on the first boundary set.

[0145] Step S504: If the initial contour vertices match the prior information of the target region shape, the first mask is used for boundary calculation.

[0146] The prior information about the target region shape can be the number of vertices in the target region. In some embodiments, the decision to update the first segmentation mask can be made by determining whether the number of initial contour vertices matches (is the same as) the prior information about the target region shape. If the number of initial contour vertices matches the prior information about the target region shape, it indicates that the first segmentation mask does not have the aforementioned defects, and therefore the first segmentation mask can be directly used for boundary calculation. This decision process can ensure that the data used for boundary calculation is valid as much as possible, improve the accuracy of boundary calculation, and thus effectively ensure the accuracy of target region segmentation.

[0147] In some embodiments, the first boundary set can be obtained in different ways. For example, the first boundary set can be obtained by performing boundary extraction operations on the first segmentation mask using a contour extraction algorithm. The specific process of the contour extraction algorithm is as follows: (1) If the target pixel in the target region is black and its 8 adjacent pixels are also black, then the target pixel is deleted (set to white); (2) Traverse each pixel of the boundary to obtain a new target region. In other embodiments, it can also be obtained by other algorithms, such as the region boundary tracking method.

[0148] In some embodiments, a polygon fitting algorithm can be used to extract vertices from the first boundary set to obtain the initial contour vertices of the target region.

[0149] In this embodiment, the integrity of the first segmentation mask is quickly determined by judging whether it matches the prior information about the target region's shape. Specifically, it's determined whether the first segmentation mask might have holes or missing corners. When the first segmentation mask matches the prior information about the target region's shape, it indicates that the first segmentation mask likely does not have holes or missing corners. Therefore, the first mask can be directly used for boundary calculation without boundary completion. This judgment process ensures that the data used for boundary calculation is valid, improving the accuracy of boundary calculation and thus effectively guaranteeing the accuracy of target region segmentation.

[0150] In optional embodiments, such as Figure 6 As shown, step S402 may further include:

[0151] Step S600: If the initial contour vertices do not match the prior shape information of the target region, perform the following operation: perform boundary completion based on the first boundary set to update the first segmentation mask; wherein, the updated first segmentation mask is used for boundary calculation.

[0152] The updated first segmentation mask M fill It is the segmentation mask after filling in the dent.

[0153] In some embodiments, it can be determined whether to update the first segmentation mask by judging whether the number of initial contour vertices matches the prior information of the target region shape (number of vertices). If it is determined that the number of initial contour vertices does not match the prior information of the target region shape (number of vertices), the first segmentation mask can be filled with boundaries according to the first boundary set, and then the updated first segmentation mask can be used for boundary calculation.

[0154] In some embodiments, a convex hull solving algorithm can be used to complete the boundaries of the first boundary set to obtain a second boundary set, which is the first boundary set after the holes are filled. That is, the first boundary set can be completed using a convex hull solving algorithm. The convex hull solving algorithm may include: Jarvis-March (stepping) algorithm, Graham (scanning) algorithm, and Andrew W., etc.

[0155] In an exemplary application, if the initial contour vertices do not match the prior information about the shape of the target region, it indicates that the first segmentation mask may have holes or missing corners (e.g., Figure 16(As shown in the left side of the image). At this point, it is necessary to complete the first segmentation mask. The first segmentation mask after boundary completion is as follows: Figure 16 As shown in the right-hand figure, the technical solution of this application determines the integrity of the first segmentation mask by performing operations such as hole detection on the first segmentation mask. When holes or other defects exist in the first segmentation mask, the boundary of the target region is filled in to update the first segmentation mask, so that the first segmentation mask can be corrected. This allows for the accurate acquisition of the target region's contour vertices based on the updated first segmentation mask, thereby making the obtained target region more precise.

[0156] The above describes how to ensure that the first segmentation mask meets the requirements (meets prior information). If it meets the requirements, the first segmentation mask can be directly used for boundary calculation; if it does not meet the requirements, the first segmentation mask is updated to meet the requirements before being used for boundary calculation. The boundary calculation method can be as follows.

[0157] In optional embodiments, such as Figure 7 As shown, step S402 may include:

[0158] Step S700: Obtain a second segmentation mask based on the first segmentation mask, wherein the second segmentation mask is a segmentation mask for the boundary of the target region.

[0159] Step S702: Obtain the first gradient map corresponding to the target image based on the target image.

[0160] Step S704: Obtain a second gradient map based on the first gradient map and the second segmentation mask. The second gradient map is the gradient map of the boundary of the target region.

[0161] Step S706: Obtain the boundary edge of the target region based on the second gradient map.

[0162] The first gradient map can be obtained in various ways, such as through the Sobel operator, the Prewitt operator, the Laplace operator, etc.

[0163] In some embodiments, the boundary of the target region can be obtained by operating on the second gradient map using an edge detection algorithm, wherein the edge detection algorithm may include: Sobel edge detection algorithm, Canny edge detection algorithm, and Laplacian edge detection algorithm. For example, obtaining the boundary of the target region based on the second gradient map can be achieved through the following operations: (1) filtering the second gradient map. (2) applying an edge detection operator to the filtered second gradient map to highlight the boundary in the image. (3) thresholding the second gradient map, marking pixels with gradient values ​​higher than a set threshold as edge points, and pixels with gradient values ​​lower than the threshold as non-edge points. (4) performing an edge connection operation on the thresholded second gradient map, connecting adjacent edge points to form continuous edge lines. The boundary of the target region is extracted based on the connected edge lines.

[0164] In some embodiments, obtaining the second gradient map based on the first gradient map and the second segmentation mask can be achieved by applying the second segmentation mask to the first gradient map to obtain the second gradient map. Specifically, this can be achieved by setting the pixel positions in the first gradient map that do not belong to the target region to zero. In this embodiment, the segmentation mask (second segmentation mask) of the target region boundary is obtained, and then the gradient map (second gradient map) of the target region boundary is obtained based on the second segmentation mask and the first gradient map, thereby calculating the boundary of the target region. The technical solution of this application embodiment determines the second gradient map of the target region boundary based on the second segmentation mask and the first gradient map, thereby determining the boundary of the target region, making the boundary of the target region obtained in this embodiment more accurate, so that the contour vertices of the target region can be calculated more accurately based on the boundary in the future.

[0165] The following provides an exemplary scheme for obtaining the segmentation mask (i.e., the second segmentation mask) of the target region boundary.

[0166] In optional embodiments, such as Figure 8 As shown, step S700 may include:

[0167] Step S800: According to the dilation and erosion algorithm, the first segmentation mask is dilated and eroded to obtain the dilated segmentation mask and the eroded segmentation mask.

[0168] Step S800: Obtain the second segmentation mask based on the dilation segmentation mask and the erosion segmentation mask.

[0169] The steps to obtain the second segmentation mask by the dilatational erosion algorithm are as follows: (1) First, perform an erosion operation on the first segmentation mask to obtain the eroded segmentation mask M. fill-erodeThe erosion operation involves setting the template size based on the target region's dimensions and sliding it along the target region's boundary line with a certain step size. During the sliding process, pixels on the boundary line are set to 0, and erosion is performed until the erosion segmentation mask M is obtained. fill-erode (2) After the erosion process is completed, the image corresponding to the erosion segmentation mask is dilated to restore the target area to its original size and allow the boundary lines to reappear, so as to obtain the dilated segmentation mask M. fill-dilate (3) Subtracting the erosion segmentation mask from the dilation segmentation mask yields the segmentation mask of the target region boundary (i.e., the second segmentation mask), which can be expressed by the formula: M fill-dilate -M fill-erode .

[0170] In this embodiment, the dilation operation can fill in holes or depressions in the first segmentation mask, thus correcting the first segmentation mask. The erosion operation can effectively remove noise and make the first segmentation mask have clearer boundaries. The technical solution of this application embodiment corrects the first segmentation mask through a dilation-erosion algorithm to obtain a second segmentation mask. This second segmentation mask is used for subsequent boundary calculation of the target region. Since the second segmentation mask has no holes or depressions, it can improve the accuracy of subsequent boundary segmentation.

[0171] The following provides an exemplary scheme for obtaining the first gradient graph.

[0172] In optional embodiments, such as Figure 9 As shown, step S702 may include:

[0173] Step S900: Obtain a first grayscale image G corresponding to the target image based on the target image.

[0174] Step S902: Perform image enhancement operation on the first grayscale image G to obtain a second grayscale image.

[0175] Step S904: Obtain the first gradient image S based on the second grayscale image.

[0176] Compared to the second grayscale image, the first grayscale image has lower clarity.

[0177] The image enhancement operations may include brightness enhancement, contrast enhancement, image sharpening, and grayscale transformation. Specifically, the first grayscale image can undergo contrast enhancement using methods such as linear transformation, histogram normalization, gamma transformation, and global histogram equalization. In some embodiments, when the image enhancement operations include brightness enhancement and contrast enhancement, performing brightness and contrast enhancement operations on the first grayscale image can improve its image quality, resulting in a clearer second grayscale image. This, in turn, allows for better detection of image gradients in the target region using the second grayscale image.

[0178] In this embodiment, a second grayscale image can be obtained by performing image enhancement on the first grayscale image. Due to the image enhancement, the second grayscale image is clearer. Furthermore, image enhancement can emphasize local characteristics of the image, suppress uninteresting features, and improve the visual effect of the image. Therefore, the technical solution of this application embodiment can emphasize the target region and improve the clarity of the grayscale image through image enhancement, so that the image gradient of the target region can be better detected subsequently using the second grayscale image.

[0179] In optional embodiments, such as Figure 10 As shown, step S904 may include:

[0180] Step S1000: Obtain the X-direction gradient map of the first gradient map in the X direction.

[0181] Step S1002: Obtain the Y-direction gradient map of the first gradient map in the Y direction.

[0182] Step S1004: The first gradient map S is obtained by merging and fusing the gradient map in the X direction and the gradient map in the Y direction.

[0183] The X-direction gradient map and Y-direction gradient map can be obtained from: gradient maps based on the Sobel operator, gradient maps based on the Prewitt operator, gradient maps based on the Laplace operator, etc., without limitation. Taking the acquisition of the corresponding gradient map based on the Sobel operator as an example, the specific steps are as follows: Based on the second grayscale image, calculate the X-direction Sobel gradient map of the first gradient map in the X direction, and the Y-direction Sobel gradient map of the first gradient map in the Y direction. Merge and fuse the X-direction Sobel gradient map and the Y-direction Sobel gradient map by taking the maximum value to obtain the first gradient map S.

[0184] For an image, areas where pixel values ​​change drastically are considered image edges. Therefore, by calculating the gradient of pixel values ​​in an image, the locations of image edges can be determined. In the optional embodiment described above, vertical edge information is obtained based on the X-direction gradient map, and horizontal edge information is obtained based on the Y-direction gradient map. By obtaining both vertical and horizontal edge information, complete edge information is obtained. By calculating a first gradient map of the target image, the boundaries of the target region can be more accurately determined subsequently based on this first gradient map.

[0185] Step S404 Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region.

[0186] The preset vertex calculation method may include: line detection algorithm, polygon fitting algorithm, horizontal segmentation algorithm, etc.

[0187] Each preset vertex calculation method can correspond to a set of contour vertices.

[0188] The following are some exemplary vertex calculation schemes.

[0189] Option 1:

[0190] When the preset vertex calculation method is a line detection algorithm, in optional embodiments, such as Figure 11 As shown, step S404 may include:

[0191] Step S1100: Perform line detection on the boundary using a line detection algorithm to obtain a set of candidate lines.

[0192] Step S1102: Obtain the number of edges of the target region based on the prior information of the target region shape.

[0193] Step S1104: Based on the number of edges, cluster the candidate lines in the candidate line set to obtain multiple target line sets. The number of edges is the same as the number of clusters, and each cluster corresponds to one target line set.

[0194] Step S1106: Based on the multiple target line sets, obtain multiple target lines, with each target line set corresponding to one target line.

[0195] Step S1108: Obtain the first contour vertex set based on the plurality of target lines.

[0196] In an exemplary application, when the target area is a rectangular object such as a billboard or display screen in a video, the number of sides of the target area is 4.

[0197] The line detection algorithms may include: Hough transform line detection algorithm, line segment detection algorithm (LSD), FLD (Fisher linear discriminant analysis) line detection algorithm, EDlines line detection algorithm, CannyLines line detection algorithm, etc. Of course, other detection algorithms may also be used, and there is no limitation here.

[0198] Clustering methods may include: K-means clustering algorithm, ISODATA (Iterative Self-Organizing Data Analysis), Bi-kmeans clustering algorithm, or other algorithms.

[0199] For example, such as Figure 18 As shown, when the target area is a rectangular billboard in the video, and the clustering algorithm is K-means clustering, the specific steps for obtaining the first contour vertex set based on the boundary can be as follows: A line detection algorithm is applied to the boundary of the target area to obtain a candidate line set. Based on the known prior information about the target area shape, the number of edges is 4. The number of edges is used as the cluster number for the K-means clustering algorithm; in this case, the cluster number is 4. K-means clustering is performed on the candidate lines in the candidate line set based on the number of edges, and outliers are removed, resulting in 4 target line sets after classification. The average of the candidate lines in each of the 4 target line sets is taken to obtain 4 target lines. Then, the intersection points of the 4 target lines are calculated to obtain the first contour vertex set of the target area.

[0200] In this embodiment, a target straight line in the target region is calculated using a straight line detection algorithm. Then, the vertices intersecting the target straight lines in the target region are calculated to determine the first contour vertex set of the target region. The first contour vertex set obtained in this way can accurately determine the contour, and thus determine the corrected target region.

[0201] Option 2:

[0202] When the preset vertex calculation method is a polygon fitting algorithm, in optional embodiments, such as Figure 12 As shown, step S404 may include:

[0203] Step S1200: Extract the contour of the boundary to obtain the outer boundary corresponding to the boundary.

[0204] Step S1200: Perform polygon fitting on the outer boundary to obtain the second contour vertex set.

[0205] A contour extraction algorithm can be applied to the boundary to obtain the outer boundary corresponding to the boundary.

[0206] A contour represents the boundary of an object in an image. In this embodiment, a contour extraction algorithm is used to extract the outer boundary (contour) of the target region. Based on this, a polygon fitting algorithm can be used to approximate the outer boundary (contour) of the target region into a polygon composed of straight lines, thereby obtaining a second set of contour vertices.

[0207] In this embodiment, a second set of contour vertices for the target region is determined using a polygon fitting algorithm. This second set of contour vertices allows for accurate contour determination, thereby identifying the corrected target region.

[0208] Step S406 Based on the one or more contour vertex sets, obtain the target contour vertex set.

[0209] The target contour vertex set is the modified set of contour vertices of the target region.

[0210] In optional embodiments, such as Figure 13 As shown, step S406 may include:

[0211] If the second set of contour vertices matches the prior shape information of the target region, perform the following operations:

[0212] Step S1300: Set the weight of the first contour vertex set to the first weight, and set the weight of the second contour vertex set to the second weight.

[0213] Step S1302: Obtain the target contour vertex set by performing a weighted summation on the first contour vertex set and the second contour vertex set.

[0214] Based on the weight W of the first contour vertex set line The weight W of the second contour vertex set fill The formula for obtaining the vertex set of the target contour can be:

[0215] V final =V line *W line +V fill *W fill .

[0216] In an exemplary application, the first weight can be set to 0.5, and the second weight can be set to 0.5.

[0217] When the second contour vertex set matches the prior shape information of the target region, the segmentation mask corresponding to the target region does not have any missing corners or other defects. Therefore, the second contour vertex set obtained based on the segmentation mask corresponding to the target region does not contain vertices formed due to missing corners or other defects, meaning the accuracy of the second contour vertex set is high (effective). With a high accuracy of the second contour vertex set, a weighted sum of the first and second contour vertex sets can effectively improve the segmentation accuracy of the target region.

[0218] The technical solution of this application embodiment determines the target contour vertex set by weighted summation of the first contour vertex set and the second contour vertex set. Compared with the first and second contour vertex sets, the target contour vertex set has higher accuracy, and therefore the corrected target region obtained based on the target contour vertex set is more accurate.

[0219] In optional embodiments, such as Figure 14 As shown, step S406 may further include:

[0220] If the first set of vertices does not match the prior shape information, perform the following operations:

[0221] Step S1400: Set the weight of the first contour vertex set to the third weight, and set the weight of the second contour vertex set to the fourth weight, wherein the third weight is greater than the fourth weight.

[0222] Step S1402: Obtain the target contour vertex set by performing a weighted summation on the first contour vertex set and the second contour vertex set.

[0223] Based on the weight W of the first contour vertex set line The weight W of the second contour vertex set fill The formula for obtaining the set of vertices of the target contour is:

[0224] V final =V line *W line +V fill *W fill .

[0225] When the first vertex set does not match the prior shape information, the segmentation mask corresponding to the target region may have missing corners. Therefore, the accuracy of the second contour vertex set obtained by the polygon fitting algorithm is low (unreliable). Given the low accuracy of the second contour vertex set, the fourth weight of the second contour vertex set is set to a small value (less than the value of the third weight). In some embodiments, the third weight can be set to 1, and the fourth weight can be set to 0. In this case, the target contour vertex set is the first contour vertex set.

[0226] In this embodiment, corresponding weights are set for the first contour vertex set and the second contour vertex set. A smaller weight value is assigned to the contour vertex set with lower accuracy, thereby reducing its influence on the target contour vertex set. After setting the corresponding weights, the first and second contour vertex sets are weighted and summed to determine the target contour vertex set. This results in a more accurate target contour vertex set, allowing for precise correction of the shape and contour of the target region based on the target contour vertex set.

[0227] Step S408 Based on the set of target contour vertices, the corrected target region is obtained.

[0228] The boundary of the first segmentation mask of the target area is extracted. Then, based on the boundary of the target area and the corresponding vertex calculation method, the target contour vertex set is obtained. The shape contour of the target area is then corrected, making the shape contour of the target area more accurate, improving stability and robustness, and enhancing the effectiveness of print advertising.

[0229] In other words, the corrected target area is a region with no depressions or missing corners, and whose shape and outline are relatively precise. This corrected target area can be used to embed objects such as billboards, creating a very good effect.

[0230] To make this application easier to understand, an exemplary application is provided below with reference to Figure 15.

[0231] S11, obtain the first segmentation mask of the target region.

[0232] S12, extract the boundary of the first segmentation mask according to the first segmentation mask; obtain the initial contour vertices according to the boundary.

[0233] S13, determine whether the initial contour vertices match the prior shape information of the target region. Two cases are as follows:

[0234] ① When the match is found, the target area has no holes or missing corners, and the boundary is calculated directly based on the first segmentation mask.

[0235] ② If there is a mismatch, the target area may have holes or missing corners. A boundary completion operation is performed to update the first segmentation mask.

[0236] S14. Obtain the boundary segmentation mask (second segmentation mask) of the target region according to the dilatational erosion algorithm.

[0237] S15 converts the target image into a grayscale image and performs brightness and contrast enhancement operations.

[0238] S16, calculate the first gradient map based on the enhanced grayscale image, and then obtain the boundary gradient map (second gradient map) of the target region based on the second segmentation mask.

[0239] S17, Calculate the boundary of the target region based on the second gradient map.

[0240] S18, obtaining the contour vertex set through two vertex calculation methods:

[0241] ①S181, the line detection algorithm, the specific steps are as follows:

[0242] S1811, perform line detection on the boundary to obtain a set of candidate lines.

[0243] S1812: Obtain the number of edges based on prior shape information, and classify the candidate line set according to the clustering algorithm.

[0244] S1813, remove outliers from the candidate line set, calculate the mean of the lines in each candidate line set, and obtain multiple target lines, with each candidate line set corresponding to one target line.

[0245] S1814 calculates the intersection points of multiple straight lines, and the intersection points are the set of vertices of the first contour.

[0246] ②S182, polygon fitting algorithm, the specific steps are as follows:

[0247] S1821, apply a contour extraction algorithm to the boundary to obtain the corresponding outer boundary.

[0248] S1822, a polygon fitting algorithm is used on the outer boundary to obtain the set of vertices of the second contour.

[0249] S19, determine whether the set of vertices of the second contour matches the prior information, with two possible outcomes:

[0250] ① When they match, the segmentation mask has no missing corners. The weight of the first contour vertex set is set as the first weight, and the weight of the second contour vertex set is set as the second weight.

[0251] ② When there is a mismatch, the segmentation mask has a missing corner. The weights are set as follows: the third weight of the first contour vertex set is greater than the fourth weight of the second contour vertex set.

[0252] S20: The first contour vertex set and the second contour vertex set are weighted and summed to obtain the contour vertices of the target region.

[0253] S21, Obtain the corrected target region based on the vertices of the target region contour.

[0254] The advantages of this technical solution are: (1) When the first segmentation mask is basically regular (without holes, missing corners, etc.) but the edges are relatively rough, the first segmentation mask can be fine-tuned to obtain a more accurate target area shape contour. (2) When the first segmentation mask has depressions or missing parts, it can be adaptively processed to obtain a corrected target area shape contour. (3) It has a very low dependence on the first segmentation mask and has extremely high robustness and stability.

[0255] It should be noted that the above solution can be used to insert advertisements into videos, or in other scenarios.

[0256] Example 2

[0257] Figure 19 The diagram schematically illustrates a block diagram of an image segmentation apparatus according to Embodiment 2 of this application. This image segmentation apparatus can be segmented into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of this application. The program module referred to in this embodiment is a series of computer program instruction segments capable of performing a specific function. The following description will specifically introduce the function of each program module in this embodiment. For example... Figure 19 As shown, the device 1900 may include: a first acquisition module 1910, a second acquisition module 1920, a third acquisition module 1930, a fourth acquisition module 1940, and a fifth acquisition module 1950, wherein:

[0258] The first acquisition module 1910 is used to acquire a first segmentation mask in the target image; wherein, the first segmentation mask is a segmentation mask corresponding to the target region in the target image;

[0259] The second acquisition module 1920 is used to acquire the boundary of the target region based on the first segmentation mask and the target image;

[0260] The third acquisition module 1930 is used to acquire one or more sets of contour vertices of the target region according to the boundary and one or more preset vertex calculation methods;

[0261] The fourth acquisition module 1940 is used to acquire a target contour vertex set based on the one or more contour vertex sets;

[0262] The fifth acquisition module 1950 is used to acquire the corrected target region based on the target contour vertex set.

[0263] In an optional embodiment, the second acquisition module 1920 is further configured to:

[0264] Based on the first segmentation mask, obtain the first boundary set of the segmentation mask;

[0265] Based on the first boundary set, obtain the initial contour vertices;

[0266] If the initial contour vertices match the prior information of the target region shape, the first mask is used for boundary calculation.

[0267] In an optional embodiment, if the initial contour vertices do not match the prior shape information of the target region, the second acquisition module 1920 is further configured to:

[0268] Boundary completion is performed based on the first boundary set to update the first segmentation mask;

[0269] The updated first segmentation mask is used for boundary calculation.

[0270] In an optional embodiment, the second acquisition module 1920 is further configured to:

[0271] Based on the first segmentation mask, a second segmentation mask is obtained, wherein the second segmentation mask is the segmentation mask of the target region boundary;

[0272] Based on the target image, obtain the first gradient map corresponding to the target image;

[0273] Based on the first gradient map and the second segmentation mask, a second gradient map is obtained, which is the gradient map of the boundary of the target region.

[0274] The boundary of the target region is obtained based on the second gradient map.

[0275] In an optional embodiment, the second acquisition module 1920 is further configured to:

[0276] According to the dilation and erosion algorithm, the first segmentation mask is dilated and eroded to obtain the dilated segmentation mask and the eroded segmentation mask.

[0277] The second segmentation mask is obtained based on the dilation segmentation mask and the erosion segmentation mask.

[0278] In an optional embodiment, the second acquisition module 1920 is further configured to:

[0279] Based on the target image, obtain a first grayscale image corresponding to the target image;

[0280] An image enhancement operation is performed on the first grayscale image to obtain a second grayscale image;

[0281] The first gradient image is obtained based on the second grayscale image.

[0282] In an optional embodiment, the second acquisition module 1920 is further configured to:

[0283] Obtain the X-direction gradient map of the first gradient map in the X direction;

[0284] Obtain the Y-direction gradient map of the first gradient map;

[0285] The first gradient map is obtained by merging and fusing the gradient map in the X direction and the gradient map in the Y direction.

[0286] In an optional embodiment, the one or more preset vertex calculation methods include a line detection algorithm; the third acquisition module 1930 is further configured to:

[0287] The boundary is subjected to line detection using a line detection algorithm to obtain a set of candidate lines;

[0288] The number of edges in the target region is obtained based on prior information about the shape of the target region.

[0289] Based on the number of edges, the candidate lines in the candidate line set are clustered to obtain multiple target line sets; wherein, the number of edges is the same as the number of clusters, and each cluster corresponds to one target line set;

[0290] Based on the multiple sets of target lines, multiple target lines are obtained, with each set of target lines corresponding to one target line;

[0291] Based on the plurality of target straight lines, obtain the first set of contour vertices.

[0292] In an optional embodiment, the third acquisition module 1930 is further configured to:

[0293] Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region, including:

[0294] Contour extraction is performed on the boundary to obtain the outer boundary corresponding to the boundary;

[0295] Polygon fitting is performed on the outer boundary to obtain the second contour vertex set.

[0296] In an optional embodiment, the fourth acquisition module 1940 is further configured to:

[0297] If the second set of contour vertices matches the prior shape information of the target region, perform the following operations:

[0298] Set the weight of the first contour vertex set to the first weight, and set the weight of the second contour vertex set to the second weight;

[0299] The target contour vertex set is obtained by performing a weighted summation on the first contour vertex set and the second contour vertex set.

[0300] In an optional embodiment, the fourth acquisition module 1940 is further configured to:

[0301] If the first set of vertices does not match the prior shape information, perform the following operations:

[0302] The weight of the first contour vertex set is set as the third weight, and the weight of the second contour vertex set is set as the fourth weight, wherein the third weight is greater than the fourth weight;

[0303] The target contour vertex set is obtained by performing a weighted summation on the first contour vertex set and the second contour vertex set.

[0304] Example 3

[0305] Figure 20 This illustration schematically depicts the hardware architecture of a computer device 10000 suitable for implementing an image segmentation method according to Embodiment 3 of this application. In some embodiments, the computer device 10000 may be a terminal device such as a smartphone, wearable device, tablet computer, personal computer, in-vehicle terminal, game console, virtual machine, workbench, digital assistant, set-top box, or robot. In other embodiments, the computer device 10000 may be a rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc. Figure 20 As shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked with each other via a system bus. Wherein:

[0306] The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of a computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as program code for image segmentation methods. In addition, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0307] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0308] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communications (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0309] It should be pointed out that, Figure 20 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0310] In this embodiment, the image segmentation method stored in memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of this application.

[0311] Example 4

[0312] This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the image segmentation method in the embodiments.

[0313] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEP ROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Of course, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the image segmentation method in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0314] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computer devices. They can be centralized on a single computer device or distributed across a network of multiple computer devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computer device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0315] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. An image segmentation method, characterized in that, The method includes: Obtain the first segmentation mask in the target image; wherein, the first segmentation mask is the segmentation mask corresponding to the target region in the target image; The boundary of the target region is obtained based on the first segmentation mask and the target image; Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region; wherein, the one or more contour vertex sets include a first contour vertex set and / or a second contour vertex set; Based on the one or more contour vertex sets, obtain the target contour vertex set; Based on the set of target contour vertices, obtain the corrected target region; The process of obtaining the target contour vertex set based on the one or more contour vertex sets includes: If the second set of contour vertices matches the prior shape information of the target region, perform the following operations: Set the weight of the first contour vertex set to the first weight, and set the weight of the second contour vertex set to the second weight; The target contour vertex set is obtained by performing a weighted summation on the first contour vertex set and the second contour vertex set. Obtaining the target contour vertex set based on the one or more contour vertex sets further includes: If the first set of contour vertices does not match the prior shape information, perform the following operations: The weight of the first contour vertex set is set as the third weight, and the weight of the second contour vertex set is set as the fourth weight, wherein the third weight is greater than the fourth weight; The target contour vertex set is obtained by performing a weighted summation on the first contour vertex set and the second contour vertex set.

2. The method according to claim 1, characterized in that, Based on the first segmentation mask and the target image, the boundary of the target region is obtained, including: Based on the first segmentation mask, obtain the first boundary set of the segmentation mask; Based on the first boundary set, obtain the initial contour vertices; If the initial contour vertices match the prior information of the target region shape, the first segmentation mask is used for boundary calculation.

3. The method according to claim 2, characterized in that, Obtaining the boundary of the target region based on the first segmentation mask and the target image further includes: If the initial contour vertices do not match the prior shape information of the target region, perform the following operations: Boundary completion is performed based on the first boundary set to update the first segmentation mask; The updated first segmentation mask is used for boundary calculation.

4. The method according to claim 1, characterized in that, The step of obtaining the boundary of the target region based on the first segmentation mask and the target image includes: Based on the first segmentation mask, a second segmentation mask is obtained, wherein the second segmentation mask is the segmentation mask of the target region boundary; Based on the target image, obtain the first gradient map corresponding to the target image; Based on the first gradient map and the second segmentation mask, a second gradient map is obtained, which is the gradient map of the boundary of the target region. The boundary of the target region is obtained based on the second gradient map.

5. The method according to claim 4, characterized in that, The step of obtaining the second segmentation mask based on the first segmentation mask includes: According to the dilation and erosion algorithm, the first segmentation mask is dilated and eroded to obtain the dilated segmentation mask and the eroded segmentation mask. The second segmentation mask is obtained based on the dilation segmentation mask and the erosion segmentation mask.

6. The method according to claim 4, characterized in that, The step of obtaining a first gradient map corresponding to the target image based on the target image includes: Based on the target image, obtain a first grayscale image corresponding to the target image; An image enhancement operation is performed on the first grayscale image to obtain a second grayscale image; The first gradient image is obtained based on the second grayscale image.

7. The method according to claim 6, characterized in that, The step of obtaining the first gradient map based on the second grayscale image includes: obtaining the X-direction gradient map of the first gradient map in the X direction; Obtain the Y-direction gradient map of the first gradient map; The first gradient map is obtained by merging and fusing the gradient map in the X direction and the gradient map in the Y direction.

8. The method according to any one of claims 1 to 7, characterized in that, The one or more preset vertex calculation methods include a line detection algorithm; Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region, including: The boundary is subjected to line detection using a line detection algorithm to obtain a set of candidate lines; The number of edges in the target region is obtained based on prior information about the shape of the target region. Based on the number of edges, the candidate lines in the candidate line set are clustered to obtain multiple target line sets; wherein, the number of edges is the same as the number of clusters, and each cluster corresponds to one target line set; Based on the multiple sets of target lines, multiple target lines are obtained, with each set of target lines corresponding to one target line; Based on the plurality of target straight lines, obtain the first set of contour vertices.

9. The method according to claim 8, characterized in that, The one or more preset vertex calculation methods include contour extraction algorithms; Based on the boundary and one or more preset vertex calculation methods, obtain one or more contour vertex sets of the target region, including: Contour extraction is performed on the boundary to obtain the outer boundary corresponding to the boundary; Polygon fitting is performed on the outer boundary to obtain the second contour vertex set.

10. An image segmentation apparatus, characterized in that, The device includes: The first acquisition module is used to acquire a first segmentation mask in the target image; wherein the first segmentation mask is a segmentation mask corresponding to the target region in the target image; The second acquisition module is used to acquire the boundary of the target region based on the first segmentation mask and the target image; The third acquisition module is used to acquire one or more contour vertex sets of the target region according to the boundary and one or more preset vertex calculation methods; wherein, the one or more contour vertex sets include a first contour vertex set and / or a second contour vertex set; The fourth acquisition module is used to acquire a target contour vertex set based on the one or more contour vertex sets; The fifth acquisition module is used to acquire the corrected target region based on the target contour vertex set; The fourth acquisition module is further configured to: If the second set of contour vertices matches the prior shape information of the target region, perform the following operations: Set the weight of the first contour vertex set to the first weight, and set the weight of the second contour vertex set to the second weight; The target contour vertex set is obtained by performing a weighted summation on the first contour vertex set and the second contour vertex set. The fourth acquisition module is further configured to: If the first set of contour vertices does not match the prior shape information, perform the following operations: The weight of the first contour vertex set is set as the third weight, and the weight of the second contour vertex set is set as the fourth weight, wherein the third weight is greater than the fourth weight; The target contour vertex set is obtained by performing a weighted summation on the first contour vertex set and the second contour vertex set.

11. A computer device, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 9.

13. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1 to 9.

Images