Large model image segmentation method, device, equipment, storage medium and program product

Abstract

The present invention provides a large-model image segmentation method, device, equipment, storage medium and program product, belonging to the field of computer technology. The method comprises: inputting a preprocessed target image into a large image segmentation model to obtain a binary segmentation map generated by the large image segmentation model; determining the boundary points of the target object in the binary segmentation map based on the target pixel points with a value of 1 in the binary segmentation map and the coordinates of the target pixel points in the target image; sorting the boundary points and connecting the sorted boundary points in sequence to obtain the mask information of the target object. The large-model image segmentation method, device, equipment, storage medium and program product provided by the present invention can greatly improve the efficiency and accuracy of image segmentation data acquisition, and effectively save time and labor costs.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a large model image segmentation method, device, equipment, storage medium and program product. Background Art

[0002] Currently, deep learning is widely used in image segmentation, and creating a dataset for training image segmentation models has become a necessary skill for researchers in the field of CV (Computer Vision). Creating an image segmentation dataset requires a large number of images, annotation files corresponding to each image, and weight files used to calculate the model. So far, the annotation files for creating image segmentation datasets still need to be manually annotated using image annotation software such as labelimg or labelme, which is very time-consuming and labor-intensive. Summary of the invention

[0003] The present invention provides a large model image segmentation method, device, equipment, storage medium and program product, which are used to solve the defects of time-consuming and labor-intensive image segmentation data acquisition in the prior art and realize efficient acquisition of image segmentation data.

[0004] In a first aspect, the present invention provides a large model image segmentation method, comprising:

[0005] Inputting the preprocessed target image into the image segmentation model to obtain a binary segmentation map generated by the image segmentation model; the preprocessed target image includes the annotation information of the target area where the target object is located; the binary segmentation map is matched with the target area;

[0006] Determine the boundary points of the target object in the binary segmentation map based on the target pixel points whose values ​​are 1 in the binary segmentation map and the coordinates of the target pixel points in the target image;

[0007] The boundary points are sorted and the sorted boundary points are connected in sequence to obtain the mask information of the target object.

[0008] In one embodiment, determining the boundary points of the target object in the binary segmentation map based on the target pixel points whose values ​​are 1 in the binary segmentation map and the coordinates of the target pixel points in the target image includes:

[0009] Determine the pixel points located in the same column based on the horizontal coordinate of each target pixel point in the target image;

[0010] Determine a first candidate boundary point based on the vertical coordinates of the pixel points located in the same column in the target image;

[0011] Determine the pixel points located in the same row based on the ordinate of each target pixel point in the target image;

[0012] Determine a second candidate boundary point based on the horizontal coordinates of the pixel points located in the same row in the target image;

[0013] The first candidate boundary points and the second candidate boundary points are deduplicated, and the deduplicated first candidate boundary points and the second candidate boundary points are used as boundary points of the target object in the binary segmentation map.

[0014] In one embodiment, determining the first candidate boundary point based on the vertical coordinates of the pixels located in the same column in the target image includes:

[0015] Determine a first pixel point from the pixel points located in the same column, and use the first pixel point as the first candidate boundary point;

[0016] The first pixel satisfies any of the following conditions:

[0017] There is no pixel point located in the same column and having a ordinate greater than the ordinate of the first pixel point by one ordinate unit;

[0018] There is no pixel point located in the same column and having a ordinate smaller than the ordinate of the first pixel point by one ordinate unit.

[0019] In one embodiment, the step of sorting the boundary points includes:

[0020] Determine the starting point from each boundary point, and regard the remaining boundary points as unprocessed boundary points;

[0021] Repeat the following traversal operation until the current point is the same as the starting point:

[0022] Determine a target adjacent point of the current point from the unprocessed boundary points;

[0023] Excluding the target adjacent point from the unprocessed boundary points to serve as sorted boundary points, and using the target adjacent point as the current point of the next traversal operation;

[0024] In the first traversal operation, the starting point is regarded as the current point.

[0025] In one embodiment, the method further comprises:

[0026] After the traversal operation is completed, determining the number of sorted boundary points;

[0027] If the number of sorted boundary points is greater than or equal to the number threshold, the sorted boundary points are connected in sequence;

[0028] If the number of sorted boundary points is less than the number threshold, the traversal operation is re-executed.

[0029] In one embodiment, the method further comprises:

[0030] If the target adjacent point of the current point is not found in the unprocessed boundary points, the current traversal operation is rolled back to determine another target adjacent point of the current point from the unprocessed boundary points in the last traversal operation.

[0031] In a second aspect, the present invention provides a large model image segmentation device, comprising:

[0032] A generating module, configured to generate a binary segmentation map based on a preprocessed target image; the preprocessed target image includes annotation information of a target area where the target object is located; and the binary segmentation map is matched with the target area;

[0033] A determination module, configured to determine a boundary point of the target object in the binary segmentation map based on a target pixel point whose value is 1 in the binary segmentation map and a coordinate of the target pixel point in the target image;

[0034] The sorting module is used to sort the boundary points and connect the sorted boundary points in sequence to obtain the mask information of the target object.

[0035] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the large model image segmentation method as described in the first aspect is implemented.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large model image segmentation method as described in the first aspect.

[0037] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the large model image segmentation method as described in the first aspect is implemented.

[0038] The large-model image segmentation method, apparatus, equipment, storage medium and program product provided by the present invention can automatically and efficiently determine the image segmentation data by simply marking the target object of the target image and generating a binary segmentation map from the image segmentation large model, and then determining the boundary points from the binary segmentation map and sorting the boundary points to obtain the mask information of the target object. Compared with the existing manual marking method, the efficiency and accuracy of image segmentation data acquisition are greatly improved, and time and labor costs are effectively saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 It is a flow chart of the large model image segmentation method provided by the present invention.

[0041] Figure 2 It is a schematic diagram of the process of sorting the boundary points in the large model image segmentation method provided by the present invention.

[0042] Figure 3 It is a structural schematic diagram of the large model image segmentation device provided by the present invention.

[0043] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Figure 1 Schematic diagram of the process of the large model image segmentation method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0046] Step 110: input the preprocessed target image into the image segmentation model to obtain a binary segmentation map generated by the image segmentation model; the preprocessed target image includes the annotation information of the target area where the target object is located; the binary segmentation map is matched with the target area;

[0047] Step 120, determining the boundary points of the target object in the binary segmentation map based on the target pixel points with a value of 1 in the binary segmentation map and the coordinates of the target pixel points in the target image;

[0048] Step 130 : sort the boundary points and connect the sorted boundary points in sequence to obtain mask information of the target object.

[0049] It should be noted that the execution subject of the large model image segmentation method provided by the present invention can be a computer device, such as a mobile phone, a tablet computer, a laptop computer, a PDA, etc.

[0050] Before step 110, the target image may be preprocessed. Specifically, image annotation software such as labelimg, labelme, etc. may be used to annotate the target object in the target image.

[0051] The target image may be any image containing a target object. The target object may be a person, an animal, a building, a background, etc. The present invention does not specifically limit the specific types of the target image and the target object.

[0052] Specifically, when annotating a target image, a target area where a target object is located in the target image may be annotated.

[0053] For example, a rectangular frame may be used to mark the target area where the target object is located, that is, only two points (diagonal points of the rectangle) are marked to define the target area. Of course, other shapes of marking frames may also be used to mark the target area where the target object is located, such as a circle, a triangle, etc., and the present invention does not specifically limit this.

[0054] It should be noted that image segmentation requires marking the overall outline of the target object, which often requires marking dozens of points. Taking the COCO dataset, which is more commonly used in computer vision, as an example, on average, the target object marking method of the present invention only takes three to four seconds to mark all the target objects in an entire image, while image segmentation takes more than two minutes (each image in the COCO dataset often has multiple target objects).

[0055] If it is expanded to a dataset with hundreds to thousands of images as a whole, if the target object labeling method of the present invention takes one hour, then it will take two to three days to label the same image set as a segmentation dataset; even the SA-1B segmentation dataset used to train various large models (such as SAM, FCN, U-Net, SegNet, DeepLab, PSPNet, etc.) has 11 million images, and this dataset was manually labeled by the entire development team in two years. If you only need to label these 11 million images as a detection dataset, it will only take about two months.

[0056] In step 110, the preprocessed target image may be input into various image segmentation models to obtain a binary segmentation map.

[0057] For example, the preprocessed target image can be input into a large image segmentation model such as SAM (Segment Anything Model), and SAM can segment the target image according to the annotation information of the target area, thereby obtaining a binary segmentation map matching the target area. The binary segmentation map contains the binary image information of the target object.

[0058] It is understandable that, in addition to SAM, the present invention can also use various image segmentation models such as FCN, U-Net, SegNet, DeepLab, PSPNet, etc. to obtain a binary segmentation map, and the present invention does not make specific limitations on this.

[0059] In step 120, the boundary points of the target object in the binary segmentation map may be determined based on the target pixel points whose values ​​are 1 in the binary segmentation map and the coordinates of the target pixel points in the target image.

[0060] It should be noted that after obtaining the binary segmentation map, the map will be stored as a 0-1 binary map of the corresponding pixel size. Then, the threshold retrieval method can be used to filter out pixels with a value of 1 from the 0-1 binary map as target pixels. The target pixels are the pixels that constitute the segmentation mask of the target object.

[0061] Subsequently, the two-dimensional coordinates of each target pixel in the target image can be obtained, and based on the two-dimensional coordinates, it can be determined whether each target pixel has an adjacent target pixel, and based on the judgment result, it can be determined whether the target pixel is a boundary point located at the junction of the binary layers.

[0062] Specifically, if the target pixel point has no adjacent target pixel point, it can be determined that the target pixel point is a boundary point at the junction; if the target pixel point has an adjacent target pixel point, it can be determined that the target pixel point is not at the junction.

[0063] In step 130, the boundary points determined in step 120 need to be sorted, and then the sorted boundary points are connected in sequence to obtain the mask information of the target object, that is, the mask contour of the target object. The mask information of the target object can be used as image segmentation data for various model training, analysis, prediction and other tasks.

[0064] The large-model image segmentation method provided by the present invention can automatically and efficiently determine the image segmentation data by simply marking the target object of the target image and generating a binary segmentation map with the image segmentation large model, and then determining the boundary points from the binary segmentation map and sorting the boundary points to obtain the mask information of the target object. Compared with the existing manual marking method, the efficiency and accuracy of obtaining image segmentation data are greatly improved, and time and labor costs are effectively saved.

[0065] In one embodiment, step 120 may specifically include:

[0066] Based on the horizontal coordinates of each target pixel point in the target image, determine the pixel points located in the same column;

[0067] Determine a first candidate boundary point based on the vertical coordinates of the pixels located in the same column in the target image;

[0068] Based on the ordinate of each target pixel point in the target image, determine the pixel points located in the same row;

[0069] Determine a second candidate boundary point based on the horizontal coordinates of the pixel points located in the same row in the target image;

[0070] The first candidate boundary points and the second candidate boundary points are deduplicated, and the deduplicated first candidate boundary points and the second candidate boundary points are used as boundary points of the target object in the binary segmentation map.

[0071] Specifically, after obtaining the two-dimensional coordinates of each target pixel point in the target image, the pixel points located in the same column can be determined based on the horizontal coordinates of the target pixel points, and then the first candidate boundary point can be determined based on the vertical coordinates of the pixel points located in the same column in the target image.

[0072] Specifically, the range of the mask of the target object in the horizontal coordinate direction, that is, [x_min, x_max], can be determined by first calculating the maximum value x_max and the minimum value x_min of the horizontal coordinate of the target pixel point in the target image.

[0073] Next, you can traverse each x value (horizontal coordinate value) from x_min to x_max one by one, and search for all pixels equal to the x value for each x value. For example, if the current x=1, then after one search, all the pixels with x=1 in the target pixel are found, and these pixels constitute the pixel points located in the column of x=1 (that is, a set of pixels with the same horizontal coordinate but different vertical coordinates); then search from x_min to x_max, and all columns of the target pixel points are completely searched.

[0074] For the pixels located in the same column, their vertical coordinates can be searched continuously.

[0075] For the pixel points in the same column, the vertical coordinate is represented as row_point[1], and the vertical coordinates y_up_site = row_point[1]+ 1 and y_down_site = row_point[1] - 1 are searched for; where 1 represents the vertical coordinate unit of the two-dimensional coordinate system of the target image.

[0076] If at least one of y_up_site or y_down_site does not exist in the current column, that is, the pixel above or below the current pixel is not found, then the point is regarded as the first pixel, that is, the pixel at the junction of the binary layers.

[0077] Among them, the adjacent upper pixel point can be represented as a pixel point located in the same column as the first pixel point and whose vertical coordinate is one vertical coordinate unit larger than the vertical coordinate of the first pixel point; the adjacent lower pixel point can be represented as a pixel point located in the same column as the first pixel point and whose vertical coordinate is one vertical coordinate unit smaller than the vertical coordinate of the first pixel point.

[0078] After completing the vertical coordinate search of the pixel points in each column, the first pixel points corresponding to each column can be obtained, and all the first pixel points constitute the first candidate boundary point set.

[0079] Next, repeat the above process by switching the order of the horizontal and vertical axes:

[0080] First, by calculating the maximum value y_max and the minimum value y_min of the ordinate of the target pixel point in the target image, the range of the mask of the target object in the ordinate direction is determined, that is, [y_min, y_max].

[0081] Next, you can traverse each y value (ordinate value) from y_min to y_max one by one, and search for all pixels equal to the y value for each y value. For example, if the current y=1, then after one search, all the pixels with y=1 in the target pixel are found, and these pixels constitute the pixels located in the row of y=1 (that is, a set of pixels with the same ordinate but different abscissas); then search from y_min to y_max, and all rows of the target pixel are completely searched.

[0082] For the pixels located in the same row, their horizontal coordinates can be searched continuously.

[0083] For the pixels in the same row, the horizontal coordinate is represented as column_point[1], and the horizontal coordinates x_right_site = column_point[1]+ 1 and x_left_site = column_point[1] - 1 are searched for; where 1 represents the horizontal coordinate unit of the two-dimensional coordinate system of the target image.

[0084] If at least one of x_up_site or x_down_site does not exist in the current row, that is, the pixel on the left or right adjacent to the current pixel is not found, then the point is regarded as the second pixel, that is, the pixel at the junction of the binary layers.

[0085] Among them, the adjacent left pixel point can be represented as a pixel point located in the same row as the second pixel point and whose horizontal coordinate is one horizontal coordinate unit smaller than the horizontal coordinate of the second pixel point; the adjacent right pixel point can be represented as a pixel point located in the same row as the second pixel point and whose horizontal coordinate is one horizontal coordinate unit larger than the horizontal coordinate of the first pixel point.

[0086] After completing the horizontal coordinate search of the pixel points in each row, the second pixel points corresponding to each row can be obtained, and all the second pixel points constitute the second candidate boundary point set.

[0087] In the above manner, all horizontal and vertical boundary points of the mask of the target object can be determined.

[0088] It is understandable that some boundary points in the first candidate boundary point set and the second candidate boundary point set may overlap, and therefore, it is necessary to perform deduplication processing on the first candidate boundary points and the second candidate boundary points.

[0089] For example, various functions, such as the torch.unique function, may be used to remove duplicate boundary points, that is, to remove boundary points with duplicate coordinates in the first candidate boundary point set and the second candidate boundary point set, so that the coordinates of the remaining boundary points are not duplicated.

[0090] The large model image segmentation method provided by the present invention determines the first candidate boundary point based on the vertical coordinates of the pixel points located in the same column, and determines the second candidate boundary point based on the horizontal coordinates of the pixel points located in the same row, and then deduplicates the first candidate boundary point and the second candidate boundary point to obtain the boundary points of the target object. This can reduce the difficulty of obtaining the boundary points and improve the accuracy, thereby improving the efficiency and accuracy of image segmentation.

[0091] In one embodiment, sorting the boundary points may include:

[0092] Determine the starting point from each boundary point, and regard the remaining boundary points as unprocessed boundary points;

[0093] Repeat the following traversal operation until the current point is the same as the starting point:

[0094] Determine the neighboring points of the current point from the unprocessed boundary points;

[0095] The adjacent points are excluded from the unprocessed boundary points as the sorted boundary points, and the adjacent points are used as the current points of the next traversal operation;

[0096] In the first traversal operation, the starting point is regarded as the current point.

[0097] It should be noted that, since the present invention processes pixel by pixel to determine the boundary points, under the known condition that the target object is a two-dimensional figure in the binary segmentation map, the mask boundary of the target object determined pixel by pixel must be a closed image that is continuous or concave or convex pixel by pixel. Then, at least two of the neighboring points of each pixel of the image (i.e., the eight points located around the pixel and closest to the pixel) must also belong to the image, as proved by the following:

[0098] There is a two-dimensional plane closed image R (R is a hollow image) that is continuous pixel by pixel. Take any pixel point P on the image, that is, P∈R; the set Q consists of the eight pixel points closest to point P, and prove that: there are at least two elements P1 and P2 in Q that satisfy P1∈R and P2∈R.

[0099] prove:

[0100] R is a closed hollow image, which means that its boundary is continuous, which can be interpreted as the pixels on the boundary are connected in the image. The hollow image is not filled, so it can be assumed that its boundary forms a closed loop.

[0101] For any point P∈R, consider its nearest 8 pixels, denoted as set Q. These pixels are defined by the standard 8-neighborhood, which are at the position closest to point P, that is, pixels in the horizontal, vertical and diagonal directions.

[0102] P∈R, since R is continuous pixel by pixel, at least some of the 8 nearby pixels intersect with R. If only one pixel in the neighborhood belongs to R, this will violate the connectivity of the closed loop, that is, any point on the boundary of the closed loop is connected to other boundary points. Therefore, multiple points in the neighborhood must be on the boundary of R.

[0103] Since R is a closed boundary and any point P∈R is taken, combined with the boundary connectivity, it can be determined that there are at least two pixel points P1 and P2∈R in Q.

[0104] Based on the above understanding, the present invention can perform the sorting operation on each boundary point, specifically:

[0105] First, determine the starting point start_point from each boundary point, and put the remaining boundary points into the boundary point set remaining_array as unprocessed boundary points.

[0106] Among them, the starting point can be determined in various ways, such as randomly determining the starting point from each boundary point, taking the point with the largest / smallest horizontal coordinate from each boundary point as the starting point, taking the point with the largest / smallest vertical coordinate from each boundary point as the starting point, etc. The present invention does not make specific limitations on this.

[0107] Next, the following traversal operations S210-S230 are repeatedly performed until the current point and the starting point are the same point:

[0108] S210 , among the eight adjacent pixel points of the current point now_point (ie, left, upper left, upper, upper right, right, lower right, lower, lower left), search for a point in the remaining_array set.

[0109] Specifically, set operations can be used to quickly determine which neighboring pixel points of now_point belong to the remaining_array set, and then store these neighboring points in the near_points list.

[0110] It should be noted that when the traversal operation is performed for the first time, the starting point start_point will be used as the current point now_point.

[0111] S220 . Determine the target adjacent point of the current point now_point from the near_points list, and exclude the target adjacent point from the remaining_array set.

[0112] It should be noted that when there are multiple adjacent points in the near_points list, it is necessary to determine the target adjacent point from the multiple adjacent points according to certain screening rules.

[0113] The filter rule can be any of the following:

[0114] Randomly select one of the multiple neighboring points as the target neighboring point;

[0115] Take any of the eight adjacent pixel points of the current point (i.e. left, upper left, upper, upper right, right, lower right, lower, lower left) as the starting point, and determine the target adjacent point according to the order in which the adjacent pixel points are arranged in a clockwise direction.

[0116] Take any one of the eight adjacent pixel points of the current point as the starting point, and determine the target adjacent point according to the order in which the adjacent pixel points are arranged in a counterclockwise direction.

[0117] Of course, the screening rules can also be set according to actual conditions, and the present invention does not make specific limitations on this.

[0118] S230: Determine whether the target adjacent point is the starting point start_point.

[0119] If the target adjacent point is not the starting point start_point, the target adjacent point is put into the sorted boundary point set result, and the target adjacent point is used as the current point now_point of the next traversal operation to execute step S210 again.

[0120] If the target adjacent point is the starting point start_point, it means that a complete closed path of the mask of the target object has been found, and the traversal operation is terminated.

[0121] When the traversal operation is completed, the sorted boundary points can be normalized (i.e., the horizontal and vertical coordinates are divided by the image size), and the results are saved in the specified file. In this way, you only need to perform sequential connection according to the boundary points in the file to obtain the complete outline of the mask of the target object.

[0122] The above overall process can be Figure 2 shown.

[0123] The large model image segmentation method provided by the present invention can obtain an accurate mask outline of the target object by sorting the boundary points by traversing adjacent points and then connecting the sorted boundary points in sequence, thereby improving the accuracy of image segmentation.

[0124] In one example, the large model image segmentation method provided by the present invention may further include:

[0125] If no adjacent point of the current point is found in the unprocessed boundary points, the current traversal operation is rolled back to determine another adjacent point of the current point from the unprocessed boundary points in the previous traversal operation.

[0126] If in a certain traversal operation, the target adjacent point in the remaining_array set is not found in the 8 neighborhoods of the current point now_point, it means that the traversal loop may be stuck in a dead end in the local area. In this case, by rolling back the traversal operation, that is, replacing the target adjacent point determined in the previous traversal operation with another target adjacent point from the remaining_array set, the traversal loop can be effectively prevented from being stuck. This ensures the smooth progress of the image segmentation process.

[0127] In one embodiment, the large model image segmentation method provided by the present invention may further include:

[0128] After the traversal operation is completed, the number of sorted boundary points is determined;

[0129] If the number of sorted boundary points is greater than or equal to the number threshold, the sorted boundary points are connected in sequence;

[0130] If the number of sorted boundary points is less than the number threshold, the traversal operation is re-executed.

[0131] It should be understood that after the sorted boundary points are determined by the traversal operation, if the number of sorted boundary points is less than the number threshold, it means that the sorted boundary points do not correspond to the specific mask of the target object, so the traversal operation needs to be re-executed. At this time, the starting point start_point can be re-determined and / or the screening rules of adjacent points can be modified to obtain a different sorted boundary point set result by re-executing the traversal operation.

[0132] If the number of sorted boundary points is less than or equal to the number threshold, it means that the sorted boundary points correspond to the specific mask of the target object. At this time, the sorted boundary points can be connected in sequence to obtain the mask contour of the target object that meets the accuracy requirements.

[0133] The large model image segmentation method provided by the present invention can ensure the accuracy of the mask information of the target object by judging the number of sorted boundary points obtained after the traversal operation is completed, and determining whether to re-execute the traversal operation or connect the sorted boundary points in sequence according to the judgment result.

[0134] In a specific example of applying the large model image segmentation method provided by the present invention, a detection data set of 300 cultivated land pictures was constructed. After consuming no more than half an hour, without manual operation, a laptop equipped with a GTX4060 graphics card can segment weeds and crops for 3,000 cultivated land pictures within two days and correctly label them. Using these automatically annotated pictures and annotation files to construct a data set, and using YOLOv8 to train a segmentation model, an accuracy rate of more than 80% can be obtained. It can be seen that the large model image segmentation method provided by the present invention has a high image segmentation accuracy.

[0135] The image understanding device provided by the present invention is described below. The image understanding device described below and the image understanding method described above can refer to each other and can achieve the same technical effects, which will not be repeated here.

[0136] Figure 3 Schematic diagram of the structure of the large model image segmentation device provided by the present invention. Figure 3 As shown, the device may include:

[0137] The generating module 310 is used to input the preprocessed target image into the image segmentation model to obtain a binary segmentation map generated by the image segmentation model; the preprocessed target image includes the annotation information of the target area where the target object is located; the binary segmentation map is matched with the target area;

[0138] A determination module 320, configured to determine a boundary point of the target object in the binary segmentation map based on a target pixel point with a value of 1 in the binary segmentation map and a coordinate of the target pixel point in the target image;

[0139] The sorting module 330 is used to sort the boundary points and connect the sorted boundary points in sequence to obtain the mask information of the target object.

[0140] In one embodiment, the determination module 320 may be specifically configured to:

[0141] Determine the pixel points located in the same column based on the horizontal coordinate of each target pixel point in the target image;

[0142] Determine a first candidate boundary point based on the vertical coordinates of the pixel points located in the same column in the target image;

[0143] Determine the pixel points located in the same row based on the ordinate of each target pixel point in the target image;

[0144] Determine a second candidate boundary point based on the horizontal coordinates of the pixel points located in the same row in the target image;

[0145] The first candidate boundary points and the second candidate boundary points are deduplicated, and the deduplicated first candidate boundary points and the second candidate boundary points are used as boundary points of the target object in the binary segmentation map.

[0146] In one embodiment, the determination module 320 may be specifically configured to:

[0147] Determine a first pixel point from the pixel points located in the same column, and use the first pixel point as the first candidate boundary point;

[0148] The first pixel satisfies any of the following conditions:

[0149] There is no pixel point located in the same column and having a ordinate greater than the ordinate of the first pixel point by one ordinate unit;

[0150] There is no pixel point located in the same column and having a ordinate smaller than the ordinate of the first pixel point by one ordinate unit.

[0151] In one embodiment, the sorting module 330 may be specifically used to:

[0152] Determine the starting point from each boundary point, and regard the remaining boundary points as unprocessed boundary points;

[0153] Repeat the following traversal operation until the current point is the same as the starting point:

[0154] Determine a target adjacent point of the current point from the unprocessed boundary points;

[0155] Excluding the target adjacent point from the unprocessed boundary points to serve as sorted boundary points, and using the target adjacent point as the current point of the next traversal operation;

[0156] In the first traversal operation, the starting point is regarded as the current point.

[0157] In one embodiment, the sorting module 330 may also be used to:

[0158] After the traversal operation is completed, determining the number of sorted boundary points;

[0159] If the number of sorted boundary points is greater than or equal to the number threshold, the sorted boundary points are connected in sequence;

[0160] If the number of sorted boundary points is less than the number threshold, the traversal operation is re-executed.

[0161] In one embodiment, the sorting module 330 may also be used to:

[0162] If the target adjacent point of the current point is not found in the unprocessed boundary points, the current traversal operation is rolled back to determine another target adjacent point of the current point from the unprocessed boundary points in the last traversal operation.

[0163] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the large model image segmentation method provided in any of the above embodiments, for example, including:

[0164] Inputting the preprocessed target image into the image segmentation model to obtain a binary segmentation map generated by the image segmentation model; the preprocessed target image includes the annotation information of the target area where the target object is located; the binary segmentation map is matched with the target area;

[0165] Determine the boundary points of the target object in the binary segmentation map based on the target pixel points whose values ​​are 1 in the binary segmentation map and the coordinates of the target pixel points in the target image;

[0166] The boundary points are sorted and the sorted boundary points are connected in sequence to obtain the mask information of the target object.

[0167] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0168] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the large model image segmentation method provided in any of the above embodiments, for example, including:

[0169] Inputting the preprocessed target image into the image segmentation model to obtain a binary segmentation map generated by the image segmentation model; the preprocessed target image includes the annotation information of the target area where the target object is located; the binary segmentation map is matched with the target area;

[0170] Determine the boundary points of the target object in the binary segmentation map based on the target pixel points whose values ​​are 1 in the binary segmentation map and the coordinates of the target pixel points in the target image;

[0171] The boundary points are sorted and the sorted boundary points are connected in sequence to obtain the mask information of the target object.

[0172] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the large model image segmentation method provided in any of the above embodiments, for example, including:

[0173] Inputting the preprocessed target image into the image segmentation model to obtain a binary segmentation map generated by the image segmentation model; the preprocessed target image includes the annotation information of the target area where the target object is located; the binary segmentation map is matched with the target area;

[0174] Determine the boundary points of the target object in the binary segmentation map based on the target pixel points whose values ​​are 1 in the binary segmentation map and the coordinates of the target pixel points in the target image;

[0175] The boundary points are sorted and the sorted boundary points are connected in sequence to obtain the mask information of the target object.

[0176] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0177] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large model image segmentation method, characterized in that: include: Inputting the preprocessed target image into the image segmentation model to obtain a binary segmentation map generated by the image segmentation model; the preprocessed target image includes the annotation information of the target area where the target object is located; the binary segmentation map is matched with the target area; Determine the boundary points of the target object in the binary segmentation map based on the target pixel points whose values ​​are 1 in the binary segmentation map and the coordinates of the target pixel points in the target image; Sorting each boundary point and sequentially connecting the sorted boundary points to obtain mask information of the target object; The step of determining the boundary points of the target object in the binary segmentation map based on the target pixel points whose values ​​are 1 in the binary segmentation map and the coordinates of the target pixel points in the target image comprises: Determine the pixel points located in the same column based on the horizontal coordinate of each target pixel point in the target image; Determine a first candidate boundary point based on the vertical coordinates of the pixel points located in the same column in the target image; Determine the pixel points located in the same row based on the ordinate of each target pixel point in the target image; Determine a second candidate boundary point based on the horizontal coordinates of the pixel points located in the same row in the target image; Deduplication of the first candidate boundary points and the second candidate boundary points is performed, and the deduplication of the first candidate boundary points and the second candidate boundary points is used as boundary points of the target object in the binary segmentation map; The determining of the first candidate boundary point based on the vertical coordinates of the pixel points located in the same column in the target image includes: Determine a first pixel point from the pixel points located in the same column, and use the first pixel point as the first candidate boundary point; The first pixel satisfies any of the following conditions: There is no pixel point located in the same column and having a ordinate greater than the ordinate of the first pixel point by one ordinate unit; There is no pixel point located in the same column and having a ordinate smaller than the ordinate of the first pixel point by one ordinate unit.

2. The large model image segmentation method according to claim 1, characterized in that: The step of sorting the boundary points comprises: Determine the starting point from each boundary point, and regard the remaining boundary points as unprocessed boundary points; Repeat the following traversal operation until the current point is the same as the starting point: Determine a target adjacent point of the current point from the unprocessed boundary points; Excluding the target adjacent point from the unprocessed boundary points to serve as sorted boundary points, and using the target adjacent point as the current point of the next traversal operation; In the first traversal operation, the starting point is regarded as the current point.

3. The large model image segmentation method according to claim 2, characterized in that: The method further comprises: After the traversal operation is completed, determining the number of sorted boundary points; If the number of sorted boundary points is greater than or equal to the number threshold, the sorted boundary points are connected in sequence; If the number of sorted boundary points is less than the number threshold, the traversal operation is re-executed.

4. The large model image segmentation method according to claim 2, characterized in that: The method further comprises: If the target adjacent point of the current point is not found in the unprocessed boundary points, the current traversal operation is rolled back to determine another target adjacent point of the current point from the unprocessed boundary points in the last traversal operation.

5. A large model image segmentation device, characterized in that: include: A generation module, used for inputting the preprocessed target image into the image segmentation model to obtain a binary segmentation map generated by the image segmentation model; the preprocessed target image includes the annotation information of the target area where the target object is located; the binary segmentation map is matched with the target area; A determination module, configured to determine a boundary point of the target object in the binary segmentation map based on a target pixel point whose value is 1 in the binary segmentation map and a coordinate of the target pixel point in the target image; A sorting module, used to sort the boundary points and connect the sorted boundary points in sequence to obtain the mask information of the target object; The determination module is specifically used for: Determine the pixel points located in the same column based on the horizontal coordinate of each target pixel point in the target image; Determine a first candidate boundary point based on the vertical coordinates of the pixel points located in the same column in the target image; Determine the pixel points located in the same row based on the ordinate of each target pixel point in the target image; Determine a second candidate boundary point based on the horizontal coordinates of the pixel points located in the same row in the target image; Deduplication of the first candidate boundary points and the second candidate boundary points is performed, and the deduplication of the first candidate boundary points and the second candidate boundary points is used as boundary points of the target object in the binary segmentation map; The determination module is also specifically used for: Determine a first pixel point from the pixel points located in the same column, and use the first pixel point as the first candidate boundary point; The first pixel satisfies any of the following conditions: There is no pixel point located in the same column and having a ordinate greater than the ordinate of the first pixel point by one ordinate unit; There is no pixel point located in the same column and having a ordinate smaller than the ordinate of the first pixel point by one ordinate unit.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the large model image segmentation method as described in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large model image segmentation method according to any one of claims 1 to 4 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the large model image segmentation method according to any one of claims 1 to 4 is implemented.

Images