Image segmentation method, image segmentation apparatus, electronic device and computer-readable storage medium

Abstract

To provide an image division method, an apparatus, a device, and a storage medium which improve accuracy of image division.SOLUTION: A method: acquires an original image; determines a division optimal path, on the basis of change of gradient corresponding to each pixel point in the original image, of a gradient vector, based on an initial divided image obtained by coarse division of the original image; and finely divides the original image on the basis of the determined division optimal path.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the field of image processing technology, and in particular to an image segmentation method, apparatus, device and storage medium. [Background technology]

[0002] With the development of artificial intelligence technology, image segmentation has gradually attracted widespread attention and become an important field of image processing. Image segmentation is an operation to divide an image into multiple non-overlapping regions according to certain characteristics and extract a target region of interest from among them.

[0003] Currently, commonly used segmentation methods suffer from over-segmentation and under-segmentation, which results in low image segmentation effectiveness and makes it impossible to directly apply the image segmentation results to subsequent tasks such as image recognition.

[0004] Therefore, how to improve the accuracy of image segmentation has become an urgent issue. Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention provides an image segmentation method, apparatus, device and storage medium for solving the above problems of the prior art. [Means for solving the problem]

[0006] The technical solutions used in the present invention are as follows:

[0007] The present invention provides a method for image segmentation, the method comprising the steps of: obtaining an original image; pre-segmenting the original image to obtain an initial segmented image including a plurality of image regions; determining a gradient vector field corresponding to each pixel point in the original image based on a gradient vector corresponding to each pixel point in the original image, where for each pixel point, the gradient vector field may indicate a change in gradient of pixel values ​​between the pixel point and a designated pixel point corresponding to the pixel point; identifying, for each of the image regions, a number of landmark points based on pixel points on an initial boundary of the image region; for each target point identified based on each of the image regions, determining a split optimal path starting from the target point based on the gradient vector field; Segmenting the original image based on the determined optimal segmentation path.

[0008] Optionally, the step of identifying a plurality of target points based on pixel points on the initial boundary of the image region specifically comprises: determining an initial set of pixel points based on each pixel point on an initial boundary of the image region; For each pixel point included in the initial pixel point set, determining a pixel point corresponding to the pixel point from the original image as a candidate target point; determining point features for the candidate landmark points, the point features including at least one of a Harris corner feature, a Sift feature, a Brisk feature, a slope, and a curvature; and identifying a plurality of target points based on point features of the plurality of candidate target points corresponding to the initial pixel point set.

[0009] Optionally, for each target point identified based on each of said image regions, determining a splitting optimal path starting from said target point based on said gradient vector field specifically comprises: determining a value corresponding to each pixel point in a matrix corresponding to the initial segmented image based on a reference pixel point and a supplementary pixel point included in the initial segmented image; determining a gradient segmented image based on a matrix corresponding to the initial segmented image and the gradient vector field; and for each target point, determining a split optimal path starting from the target point based on the gradient split image; the reference pixel points are pixel points on the initial boundary of each of the image regions; The supplementary pixel point is a pixel point in the initial segmented image that is in the vicinity of the reference pixel point.

[0010] Optionally, for each target point identified based on each of said image regions, determining a splitting optimal path starting from said target point based on said gradient vector field specifically comprises: sorting each of the target points based on its location in the initial segmented image; for each sorted target point, defining the target point and a target point next to the target point as a target point set, defining the target point as a start point of the target point set, and defining the target point next to the target point as an end point of the target point set; For each set of target points, determining a split optimal path from a start point of the set of target points to an end point of the set of target points as a split optimal path corresponding to the set of target points based on the gradient vector field.

[0011] Optionally, the step of determining a split optimal path from a start point of the set of target points to an end point of the set of target points based on the gradient vector field specifically comprises: setting the starting point as a parent path point; determining an optimal path point from a plurality of candidate path points based on the parent path point and the gradient vector field, the plurality of candidate path points being neighboring pixel points of the parent path point; If the optimal path point is not the end point, the optimal path point is set as a new parent path point, and the step of determining the optimal path point is repeated; If the optimal path point is the end point, determining a split optimal path from the start point of the set of target points to the end point of the set of target points based on all the determined optimal path points.

[0012] Optionally, the step of determining an optimal path point from a plurality of candidate path points based on the parent path points and the gradient vector field specifically comprises: determining a loss value corresponding to a path from the parent path point to each of the candidate path points based on the gradient vector field as a loss value corresponding to the candidate path point; and determining the candidate route point for which the loss value is smallest as the optimal route point.

[0013] Optionally, the step of determining an optimal path point from a plurality of candidate path points based on the parent path points and the gradient vector field specifically comprises: determining a loss value corresponding to each of the candidate path points based on a distance from each of the candidate path points to the end point and the gradient vector field; and determining the candidate route point for which the loss value is smallest as the optimal route point.

[0014] The present invention further provides an image segmentation apparatus, comprising: an acquisition module for acquiring an original image; a pre-segmentation module for pre-segmenting the original image to obtain an initial segmented image including a plurality of image regions; a determination module for determining a gradient vector field corresponding to each pixel point in the original image based on a gradient vector corresponding to each pixel point in the original image, where for each pixel point, the gradient vector field can indicate a gradient change in pixel value between the pixel point and a designated pixel point corresponding to the pixel point; an identification module for identifying, for each of the image regions, a number of landmark points based on respective pixel points on an initial boundary of the image region; a path optimization module for determining, for each target point identified based on each of the image regions, a split optimal path originating from the target point based on the gradient vector field; a segmentation module for segmenting the original image based on the determined segmentation optimal path.

[0015] Optionally, the particular module specifically: determining an initial set of pixel points based on each pixel point on an initial boundary of the image region; For each pixel point included in the initial pixel point set, determining a pixel point corresponding to the pixel point from the original image as a candidate target point; determining point features for the candidate landmark points, the point features including at least one of a Harris corner feature, a Sift feature, a Brisk feature, a slope, and a curvature; A plurality of target points are identified based on point features of the plurality of candidate target points corresponding to the initial pixel point set.

[0016] Optionally, the route optimization module specifically: determining values ​​corresponding to each pixel point in a matrix corresponding to the initial segmented image based on the reference pixel points and the supplementary pixel points included in the initial segmented image; determining a gradient segmented image based on a matrix corresponding to the initial segmented image and the gradient vector field; for each target point, a segmented optimal path is determined starting from the target point based on the gradient segmentation image; the reference pixel points are pixel points on the initial boundary of each of the image regions; The supplementary pixel point is a pixel point in the initial segmented image that is in the vicinity of the reference pixel point.

[0017] Optionally, the route optimization module specifically: sorting each of the target points based on its position in the initial segmented image; For each sorted target point, the target point and the next target point of the target point are set as a target point set, the target point is set as a start point of the target point set, and the next target point of the target point is set as an end point of the target point set; For each set of goal points, a split optimal path from the start of the goal set to the end of the goal set is used to determine a split optimal path corresponding to the goal set based on the gradient vector field.

[0018] Optionally, the route optimization module specifically: The starting point is a parent path point, determining an optimal path point from a plurality of candidate path points based on the parent path point and the gradient vector field, the plurality of candidate path points being neighboring pixel points of the parent path point; If the optimal path point is not the end point, the optimal path point is set as a new parent path point, and the step of determining the optimal path point is repeated; If the optimal path point is the end point, it is used to determine a split optimal path from the start point of the set of destination points to the end point of the set of destination points based on all the determined optimal path points.

[0019] Optionally, the route optimization module specifically: determining a loss value corresponding to a path from the parent path point to each of the candidate path points based on the gradient vector field as a loss value corresponding to the candidate path point; The candidate path point for which the loss value is smallest is used to determine the optimal path point.

[0020] Optionally, the route optimization module specifically: determining a loss value corresponding to each of the candidate path points based on a distance from each of the candidate path points to the end point and the gradient vector field; The candidate path point for which the loss value is smallest is used to determine the optimal path point.

[0021] The present invention further provides a computer-readable storage medium, the computer-readable storage medium storing a computer program, which, when executed by a processor, performs the above image segmentation method.

[0022] The present invention further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the electronic device implementing the image segmentation method described above when the processor executes the computer program. Effect of the Invention

[0023] At least one of the above technical solutions used in the present invention can achieve the following beneficial effects:

[0024] The image segmentation method provided by the present invention includes first obtaining an original image, pre-segmenting the original image to obtain an initial segmented image including a plurality of image regions, determining a gradient vector field corresponding to the original image based on a gradient vector corresponding to each pixel point in the original image, where for each pixel point, the gradient vector field can indicate a change in gradient of pixel values ​​between the pixel point and a designated pixel point corresponding to the pixel point, identifying, for each of the image regions, a plurality of target points based on each pixel point on an initial boundary of the image region, determining, for each target point identified based on each of the image regions, a segmentation optimal path starting from the target point based on the gradient vector field, and segmenting the original image based on the determined segmentation optimal path.

[0025] As can be seen from the above method, based on the initial segmented image obtained by coarse segmentation, the original image can be finely segmented by path optimization based on the change in gradient of the gradient vector corresponding to each pixel point in the original image, thereby improving the accuracy of image segmentation. [Brief description of the drawings]

[0026] The accompanying drawings described herein are used to enhance understanding of the present invention and constitute a part of the present invention, and the exemplary embodiments of the present invention and their description are used to explain the present invention and do not constitute undue limitations of the present invention. [Figure 1] 1 is a schematic diagram showing the flow of an image segmentation method provided in the present invention. [Diagram 2] FIG. 2 is a schematic diagram showing the process of the histogram equalization method provided in the present invention. [Diagram 3] FIG. 2 is a schematic diagram showing the process of segmenting an original image provided in the present invention; [Figure 4] 1 is a schematic diagram showing an image segmentation device provided in the present invention; [Diagram 5] 1 is a schematic diagram showing an electronic device provided by the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described below clearly and completely in conjunction with specific embodiments of the present invention and corresponding accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of them. All other embodiments that can be obtained by those skilled in the art based on the embodiments of the present invention without creative efforts belong to the protection scope of the present invention.

[0028] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Below, the technical solutions provided in each embodiment of the present invention will be described in detail in conjunction with the accompanying drawings.

[0029] FIG. 1 is a schematic diagram showing the flow of an image segmentation method provided in the present invention, which may include steps S101 to S106.

[0030] In S101, an original image is acquired.

[0031] In S102, the original image is pre-segmented to obtain an initial segmented image including a plurality of image regions.

[0032] In the present invention, the executing entity for executing the image segmentation method may be a fixed device such as a server, a computer, a mobile phone, etc., or an unmanned device such as an unmanned vehicle, a drone, etc. For convenience of explanation, the image segmentation method provided by the present invention will be described below using only a server as an example of the executing entity.

[0033] In the present invention, the server can finely divide the original image based on the initial segmented image. Before that, the server may obtain the original image and pre-segment (i.e., coarsely segment) the obtained original image to obtain the initial segmented image, where the initial segmented image has the same size as the original image.

[0034] Here, the initial segmented image is segmented using the thresholding method. (A segmentation method that sets an appropriate threshold and divides the image into different regions for pixels above the threshold and pixels below the threshold) , Edge detection segmentation method, Region growing Partitioning (a method of partitioning that starts with an extension point (which can be a pixel or a region) and merges nearby pixels or regions that have similar characteristics or properties to this extension point to form a new extension point, and then extends it until it can no longer be extended); Split and Merge Method (A segmentation method that starts with the whole image, continues to segment the image to obtain multiple sub-regions, and merges the sub-regions that belong to the foreground to obtain the foreground.) ,Watershed Algorithm Based on Partitioning method, Graph Cut Based on Segmentation method, segmentation method based on active contour model, deep learning segmentation method (A segmentation method that gradually learns feature representations from the pixel level to the region level to achieve pixel-level classification) Alternatively, the image may be a roughly divided image obtained by a division method such as the above.

[0035] In addition, in practical application scenarios, the contrast of the original image acquired by the server is low, so that the contour boundary of the original image is not clear, so the server may perform image enhancement on the acquired original image to obtain an enhanced original image.

[0036] Here, the method by which the server performs image enhancement on the original image may be histogram equalization. Specifically, the server may obtain a pixel value distribution corresponding to each pixel point of the original image, and equalize the pixel value distribution corresponding to each pixel point of the original image using a predetermined histogram equalization algorithm, as shown in FIG. 2.

[0037] FIG. 2 is a schematic diagram showing the process of the histogram equalization method provided in the present invention.

[0038] In FIG. 2, the histogram on the left shows the pixel value (i.e., 0-255) distribution corresponding to each pixel point of the original image, that is, the number of pixel points corresponding to each pixel value among each pixel point included in the original image. For example, if the original image contains 5000 pixel points with a pixel value of 120, the histogram on the left shows that the pixel values ​​of the pixel points included in the original image are mostly between 100 and 180, so the contrast of the original image is not high, that is, the difference between each image area of ​​the original image is not clear. The histogram on the right of FIG. 2 shows the pixel value distribution corresponding to each pixel point of the enhanced original image, and it can be seen from the histogram on the right that the server can adjust the pixel values ​​corresponding to at least some pixel points of the original image by the histogram equalization algorithm, so that the pixel values ​​corresponding to the pixel points of the original image are distributed as evenly as possible between 0 and 255, thereby enhancing the contrast and clarity of the image.

[0039] In S103, a gradient vector field corresponding to each pixel point in the original image is determined based on a gradient vector corresponding to each pixel point in the original image, and for each pixel point, the gradient vector field can indicate a gradient change in pixel value between the pixel point and a designated pixel point corresponding to the pixel point.

[0040] Furthermore, the server may determine a gradient vector field corresponding to the original image based on the gradient vector corresponding to each pixel point in the original image, where for each pixel point, the gradient vector field may indicate a gradient change in pixel value between the pixel point and a designated pixel point corresponding to the pixel point, where the designated pixel point may be an adjacent pixel point adjacent to the pixel point.

[0041] Note that in the original image, the closer a pixel point is to a region boundary, the greater the change in gradient of the corresponding gradient vector (since the pixel value of a pixel point on a region boundary may be completely different from the pixel values ​​of adjacent pixel points, the greater the change in gradient of the gradient vector corresponding to that pixel point on the region boundary), and the change in gradient of the gradient vector corresponding to a pixel point in a smooth portion within the region of the image is closer to 0. The change in gradient here may be determined based on the amplitude value of the gradient vector, and the greater the amplitude value corresponding to the gradient vector of a pixel point, the greater the change in gradient corresponding to the gradient vector of this pixel point.

[0042] Here, the server may determine the gradient vector field with the minimized energy function. Specifically, the following formula may be referred to:

number

[0043] In the above formula,

number

number

[0044] Therefore, the above minimization energy function may be transformed into the following Euler equation:

number

[0045] In the above equation, ∇ 2 is the Laplace operator, which transforms the above Euler equation into By law Iteration target The gradient vector field of the original image can be calculated by

[0046] At S104, for each of the image regions, a number of landmark points are identified based on pixel points on an initial boundary of the image region.

[0047] In the present invention, the server may determine an initial pixel point set for each image region based on each pixel point on the initial boundary of the image region. Next, for each pixel point included in the initial pixel point set, a pixel point corresponding to the pixel point is determined as a candidate target point from the original image, point features of the candidate target point are determined, and multiple target points can be identified based on the point features of the multiple candidate target points corresponding to the initial pixel point set. The point features here may include at least one of Harris corner features, Sift features, Brisk features, slope, and curvature.

[0048] Here, the method for the server to identify the multiple target points based on the point features of the multiple candidate target points corresponding to the initial pixel point set may be to determine, for each candidate target point, whether or not each point feature of the candidate target point satisfies a predetermined condition value. Here, the point feature satisfying the predetermined condition value may be, for example, that the inclination satisfies a predetermined threshold value, or that the curvature satisfies a predetermined threshold value.

[0049] The initial divided image may be a binarized image, that is, the initial divided image is in the form of a matrix. In the matrix corresponding to the initial divided image, the value corresponding to the pixel point belonging to the target area is 1, and the value corresponding to the pixel point not belonging to the target area is 0. Therefore, in the above, when identifying the target point, the coordinate value corresponding to each pixel point on the initial boundary of each image area in the initial divided image is determined by a specified algorithm, so that it is possible to identify the pixel point corresponding to the coordinate value corresponding to the pixel point on the initial boundary of each image area in the initial divided image from the original image. The specified algorithm here is OpenCV's findContours function may be also possible.

[0050] At S105, for each target point identified based on each of the image regions, a split optimal path starting from the target point is determined based on the gradient vector field.

[0051] Before the server determines, for each target point identified based on each image region, an optimal division path starting from the target point based on the gradient vector field, the server may further set pixel points on the initial boundary of each image region included in the initial division image as reference pixel points, and for each reference pixel point, set pixel points in the initial division image that are near the reference pixel point as supplementary pixel points corresponding to the reference pixel point.

[0052] Furthermore, the server may determine values ​​corresponding to each pixel point in a matrix corresponding to the initial segmented image based on reference pixel points and supplementary pixel points included in the initial segmented image, and determine a gradient segmentation image based on the matrix corresponding to the initial segmented image and a gradient vector field, and then, for each target point, determine an optimal segmentation path starting from the target point based on the gradient segmentation image.

[0053] In the present invention, the neighboring pixel points of a certain pixel point may be pixel points located within a designated area of ​​the certain pixel point. The designated area here may be 8 neighbors, 16 neighbors, or the like.

[0054] The above eight neighboring pixel points may be eight pixel points, which are the pixel point adjacent to the left, the pixel point adjacent to the right, the pixel point adjacent below, the pixel point adjacent above, the pixel point adjacent to the upper left, the pixel point adjacent to the upper right, the pixel point adjacent to the lower left, and the pixel point adjacent to the lower right of the pixel point.

[0055] Furthermore, the server may adjust values ​​corresponding to each pixel point in the matrix corresponding to the initial segmented image based on the reference pixel point and the supplementary pixel point. For example, the server may adjust values ​​corresponding to the reference pixel point and the supplementary pixel point to 1, and adjust values ​​corresponding to other pixel points to 255.

[0056] Further, the server may determine a gradient segmentation image based on a matrix corresponding to the initial segmentation image and the gradient vector field, and for each target point, determine a segmentation optimal path starting from the target point based on the gradient segmentation image.

[0057] Here, the gradient divided image is obtained by multiplying, for each pixel point in the initial divided image, the value corresponding to that pixel point in the matrix corresponding to the initial divided image by the amplitude value corresponding to that pixel point in the gradient vector field.

[0058] In the above, the reason why the gradient division image is determined is that when the division optimum path is subsequently determined, the division optimum path can be determined by calculating the loss value corresponding to each pixel point. At this time, since the values ​​of the reference pixel point and the supplementary pixel point in the matrix corresponding to the initial division image are adjusted to 1, when multiplied, the values ​​of the reference pixel point and the supplementary pixel point in the gradient division image become the amplitude values ​​corresponding to the reference pixel point and the supplementary pixel point in the gradient vector field. Since the values ​​of other pixel points in the matrix corresponding to the initial division image are adjusted to 255, at this time, the values ​​of other pixel points in the gradient division image become 255 times the amplitude values ​​corresponding to other pixel points in the gradient vector field. When the loss value is subsequently calculated, the larger the value corresponding to each pixel point in the gradient division image, the larger the loss, and as a result, the above other pixel points are excluded from the division optimum path.

[0059] Furthermore, after the server determines the gradient segmentation image, the target points may be sorted based on the position of each target point in the initial segmentation image. The sorting method here may be set according to actual needs, for example, sorting in the clockwise direction of the initial boundary of each image region in the initial segmentation image. If the image region is a person, the target points may be sorted in the following order: from the target point belonging to the boundary of the person's head region, to the pixel point corresponding to the boundary of the person's left face region, to the pixel point corresponding to the boundary of the person's left neck region, to the pixel point corresponding to the boundary of the person's left shoulder region, and so on, until returning to the boundary of the person's head region.

[0060] In addition, after the server sorts each target point, for each sorted target point, the target point and its next target point may be a target point set, where the target point is a start point of the target point set, and the next target point is an end point of the target point set. For each target point set, a split optimal path from the start point of the target point set to the end point of the target point set is determined as the split optimal path corresponding to the target point set based on the gradient vector field.

[0061] For example, if there are five target points, A, B, C, D, and E, five target point sets are obtained: (A, B), (B, C), (C, D), (D, E), and (E, A). Taking the first target point set (A, B) as an example, target point A is the start point of the target point set, and target point B is the end point of the target point set. Route Query Used for The algorithm may determine the splitting optimal path from target point A to target point B as the splitting optimal path corresponding to the target point set, and then the original image can be split based on the splitting optimal path corresponding to each target point set.

[0062] Furthermore, for each set of goal points, the server may determine a split optimal path from the start point of the set of goal points to the end point of the set of goal points based on the gradient vector field as the split optimal path corresponding to the set of goal points.

[0063] Here, for each target point, the server may determine a split optimal route from the target point (i.e., the starting point) to the next target point (i.e., the end point) by multiple route queries.

[0064] Specifically, the server may determine an optimal route point from a plurality of candidate route points based on the parent route point and the gradient vector field, the plurality of candidate route points being neighboring pixel points of the parent route point, and if the optimal route point is not the end point, the server may determine the optimal route point as a new parent route point and repeat the step of determining the optimal route point, and if the optimal route point is the end point, may determine a split optimal route from the starting point of the target point set to the end point of the target point set based on all the determined optimal route points.

[0065] For each route query, the server determines a parent route point of the route query in that round and each candidate route point of the route query in that round, determines whether each candidate route point includes an end point, and if not, determines a loss value corresponding to the route from the parent route point to each candidate route point based on the gradient division image as the loss value corresponding to the candidate route point, and may determine the candidate route point with the smallest corresponding loss value as the optimal route point determined for the route query in that round.

[0066] In each route query, the server may add a neighboring pixel point of a parent route point of the route query to a candidate route point set, and set each candidate route point included in the obtained candidate route point set as each candidate route point of the route query in that round. Furthermore, after an optimal route point is determined in each route query, the server may delete the optimal route point determined in the route query in that round from the route candidate point set.

[0067] In the above, for each candidate path point, the server determines a loss value corresponding to the candidate path point by at least one of a value corresponding to the candidate path point in a gradient segmentation image and a distance from the candidate path point to the end point.

[0068] Preferably, the server may determine a loss value corresponding to the candidate path point based on both the value corresponding to the candidate path point in the gradient division image and the distance from the candidate path point to the end point. Specifically, the following formula may be referred to:

number

[0069] In the above formula, cf(i) is the loss value corresponding to the candidate path point, and cf v (i) is the value corresponding to the candidate path point in the gradient division image, and cf m (i) is the distance from the candidate route point to the end point, and w is the weight.

[0070] Here, the distance from the candidate route point to the end point may be the distance required to move the candidate route point to the end point in the horizontal and vertical directions, for example, the moving distance per pixel point in the horizontal and vertical directions may be 1, and the moving distance in the diagonal direction may be 1.4.

[0071] Furthermore, if the server determines that the end point is included in each candidate route point in the route query, it may determine a split optimal route from the starting point to the end point as the split optimal route corresponding to the target point set based on the starting point, the end point, and the optimal route point determined in each route query.

[0072] In S106, the original image is divided based on the determined optimum division path.

[0073] In the present invention, the server may determine a splitting curve based on the splitting optimal path corresponding to each target point set, and then can split the original image based on the determined splitting curve.

[0074] Here, the server may smooth the split optimal paths corresponding to each obtained set of target points to obtain a closed curve, which can then be the split curve determined to split the image.

[0075] The above image segmentation may be a segmentation of the original image based on a segmentation curve, or a further fine segmentation of the initial segmented image based on a segmentation region.

[0076] To explain the above in more detail, the flow of dividing an original image by the above image dividing method will be described below in conjunction with FIG.

[0077] FIG. 3 is a schematic diagram showing the process of segmenting an original image provided in the present invention.

[0078] In FIG. 3 , after the server obtains an original image and an initial segmented image corresponding to the original image, it may enhance the original image to obtain an enhanced original image, determine a gradient vector field corresponding to the enhanced original image, and determine reference pixel points and supplementary pixel points contained in the initial segmented image from the initial segmented image.

[0079] Further, the server may identify multiple target points from pixel points included in the enhanced original image and the initial segmented image, and then determine an optimized gradient vector field based on the determined gradient vector field corresponding to the enhanced original image and the reference pixel points and supplementary pixel points included in the initial segmented image identified from the initial segmented image, and then determine a segmentation optimal path between every two adjacent target points, thereby segmenting the image based on each determined segmentation optimal path.

[0080] As can be seen from the above, the server can finely divide the original image by route optimization based on the initial segmented image obtained by coarse segmentation and based on the change in gradient of the gradient vector corresponding to each pixel point in the original image, thereby improving the accuracy of image segmentation.

[0081] The above is the image segmentation method provided in one or more embodiments of the present invention. Based on the same idea, the present invention further provides a corresponding image segmentation apparatus, as shown in FIG.

[0082] FIG. 4 is a schematic diagram of an image segmentation device provided by the present invention, the device comprising: an acquisition module 401 for acquiring an original image; a pre-segmentation module 402 for pre-segmenting the original image to obtain an initial segmented image including a plurality of image regions; A determination module 403 for determining a gradient vector field corresponding to each pixel point in the original image based on a gradient vector corresponding to each pixel point in the original image, where for each pixel point, the gradient vector field can indicate a gradient change in pixel value between the pixel point and a designated pixel point corresponding to the pixel point; an identification module 404 for identifying, for each said image region, a number of landmark points based on respective pixel points on an initial boundary of said image region; a path optimization module 405 for determining, for each target point identified based on each of the image regions, a split optimal path starting from the target point based on the gradient vector field; a segmentation module 406 for segmenting the original image based on the determined segmentation optimal path.

[0083] Optionally, the identification module 404 may specifically: determining an initial set of pixel points based on each pixel point on an initial boundary of the image region; For each pixel point included in the initial pixel point set, determining a pixel point corresponding to the pixel point from the original image as a candidate target point; determining point features for the candidate landmark points, the point features including at least one of a Harris corner feature, a Sift feature, a Brisk feature, a slope, and a curvature; A plurality of target points are identified based on point features of the plurality of candidate target points corresponding to the initial pixel point set.

[0084] Optionally, the route optimization module 405 specifically: determining values ​​corresponding to each pixel point in a matrix corresponding to the initial segmented image based on the reference pixel points and the supplementary pixel points included in the initial segmented image; determining a gradient segmented image based on a matrix corresponding to the initial segmented image and the gradient vector field; for each target point, a segmented optimal path is determined starting from the target point based on the gradient segmentation image; the reference pixel points are pixel points on the initial boundary of each of the image regions; The supplementary pixel point is a pixel point in the initial segmented image that is in the vicinity of the reference pixel point.

[0085] Optionally, the route optimization module 405 specifically: sorting each of the target points based on its position in the initial segmented image; For each sorted target point, the target point and the next target point of the target point are set as a target point set, the target point is set as a start point of the target point set, and the next target point of the target point is set as an end point of the target point set; For each set of goal points, a split optimal path from the start of the goal set to the end of the goal set is used to determine a split optimal path corresponding to the goal set based on the gradient vector field.

[0086] Optionally, the route optimization module 405 specifically: The starting point is a parent path point, determining an optimal path point from a plurality of candidate path points based on the parent path point and the gradient vector field, the plurality of candidate path points being neighboring pixel points of the parent path point; If the optimal path point is not the end point, the optimal path point is set as a new parent path point, and the step of determining the optimal path point is repeated; If the optimal path point is the end point, it is used to determine a split optimal path from the start point of the set of destination points to the end point of the set of destination points based on all the determined optimal path points.

[0087] Optionally, the route optimization module 405 specifically: determining a loss value corresponding to a path from the parent path point to each of the candidate path points based on the gradient vector field as a loss value corresponding to the candidate path point; The candidate path point for which the loss value is smallest is used to determine the optimal path point.

[0088] Optionally, the route optimization module 405 specifically: determining a loss value corresponding to each of the candidate path points based on a distance from each of the candidate path points to the end point and the gradient vector field; The candidate path point for which the loss value is smallest is used to determine the optimal path point.

[0089] The present invention further provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the method of FIG. 1 above.

[0090] The present invention further provides an electronic device as shown in Fig. 5. As shown in Fig. 5, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, an internal memory, and a non-volatile memory, and may of course include other hardware required for operation. The processor loads the corresponding computer program from the non-volatile memory into the internal memory and executes it to implement the method described in Fig. 1 above.

[0091] Of course, in addition to realization by software, the present invention does not exclude other realization methods, such as logical devices or a combination of hardware and software. In other words, the entity that executes the following processing steps is not limited to each logical unit, but may be hardware or a logical device.

[0092] In the 1990s, improvements in a technology could be clearly divided into hardware improvements (such as improvements in circuit structures such as diodes, transistors, and switches) and software improvements (improvements in method flows). However, with the development of technology, many of the current method flow improvements can be considered as direct improvements to hardware circuit structures. Designers can mostly obtain the corresponding hardware circuit structures by programming the improved method flows into the hardware circuits. Therefore, it cannot be said that method flow improvements cannot be realized by hardware physical modules. For example, Programmable Logic Devices (PLDs) (e.g. Field Programmable Gate Arrays (FPGAs)) are such integrated circuits, whose logical functions are determined by the user's programming of the device. Instead of chip manufacturers designing and manufacturing dedicated integrated circuit chips, designers program and "integrate" digital systems onto a single PLD. Nowadays, instead of handcrafting integrated circuit chips, this programming is mostly achieved using software called a "logic compiler", which is similar to the software compiler used to write programs. In order to compile the original code, it must be written in a specific programming language, which is called a Hardware Description Language (HDL). There is not just one type of HDL, but many others, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java (Registered Trademark)There are many types of hardware description language, such as VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. (Registered Trademark) It will be apparent to those skilled in the art that a hardware circuit that realizes the logical method flow can be easily obtained by simply programming the method flow logically into an integrated circuit using some of the above hardware description languages.

[0093] The controller may be implemented in any suitable manner, for example the controller may take the form of a microprocessor or processor, a computer readable storage medium storing computer readable program code (e.g. software or firmware) executable by the (micro)processor, logic gates, switches, Application Specific Integrated Circuits (ASICs), programmable logic controllers and embedded microcontrollers, examples of which include the ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, Silicone Labs The memory controller may also be implemented as part of the control logic of the memory, including but not limited to microcontrollers such as C8051F320. It will be clear to those skilled in the art that in addition to implementing the controller with pure computer-readable program code, it is entirely possible to logically program the method steps to cause the controller to perform the same functions in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Thus, such controllers may be considered as hardware components, and the devices contained therein for implementing various functions may also be considered as structures within the hardware components. Alternatively, the devices for implementing various functions may also be considered as software modules implementing methods, or as structures within the hardware components.

[0094] The system, device, module or unit described in the above embodiments may be specifically realized by a computer chip, an entity, or a product having some function. A typical realizing device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a mobile phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet, a wearable device, or any combination of these devices.

[0095] For the sake of convenience, the above-mentioned device will be described by dividing it into various units according to their functions. Of course, when implementing the present invention, the functions of each unit can be realized by the same or multiple pieces of software and / or hardware.

[0096] As will be appreciated by those skilled in the art, the embodiments of the present invention may be provided as a method, a system, or a computer program product. Thus, the present invention may employ an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware. Furthermore, the present invention may take the form of a computer program product embodied in one or more computer usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) that contain computer usable program code.

[0097] The present invention will be described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, may be realized by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, whereby the instructions executed by the processor of the computer or other programmable data processing device generate an apparatus for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0098] These computer program instructions may be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture that includes an instruction apparatus that implements the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0099] These computer program instructions may be loaded into a computer or other programmable data processing device such that a sequence of operational steps are executed on the computer or other programmable device to generate a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0100] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0101] The memory may be a volatile memory, a random access memory, or any other type of computer-readable storage medium. (RAM) and / or non-volatile memory, such as read only memory (ROM) or flash memory (flash RAM). Memory is one example of a computer-readable storage medium.

[0102] Computer-readable storage media include non-volatile and volatile media, removable and non-removable media, and may implement information storage by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Computer storage media may include Phase Change RAM (PRAM), Static Random-Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), and other types of memory. (Registered Trademark) ), flash memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible to a computing device. As defined herein, computer readable storage media does not include transitory storage computer readable storage media, such as modulated data signals and carriers.

[0103] Additionally, the terms "comprise", "contain" or any other variation thereof are intended to include a non-exclusive inclusion, whereby a process, method, article or device that comprises a set of elements not only includes those elements, but also includes other elements not expressly listed, or includes the inherent elements of such process, method, article or device. Absent more limitations, an element qualified by the phrase "comprises a..." does not exclude the presence of further identical elements in a process, method, article or device that includes said element.

[0104] As will be appreciated by those skilled in the art, the embodiments of the present invention may be provided as a method, a system, or a computer program product. Thus, the present invention may employ an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware. Furthermore, the present invention may take the form of a computer program product embodied in one or more computer usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) that contain computer usable program code.

[0105] The present invention relates generally to computer-executable instructions, such as program modules, that are executed by a computer. context Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

[0106] Each embodiment of the present invention will be described in a stepwise manner, and the same or similar parts between each embodiment may be referred to each other, and the emphasis of each embodiment is on the differences from other embodiments. In particular, the system embodiment is basically similar to the method embodiment, so it will be briefly described, and the relevant parts may be referred to the description of part of the method embodiment.

[0107] The above is merely an embodiment of the present invention, and is not used to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. acquiring an original image including a plurality of segmentation objects from an image acquisition device, the original image being a grayscale image; A step of pre-segmenting the original image to obtain an initial segmented image including a plurality of image regions, the pre-segmentation being one of a thresholding segmentation method, an edge detection segmentation method, a region growing segmentation method, a split-merge segmentation method, a watershed algorithm-based segmentation method, a graph cut-based segmentation method, an active contour model-based segmentation method, and a deep learning segmentation method, and each of the image regions includes one of the segmentation targets; determining a gradient vector field corresponding to each pixel point in the original image based on a gradient vector corresponding to each pixel point in the original image, where for each pixel point, the gradient vector field can indicate a gradient change in pixel value between the pixel point and an adjacent pixel point adjacent to the pixel point, and the pixel value is a gray value of the pixel point; identifying, for each of the image regions, from each pixel point on an initial boundary of the image region, a number of target points that satisfy a predetermined condition, the predetermined condition being that the target points are detected as corner points by Harris corner detection; sorting each of the target points along an initial boundary of the image region based on the location of each of the target points in the initial segmented image; for each sorted target point, defining the target point and a target point next to the target point as a target point set, defining the target point as a start point of the target point set, and defining the target point next to the target point as an end point of the target point set; For each target point set, setting the starting point as a parent path point; determining neighboring pixel points of the parent path point as candidate path points; determining a loss value corresponding to each of the candidate path points based on a distance from each of the candidate path points to the end point and the gradient vector field, the loss value being an index value for evaluating whether a candidate path point can be used as an optimal path point; determining the candidate route point that has the smallest loss value as the optimal route point; If the optimal path point is not the end point, the optimal path point is set as a new parent path point, and the step of determining the optimal path point is repeated; If the optimal path point is the end point, determining a split optimal path from a start point of the set of target points to an end point of the set of target points based on all the determined optimal path points; Segmenting the original image based on the determined optimal segmentation path.

13. An image segmentation method comprising:

2. The step of identifying a plurality of target points that satisfy a predetermined condition from each pixel point on the initial boundary of the image region includes: determining an initial set of pixel points including all pixel points on an initial boundary of the image region; For each pixel point included in the initial pixel point set, determining, from the original image, a pixel point having the same coordinates as the pixel point in the initial segmented image as a candidate target point; determining point features for the candidate landmark points, the point features including features detected by Harris corner detection; and identifying a plurality of target points whose point features satisfy the predetermined condition from a plurality of the candidate target points corresponding to the initial pixel point set.

2. The image segmentation method according to claim 1.

3. an acquisition module for acquiring an original image including a plurality of segmentation targets from an image acquisition device, the original image being a grayscale image; a pre-segmentation module for pre-segmenting the original image to obtain an initial segmented image including a plurality of image regions, the pre-segmentation being one of a thresholding segmentation method, an edge detection segmentation method, a region growing segmentation method, a split-merge segmentation method, a watershed algorithm-based segmentation method, a graph cut-based segmentation method, an active contour model-based segmentation method, and a deep learning segmentation method, and each of the image regions includes one of the segmentation targets; a determination module for determining a gradient vector field corresponding to each pixel point in the original image based on a gradient vector corresponding to each pixel point in the original image, where for each pixel point, the gradient vector field can indicate a gradient change in pixel value between the pixel point and an adjacent pixel point adjacent to the pixel point, and the pixel value is a gray value of the pixel point; an identification module for identifying, for each said image region, from each pixel point on an initial boundary of said image region, a number of target points satisfying a predetermined condition, said predetermined condition being that said target points are detected as corner points by Harris corner detection; sorting each of the target points along an initial boundary of the image region based on the location of each of the target points in the initial segmented image; For each sorted target point, the target point and the next target point of the target point are set as a target point set, the target point is set as a start point of the target point set, and the next target point of the target point is set as an end point of the target point set; For each target point set, The starting point is a parent path point, A pixel point adjacent to the parent path point is set as a candidate path point. determining a loss value corresponding to each of the candidate route points based on a distance from each of the candidate route points to the end point and the gradient vector field, the loss value being an index value for evaluating whether or not a candidate route point can be used as an optimal route point; The candidate route point that has the smallest loss value is set as the optimal route point; If the optimal route point is not the end point, the optimal route point is set as a new parent route point, and the process of determining the optimal route point is repeated; a route optimization module for determining a split optimal route from a start point of the set of destination points to an end point of the set of destination points based on all the determined optimal route points if the optimal route point is the end point; a segmentation module for segmenting the original image based on the determined segmentation optimal path.

1. An image segmentation device comprising:

4. A computer-readable storage medium storing a computer program, the computer program being executed by a processor to perform the method according to any one of claims 1 to 2. A computer-readable storage medium comprising:

5. An electronic device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, the electronic device performing the method according to any one of claims 1 to 2 when the processor executes the computer program.

1. An electronic device comprising:

Images