An image segmentation method to be segmented, a material sorting device and a readable storage medium

Abstract

The present disclosure relates to the field of material sorting, and particularly relates to a to-be-segmented image segmentation method, a material sorting device and a readable storage medium. The to-be-segmented image segmentation method comprises: determining gradient information of each point in the to-be-segmented image according to the to-be-segmented image; determining a plurality of concave points in the to-be-segmented image according to the to-be-segmented image; pairing the plurality of concave points two by two to obtain one or more concave point pairs; determining a segmentation path of the to-be-segmented image according to the concave point pairs and the gradient information of each point in the to-be-segmented image; and segmenting the to-be-segmented image according to the segmentation path to obtain a plurality of target images. Through the method of the present disclosure, the actual edges of a plurality of overlapping targets can be quickly and accurately determined, the to-be-segmented image can be more accurately segmented, and in actual application, quick detection can be achieved, better real-time performance can be achieved, and the efficiency and accuracy of image segmentation are effectively improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image processing, in particular to a to-be-segmented image segmentation method, a material sorting device and a readable storage medium. BACKGROUND

[0002] In the process of material sorting, there is often overlap between materials, so it is necessary to segment the material image to determine the position of each material. For the case where materials mutually occlude each other, the material overlap part is too large, resulting in that the overlap area between materials in the imaged image is nested too deeply, which seriously affects the subsequent recognition and positioning of the material boundary, and reduces the accuracy of segmentation. The current image segmentation method for materials cannot accurately segment the case where materials mutually occlude each other and the overlap area is nested too deeply, and the accuracy of the segmentation result is poor. In addition, the current image segmentation method for materials requires a large amount of labeled data for model training, the computational complexity of image segmentation is high, the efficiency is low, the robustness is poor, and it cannot meet the real-time segmentation demand. SUMMARY

[0003] To overcome the problems in the related art, an exemplary embodiment of the present disclosure provides a to-be-segmented image segmentation method, wherein the to-be-segmented image includes a plurality of target images, comprising: determining gradient information of each point in the to-be-segmented image according to the to-be-segmented image, wherein the gradient information includes a gradient amplitude; determining a plurality of concave points in the to-be-segmented image according to the to-be-segmented image; pairing the plurality of concave points two by two to obtain one or more concave point pairs; determining a segmentation path of the to-be-segmented image according to the concave point pairs and the gradient information of each point in the to-be-segmented image; and segmenting the to-be-segmented image according to the segmentation path to obtain a plurality of target images.

[0004] In some embodiments, the pairing the plurality of concave points two by two to obtain one or more concave point pairs comprises: determining a corresponding concave region of each of the concave points; and pairing each concave point with a concave point belonging to another concave region to obtain a concave point pair.

[0005] In some embodiments, the determining a segmentation path of the to-be-segmented image according to the concave point pairs and the gradient information of each point in the to-be-segmented image comprises: based on one or more concave point pairs, confirming one or more candidate paths according to the gradient information; and determining the segmentation path based on the candidate paths.

[0006] In some embodiments, the confirming one or more candidate paths according to the gradient information based on the one or more concave point pairs comprises: determining a current concave point pair, wherein the current concave point pair comprises a first concave point and a second concave point, the first concave point is a concave point with the largest gradient amplitude among all current concave points, and the second concave point is a concave point with the largest gradient amplitude among concave points paired with the first concave point; connecting the first concave point and the second concave point to obtain one or more passing points on the connecting line to obtain an initial path; updating the initial path based on the gradient information to obtain a current path; if the current path passes through other concave points other than the current concave point pair, ignoring the second concave point in the current concave point pair and returning to perform the determining of the current concave point pair; and if the current path does not pass through other concave points other than the current concave point pair, determining the current path as a candidate path and performing the determining of the segmentation path based on the candidate path.

[0007] In some embodiments, the updating the initial path based on the gradient information to obtain a current path comprises: updating the passing points according to the gradient information to obtain a process path, so that an average gradient amplitude of the process path is greater than or equal to an average gradient amplitude before updating; if the average gradient amplitude of the process path is equal to the average gradient amplitude before updating, or the average gradient amplitude of the process path is greater than or equal to a first threshold value, stopping updating and taking the process path as the current path; if the average gradient amplitude of the process path is greater than the average gradient amplitude before updating, and a number of times of updating is less than a number threshold value, returning to perform the updating the passing points according to the gradient information to obtain a process path; and if the average gradient amplitude of the process path is greater than the average gradient amplitude before updating, and the number of times of updating is equal to the number threshold value, returning to perform the determining of the current concave point pair.

[0008] In some embodiments, the confirming one or more candidate paths according to the gradient information based on the one or more concave point pairs comprises: determining all concave point pairs; connecting each concave point pair to obtain one or more initial paths on each connecting line; updating the initial paths based on the gradient information to obtain one or more current paths; if the current path passes through other concave points other than the current concave point pair, ignoring the current path; and if the current path does not pass through other concave points other than the current concave point pair, determining the current path as a candidate path.

[0009] In some embodiments, the updating the initial path based on the gradient information to obtain one or more current paths comprises: updating the passing point according to the gradient information to obtain a process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before the updating; if the average gradient amplitude of the process path is equal to the average gradient amplitude before the updating, or the average gradient amplitude of the process path is greater than or equal to a first threshold value, then stopping the updating and taking the process path as one of the current paths; if the average gradient amplitude of the process path is greater than the average gradient amplitude before the updating, and the number of times of updating is less than a number threshold value, then returning to perform the updating of the passing point according to the gradient information to obtain the process path; if the average gradient amplitude of the process path is greater than the average gradient amplitude before the updating, and the number of times of updating is equal to the number threshold value, then determining whether the average gradient amplitude of the process path is greater than a second threshold value, wherein the second threshold value is less than or equal to the first threshold value; if the average gradient amplitude of the process path is greater than or equal to the second threshold value, then taking the process path as one of the current paths; and if the average gradient amplitude of the process path is less than the second threshold value, then ignoring the process path.

[0010] In some embodiments, the determining the segmentation path based on the candidate paths further comprises: if the number of the candidate paths is equal to 1, then taking the candidate path as the segmentation path; and if the number of the candidate paths is greater than 1, then determining the segmentation path according to the gradient information of the candidate paths.

[0011] In some embodiments, the determining the segmentation path according to the gradient information of the candidate paths comprises: taking the candidate path with an average gradient amplitude greater than or equal to a gradient threshold value as the segmentation path; or taking the candidate path with the maximum average gradient amplitude as the segmentation path; or sorting the candidate paths according to the average gradient amplitudes of the candidate paths from large to small, and taking the first N-1 candidate paths as the segmentation paths, wherein N is the number of targets contained in the image.

[0012] In some embodiments, the determining the plurality of concave points in the image to be segmented according to the image to be segmented comprises: determining candidate points according to the curvatures of the points in the image; if the depth information of the candidate point is greater than or equal to a depth threshold value, then determining the candidate point as a concave point; and if the depth information of the candidate point is less than the depth threshold value, then ignoring the candidate point.

[0013] In some embodiments, the determining the plurality of concave points in the image to be segmented according to the image to be segmented comprises: performing image preprocessing on the image to be segmented to obtain a preprocessed image; and determining the concave points in the image according to the preprocessed image.

[0014] In a second aspect, the present disclosure also provides a material sorting device, wherein the material sorting comprises: an image segmentation module configured to perform the image segmentation method according to any one of the above embodiments; and a material sorting module configured to perform target identification and sorting according to the target image obtained by the image segmentation module.

[0015] In a third aspect, the present disclosure also provides a computer-readable storage medium storing a program for performing the image segmentation method according to any one of the above embodiments.

[0016] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure.

[0017] The technical solution provided by the embodiments of the present disclosure can have the following beneficial effects: by identifying the concave points of the image to be segmented, determining the path of the concave point connection line, and segmenting the image in combination with the gradient information of the image, the image can be accurately identified and segmented in the case of large overlapping area and serious image nesting, and the segmentation path is the contour edge of the overlapping area between multiple targets. By using the method of the present disclosure, the actual edges of multiple overlapping targets can be quickly and accurately determined, the image to be segmented can be more accurately segmented, the image can be quickly detected in actual application, and better real-time performance is achieved, thereby effectively improving the efficiency and accuracy of image segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0018] The present disclosure can be better understood by describing the exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which:

[0019] Figure 1 is a structure diagram of a sorting device according to an exemplary embodiment of the present disclosure;

[0020] Figure 2 is an image to be segmented according to an exemplary embodiment of the present disclosure;

[0021] Figure 3 is a flowchart of an image segmentation method according to an exemplary embodiment of the present disclosure;

[0022] Figure 4 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure;

[0023] Figure 5 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure;

[0024] Figure 6 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure;

[0025] Figure 7 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0026] Figure 8 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0027] Figure 9 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0028] Figure 10 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0029] Figure 11 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0030] Figure 12 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0031] Figure 13 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0032] Figure 14 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0033] Figure 15 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0034] Figure 16 is a schematic block diagram of a material sorting system according to an exemplary embodiment of the present disclosure;

[0035] Figure 17 is a schematic block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The specific embodiments of the present application will now be described in detail, it should be pointed out that in the specific description of these embodiments, for the purpose of concise and brief description, the present specification can not make a detailed description of all the features of the actual embodiments. It should be understood that in the actual implementation of any embodiment, as in the process of any engineering or design project, in order to achieve the specific goals of the developers, to meet the system related or business related constraints, often make a variety of specific decisions, and this will change from one embodiment to another. In addition, it should be understood that although the effort made in this development process can be complex and lengthy, however, for those skilled in the art related to the disclosed content of the present application, some design, manufacture or production changes based on the disclosed technology content are only routine technical means, and should not be understood as insufficient disclosure.

[0037] Unless otherwise defined, the technical terms or scientific terms used in the claims and the specification should be understood as the usual meaning understood by those skilled in the art. The "first", "second" and similar words used in the patent application specification and claims of the present application do not represent any order, quantity or importance, but are used to distinguish different components. "One" or "a" and similar words do not represent a quantity limitation, but represent the existence of at least one. "Include" or "contain" and similar words mean that the elements or objects appearing before "include" or "contain" cover the elements or objects listed after "include" or "contain" and their equivalent elements, and do not exclude other elements or objects. "Connected" or "connected" and similar words are not limited to physical or mechanical connection, nor are they direct or indirect connection.

[0038] The image segmentation method provided by the present disclosure can be applied to a material sorting device 100, which can be used for sorting materials, such as Figure 1 As shown, it can include a feeding mechanism 110, a conveying mechanism 120, a detection mechanism 130 and a sorting device 140. The feeding mechanism 110 is used to feed the material to be sorted into the conveying mechanism 120. The conveying mechanism 120 can be a conveyor belt or a chute structure, which is used to convey the material to be sorted fed by the feeding mechanism 110. The detection mechanism 130 is used to detect the material conveyed on the conveying mechanism 120 to detect whether the material is a material to be rejected. The material to be rejected refers to the material to be separated by the sorting device. The material to be rejected can be the required material or the non-required material, as long as the sorting of the material can be realized. The sorting device 140 is used to reject the material to be rejected. The material sorting device 100 can be used for sorting of ore, and can also be used for food sorting or waste sorting tasks.

[0039] In the process of material sorting, due to the irregular shape and various surface textures of the material itself, and the poor effect of image imaging, as shown in Figure 2 (a), Figure 2 (e), the target materials to be sorted overlap each other, and in the case of excessive overlap area, the occlusion between multiple materials is serious, especially in the case of ore, the shape of the ore is irregular, the surface texture is complex, and in the process of material sorting, the ore is easy to roll during feeding by the feeding mechanism 110 into the transmission mechanism 120, resulting in the overlapping of the ore. The overlapping area may be large. In the obtained image, multiple targets are included, and the overlapping area between the targets is deeply nested, making it difficult to identify and locate the boundary of the material, resulting in poor accuracy when segmenting the overlapping materials.

[0040] In some related technologies, by training a machine learning model, an instance segmentation technology is used to segment multiple overlapping or adhering materials. However, due to the instance segmentation technology based on machine learning, a model needs to be established for the target material, and a large amount of high-quality labeled data is needed for data training. In application scenarios such as ore classification and garbage classification, it is difficult to obtain high-quality labeled data, the quality of model training is crossed, and due to the high complexity of calculation, the material segmentation method based on machine learning requires a longer response time, which cannot meet the real-time segmentation requirements in actual production.

[0041] In some other related technologies, concave points are obtained by concave point detection, the concave points are paired and connected to obtain a segmentation line, which can quickly segment the image. However, for the case of large-area overlap and serious nesting of the target image, pairing the concave points and directly connecting the lines as the segmentation line has poor segmentation accuracy. In the case of large-area overlap of the target, part of the target A is nested in the target B in the image. According to the concave point pairing and line connection for segmentation, the part of the target A nested in the target B will be attributed to the target B after segmentation, and the segmentation result does not match the actual situation, resulting in the fact that the target B after segmentation still contains the target A, which easily leads to incorrect identification and incorrect sorting in the subsequent identification and sorting process, resulting in low accuracy of material sorting. Moreover, for some cases of large-area overlap of the target, the concave point identification may not be able to accurately obtain the concave points, resulting in incorrect segmentation, and the accuracy of image segmentation is low.

[0042] To solve the above problems, as shown in Figure 3 The present disclosure provides a method for segmenting a to-be-segmented image, wherein the to-be-segmented image includes multiple target images, which can include steps S110 to S150, which are described in detail below.

[0043] At step S110, gradient information of each point in the image to be segmented is determined according to the image to be segmented, where the gradient information includes a gradient magnitude. The image to be segmented can be an image including multiple targets that overlap each other, and the image can be acquired by an image acquisition device. Then, the image of the multiple targets that overlap each other is extracted from the image to obtain the image to be segmented. The gradient information of the image can reflect the change in grayscale or color of the image to be segmented. The gradient information can include a gradient magnitude, which can be the gradient size at each pixel point in the image to be segmented. The greater the gradient magnitude, the more intense the change in the image, which can correspond to an edge or other features in the image. Therefore, in the process of determining the segmentation path 10 of the image to be segmented, the edge of the overlapping targets in the overlapping part in the image to be segmented can be found according to the gradient magnitude. The gradient information in the image to be segmented can be acquired by an image detection algorithm, so as to obtain the gradient magnitude and gradient direction of the image. The image detection algorithm can include a Sobel operator, a Scharr operator, or a Laplace operator.

[0044] At step S120, multiple concave points in the image to be segmented are determined according to the image to be segmented. The concave point can be the lowest point on the edge of a continuous pattern in the image to be segmented that is concave downward. The image can be first subjected to contour extraction, and the concave points can be determined according to the image by a concave point recognition algorithm. The concave points in the image can also be recognized by a curvature-based method. The curvature information of each point on the contour of the image is calculated, and the concave points in the image are determined by screening according to the curvature information. For multiple materials that overlap each other, the nesting of the overlapping regions in the images of the multiple materials can be too deep, which can make it difficult to recognize the concave points. In the image segmentation method of the present disclosure, the related parameters for recognizing the concave points can be adjusted during the process of recognizing the concave points, multiple candidate concave points can be recognized, and the candidate concave points can be screened to determine the final concave points.

[0045] In some embodiments, as Figure 4As shown, in step S120, the plurality of concave points in the image to be segmented are determined according to the image to be segmented, which can include: step 121, determining the candidate points according to the point curvature in the image. The contour of the image to be segmented can be extracted, the point curvature of each point on the contour of the image to be segmented is calculated, and the contour points greater than the point curvature threshold value can be determined as candidate points by setting the point curvature threshold value. Due to the mutual overlapping of multiple materials, the nested depth of the overlapping area in the image is too deep, and the concave point curvature may be small and difficult to identify. Therefore, a small point curvature threshold value can be set to ensure that the concave points in the image can be determined as candidate points. To calculate the contour point curvature, a point can be extracted as a detection point C first, and two adjacent points of the detection point C are selected, which are a predecessor point P and a successor point N. The vectors corresponding to CP and PN are calculated, and the cross product of the two vectors is calculated, which is denoted as CrossProduct. The lengths of the two vectors are calculated, which are denoted as CPL and PNL respectively. The tangent value of the vector angle of CP and PN is calculated, which is denoted as TanTHeta. Finally, the curvature of the detection point in the image is calculated according to the formula abs(4.0 / tan(TanTHeta / 2.0)), and the curvature of each point on the contour of the image to be segmented is obtained. The candidate points are determined according to the curvature threshold value.

[0046] If the depth information of the candidate point is greater than or equal to the depth threshold value, the candidate point is determined as a concave point. If the depth information of the candidate point is less than the depth threshold value, the candidate point is ignored. The depth information of the candidate point is the shortest distance between the candidate point and the convex hull, and the depth information of the candidate point can also be the perpendicular distance between the candidate point and the line connecting the two endpoints of the concave region in which the candidate point is located. The concave region is a region in the image to be segmented that is concave, and each concave region can include a plurality of candidate points. The convex hull can be an outer convex polygon of the contour of the image to be segmented. The position of the candidate point can be determined according to the information of the candidate point, a perpendicular line is drawn from the candidate point to the edge of the convex hull, and the perpendicular distance is the depth of the concave point. The endpoints of the concave region can be determined first, and then a perpendicular line is drawn from the candidate point to the line connecting the endpoints of the concave region, and the perpendicular distance is the depth of the concave point. The depth threshold value can be set, which can be the minimum depth value of the concave point. When the depth of the candidate point is greater than or equal to the depth threshold value, the candidate point can be determined as a concave point. When the depth of the candidate point is less than the depth threshold value, the candidate point is not a concave point and is ignored.

[0047] According to the point curvature determination alternative point, set a smaller point curvature threshold, can ensure that all concave points in the image to be segmented can be identified as alternative points, can improve the accuracy of concave point identification, the selected alternative points may also include other points in addition to concave points, therefore, by judging the depth information of the alternative points, the alternative points with depth greater than or equal to the depth threshold are reserved as concave points, the data amount is reduced, so that the subsequent concave point pairing and other operations are more rapid, at the same time, the interference points can be excluded, the invalid data is reduced, so as to improve the efficiency of the subsequent concave point pairing and other operations.

[0048] In step S130, a plurality of concave points are paired two by two to obtain one or more concave point pairs. The concave points in the image can be paired two by two, so that each concave point forms a concave point pair with other concave points. Each concave point can be paired multiple times, and each concave point can be paired into multiple concave point pairs.

[0049] In some embodiments, all concave points in the image to be segmented can be paired respectively, so that each concave point is paired with all the remaining concave points to obtain a concave point pair. Each concave point can be paired multiple times. By pairing all the concave points, it can be ensured that the concave points in the image to be segmented can avoid data omission in the subsequent process of determining the segmentation path 10 according to the concave point pairs and the gradient information. All concave points and their concave point pairs can be traversed to determine the segmentation path 10, which can ensure the accuracy of the image.

[0050] In some embodiments, all concave points in the image to be segmented can be paired respectively, so that each concave point is paired with all the remaining concave points to obtain a concave point pair. Each concave point can be paired multiple times. By pairing all the concave points, it can be ensured that the concave points in the image to be segmented can avoid data omission in the subsequent process of determining the segmentation path 10 according to the concave point pairs and the gradient information. All concave points and their concave point pairs can be traversed to determine the segmentation path 10, which can ensure the accuracy of the image.

[0051] Step S140, according to the concave point pair and the gradient information of each point in the image to be segmented, the segmentation path 10 of the image to be segmented is determined. The segmentation path 10 can be a line in the image, and the starting point and the ending point are different concave points in the image. The segmentation path 10 is the edge of the overlapping material in the image. The concave point pair can be taken, and the two concave points are taken as the starting point and the ending point of the segmentation path 10 respectively. The line connecting the concave point pair is adjusted according to the gradient information of each point in the image, so as to obtain the segmentation path 10 of the image. The gradient can be the change rate and direction of the pixel value in the image. The gradient information can include the gradient amplitude and the gradient direction. The gradient amplitude can represent the gray change rate or color change rate of the image at the current point. The gradient direction can represent the direction in which the image changes fastest at the current point. Due to the nesting and large-area overlapping of the target image, the method of directly connecting the concave points to segment the image will cause a target part to be segmented and belong to another target, resulting in inaccurate segmentation. The method of determining the segmentation path 10 according to the gradient information can quickly and accurately determine the edge profile of the overlapping area according to the gradient information, and determine it as the segmentation path 10, so as to make the segmentation more accurate, and the segmentation path 10 is closer to the edge profile of the actual overlapping area. According to the number of materials in the image, the segmentation path 10 can have one or more. For an image including two different materials, the segmentation path 10 has only one. For an image including three or more different materials, the segmentation path 10 can have multiple.

[0052] Step S150, according to the segmentation path 10, the image to be segmented is segmented to obtain a plurality of target images. According to the segmentation path 10, the image of the overlapping material can be segmented, as shown in Figure 2 (d)、 Figure 2 (h) shown, so that each material in the image is separated from each other.

[0053] By the image segmentation method of the present disclosure, the positions of the material overlap and serious occlusion in the image to be segmented can be determined more quickly by determining the concave points in the image to be segmented and pairing them into concave point pairs, so that the identification and response speed is faster. By obtaining the gradient information of each point in the image to be segmented, the gray level change rate or color change rate of the image and the direction in which the gray level or color of the image changes most quickly at each point are obtained. According to the gradient information, the area where the material overlap and serious occlusion occur can be located more accurately. The segmentation path 10 of the image is determined in combination with the concave points and the gradient information, so that the image can be accurately segmented in the case of material overlap, nesting and serious occlusion. The segmentation speed of the material overlap image can be improved while the accuracy of image segmentation is effectively improved, and the segmentation efficiency is better. The segmentation path 10 is not a direct connection of the concave points, but can accurately identify the target contour edge of the overlap area. The concave point detection can quickly obtain the start and end positions of the possible segmentation line, avoiding a large amount of invalid calculation. According to the gradient information and in combination with the concave point information, the accurate segmentation line can be obtained, which is closer to the target contour edge of the overlap area, and the segmentation accuracy is higher. In the process of sorting the materials, the image can be segmented in real time, which is more efficient and can improve the accuracy of sorting.

[0054] In some embodiments, as shown in FIG. 1, the step S130 of pairing the plurality of concave points two by two to obtain one or more concave point pairs can include steps S131 and S132. Figure 5

[0055] In step S131, the corresponding concave region of each concave point is determined. The concave region can be a region in the image to be segmented where a concave point is located. For the image to be segmented, it includes a plurality of target images that overlap with each other or are occluded. The concave region can be located at the position where the different target images overlap. The concave point is located in the concave region, and there can be multiple concave points in one concave region.

[0056] In step S132, each concave point is paired with a concave point belonging to another concave region to obtain a concave point pair. According to the concave point and the concave region corresponding thereto, the concave points belonging to different concave regions can be paired to obtain the concave point pair. Since the target images overlap, the overlapping positions on both sides will form two different concave regions. Therefore, according to the two concave points belonging to the same region, the segmentation path 10 cannot be determined. The concave points of the multiple targets in the case of overlap must belong to different concave regions, so for the two concave points belonging to the same concave region, pairing is not required.

[0057] ​By step 131, step 132, the concave points can be distinguished according to the concave regions where they are located, and the concave points belonging to different concave regions are paired to obtain one or more concave point pairs. The concave points can be screened and classified according to the concave regions. Since the multiple target images overlap, their concave points must belong to different concave regions. In this embodiment, the concave points belonging to the same concave region are avoided to be paired as concave point pairs, which can effectively reduce the data amount and effectively reduce the data calculation amount of subsequent image segmentation, thereby reducing the actual image segmentation required, and improving the calculation speed and efficiency of image segmentation. The segmentation method of the image to be segmented has better real-time performance when applied to the material sorting equipment.

[0058] In some embodiments, as shown in step S140, the gradient information of each point in the image to be segmented and the concave point pairs are used to determine the segmentation path 10 of the image to be segmented, which can include: Figure 6

[0059] Step S141, based on one or more concave point pairs, determines one or more candidate paths according to the gradient information. The candidate path can be determined by combining the concave point pairs and the gradient information. The line connecting the concave point pairs can be adjusted according to the gradient information of each point on the line. The direction of the fastest change in gray scale of each point on the line connecting the concave point pairs can be determined by the gradient direction of each point in the image, so as to adjust the line connecting the concave point pairs along the direction of the fastest change in gray scale. Meanwhile, the point with a larger gradient amplitude in the direction of the fastest change in gray scale can be found by the gradient amplitude of each point in the image, so as to adjust the line connecting the concave point pairs. The line connecting the concave point pairs can be adjusted multiple times, and the final result after adjustment is taken as the candidate path of the image, so as to determine the candidate path of the image.

[0060] Step S142, based on the candidate path, determines the segmentation path 10. One or more candidate paths can be determined according to the concave point pairs and the gradient information. The final segmentation path 10 can be determined by screening the candidate paths. The segmentation path 10 is the final edge of the overlapping part of the target image, as shown in Figure 2 (c)、 Figure 2 (g), the segmentation path 10 can have one or more. For the image to be segmented including two target images, the segmentation path 10 can be one. For the image to be segmented including more than two target images, the segmentation path 10 can have multiple.

[0061] ​By steps S141 and S142, according to the concave point pairs and the gradient information of each point in the image, one or more candidate paths can be confirmed, which are not directly connected to the concave points, but are obtained by adjusting the connection of the concave points according to the gradient information, and have higher accuracy. By screening the candidate paths, the final segmentation path 10 is determined, which is the actual edge of the multiple targets in the overlapping or nested position, and can determine the specific edge contour of each target in the image to be segmented, so that the image segmentation is more accurate, effectively improves the accuracy of image segmentation, and effectively avoids false recognition in the actual material sorting process, and has higher sorting accuracy.

[0062] In some embodiments, as shown in Figure 7 Step S141 can include steps S1411 to S1413.

[0063] Step S1411, determine the current concave point pair, wherein the current concave point pair includes a first concave point and a second concave point, the first concave point is the concave point with the largest gradient amplitude in the current all concave points, and the second concave point is the concave point with the largest gradient amplitude in the concave point paired with the first concave point. All concave points can be sorted in descending order of gradient amplitude, and the current concave point pair can be determined. The concave point with the largest gradient amplitude in the current all concave points can be selected as the first concave point, and the concave point with the largest gradient amplitude in the concave point paired with the first concave point can be selected as the second concave point. Since the size of the gradient amplitude can represent the rate of change of the image, the larger the gradient amplitude, the more obvious the change of the image at the current point, therefore, in the process of determining the concave point pair, the first concave point and the second concave point with larger gradient amplitude can be selected, and the path determined according to the concave point with larger gradient amplitude has the highest possibility of being a segmentation path. Especially in the case of pairing concave points belonging to different concave regions, the possibility of the concave point pair being an end point of the segmentation path can be further improved.

[0064] Step S1412, connecting the first concave point and the second concave point, and taking one or more passing points on the connecting line to obtain an initial path. The initial path can be a line segment inside the image to be segmented. The initial path can be determined as a line segment with the first concave point as the starting point and the second concave point as the ending point. One or more points on the initial path can be taken as passing points for subsequent adjustment and updating of the initial path. The initial path can be a straight line segment directly connecting the first concave point and the second concave point, or an arbitrary line segment with the first concave point and the second concave point as end points. The number of passing points can be fixed, and the number can be 3-20, such as 3, 5, or 10, which can be set at equal intervals. The number of passing points can also be determined according to the length of the initial path, which can be 1-10 cm or 3-20 pixel distances, for example, 1 passing point every 5 or 10 pixel distances. The more the number of passing points selected, the more points can be adjusted in the subsequent adjustment, and the relative accuracy is improved. The fewer the number of passing points selected, the lower the computational complexity and the higher the computational efficiency.

[0065] Step S1413, updating the initial path based on the gradient information to obtain a current path. The passing points of the initial path can be updated based on the gradient amplitude and gradient direction to obtain updated passing points, so that the gradient amplitude of the updated passing points is greater than the gradient amplitude of the passing points of the initial path. The updated passing points are connected with the first concave point and the second concave point to obtain the updated current path. The direction of the updated initial path can also be determined according to the gradient direction of the initial path, and the entire path can be updated according to the gradient amplitude of the initial path, so that the gradient amplitude of the updated current path is greater than the gradient amplitude of the initial path. The updating of the initial path based on the gradient information can be repeated multiple times to update the initial path multiple times.

[0066] According to the current path, all points on the current path can be determined. Since the final segmentation path 10 is the edge of the target image with overlap, the segmentation path 10 starts from one concave point, passes through the inside of the image to be segmented, and finally ends at another concave point. In the segmentation path 10, only the points at both ends are concave points, and the remaining points are inside the image to be segmented. Therefore, if the current path passes through other concave points other than the current concave point pair, the current path cannot be the segmentation path 10, and the second concave point in the current concave point pair is ignored and the determination of the current concave point pair is returned. If the current path does not pass through other concave points other than the current concave point pair, the current path can be the segmentation path 10, and the current path can be determined as a candidate path, and the determination of the segmentation path 10 based on the candidate path is performed.

[0067] In some embodiments, the image to be segmented may include multiple targets. After determining the segmentation path according to steps S1411 to S1413 and S142, one segmentation path 10 can be obtained. However, the image to be segmented may include three or more targets, therefore two or more segmentation paths 10 need to be determined. Therefore, in this case, the number of targets can be determined by target detection, and the number of required segmentation paths can be determined based on the number of targets. If the number of targets is greater than 2, the number of required segmentation paths is greater than 1. Alternatively, the number of required segmentation paths can be determined based on the number of deeply concave regions in the image. The degree of concavity of a concave region can be determined based on the edge curvature of the concave region. For concave regions with large edge curvature, it can be considered that the current concave region contains the endpoints of the segmentation path. The degree of concavity of a concave region can also be determined by the depth of the concave point, which is the distance between the concave point and the line connecting the endpoints of its concave region. When the average depth of the concave points in a concave region reaches a large value, it can be considered that the current concave region contains the endpoints of the segmentation path. When the number of deeply concave regions is greater than 2, the number of required segmentation paths can be determined to be greater than 1. Depending on the required number of segmentation paths, if the required number of segmentation paths is greater than 1, after determining a segmentation path by executing steps S1411 to S1413 and S142, steps S1411 to S1413 and S142 can be executed again to determine a new segmentation path, until the number of segmentation paths obtained matches the required number of segmentation paths. Since the path determined by the concave point with the larger gradient magnitude has the highest probability of being segmentation path 10, in this embodiment, by determining the current concave point pair and prioritizing the connection and path update of the concave point pair containing the concave point with the largest gradient magnitude, less data can be processed and segmentation path 10 can be determined faster in the process of determining segmentation path 10 of the image to be segmented; by determining whether the points on the current path pass through other concave points to determine candidate paths, the current path can be filtered to remove interfering data, reducing the amount of data to be processed, thereby effectively improving the speed of segmenting the image to be segmented and improving segmentation efficiency.

[0068] In some embodiments, such as Figure 8 As shown, step 1413, updating the initial path based on gradient information to obtain the current path, may include:

[0069] In step 14131, the passing point is updated according to the gradient information, the process path is obtained, and the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before updating; the passing point is updated based on the gradient information, a straight line perpendicular to the initial path can be drawn from the passing point, and a pixel point with a gradient greater than the current passing point is searched for on the straight line according to the gradient information of each point in the to-be-segmented image, and the pixel point is determined as a new passing point. After updating all the passing points, the new passing points, the first concave point and the second concave point are connected according to the gradient direction of each new passing point, and the process path is obtained. The initial path is updated based on the gradient information, a straight line can be drawn from the passing point along the gradient direction of the passing point, and a pixel point with a gradient greater than the current passing point is searched for on the straight line according to the gradient information of each point in the to-be-segmented image, and the pixel point is determined as a new passing point. After updating all the passing points, the new passing points, the first concave point and the second concave point are connected according to the gradient direction of each new passing point, and the process path is obtained. For the process path, the average gradient amplitude can be greater than the gradient amplitude before updating, or can be equal to the gradient amplitude before updating.

[0070] If the average gradient amplitude of the process path is equal to the average gradient amplitude before updating, it can be determined that the average gradient amplitude of the process path has reached the maximum value, the edge profile closest to the target of the overlapping region, and the passing points on the process path will not change even if they are updated again. The process path cannot be updated again, so the updating can be stopped, and the process path can be taken as the current path. If the average gradient amplitude of the process path is greater than or equal to a first threshold value, the process path can also be stopped updating and taken as the current path, wherein the first threshold value can be set as a large number. When the average gradient amplitude of the process path is greater than or equal to the first threshold value, the average gradient amplitude is large, and it can be considered that the process path has met the condition of the current path, the updating is stopped, and the process path is determined as the current path.

[0071] If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, it is known that the average gradient amplitude of the process path has not reached the maximum value, and the passing points on the process path can still change after the next update, and the process path can still be updated. A number threshold, i.e., a threshold of the number of updates, can be set to limit the maximum number of updates. If the number of updates reaches the number threshold but the current path is still not determined, it can be determined that the current concave point pair is not a correct concave point pair that can determine the segmentation path 10, and the current concave point pair can be excluded and stopped in time to reduce the influence of the interference data. If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is less than the number threshold, the process path can be continuously updated, and the method of updating the passing points according to the gradient information to obtain the process path can be returned to be executed. If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is equal to the number threshold, the method of determining the current concave point pair is returned to be executed.

[0072] In some embodiments, the number threshold can be set to 10-200 times, for example, 100 times. By setting the number threshold, the number of iterations can be prevented from being too large, which can cause the image segmentation to take too long and affect the subsequent process of material sorting. A relatively low number threshold can improve efficiency and reduce iteration time, and a relatively high number threshold can improve accuracy and avoid missed identification. The number threshold can be 100 times, which can ensure that the update finds the segmentation path and can avoid the case that the number is too small to find the segmentation path.

[0073] By updating the passing points according to the gradient information to obtain the process path, and screening the process path according to the gradient amplitude to determine the current path, the method can effectively reduce redundant data processing, timely eliminate interference data, improve the speed of determining the current path, and thus effectively improve the efficiency, so that the segmentation process of the image to be segmented has better real-time performance. In some embodiments, as shown in FIG. 1 1, step S141 can include: Figure 9

[0074] In step S1414, all concave point pairs are determined. The concave point pairs obtained in step S120 can be determined, and the concave points included in each concave point pair and the concave point information included in each concave point pair can be determined, which can include position information, gradient information, etc.

[0075] ​Step S1415: Connect each pair of concave points with a line, and select one or more passing points on each line to obtain one or more initial paths. Each pair of concave points can be a line segment within the image to be segmented. Each initial path uses the two concave points of its corresponding pair as endpoints. One or more points can be selected as passing points in each initial path for subsequent adjustment and updating. The initial path can be a straight line segment directly connecting pairs of concave points, or any line segment with the two concave points of the pair as endpoints.

[0076] Step S1416: Update the initial path based on gradient information to obtain one or more current paths. This can be done by updating the points passed through based on gradient magnitude and direction, ensuring the gradient magnitude of the updated points is greater than that of the points on the initial path. Connecting the updated points to their corresponding concave points yields the updated current path. Alternatively, the direction of the updated initial path can be determined based on its gradient direction, and the entire path can be updated based on its gradient magnitude, ensuring the gradient magnitude of the updated current path is greater than that of the initial path. Updating the initial path based on gradient information can be repeated multiple times, allowing for multiple updates to the initial path.

[0077] Based on the current path, all points on the current path can be determined. Since the final segmentation path 10 is the edge of the overlapping target image, segmentation path 10 starts from one concave point, passes through the interior of the image to be segmented, and ends at another concave point. In segmentation path 10, only the two ends are concave points; the remaining points are all inside the image to be segmented. Therefore, if the current path passes through concave points other than the current concave point pair, the current path cannot be segmentation path 10 and is ignored. If the current path does not pass through concave points other than the current concave point pair, the current path may be segmentation path 10, and therefore, the current path can be identified as a candidate path.

[0078] By connecting all pairs of concave points to determine the initial path and updating the path, candidate paths can be obtained based on each pair of concave points, avoiding omissions of candidate paths and effectively improving the accuracy of image segmentation. Furthermore, when applied to sorting devices, the method in this embodiment has simpler program execution logic, is applicable to various different devices, and has better versatility.

[0079] In some embodiments, such as Figure 10 As shown, step S1416, updating the initial path based on gradient information to obtain one or more current paths, may include:

[0080] Step S14161, updating the passing points according to the gradient information to obtain a process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before the updating. The updating is performed on each initial path, the passing points on the initial path are updated according to the gradient information, a perpendicular line is drawn on the initial path, the foot of the perpendicular line is located on the passing point, a pixel point with a gradient amplitude greater than the current passing point is found on the perpendicular line according to the gradient amplitudes of the points in the image, and the pixel point is determined as a new passing point. After the updating of all the passing points on the initial path is completed, the concave point pairs and the new passing points are connected according to the gradient directions of the passing points to obtain the process path. A straight line can also be drawn along the gradient direction of the passing point, a pixel point with a gradient amplitude greater than the current passing point is found on the straight line, and the pixel point is determined as a new passing point. After the updating of all the passing points on the initial path is completed, the concave point pairs and the new passing points are connected according to the gradient directions of the passing points to obtain the process path. The average gradient amplitude of the process path can be greater than the average gradient amplitude of the initial path before the updating, or can be equal to the average gradient amplitude before the updating.

[0081] If the average gradient amplitude of the process path is equal to the average gradient amplitude before the updating, or the average gradient amplitude of the process path is greater than or equal to the first threshold value, the updating is stopped, and the process path is taken as a current path. If the average gradient amplitude of the process path is equal to the average gradient amplitude before the updating, it can be determined that the average gradient amplitude of the process path has reached the maximum value, the passing points on the process path cannot be changed again after the updating, and the process path determined by the current concave point pair has been updated to the edge contour closest to the overlapping target region. Therefore, the updating can be stopped, and the process path is taken as a current path. The first threshold value can also be set to determine the current path, and the first threshold value can be set to a relatively large value. If the average gradient amplitude of the process path is greater than or equal to the first threshold value, it can be determined that the gradient amplitude of the process path has met the requirement of the current path, and the updating can be directly stopped, and the process path is determined as the current path.

[0082] If the average gradient amplitude of the process path is greater than the average gradient amplitude before the updating, and the number of times of updating is less than the number threshold value, the updating of the passing points according to the gradient information to obtain the process path is performed again. If the average gradient amplitude of the process path is greater than the average gradient amplitude before the updating, it indicates that the process path has not reached the maximum average gradient amplitude, and the passing points on the process path can be further updated to obtain a new process path. The number threshold value can be set to limit the number of times of updating. For the case that the number of times of updating does not reach the number threshold value and the average gradient amplitude of the process path is greater than the average gradient amplitude before the updating, it can be determined that the process path can still be updated, and the updating of the passing points according to the gradient information to obtain the process path is performed again to continue the updating of the process path.

[0083] For the case that the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is equal to the number threshold, the process path can have reached its maximum average gradient amplitude. Since the number of updates has reached the number threshold, the process path cannot be updated again, and therefore it cannot be determined whether the process path can be a current path. A second threshold can be set to determine whether the average gradient amplitude of the process path is greater than the second threshold to determine whether the process path can be determined as a current path. The second threshold can be less than or equal to the first threshold, so that the omission of part of the path can be effectively avoided in the process of determining the current path. If the average gradient amplitude of the process path is greater than or equal to the second threshold, it can be determined that the average gradient amplitude of the process path is large and close to the edge profile of the overlapping region target, and the process path can be determined as a current path for subsequent judgment and selection. If the average gradient amplitude of the process path is less than the second threshold, it can be known that the process path is far from the edge profile of the overlapping region target, and therefore the process path can be ignored and other process paths can be updated and judged.

[0084] By updating the passing points according to the gradient information to obtain the process path, and screening the process path according to the gradient amplitude and determining the current path, the interference data can be ignored in time and unnecessary data processing can be avoided, so that the speed of determining the current path can be effectively improved. By setting the first threshold, the judgment of the current path can be facilitated and the computing power can be saved. By setting the second threshold, the process path that is close to the edge profile of the overlapping region target but has reached the number threshold can be confirmed, the omission of the current path can be effectively avoided, the accuracy and comprehensiveness of the data can be ensured, and the segmentation path can have higher accuracy.

[0085] In some embodiments, as shown in FIG. 14B, step S142 of determining the segmentation path 10 based on the candidate path can further include: Figure 11

[0086] The segmentation path 10 is determined according to the number of candidate paths. Since the segmentation path 10 of the image to be segmented can have one or more paths, and the number of candidate paths can be greater than the final segmentation path 10, the candidate paths need to be screened. The segmentation path 10 can be determined in different ways according to the number of candidate paths.

[0087] If the number of candidate paths is equal to 1, the candidate path is determined as the segmentation path 10. In the case of one candidate path, only one candidate path meets the condition of the segmentation path 10, and the current candidate path can be directly determined as the segmentation path 10, and it can be determined that the image to be segmented contains two targets.

[0088] ​If the number of candidate paths is greater than 1, the segmentation path 10 is determined according to the gradient information of the candidate paths. If the number of candidate paths is greater than 1, the number of segmentation paths 10 can be one or more. For the case where the to-be-segmented image contains two targets, the number of segmentation paths 10 is one. For the case where the to-be-segmented image contains three or more targets, the number of segmentation paths 10 is greater than one. Therefore, the candidate paths need to be screened to determine the segmentation path 10. The segmentation path 10 can be determined according to the gradient information of the candidate paths. In this case, the candidate path with a larger gradient amplitude can be determined as the segmentation path 10.

[0089] Different segmentation path 10 determination methods are adopted according to different numbers of candidate paths, which can accurately determine the segmentation path 10 for different to-be-segmented images. For the case where the number of candidate paths is one, the segmentation path 10 is directly determined without the need for more data processing, which can effectively improve the efficiency.

[0090] In some embodiments, as shown in FIG. 14B, step S142 of determining the segmentation path 10 according to the gradient information of the candidate paths can include: Figures 12 to 14

[0091] The candidate path with an average gradient amplitude greater than or equal to a gradient threshold value is determined as the segmentation path 10. Since the larger the gradient amplitude, the more obvious the change of the image, and the more likely it is the edge of the image, for the candidate path with a larger average gradient amplitude, it can be determined that the candidate path is closer to the edge of the overlapping region of the multiple overlapping targets. Therefore, the gradient threshold value can be set. For the candidate path with an average gradient amplitude greater than or equal to the gradient threshold value, it can be determined as the segmentation path 10.

[0092] The candidate path with the largest average gradient amplitude is determined as the segmentation path 10. For the case where the to-be-segmented image contains two targets, the number of segmentation paths is one. Therefore, the candidate path with the largest average gradient amplitude can be determined as the segmentation path 10.

[0093] The candidate paths are sorted according to the average gradient amplitude from large to small, and the first N-1 candidate paths are recorded as the segmentation path 10, where N is the number of targets contained in the image. The target recognition can also be performed on the to-be-segmented image to determine the number N of targets in the to-be-segmented image. The bounding box of each target in each to-be-segmented image can be obtained through the target recognition algorithm. According to the number of bounding boxes, the number N of targets in the to-be-segmented image can be determined. For the N targets that overlap, N-1 segmentation paths are needed to segment the targets to make them independent. Therefore, the number of segmentation paths can be determined as N-1. The candidate paths can be sorted according to the average gradient amplitude from large to small. For the first N-1 candidate paths, they can be determined as the segmentation path 10.

[0094] ​By determining the segmentation path 10 based on the magnitude of the gradient, the correct segmentation path 10 can be accurately identified, redundant candidate paths can be discarded, and oversegmentation or incorrect segmentation can be avoided, thus effectively improving the accuracy of the image segmentation method.

[0095] In some embodiments, such as Figure 15 As shown, step S120, determining multiple concave points in the image to be segmented, may include:

[0096] Step S122: Based on the image to be segmented, perform image preprocessing to obtain a preprocessed image. Image preprocessing may include noise reduction, such as Gaussian smoothing of the acquired image to be segmented to remove jagged edges and prevent them from affecting concave point extraction, thus making the edges of the image to be segmented smoother. Image preprocessing may also include contour extraction, extracting the common external contours of multiple targets from the image to be segmented, which facilitates the calculation of contour point curvature and the extraction of concave points. Image preprocessing may also include binarization, such as... Figure 2 (b) Figure 2 As shown in (f), the image can be binarized. When contour processing is required for the image to be segmented, binarization can reduce the amount of data calculation in the contour extraction process and obtain a clearer contour.

[0097] Step S123: Determine the concave points in the image based on the preprocessed image. Determining concave points in the image based on the preprocessed image reduces the amount of data computation during concave point determination, improves the quality of concave point extraction, obtains more accurate concave point information, eliminates some data interference, and improves the accuracy of segmentation in subsequent steps.

[0098] Based on the same inventive concept, such as Figure 16As shown, this disclosure also provides a material sorting device 200, which includes: an image segmentation module 210 for executing the image segmentation method as described in any of the foregoing embodiments; and a material sorting module 220 for performing target recognition and sorting based on the target image obtained by the image segmentation module 210. The image segmentation module 210 can segment the image according to the image segmentation method of any of the foregoing embodiments, enabling rapid and accurate image segmentation and distinguishing multiple overlapping and nested targets in the image to be segmented. The material sorting module 220 may further include a target recognition unit and a material sorting unit, respectively used for target recognition based on the target image obtained by the image segmentation module 210 and for sorting the targets. Regarding the material sorting device in the above embodiments, the specific methods by which each module performs its operation have been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0099] Based on the same inventive concept, this disclosure also provides a computer-readable storage medium storing a program for performing the image segmentation method of any of the foregoing embodiments.

[0100] like Figure 17 As shown, one embodiment of this disclosure provides an electronic device 300. The electronic device 300 includes a memory 310, a processor 320, and an input / output (I / O) interface 330. The memory 310 is used to store instructions. The processor 320 is used to execute the image segmentation method of this disclosure embodiment by calling the instructions stored in the memory 310. The processor 320 is connected to both the memory 310 and the I / O interface 330, for example, via a bus system and / or other forms of connection mechanism (not shown). The memory 310 can be used to store programs and data, including the program for the image segmentation method involved in the embodiments of this disclosure. The processor 320 executes various functional applications and data processing of the electronic device 300 by running the program stored in the memory 310.

[0101] The processor 320 in the embodiments of the present disclosure can be implemented in at least one of a hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), and can be a combination of one or several of a central processing unit (CPU) or other forms of processing units having data processing and / or instruction execution capabilities.

[0102] The memory 310 in the embodiments of the present disclosure can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, a random access memory (RAM), a cache, and / or the like. The non-volatile memory can include, for example, a read-only memory (ROM), a flash memory, a hard disk drive (HDD), a solid-state drive (SSD), and / or the like.

[0103] In the embodiments of the present disclosure, the I / O interface 330 can be used to receive input instructions (such as digital or character information, and generate key signal inputs related to user settings and function control of the electronic device 300, etc.), and can also output various information to the outside (such as images or sounds, etc.). The I / O interface 330 in the embodiments of the present disclosure can include one or more of a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel, etc.

[0104] The present application uses specific words to describe the embodiments of the present application. As "one embodiment", "an embodiment", and / or "some embodiments" means a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that the "an embodiment" or "one embodiment" or "an alternative embodiment" mentioned in different places in the specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of the present application can be properly combined.

[0105] As used in the description of the application herein, the articles "a", "an", "the", and "at least one" are intended to mean that there is one or more (i.e., one or more than one) of the elements. As used in the description of the application herein, the term "includes" means includes but is not limited to, and the transition term "comprising" means including, but not limited to. As used in the description of the application herein, the term "consisting essentially of means including the elements listed and any other elements that do not materially affect the basic and novel characteristics of the application.

[0106] It should also be noted that all the diverse features described above in the context of the embodiments of the application may, and often are, combined in any permutation and combination. It is intended that the application embrace all such permutations and combinations. Furthermore, it is intended that the features described above in the context of the embodiments of the application are not limited to the embodiments described above, but are applicable to any embodiment of the application. It is intended that the features described above in the context of the embodiments of the application are not limited to the embodiments described above, but are applicable to any embodiment of the application. It is intended that the features described above in the context of the embodiments of the application are not limited to the embodiments described above, but are applicable to any embodiment of the application. It is intended that the features described above in the context of the embodiments of the application are not limited to the embodiments described above, but are applicable to any embodiment of the application. It is intended that the features described above in the context of the embodiments of the application are not limited to the embodiments described above, but are applicable to any embodiment of the application. It is intended that the features described above in the context of the embodiments of the application are not limited to the embodiments described above, but are applicable to any embodiment of the application.

Claims

1. A method of segmenting an image to be segmented, wherein, The image to be segmented includes a plurality of target images, comprising: According to the image to be segmented, the gradient information of each point in the image to be segmented is determined, wherein the gradient information includes gradient amplitude; According to the image to be segmented, a plurality of concave points in the image to be segmented are determined; The plurality of concave points are paired two by two to obtain one or more concave point pairs; Based on one or more concave point pairs, one or more candidate paths are confirmed according to the gradient information; Based on the candidate path, a segmentation path is determined; Wherein, based on one or more concave point pairs, one or more candidate paths are confirmed according to the gradient information, comprising: determining all concave point pairs; connecting each concave point pair, and taking one or more passing points on each connecting line to obtain one or more initial paths; updating the initial path based on the gradient information to obtain one or more current paths; if the current path does not pass through other concave points except the current concave point pair, the current path is determined as a candidate path; Wherein, based on the gradient information, the initial path is updated to obtain one or more current paths, comprising: updating the passing point according to the gradient information to obtain a process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before updating; if the average gradient amplitude of the process path is greater than the average gradient amplitude before updating, and the number of updates is less than the number threshold, return to execute the updating of the passing point according to the gradient information to obtain the process path; if the average gradient amplitude of the process path is equal to the average gradient amplitude before updating, or the average gradient amplitude of the process path is greater than or equal to the first threshold, stop updating, and take the process path as a current path; According to the segmentation path, the image to be segmented is segmented to obtain a plurality of target images.

2. The image segmentation method of claim 1, wherein, The plurality of concave points are paired two by two to obtain one or more concave point pairs, comprising: Determine the corresponding concave region of each concave point; Pair each concave point with a concave point belonging to other concave regions to obtain a concave point pair.

3. The image segmentation method of claim 1, wherein, Based on one or more concave point pairs, one or more candidate paths are confirmed according to the gradient information, further comprising: Determine the current concave point pair, wherein the current concave point pair includes a first concave point and a second concave point, the first concave point is the concave point with the largest gradient amplitude in the current all concave points, and the second concave point is the concave point with the largest gradient amplitude in the concave point paired with the first concave point; Connecting the first concave point and the second concave point, and taking one or more passing points on the connecting line to obtain an initial path; Based on the gradient information, the initial path is updated to obtain a current path; If the current path passes through other concave points except the current concave point pair, ignore the second concave point in the current concave point pair and return to execute the determination of the current concave point pair; If the current path does not pass through other concave points except the current concave point pair, the current path is determined as a candidate path, and the segmentation path is determined based on the candidate path.

4. The image segmentation method of claim 3, wherein, The initial path is updated based on the gradient information to obtain a current path, further comprising: If the average gradient amplitude of the process path is greater than the average gradient amplitude before updating, and the number of updating times is equal to the number threshold, it is determined that the current concave point pair.

5. The image segmentation method of claim 1, wherein, The determining one or more candidate paths based on the gradient information further comprises: If the current path passes through other concave points other than the current concave point pair, the current path is ignored.

6. The image segmentation method of claim 5, wherein, The updating the initial path based on the gradient information to obtain one or more current paths further comprises: If the average gradient amplitude of the process path is greater than the average gradient amplitude before updating, and the number of updating times is equal to the number threshold, it is determined whether the average gradient amplitude of the process path is greater than a second threshold, wherein the second threshold is less than or equal to the first threshold. If the average gradient amplitude of the process path is greater than or equal to the second threshold, the process path is determined as one of the current paths. If the average gradient amplitude of the process path is less than the second threshold, the process path is ignored.

7. The image segmentation method of claim 1, wherein, The determining the segmentation path based on the candidate paths further comprises: If the number of candidate paths is equal to 1, the candidate path is determined as the segmentation path. If the number of candidate paths is greater than 1, the segmentation path is determined according to the gradient information of the candidate paths.

8. The image segmentation method of claim 7, wherein, The determining the segmentation path according to the gradient information of the candidate paths comprises: the candidate path with an average gradient amplitude greater than or equal to a gradient threshold is determined as the segmentation path; or the candidate path with the maximum average gradient amplitude is determined as the segmentation path; or the first N-1 candidate paths are recorded as the segmentation path according to the average gradient amplitudes of the candidate paths in descending order, wherein N is the number of targets contained in the image.

9. The image segmentation method of claim 1, wherein, The determining a plurality of concave points in the image to be segmented comprises: determining a candidate point according to the curvature of a point in the image; if the depth information of the candidate point is greater than or equal to a depth threshold, the candidate point is determined as a concave point; if the depth information of the candidate point is less than the depth threshold, the candidate point is ignored.

10. The image segmentation method of claim 1, wherein, The determining a plurality of concave points in the image to be segmented comprises: performing image preprocessing on the image to be segmented to obtain a preprocessed image; determining the concave points in the image according to the preprocessed image.

11. A material sorting apparatus wherein, The material sorting device comprises: an image segmentation module configured to perform the image segmentation method according to any one of claims 1-10; a material sorting module configured to perform target identification and sorting according to the target image obtained by the image segmentation module.

12. A computer readable storage medium storing a program for performing the image segmentation method according to any one of claims 1-10.

Images