To-be-segmented image segmentation method, material sorting device and readable storage medium

Abstract

The invention relates to the field of material sorting, in particular to a to-be-segmented image segmentation method, a material sorting device and a readable storage medium. The to-be-segmented image segmentation method comprises the following steps: determining gradient information of each point in a to-be-segmented image according to the to-be-segmented image; determining a plurality of concave points in the to-be-segmented image according to the to-be-segmented image; pairing the plurality of concave points in pairs to obtain one or more concave point pairs; determining a segmentation path of the to-be-segmented image according to the concave point pairs and gradient information of each point in the to-be-segmented image; and segmenting the to-be-segmented image according to the segmentation path to obtain a plurality of target images. Through the method disclosed by the invention, the actual edges of a plurality of overlapped targets can be rapidly and accurately determined, the to-be-segmented image can be segmented more accurately, rapid detection can be performed in actual application, better real-time performance is achieved, and the image segmentation efficiency and the segmentation accuracy are effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a method for segmenting an image to be segmented, a material sorting device, and a readable storage medium. Background Art

[0002] During the material sorting process, materials often overlap, so the material image needs to be segmented to determine the position of each material. In the case of serious mutual occlusion between materials, the overlapping part of the materials is too large, resulting in the overlapping area between the materials in the image being too deeply nested, which seriously affects the subsequent recognition and positioning of the material boundaries and reduces the accuracy of segmentation. The current image segmentation method for materials cannot accurately perform image segmentation in such cases where materials occlude each other and the overlapping area is too deeply nested, and the accuracy of the segmentation results is poor. In addition, some current image segmentation methods for materials require a large amount of labeled data for model training. The computational complexity of image segmentation is high, the efficiency is low, and the robustness is poor, which cannot meet the needs of real-time segmentation. Summary of the invention

[0003] To overcome the problems existing in the related art, an exemplary embodiment of the present disclosure provides a method for segmenting an image to be segmented, wherein the image to be segmented includes multiple target images, including: determining gradient information of each point in the image to be segmented according to the image to be segmented, wherein the gradient information includes gradient amplitude; determining multiple concave points in the image to be segmented according to the image to be segmented; pairing the multiple concave points in pairs to obtain one or more concave point pairs; determining a segmentation path of the image to be segmented according to the concave point pairs and the gradient information of each point in the image to be segmented; and segmenting the image to be segmented according to the segmentation path to obtain multiple target images.

[0004] In some embodiments, pairing the plurality of concave points in pairs to obtain one or more concave point pairs includes: determining a concave area corresponding to each of the concave points; and pairing each concave point with concave points belonging to other concave areas to obtain concave point pairs.

[0005] In some embodiments, determining the segmentation path of the image to be segmented based on the concave point pairs and the gradient information of each point in the image to be segmented includes: based on one or more concave point pairs, confirming one or more candidate paths according to the gradient information; and determining the segmentation path based on the candidate paths.

[0006] In some embodiments, the step of confirming one or more candidate paths based on the gradient information based on one or more concave point pairs includes: determining a current concave point pair, wherein the current concave point pair includes a first concave point and a second concave point, the first concave point being a concave point with a maximum gradient amplitude among all current concave points, and the second concave point being a concave point with a maximum gradient amplitude among concave points paired with the first concave point; connecting the first concave point and the second concave point, taking one or more passing points on the connecting line to obtain an initial path; updating the initial path based on the gradient information to obtain a current path; if the current path passes through other concave points other than the current concave point pair, ignoring the second concave point in the current concave point pair and returning to execute the step of determining the current concave point pair; if the current path does not pass through other concave points other than the current concave point pair, determining the current path as a candidate path and executing the step of determining the segmented path based on the candidate path.

[0007] In some embodiments, updating the initial path based on the gradient information to obtain the current path includes: updating the passing points according to the gradient information to obtain a process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before the update; if the average gradient amplitude of the process path is equal to the average gradient amplitude before the update, or the average gradient amplitude of the process path is greater than or equal to a first threshold, stopping the update and taking the process path as the current path; if the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is less than the number threshold, returning to execute the updating of the passing points according to the gradient information to obtain the process path; if the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is equal to the number threshold, returning to execute the determination of the current concave point pair.

[0008] In some embodiments, based on one or more concave point pairs and according to the gradient information, one or more candidate paths are confirmed, including: determining all concave point pairs; connecting each concave point pair, taking one or more passing points on each connecting line to obtain one or more initial paths; updating the initial path based on the gradient information to obtain one or more current paths; if the current path passes through other concave points other than the current concave point pair, ignoring the current path; if the current path does not pass through other concave points other than the current concave point pair, determining the current path as a candidate path.

[0009] In some embodiments, the updating of the initial path based on the gradient information to obtain one or more current paths includes: updating the passing points according to the gradient information to obtain a process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before the update; if the average gradient amplitude of the process path is equal to the average gradient amplitude before the update, or the average gradient amplitude of the process path is greater than or equal to a first threshold, stopping the update and taking the process path as one of the current paths; if the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is less than the number threshold, returning to execute the updating of the passing points according to the gradient information to obtain the process path; if the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is equal to the number threshold, confirming whether the average gradient amplitude of the process path is greater than a second threshold, wherein the second threshold is less than or equal to the first threshold; if the average gradient amplitude of the process path is greater than or equal to the second threshold, determining the process path as one of the current paths; if the average gradient amplitude of the process path is less than the second threshold, ignoring the process path.

[0010] In some embodiments, determining the segmentation path based on the candidate paths further includes: if the number of the candidate paths is equal to 1, determining the candidate paths as segmentation paths; if the number of the candidate paths is greater than 1, determining the segmentation paths according to the gradient information of the candidate paths.

[0011] In some embodiments, determining the segmentation path based on the gradient information of the candidate paths includes: determining the candidate paths whose average gradient amplitude is greater than or equal to a gradient threshold as the segmentation path; or, determining the candidate paths with the largest average gradient amplitude as the segmentation path; or, sorting the candidate paths from large to small according to their average gradient amplitudes, and recording the first N-1 candidate paths as the segmentation path, where N is the number of targets contained in the image.

[0012] In some embodiments, determining multiple concave points in the image to be segmented based on the image to be segmented includes: determining candidate points based on the point curvature in the image; if the depth information of the candidate point is greater than or equal to a depth threshold, determining it as the concave point; if the depth information of the candidate point is less than the depth threshold, ignoring the candidate point.

[0013] In some embodiments, determining multiple concave points in the image to be segmented according to the image to be segmented includes: performing image preprocessing according to the image to be segmented to obtain a preprocessed image; and determining concave points in the image according to the preprocessed image.

[0014] In a second aspect, the present disclosure further provides a material sorting device, wherein the material sorting includes: an image segmentation module, used to execute the image segmentation method as described in any of the above embodiments; a material sorting module, used to perform target identification and sorting according to the target image obtained by the image segmentation module.

[0015] In a third aspect, the present disclosure further provides a computer-readable storage medium, which stores the following program, and the program is used to execute the image segmentation method as described in any of the above embodiments.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0017] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by identifying concave points in the image to be segmented, determining the path of the concave point connection lines and segmenting the image in combination with the gradient information of the image, accurate identification and segmentation can be performed for situations where the overlapping area is large and the image is severely nested, and the segmentation path is the contour edge of the overlapping area between multiple targets. Through the method of the present disclosure, the actual edges of multiple overlapping targets can be determined quickly and accurately, the image to be segmented can be segmented more accurately, and rapid detection can be performed in practical applications, with better real-time performance, effectively improving the efficiency and accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present disclosure may be better understood by describing exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which:

[0019] Figure 1 It is a schematic diagram of the structure of a sorting device according to an exemplary embodiment of the disclosure;

[0020] Figure 2 is an image to be segmented according to an exemplary embodiment of the disclosure;

[0021] Figure 3 It is a flowchart of a method for segmenting an image to be segmented according to an exemplary embodiment of the disclosure;

[0022] Figure 4 is a schematic flow chart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0023] Figure 5 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0024] Figure 6 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0025] Figure 7 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0026] Figure 8 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0027] Fig. 9 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0028] Fig.10 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0029] Fig.11 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0030] Fig.12 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0031] Fig.13 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0032] Fig.14 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0033] Fig.15 is a flowchart of a method for segmenting an image to be segmented according to another exemplary embodiment of the present disclosure;

[0034] Fig.16 is a schematic block diagram of a material sorting system according to an exemplary embodiment of the disclosure;

[0035] Fig.17 It is a schematic block diagram of an electronic device according to an exemplary embodiment of the disclosure. DETAILED DESCRIPTION

[0036] The specific embodiments of the present invention will be described below. It should be noted that in the specific description of these embodiments, in order to provide a concise description, it is impossible for this specification to provide a detailed description of all the features of the actual embodiments. It should be understood that in the actual implementation of any embodiment, just as in the process of any engineering project or design project, in order to achieve the specific goals of the developer and to meet system-related or business-related restrictions, various specific decisions are often made, and this will also change from one embodiment to another. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the content disclosed by the present invention, some changes such as design, manufacturing or production based on the technical content disclosed in this disclosure are just conventional technical means, and should not be understood as insufficient content of this disclosure.

[0037] Unless otherwise defined, the technical or scientific terms used in the claims and the specification shall have the usual meaning understood by persons with ordinary skills in the technical field to which the invention belongs. The words "first", "second" and similar words used in the patent application specification and the claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "One" or "one" and other similar words do not indicate a quantitative limitation, but indicate the existence of at least one. "Include" or "comprises" and other similar words mean that the elements or objects appearing before "include" or "comprises" include the elements or objects listed after "include" or "comprises" and their equivalent elements, and do not exclude other elements or objects. "Connected" or "connected" and other similar words are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0038] The image segmentation method provided in the present disclosure can be applied to a material sorting device 100, and the material sorting device 100 can be used to sort materials, such as Figure 1 As shown, it may include a feeding mechanism 110, a transmission mechanism 120, a detection mechanism 130 and a sorting device 140. The feeding mechanism 110 is used to feed the material to be sorted into the transmission mechanism 120. The transmission mechanism 120 may be a structure such as a conveyor belt or a chute, which is used to transport the material to be sorted fed by the feeding mechanism 110. The detection mechanism 130 is used to detect the material transported on the transmission mechanism 120 to detect whether the material is the material to be rejected; the material to be rejected refers to the material to be separated by the sorting equipment. The material to be rejected may be the required material or the unrequired material, as long as the material can be sorted. The sorting device 140 is used to reject the material to be rejected. The material sorting equipment 100 can be used for the sorting of ores, and can also be used for tasks such as food sorting or waste sorting.

[0039] In the process of material sorting, due to the irregular shapes and diverse surface textures of the materials themselves, as well as the poor imaging effect, such as Figure 2 (a) Figure 2 As shown in (e), the target materials to be sorted overlap each other. When the overlapping area is too large, multiple materials are seriously blocked. In particular, when the material is ore, the ore has an irregular shape and a complex surface texture. In the process of material sorting, the feeding mechanism 110 feeds the ore into the transmission mechanism 120. During the feeding process, the ore is prone to rolling, resulting in the ore overlapping each other, and the overlapping area may be large. In the acquired image, multiple targets are included, and the overlapping areas between the targets are too deeply nested, making it difficult to identify and locate the boundaries of the materials, resulting in poor accuracy when segmenting the overlapping materials.

[0040] In some related technologies, instance segmentation technology is used to segment multiple overlapping or adhered materials by training machine learning models. However, since the instance segmentation technology based on machine learning needs to establish a model for the target material and requires a large amount of high-quality labeled data for data training, it is difficult to obtain high-quality labeled data in application scenarios such as ore classification and garbage classification. The quality of model training is mixed, and due to the high complexity of calculation, the material segmentation method based on machine learning requires a longer response time and cannot meet the needs of real-time segmentation in actual production.

[0041] In other related technologies, concave points are obtained through concave point detection, and the concave points are paired and connected to obtain the segmentation line, so that image segmentation can be performed quickly. However, in the case of large-area overlap and serious nesting of target images, the concave points are paired and directly connected as the segmentation line, and the accuracy of segmentation is poor. In the case of large-area overlap of targets, in the image, target A is partially nested in target B. Segmentation based on concave point pairing and connecting lines will cause the part of target A that is nested in target B to be attributed to target B after segmentation. The segmentation result is inconsistent with the actual situation, resulting in target B still containing target A after segmentation. In the subsequent recognition and sorting process, it is easy to have wrong recognition and wrong sorting, resulting in low accuracy of material sorting. In addition, in the case of large-area overlap of some targets, it may cause the concave point recognition to fail to accurately obtain the concave points and fail to obtain the concave points, resulting in wrong segmentation and low accuracy of image segmentation.

[0042] To solve the above problems, Figure 3 As shown, the present disclosure provides a method for segmenting an image to be segmented, wherein the image to be segmented includes multiple target images, and may include: steps S110 to S150, and the above steps are specifically described below:

[0043] Step S110, according to the image to be segmented, determine the gradient information of each point in the image to be segmented, wherein the gradient information includes the gradient amplitude. The image to be segmented may be an image including multiple overlapping targets, and the image may be acquired by an image acquisition device, and then the image of the multiple overlapping targets is extracted according to the image, which is the image to be segmented. The gradient information of the image can reflect the grayscale change or color change of the image to be segmented. The gradient information may include the gradient amplitude, and the gradient amplitude may be the gradient size at each pixel point in the image to be segmented. The larger the gradient amplitude, the more intense the change in the image, which may correspond to features such as edges in the image. Therefore, in the process of determining the segmentation path 10 of the image to be segmented, the edges of the overlapping targets in the overlapping part of the image to be segmented can be found according to the gradient amplitude. The gradient information in the image to be segmented can be acquired by an image detection algorithm, thereby obtaining the gradient amplitude and gradient direction of the image. The image detection algorithm may include a Sobel operator, a Scharr operator, or a Laplace operator, etc.

[0044] Step S120, based on the image to be segmented, determine multiple concave points in the image to be segmented. The concave point can be the lowest point on the edge of the continuous pattern in the image to be segmented that is concave downward. The image can be firstly contour extracted, and the concave points can be determined based on the image by a concave point recognition algorithm. The concave points in the image can also be identified by a curvature-based method, and the curvature information of each point on the contour of the image can be calculated, and the concave points in the image can be determined by screening based on the curvature information. For multiple overlapping materials, the overlapping areas in their images are nested too deeply, which may make it difficult to identify the concave points. In the image segmentation method disclosed in the present invention, during the process of concave point recognition, the relevant parameters of concave point recognition can be adjusted to identify multiple candidate concave points, and then the candidate concave points can be screened to determine the final concave points.

[0045] In some embodiments, Figure 4As shown, step S120, according to the image to be segmented, determines multiple concave points in the image to be segmented, which may include: step 121, determining candidate points according to the point curvature in the image. The contour of the image to be segmented can be extracted, and the point curvature of each point on the contour of the image to be segmented can be calculated. The point curvature threshold can be set, and the contour points greater than the point curvature threshold can be determined as candidate points. Due to the overlapping areas of the multiple materials overlapping each other, the overlapping areas in the image are nested too deeply, and there may be concave points with small curvatures that are difficult to identify. Therefore, a smaller point curvature threshold can be set to ensure that the concave points in the image can be determined as candidate points. To calculate the contour point curvature, a point can be first extracted as the point C to be checked, and two adjacent points of the point C to be checked are selected, and the two adjacent points are the predecessor point P and the successor point N. The vectors corresponding to CP and PN are calculated respectively, and the cross product of the two vectors is calculated, which is recorded as CrossProduct; then the modulus lengths of the two vectors are calculated respectively, which are recorded as CPL and PNL respectively; the tangent value of the vector angle between CP and PN is calculated, which is recorded as TanTheta. Finally, the curvature of the point to be checked in the image is calculated according to the formula abs(4.0 / tan(TanTHeta / 2.0)), the curvature of each point on the contour of the image to be segmented is obtained, and the candidate point is determined according to the curvature threshold.

[0046] If the depth information of the candidate point is greater than or equal to the depth threshold, it is determined to be a concave point. If the depth information of the candidate point is less than the depth threshold, the candidate point is ignored. The depth information of the candidate point is the shortest distance between the candidate point and the convex hull. The depth information of the candidate point can also be the vertical distance between the candidate point and the two endpoints of the concave area where it is located, wherein the concave area is the area where the depression occurs in the image to be segmented, each concave area can include multiple candidate points, and the convex hull can be a convex polygon circumscribed by the contour of the image to be segmented. The position of the candidate point can be determined according to the information of the candidate point, and a perpendicular line is drawn from the candidate point to the edge of the convex hull, and the perpendicular distance is the depth of the concave point; or the concave area where the candidate point is located can be first determined, the endpoints of the concave area are connected, and then a perpendicular line is drawn from the candidate point to the endpoints of the concave area, and the perpendicular distance is the depth of the concave point. A depth threshold can be set, and the depth threshold can be the minimum depth value of the concave point. When the depth of the candidate point is greater than or equal to the depth threshold, the candidate point can be determined as a concave point. When the candidate point is less than the depth threshold, it means that the candidate point is not a concave point and the candidate point is ignored.

[0047] Determining candidate points based on point curvature and setting a smaller point curvature threshold can ensure that all concave points in the image to be segmented can be identified as candidate points, which can improve the accuracy of concave point identification. The selected candidate points may also include other points besides concave points. Therefore, by judging the depth information of the candidate points, the candidate points with depth greater than or equal to the depth threshold are retained as concave points, reducing the amount of data, thereby making subsequent operations such as concave point pairing faster. At the same time, interference points can be eliminated and invalid data can be reduced, thereby improving the efficiency of subsequent operations such as concave point pairing.

[0048] Step S130, multiple concave points are paired in pairs to obtain one or more concave point pairs. The concave points in the image can be paired in pairs, so that each concave point and other concave points form a concave point pair, each concave point can be paired multiple times, and each concave point can be paired into multiple concave point pairs.

[0049] In some embodiments, all the concave points in the image to be segmented can be paired separately, so that each concave point is paired with all other concave points to obtain a concave point pair, and each concave point can be paired multiple times. By pairing all the concave points, it is possible to ensure that the concave points in the image to be segmented can avoid data omission in the subsequent process of determining the segmentation path 10 according to the concave point pairs and gradient information, and it is possible to traverse all the concave points and their concave point pairs to determine the segmentation path 10, so as to ensure the accuracy of the image.

[0050] In other embodiments, all the concave points may be partially paired, or each concave point may be paired with other concave points based on factors such as the position and distance of the concave points in the image to be segmented, thereby excluding some pairs of concave points that are obviously not the endpoints of the segmentation path, further reducing the amount of calculation and improving the accuracy and efficiency of image segmentation.

[0051] Step S140, determining the segmentation path 10 of the image to be segmented according to the concave point pair and the gradient information of each point in the image to be segmented. The segmentation path 10 can be a line in the image, with the starting point and the end point being different concave points in the image, and the segmentation path 10 is the edge of the overlapping materials in the image. The concave point pair can be taken, and the two concave points are respectively used as the starting point and the end point of the segmentation path 10, and the concave point pair is connected. According to the gradient information of each point in the image, the concave points in the image are connected and adjusted, so as to obtain the segmentation path 10 of the image. The gradient can be the rate of change and direction of the pixel value in the image, and the gradient information can include the gradient amplitude and the gradient direction. The gradient amplitude can represent the grayscale change rate or color change rate of the image at the current point, and the gradient direction can represent the direction in which the image changes fastest at the current point. Due to the nesting and large-area overlap of the target images, the method of directly connecting the concave points to segment the image will result in a target part being segmented and belonging to another target, resulting in inaccurate segmentation; the method of determining the segmentation path 10 based on the gradient information, because the gradient information can reflect the grayscale change rate and direction of the image, and when the targets are nested and overlapped in a large area, the grayscale change is obvious at the junction of different target images, that is, the edge contour of the overlapping area. According to the gradient information, the edge contour of the overlapping area can be quickly and accurately determined and determined as the segmentation path 10, which can make the segmentation more accurate and the segmentation path 10 closer to the edge contour of the actual overlapping area. Depending on the number of materials in the image, there can be one or more segmentation paths 10. For an image including two different materials, there is only one segmentation path 10; for an image including three or more different materials, there can be multiple segmentation paths 10.

[0052] Step S150, segment the image to be segmented according to the segmentation path 10 to obtain multiple target images. The overlapping material images can be segmented according to the segmentation path 10, such as Figure 2 (d) Figure 2 As shown in (h), each material in the image is separated from each other.

[0053] By the image segmentation method disclosed in the present invention, by determining the concave points in the image to be segmented and pairing them into concave point pairs, the position where the material overlaps and is severely blocked in the image to be segmented can be determined more quickly, and the recognition and response speed can be faster. By obtaining the gradient information of each point in the image to be segmented, the grayscale change rate or color change rate of the image, as well as the direction in which the grayscale change or color change is fastest at each point of the image, can be obtained. According to the gradient information, the area where the material overlaps and is severely blocked can be more accurately located. By combining the concave points and the gradient information to determine the segmentation path 10 of the image, the image can be accurately segmented when the material overlaps, is nested, and is severely blocked. While improving the segmentation speed of the material overlapping image, the accuracy of image segmentation can be effectively improved, and the segmentation efficiency can be better. The segmentation path 10 is not obtained by directly connecting the concave points, but can relatively accurately identify the edge of the target contour of the overlapping area. The concave point detection can quickly obtain the possible starting and ending positions of the segmentation line, avoiding a large number of invalid calculations. Then, according to the gradient information, the accurate segmentation line can be obtained in combination with the concave point information, which is closer to the edge of the target contour of the overlapping area, and has a higher segmentation accuracy. In the process of sorting materials, image segmentation can be performed in real time, which is more efficient and can improve the accuracy of sorting.

[0054] In some embodiments, Figure 5 As shown, step S130, pairing a plurality of concave points in pairs to obtain one or more concave point pairs, may include step S131 and step S132.

[0055] Step S131, determining the concave area corresponding to each concave point. The concave area may be a concave area in the image to be segmented. For the image to be segmented, including multiple overlapping or occluded target images, the concave area may be located at a position where different target images overlap. The concave point is located in the concave area, and there may be multiple concave points in one concave area.

[0056] Step S132, pairing each concave point with a concave point belonging to another concave area to obtain a concave point pair. According to the concave points and the concave areas corresponding to them, the concave points belonging to different concave areas can be paired to obtain concave point pairs. Since when the target images overlap, the overlapping positions on both sides will form two different concave areas, therefore, the segmentation path 10 cannot be determined according to two concave points belonging to the same area. When multiple targets overlap, their concave points must belong to different concave areas, therefore, there is no need to pair two concave points belonging to the same concave area.

[0057] Through step 131 and step 132, the concave points can be distinguished according to the concave areas where they are located, and then the concave points belonging to different concave areas can be paired to obtain one or more concave point pairs, and the concave points can be screened and classified according to the concave areas. Since the concave points of multiple target images must belong to different concave areas when they overlap, in this embodiment, the concave points belonging to the same concave area are avoided from being paired into concave point pairs, which can effectively reduce the amount of data and the amount of data calculation for subsequent image segmentation, thereby reducing the actual required for image segmentation, and can improve the calculation speed and efficiency of image segmentation, so that the segmentation method of the image to be segmented has better real-time performance when applied to material sorting equipment.

[0058] In some embodiments, Figure 6 As shown, step S140, determining the segmentation path 10 of the image to be segmented according to the concave point pairs and the gradient information of each point in the image to be segmented, may include:

[0059] Step S141, based on one or more concave point pairs, one or more candidate paths are confirmed according to the gradient information. The candidate paths can be determined by combining the concave point pairs with the gradient information, and the concave point pairs can be connected, and each point on the concave point pair connection line can be adjusted according to the gradient information. The direction of the fastest grayscale change at each point on the concave point pair connection line can be determined by the gradient direction of each point in the image, so as to adjust the concave point pair connection line along the direction of the fastest grayscale change. At the same time, the gradient amplitude of each point in the image can be used to find a point with a larger gradient amplitude in the direction of the fastest grayscale change, so as to adjust the concave point pair connection line. The concave point pair connection line can be adjusted multiple times, and the final result after adjustment is used as the candidate path of the image, so as to determine the candidate path of the image.

[0060] Step S142, based on the candidate paths, determine the segmentation path 10. One or more candidate paths can be determined based on the concave point pairs and the gradient information, and the candidate paths can be screened to determine the final segmentation path 10, which is the edge of the overlapping part of the final target image, such as Figure 2 (c) Figure 2 As shown in (g), there can be one segmentation path 10 or more. For an image to be segmented including two target images, there can be one segmentation path 10; for an image to be segmented including more than two target images, there can be more than one segmentation path 10.

[0061] Through step S141 and step S142, one or more candidate paths can be confirmed according to the gradient information of the concave point pairs and each point in the image. The candidate paths are not directly connected to the concave points, but are obtained by adjusting the concave point connection according to the gradient information, which has higher accuracy. By screening the candidate paths, the final segmentation path 10 is determined. The segmentation path 10 is the actual edge of multiple targets at overlapping or nested positions. The specific edge contour of each target in the image to be segmented can be determined, making the image segmentation more accurate, effectively improving the accuracy of image segmentation, and effectively avoiding the situation of misidentification in the actual material sorting process, with higher sorting accuracy.

[0062] In some embodiments, Figure 7 As shown, step S141 confirms one or more candidate paths based on one or more concave point pairs and according to gradient information, and may include: steps S1411 to S1413.

[0063] Step S1411, determining the current concave point pair, wherein the current concave point pair includes a first concave point and a second concave point, the first concave point is the concave point with the largest gradient amplitude among all the current concave points, and the second concave point is the concave point with the largest gradient amplitude among the concave points paired with the first concave point. All the concave points can be sorted in descending order according to the gradient amplitude to determine the current concave point pair, and the concave point with the largest gradient amplitude among all the current concave points can be selected as the first concave point, and according to the concave point pair, the concave point with the largest gradient amplitude among the concave points paired with the first concave point can be selected as the second concave point. Since the magnitude of the gradient amplitude can represent the rate of change of the image, the larger the gradient amplitude, the more obvious the change of the image at the current point. Therefore, in the process of determining the concave point pair, the first concave point and the second concave point with the larger gradient amplitude can be selected, and the path determined by the concave point with the larger gradient amplitude is the most likely to be the segmentation path. In particular, when the concave points belonging to different concave areas are paired, the possibility of the above-mentioned concave point pair being the endpoint of the segmentation path can be further determined.

[0064] Step S1412, connect the first concave point and the second concave point, select one or more passing points on the connecting line, and obtain the initial path. The initial path can be a line segment inside the image to be segmented, and a line segment can be determined with the first concave point as the starting point and the second concave point as the end point, and the line segment is determined as the initial path. One or more points can be taken as passing points on the initial path, so that the initial path can be adjusted and updated later. The initial path can be a straight line segment directly connected to the first concave point and the second concave point, or it can be any line segment with the first concave point and the second concave point as the end points. The passing points can select a fixed value, and the number can be selected from 3 to 20, such as 3, 5, and 10, and can be set equidistantly. The passing points can also determine the value according to the length of the initial path, and the distance can be 1 to 10 cm, or it can be a distance of 3 to 20 pixels, for example, one passing point is selected for every 5 or 10 pixel distances. The more the passing points are selected, the more points can be adjusted in the subsequent adjustment, and the relative accuracy is improved; the fewer the passing points are selected, the less the calculation amount can be reduced and the calculation efficiency can be improved.

[0065] Step S1413, based on the gradient information, the initial path is updated to obtain the current path. The passing points of the initial path can be updated based on the gradient amplitude and gradient direction to obtain the updated passing points, so that the gradient amplitude of the updated passing points is greater than the gradient amplitude of the passing points of the initial path, and the updated passing points are connected with the first concave point and the second concave point to obtain the updated current path. The direction of the updated initial path can also be determined based on the gradient direction of the initial path, and the path can be updated as a whole based on the gradient amplitude of the initial path, so that the gradient amplitude of the updated current path is greater than the gradient amplitude of the initial path. The update of the initial path based on the gradient information can be repeated multiple times, and the initial path can be updated multiple times.

[0066] According to the current path, all the points on the current path can be determined. Since the final segmentation path 10 is the edge of the overlapping target image, the segmentation path 10 starts from a concave point, passes through the inside of the image to be segmented, and finally ends at another concave point. In the segmentation path 10, only the points at the two ends are concave points, and the remaining points are all inside the image to be segmented. Therefore, if the current path passes through other concave points other than the current concave point pair, the current path cannot be the segmentation path 10, and the second concave point in the current concave point pair is ignored and the current concave point pair is determined. If the current path does not pass through other concave points other than the current concave point pair, the current path may be the segmentation path 10, so the current path can be determined as a candidate path, and the segmentation path 10 is determined based on the candidate path.

[0067] In some embodiments, the image to be segmented may include multiple targets. When the segmentation path is determined according to steps S1411 to S1413 and step S142, a segmentation path 10 can be obtained. However, the image to be segmented may include three or more targets, so two or more segmentation paths 10 need to be determined. Therefore, in this case, the number of targets can be determined by target detection, and the number of required segmentation paths can be determined according to the number of targets. When the number of targets is greater than 2, the number of required segmentation paths is greater than 1. The number of required segmentation paths can also be determined according to the number of concave areas with a deeper concave degree in the image. The degree of concaveness of the concave area can be determined according to the edge curvature of the concave area. For concave areas with a larger edge curvature, it can be considered that the current concave area contains the endpoints of the segmentation path. The degree of concaveness of the concave area can also be determined by the depth of the concave point. The depth is the distance between the concave point and the line connecting the endpoints of the concave area where it is located. When the average depth of the concave points in a concave area reaches a larger value, it can be considered that the current concave area contains the endpoints of the segmentation path. When the number of concave areas with a deeper concave degree is greater than 2, it can be determined that the number of required segmentation paths is greater than 1. According to the number of required segmentation paths, when it is determined that the number of required segmentation paths is greater than 1, after executing steps S1411 to S1413 and step S142 to determine a segmentation path, steps S1411 to S1413 and step S142 can be executed again to determine a new segmentation path until the number of segmentation paths obtained is consistent with the number of required segmentation paths. Since the path determined according to the concave point with a larger gradient amplitude is most likely to be the segmentation path 10, in this embodiment, by determining the current concave point pair, and preferentially connecting the concave point pair where the concave point with the largest gradient amplitude is located and updating the path, less data can be processed in the process of determining the segmentation path 10 of the image to be segmented, and the segmentation path 10 can be determined more quickly; by determining whether the point on the current path passes through other concave points to determine the candidate path, the current path can be screened, interference data can be removed, and the amount of data to be processed can be reduced, thereby effectively improving the speed of segmenting the image to be segmented and improving the segmentation efficiency.

[0068] In some embodiments, Figure 8 As shown, step 1413, updating the initial path based on the gradient information to obtain the current path, may include:

[0069] Step 14131, update the passing points according to the gradient information to obtain the process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before the update; update the passing points based on the gradient information, and draw a straight line perpendicular to the initial path from the passing points, and find a pixel point with a gradient greater than the current passing point on the straight line according to the gradient information of each point in the image to be segmented, and determine it as a new passing point. After all the passing points are updated, the new passing point is connected with the first concave point and the second concave point according to the gradient direction of each new passing point to obtain the process path. Update the initial path based on the gradient information, and draw a straight line from the passing points along the gradient direction of the passing points, and find a pixel point with a gradient greater than the current passing point on the straight line according to the gradient information of each point in the image to be segmented, and determine it as a new passing point. After all the passing points are updated, the new passing point is connected with the first concave point and the second concave point according to the gradient direction of each new passing point to obtain the process path. For the process path, its average gradient amplitude can be greater than the gradient amplitude before the update, or it can be equal to the gradient amplitude before the update.

[0070] If the average gradient amplitude of the process path is equal to the average gradient amplitude before the update, it can be determined that the average gradient amplitude of the process path has reached the maximum value, the edge contour closest to the target in the overlapping area, and the passing points on the process path will not change even if they are updated again. The process path can no longer be updated, so the update can be stopped and the process path can be used as the current path. If the average gradient amplitude of the process path is greater than or equal to the first threshold, the process path can also be stopped from being updated and used as the current path, where the first threshold can be set to a larger value. When the average gradient amplitude of the process path is greater than or equal to the first threshold, the average gradient amplitude is larger, and it can be considered that the process path has met the conditions for being the current path, and the update is stopped, and the process path is determined as the current path.

[0071] If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, it can be known that the average gradient amplitude of the process path has not reached the maximum value, and the passing points on the process path may still change after the next update, and the process path can still be updated. A number threshold can be set, that is, a threshold of the number of updates, to limit the maximum number of updates. If the number threshold is reached but the current path is still not determined, it can be determined that the current concave point pair is not the correct concave point pair that can determine the segmentation path 10. The current concave point pair can be excluded and stopped in time to reduce the impact of interference data. If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is less than the number threshold, the process path can continue to be updated, and the execution can be returned to update the passing points according to the gradient information to obtain the process path; if the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is equal to the number threshold, the execution returns to determine the current concave point pair.

[0072] In some embodiments, the number threshold can be set to 10-200 times, for example, 100 times. By setting the number threshold, it is possible to avoid too many iterative updates, which may result in too long image segmentation response time and affect the subsequent process of material sorting. Setting a relatively low number threshold can improve efficiency and reduce iteration time; setting a relatively high number threshold can improve accuracy and avoid missed recognition. The number threshold can be 100 times, which can ensure that the segmentation path is updated and found, and can also avoid the situation where the segmentation path is not updated due to too few times.

[0073] By updating the passing points according to the gradient information, obtaining the process path, and screening the process path according to the gradient amplitude, the method of determining the current path can effectively reduce redundant data processing, timely remove interference data, and increase the speed of determining the current path process, thereby effectively improving efficiency and making the segmentation process of the image to be segmented more real-time. In some embodiments, such as Fig. 9 As shown, step S141 confirms one or more candidate paths based on one or more concave point pairs and according to gradient information, which may include:

[0074] Step S1414, determining all concave point pairs. All concave point pairs obtained in step S120 may be determined to determine the concave points included in each concave point pair and the concave point information included in each concave point pair, which may include position information, gradient information, and the like.

[0075] Step S1415, connect each concave point pair, take one or more passing points on each connecting line, and obtain one or more initial paths. Connect each concave point pair to obtain one or more initial paths. The initial path can be a line segment inside the image to be segmented. Each initial path uses two concave points in its corresponding concave point pair as endpoints. One or more points can be taken as passing points in each initial path to adjust and update the initial path later. The initial path can be a straight line segment directly connected to the concave point pair, or it can be an arbitrary line segment with two concave points of the concave point pair as endpoints.

[0076] Step S1416, based on the gradient information, the initial path is updated to obtain one or more current paths. The passing points can be updated based on the gradient amplitude and gradient direction to obtain updated passing points, so that the gradient amplitude of the updated passing points is greater than the gradient amplitude of the passing points of the initial path, and the updated passing points are connected with the two concave points of the corresponding concave point pair to obtain the updated current path. The direction of the updated initial path can also be determined according to the gradient direction of the initial path, and the path can be updated as a whole according to the gradient amplitude of the initial path, so that the gradient amplitude of the updated current path is greater than the gradient amplitude of the initial path. The update of the initial path based on the gradient information can be repeated multiple times, and the initial path can be updated multiple times.

[0077] According to the current path, all points on the current path can be determined. Since the final segmentation path 10 is the edge of the target image where the overlap occurs, the segmentation path 10 starts from a concave point, passes through the inside of the image to be segmented, and finally ends at another concave point. In the segmentation path 10, only the points at both ends are concave points, and the remaining points are all inside the image to be segmented. Therefore, if the current path passes through other concave points other than the current concave point pair, the current path cannot be the segmentation path 10, and the current path is ignored. If the current path does not pass through other concave points other than the current concave point pair, the current path may be the segmentation path 10, so the current path can be determined as a candidate path.

[0078] By connecting all the concave point pairs, determining the initial path and updating the path, the candidate path determined according to each concave point pair can be obtained, the omission of the candidate path can be avoided, and the accuracy of image segmentation can be effectively improved. At the same time, when applied to sorting equipment, the method in this embodiment has a simpler program running logic, can be applied to a variety of different equipment, and has better versatility.

[0079] In some embodiments, Fig.10 As shown, step S1416, updating the initial path based on the gradient information to obtain one or more current paths, may include:

[0080] Step S14161, update the passing points according to the gradient information to obtain the process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before the update. For each initial path, each passing point on the initial path can be updated according to the gradient information. A vertical line can be drawn to the initial path, and the foot of the vertical line is located at the passing point. According to the gradient amplitude of each point in the image, a pixel point with a gradient amplitude greater than the current passing point is found on the vertical line, and it is determined as a new passing point. After all the passing points on the initial path are updated, the concave point pair and the new passing point can be connected according to the gradient direction of the passing point to obtain the process path. A straight line can also be drawn along the gradient direction of the passing point, and a pixel point with a gradient amplitude greater than the current passing point is found on the straight line, and it is determined as a new passing point. After all the passing points on the initial path are updated, the concave point pair and the new passing point can be connected according to the gradient direction of the passing point to obtain the process path. The average gradient amplitude of the process path can be greater than the average gradient amplitude of the initial path before the update, or it may be equal to the average gradient amplitude before the update.

[0081] If the average gradient amplitude of the process path is equal to the average gradient amplitude before the update, or the average gradient amplitude of the process path is greater than or equal to the first threshold, then the update is stopped and the process path is used as a current path. If the average gradient amplitude of the process path is equal to the average gradient amplitude before the update, it can be determined that the average gradient amplitude of the process path has reached the maximum value, and the passing points on the process path cannot change even if they are updated again. The process path determined by the current concave point pair has been updated to the edge contour closest to the overlapping target area, and the update can be stopped, and the process path can be used as a current path. The first threshold can also be set to determine the current path, and the first threshold can be set to a larger value. If the average gradient amplitude of the process path is greater than or equal to the first threshold, it can be determined that the gradient amplitude of the process path has met the requirements as the current path, and the update can be stopped directly, and the process path can be determined as the current path.

[0082] If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is less than the number threshold, then return to execute to update the passing points according to the gradient information to obtain the process path. If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, it indicates that the process path has not yet reached the maximum average gradient amplitude, and the passing points on the process path can continue to be updated to obtain a new process path. A number threshold can be set to limit the number of updates. If the number of updates does not reach the number threshold, and the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, it can be determined that the process path can still be updated, and it can return to execute to update the passing points according to the gradient information to obtain the process path, and continue to update the process path.

[0083] In the case where the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is equal to the number threshold, the process path may have reached its maximum average gradient amplitude. Since the number of updates has reached the number threshold, it cannot be updated again, so it is impossible to confirm whether the process path can be used as a current path. A second threshold can be set to confirm whether the average gradient amplitude of the process path is greater than the second threshold to determine whether the process path can be determined as the current path. The second threshold can be less than or equal to the first threshold, so that in the process of determining the current path, the omission of part of the path can be effectively avoided. If the average gradient amplitude of the process path is greater than or equal to the second threshold, it can be determined that the average gradient amplitude of this process path is large and close to the edge contour of the overlapping area target. This process path can be determined as a current path and subsequent judgment and selection are performed on it. If the average gradient amplitude of the process path is less than the second threshold, it can be known that this process path is far from the edge contour of the overlapping area target, so this process path can be ignored, and other process paths can continue to be updated and judged.

[0084] By updating the passing points according to the gradient information, obtaining the process path, and filtering the process path according to the gradient amplitude and determining the current path, the interference data can be ignored in time and some unnecessary data processing can be avoided, so the speed of determining the current path can be effectively improved. By setting the first threshold, it is convenient to judge the current path and save more computing power. By setting the second threshold, some process paths whose update times reach the threshold but are close to the edge contour of the target in the overlapping area can be confirmed, which can effectively avoid missing the current path, ensure the accuracy and comprehensiveness of the data, and thus ensure that the segmented path has higher accuracy.

[0085] In some embodiments, Fig.11 As shown, step S142, determining the segmentation path 10 based on the candidate path, may also include:

[0086] Determine the segmentation path 10 according to the number of candidate paths. Since there may be one or more segmentation paths 10 for the image to be segmented, and the number of candidate paths may be greater than the final segmentation path 10, the candidate paths need to be screened, and the segmentation path 10 may be determined in different ways according to the number of candidate paths.

[0087] If the number of candidate paths is equal to 1, the candidate path is determined as segmentation path 10. When the number of candidate paths is 1, only one candidate path satisfies the condition of being segmentation path 10, and the current candidate path can be directly determined to be segmentation path 10, and it can be determined that the current image to be segmented contains an image of two targets.

[0088] If the number of candidate paths is greater than 1, the segmentation path 10 is determined according to the gradient information of the candidate paths. If the number of candidate paths is greater than 1, the number of segmentation paths 10 can be one or more. For the case where the image to be segmented contains two targets, the number of segmentation paths 10 is 1, and for the case where the image to be segmented contains three or more targets, the number of segmentation paths 10 is greater than 1. Therefore, it is necessary to screen the candidate paths to determine the segmentation path 10, and the segmentation path 10 can be determined according to the gradient information of the candidate paths, wherein the candidate path with a larger gradient amplitude can be determined as the segmentation path 10.

[0089] Different methods for determining the segmentation path 10 are adopted according to the number of candidate paths, which can accurately determine the segmentation path 10 for the images to be sorted in different situations. When the number of candidate paths is 1, the segmentation path 10 is directly determined without further data processing, which can effectively improve efficiency.

[0090] In some embodiments, Figure 12 to Figure 14 As shown, step S142, determining the segmentation path 10 according to the gradient information of the candidate path, may include:

[0091] The candidate path with an average gradient magnitude greater than or equal to the gradient threshold is determined as the segmentation path 10. Since the larger the gradient magnitude is, the more obvious the image change is, and the more likely it is the edge of the image, for the candidate path with a larger average gradient magnitude, it can be determined that the candidate path is closer to the edge of the overlapping area of ​​the overlapping multiple targets. Therefore, a gradient threshold can be set, and the candidate path with an average gradient magnitude greater than or equal to the gradient threshold can be determined as the segmentation path 10.

[0092] The candidate path with the largest average gradient magnitude is determined as segmentation path 10. When the image to be segmented contains two objects, the number of segmentation paths is 1, so the candidate path with the largest average gradient magnitude can be determined as segmentation path 10.

[0093] The candidate paths are sorted from large to small according to their average gradient amplitude, and the first N-1 candidate paths are recorded as segmentation paths 10, where N is the number of targets contained in the image. The image to be segmented can also be identified to determine the number of targets N in the image to be segmented. The bounding box of each target in each image to be segmented can be obtained by the target recognition algorithm, and the number of targets N in the image to be segmented can be determined according to the number of bounding boxes. For N overlapping targets, N-1 segmentation paths are required to segment the targets so that they are independent of each other. Therefore, the number of segmentation paths can be determined to be N-1. The candidate paths can be sorted from large to small according to their average gradient amplitude, and the first N-1 candidate paths can be confirmed as segmentation paths 10.

[0094] By determining the segmentation path 10 according to the magnitude of the gradient amplitude, the correct segmentation path 10 can be accurately determined, redundant candidate paths can be discarded, over-segmentation or erroneous segmentation can be avoided, and the accuracy of the image segmentation method to be segmented can be effectively improved.

[0095] In some embodiments, Fig.15 As shown, step S120, determining a plurality of concave points in the image to be segmented according to the image to be segmented, may include:

[0096] Step S122, performing image preprocessing on the image to be segmented to obtain a preprocessed image. Image preprocessing may include noise reduction processing, which may perform Gaussian smoothing on the acquired image to be segmented, remove burrs on the edge of the image to be segmented, prevent burrs on the image to be segmented from affecting the extraction of concave points, and make the edge of the image to be segmented smoother. Image preprocessing may also include contour extraction, which may perform contour extraction on the image to be segmented to extract the common external contours of multiple targets, which may facilitate the calculation of the curvature of the contour points and the extraction of concave points. Image preprocessing may also include binarization processing, such as Figure 2 (b) Figure 2 As shown in (f), the image can be binarized. When contour processing is required for the segmented image, the binarization process can reduce the amount of data calculation in the contour extraction process and obtain a clearer contour.

[0097] Step S123, determining the concave points in the image according to the preprocessed image. The concave points of the image can be determined according to the preprocessed image. After preprocessing, the concave points of the preprocessed image are determined, which can reduce the amount of data calculation in the concave point determination process, improve the quality of concave point extraction, obtain more accurate concave point information, eliminate certain data interference, and improve the accuracy of segmentation in the subsequent steps.

[0098] Based on the same inventive concept, Fig.16As shown, the present disclosure also provides a material sorting device 200, wherein the material sorting device 200 includes: an image segmentation module 210, which is used to execute the image segmentation method as in any of the aforementioned embodiments; and a material sorting module 220, which is used to identify and sort targets according to the target image obtained by the image segmentation module 210. The image segmentation module 210 can segment the image according to the image segmentation method to be segmented in any of the aforementioned embodiments, and can quickly and accurately complete the image segmentation, and distinguish multiple overlapping and nested targets in the image to be segmented. The material sorting module 220 may further include a target recognition unit and a material sorting unit, which are respectively used to identify targets according to the target image obtained by the image segmentation module 210, and to sort targets. Regarding the material sorting device in the above-mentioned embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be explained in detail here.

[0099] Based on the same inventive concept, the present disclosure further provides a computer-readable storage medium, which stores the following program, and the program is used to execute the method for segmenting an image to be segmented in any of the aforementioned embodiments.

[0100] like Fig.17 As shown, an embodiment of the present disclosure provides an electronic device 300. The electronic device 300 includes a memory 310, a processor 320, and an input / output (I / O) interface 330. The memory 310 is used to store instructions. The processor 320 is used to call the instructions stored in the memory 310 to execute the image segmentation method to be segmented of the embodiment of the present disclosure. The processor 320 is connected to the memory 310 and the I / O interface 330 respectively, for example, through a bus system and / or other forms of connection mechanisms (not shown). The memory 310 can be used to store programs and data, including the program of the image segmentation method involved in the embodiment of the present disclosure. The processor 320 executes various functional applications and data processing of the electronic device 300 by running the program stored in the memory 310.

[0101] In the embodiment of the present disclosure, the processor 320 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 320 can be a central processing unit (CPU) or one or a combination of other forms of processing units with data processing capabilities and / or instruction execution capabilities.

[0102] The memory 310 in the embodiment of the present disclosure may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.

[0103] In the embodiment of the present disclosure, the I / O interface 330 may be used to receive input instructions (such as digital or character information, and key signal input related to user settings and function control of the electronic device 300), and may also output various information to the outside (such as images or sounds). In the embodiment of the present disclosure, the I / O interface 330 may include one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel.

[0104] The present application uses specific words to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0105] In the context of this application, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0106] Similarly, it should be noted that in order to simplify the description disclosed in this application and thus help understand one or more application embodiments, in the foregoing description of the embodiments of this application, multiple features are sometimes merged into one embodiment, drawing or description thereof. However, this disclosure method does not mean that the features required by the object of this application are more than the features mentioned in the claims. In fact, the features of the embodiment are less than all the features of the single embodiment disclosed above. The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is only an example and does not constitute a limitation of this application. Although it is not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this application. Such modifications, improvements and corrections are suggested in this application, so such modifications, improvements and corrections still belong to the spirit and scope of the embodiments of this application.

Claims

1. A method for segmenting an image to be segmented, wherein: The image to be segmented includes a plurality of target images, including: Determine, according to the image to be segmented, the gradient information of each point in the image to be segmented, wherein the gradient information includes the gradient amplitude; According to the image to be segmented, determining a plurality of concave points in the image to be segmented; Pairing the plurality of concave points in pairs to obtain one or more concave point pairs; Determining a segmentation path of the image to be segmented according to the concave point pairs and the gradient information of each point in the image to be segmented; The image to be segmented is segmented according to the segmentation path to obtain a plurality of target images.

2. The image segmentation method according to claim 1, wherein: The step of pairing the plurality of concave points in pairs to obtain one or more concave point pairs comprises: Determine a corresponding concave area of ​​each of the concave points; Each concave point is paired with a concave point belonging to another concave region to obtain a concave point pair.

3. The image segmentation method according to claim 1 or 2, wherein: The step of determining a segmentation path of the image to be segmented according to the concave point pairs and the gradient information of each point in the image to be segmented comprises: Based on the one or more concave point pairs, and according to the gradient information, confirm one or more candidate paths; Based on the candidate paths, the segmentation path is determined.

4. The method for segmenting an image to be segmented according to claim 3, wherein: The step of confirming one or more candidate paths based on the one or more concave point pairs and according to the gradient information includes: Determine a current concave point pair, wherein the current concave point pair includes a first concave point and a second concave point, the first concave point is a concave point with a maximum gradient amplitude among all current concave points, and the second concave point is a concave point with a maximum gradient amplitude among concave points paired with the first concave point; Connect the first concave point and the second concave point, and select one or more passing points on the connecting line to obtain an initial path; Update the initial path based on the gradient information to obtain a current path; If the current path passes through other concave points other than the current concave point pair, the second concave point in the current concave point pair is ignored and the process of determining the current concave point pair is returned to be executed; If the current path does not pass through other concave points other than the current concave point pair, the current path is determined as a candidate path and the segmentation path is determined based on the candidate path.

5. The method for segmenting an image to be segmented according to claim 4, wherein: The updating of the initial path based on the gradient information to obtain the current path includes: Update the passing points according to the gradient information to obtain a process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before the update; If the average gradient amplitude of the process path is equal to the average gradient amplitude before updating, or the average gradient amplitude of the process path is greater than or equal to a first threshold, then the updating is stopped and the process path is used as the current path; If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is less than the number threshold, then return to execute the updating of the passing points according to the gradient information to obtain the process path; If the average gradient magnitude of the process path is greater than the average gradient magnitude before updating, and the number of updates is equal to the number threshold, the process returns to executing the determination of the current concave point pair.

6. The method for segmenting an image to be segmented according to claim 3, wherein: The step of confirming one or more candidate paths based on the one or more concave point pairs and according to the gradient information includes: Determine all pairs of concave points; Connect each pair of concave points and select one or more passing points on each connecting line to obtain one or more initial paths; updating the initial path based on the gradient information to obtain one or more current paths; If the current path passes through other concave points other than the current concave point pair, the current path is ignored; If the current path does not pass through other concave points other than the current concave point pair, the current path is determined as a candidate path.

7. The method for segmenting an image to be segmented according to claim 6, wherein: The updating of the initial path based on the gradient information to obtain one or more current paths includes: Update the passing points according to the gradient information to obtain a process path, so that the average gradient amplitude of the process path is greater than or equal to the average gradient amplitude before the update; If the average gradient amplitude of the process path is equal to the average gradient amplitude before updating, or the average gradient amplitude of the process path is greater than or equal to the first threshold, then the updating is stopped and the process path is used as the current path; If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is less than the number threshold, then return to execute the updating of the passing points according to the gradient information to obtain the process path; If the average gradient amplitude of the process path is greater than the average gradient amplitude before the update, and the number of updates is equal to the number threshold, confirming whether the average gradient amplitude of the process path is greater than a second threshold, wherein the second threshold is less than or equal to the first threshold; If the average gradient amplitude of the process path is greater than or equal to the second threshold, determining the process path as the current path; If the average gradient magnitude of the process path is less than the second threshold, the process path is ignored.

8. The method for segmenting an image to be segmented according to claim 3, wherein: The determining the segmentation path based on the candidate path further includes: If the number of the candidate paths is equal to 1, the candidate path is determined as a segmentation path; If the number of the candidate paths is greater than 1, the segmentation path is determined according to the gradient information of the candidate paths.

9. The method for segmenting an image to be segmented according to claim 8, wherein: The determining the segmentation path according to the gradient information of the candidate path includes: Determine the candidate path whose average gradient magnitude is greater than or equal to the gradient threshold as the segmentation path; or, Determine the candidate path with the largest average gradient amplitude as the segmentation path; or, The candidate paths are sorted from large to small according to their average gradient amplitudes, and the first N-1 candidate paths are recorded as the segmentation paths, where N is the number of objects contained in the image.

10. The method for segmenting an image to be segmented according to claim 1, wherein: The step of determining a plurality of concave points in the image to be segmented according to the image to be segmented comprises: determining candidate points according to the curvature of points in the image; If the depth information of the candidate point is greater than or equal to the depth threshold, it is determined to be the concave point; If the depth information of the candidate point is less than the depth threshold, the candidate point is ignored.

11. The method for segmenting an image to be segmented according to claim 1, wherein: The step of determining a plurality of concave points in the image to be segmented according to the image to be segmented comprises: Performing image preprocessing according to the image to be segmented to obtain a preprocessed image; According to the preprocessed image, concave points in the image are determined.

12. A material sorting device, wherein: The material sorting device comprises: An image segmentation module, used to execute the image segmentation method according to claims 1-11; The material sorting module is used to perform target recognition and sorting according to the target image obtained by the image segmentation module.

13. A computer-readable storage medium storing the following program, wherein the program is used to execute the image segmentation method according to any one of claims 1 to 11.

Images