Image segmentation method, target sorting method and material sorting system

Abstract

The invention relates to the field of material sorting, in particular to an image segmentation method, a target sorting method and a material sorting system. The image segmentation method comprises the following steps: acquiring an image containing a plurality of targets; performing target identification according to the image to obtain an external frame of each target; determining edge information of continuous patterns in the image based on the image; based on the edge information of the continuous pattern comprising the multiple targets, carrying out pit identification to obtain multiple pits; determining segmentation lines of the multiple targets in the continuous pattern comprising the multiple targets based on the multiple concave points of the continuous pattern comprising the multiple targets and the multiple corresponding external connection frames; and obtaining a contour of each target in the continuous pattern comprising the plurality of targets based on the edge information of the continuous pattern comprising the plurality of targets and the segmentation lines. According to the invention, the contour of each target in the image can be extracted more accurately, the accuracy of image segmentation is improved, the detection is rapid, the real-time performance is better, and the efficiency and accuracy of image segmentation are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of material sorting, and in particular to an image segmentation method, a target sorting method, a material sorting system, and a computer-readable storage medium. Background Art

[0002] In the material sorting process, it is necessary to collect and segment the image of each material, and distinguish the overlapping or sticky materials to facilitate the subsequent identification and classification of the materials. However, due to the different shapes and sizes of the materials, and the close arrangement during the sorting process, the collected images are often accompanied by adhesion and overlap, so the images of the materials need to be segmented. The current material image segmentation technology has low accuracy, especially for the case of adhesion and overlap of three or more target images, which is prone to over-segmentation and missed segmentation, and cannot meet the accuracy requirements in the material sorting process; or in other image segmentation technologies, the calculation is large, time-consuming, poor real-time performance, low efficiency, and cannot meet production needs. Summary of the invention

[0003] To overcome the problems existing in the related art, an exemplary embodiment of the present disclosure provides an image segmentation method in a first aspect, comprising: acquiring an image including multiple targets; performing target recognition based on the image to obtain a bounding box for each target; determining edge information of a continuous pattern in the image based on the image, wherein each continuous pattern includes one or more targets; performing concave point recognition based on the edge information of the continuous pattern including multiple targets to obtain multiple concave points; determining a dividing line of multiple targets in the continuous pattern including multiple targets based on the multiple concave points and the corresponding multiple bounding boxes of the continuous pattern including multiple targets; and obtaining the outline of each target in the continuous pattern including multiple targets based on the edge information of the continuous pattern including multiple targets and the dividing line.

[0004] In some embodiments, the image segmentation method further includes: confirming the number of external frames corresponding to the edge information of each of the continuous patterns in the image; if the edge information of the continuous pattern corresponds to one external frame, the continuous pattern includes one target, and based on the edge information of the continuous pattern, the outline of the one target is obtained; if the edge information of the continuous pattern corresponds to multiple external frames, the continuous pattern includes multiple targets, and concave point recognition is performed based on the edge information of the continuous pattern including multiple targets to obtain multiple concave points.

[0005] In some embodiments, the dividing lines of multiple targets in the continuous pattern including multiple targets are determined based on the multiple concave points and the corresponding multiple external frames of the continuous pattern including multiple targets, including: determining the overlapping area between each two intersecting external frames according to each of the external frames including multiple targets; and determining the dividing line of the multiple targets based on each of the overlapping areas and the multiple concave points in the overlapping areas.

[0006] In some embodiments, determining the segmentation line of the multiple targets based on each overlapping area and the multiple concave points within the overlapping area includes: pairing the multiple concave points within each overlapping area in pairs to form one or more concave point pairs, wherein each concave point belongs to only one concave point pair; connecting the two concave points corresponding to each concave point pair to form the segmentation line.

[0007] In some embodiments, pairing the multiple concave points in each overlapping area in pairs to form one or more concave point pairs includes: determining the number of the concave points in each overlapping area; if the overlapping area contains two concave points, determining the two concave points as a concave point pair; if the overlapping area contains three or more concave points, determining two unpaired concave points therein as a concave point pair.

[0008] In some embodiments, pairing the multiple concave points in each overlapping area in pairs to form one or more concave point pairs includes: determining the number of unpaired concave points in the overlapping area that currently contains unpaired concave points; if the number of unpaired concave points in the overlapping area that currently contains unpaired concave points is equal to 2, determining the two concave points as a concave point pair; if the number of unpaired concave points in the overlapping area that currently contains unpaired concave points is greater than 2, determining the number of unpaired concave points in the next overlapping area that contains unpaired concave points until all concave points are paired.

[0009] In some embodiments, determining edge information of continuous patterns in the image based on the image further includes: performing binarization processing on the image to obtain a binary image; and determining edge information of continuous patterns in the image by contour extraction based on the binary image.

[0010] In some embodiments, the method further includes: obtaining a smooth target contour through morphological operations according to the contour of each target.

[0011] In a second aspect, the present disclosure further provides a target sorting method, comprising: the image segmentation method as described in the first aspect; and classifying each target through target recognition according to the outline of each target.

[0012] In a third aspect, the present disclosure further provides a material sorting system for sorting materials, wherein the material sorting system comprises: an image segmentation module for executing the image segmentation method as described in the first aspect.

[0013] In some embodiments, the material sorting system includes: an image recognition module, which is used to identify and classify each of the materials in the segmented image.

[0014] In a fourth aspect, the present disclosure further provides a computer-readable storage medium, which stores the following program, wherein the program is used to execute the image segmentation method as described in the first aspect.

[0015] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: quickly locating the number and position of targets through target recognition, and determining the segmentation line of an image containing multiple targets in combination with the concave point detection results of edge information, thereby obtaining the outline of each target in the image, and being able to more accurately extract the outline of each target in the image, effectively improving the accuracy of image segmentation, while being able to quickly detect and have better real-time performance, effectively improving the efficiency and accuracy of image segmentation.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present disclosure may be better understood by describing exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which:

[0018] Figure 1 It is a schematic diagram of the structure of a sorting device according to an exemplary embodiment of the disclosure;

[0019] Figure 2 is a flowchart of an image segmentation method according to an exemplary embodiment of the disclosure;

[0020] Figure 3 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure.

[0021] Figure 4 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure.

[0022] Figure 5 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure;

[0023] Figure 6 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure;

[0024] Figure 7 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure;

[0025] Figure 8 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure;

[0026] Fig. 9 is a schematic diagram of a process of segmenting an image according to an image segmentation method according to an exemplary embodiment of the disclosure;

[0027] Fig.10 is a schematic diagram of a process of segmenting an image according to an image segmentation method according to another exemplary embodiment of the present disclosure;

[0028] Fig.11 is a flowchart of an image segmentation method according to another exemplary embodiment of the present disclosure;

[0029] Fig.12 is a schematic block diagram of a material sorting system according to an exemplary embodiment of the disclosure;

[0030] Fig.13 It is a schematic block diagram of an electronic device according to an exemplary embodiment of the disclosure. DETAILED DESCRIPTION

[0031] The specific implementation methods of the present disclosure will be described below. It should be noted that in the specific description of these implementation methods, in order to provide a concise description, it is impossible for this specification to provide a detailed description of all the features of the actual implementation methods. It should be understood that in the actual implementation of any implementation method, just as in the process of any engineering project or design project, in order to achieve the specific goals of the developer and to meet system-related or business-related restrictions, various specific decisions are often made, and this will also change from one implementation method to another. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the content disclosed by the present disclosure, some changes in design, manufacturing or production based on the technical content disclosed in the present disclosure are just conventional technical means, and should not be understood as insufficient content of the present disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in the claims and the specification shall have the usual meaning understood by persons with ordinary skills in the technical field to which the present disclosure belongs. The words "first", "second" and similar words used in the patent application specification and the claims of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "one" or "one" do not indicate a quantitative limitation, but indicate the existence of at least one. Words such as "include" or "comprise" and similar words mean that the elements or objects appearing before "include" or "comprise" include the elements or objects listed after "include" or "comprise" and their equivalent elements, and do not exclude other elements or objects. Words such as "connect" or "connected" and similar words are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0033] The image segmentation method provided by the present disclosure can be applied to the segmentation of the edge of the target image. In some embodiments, it can be applied to the sorting equipment of ore, food or other fields. Figure 1 As shown, the material sorting equipment 100 can be used to sort materials, and may include a feeding mechanism 110, a transmission mechanism 120, a detection mechanism 130 and a sorting device 140. The feeding mechanism 110 is used to feed the material to be sorted into the transmission mechanism 120. The transmission mechanism 120 may be a structure such as a conveyor belt or a chute, which is used to transport the material to be sorted fed by the feeding mechanism 110. The detection mechanism 130 is used to detect the material transported on the transmission mechanism 120 to detect whether the material is the material to be rejected; the material to be rejected refers to the material to be separated by the sorting equipment. The material to be rejected may be the required material or the unrequired material, as long as the material can be sorted. The sorting device 140 is used to reject the material to be rejected. The material sorting equipment can be used for the sorting of ores, and can also be used for tasks such as food sorting or waste sorting. Especially in the process of ore sorting, since the shape of the ore is more irregular and the relative positional relationship between multiple ores is more uncertain, the materials to be sorted may overlap or stick together during the sorting process of the material sorting equipment. Therefore, the detection mechanism 130 of the material sorting equipment needs to segment the overlapping or sticky multiple targets when performing material detection in order to distinguish the overlapping or sticky multiple targets.

[0034] In some related technologies, multiple overlapping or adhered targets can be segmented by using a segmentation method based on machine learning, and an instance segmentation model of the target can be established by using instance segmentation technology to segment multiple overlapping and adhered targets. However, the instance segmentation method based on machine learning requires the establishment of a model for the target material and a large amount of data training, which is costly and has a large model size. Therefore, when performing segmentation, a large amount of calculation is required, and the target segmentation speed is slow and inefficient.

[0035] In other related technologies, multiple overlapping or adhered targets can be segmented by traditional algorithms, such as morphological segmentation algorithms, watershed algorithms, and concave point detection and matching segmentation algorithms. Since the images of materials such as ore to be sorted are complex and diverse, the accuracy of segmentation is low when segmenting by traditional algorithms, and it is easy to make wrong segmentations or omissions, thus affecting the accuracy of material sorting. In some cases, three or more targets may overlap or stick together. The segmentation line is determined by the traditional concave point detection algorithm. Concave point detection can confirm four or more concave points, so it is necessary to pair the concave points. The traditional algorithm generally pairs the two concave points with the shortest distance and connects them to form a segmentation line. For the case where the number of concave points is more than four, pairing errors are prone to occur, resulting in target segmentation errors, which seriously affect the accuracy of material sorting.

[0036] The detection mechanism 130 of the material sorting equipment needs to have good real-time and accuracy. In order to improve production and efficiency, the material transmission speed of the transmission mechanism 120 is constantly increasing. The detection mechanism 130 is required to detect the material in real time in a shorter time. The segmentation method based on machine learning cannot meet the real-time requirements, and the segmentation method based on traditional algorithms has poor accuracy and is prone to over-segmentation, missed segmentation and wrong segmentation, and cannot meet the accuracy requirements.

[0037] To solve the above problems, Figure 2 As shown, the present disclosure provides an image segmentation method, which may include: step S10 to step S60, and the above steps are specifically described below:

[0038] Step S10, acquiring an image containing multiple targets. The image can be acquired through an image acquisition device, which can be a camera, an X-ray image acquisition device, or a CT scanning device. Step S10 can also acquire an image containing multiple targets in a memory by reading the storage. In some cases, an image containing multiple targets can also be acquired by generating an image. The image can contain multiple targets, and the multiple targets can be scattered throughout the image. There may also be overlap or adhesion between the multiple targets. For material sorting equipment, the image acquisition device can be used to acquire images of materials on the transmission mechanism 120 in the material sorting system, such as Fig. 9 As shown in (a), the material can be ore to be sorted, or food to be sorted, or waste to be sorted.

[0039] Step S20, target recognition is performed based on the image to obtain the bounding box of each target. Target recognition can be performed on an image containing multiple targets to determine the number of targets and obtain the bounding box of each target. Target recognition of an image can be accomplished by a target recognition algorithm. Target recognition can be performed by confirming the bounding box of a target through pattern recognition. It can also be based on a deep learning model, such as a convolutional neural network, to extract features and identify targets in the image. For example, target recognition can be performed using a PP-YOLOE network. The accuracy of target recognition can be improved by training a deep learning model, increasing the training data set, training rounds, improving the model structure, adjusting parameters, and the like. Fig. 9 (b) Fig.10 As shown in (b), the bounding box of each target in the image can be rectangular or circular. The bounding box of each target can contain its corresponding target. When multiple targets are close to each other and overlap or stick together, Fig. 9 (b) Fig.10 As shown in (b), there may be a certain overlap between the bounding boxes corresponding to each overlapping or adhering target. The bounding box may also be non-visual. Through the target recognition algorithm, the number of targets in the image can be determined, the position of each target can be determined, the bounding box of each target can be determined, and the position and size data of each bounding box can be stored. Step S20 does not obtain the outer contour of each target through target recognition, but obtains the bounding box containing all the information of the corresponding target. Therefore, the target recognition model is small in size, less in calculation, fast in response and high in recognition accuracy.

[0040] Step S30, based on the image, determining edge information of continuous patterns in the image, wherein each continuous pattern includes one or more targets. The image can be processed by an image processing algorithm to determine edge information of continuous patterns in the image. The continuous patterns can be internally connected and have continuous external contours. One target can be a continuous pattern, and multiple overlapping or adhered targets can also be a continuous pattern. Fig. 9 (c) Fig.10 As shown in (c), there may be one or more continuous patterns, and each continuous pattern may include one or more targets. The edge information of a continuous pattern including one target is the edge information of a single target that does not overlap or adhere to other targets in the image. A continuous pattern may also include multiple overlapping or adhered targets. The edge information of a continuous pattern including multiple targets is the total edge information of the area formed by multiple overlapping or adhered targets in the image. The edge information of a continuous pattern may be the peripheral contour information of the continuous pattern, or it may be the shape information of a continuous pattern including contour information.

[0041] In some embodiments, Figure 3As shown, step S30, based on the image, determines the edge information of the continuous pattern in the image, which may include: step S31, based on the image, performing binarization processing to obtain a binary image; step S32, based on the binary image, determining the edge information of the continuous pattern in the image by contour extraction. Fig. 9 (c) Fig.10 As shown in (a), a binarization process can be performed on the image, and the grayscale values ​​of the pixels on the image can be set to 0 or 255 to obtain a binary image. The binarization process can separate the target from the background, making the target clearer and facilitating subsequent image processing. The binarization process converts each target in the image into a continuous pattern with the same grayscale value. The continuous pattern can be a complete color block with internal interconnection and continuous external contours. The binarization process can convert multiple overlapping or adhered targets into a continuous pattern in the binary image, thereby merging multiple overlapping or adhered targets into a complete color block with internal interconnection and continuous external contours. The binarization process can reduce the grayscale levels on the image from multiple to two, effectively simplifying the image information and reducing the amount of calculation in the subsequent image detection process. Fig. 9 (d) Fig.10 As shown in (c), according to the binary image, the edge information of the continuous pattern in the image can be determined by contour extraction. The binary image is subjected to contour extraction to extract the edge information of each continuous pattern, wherein the continuous pattern may include a single target or may include multiple overlapping or adhered targets. The edge information of the continuous pattern in the binary image is obtained by contour extraction. The edge information of the continuous pattern may be the contour line of the continuous pattern, or the continuous pattern may be subjected to edge detection to obtain the edge information of the continuous pattern. Through steps S31 and S32, the image information can be simplified, the amount of calculation in the image segmentation process can be reduced, the efficiency of image segmentation can be improved, and clearer target edge information can be extracted, thereby improving the accuracy of image segmentation.

[0042] In some other embodiments, step S30 may include: performing binarization processing according to the image to obtain edge information of continuous patterns in the determined image. Performing binarization processing according to the image and setting the grayscale value of the pixels on the image to 0 or 255 can simplify the image, thereby effectively reducing the amount of calculation for subsequent detection of the image and improving the efficiency of image segmentation. In the image after binarization processing, the color blocks are internally connected and have continuous external contours. According to the image after binarization processing, edge information can be obtained. The edge information can be the shape information of the continuous pattern, and the image edge information can be directly obtained according to the shape information of the continuous pattern. The contour can also be extracted according to the shape information of the continuous pattern to obtain the image edge information. Through binarization, the target can be clearly distinguished from the background, making the target clearer. The binarization processing has a smaller amount of data calculation, improves the speed of obtaining edge information, thereby improving the speed of image segmentation, and has better real-time performance.

[0043] In some other embodiments, step S30 may include: determining edge information of continuous patterns in the image by contour extraction. Contour extraction can be performed directly on the image, and for an independent target, the edge information of the target, that is, the contour of the target, can be directly obtained. For multiple targets that overlap and adhere, the edge information of the continuous pattern formed by the multiple targets, that is, the outer contour line of the continuous pattern, can be obtained by contour extraction. Through contour extraction, the contour line of each continuous pattern in the image can be directly obtained, and the obtained contour information is more accurate, and the accuracy of obtaining the target edge information is high, thereby effectively improving the accuracy of image segmentation.

[0044] Step S40, based on the edge information of the continuous pattern including multiple targets, concave point recognition is performed to obtain multiple concave points. A concave point may be the lowest point on the edge of the continuous pattern that is depressed downward. On the edge of the continuous pattern, a concave point may be located at a position where the edge contour changes sharply. In the case where the targets overlap or stick together, a concave point may be located at the edge of the junction between two overlapping or sticking targets. Fig. 9 (d) Fig.10 As shown in (c), for the edge information of the continuous pattern, the edge information of the continuous pattern including multiple targets can be extracted, and multiple concave points therein can be identified through the concave point recognition algorithm. The concave point recognition can obtain the curvature of each pixel point on the edge by analyzing the edge information of the continuous pattern including multiple targets, and determine the concave points by analyzing the change of the curvature of the points on the edge. The concave point recognition can also determine the concave points by detecting the gradient information of each pixel point in the edge information of the continuous pattern and differentiating the gradient information through methods such as angle difference. By processing the edge information of the continuous pattern of multiple targets through concave point recognition, the concave points can be obtained more quickly, time can be saved, the efficiency of image segmentation can be improved, and the image segmentation method has better real-time performance when applied to the detection mechanism 130 of the sorting equipment.

[0045] Step S50, based on the multiple concave points of the continuous pattern including the multiple targets and the corresponding multiple external frames, determine the segmentation lines of the multiple targets in the continuous pattern including the multiple targets. Through the multiple concave points of the continuous pattern including the multiple targets and the corresponding multiple external frames, when the multiple targets overlap and adhere, the multiple targets can form a continuous pattern, and the multiple targets are included in a continuous pattern. For two targets that overlap or adhere, the junction of their edges includes a concave point. There can be multiple concave points in the continuous pattern. In some cases, when there are three or more targets overlapping and adhering, four or more concave points can appear in a continuous pattern, and there may be overlaps between the external frames corresponding to each target. The existing concave point pairing method generally subtracts the convex closure in the image from the concave figure to obtain a concave area, obtains the two points with the shortest distance in the concave area as concave points, and pairs them. In some cases, the two concave points with the shortest distance may not belong to the same target, and the concave points may be incorrectly paired, causing errors in subsequent image segmentation. The present disclosure integrates target detection and concave point detection. The bounding box obtained by target detection includes position information, which can be used to confirm the area where the target is located. The concave points can be paired with different bounding boxes as boundaries, wherein the determined segmentation line is not the edge of the actual target, but a boundary line that can separate each target. The boundary of each target can be distinguished by the segmentation line, which is convenient for subsequent identification of each target. Through step S150 of the present disclosure, the segmentation line of multiple targets can be determined by comprehensively considering the concave point information of the bounding box and its interior. The segmentation line of multiple targets can be determined by matching the concave points in pairs and connecting the two paired concave points as endpoints, so that the pairing of the concave points is more accurate, thereby improving the accuracy of image segmentation.

[0046] Step S60, based on the edge information and the segmentation line of the continuous pattern including the multiple targets, obtain the contour of each target in the continuous pattern including the multiple targets. Fig. 9 (g) Fig.10 As shown in (f), the continuous pattern including multiple targets can be divided into multiple patterns according to the dividing line, and the closed curve composed of the edge information of each pattern and the dividing line is the outline of each target in the continuous pattern including multiple targets.

[0047] By the image segmentation method disclosed in the present invention, target recognition is performed through a deep learning algorithm to obtain the external frame of each target in the image, which can more quickly and accurately determine the number of targets in the image and the information of the area where they are located, thereby improving the efficiency of target recognition and image segmentation. The concave points between overlapping and adhered targets are obtained through a concave point recognition algorithm, which has a faster recognition speed. The concave points are combined with the edge information of a continuous pattern including multiple targets to confirm the segmentation line, and finally the overlapping and adhered targets can be segmented to obtain the outline of each target, with higher segmentation accuracy. Through the combination of a deep learning algorithm and image recognition technology, the speed of image segmentation can be effectively improved while ensuring the accuracy of image segmentation. When the image segmentation method is applied to actual production, it has better real-time performance. In the process of sorting materials, image segmentation can be performed in real time, which is more efficient and can improve the accuracy of sorting, thereby improving the material conveying speed and recognition efficiency in the material sorting process to improve production capacity. The traditional method of determining the segmentation line by concave point detection is prone to image segmentation errors caused by concave point pairing errors when three or more targets overlap or stick to each other, and the accuracy is low; the image segmentation method provided by the present disclosure can combine concave point detection with neural network target detection technology, and pair the concave points by combining the external frame obtained by target detection with the concave point information to determine the concave point pair, which can effectively avoid the situation of concave point pairing errors and effectively improve the accuracy of image segmentation. The target position and number are determined by target recognition, the external frame of the target is obtained, and the segmentation line of the image containing multiple targets is determined in combination with the concave point detection result of the edge information, so as to obtain the outline of each target in the image, and the outline of each target in the image can be extracted more accurately. At the same time, it can avoid the situation that the traditional instance segmentation technology takes a long time, can detect quickly, has better real-time performance, effectively improves the efficiency of image segmentation, and meets the real-time requirements of image segmentation for subsequent target sorting operations.

[0048] To better illustrate the advantages of the image segmentation method of the embodiment of the present disclosure compared with the traditional method, please refer to the test results of the performance test, as shown in Table 1:

[0049] Table 1:

[0050] method Segmentation accuracy (%) Calculation time (ms) Traditional pit detection algorithm 80 5 Instance segmentation techniques 95 80 The present invention discloses an image segmentation method 94 20

[0051] The test is to perform image segmentation on the same image (ore image), as shown in Table 1, wherein the traditional concave point detection algorithm is a method for segmenting an image through a concave point recognition algorithm, and the instance segmentation technology is a segmentation method based on machine learning, and the edge contour of each target is confirmed according to the instance segmentation model, thereby performing image segmentation. It can be clearly seen from Table 1 that the accuracy of the image segmentation method provided by the present disclosure reaches 94%, which is much higher than the 80% of the traditional concave point detection algorithm, and is close to the 95% of the instance segmentation technology. It has a high segmentation accuracy and can meet the accuracy requirements of material sorting. At the same time, for the time consumed, the image segmentation method provided by the present disclosure consumes 20ms, which is longer than the 5ms of the traditional algorithm, but much lower than the 80ms of the instance segmentation technology. It consumes less time and has a higher computational efficiency, which can meet the real-time requirements of material sorting. Comprehensive segmentation accuracy and computational time consumption data, the image segmentation method provided by the present disclosure can significantly improve the accuracy of segmentation while ensuring a shorter computational time consumption, and the image segmentation efficiency is high, which can simultaneously meet the real-time requirements and accuracy requirements of material sorting.

[0052] In some embodiments, Figure 4 As shown, the image segmentation method may further include: step S70, confirming the number of external frames corresponding to the edge information of each continuous pattern in the image; if the edge information of the continuous pattern corresponds to one external frame, the continuous pattern includes one target, and executing step S80, obtaining the outline of one target based on the edge information of the continuous pattern; if the edge information of the continuous pattern corresponds to multiple external frames, the continuous pattern includes multiple targets, and executing step S40, performing concave point recognition based on the edge information of the continuous pattern including multiple targets to obtain multiple concave points. By confirming the number of external frames corresponding to the edge information of each continuous pattern in the image, the number of targets corresponding to the edge information of each continuous pattern in the image can be confirmed. When the edge information of the continuous pattern corresponds to one external frame, it can be determined that the current continuous pattern only includes one target, so the outline of the current continuous pattern can be directly obtained, that is, the outline of an independent target. When the edge information of a continuous pattern corresponds to multiple bounding boxes, it can be determined that the current continuous pattern includes multiple targets that overlap or adhere. At this time, if the contour of the current continuous pattern is obtained, the common outer contour of the multiple overlapping or adhering targets will be obtained, and the edges of each target cannot be distinguished. Therefore, when the edge information of a continuous pattern corresponds to multiple bounding boxes, in order to facilitate image segmentation, the edge information of the continuous pattern needs to be processed. The concave points can be quickly obtained through the concave point recognition algorithm to facilitate the subsequent determination of the target segmentation line. By confirming the number of bounding boxes corresponding to the edge information of each continuous pattern in the image, different processing methods are used according to the number of bounding boxes. Only the continuous pattern containing multiple targets can be concave point recognized, which can effectively save computing power and improve the efficiency of image segmentation.

[0053] In some embodiments, Figure 5 As shown, step S50 determines the dividing lines of multiple targets in the continuous pattern including multiple targets based on multiple concave points and corresponding multiple external frames of the continuous pattern including multiple targets, which may include: step S51, determining the overlapping area between each two intersecting external frames according to each external frame including multiple targets; step S52, determining the dividing line of multiple targets based on each overlapping area and multiple concave points in the overlapping area. The continuous pattern including multiple targets may correspond to multiple external frames, each of which corresponds to one target. Since the external frame is not the outer contour line of each target, but an area containing all the information of the corresponding target, it is easy to determine the number of targets and the location of the targets. The range selected by the external frame is larger than the range selected by the outer contour line of the corresponding target. Therefore, for multiple overlapping or sticky targets, there may be a certain overlapping area between the corresponding multiple external frames. The overlapping area between each two intersecting external frames is determined, and the position of the dividing line can be located according to the overlapping area. Since the overlapping area is located at the junction of adjacent targets of the continuous pattern including multiple targets, and the concave point is also located at the junction of the edges of adjacent targets of the continuous pattern including multiple targets, the concave point can be located in the overlapping area, and there can be multiple concave points in each overlapping area. Since the concave points are located at the junction of the edges of adjacent targets of the continuous pattern including multiple targets, and each overlapping area can determine the position where the corresponding two targets overlap and adhere, according to an overlapping area and the multiple concave points inside it, the position where the two targets corresponding to the overlapping area overlap and adhere can be determined. Therefore, the segmentation line of multiple targets can be determined based on each overlapping area and the multiple concave points in the overlapping area. The concave points located in the same overlapping area can be connected in pairs with the concave points as endpoints to determine the segmentation line. By jointly analyzing the overlapping area of ​​the external frame and the concave points inside it, the concave points belonging to the same target can be confirmed more quickly and accurately, and then the segmentation line can be determined according to the concave points belonging to the same target, which can make the image segmentation more accurate.

[0054] In some embodiments, Figure 6 As shown, step S52 determines the segmentation lines of multiple targets based on each overlapping area and multiple concave points in the overlapping area, which may include: step S521, pairing multiple concave points in each overlapping area in pairs to form one or more concave point pairs, wherein each concave point belongs to only one concave point pair; step S522, connecting two concave points corresponding to each concave point pair to form a segmentation line. Fig. 9 (e) Fig.10As shown in (d), the overlapping area can be used to determine the position where the two targets overlap and adhere, and the concave point is located at the junction of the edge contours of the two targets. Therefore, multiple concave points in each overlapping area can be paired to form one or more concave point pairs. Each concave point pair can correspond to two overlapping and adhering targets, so as to determine the segmentation line based on the concave point pair. Each concave point belongs to only one concave point pair and cannot be paired with multiple concave points, so as to ensure the accuracy of image segmentation and avoid over-segmentation caused by multiple pairings of a concave point. The two concave points corresponding to each concave point pair can be connected as endpoints to form a segmentation line. Each segmentation line can segment two overlapping and adhering targets. The segmentation line is not the actual edge contour line of the two targets, but a line segment used to distinguish two different targets. By confirming the segmentation line, it is easy to confirm the position of the target and facilitate the subsequent classification and detection of each target. By confirming the concave point pairs of the concave points in the overlapping area and connecting them to obtain the segmentation line, it is possible to avoid over-segmentation and missed segmentation, thereby improving the accuracy of image segmentation.

[0055] In some embodiments, Figure 7 As shown, step S521, pairing multiple concave points in each overlapping area to form one or more concave point pairs, may include: step S5211, determining the number of concave points in each overlapping area; step S5212, if the overlapping area contains two concave points, determining the two concave points as a concave point pair; step S5213, if the overlapping area contains three or more concave points, determining the two unpaired concave points as a concave point pair. In the process of concave point pairing, the number of concave points contained in each overlapping area may be determined first, such as Fig. 9 As shown in , an overlapping area may contain two concave points. In this case, the two concave points can be directly paired as a concave point pair, and the line connecting them is the dividing line of the two objects corresponding to the overlapping area. Therefore, all overlapping areas containing two concave points can be paired first. Fig. 9 As shown in (e), the concave points 21 and 22 located in the overlapping area of ​​the frame 11 and the frame 12 can be paired, and the concave points 23 and 24 located in the overlapping area of ​​the frame 12 and the frame 13 can be paired. After all the concave points are paired, Fig. 9 As shown in (f), all pairs of concave points are connected, concave points 21 and 22 are connected to confirm the dividing line 31; concave points 23 and 24 are connected to confirm the dividing line 32. Fig.10As shown, an overlapping area may also contain three or more concave points. After all overlapping areas containing two concave points have been paired, the overlapping areas containing three or more concave points can be paired, among which some concave points have been paired. Since each concave point can only be paired once, the number of unpaired concave points can be confirmed for overlapping areas containing three or more concave points. When the overlapping area only contains two unpaired concave points, the concave points can be paired and determined as concave point pairs. Concave point pairing can be performed on overlapping areas containing two unpaired concave points first. For overlapping areas with more than two unpaired concave points, after all overlapping areas containing two unpaired concave points have been paired, the number of unpaired concave points can be detected again, and the concave point pairing can be continued for overlapping areas containing two unpaired concave points until all concave points in the image have been paired. Finally, all pairs of concave points are connected to confirm the segmentation line. As shown Fig.10 As shown in (d), the overlapping area of ​​frame 14 and frame 16 contains three concave points, namely concave point 25, concave point 26, and concave point 27, and the overlapping area between frame 15 and frame 16 contains two concave points, namely concave point 27 and concave point 28. Therefore, the concave points in the overlapping area of ​​frame 15 and frame 16 are first paired. After the pairing is completed, the overlapping area of ​​frame 14 and frame 16 contains three concave points, among which concave point 27 is a paired concave point, and there are two unpaired concave points, namely concave point 25 and concave point 26. The concave points 25 and concave point 26 in the overlapping area of ​​frame 14 and frame 16 are paired to complete the pairing of all concave points. Fig.10 As shown in (e), all pairs of concave points are connected to confirm the dividing line, and concave points 25 and 26 are connected to confirm the dividing line 33; concave points 27 and 28 are connected to confirm the dividing line 34. If an overlapping area contains only one unpaired concave point, it can be considered that there is a problem with the concave point detection, the system reports an error, and the detection can be automatically suspended, and then the problem can be manually checked. The method disclosed in this embodiment is more suitable for situations where there are many overlapping or adhered targets. It can orderly pair the concave points for each continuous pattern containing multiple targets, and has a higher pairing efficiency. By confirming the number of concave points in each overlapping area and pairing the concave points in different overlapping areas in sequence according to the number of concave points, it is possible to save computing power and improve the efficiency of concave point pairing, thereby improving the efficiency of image segmentation and ensuring the real-time performance of image segmentation.

[0056] In some embodiments, Figure 8As shown, step S521, pairing multiple concave points in each overlapping area in pairs to form one or more concave point pairs, may include: step S5214, determining the number of unpaired concave points in the overlapping area currently containing unpaired concave points; step S5215, if the number of unpaired concave points in the overlapping area currently containing unpaired concave points is equal to 2, then determining the two concave points as a concave point pair; step S5216, if the number of unpaired concave points in the overlapping area currently containing unpaired concave points is greater than 2, then determining the number of unpaired concave points in the next overlapping area containing unpaired concave points, until all concave points are paired. In the process of concave point pairing, each overlapping area contains unpaired concave points, and any overlapping area containing unpaired concave points can be taken as the starting point to pair the concave points in the overlapping area, and then the concave points in each overlapping area containing unpaired concave points are paired in sequence. First, the number of unpaired concave points in the overlapping region currently containing unpaired concave points can be determined. When the number of unpaired concave points in the overlapping region currently containing unpaired concave points is equal to 2, the two concave points can be determined as a concave point pair, and the concave point pairing of the current overlapping region is completed. Then, the next overlapping region containing unpaired concave points can be identified. Fig. 9 As shown in (e), the overlapping area of ​​frame 11 and frame 12 can be identified first, and the overlapping area includes two unpaired concave points, namely concave point 21 and concave point 22, which are paired; then the overlapping area of ​​frame 12 and frame 13 is identified, and the overlapping area includes two unpaired concave points, namely concave point 23 and concave point 24, which are paired. At this time, all the concave points in the overlapping area have been paired, and can be as shown in FIG. Fig. 9 As shown in (f), the concave point pairs are connected to determine the dividing line. Concave points 21 and 22 are connected to confirm dividing line 31; concave points 23 and 24 are connected to confirm dividing line 32. When the number of unpaired concave points in the overlapping area currently containing unpaired concave points is greater than 2, the current overlapping area is retained and the concave point pairing is not performed temporarily. The number of unpaired concave points in the next overlapping area containing unpaired concave points is confirmed first. After all overlapping areas have completed the confirmation of the number of concave points, for the overlapping areas that have not yet been paired, the number of unpaired concave points inside is confirmed again, and when the number of unpaired concave points in the overlapping area is equal to 2, the two concave points are confirmed as a concave point pair until all concave points are paired. Fig.10As shown in (d), the overlapping area of ​​frame 14 and frame 16 can be identified first, and the overlapping area includes three unpaired concave points, namely concave point 25, concave point 26, and concave point 27, so the current area is retained and the concave point pairing is not performed temporarily; then the overlapping area of ​​frame 15 and frame 16 is identified, and the overlapping area includes two unpaired concave points, namely concave point 27 and concave point 28, which are paired; then the overlapping area of ​​frame 14 and frame 16 is identified, and the overlapping area includes three concave points, including one paired concave point 27, and the remaining two unpaired concave points are concave point 25 and concave point 26, and the two unpaired concave points are paired. At this time, the concave points in all overlapping areas have been paired, and can be as shown in FIG. Fig.10 As shown in (e), the concave point pairs are connected to determine the segmentation line, and concave points 25 and 26 are connected to determine the segmentation line 33; concave points 27 and 28 are connected to determine the segmentation line 34. The method disclosed in this embodiment is more suitable for situations with fewer overlapping or sticky targets, has better concave point pairing efficiency for loosely arranged materials, and has low requirements for computing power, which is easy to implement. Through the method of this embodiment, each overlapping area can be traversed in order and concave point pairing can be performed on it, which can ensure that all concave points are paired, improve the accuracy of concave point pairing, effectively avoid missed segmentation or over-segmentation, and improve the accuracy of image segmentation.

[0057] In some embodiments, Fig.11 As shown, the image segmentation method may also include: step S90, according to the outline of each target, through morphological operation, a smooth target outline is obtained. Through step S60, the outline of each target is obtained. For a continuous pattern including a single target, some noise may appear in the process of extracting its outline information, resulting in unclear outline or jagged edges. Morphological operations can be used to remove the noise of the target outline through corrosion or expansion operations, so as to obtain a smoother and clearer target outline. For a continuous pattern including multiple targets, multiple targets in the continuous pattern are segmented by a segmentation line, wherein the outline of a single target includes the outer contour line of the target and the corresponding segmentation line. Since the segmentation line is a straight line connecting two concave points, and the outer contour line of the continuous pattern is a curve, the line at the junction of the two may produce jagged edges or be not smooth enough, or a broken outline may appear. Through morphological operations, the target outline can be closed, so that the target outline is smoother, continuous and natural. Through step S90, the noise of the target contour can be removed to obtain a smoother and clearer target contour, which effectively improves the accuracy of target segmentation and can also improve the accuracy and stability of subsequent operations such as classification and sorting.

[0058] Based on the same inventive concept, the present disclosure also provides a target sorting method, which may include: an image segmentation method as in any of the aforementioned embodiments; and classifying each target through target recognition according to the contour of each target. The target sorting method may first obtain the contour of each target through the image segmentation method as in any of the aforementioned embodiments. For multiple targets that overlap or adhere, multiple different targets may be distinguished according to the contour of each target, and target recognition may be performed on multiple different targets to determine that each target is a required material or a material to be removed, thereby classifying multiple different targets, and facilitating the subsequent sorting of each target according to its category. Through the target sorting method, according to the image, the overlapping or adhered targets in the image are segmented to obtain the contour of each target, and the image segmentation speed is fast and the accuracy is good, so that a relatively accurate target contour can be obtained. Then, the target position is confirmed according to the contour of each target and each target is identified, thereby classifying the target. When the target sorting method of the present disclosure is applied to material sorting equipment, it can enable the detection mechanism 130 to quickly segment and identify the target image, meeting the real-time and accuracy requirements of the detection mechanism 130.

[0059] Based on the same inventive concept, Fig.12 As shown, the present disclosure also provides a material sorting system 200 for sorting materials, wherein the material sorting system 200 may include: an image segmentation module 210, which executes the image segmentation method as described in any of the above embodiments. The image segmentation module 210 may be used to execute the image segmentation method, obtain a target image, process and segment multiple overlapping or contiguous targets, and obtain the contour of each target in the image.

[0060] In some embodiments, Fig.12 As shown, the material sorting system 200 may include: an image recognition module 220, which is used to identify and classify each material in the segmented image. The image recognition module 220 can obtain the outline of each target output by the image segmentation module, locate each target according to the outline of each target, and identify and classify each target.

[0061] Regarding the material sorting system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0062] like Fig.13As shown, an embodiment of the present disclosure provides an electronic device 300. The electronic device 300 includes a memory 310, a processor 320, and an input / output (I / O) interface 330. The memory 310 is used to store instructions. The processor 320 is used to call the instructions stored in the memory 310 to execute the image segmentation method of the embodiment of the present disclosure. The processor 320 is connected to the memory 310 and the I / O interface 330 respectively, for example, through a bus system and / or other forms of connection mechanisms (not shown). The memory 310 can be used to store programs and data, including the program of the image segmentation method involved in the embodiment of the present disclosure. The processor 320 executes various functional applications and data processing of the electronic device 300 by running the program stored in the memory 310.

[0063] In the embodiment of the present disclosure, the processor 320 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 320 can be a central processing unit (CPU) or one or a combination of other forms of processing units with data processing capabilities and / or instruction execution capabilities.

[0064] The memory 310 in the embodiment of the present disclosure may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.

[0065] In the embodiment of the present disclosure, the I / O interface 330 may be used to receive input instructions (such as digital or character information, and key signal input related to user settings and function control of the electronic device 300), and may also output various information to the outside (such as images or sounds). In the embodiment of the present disclosure, the I / O interface 330 may include one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel.

[0066] It is to be understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0067] The methods and devices involved in the embodiments of the present disclosure can be implemented using standard programming techniques, using rule-based logic or other logic to implement various method steps. It should also be noted that the words "device" and "module" used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0068] Any steps, operations or procedures described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product including a computer-readable medium containing computer program code, which can be executed by a computer processor to perform any or all of the described steps, operations or procedures.

[0069] The foregoing description of the implementation of the present disclosure has been given for the purpose of illustration and description. The foregoing description is not exhaustive nor is it intended to limit the present disclosure to the exact form disclosed, and various variations and modifications may exist in accordance with the above teachings or may be obtained from the practice of the present disclosure. These embodiments are selected and described in order to illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can utilize the present disclosure in various embodiments and various modifications suitable for the specific purpose contemplated.

[0070] The present application uses specific words to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0071] In the context of this application, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0072] Similarly, it should be noted that in order to simplify the description of the disclosure of this application and thus help understand one or more application embodiments, in the above description of the embodiments of this application, multiple features are sometimes merged into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the object of this application are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0073] The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is only for example and does not constitute a limitation of the present application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements and amendments to the present application. Such modifications, improvements and amendments are suggested in the present application, so such modifications, improvements and amendments still belong to the spirit and scope of the embodiments of the present application.

Claims

1. An image segmentation method, comprising: Acquire an image containing multiple objects; Performing target recognition according to the image to obtain an external bounding box of each target; Based on the image, determining edge information of continuous patterns in the image, wherein each of the continuous patterns includes one or more objects; Based on edge information of a continuous pattern including multiple targets, concave point recognition is performed to obtain multiple concave points; Determining a segmentation line of a plurality of objects in the continuous pattern including a plurality of objects based on the plurality of concave points and the corresponding plurality of external bounding boxes of the continuous pattern including a plurality of objects; Based on the edge information of the continuous pattern including the multiple objects and the segmentation line, the outline of each object in the continuous pattern including the multiple objects is obtained.

2. The image segmentation method according to claim 1, further comprising: Determine the number of bounding boxes corresponding to the edge information of each of the continuous patterns in the image; If the edge information of the continuous pattern corresponds to an external frame, the continuous pattern includes an object, and the contour of the object is obtained based on the edge information of the continuous pattern; If the edge information of the continuous pattern corresponds to a plurality of bounding boxes, the continuous pattern includes a plurality of targets, and the concave point recognition is performed based on the edge information of the continuous pattern including the plurality of targets to obtain a plurality of concave points.

3. The image segmentation method according to claim 1, wherein: The determining, based on the plurality of concave points and the corresponding plurality of external frames of the continuous pattern including the plurality of targets, the segmentation lines of the plurality of targets in the continuous pattern including the plurality of targets comprises: According to each of the outer bounding boxes including a plurality of objects, determining an overlapping area between every two intersecting outer bounding boxes; Based on each of the overlapping regions and a plurality of concave points within the overlapping regions, segmentation lines of the plurality of objects are determined.

4. The image segmentation method according to claim 3, wherein: The step of determining the segmentation lines of the multiple objects based on each of the overlapping regions and the multiple concave points in the overlapping regions comprises: Pairing the plurality of concave points in each of the overlapping regions in pairs to form one or more concave point pairs, wherein each of the concave points belongs to only one concave point pair; The two concave points corresponding to each concave point pair are connected to form the dividing line.

5. The image segmentation method according to claim 4, wherein: The step of pairing the plurality of concave points in each overlapping region in pairs to form one or more concave point pairs comprises: Determining the number of the concave points in each of the overlapping regions; If the overlapping area includes two concave points, the two concave points are determined as a concave point pair; If the overlapping region includes three or more than three concave points, two unpaired concave points therein are determined as a concave point pair.

6. The image segmentation method according to claim 4, wherein: The step of pairing the plurality of concave points in each overlapping region in pairs to form one or more concave point pairs comprises: Determine the number of unpaired concave points in the overlapping region currently containing the unpaired concave points; If the number of unpaired concave points in the overlapping region currently containing unpaired concave points is equal to 2, the two concave points are determined as a concave point pair; If the number of unpaired concave points in the overlapping region currently containing unpaired concave points is greater than 2, the number of unpaired concave points in the next overlapping region containing unpaired concave points is determined until all concave points are paired.

7. The image segmentation method according to claim 1, wherein: The step of determining edge information of continuous patterns in the image based on the image further includes: According to the image, binarization processing is performed to obtain a binary image; According to the binary image, edge information of continuous patterns in the image is determined by contour extraction.

8. The image segmentation method according to claim 1, further comprising: According to the outline of each target, a smooth target outline is obtained through morphological operations.

9. A target sorting method comprising: The image segmentation method as claimed in claims 1 to 8; According to the outline of each target, each target is classified through target recognition.

10. A material sorting system for sorting materials, wherein: The material sorting system comprises: An image segmentation module executes the image segmentation method as described in claims 1-8.

11. The material sorting system according to claim 10, wherein: The material sorting system comprises: The image recognition module is used to identify and classify the materials in the segmented images.

12. A computer-readable storage medium storing the following program, wherein the program is used to execute the image segmentation method according to any one of claims 1 to 8.

Images