An image segmentation method, a target sorting method, and a material sorting system

Abstract

This disclosure relates to the field of material sorting, specifically to an image segmentation method, a target sorting method, and a material sorting system. The image segmentation method includes: acquiring an image containing multiple targets; performing target recognition based on the image to obtain the bounding box of each target; determining edge information of a continuous pattern in the image based on the image; identifying concave points based on the edge information of the continuous pattern containing multiple targets to obtain multiple concave points; determining the dividing lines of the multiple targets in the continuous pattern based on the multiple concave points and their corresponding bounding boxes; and obtaining the contour of each target in the continuous pattern based on the edge information and dividing lines. This disclosure can more accurately extract the contour of each target in an image, improving the accuracy of image segmentation, enabling rapid detection, providing better real-time performance, and improving both the efficiency and accuracy of image segmentation.

Description

Technical Field

[0001] This disclosure relates to the field of material sorting, specifically to an image segmentation method, a target sorting method, and a material sorting system, and to a computer-readable storage medium. Background Technology

[0002] In the material sorting process, it is necessary to acquire and segment images of each material to distinguish overlapping or adhered materials, facilitating subsequent identification and classification. However, due to the diverse shapes and sizes of materials, and their close arrangement during sorting, the acquired images often exhibit adhesion and overlap, necessitating image segmentation. Current material image segmentation technologies suffer from low accuracy, especially when dealing with overlapping or adhered images of three or more targets, leading to oversegmentation and undersegmentation, failing to meet the accuracy requirements of material sorting. Furthermore, other image segmentation techniques are computationally intensive, time-consuming, lack real-time performance, and are inefficient, failing to meet production demands. Summary of the Invention

[0003] To overcome the problems existing in related technologies, an exemplary embodiment of this disclosure provides an image segmentation method in the first aspect, comprising: acquiring an image containing a plurality of targets; performing target recognition based on the image to obtain a bounding box for each target; determining edge information of a continuous pattern in the image based on the image, wherein each continuous pattern includes one or more targets; performing concave point recognition based on the edge information of the continuous pattern including the plurality of targets to obtain a plurality of concave points; determining a segmentation line of the plurality of targets in the continuous pattern including the plurality of targets based on the plurality of concave points and the corresponding plurality of bounding boxes; and obtaining the contour of each target in the continuous pattern including the plurality of targets based on the edge information of the continuous pattern including the plurality of targets and the segmentation line.

[0004] In some embodiments, the image segmentation method further includes: determining the number of bounding boxes corresponding to the edge information of each continuous pattern in the image; if the edge information of the continuous pattern corresponds to one bounding box, then the continuous pattern includes one target, and the contour of the target is obtained based on the edge information of the continuous pattern; if the edge information of the continuous pattern corresponds to multiple bounding boxes, then the continuous pattern includes multiple targets, and the concave point recognition is performed based on the edge information of the continuous pattern including multiple targets to obtain multiple concave points.

[0005] In some embodiments, determining the dividing line of the multiple targets in the continuous pattern comprising multiple targets based on the multiple concave points and the corresponding multiple bounding boxes of the continuous pattern comprising multiple targets includes: determining an overlapping region between every two intersecting bounding boxes based on each of the bounding boxes comprising multiple targets; and determining the dividing line of the multiple targets based on each overlapping region and the multiple concave points within the overlapping region.

[0006] In some embodiments, determining the dividing line of the plurality of targets based on each of the overlapping regions and the plurality of concave points within the overlapping regions includes: pairing the plurality of concave points in each of the overlapping regions to form one or more concave point pairs, wherein each concave point belongs to only one concave point pair; and connecting the two concave points corresponding to each concave point pair to form the dividing line.

[0007] In some embodiments, pairing multiple concave points in each overlapping region to form one or more concave point pairs includes: determining the number of concave points in each overlapping region; if the overlapping region contains two concave points, then determining the two concave points as a concave point pair; if the overlapping region contains three or more concave points, then determining two unpaired concave points as a concave point pair.

[0008] In some embodiments, pairing multiple concave points in each overlapping region to form one or more concave point pairs includes: determining the number of unpaired concave points in the current overlapping region containing unpaired concave points; if the number of unpaired concave points in the current overlapping region containing unpaired concave points is equal to 2, then determining the two concave points as a concave point pair; if the number of unpaired concave points in the current overlapping region containing unpaired concave points is greater than 2, then determining the number of unpaired concave points in the next overlapping region containing unpaired concave points, until all concave points are paired.

[0009] In some embodiments, determining the edge information of a continuous pattern in the image based on the image further includes: performing binarization processing on the image to obtain a binary image; and determining the edge information of the continuous pattern in the image by contour extraction based on the binary image.

[0010] In some embodiments, the method further includes obtaining a smooth target contour by morphological operations based on the contour of each target.

[0011] Secondly, this disclosure also provides a target sorting method, including: the image segmentation method as described in the first aspect; classifying each target according to its contour through target recognition.

[0012] Thirdly, this disclosure also provides a material sorting system for sorting materials, wherein the material sorting system includes: an image segmentation module that performs the image segmentation method as described in the first aspect.

[0013] In some embodiments, the material sorting system includes an image recognition module for identifying and classifying the materials in the segmented image.

[0014] Fourthly, this disclosure also provides a computer-readable storage medium storing a program for performing the image segmentation method as described in the first aspect.

[0015] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: by quickly locating the number and position of targets through target recognition, and combining the concave point detection results of edge information, the segmentation line of an image containing multiple targets is determined, thereby obtaining the contour of each target in the image. This can more accurately extract the contour of each target in the image, effectively improving the accuracy of image segmentation. At the same time, it can detect quickly, has better real-time performance, and effectively improves the efficiency and accuracy of image segmentation.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0017] This disclosure can be better understood by describing exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which:

[0018] Figure 1 This is a schematic diagram of a sorting device structure shown according to an exemplary embodiment disclosed in a publication;

[0019] Figure 2 This is a schematic diagram of an image segmentation method according to an exemplary embodiment of a publication;

[0020] Figure 3 This is a schematic flowchart of an image segmentation method according to another exemplary embodiment disclosed.

[0021] Figure 4 This is a schematic flowchart of an image segmentation method according to another exemplary embodiment disclosed.

[0022] Figure 5 This is a schematic diagram of an image segmentation method flow according to another exemplary embodiment disclosed;

[0023] Figure 6 This is a schematic diagram of an image segmentation method flow according to another exemplary embodiment disclosed;

[0024] Figure 7 This is a schematic diagram of an image segmentation method flow according to another exemplary embodiment disclosed;

[0025] Figure 8 This is a schematic diagram of an image segmentation method flow according to another exemplary embodiment disclosed;

[0026] Figure 9 This is a schematic diagram illustrating the process of segmenting an image according to an image segmentation method, based on an exemplary embodiment of a disclosed document.

[0027] Figure 10 This is a schematic diagram illustrating the process of segmenting an image according to an image segmentation method, based on an exemplary embodiment disclosed in another publication.

[0028] Figure 11 This is a schematic diagram of an image segmentation method flow according to another exemplary embodiment disclosed;

[0029] Figure 12 This is a schematic block diagram of a material sorting system illustrated in an exemplary embodiment of a publication;

[0030] Figure 13 This is a schematic block diagram of an electronic device illustrated according to an exemplary embodiment of a published document. Detailed Implementation

[0031] The following describes specific embodiments of this disclosure. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. It should be understood that, in the actual implementation of any embodiment, just as in any engineering or design project, various specific decisions are often made to achieve the developer's specific goals and to meet system-related or business-related constraints, and this can change from one embodiment to another. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this disclosure, changes in design, manufacturing, or production based on the technical content disclosed in this disclosure are merely conventional technical means and should not be construed as insufficient content of this disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in the claims and description shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this patent application description and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. The terms “an” or “a” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising” or “including” and similar terms mean that the element or object preceding “comprising” or “including” encompasses the element or object listed following “comprising” or “including” and its equivalents, and do not exclude other elements or objects. The terms “connected” or “linked” and similar terms are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0033] The image segmentation method provided in this disclosure can be applied to the segmentation of target image edges. In some embodiments, it can be applied to sorting equipment in the fields of ores, food, or others. Figure 1 As shown, the material sorting equipment 100 can be used to sort materials and may include a feeding mechanism 110, a conveying mechanism 120, a detection mechanism 130, and a sorting device 140. The feeding mechanism 110 feeds the material to be sorted into the conveying mechanism 120. The conveying mechanism 120 can be a conveyor belt or chute, etc., used to transport the material to be sorted fed by the feeding mechanism 110. The detection mechanism 130 is used to detect the material conveyed on the conveying mechanism 120 to determine whether the material is to be rejected; the material to be rejected refers to the material that will be separated by the sorting equipment. The material to be rejected can be desired material or undesirable material, as long as the material can be sorted. The sorting device 140 is used to reject the material to be rejected. The material sorting equipment can be used for ore sorting, food sorting, or waste sorting, etc. Especially in the process of ore sorting, due to the more irregular shape of the ore and the more uncertain relative positions of multiple ores, the materials to be sorted may overlap or stick together during the sorting process of the material sorting equipment. Therefore, when the material sorting equipment's detection mechanism 130 performs material detection, it needs to separate the multiple overlapping or sticking targets in order to distinguish them.

[0034] In some related technologies, machine learning-based segmentation methods can be used to segment multiple overlapping or adhered targets. Instance segmentation techniques are used to establish instance segmentation models of the targets, enabling the segmentation of multiple overlapping or adhered targets. However, machine learning-based instance segmentation methods require building models of the target materials and training on a large amount of data, resulting in high costs and large model sizes. Therefore, segmentation requires a large amount of computation, leading to slow target segmentation speed and low efficiency.

[0035] In other related technologies, traditional algorithms can be used to segment multiple overlapping or adhered targets, such as morphological segmentation algorithms, watershed algorithms, and concave point detection and matching segmentation algorithms. However, due to the complexity and diversity of images of materials such as ores awaiting sorting, the accuracy of segmentation using traditional algorithms is low, prone to errors or omissions, thus affecting the accuracy of material sorting. In some cases, three or more targets may overlap or adhere. Traditional concave point detection algorithms can determine the segmentation line and identify four or more concave points. Therefore, concave points need to be paired. Traditional algorithms generally pair and connect the two concave points with the shortest distance to form the segmentation line. When there are more than four concave points, pairing errors are prone to occur, leading to incorrect target segmentation and seriously affecting the accuracy of material sorting.

[0036] For the detection mechanism 130 of the material sorting equipment, good real-time performance and accuracy are required. In order to improve output and efficiency, the material transmission speed of the transmission mechanism 120 is constantly increasing. The detection mechanism 130 needs to detect the material in real time within a short period of time. The segmentation method based on machine learning cannot meet the real-time requirements, while the segmentation method based on traditional algorithms has poor accuracy and is prone to over-segmentation, under-segmentation and incorrect segmentation, which cannot meet the accuracy requirements.

[0037] To solve the above problems, such as Figure 2 As shown, this disclosure provides an image segmentation method, which may include steps S10 to S60, and the above steps are described in detail below:

[0038] Step S10: Acquire an image containing multiple targets. This can be done using an image acquisition device, such as a camera, X-ray image acquisition device, or CT scan device. Alternatively, Step S10 can be achieved by reading from storage to acquire an image containing multiple targets from memory. In some cases, an image containing multiple targets can also be generated. The image can contain multiple targets, which may be scattered throughout the image and may overlap or be adhered to each other. For material sorting equipment, an image acquisition device can be used to acquire images of the materials on the conveyor mechanism 120 in the material sorting system, such as... Figure 9 As shown in (a), the material can be ore to be sorted, food to be sorted, or waste to be sorted, etc.

[0039] Step S20: Perform target recognition based on the image to obtain the bounding box of each target. Target recognition can be performed on images containing multiple targets to determine the number of targets and obtain the bounding box of each target. Target recognition in images can be accomplished using target recognition algorithms. Target recognition can be achieved by confirming the bounding box of the target through pattern recognition; it can also be based on deep learning models, such as convolutional neural networks, to extract features and recognize targets in the image. For example, the PP-YOLOE network can be used for target recognition. The accuracy of target recognition can be improved by training deep learning models, increasing the training dataset, increasing the number of training epochs, improving the model structure, and adjusting parameters. Figure 9 (b) Figure 10 As shown in (b), the bounding box of each target in the image can be rectangular or fitted as a circular bounding box. Each target's bounding box can contain its corresponding target. In cases where multiple targets are close together and overlap or adhere to each other, such as... Figure 9 (b) Figure 10 As shown in (b), the bounding boxes corresponding to each overlapping or connected target can have a certain degree of overlap. The bounding boxes can also be non-visual. Through target recognition algorithms, the number of targets in the image can be determined, the position of each target can be determined, the bounding box of each target can be determined, and the position and size data of each bounding box can be stored. Step S20 does not obtain the outer contour line of each target through target recognition, but rather obtains the bounding box containing all the information of the corresponding target. Therefore, the target recognition model is smaller, requires less computation, has a faster response, and higher recognition accuracy.

[0040] Step S30: Based on the image, determine the edge information of continuous patterns in the image, where each continuous pattern includes one or more targets. Image processing algorithms can be used to process the image to determine the edge information of continuous patterns. Continuous patterns can be internally connected and have continuous external contours; a single target can be a continuous pattern, and multiple overlapping or connected targets can also form a continuous pattern. Figure 9 (c) Figure 10 As shown in (c), there can be one or more continuous patterns, and each continuous pattern can include one or more targets. The edge information of a continuous pattern including one target is the edge information of a single target in the image that does not overlap or adhere to other targets. A continuous pattern can also include multiple overlapping or adhered targets. The edge information of a continuous pattern including multiple targets is the total edge information of the region formed by multiple overlapping or adhered targets in the image. The edge information of a continuous pattern can be the outer contour information of the continuous pattern, or it can be the shape information of the continuous pattern that includes contour information.

[0041] In some embodiments, such as Figure 3As shown, step S30, determining the edge information of continuous patterns in the image based on the image, may include: step S31, performing binarization processing on the image to obtain a binary image; step S32, determining the edge information of continuous patterns in the image through contour extraction based on the binary image. Figure 9 (c) Figure 10 As shown in (a), binarization can be performed on the image, setting the grayscale value of each pixel to 0 or 255 to obtain a binary image. Binarization separates the target from the background, making the target clearer and facilitating subsequent image processing. Binarization transforms each target in the image into a continuous pattern with the same grayscale value. This continuous pattern can be a complete color block with internal connectivity and continuous external contours. Binarization can also transform multiple overlapping or adhered targets into a continuous pattern in the binary image, thus merging them into a single complete color block with internal connectivity and continuous external contours. By reducing the number of grayscale levels in the image from multiple to two, binarization effectively simplifies image information and reduces the computational load in subsequent image detection processes. Figure 9 (d) Figure 10 As shown in (c), based on the binary image, the edge information of continuous patterns in the image can be determined through contour extraction. Contour extraction is performed on the binary image to extract the edge information of each continuous pattern. The continuous pattern may include a single target or multiple overlapping or connected targets. The edge information of the continuous pattern in the binary image is obtained through contour extraction. The edge information of the continuous pattern can be the contour line of the continuous pattern, or edge detection can be performed on the continuous pattern to obtain its edge information. Through steps S31 and S32, image information can be simplified, the computational load in the image segmentation process can be reduced, and the efficiency of image segmentation can be improved. At the same time, clearer target edge information can be extracted, thereby improving the accuracy of image segmentation.

[0042] In some embodiments, step S30 may include: performing binarization processing on the image to obtain edge information of continuous patterns in the image. Binarization processing sets the grayscale value of pixels in the image to 0 or 255, simplifying the image and effectively reducing the computational load for subsequent image detection, thus improving the efficiency of image segmentation. The binarized image consists of color blocks that are internally connected and have continuous external contours. Edge information can be obtained from the binarized image; this edge information can be the shape information of the continuous pattern, which can be directly used to obtain the image edge information. Alternatively, contours can be extracted based on the shape information of the continuous pattern to obtain the image edge information. Binarization makes the target clearly distinguishable from the background, making the target clearer. Binarization processing requires less data computation, increasing the speed of edge information acquisition, thereby improving the speed of image segmentation and providing better real-time performance.

[0043] In some embodiments, step S30 may include: determining the edge information of a continuous pattern in the image through contour extraction. Contour extraction can be performed directly on the image; for a single target, the edge information, i.e., the target's contour, can be directly obtained. For multiple targets that overlap or adhere together, contour extraction can obtain the edge information of the continuous pattern formed by the multiple targets, i.e., the outer contour line of the continuous pattern. Through contour extraction, the contour line of each continuous pattern in the image can be directly obtained, and the obtained contour information is more accurate, resulting in high accuracy in obtaining target edge information, thereby effectively improving the accuracy of image segmentation.

[0044] Step S40: Based on the edge information of a continuous pattern including multiple targets, concave point identification is performed to obtain multiple concave points. A concave point can be the lowest point of downward indentation on the edge of the continuous pattern. On the edge of the continuous pattern, a concave point can be located at a position where the edge contour changes abruptly. In the case of overlapping or adhered targets, a concave point can be located at the edge of the boundary between two overlapping or adhered targets. For example... Figure 9 (d) Figure 10 As shown in (c), for the edge information of a continuous pattern, the edge information of the continuous pattern including multiple targets can be extracted, and multiple concave points can be identified through a concave point recognition algorithm. Concave point recognition can be achieved by analyzing the edge information of the continuous pattern including multiple targets, obtaining the curvature of each pixel on the edge, and determining the concave point by analyzing the curvature changes of the points on the edge. Concave point recognition can also be achieved by methods such as angle difference, by detecting the gradient information of each pixel in the edge information of the continuous pattern, and performing difference on the gradient information to determine the concave point. Processing the edge information of a continuous pattern with multiple targets through concave point recognition can obtain concave points more quickly, save time, improve the efficiency of image segmentation, and enable the image segmentation method to have better real-time performance when applied to the detection mechanism 130 of the sorting equipment.

[0045] Step S50: Based on multiple concave points and corresponding bounding boxes of a continuous pattern including multiple targets, determine the segmentation lines of multiple targets in the continuous pattern including multiple targets. Through multiple concave points and corresponding bounding boxes of a continuous pattern including multiple targets, multiple targets can form a continuous pattern when they overlap or adhere to each other. A continuous pattern includes multiple targets. For two targets that overlap or adhere to each other, their edges intersect at a concave point. A continuous pattern can contain multiple concave points. In some cases, when three or more targets overlap or adhere to each other, a continuous pattern can contain four or more concave points. The bounding boxes corresponding to each target may overlap. Existing concave point pairing methods generally subtract the concave shape from the convex closure in the image to obtain the concave region. The two points with the shortest distance in the concave region are then selected as concave points and paired. In some cases, the two shortest-distance concave points may not belong to the same target, potentially leading to incorrect concave point pairing and causing errors in subsequent image segmentation. This disclosure integrates object detection and concave point detection. The bounding box obtained from object detection includes positional information, which can be used to confirm the region where the object is located. Concave points can be combined and paired using different bounding boxes as boundaries. The determined segmentation line is not the actual edge of the object, but rather a boundary line that separates each object. The segmentation line can distinguish the boundaries of each object, facilitating subsequent object recognition. Through step S150 of this disclosure, the segmentation line of multiple objects can be determined by comprehensively considering the bounding box and its internal concave point information. By pairing concave points and connecting the endpoints of the paired concave points, the segmentation line of multiple objects can be determined, making the pairing of concave points more accurate, thereby improving the accuracy of image segmentation.

[0046] Step S60: Based on the edge information and dividing lines of the continuous pattern including multiple targets, the contour of each target in the continuous pattern including multiple targets is obtained. For example... Figure 9 (g) Figure 10 As shown in (f), a continuous pattern containing multiple targets can be divided into multiple patterns according to the dividing line. The closed curve formed by the edge information of each pattern and the dividing line is the outline of each target in the continuous pattern containing multiple targets.

[0047] The image segmentation method disclosed herein utilizes deep learning algorithms for target recognition, obtaining bounding boxes for each target in the image. This enables faster and more accurate determination of the number of targets and their locations, improving the efficiency of target recognition and image segmentation. A concave point recognition algorithm is used to identify concave points between overlapping and adhered targets, resulting in faster recognition speed. Combining the concave points with edge information of continuous patterns containing multiple targets confirms the segmentation line, ultimately segmenting overlapping and adhered targets and obtaining the contour of each target with higher segmentation accuracy. The combination of deep learning algorithms and image recognition technology effectively improves image segmentation speed while maintaining accuracy. When applied to actual production, this method offers better real-time performance. During material sorting, real-time image segmentation is possible, leading to greater efficiency and improved sorting accuracy. This, in turn, increases material conveying speed and recognition efficiency during material sorting, thereby increasing production capacity. Traditional methods for determining segmentation lines using concave point detection are prone to errors in image segmentation due to mismatched concave points when three or more targets overlap or are adhered to each other, resulting in low accuracy. The image segmentation method provided in this disclosure combines concave point detection with neural network target detection technology. By combining the bounding box obtained from target detection with concave point information, concave point pairs are matched, effectively avoiding mismatched concave point pairing and significantly improving the accuracy of image segmentation. By identifying the target location and quantity and obtaining the target's bounding box, and combining the concave point detection results with edge information, the segmentation line of an image containing multiple targets is determined, thereby obtaining the contour of each target in the image. This allows for more accurate extraction of the contour of each target in the image, while avoiding the time-consuming nature of traditional instance segmentation techniques. It enables rapid detection, has better real-time performance, and effectively improves the efficiency of image segmentation, meeting the real-time requirements of subsequent target sorting operations.

[0048] To better illustrate the advantages of the image segmentation method of this disclosure compared with traditional methods, please refer to the performance test results, as shown in Table 1:

[0049] Table 1:

[0050] method Segmentation accuracy (%) Calculation time (ms) Traditional concave detection algorithm 80 5 Instance segmentation techniques 95 80 This disclosure discloses an image segmentation method. 94 20

[0051] The test involved image segmentation of identical images (ore images), as shown in Table 1. The traditional concave point detection algorithm segments the image using concave point recognition, while the instance segmentation technique is a machine learning-based segmentation method that identifies the edge contours of each target based on the instance segmentation model. Table 1 clearly shows that the image segmentation method provided in this disclosure achieves an accuracy of 94%, significantly higher than the 80% of the traditional concave point detection algorithm and close to the 95% of the instance segmentation technique, demonstrating high segmentation accuracy and meeting the accuracy requirements for material sorting. Furthermore, regarding computation time, the image segmentation method provided in this disclosure takes 20ms, longer than the 5ms of the traditional algorithm but significantly shorter than the 80ms of the instance segmentation technique, indicating higher computational efficiency and meeting the real-time requirements of material sorting. Considering both segmentation accuracy and computation time, the image segmentation method provided in this disclosure significantly improves segmentation accuracy while maintaining short computation time, demonstrating high image segmentation efficiency and simultaneously meeting the real-time and accuracy requirements of material sorting.

[0052] In some embodiments, such as Figure 4 As shown, the image segmentation method may further include: step S70, confirming the number of bounding boxes corresponding to the edge information of each continuous pattern in the image; if the edge information of the continuous pattern corresponds to one bounding box, then the continuous pattern includes one target, and step S80 is executed to obtain the contour of a target based on the edge information of the continuous pattern; if the edge information of the continuous pattern corresponds to multiple bounding boxes, then the continuous pattern includes multiple targets, and step S40 is executed to perform concave point recognition based on the edge information of the continuous pattern including multiple targets to obtain multiple concave points. By confirming the number of bounding boxes corresponding to the edge information of each continuous pattern in the image, the number of targets corresponding to the edge information of each continuous pattern in the image can be confirmed. When the edge information of the continuous pattern corresponds to one bounding box, it can be determined that the current continuous pattern includes only one target, so the contour of the current continuous pattern can be directly obtained, which is the contour of an independent target. When the edge information of a continuous pattern corresponds to multiple bounding boxes, it can be determined that the current continuous pattern includes multiple overlapping or connected targets. However, if the contour of the current continuous pattern is obtained in this case, the common outer contour of the overlapping or connected targets will be obtained, making it impossible to distinguish the edges of each target. Therefore, when the edge information of a continuous pattern corresponds to multiple bounding boxes, the edge information of the continuous pattern needs to be processed to facilitate image segmentation. Concave points can be quickly obtained using a concave point recognition algorithm, facilitating subsequent determination of the target segmentation lines. By confirming the number of bounding boxes corresponding to the edge information of each continuous pattern in the image, different processing methods can be adopted according to the number of bounding boxes. Concave point recognition can be performed only on continuous patterns containing multiple targets, effectively saving computational power and improving the efficiency of image segmentation.

[0053] In some embodiments, such as Figure 5 As shown, step S50, based on multiple concave points of a continuous pattern including multiple targets and corresponding multiple bounding boxes, determines the dividing lines of multiple targets in the continuous pattern including multiple targets. This may include: step S51, determining the overlapping area between every two intersecting bounding boxes based on each bounding box including multiple targets; and step S52, determining the dividing lines of multiple targets based on each overlapping area and multiple concave points within the overlapping area. A continuous pattern including multiple targets may correspond to multiple bounding boxes, each bounding box corresponding to one target. Since the bounding box is not the outer contour line of each target, but rather an area containing all the information of the corresponding target, it is convenient to determine the number of targets and their locations. The range selected by the bounding box is larger than the range selected by the outer contour line of the corresponding target. Therefore, for multiple overlapping or adhered targets, there may be a certain overlap area between the corresponding multiple bounding boxes. By determining the overlap area between every two intersecting bounding boxes, the position of the dividing line can be located based on the overlap area. Since the overlapping region is located at the boundary of adjacent targets in a continuous pattern containing multiple targets, and the concave point is also located at the boundary of adjacent target edges in a continuous pattern containing multiple targets, the concave point can be located within the overlapping region, and each overlapping region can contain multiple concave points. Because the concave point is located at the boundary of adjacent target edges in a continuous pattern containing multiple targets, and each overlapping region can determine the location where two corresponding targets overlap and adhere. Based on an overlapping region and its multiple concave points, the location where the two targets corresponding to the overlapping region overlap and adhere can be determined. Therefore, the segmentation line of multiple targets can be determined based on each overlapping region and its multiple concave points. The segmentation line can be determined by connecting concave points located within the same overlapping region pairwise, using the concave point as an endpoint. By jointly analyzing the overlapping region of the circumbound box and its internal concave points, concave points belonging to the same target can be identified more quickly and accurately. Determining the segmentation line based on the concave points belonging to the same target makes image segmentation more accurate.

[0054] In some embodiments, such as Figure 6 As shown, step S52, based on each overlapping region and multiple concave points within the overlapping region, determines the dividing line of multiple targets, which may include: step S521, pairing multiple concave points in each overlapping region to form one or more concave point pairs, wherein each concave point belongs to only one concave point pair; step S522, connecting the two concave points corresponding to each concave point pair to form a dividing line. Figure 9 (e) Figure 10As shown in (d), the overlapping region can be used to determine the location where two targets overlap and adhere. Concave points are located at the intersection of the edge contours of the two targets. Therefore, multiple concave points within each overlapping region can be paired to form one or more concave point pairs. Each concave point pair corresponds to two overlapping and adhered targets, thus determining the segmentation line based on the concave point pairs. Each concave point belongs to only one concave point pair and cannot be paired with multiple concave points, ensuring the accuracy of image segmentation and avoiding oversegmentation caused by multiple pairings of a single concave point. The two concave points corresponding to each concave point pair can be connected as endpoints to form a segmentation line. Each segmentation line can separate two overlapping and adhered targets. The segmentation line is not the actual edge contour of the two targets, but rather a line segment used to distinguish two different targets. Identifying the segmentation line facilitates the confirmation of the target's location, making subsequent classification and detection of each target easier. By identifying concave point pairs within the overlapping region and connecting them to obtain the segmentation line, oversegmentation and undersegmentation can be avoided, improving the accuracy of image segmentation.

[0055] In some embodiments, such as Figure 7 As shown, step S521, which pairs multiple concave points within each overlapping region to form one or more concave point pairs, may include: step S5211, determining the number of concave points within each overlapping region; step S5212, if the overlapping region contains two concave points, then determining the two concave points as a concave point pair; step S5213, if the overlapping region contains three or more concave points, then determining the two unpaired concave points as a concave point pair. During the concave point pairing process, the number of concave points contained within each overlapping region can be determined first, such as... Figure 9 As shown, an overlapping region may contain two concave points. In this case, the two concave points can be directly paired, and the line connecting them is the dividing line between the two targets corresponding to the overlapping region. Therefore, all overlapping regions containing two concave points can be paired first. Figure 9 As shown in (e), concave points 21 and 22 located in the overlapping area of ​​frames 11 and 12 can be paired, and concave points 23 and 24 located in the overlapping area of ​​frames 12 and 13 can be paired. After all the concave points have been paired, then... Figure 9 As shown in (f), connect all pairs of concave points; connect concave points 21 and 22 to confirm dividing line 31; connect concave points 23 and 24 to confirm dividing line 32. Figure 10As shown, an overlapping region may contain three or more concave points. After all overlapping regions containing two concave points have been paired, concave point pairing can be performed on overlapping regions containing three or more concave points. Some concave points have already been paired. Since each concave point can only be paired once, for overlapping regions containing three or more concave points, the number of unpaired concave points can be determined. When the overlapping region contains only two unpaired concave points, the concave points can be paired to determine a concave point pair. First, concave point pairing can be performed on overlapping regions containing two unpaired concave points. For overlapping regions with more than two unpaired concave points, after all overlapping regions containing two unpaired concave points have been paired, the number of unpaired concave points is detected again, and concave point pairing continues on overlapping regions containing two unpaired concave points until all concave points in the image have been paired. Finally, all concave point pairs are connected to confirm the dividing line. For example... Figure 10 As shown in (d), the overlapping area of ​​boxes 14 and 16 contains three concave points, namely concave point 25, concave point 26, and concave point 27, while the overlapping area between boxes 15 and 16 contains two concave points, namely concave point 27 and concave point 28. Therefore, the concave points in the overlapping area of ​​boxes 15 and 16 are first paired. After pairing, the overlapping area of ​​boxes 14 and 16 contains three concave points, of which concave point 27 is a paired concave point, and the remaining two unpaired concave points 25 and 26 are paired. The concave points 25 and 26 in the overlapping area of ​​boxes 14 and 16 are then paired to complete the pairing of all concave points. Figure 10 As shown in (e), connect all pairs of concave points to confirm the dividing line. Connect concave points 25 and 26 to confirm dividing line 33; connect concave points 27 and 28 to confirm dividing line 34. If an overlapping region contains only one unpaired concave point, it can be considered that there is a problem with concave point detection, the system will report an error, and detection can be automatically paused, followed by manual troubleshooting. The method disclosed in this embodiment is more suitable for situations with many overlapping or adhered targets, and can orderly pair concave points for each continuous pattern containing multiple targets, resulting in higher pairing efficiency. By confirming the number of concave points in each overlapping region, and pairing concave points in different overlapping regions in sequence according to the number of concave points, computing power can be saved, the efficiency of concave point pairing can be improved, thereby improving the efficiency of image segmentation and ensuring the real-time performance of image segmentation.

[0056] In some embodiments, such as Figure 8As shown, step S521, which pairs multiple concave points within each overlapping region to form one or more concave point pairs, may include: step S5214, determining the number of unpaired concave points in the current overlapping region containing unpaired concave points; step S5215, if the number of unpaired concave points in the current overlapping region containing unpaired concave points is equal to 2, then the two concave points are determined as a concave point pair; step S5216, if the number of unpaired concave points in the current overlapping region containing unpaired concave points is greater than 2, then the process of determining the number of unpaired concave points in the next overlapping region containing unpaired concave points is performed until all concave points are paired. During the concave point pairing process, each overlapping region contains unpaired concave points. One overlapping region containing unpaired concave points can be arbitrarily selected as the starting point, and the concave points within that overlapping region are paired. Then, the pairing of concave points within each overlapping region containing unpaired concave points is completed sequentially. First, the number of unpaired dimples in the current overlapping region containing unpaired dimples can be determined. If the number of unpaired dimples in the current overlapping region containing unpaired dimples is equal to 2, then the two dimples can be identified as a dimple pair, completing the dimple pairing for the current overlapping region. Subsequently, the next overlapping region containing unpaired dimples can be identified. For example... Figure 9 As shown in (e), the overlapping area between boxes 11 and 12 can be identified first. This overlapping area includes two unpaired concave points, namely concave point 21 and concave point 22, which are then paired. Next, the overlapping area between boxes 12 and 13 can be identified. This overlapping area also includes two unpaired concave points, namely concave point 23 and concave point 24, which are then paired. At this point, all concave points within the overlapping areas have been paired, and the process can proceed as follows: Figure 9 As shown in (f), connect the concave point pairs to determine the dividing line; connect concave points 21 and 22 to confirm dividing line 31; connect concave points 23 and 24 to confirm dividing line 32. If the number of unpaired concave points in the current overlapping region containing unpaired concave points is greater than 2, then the current overlapping region is temporarily left unpaired. First, the number of unpaired concave points in the next overlapping region containing unpaired concave points is confirmed. After all overlapping regions have completed one confirmation of the number of concave points, for overlapping regions that have still not been paired, the number of unpaired concave points within them is confirmed again. If the number of unpaired concave points in the overlapping region is equal to 2, the two concave points are confirmed as a pair, until all concave points are paired. Figure 10As shown in (d), the overlapping area between boxes 14 and 16 can be identified first. This overlapping area includes three unpaired concave points, namely concave point 25, concave point 26, and concave point 27. Therefore, the current area is retained without concave point pairing. Then, the overlapping area between boxes 15 and 16 is identified. This overlapping area includes two unpaired concave points, namely concave point 27 and concave point 28, which are paired. Next, the overlapping area between boxes 14 and 16 is identified. This overlapping area includes three concave points, including one paired concave point 27, and the remaining two unpaired concave points are concave point 25 and concave point 26. The two unpaired concave points are paired. At this point, the concave points in all overlapping areas have been paired, and can be processed as follows. Figure 10 As shown in (e), connecting the concave point pairs determines the dividing line; connecting concave points 25 and 26 determines the dividing line 33; connecting concave points 27 and 28 determines the dividing line 34. The method disclosed in this embodiment is more suitable for situations with fewer overlapping or adhered targets, offering better concave point pairing efficiency for loosely arranged materials, and requiring less computational power, making it easy to implement. This method allows for sequential traversal of each overlapping region and concave point pairing, ensuring all concave points are paired, improving the accuracy of concave point pairing, effectively avoiding missed or over-segmentation, and improving the accuracy of image segmentation.

[0057] In some embodiments, such as Figure 11 As shown, the image segmentation method may further include: Step S90, obtaining a smooth target contour through morphological operations based on the contour of each target. Step S60 obtains the contour of each target. For continuous patterns containing a single target, noise may occur during the extraction of its contour information, resulting in unclear contours or jagged edges. Morphological operations can be used to remove noise from the target contour through erosion or dilation operations, thereby obtaining a smoother and clearer target contour. For continuous patterns containing multiple targets, the multiple targets within the continuous pattern are segmented by segmentation lines. The contour of a single target includes the target's outer contour line and the corresponding segmentation line. Since the segmentation line is a straight line connecting two concave points, while the outer contour line of the continuous pattern is a curve, the line at the intersection of the two may produce jagged edges or be insufficiently smooth, or there may be broken contours. Morphological operations can be used to perform a closing operation on the target contour, thereby making the target contour smoother, more continuous, and more natural. Step S90 removes noise from the target contour, resulting in a smoother and clearer target contour, which effectively improves the accuracy of target segmentation and enhances the accuracy and stability of subsequent classification and sorting operations.

[0058] Based on the same inventive concept, this disclosure also provides a target sorting method, which may include: the image segmentation method as described in any of the foregoing embodiments; and classifying each target according to its contour through target recognition. The target sorting method may first obtain the contour of each target using the image segmentation method as described in any of the foregoing embodiments. For multiple targets that overlap or adhere, multiple different targets can be distinguished based on their contours, and target recognition can be performed on these multiple different targets to determine whether each target is a desired material or a material to be rejected, thereby classifying the multiple different targets and facilitating subsequent sorting of each target according to its category. Through the target sorting method, overlapping or adhered targets in the image are segmented according to the image to obtain the contour of each target. Image segmentation is fast and accurate, and can obtain relatively accurate target contours. Then, the target position is confirmed based on the contour of each target, and target recognition is performed on each target, thereby classifying the targets. When applied to material sorting equipment, the target sorting method of this disclosure enables the detection mechanism 130 to quickly segment and recognize target images, meeting the real-time and accuracy requirements of the detection mechanism 130.

[0059] Based on the same inventive concept, such as Figure 12 As shown, this disclosure also provides a material sorting system 200 for sorting materials. The material sorting system 200 may include an image segmentation module 210, which executes the image segmentation method as described in any of the foregoing embodiments. The image segmentation module 210 can be used to execute the image segmentation method, acquire a target image, process it, and segment multiple overlapping or adhered targets to obtain the contour of each target in the image.

[0060] In some embodiments, such as Figure 12 As shown, the material sorting system 200 may include an image recognition module 220, used to identify and classify each material in the segmented image. The image recognition module 220 can acquire the contour of each target output by the image segmentation module, locate each target based on its contour, and identify and classify each target.

[0061] Regarding the material sorting system in the above embodiments, the specific methods by which each module performs its operations have been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0062] like Figure 13As shown, one embodiment of this disclosure provides an electronic device 300. The electronic device 300 includes a memory 310, a processor 320, and an input / output (I / O) interface 330. The memory 310 is used to store instructions. The processor 320 is used to execute the image segmentation method of this disclosure embodiment by calling the instructions stored in the memory 310. The processor 320 is connected to both the memory 310 and the I / O interface 330, for example, via a bus system and / or other forms of connection mechanism (not shown). The memory 310 can be used to store programs and data, including the program for the image segmentation method involved in the embodiments of this disclosure. The processor 320 executes various functional applications and data processing of the electronic device 300 by running the program stored in the memory 310.

[0063] In this embodiment of the disclosure, the processor 320 may be implemented in at least one of the following hardware forms: digital signal processor (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 320 may be one or a combination of several of the following: central processing unit (CPU) or other processing units with data processing capability and / or instruction execution capability.

[0064] The memory 310 in this embodiment may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD).

[0065] In this embodiment of the disclosure, the I / O interface 330 can be used to receive input instructions (such as numeric or character information, and to generate key signal inputs related to user settings and function control of the electronic device 300), and can also output various information (such as images or sounds) to the outside. In this embodiment of the disclosure, the I / O interface 330 may include one or more of the following: a physical keyboard, function keys (such as volume control keys, power buttons, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel.

[0066] It is understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0067] The methods and apparatus disclosed herein can be implemented using standard programming techniques, and various method steps can be implemented using rule-based logic or other logic. It should also be noted that the terms "apparatus" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0068] Any step, operation, or procedure described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product comprising a computer-readable medium containing computer program code, which is executable by a computer processor to perform any or all of the described steps, operations, or procedures.

[0069] The foregoing description of embodiments of this disclosure has been provided for purposes of illustration and description. The foregoing description is not exhaustive and is not intended to limit this disclosure to the exact form disclosed; various modifications and variations may be made in accordance with the foregoing teachings, or may be derived from practice of this disclosure. These embodiments were chosen and described to illustrate the principles of this disclosure and its practical application, enabling those skilled in the art to utilize this disclosure in various implementations and modifications suitable for the particular purpose conceived.

[0070] This application uses specific terms to describe embodiments of the application. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of the application. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics in one or more embodiments of the application can be appropriately combined.

[0071] In the context of this application, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0072] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this disclosure method does not imply that the subject matter of the present application requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

[0073] The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the embodiments of this application.

Claims

1. An image segmentation method, comprising: obtaining an image containing a plurality of objects; performing object recognition based on the image to obtain a bounding box of each of the objects; determining edge information of a continuous pattern in the image based on the image, wherein each of the continuous patterns comprises one or more objects; performing concave point recognition based on the edge information of the continuous pattern comprising a plurality of objects to obtain a plurality of concave points; determining a segmentation line of the plurality of objects in the continuous pattern comprising a plurality of objects based on the plurality of concave points and the corresponding plurality of bounding boxes of the continuous pattern comprising a plurality of objects; obtaining a contour of each of the objects in the continuous pattern comprising a plurality of objects based on the edge information of the continuous pattern comprising a plurality of objects and the segmentation line; wherein the determining of the segmentation line of the plurality of objects in the continuous pattern comprising a plurality of objects based on the plurality of concave points and the corresponding plurality of bounding boxes of the continuous pattern comprising a plurality of objects comprises: determining an overlapping region between each two intersecting bounding boxes according to each of the bounding boxes comprising a plurality of objects; determining the segmentation line of the plurality of objects based on each of the overlapping regions and a plurality of concave points in the overlapping region; wherein the determining of the segmentation line of the plurality of objects based on each of the overlapping regions and a plurality of concave points in the overlapping region comprises: pairing the plurality of concave points in each of the overlapping regions two by two to form one or more concave point pairs, wherein each of the concave points belongs to only one concave point pair; connecting the two concave points corresponding to each of the concave point pairs to form the segmentation line.

2. The image segmentation method of claim 1, further comprising: confirming the number of bounding boxes corresponding to the edge information of each of the continuous patterns in the image; if the edge information of a continuous pattern corresponds to one bounding box, the continuous pattern comprises one object, and obtaining a contour of the one object based on the edge information of the continuous pattern; if the edge information of a continuous pattern corresponds to a plurality of bounding boxes, the continuous pattern comprises a plurality of objects, and performing the concave point recognition based on the edge information of the continuous pattern comprising a plurality of objects to obtain a plurality of concave points.

3. The image segmentation method of claim 1, wherein, the pairing of the plurality of concave points in each of the overlapping regions two by two to form one or more concave point pairs comprises: determining the number of the concave points in each of the overlapping regions; if the overlapping region contains two of the concave points, the two concave points are determined as a concave point pair; if the overlapping region contains three or more of the concave points, two of the concave points that are not paired are determined as a concave point pair.

4. The image segmentation method of claim 1, wherein, the pairing of the plurality of concave points in each of the overlapping regions two by two to form one or more concave point pairs comprises: determining the number of unpaired concave points in a current overlapping region containing unpaired concave points; if the number of unpaired concave points in the current overlapping region containing unpaired concave points is equal to 2, the two concave points are determined as a concave point pair; if the number of unpaired concave points in the current overlapping region containing unpaired concave points is greater than 2, determining the number of unpaired concave points in a next overlapping region containing unpaired concave points is performed until all concave points are paired.

5. The image segmentation method of claim 1, wherein, The determining the edge information of the continuous pattern in the image based on the image further comprises: performing a binarization process on the image to obtain a binary image; determining the edge information of the continuous pattern in the image by contour extraction based on the binary image.

6. The image segmentation method of claim 1, further comprising: obtaining a smoothed target contour by morphological operation based on the contour of each target.

7. A target sorting method, comprising: the image segmentation method of any one of claims 1-6; classifying each target by target recognition based on the contour of each target.

8. A material sorting system for sorting material, wherein, The material sorting system comprises: an image segmentation module configured to perform the image segmentation method of any one of claims 1-6.

9. The material sorting system of claim 8, wherein, The material sorting system comprises: an image recognition module configured to recognize and classify each material in the segmented image.

10. A computer readable storage medium storing a program for performing the image segmentation method of any one of claims 1-6.

Images