Robot sorting system and method based on vision

Abstract

The invention discloses a robot sorting system and method based on vision, relates to the technical field of robot sorting, and constructs a complete closed-loop system from product image acquisition, defect analysis to classified sorting through deep integration of a machine vision detection system and automatic sorting equipment. By performing abnormal marking and geometric feature analysis on product images and accurately comparing a product defect high-frequency generation area and a problem feedback frequent generation area which are obtained by analyzing historical data and product after-sales problem feedback data, high-probability problem products are quickly screened out, the sorting speed is greatly increased, and the sorting efficiency is improved. Meanwhile, the requirement for real-time detection is met, the yield of produced products is effectively guaranteed, rapid sorting and judgment of the products are achieved by researching the highest bearing capacity of different areas of the products to different defects, and the production efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot sorting, and in particular to a vision-based robot sorting system and a sorting method. Background Art

[0002] In current assembly line production systems, sorting is a core process, requiring different types of materials or workpieces to be categorized and placed in their corresponding locations. Traditional manual sorting is inefficient and lacks accuracy, while vision-based sorting robots can quickly and efficiently complete high-quality sorting operations.

[0003] Patent publication number CN115488876A discloses a robot sorting method and device based on machine vision. The method includes: a workpiece conveying device transports the workpiece to the collection area of ​​a visual inspection platform; an image acquisition device collects image information of the workpiece; an information transmission device transmits the image information to an image processing device for image processing, and outputs the image processing result information; a controller on the sorting robot receives the image processing result information, causes the sorting robot to pick up the workpiece, and places the workpiece into a silo. This invention uses the controller on the sorting robot to receive the image processing result information and simulate the motion trajectory of the sorting robot, effectively integrating machine vision into the sorting robot's sorting work, thereby improving the sorting robot's work efficiency.

[0004] While this robotic sorting system significantly improves sorting efficiency, compared to manual sorting, it can only sort products based on color, shape, and size. It cannot identify defective products during sorting, meaning it cannot sort based on product quality. Therefore, this sorting process often requires a second round of manual sorting to ensure product yield. However, manual inspection is inefficient, and the results are easily affected by subjective factors, making it inadequate for real-time inspection. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this paper proposes a vision-based robotic sorting system and method to address the inefficiency of production methods that require secondary manual inspections. By incorporating deep learning technology, defective products can be identified and sorted, improving overall factory production efficiency and yield rates.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] A vision-based robot sorting method specifically includes the following steps:

[0008] Step 1: collect product image information, and perform enhancement, denoising, grayscale and binarization processing on the collected image;

[0009] Step 2: Perform connected domain analysis on the binary image obtained in step 1, calculate the grayscale mean of each connected domain, mark abnormal connected domains according to the grayscale mean, and combine historical data analysis to obtain the high-frequency defect occurrence area. Perform secondary abnormal marking on the abnormal connected domains in this area.

[0010] Step 3: Identify the defective areas by calculating the geometric features of each abnormal connected domain, use the weighted comprehensive scoring method to judge the severity of each defective area, and output the defect detection results;

[0011] Step 4: Mark areas with frequent problem feedback, formulate product sorting strategies based on secondary anomaly markings, historical defect high-frequency areas, and defect detection results, and sort products according to the sorting strategies.

[0012] As a further solution of the present invention, the grayscale mean of the connected domain in step 2 is specifically calculated as follows:

[0013] For each pixel i in the connected network, obtain the intensity values ​​of its corresponding red, green, and blue channels, denoted as R, G, and B respectively. The grayscale value I can be obtained by I = 0.299R + 0.587G + 0.114B;

[0014] Add the grayscale values ​​I of all pixels in the connected domain i and divide it by the total number of pixels n in the connected domain to get the grayscale mean μi of the connected domain, that is:

[0015] Grayscale mean

[0016] As a further solution of the present invention, the specific method for determining the high-frequency occurrence area of ​​historical defects in step 2 is:

[0017] Segment the grayscale image of the product image to obtain segmented images and label them s, where s = 1, 2, ..., k, where k represents the number of segmented regions. For the same segmented image s of all product images, there is a defect occurrence count d, which is initially 0. Whenever a defective region appears in segmented region s, the defect occurrence count d of segmented image s is increased by one. This operation is repeated for each product image to be sorted.

[0018] The number of defects occurring in the same segmented image s of the product images of N products inspected in the past is recorded as D, where N is a fixed number manually set by comprehensively considering historical data. Then D = d2-d1, d1 and d2 are the d values ​​of the first product and the last product among the N products inspected, respectively. The segmented image s whose number of defect occurrences D exceeds the manually set threshold is recorded as a historical defect high-frequency occurrence area.

[0019] As a further solution of the present invention, the specific determination method for performing secondary abnormal marking on abnormal connected regions in step 2 is:

[0020] The system reads the historical defect-frequently occurring areas fed back during defect inspection of the previous product, overlaps and compares them with the image of the completed abnormal connected domain marking, and marks the abnormal connected domain located in the historical defect-frequently occurring areas as a secondary abnormality.

[0021] As a further solution of the present invention, the specific calculation method of the weighted comprehensive score in step 3 is:

[0022] Combined with historical data analysis, the influence of different geometric features on different defects is weighted. First, the normalized scores of each geometric feature of the target defect are calculated using the normalization method. For example, the normalized score of the area can be used. To determine, then add up the weighted scores to get the weighted comprehensive score.

[0023] As a further solution of the present invention, the specific method for determining the area with frequent problem feedback in step 4 is as follows:

[0024] The method of segmenting the grayscale image of the product image to obtain the segmented image is continued, and the label is denoted as s, s = 1, 2, ..., k, where s represents the number of segmented areas. According to the problem feedback after the product is sold, for the defects reported by customers, the segmented area s in which they are located is counted, and the ratio of the number of defect feedbacks in segmented area s to the total number of product after-sales problem feedback per unit time is recorded as the feedback ratio R. The feedback ratios of segmented areas 1, 2, ..., 9 are recorded as R1, R2, ..., R9 respectively. A feedback ratio warning threshold is set. When the feedback ratio of segmented area s is greater than the feedback ratio warning threshold, segmented area s is marked as an area with frequent problem feedback.

[0025] As a further solution of the present invention, the determination rules based on which the sorting strategy is formulated in step 4 are specifically:

[0026] The problem feedback-prone areas are overlapped and compared with the historical defect-frequently occurring areas where the connected domain with secondary anomaly markers is located. If the two areas overlap, the product is judged as scrap. If there is no overlap, the defect detection results are combined to make a judgment, and the product with defects in the area in the defect detection results is judged as scrap.

[0027] For products with secondary abnormality marks that have not been judged as scrap, a sampling inspection strategy is implemented. Products that have not been inspected are judged as reworkable products, and the inspected products are sorted according to the inspection results.

[0028] For general products, image segmentation is performed. Based on the different tolerances of different segmented areas for different defects and combined with the defect detection results, it is checked whether there are defects in each segmented area that exceed the maximum tolerance level of the area. If so, the product is directly judged as scrap; if there are defects but the defects do not exceed the maximum tolerance level of the segmented area, it is judged as a reworkable product; if the defect detection results are normal, the product is directly judged as qualified.

[0029] As a further solution of the present invention, the sampling detection strategy implemented in the judgment rule based on the sorting strategy is specifically:

[0030] For products with secondary anomaly marks that haven't been deemed scrapped, a sampling inspection strategy is implemented. The sampling inspection frequency is denoted as f, with a non-zero starting value f0. f is determined by the average D value of the historically high-frequency defect region for all products inspected within a unit timeframe. When the qualified rate of products inspected within a unit timeframe exceeds 60%, a comprehensive inspection strategy is implemented. When the qualified rate of products inspected within a unit timeframe falls below 60%, the strategy is switched back to sampling inspection.

[0031] A vision-based robotic sorting system, specifically comprising:

[0032] The image acquisition module uses a megapixel high-resolution industrial camera to capture image information of the target product under uniform illumination from the light source system. The collected high-definition image information is transmitted to the pre-processing module;

[0033] The pre-processing module performs enhancement, denoising, grayscale and binarization on the obtained product images;

[0034] The abnormal marking module performs connected domain analysis on the binary image of the target product, calculates its grayscale mean, and marks connected domains that exceed the set threshold as abnormal connected domains. At the same time, based on the historical defect high-frequency area feedback from the defect detection module, the abnormal connected domains located in this area are secondary marked as abnormal and fed back to the sorting strategy module.

[0035] The defect detection module calculates the geometric features of each abnormal connected domain, compares them with the manually set threshold, identifies the defective areas, and then uses a weighted comprehensive scoring method to determine the severity of each defective area and output the defect detection results;

[0036] The sorting strategy module evaluates products, formulates sorting strategies, and determines product sorting categories and routes based on product after-sales problem feedback, abnormal marking feedback from historically high-frequency defect areas, and defect detection results.

[0037] The sorting execution module reads the defect detection results and sorts the products to designated areas according to the sorting strategy.

[0038] The present invention provides a robot sorting system and a sorting method based on machine vision. Compared with the existing technology, it has the following advantages:

[0039] The present invention deeply integrates the machine vision inspection system with the automated sorting equipment to build a complete closed-loop system from product image acquisition and defect analysis to classification and sorting. By marking abnormalities and analyzing geometric features of product images, it can accurately identify defects on the product surface, such as cracks, holes, scratches, etc., meeting the requirements of real-time detection. By implementing a strict sorting strategy for these products, the yield rate of production products is effectively guaranteed.

[0040] In product image processing, the present invention analyzes historical data and product after-sales problem feedback data to determine the product's high-frequency defect occurrence areas and problem feedback areas. By accurately comparing these areas, high-probability problem products are quickly screened out, greatly improving the sorting speed. By studying the maximum tolerance of different areas of the product to different defects, the present invention realizes rapid sorting and judgment of the products, achieving a significant improvement in production efficiency compared to the efficiency of traditional manual inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a process of a vision-based robot sorting method of the present invention;

[0042] Figure 2 This is a schematic diagram of the framework of a vision-based robot sorting system of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] Example 1

[0045] See also Figure 2 As shown, the present invention provides a vision-based robot sorting system, comprising:

[0046] The image acquisition module uses a megapixel high-resolution industrial camera to capture image information of the target product under uniform illumination from the light source system. The collected high-definition image information is transmitted to the pre-processing module;

[0047] The pre-processing module performs enhancement, denoising, grayscale and binarization on the obtained product images;

[0048] The abnormal marking module performs connected domain analysis on the binary image of the target product, calculates its grayscale mean, and marks connected domains that exceed the set threshold as abnormal connected domains. At the same time, based on the historical defect high-frequency area feedback from the defect detection module, the abnormal connected domains located in this area are secondary marked as abnormal and fed back to the sorting strategy module.

[0049] The defect detection module calculates the geometric features of each abnormal connected domain, compares them with the manually set threshold, identifies the defective areas, and then uses a weighted comprehensive scoring method to determine the severity of each defective area and output the defect detection results;

[0050] The sorting strategy module evaluates products, formulates sorting strategies, and determines product sorting categories and routes based on product after-sales problem feedback, abnormal marking feedback from historically high-frequency defect areas, and defect detection results.

[0051] The sorting execution module reads the defect detection results and sorts the products to designated areas according to the sorting strategy.

[0052] See also Figure 1 As shown, the present invention provides a vision-based robot sorting method, which is as follows:

[0053] Obtain high-definition image information of the target product to be inspected from the image acquisition module, perform filtering, denoising, and image enhancement on the product image to obtain a preprocessed product image, and convert the preprocessed product image into grayscale. The weighted average method can be used to obtain the intensity values ​​of the corresponding red, green, and blue channels for each pixel i in the area, which are denoted as R, G, and B respectively. The grayscale value I can be obtained by I = 0.299R + 0.587G + 0.114B.

[0054] For example, there is a pixel i with R=100, G=120, and B=80. According to the above formula, Gray=0.299x100+0.587x120+0.114x80=29.9+70.44+9.12=109.46. This value is the grayscale value of the pixel after conversion. By performing the conversion operation on all pixels in the image, the grayscale image of the product image can be obtained.

[0055] Set an optimal global threshold derived from historical data, and perform binarization on the grayscale image obtained above. For pixel i, if its grayscale value is greater than the global threshold, it is assigned a value of 255; otherwise, it is assigned to 0, and a binary image is obtained.

[0056] For example, assuming that the optimal global threshold is set to 120, through the above calculation, all pixels with grayscale values ​​greater than 120 are assigned a grayscale value of 255, and all pixels with grayscale values ​​less than 120 are assigned a grayscale value of 0. This will result in a black and white image, which is a binary image of the product image.

[0057] Perform connected domain analysis on the binary image (usually using a two-pass scanning method or a seed filling method), find and mark each connected domain in the image, and obtain the grayscale image of any connected domain as the analysis target. The pixels in the region are sequentially labeled as i, i = 1, 2, 3...n, where n is the total number of pixels covered by the region, that is, the area of ​​the region;

[0058] Add the grayscale values ​​I of all pixels in the connected domain i and divide it by the total number of pixels n in the connected domain to get the grayscale mean μi of the connected domain, that is:

[0059] Grayscale mean

[0060] The grayscale mean μi is then compared with the set mean threshold range, which is obtained by comprehensive analysis of historical qualified product inspection data. When the grayscale mean exceeds the threshold range, the connected domain is marked as an abnormal connected domain; otherwise, it is marked as a normal area. The normal area is the characteristics of the product itself, such as the grooves and connection gaps of the product itself.

[0061] For example, the total number of pixels in the first connected domain is 3600, and the sum of all grayscale values ​​in the region is 482400, so its grayscale mean μi = 482400 / 3600 = 134; the total number of pixels in the second connected domain is 3000, and the sum of all grayscale values ​​in the region is 345000, so its grayscale mean μi = 345000 / 3000 = 115. Assuming that the set threshold range is [110, 120], then:

[0062] Connected domain No. 1 μi=134>120, which is not within the threshold range, so it should be marked as an abnormal connected domain;

[0063] The second connected region 110<μi=115<120 is within the threshold range, so it should be marked as a normal area.

[0064] Perform the above operations on all connected domains marked in the image and mark them one by one.

[0065] The defect detection module reads the historical defect high-frequency area fed back by the previous product sorting, overlaps and compares it with the image of the above-mentioned abnormal connected domain marking, and marks the abnormal connected domain located in the historical defect high-frequency area as a secondary abnormality. At the same time, the information is fed back to the sorting strategy module.

[0066] For example, in the above example, connected domain No. 1 has been marked as an abnormal connected domain. If it happens to overlap with the historical defect high-frequency area fed back by the defect detection module in the previous product sorting, it is necessary to mark connected domain No. 1 as abnormal a second time and feed the secondary abnormal marking information back to the sorting strategy module.

[0067] All the abnormal connected regions marked above are recorded as the detection regions and labeled with c = 1, 2, ..., r, where r represents the number of detection regions. The geometric features of each detection region are calculated, such as area, aspect ratio, circularity, edge gradient mean, grayscale variance, etc.

[0068] By comparing these geometric features with the manually set normal threshold range, the areas to be inspected that belong to defect areas are identified, such as:

[0069] If the area and aspect ratio are greater than the manually set threshold and the grayscale variance is less than the manually set threshold, it can be judged as a crack defect; if the circularity is less than the manually set threshold and the grayscale variance is greater than the manually set threshold, it can be judged as a hole defect; if the regional aspect ratio, area and edge gradient mean are all greater than the manually set threshold, it can be judged as a scratch; the area to be detected whose geometric features are all within the threshold range can be judged as a normal area.

[0070] For example, the area to be detected with a quantity label of 15 has an aspect ratio of 4, which is greater than the manually set threshold of 3; an area of ​​800, which is greater than the manually set threshold of 700; an edge gradient mean of 70, which is greater than the manually set threshold of 60; and other geometric features are all within the manually set threshold range. Therefore, the area to be detected No. 15 meets the scratch determination conditions and is determined to be a scratch.

[0071] For the area to be inspected that is identified as a defect, a weighted comprehensive scoring method is used based on the defect type and geometric characteristics. Combined with historical data analysis, the influence of different geometric characteristics on different defects is weighted. First, the normalized score of each geometric feature of the defect is calculated using the normalization method (for example, the normalized score of the area can be used). The weighted scores are then added together to form a comprehensive score, and the severity is divided into mild, moderate, and severe according to the comprehensive score.

[0072] For example, consider a crack defect. Based on historical data analysis, the five geometric features (area, aspect ratio, circularity, edge gradient mean, and grayscale variance) have the greatest impact on its severity. Therefore, the weights assigned to each geometric feature in the comprehensive score are: area accounts for 30%, aspect ratio accounts for 25%, circularity accounts for 5%, edge gradient mean accounts for 15%, and grayscale variance accounts for 25%. Assuming the normalized scores of area, aspect ratio, circularity, edge gradient mean, and grayscale variance are 0.9, 1.0, 0.1, 0.3, and 0.8, respectively, the comprehensive score is 0.9 × 30% + 1.0 × 25% + 0.1 × 5% + 0.3 × 15% + 0.8 × 25% = 0.77. Assume the severity score is categorized as follows: a comprehensive score < 0.4 is considered mild; 0.4 ≤ a comprehensive score < 0.7 is considered moderate; and a comprehensive score > 0.7 is considered severe. At this time, 0.77>0.7, so the severity level of this crack is severe.

[0073] After each inspection of a to-be-inspected area is completed, no matter the result is a defect area or a normal area, the mark of the to-be-inspected area is changed to the mark of the inspected area, and the inspection of all abnormal connected domains is completed in sequence.

[0074] After all the areas to be detected are identified, the grayscale image of the product is divided into nine equal parts to obtain segmented images and labeled as s, where s = 1, 2, ..., 9, where s represents the number of segmented areas. At the same time, any group of segmented images s is obtained as the analysis target:

[0075] For the same segmented image s of all product images, there is a defect occurrence count d (the initial value of d is 0). When the defect area detected in the above step exists in the current segmented area s, the defect occurrence count d of the segmented image s is increased by one. This operation is performed for each product image to be sorted.

[0076] The number of defects occurring in the same segmented image s of the product images of N products inspected in the past is recorded as D, where N is a fixed number manually set by comprehensively considering historical data. Then D = d2 - d1, d1 and d2 are the d values ​​of the first and last products in the N products inspected, respectively. The segmented image s whose defect occurrence number D exceeds the manually set threshold is recorded as a historical defect high-frequency occurrence area, and this result is fed back to the abnormality marking module and the sorting strategy module.

[0077] For example, assuming that the manually set N value is 300, the above-mentioned geometric feature defect detection method has determined that the detected area No. 15 is a scratch. Assuming that all the inspected products are numbered in sequence, the detected area No. 15 belongs to product No. 500. After all the areas to be inspected are identified, the product image grayscale image is divided into nine equal parts to obtain segmented images and labeled. Assuming that the number of defects found in the historical inspected products in the segmented area No. 5 is d = 100, and in this sorting defect detection, it is found that the detected area No. 15 is also in the segmented area No. 5, then d = 100 + 1 = 101; when inspecting product No. 200, d = 51 in the segmented area No. 5, then the number of defects in the 300 products inspected in the past is D = 101-51 = 50. Assuming that the manually set segmented area D value is less than 30, it belongs to the normal area, then the segmented area No. 5 will be marked as a historical defect high-frequency area, and this information will be fed back to the abnormal marking module and the sorting strategy module.

[0078] At the same time, when all the areas to be detected in the product image are identified, the defect type, defect severity and segmented area s where the defect is located are comprehensively organized into defect detection results and output to the sorting strategy module.

[0079] In the sorting strategy module, the image segmentation method used in the defect detection module is used. Based on the after-sales problem feedback of the product, the segmented region s where the defect reported by the customer is located is counted. The ratio of the number of defect feedbacks in segmented region s to the total number of after-sales problem feedbacks per unit time is recorded as the feedback ratio R. The feedback ratios of segmented regions 1, 2, ..., 9 are recorded as R1, R2, ..., R9, respectively. A feedback ratio warning threshold is set. When the feedback ratio of segmented region s is greater than the feedback ratio warning threshold, segmented region s is marked as a region with frequent problem feedbacks.

[0080] For example, suppose that according to the collected product after-sales problem feedback information, the total number of after-sales problem feedback about product defects in one month is 100, and in 55 of the problem feedbacks, the defect locations all appear in segmented area 5. Therefore, the feedback ratio N5 of segmented area 5 is 55÷100=55%. Assuming that the feedback ratio warning threshold is set to 50%, 55%>50%, so segmented area 5 will be recorded as an area with frequent problem feedback.

[0081] In the sorting strategy module, the problem feedback-prone area is compared with the historical defect-frequently occurring area where the connected domain with the secondary anomaly mark is located. If the two areas overlap, the product is judged as scrap. If there is no overlap, the defect detection results are combined to make a judgment, and the product with defects in the area in the defect detection results is judged as scrap.

[0082] For products with secondary anomaly flags reported by the anomaly marking module but not yet deemed scrap, a sampling inspection strategy is implemented. Because they already have secondary anomaly flags, the probability of defects is very high. To improve inspection efficiency, products not sampled for inspection are directly classified as reworkable. Products sampled for inspection are sent to the defect detection module for inspection, their secondary anomaly flags are removed, and they are sorted according to the general product sorting strategy. The sampling inspection frequency is denoted as f (the starting value f0 is not 0), which is determined by the average D value of the historical defect high-frequency area of ​​all products inspected per unit time.

[0083] For example, suppose the system is set to that when the average D value of the historical defect high-frequency area where the defects of the products inspected within ten minutes are located is less than or equal to 40, the sampling frequency f = f0, and the value of f0 is 20 samplings per 100 products. When the average D value of the historical defect high-frequency area where the defects of the products inspected within ten minutes are located is greater than 40, the value of f is 25 samplings per 100 products. Assuming that the average D value of the historical defect high-frequency area where the defects of the products inspected within ten minutes is calculated to be 60 at this time, 60>40, so the sampling frequency f is updated to 25 samplings per 100 products.

[0084] When the qualified rate of products sampled and inspected per unit time is greater than 60%, it means that the probability of defects in such products in the recent period has decreased. At this time, the comprehensive inspection strategy is switched to and products without defects are judged as qualified products. When the qualified rate of products sampled and inspected per unit time is less than 60%, the sampling inspection strategy is switched back to.

[0085] For general products that are not marked as abnormal twice, they are segmented using the image segmentation method of the defect detection module. For the segmented area s, refer to historical data to obtain its maximum tolerance level for different defect severities. Based on the defect detection results, search whether there are defects exceeding its maximum tolerance level in the segmented area s. If so, no other segmented areas will be searched and the product will be directly judged as scrap. If there are defects but none of the defects exceed the maximum tolerance level of the segmented area, the product will be judged as a reworkable product. If the defect detection results are normal, the product will be directly judged as a qualified product without further segmented area search.

[0086] For example, there is a product. According to historical data, the tolerance levels of segmented areas 2 to 9 in its segmented image for cracks, holes, scratches, etc. are all at the highest level - severe. However, segmented area 1 has a maximum tolerance level of slight for cracks and holes, and a maximum tolerance level of medium for scratches. Combined with the defect detection results fed back by the defect detection module, it is known that there is a crack defect in segmented area 1, with an overall score of 0.55 and a severity of medium. At this time, the defect has exceeded the maximum tolerance level of segmented area 1 for crack defects, so it is directly judged as scrap in the sorting strategy, and no search and judgment is performed on other segmented areas.

[0087] After the judgment is completed, the sorting strategy is generated and output to the sorting execution module.

[0088] The sorting execution module reads the defect detection results and sorts the products to designated areas according to the sorting strategy.

[0089] The above contents are merely examples and explanations of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in similar ways. As long as they do not deviate from the invention or exceed the scope defined by the claims, they should all fall within the scope of protection of the present invention.

Claims

1. A vision-based robot sorting method, characterized in that: The specific steps include: Step 1: collect product image information, and perform enhancement, denoising, grayscale and binarization processing on the collected image; Step 2: Perform connected domain analysis on the binary image obtained in step 1, calculate the grayscale mean of each connected domain, mark abnormal connected domains according to the grayscale mean, read the historical defect high-frequency occurrence area in the historical inspection data, and perform secondary abnormal marking on the abnormal connected domains located in this area; Step 3: Identify the defective areas by calculating the geometric features of each abnormal connected domain, use the weighted comprehensive scoring method to judge the severity of each defective area, and output the defect detection results; Step 4: Mark areas with frequent problem feedback, formulate product sorting strategies based on secondary anomaly markings, historical defect high-frequency areas, and defect detection results, and sort products according to the sorting strategies.

2. A vision-based robot sorting method according to claim 1, characterized in that: The specific calculation method of the grayscale mean of the connected domain in step 2 is: For each pixel i in the connected network, obtain the intensity values ​​of its corresponding red, green, and blue channels, denoted as R, G, and B respectively. The grayscale value I can be obtained by I = 0.299R + 0.587G + 0.114B; Add the grayscale values ​​I of all pixels in the connected domain i and divide it by the total number of pixels n in the connected domain to get the grayscale mean μi of the connected domain, that is: Grayscale mean 3. A vision-based robot sorting method according to claim 1, characterized in that: The specific method for determining the high-frequency area of ​​historical defects in step 2 is as follows: Segment the grayscale image of the product image to obtain segmented images and label them s, where s = 1, 2, ..., k, where k represents the number of segmented regions. For the same segmented image s of all product images, there is a defect occurrence count d, which is initially 0. Whenever a defective region appears in segmented region s, the defect occurrence count d of segmented image s is increased by one. This operation is repeated for each product image to be sorted. The number of defects occurring in the same segmented image s of the product images of N products inspected in the past is recorded as D, where N is a fixed number manually set by comprehensively considering historical data. Then D = d2-d1, d1 and d2 are the d values ​​of the first product and the last product among the N products inspected, respectively. The segmented image s whose number of defect occurrences D exceeds the manually set threshold is recorded as a historical defect high-frequency occurrence area.

4. A vision-based robot sorting method according to claim 1, characterized in that: The specific determination method for secondary abnormal marking of abnormal connected areas in step 2 is: The system reads the historical defect-frequently occurring areas fed back during defect inspection of the previous product, overlaps and compares them with the image of the completed abnormal connected domain marking, and marks the abnormal connected domain located in the historical defect-frequently occurring areas as a secondary abnormality.

5. The vision-based robot sorting method according to claim 1, characterized in that: The specific calculation method of the weighted comprehensive score in step 3 is: Combined with historical data analysis, the influence of different geometric features on different defects is weighted. First, the normalized scores of each geometric feature of the target defect area are calculated using the normalization method. For example, the normalized score of the area can be used. To determine, then add up the weighted scores to serve as the weighted comprehensive score for the defect area.

6. A vision-based robot sorting method according to claim 1, characterized in that: The specific method for determining areas with frequent problem feedback in step 4 is as follows: The method of segmenting the grayscale image of the product image to obtain the segmented image is continued, and the label is denoted as s, s = 1, 2, ..., k, where s represents the number of segmented areas. According to the problem feedback after the product is sold, for the defects reported by customers, the segmented area s in which they are located is counted, and the ratio of the number of defect feedbacks in segmented area s to the total number of product after-sales problem feedback per unit time is recorded as the feedback ratio R. The feedback ratios of segmented areas 1, 2, ..., 9 are recorded as R1, R2, ..., R9 respectively. A feedback ratio warning threshold is set. When the feedback ratio of segmented area s is greater than the feedback ratio warning threshold, segmented area s is marked as an area with frequent problem feedback.

7. The vision-based robot sorting method according to claim 1, characterized in that: The specific judgment rules for formulating the sorting strategy in step 4 are: The problem feedback-prone areas are overlapped and compared with the historical defect-frequently occurring areas where the connected domain with secondary anomaly markers is located. If the two areas overlap, the product is judged as scrap. If there is no overlap, the defect detection results are combined to make a judgment, and the product with defects in the area in the defect detection results is judged as scrap. For products with secondary abnormality marks that have not been judged as scrap, a sampling inspection strategy is implemented. Products that have not been inspected are judged as reworkable products, and the inspected products are sorted according to the inspection results. For general products, image segmentation is performed. Based on the different tolerances of different segmented areas for different defects and combined with the defect detection results, it is checked whether there are defects in each segmented area that exceed the maximum tolerance level of the area. If so, the product is directly judged as scrap; if there are defects but the defects do not exceed the maximum tolerance level of the segmented area, it is judged as a reworkable product; if the defect detection results are normal, the product is directly judged as qualified.

8. A vision-based robot sorting method according to claim 7, characterized in that: The specific sampling and testing strategies implemented in the judgment rules based on the sorting strategy are as follows: For products with secondary abnormality marks that have not been judged as scrap, a sampling inspection strategy is implemented. The sampling inspection frequency is denoted as f. The starting value f0 is not 0. f is determined by the average D value of the historical high-frequency defect area of ​​all products inspected within a unit time. When the qualified rate of products inspected within a unit time is greater than 60%, the comprehensive inspection strategy is switched to. When the qualified rate of products inspected in full per unit time is less than 60%, the sampling inspection strategy is switched back.

9. A vision-based robotic sorting system, characterized in that: The system is used to execute the vision-based robot sorting method described in any one of claims 1 to 8.

Images