A vision-based robotic sorting system and method

Abstract

This invention discloses a vision-based robotic sorting system and method, relating to the field of robotic sorting technology. Through deep integration of a machine vision inspection system and automated sorting equipment, a complete closed-loop system is constructed, encompassing product image acquisition, defect analysis, and classification sorting. By anomaly marking and geometric feature analysis of product images, and by accurately comparing high-frequency defect areas and frequently reported problem areas derived from historical data and after-sales feedback data, high-probability defective products are quickly identified, significantly improving sorting speed. Simultaneously, it meets the requirements of real-time detection, effectively ensuring the yield rate of manufactured products. Furthermore, by studying the maximum tolerance of different areas of the product to various defects, rapid product sorting and judgment are achieved, significantly improving production efficiency.

Description

Technical Field

[0001] This invention relates to the field of robot sorting technology, specifically to a vision-based robot sorting system and sorting method. Background Technology

[0002] In current assembly line production systems, sorting is a core process, requiring the categorization and placement of different types of materials or workpieces into their appropriate locations. Traditional manual sorting is inefficient and lacks accuracy, while vision-based sorting robots can quickly complete high-quality sorting operations in a short time.

[0003] Patent publication number CN115488876A discloses a robot sorting method and apparatus based on machine vision. The method includes: a workpiece conveying device transporting workpieces to the acquisition area of ​​a vision inspection platform; an image acquisition device acquiring image information of the workpieces; an information transmission device transmitting the image information to an image processing device for image processing and outputting the image processing result information; and a controller on the sorting robot receiving the image processing result information, causing the sorting robot to pick up the workpiece and place it into a hopper. This invention, by receiving the image processing result information on the controller of the sorting robot and simulating the motion trajectory of the sorting robot, effectively integrates machine vision into the sorting robot's sorting work, thereby improving the working efficiency of the sorting robot.

[0004] While this robotic sorting system significantly improves sorting efficiency, compared to manual sorting, it can only sort products based on color, shape, and size, and cannot identify defective products during sorting; that is, it cannot sort products based on quality. Therefore, a second manual sorting is often required after this initial sorting to ensure product yield. However, manual inspection is inefficient, and the results are easily affected by subjective human factors, failing to meet the requirements of real-time inspection. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a vision-based robotic sorting system and method, solving the problem of low efficiency in production methods that require secondary manual inspection. By introducing deep learning technology, defective products can be identified and sorted, thereby improving the overall production efficiency and yield rate of the factory.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A vision-based robotic sorting method specifically includes the following steps:

[0008] Step 1: Collect product image information and perform enhancement, noise reduction, grayscale conversion, and binarization processing on the collected images;

[0009] Step 2: Perform connected component analysis on the binarized image obtained in Step 1, calculate the gray-scale mean of each connected component, mark abnormal connected components based on the gray-scale mean, and combine historical data analysis to obtain the high-frequency occurrence area of ​​historical defects. Perform secondary anomaly marking on the abnormal connected components located in this area.

[0010] Step 3: Identify the defective regions by calculating the geometric features of each abnormal connected region, use a weighted comprehensive scoring method to determine the severity of each defective region, and output the defect detection results.

[0011] Step 4: Mark areas with frequent problem feedback. Based on secondary anomaly marking, historical high-frequency defect areas, and defect detection results, formulate a product sorting strategy and sort the products according to the sorting strategy.

[0012] As a further aspect of the present invention, the specific calculation method for the average gray value of the connected components in step two is as follows:

[0013] For each pixel i within the connected region, obtain the intensity values ​​of its corresponding red, green, and blue channels, denoted as R, G, and B respectively. The grayscale value I can be obtained from I = 0.299R + 0.587G + 0.114B.

[0014] Adding the gray values ​​I of all pixels within connected region i and then dividing by the total number of pixels n in that connected region yields the average gray value μi of that connected region, i.e.:

[0015] Gray mean

[0016] As a further aspect of the present invention, the specific method for determining the high-frequency occurrence area of ​​historical defects in step two is as follows:

[0017] The grayscale image of the product is segmented to obtain segmented images and labeled as s, s = 1, 2, ..., k, where k represents the number of segmented regions. For the same segmented image s of all product images, there is a number of defects d, the initial value of d is 0. Whenever a defective region appears on the segmented region s, the number of defects d in the segmented image s is incremented by one. This operation is performed for each product image to be sorted.

[0018] The number of defects occurring in the same segmented image s of the product images of N products that have been inspected in the past is denoted as D, where N is a fixed number set manually by combining historical data. Then D = d2 - d1, where d1 and d2 are the d values ​​of the first and last products among the N products that have been inspected, respectively. The segmented image s where the number of defects D exceeds the manually set threshold is denoted as the high-frequency defect occurrence area in history.

[0019] As a further aspect of the present invention, the specific method for determining the secondary anomaly marking of the abnormal connected region in step two is as follows:

[0020] Read the historical high-frequency defect areas returned during the defect detection of the previous product, compare them with the image that has been marked with abnormal connected components, and mark the abnormal connected components located in the historical high-frequency defect areas with secondary anomalies.

[0021] As a further aspect of the present invention, the specific calculation method for the weighted comprehensive score in step three is as follows:

[0022] Based on historical data analysis, weights are applied according to the influence of different geometric features on different defects. First, the normalized score of each geometric feature of the target defect is calculated using a normalization method. For example, the normalized score for the area can be calculated using... To determine the total score, the weighted scores are summed together to obtain the weighted composite score.

[0023] As a further aspect of the present invention, the specific method for determining the frequently occurring problem feedback areas in step four is as follows:

[0024] The method of segmenting the grayscale image of the product to obtain segmented images is adopted. The segments are labeled as s, where s = 1, 2, ..., k, and s represents the number of segmented regions. Based on the feedback of after-sales issues, for the defects reported by customers, the segmented regions s in which they are located are counted. The proportion of defect feedback in segmented region s to the total number of after-sales issues within a unit of time is denoted as the feedback ratio R. The feedback ratios of segmented regions 1, 2, ..., 9 are denoted as R1, R2, ..., R9, respectively. A feedback ratio warning threshold is set. When the feedback ratio of segmented region s is greater than the feedback ratio warning threshold, segmented region s is marked as a region with frequent problem feedback.

[0025] As a further aspect of the present invention, the judgment rule based on formulating the sorting strategy in step four is specifically as follows:

[0026] The system compares the areas with frequent problem reports with the historical high-frequency defect areas where connected regions with secondary anomaly markers are located. If the two areas overlap, the product is determined to be defective. If they do not overlap, the system combines the defect detection results to determine the defective products that have defects in the area.

[0027] For products with secondary anomaly markers that are not judged as defective, a sampling inspection strategy is implemented. Products that are not sampled are judged as reworkable, and products that are sampled are sorted according to the inspection results.

[0028] For general products, image segmentation is performed. Based on the different tolerance levels of different segmented regions for different defects, and combined with the defect detection results, it is searched to see if there are any defects in each segmented region that exceed the highest tolerance level of that region. If so, the product is directly judged as scrap. If there are defects but none of them exceed the highest tolerance level of their respective segmented regions, the product is judged as reworkable. If the defect detection results are normal, the product is directly judged as qualified.

[0029] As a further aspect of the present invention, the sampling detection strategy implemented in the judgment rules upon which the sorting strategy is based is specifically as follows:

[0030] For products with secondary anomaly markers but not classified as defective, a sampling inspection strategy is implemented. The sampling frequency is denoted as f, with an initial value f0 that is not 0. f is determined by the average of the historical high-frequency defect occurrence areas D values ​​of all products inspected within a unit of time. When the pass rate of the sampled products within a unit of time is greater than 60%, the strategy is switched to full inspection; when the pass rate of the full inspection products within a unit of time is less than 60%, the strategy is switched back to sampling inspection.

[0031] A vision-based robotic sorting system, specifically comprising:

[0032] The image acquisition module uses a megapixel high-resolution industrial camera to acquire image information of the target product under uniform illumination from the light source system. The acquired high-definition image information is then transmitted to the preprocessing module.

[0033] The preprocessing module performs enhancement, noise reduction, grayscale conversion, and binarization on the obtained product images;

[0034] The anomaly marking module performs connected component analysis on the binarized image of the target product, calculates its grayscale mean, and marks connected components that exceed the set threshold range as abnormal connected components. At the same time, based on the historical high-frequency defect occurrence areas fed back by the defect detection module, the abnormal connected components located in the area are marked as abnormal again and fed back to the sorting strategy module.

[0035] The defect detection module calculates the geometric features of each abnormal connected region, compares them with manually set thresholds, identifies the defective regions, and then uses a weighted comprehensive scoring method to determine the severity of each defective region and outputs the defect detection results.

[0036] The sorting strategy module evaluates products based on feedback from after-sales issues, feedback from abnormal markings in high-frequency defect areas in history, and defect detection results, formulates sorting strategies, and determines the sorting categories and paths of products.

[0037] The sorting execution module reads the defect detection results and sorts the products to designated areas according to the sorting strategy.

[0038] This invention provides a machine vision-based robotic sorting system and method. Compared with existing technologies, it has the following advantages:

[0039] This invention establishes a complete closed-loop system from product image acquisition and defect analysis to classification and sorting by deeply integrating machine vision inspection system with automated sorting equipment. Through anomaly marking and geometric feature analysis of product images, it achieves accurate identification of defects such as cracks, holes, and scratches on product surfaces, meeting the requirements of real-time detection. Furthermore, by implementing strict sorting strategies for these products, it effectively ensures the yield rate of manufactured products.

[0040] In product image processing, this invention analyzes historical data and after-sales feedback data to identify high-frequency defect areas and areas with frequent problem feedback. By accurately comparing these areas, high-probability defective products can be quickly screened out, significantly improving sorting speed. Furthermore, by studying the maximum tolerance of different product areas to different defects, rapid product sorting and judgment can be achieved, resulting in a significant improvement in production efficiency compared to traditional manual inspection. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of a vision-based robot sorting method according to the present invention.

[0042] Figure 2 This is a schematic diagram of the framework of a vision-based robotic sorting system according to the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Example 1

[0045] Please see Figure 2 As shown, the present invention provides a vision-based robotic sorting system, comprising:

[0046] The image acquisition module uses a megapixel high-resolution industrial camera to acquire image information of the target product under uniform illumination from the light source system. The acquired high-definition image information is then transmitted to the preprocessing module.

[0047] The preprocessing module performs enhancement, noise reduction, grayscale conversion, and binarization on the obtained product images;

[0048] The anomaly marking module performs connected component analysis on the binarized image of the target product, calculates its grayscale mean, and marks connected components that exceed the set threshold range as abnormal connected components. At the same time, based on the historical high-frequency defect occurrence areas fed back by the defect detection module, the abnormal connected components located in the area are marked as abnormal again and fed back to the sorting strategy module.

[0049] The defect detection module calculates the geometric features of each abnormal connected region, compares them with manually set thresholds, identifies the defective regions, and then uses a weighted comprehensive scoring method to determine the severity of each defective region and outputs the defect detection results.

[0050] The sorting strategy module evaluates products based on feedback from after-sales issues, feedback from abnormal markings in high-frequency defect areas in history, and defect detection results, formulates sorting strategies, and determines the sorting categories and paths of products.

[0051] The sorting execution module reads the defect detection results and sorts the products to designated areas according to the sorting strategy.

[0052] Please see Figure 1 As shown, this invention provides a vision-based robot sorting method, as detailed below:

[0053] The high-definition image information of the target product to be detected is obtained from the image acquisition module. The product image is filtered, denoised and enhanced to obtain a preprocessed product image. The preprocessed product image is then converted to grayscale. A weighted average method can be used. For each pixel i in the region, the intensity values ​​of its corresponding red, green and blue channels are obtained and denoted as R, G and B respectively. The grayscale value I can be obtained from I = 0.299R + 0.587G + 0.114B.

[0054] For example, given a pixel i with R=100, G=120, and B=80, according to the formula above, Gray=0.299x100+0.587x120+0.114x80=29.9+70.44+9.12=109.46. This value is the grayscale value of that pixel after conversion. By performing the conversion operation on all pixels in the image, the grayscale image of the product can be obtained.

[0055] Set an optimal global threshold derived from historical data, and perform binarization on the grayscale image obtained above. For pixel i, if its grayscale value is greater than the global threshold, assign it a value of 255; otherwise, assign it a value of 0 to obtain a binary image.

[0056] For example, assuming the optimal global threshold is set to 120, then through the above calculation, all pixels with a gray value greater than 120 will have their gray value assigned to 255, and all pixels with a gray value less than 120 will have their gray value assigned to 0. This will result in a black and white image, which is the binary image of the product image.

[0057] Connectivity analysis is performed on the obtained binary image (usually using a two-pass scan method or a seed fill method). Each connected component in the image is identified and marked. At the same time, the grayscale image of any connected component is obtained as the analysis target. The pixels in the region are sequentially labeled as i, i = 1, 2, 3...n, where n is the total number of pixels covered by the region, which is also the area of ​​the region.

[0058] Adding the gray values ​​I of all pixels within connected region i and then dividing by the total number of pixels n in that connected region yields the average gray value μi of that connected region, i.e.:

[0059] Gray mean

[0060] Next, the grayscale mean μi is compared with the set mean threshold range, which is obtained by comprehensive analysis of historical qualified product inspection data. When the grayscale mean exceeds the threshold range, the connected region is marked as an abnormal connected region; otherwise, it is marked as a normal region. The normal region is the feature of the product itself, such as the grooves and seams of the product itself.

[0061] For example, if the total number of pixels in connected component 1 is 3600, and the sum of all gray values ​​within its region is 482400, then its average gray value μi = 482400 / 3600 = 134; if the total number of pixels in connected component 2 is 3000, and the sum of all gray values ​​within its region is 345000, then its average gray value μi = 345000 / 3000 = 115. Assuming the set threshold range is [110, 120], then:

[0062] The first connected component μi = 134 > 120, which is outside the threshold range, so it should be marked as an abnormal connected component.

[0063] The second connected region, 110 < μi = 115 < 120, is within the threshold range, so it should be marked as a normal region.

[0064] Perform the above operation on all connected components marked in the image, marking them one by one.

[0065] The system reads the historical high-frequency defect area fed back by the defect detection module in the previous product sorting, compares it with the image that has been marked with abnormal connected components, marks the abnormal connected components located in the historical high-frequency defect area with secondary abnormality, and feeds the information back to the sorting strategy module.

[0066] For example, in the above example, the first connected component has been marked as an abnormal connected component. If it happens to overlap with the historical high-frequency defect area fed back by the defect detection module in the previous product sorting, the first connected component needs to be marked as abnormal again, and the information of the second abnormal marking should be fed back to the sorting strategy module.

[0067] All the abnormal connected components marked above are denoted as regions to be detected and labeled as c = 1, 2, ..., r, where r represents the number of regions to be detected. Calculate the geometric features of each region to be detected, such as: area, aspect ratio, roundness, mean edge gradient, and gray-level variance.

[0068] By comparing these geometric features with manually set normal threshold ranges, the areas to be detected that belong to the defect region are identified, such as:

[0069] Areas with an area and aspect ratio greater than the manually set threshold, and grayscale variance less than the manually set threshold, can be identified as crack defects; areas with a roundness less than the manually set threshold, and grayscale variance greater than the manually set threshold, can be identified as hole defects; areas with an aspect ratio, area, and mean edge gradient all greater than the manually set threshold can be identified as scratches; and areas whose geometric features are all within the threshold range can be identified as normal areas.

[0070] For example, the area to be detected with the quantity number 15 has an aspect ratio of 4, which is greater than the manually set threshold of 3; an area of ​​800, which is greater than the manually set threshold of 700; an average edge gradient of 70, which is greater than the manually set threshold of 60; and other geometric features are all within the range of the manually set thresholds. Therefore, the area to be detected with the quantity number 15 meets the scratch determination criteria and is determined to be a scratch.

[0071] For areas identified as defects, a weighted comprehensive scoring method is used based on the defect type and geometric features, combined with historical data analysis. Weights are applied according to the influence of different geometric features on different defects. First, the normalized score of each geometric feature of the defect is calculated using a normalization method (e.g., the normalized score for area can be calculated using...). (To determine), then the weighted scores are summed to obtain a comprehensive score, and the severity is classified as mild, moderate, or severe based on the comprehensive score.

[0072] For example, consider a crack defect. Based on historical data analysis, among its five geometric features (area, aspect ratio, roundness, mean edge gradient, and grayscale variance), area, aspect ratio, and grayscale variance have a significant impact on its severity. Therefore, the weighting of each geometric feature in the overall score is as follows: area 30%, aspect ratio 25%, roundness 5%, mean edge gradient 15%, and grayscale variance 25%. Assuming the normalized scores for these five features are 0.9, 1.0, 0.1, 0.3, and 0.8 respectively, the overall score is 0.9 × 30% + 1.0 × 25% + 0.1 × 5% + 0.3 × 15% + 0.8 × 25% = 0.77. The severity rating is categorized as follows: overall score < 0.4 (mild); 0.4 ≤ overall score < 0.7 (moderate); overall score > 0.7 (severe). Since 0.77 > 0.7, the severity level of this crack is severe.

[0073] After each region to be inspected is inspected, regardless of whether the result is a defective region or a normal region, the region to be inspected is changed to a region that has been inspected. This process is repeated for all abnormal connected regions.

[0074] After all regions to be detected have been identified, the grayscale image of the product is divided into nine equal parts to obtain segmented images, which are labeled as s, where s = 1, 2, ..., 9, and s represents the number of segmented regions. Simultaneously, any set of segmented images s is used as the analysis target.

[0075] For the same segmented image s of all product images, there is a number of defects d (the initial value of d is 0). When the defective region detected in the above steps exists in the current segmented region s, the number of defects d of the segmented image s is incremented by one. This operation is performed for each product image to be sorted.

[0076] The number of defects occurring in the same segmented image s of the product images of N products that have been inspected in the past is denoted as D, where N is a fixed number set manually by combining historical data. Then D = d2 - d1, where d1 and d2 are the d values ​​of the first and last products among the N products that have been inspected, respectively. The segmented image s where the number of defects D exceeds the manually set threshold is marked as a high-frequency defect area in history, and this result is fed back to the anomaly marking module and the sorting strategy module.

[0077] For example, assuming the manually set N value is 300, and the geometric feature defect detection method described above has determined that the detected area 15 is a scratch, and assuming all detected products are numbered sequentially, the detected area 15 belongs to product number 500, after all areas to be detected are identified, the grayscale image of the product is divided into nine equal parts to obtain segmented images and labeled. Assuming analysis of segmented area 5 shows that the number of defects found in segmented area 5 in the past is d = 100, and the detected area 15 is also found in segmented area 5 during this sorting defect detection, then d = 100 + 1 = 101; while when detecting product number 200, segmented area 5 has d = 51, then the number of defects in the past 300 products is D = 101 - 51 = 50. Assuming the manually set segmented area D value is less than 30 and is considered a normal area, segmented area 5 will be marked as a high-frequency defect area in history, and this information will be fed back to the anomaly marking module and the sorting strategy module.

[0078] Meanwhile, once all the areas to be detected in the product image have been identified, the product's defect type, defect severity, and the segmented region s where the defect is located are comprehensively compiled into a defect detection result and output to the sorting strategy module.

[0079] In the sorting strategy module, the image segmentation method of the defect detection module is adopted. Based on the feedback of product after-sales issues, for the defects reported by customers, the segmentation region s in which they are located is counted. The proportion of the number of defect feedbacks in segmentation region s per unit time to the total number of product after-sales issues is recorded as the feedback ratio R. The feedback ratios of segmentation regions 1, 2, ..., 9 are recorded as R1, R2, ..., R9, respectively. A feedback ratio warning threshold is set. When the feedback ratio of segmentation region s is greater than the feedback ratio warning threshold, segmentation region s is marked as a region with frequent problem feedback.

[0080] For example, suppose that according to the collected product after-sales feedback information, there are a total of 100 feedback issues regarding product defects within a month. In 55 of these feedback issues, the defect location is in segmented region 5. Therefore, the feedback ratio N5 of segmented region 5 is 55 ÷ 100 = 55%. Assuming that the set feedback ratio warning threshold is 50%, 55% > 50%, so segmented region 5 will be recorded as a region with frequent problem feedback.

[0081] In the sorting strategy module, the areas with frequent problem feedback are compared with the historical high-frequency defect areas where the connected regions with secondary anomaly markers are located. If the two areas overlap, the product is judged as scrap. If they do not overlap, the defect detection results are combined to make a judgment, and the products with defects in the area in the defect detection results are judged as scrap.

[0082] For products with secondary anomaly markers reported by the anomaly marking module but not yet judged as scrap, a sampling inspection strategy is implemented. Since these products already have secondary anomaly markers, the probability of them having defects is very high. To improve inspection efficiency, products not sampled for inspection are directly judged as reworkable. Products sampled for inspection are sent to the defect inspection module for testing, and their secondary anomaly markers are removed. They are then sorted according to the general product sorting strategy. The sampling frequency is denoted as f (the initial value f0 is not 0), and f is determined by the average of the historical high-frequency defect occurrence areas D of all products inspected per unit time.

[0083] For example, suppose the system is set so that when the average value of the historical high-frequency occurrence area D of the inspected product defects within ten minutes is less than or equal to 40, the sampling frequency f = f0, where f0 is 20 products out of every 100 products. When the average value of the historical high-frequency occurrence area D of the inspected product defects within ten minutes is greater than 40, f is 25 products out of every 100 products. Suppose that the average value of the historical high-frequency occurrence area D of the inspected products within ten minutes is calculated to be 60, and 60 > 40, so the sampling frequency f is updated to 25 products out of every 100 products.

[0084] When the pass rate of sampled products within a unit of time is greater than 60%, it means that the probability of defects in such products has decreased recently. At this time, the strategy is switched to full inspection, and products without defects are judged as qualified. When the pass rate of full inspection within a unit of time is less than 60%, the strategy is switched back to sampling inspection.

[0085] For general products not marked with secondary anomalies, the image segmentation method of the defect detection module is used to segment them. For each segmented region s, the highest tolerance level for different defect severity is determined by referring to historical data. Based on the defect detection results, it is searched to see if there are any defects in segmented region s that exceed its highest tolerance level. If so, other segmented regions are not searched, and the product is directly identified as scrap. If there are defects but none of them exceed the highest tolerance level of their respective segmented regions, the product is identified as reworkable. If the defect detection results are normal, the product is directly identified as qualified and no further segmented region search is performed.

[0086] For example, consider a product whose segmented image shows that regions 2 to 9 have the highest tolerance level for cracks, holes, and scratches—severe. However, segmented region 1 has a maximum tolerance level of slight for cracks and holes, and a maximum tolerance level of medium for scratches. Based on the defect detection results from the defect detection module, segmented region 1 has a crack defect with a comprehensive score of 0.55, and its severity is judged as medium. Since this defect exceeds the maximum tolerance level for crack defects in segmented region 1, it is directly judged as a scrap product in the sorting strategy, and no further search and judgment are performed on other segmented regions.

[0087] After the judgment is completed, a sorting strategy is generated and output to the sorting execution module.

[0088] The sorting execution module reads the defect detection results and sorts the products to designated areas according to the sorting strategy.

[0089] The above description is merely an example and illustration of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the invention or exceed the scope defined in the claims, they should all fall within the protection scope of the present invention.

Claims

1. A vision-based robot sorting method, characterized in that, Specifically, the following steps are included: Step 1: Collect product image information and perform enhancement, noise reduction, grayscale conversion, and binarization processing on the collected images; Step 2: Perform connected component analysis on the binarized image obtained in Step 1, calculate the gray-scale mean of each connected component, mark abnormal connected components based on the gray-scale mean, read the historical defect high-frequency occurrence area in the historical detection data, and perform secondary anomaly marking on the abnormal connected components located in the area. Step 3: Identify the defective regions by calculating the geometric features of each abnormal connected region, use a weighted comprehensive scoring method to determine the severity of each defective region, and output the defect detection results. Step 4: Mark areas with frequent problem feedback. Based on secondary anomaly marking, historical high-frequency defect areas, and defect detection results, formulate product sorting strategies and sort products according to these strategies. The specific method for determining areas with frequent problem feedback in step four is as follows: The method of segmenting the grayscale image of the product to obtain the segmented image is adopted. The segments are labeled as s, where s = 1, 2, ..., k, and s represents the number of segmented regions. Based on the feedback of after-sales issues, for the defects reported by customers, the segmented regions s in which they are located are counted. The proportion of defect feedback in segmented region s to the total number of after-sales issues within a unit of time is denoted as the feedback ratio R. The feedback ratios of segmented regions 1, 2, ..., 9 are denoted as R1, R2, ..., R9, respectively. A feedback ratio warning threshold is set. When the feedback ratio of segmented region s is greater than the feedback ratio warning threshold, segmented region s is marked as a region with frequent problem feedback.

2. The vision-based robot sorting method according to claim 1, characterized in that, The specific calculation method for the gray-scale mean of the connected components in step two is as follows: For each pixel i within the connected region, obtain the intensity values ​​of its corresponding red, green, and blue channels, denoted as R, G, and B respectively. The grayscale value I can be obtained from I = 0.299R + 0.587G + 0.114B. Adding the gray values ​​I of all pixels within connected region i and then dividing by the total number of pixels n in that connected region yields the average gray value μi of that connected region, i.e.: Gray mean .

3. The vision-based robot sorting method according to claim 1, characterized in that, The specific method for determining the high-frequency occurrence area of ​​historical defects in step two is as follows: The grayscale image of the product is segmented to obtain segmented images and labeled as s, s=1, 2, ..., k, where k represents the number of segmented regions. For the same segmented image s of all product images, there is a number of defects d, the initial value of d is 0. Whenever a defective region appears on the segmented region s, the number of defects d in the segmented image s is incremented by one. This operation is performed for each product image to be sorted. Let D be the number of defects occurring in the same segmented image s of N previously inspected product images, where N is a fixed number determined manually by combining historical data. , Let d be the first and last products out of N products that have been inspected. The segmented image s where the number of defects D exceeds the manually set threshold is recorded as the high-frequency region of historical defects.

4. The vision-based robot sorting method according to claim 1, characterized in that, The specific method for determining the secondary anomaly marking of abnormal connected regions in step two is as follows: Read the historical high-frequency defect areas returned during the defect detection of the previous product, compare them with the image that has been marked with abnormal connected components, and mark the abnormal connected components located in the historical high-frequency defect areas with secondary anomalies.

5. The vision-based robot sorting method according to claim 1, characterized in that, The specific calculation method for the weighted composite score in step three is as follows: Based on historical data analysis, weighted averages are applied according to the influence of different geometric features on different defects. First, normalized scores are calculated for each geometric feature of the target defect area using a normalization method. For example, the normalized score for the area can be calculated using... To determine the score, the weighted scores are summed to obtain the weighted overall score for the defective area.

6. The vision-based robot sorting method according to claim 1, characterized in that, The specific judgment rules used to formulate the sorting strategy in step four are as follows: The system compares the areas with frequent problem reports with the historical high-frequency defect areas where connected regions with secondary anomaly markers are located. If the two areas overlap, the product is determined to be defective. If they do not overlap, the system combines the defect detection results to determine the defective products in the area that are found to be defective. For products with secondary anomaly markers that are not judged as defective, a sampling inspection strategy is implemented. Products that are not sampled are judged as reworkable, and products that are sampled are sorted according to the inspection results. For general products, image segmentation is performed. Based on the different tolerance levels of different segmented regions for different defects, and combined with the defect detection results, it is searched to see if there are any defects in each segmented region that exceed the highest tolerance level of that region. If so, the product is directly judged as scrap. If there are defects but none of them exceed the highest tolerance level of their respective segmented regions, the product is judged as reworkable. If the defect detection results are normal, the product is directly judged as qualified.

7. A vision-based robot sorting method according to claim 6, characterized in that, The sampling inspection strategy implemented in the judgment rules upon which the sorting strategy is based is specifically as follows: For products with secondary anomaly markers but not classified as defective, a sampling inspection strategy is implemented, with the sampling frequency denoted as f, and the initial value being... The value of f is not 0. It is determined by the average value of D values ​​of the historical high-frequency defect occurrence area of ​​all products inspected within a unit time. When the pass rate of the sampled products within a unit time is greater than 60%, the strategy is switched to full inspection. When the pass rate of the full inspection products within a unit time is less than 60%, the strategy is switched back to sampling inspection.

8. A vision-based robotic sorting system, characterized in that, The system is used to perform a vision-based robotic sorting method as described in any one of claims 1 to 7.

Images