Material detecting and sorting system and method based on machine vision

Abstract

According to the material detecting and sorting system and method based on machine vision, the global image of the material cluster is collected through machine vision, the material space distribution feature information in the material cluster is obtained, all sorting executable areas in the material cluster are positioned, the execution space range of material sorting is accurately limited, and the sorting efficiency is improved. The situation that the detection workload is increased due to repeated visual recognition on a large-size material cluster is avoided; a global image of the sorting executable area is collected through machine vision, material type distribution feature information in the sorting executable area is obtained, a material sorting grabbing path is determined accordingly, and accurate navigation is provided for material sorting in the sorting executable area; the material sorting operation dynamic image is collected based on the material sorting and grabbing path and machine vision, so that the material sorting action state is adjusted, material sorting and grabbing can be conducted in order in the sorting executable area, unified machine vision recognition and sorting of materials with the same attribute are achieved, and the material sorting efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision recognition, and in particular to a material detection and sorting system and method based on machine vision. Background Art

[0002] Machine vision is an important means of automated and intelligent production operations. Through machine vision technology, comprehensive environmental recognition of production operation scenes can be carried out, which facilitates timely response of production operation equipment and appropriate action adjustments. In particular, in large-scale product production scenarios, a large number of materials need to be inspected and sorted, and different materials need to be placed on appropriate production lines. In order to ensure the efficiency and accuracy of material inspection and sorting, it is necessary to use automated equipment such as manipulators and machine vision technology to complete material inspection and sorting. The existing manipulators perform material inspection and sorting operations by directly performing machine vision identification on all materials, calibrating the current sorted materials, and then controlling the manipulators to directly grab the sorted materials. Although the above method can ensure the accuracy of material inspection and sorting, it can only perform machine vision identification and sorting on a single material at a time, and cannot perform synchronous machine vision identification and sorting on materials with the same properties, which reduces the efficiency and speed of material inspection and sorting. Summary of the invention

[0003] The object of the present invention is to provide a material detection and sorting system and method based on machine vision, wherein the machine vision collects a global image of a material cluster and obtains the spatial distribution characteristic information of the materials in the material cluster, thereby locating all sorting executable areas in the material cluster, accurately limiting the execution space range of material sorting, and avoiding repeated visual recognition of large-volume material clusters to increase the detection workload; the machine vision collects a global image of the sorting executable area and obtains the material type distribution characteristic information in the sorting executable area, thereby determining the material sorting grabbing path, and providing accurate navigation for material sorting in the sorting executable area; based on the material sorting grabbing path, the machine vision collects dynamic images of the material sorting operation, thereby adjusting the material sorting action state, and being able to orderly perform material sorting and grabbing in the sorting executable area, realizing unified machine vision recognition and sorting of materials with the same attributes, and improving material sorting efficiency and accuracy.

[0004] The present invention is achieved through the following technical solutions:

[0005] A material detection and sorting system based on machine vision, comprising:

[0006] A first machine vision recognition module is used to perform a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; perform material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of materials within the material cluster;

[0007] A sorting executable area positioning module, used to locate all sorting executable areas in the material cluster based on the material spatial distribution feature information;

[0008] A second machine vision recognition module is used to perform a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; recognize the global image of the sorting area to obtain material type distribution feature information in the sorting executable area;

[0009] A material sorting and grabbing path determination module, used to determine the material sorting and grabbing path of the sorting executable area based on the material type distribution characteristic information;

[0010] A third machine vision recognition module is used to perform a third machine vision acquisition on the material sorting operation in the sorting executable area based on the material sorting grabbing path to obtain a dynamic image of the material sorting operation;

[0011] The material sorting action adjustment module is used to adjust the material sorting action state based on the material sorting operation dynamic image.

[0012] Optionally, the first machine vision recognition module is used to perform a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; perform material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of materials within the material cluster, including:

[0013] Based on the spatial boundary position of the material cluster, the plane area where the material cluster is located is divided into a number of visual acquisition sub-areas with regular shapes; based on the boundary position of each visual acquisition sub-area, all visual acquisition sub-areas are sequentially subjected to machine vision acquisition to obtain a number of material cluster sub-images, and all material cluster sub-images are spliced ​​into a global image of the material cluster;

[0014] Perform pixel edge sharpening and pixel contour recognition processing on the global image of the material cluster to obtain edge contour information of all materials in the global image of the material cluster; based on the edge contour information, obtain the spatial gap size information between any two adjacent materials in the material cluster, and use it as the spatial distribution feature information of the materials in the material cluster;

[0015] The sorting executable area positioning module is used to locate all sorting executable areas in the material cluster based on the material spatial distribution feature information, including:

[0016] Based on the material spatial distribution characteristic information, the spatial overlapping status of all materials in the material cluster is analyzed to obtain the spatial overlapping area information of all materials in the material cluster, so as to locate all sorting executable areas in the material cluster.

[0017] Optionally, before performing material contour recognition on the material cluster global image, performing contrast adjustment on the material cluster global image includes:

[0018] Extracting the grayscale value corresponding to each pixel point contained in the material cluster global map;

[0019] Compare the grayscale value corresponding to each pixel with a preset first grayscale threshold and a second grayscale threshold;

[0020] Retrieving the grayscale value corresponding to the pixel point lower than the first grayscale threshold as the first grayscale value;

[0021] Retrieving the grayscale value corresponding to the pixel point that is higher than the second grayscale threshold as the second grayscale value;

[0022] Obtaining a contrast adjustment coefficient using the first grayscale value and the second grayscale value;

[0023] The contrast adjustment coefficient is obtained by the following formula:

[0024]

[0025] Wherein, S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; F 02i represents the gray value of the i-th second gray value; F 01i represents the gray value of the i-th first gray value; F 02z represents the grayscale value middle value of m second grayscale values; F 01z represents the middle value of the grayscale value of n first grayscale values; F z Represents the central gray value of the global graph of material clusters; F 01 and F 02 Represents the first grayscale threshold and the second grayscale threshold

[0026] Using the contrast adjustment coefficient to adjust the contrast of the material cluster global map, obtain the material cluster global map after contrast adjustment, and replace the original unadjusted material cluster global map;

[0027] Among them, the adjusted contrast value is obtained by the following formula:

[0028]

[0029] Among them, D t represents the contrast value after adjustment; D0 represents the contrast value before adjustment; S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; k represents the total number of pixels contained in the global map of the material cluster.

[0030] Optionally, the second machine vision recognition module is used to perform a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; recognize the global image of the sorting area to obtain material type distribution feature information in the sorting executable area, including:

[0031] Synchronously performing RGB machine vision acquisition and near-infrared spectrum vision acquisition on the sorting executable area to obtain an RGB global image and a near-infrared spectrum global image of the sorting executable area;

[0032] Perform material contour recognition on the RGB global image to obtain the position information and shape and size information of all materials in the sorting executable area; perform near-infrared reflectivity recognition on the near-infrared spectral global image to obtain the material type information of all materials in the sorting executable area; based on the position information, the shape and size information and the material type information, generate the shape and size, material type and position mapping relationship information of the materials in the sorting executable area, and use this as the material type distribution feature information in the sorting executable area;

[0033] The material sorting grabbing path determination module is used to determine the material sorting grabbing path of the sorting executable area based on the material type distribution characteristic information, including:

[0034] Based on the material type distribution characteristic information, all materials in the sorting executable area are divided into several material clusters; wherein all materials under each material cluster can be sorted and grasped with the same grasping action; based on the location information of all materials under each material cluster, the sorting and grasping path of all materials under each material cluster is determined.

[0035] Optionally, the third machine vision recognition module is used to perform a third machine vision acquisition on the material sorting operation in the sorting executable area based on the material sorting grabbing path to obtain a dynamic image of the material sorting operation, including:

[0036] Based on the material sorting grabbing path, dynamically track and visually collect the material sorting operation in the sorting executable area to obtain a dynamic image of the material sorting operation;

[0037] The material sorting action adjustment module is used to adjust the material sorting action state based on the material sorting operation dynamic image, including:

[0038] The dynamic image of the material sorting operation is analyzed to determine the relative distance and relative orientation information between the sorting and grabbing robot and the materials to be sorted and grabbed; based on the relative distance and relative orientation information, the moving speed and sorting and grabbing action posture of the sorting and grabbing robot relative to the materials to be sorted and grabbed are adjusted.

[0039] A material detection and sorting method based on machine vision, comprising:

[0040] Performing a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; performing material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of the materials within the material cluster; locating all sorting executable areas within the material cluster based on the spatial distribution feature information of the materials;

[0041] Performing a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; identifying the global image of the sorting area to obtain material type distribution characteristic information in the sorting executable area; and determining a material sorting grabbing path in the sorting executable area based on the material type distribution characteristic information;

[0042] Based on the material sorting grabbing path, the material sorting operation in the sorting executable area is collected by a third machine vision to obtain a dynamic image of the material sorting operation; based on the dynamic image of the material sorting operation, the material sorting action state is adjusted.

[0043] Optionally, performing a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; performing material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of materials within the material cluster; and locating all sorting executable areas within the material cluster based on the spatial distribution feature information of the materials, including:

[0044] Based on the spatial boundary position of the material cluster, the plane area where the material cluster is located is divided into a number of visual acquisition sub-areas with regular shapes; based on the boundary position of each visual acquisition sub-area, all visual acquisition sub-areas are sequentially subjected to machine vision acquisition to obtain a number of material cluster sub-images, and all material cluster sub-images are spliced ​​into a global image of the material cluster;

[0045] Perform pixel edge sharpening and pixel contour recognition processing on the global image of the material cluster to obtain edge contour information of all materials in the global image of the material cluster; based on the edge contour information, obtain the spatial gap size information between any two adjacent materials in the material cluster, and use it as the spatial distribution feature information of the materials in the material cluster;

[0046] Based on the material spatial distribution characteristic information, the spatial overlapping status of all materials in the material cluster is analyzed to obtain the spatial overlapping area information of all materials in the material cluster, so as to locate all sorting executable areas in the material cluster.

[0047] Optionally, before performing material contour recognition on the material cluster global image, performing contrast adjustment on the material cluster global image includes:

[0048] Extracting the grayscale value corresponding to each pixel point contained in the material cluster global map;

[0049] Compare the grayscale value corresponding to each pixel with a preset first grayscale threshold and a second grayscale threshold;

[0050] Retrieving the grayscale value corresponding to the pixel point lower than the first grayscale threshold as the first grayscale value;

[0051] Retrieving the grayscale value corresponding to the pixel point that is higher than the second grayscale threshold as the second grayscale value;

[0052] Obtaining a contrast adjustment coefficient using the first grayscale value and the second grayscale value;

[0053] The contrast adjustment coefficient is obtained by the following formula:

[0054]

[0055] Wherein, S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; F 02i represents the gray value of the i-th second gray value; F 01i represents the gray value of the i-th first gray value; F 02z represents the grayscale value middle value of m second grayscale values; F 01z represents the middle value of the grayscale value of n first grayscale values; F z Represents the central gray value of the global graph of material clusters; F 01 and F 02 Represents the first grayscale threshold and the second grayscale threshold

[0056] Using the contrast adjustment coefficient to adjust the contrast of the material cluster global map, obtain the material cluster global map after contrast adjustment, and replace the original unadjusted material cluster global map;

[0057] Among them, the adjusted contrast value is obtained by the following formula:

[0058]

[0059] Among them, Dt represents the contrast value after adjustment; D0 represents the contrast value before adjustment; S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; k represents the total number of pixels contained in the global map of the material cluster.

[0060] Optionally, performing a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; identifying the global image of the sorting area to obtain material type distribution feature information in the sorting executable area; and determining a material sorting path in the sorting executable area based on the material type distribution feature information, including:

[0061] Synchronously performing RGB machine vision acquisition and near-infrared spectrum vision acquisition on the sorting executable area to obtain an RGB global image and a near-infrared spectrum global image of the sorting executable area;

[0062] Perform material contour recognition on the RGB global image to obtain the position information and shape and size information of all materials in the sorting executable area; perform near-infrared reflectivity recognition on the near-infrared spectral global image to obtain the material type information of all materials in the sorting executable area; based on the position information, the shape and size information and the material type information, generate the shape and size, material type and position mapping relationship information of the materials in the sorting executable area, and use this as the material type distribution feature information in the sorting executable area;

[0063] Based on the material type distribution characteristic information, all materials in the sorting executable area are divided into several material clusters; wherein all materials under each material cluster can be sorted and grasped with the same grasping action; based on the location information of all materials under each material cluster, the sorting and grasping path of all materials under each material cluster is determined.

[0064] Optionally, based on the material sorting grabbing path, performing a third machine vision acquisition on the material sorting operation in the sorting executable area to obtain a material sorting operation dynamic image; and based on the material sorting operation dynamic image, adjusting the material sorting action state, including:

[0065] Based on the material sorting grabbing path, dynamically track and visually collect the material sorting operation in the sorting executable area to obtain a dynamic image of the material sorting operation;

[0066] The dynamic image of the material sorting operation is analyzed to determine the relative distance and relative orientation information between the sorting and grabbing robot and the materials to be sorted and grabbed; based on the relative distance and relative orientation information, the moving speed and sorting and grabbing action posture of the sorting and grabbing robot relative to the materials to be sorted and grabbed are adjusted.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] The present application provides a material detection and sorting system and method based on machine vision. Machine vision collects global images of material clusters and obtains spatial distribution feature information of materials in the material clusters, thereby locating all sorting executable areas in the material clusters, accurately limiting the execution space range of material sorting, and avoiding repeated visual recognition of large-volume material clusters to increase the detection workload; machine vision collects global images of the sorting executable area and obtains material type distribution feature information in the sorting executable area, thereby determining the material sorting grasping path, and providing accurate navigation for material sorting in the sorting executable area; based on the material sorting grasping path, machine vision collects dynamic images of material sorting operations, thereby adjusting the material sorting action state, and being able to orderly sort and grasp materials in the sorting executable area, realizing unified machine vision recognition and sorting of materials with the same attributes, and improving material sorting efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0070] Figure 1 A structural schematic diagram of a material detection and sorting system based on machine vision provided by the present invention.

[0071] Figure 2 A schematic flow chart of a material detection and sorting method based on machine vision provided by the present invention. DETAILED DESCRIPTION

[0072] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some structures related to the present application are shown in the accompanying drawings, rather than all structures. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0073] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0074] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0075] See also Figure 1 As shown, an embodiment of the present application provides a material detection and sorting system based on machine vision. The material detection and sorting system based on machine vision includes:

[0076] A first machine vision recognition module is used to perform a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; perform material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of materials within the material cluster;

[0077] A sorting executable area positioning module is used to locate all sorting executable areas within the material cluster based on the spatial distribution feature information of the material;

[0078] The second machine vision recognition module is used to perform a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; recognize the global image of the sorting area to obtain material type distribution feature information in the sorting executable area;

[0079] A material sorting grabbing path determination module is used to determine the material sorting grabbing path of the sorting executable area based on the material type distribution characteristic information;

[0080] A third machine vision recognition module is used to perform a third machine vision acquisition of the material sorting operation in the sorting executable area based on the material sorting grabbing path to obtain a dynamic image of the material sorting operation;

[0081] The material sorting action adjustment module is used to adjust the material sorting action state based on the material sorting operation dynamic image.

[0082] The beneficial effects of the above embodiments are as follows: the machine vision-based material detection and sorting system uses machine vision to collect global images of material clusters and obtain spatial distribution feature information of materials within the material cluster, thereby locating all sorting executable areas within the material cluster, accurately limiting the execution space range of material sorting, and avoiding repeated visual recognition of large-volume material clusters to increase the detection workload; machine vision collects global images of the sorting executable area and obtains material type distribution feature information within the sorting executable area, thereby determining the material sorting grabbing path, and providing accurate navigation for material sorting in the sorting executable area; based on the material sorting grabbing path, machine vision collects dynamic images of material sorting operations, thereby adjusting the material sorting action state, and being able to perform material sorting and grabbing in an orderly manner within the sorting executable area, realizing unified machine vision recognition and sorting of materials with the same attributes, and improving material sorting efficiency and accuracy.

[0083] In another embodiment, the first machine vision recognition module is used to perform a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; perform material contour recognition on the global image of the material cluster to obtain material spatial distribution feature information within the material cluster, including:

[0084] Based on the spatial boundary position of the material cluster, the plane area where the material cluster is located is divided into a number of visual acquisition sub-areas with regular shapes; based on the boundary position of each visual acquisition sub-area, all visual acquisition sub-areas are sequentially subjected to machine vision acquisition to obtain a number of material cluster sub-images, and all material cluster sub-images are spliced ​​into a global image of the material cluster;

[0085] Perform pixel edge sharpening and pixel contour recognition processing on the global image of the material cluster to obtain edge contour information of all materials in the global image of the material cluster; based on the edge contour information, obtain the spatial gap size information between any two adjacent materials in the material cluster, and use it as the spatial distribution feature information of the materials in the material cluster;

[0086] The sorting executable area positioning module is used to locate all sorting executable areas in the material cluster based on the material spatial distribution feature information, including:

[0087] Based on the spatial distribution characteristic information of the material, the spatial overlapping status of all materials in the material cluster is analyzed to obtain the spatial overlapping area information of all materials in the material cluster, so as to locate all sorting executable areas in the material cluster.

[0088] The beneficial effect of the above-mentioned embodiment is that in the material detection and sorting scenario, it is necessary to accurately detect and sort each material inside a large-volume material cluster, so as to correctly transfer each sorted material to the corresponding production process. The material cluster contains a huge number of materials and different types of materials, and the distribution position and overlapping and interlacing of different types of materials in the material set are not the same. If the material cluster is globally machine-visually identified every time the material detection and sorting are performed, the recognition workload will increase and the real-time performance of the detection and sorting will be reduced. In order to perform small-scale machine-visual identification on the material cluster, it is necessary to divide the material cluster into machine-visual identification areas in advance, so that each time the IoT detection and sorting is performed, only the materials in the small area need to be machine-visually identified, which effectively reduces the workload of visual identification. In order to first perform global machine-visual identification on the material cluster, based on the spatial boundary position of the material cluster, the plane area where the material cluster is located is divided into several visual acquisition sub-areas with regular shapes, so that all visual acquisition sub-areas are sequentially machine-visually acquired, and several material cluster sub-images corresponding to all visual acquisition sub-areas are obtained, so that all material cluster sub-images are spliced ​​into a global image of the material cluster, and the material cluster is globally visually represented. Then, the global image of the material cluster is subjected to pixel edge sharpening and pixel contour recognition processing to obtain the edge contour information of all materials in the global image of the material cluster, which is the edge contour information of the outer surface of each material. The edge contours of any two adjacent materials in the material cluster are compared to determine the spatial gap size information between any two adjacent materials, thereby comprehensively identifying the size of the interval between adjacent materials in the material cluster. When some materials in the material cluster overlap with each other, it will cause great difficulty in identifying and operating the robot to detect and sort these materials, which is not conducive to the robot to quickly detect and sort out the corresponding materials. In order to select areas with larger spatial gaps between materials in the material cluster to avoid interference in material visual recognition for sorting operations, the spatial overlapping status of all materials in the material cluster is analyzed based on the spatial gap size information between any two adjacent materials in the material cluster to obtain the spatial overlapping area information of all materials in the material cluster. If the average spatial overlapping area between materials in a certain area in the material cluster is less than the preset area threshold, the corresponding area is determined as the sorting executable area in the material cluster, and the detection and sorting operations of the robot are limited to the sorting executable area to avoid global visual recognition of the material cluster and affect the material sorting efficiency.

[0089] In another embodiment, before performing material contour recognition on the material cluster global image, performing contrast adjustment on the material cluster global image includes:

[0090] Extracting the grayscale value corresponding to each pixel point contained in the material cluster global map;

[0091] Compare the grayscale value corresponding to each pixel with a preset first grayscale threshold and a second grayscale threshold;

[0092] Retrieving the grayscale value corresponding to the pixel point lower than the first grayscale threshold as the first grayscale value;

[0093] Retrieving the grayscale value corresponding to the pixel point that is higher than the second grayscale threshold as the second grayscale value;

[0094] Obtaining a contrast adjustment coefficient using the first grayscale value and the second grayscale value;

[0095] The contrast adjustment coefficient is obtained by the following formula:

[0096]

[0097] Wherein, S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; F 02i represents the gray value of the i-th second gray value; F 01i represents the gray value of the i-th first gray value; F 02z represents the grayscale value middle value of m second grayscale values; F 01z represents the middle value of the grayscale value of n first grayscale values; F z Represents the central gray value of the global graph of material clusters; F 01 and F 02 Represents the first grayscale threshold and the second grayscale threshold

[0098] Using the contrast adjustment coefficient to adjust the contrast of the material cluster global map, obtain the material cluster global map after contrast adjustment, and replace the original unadjusted material cluster global map;

[0099] Among them, the adjusted contrast value is obtained by the following formula:

[0100]

[0101] Among them, D t represents the contrast value after adjustment; D0 represents the contrast value before adjustment; S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; k represents the total number of pixels contained in the global map of the material cluster.

[0102] The beneficial effect of the above embodiment is that by extracting the grayscale value of each pixel in the global image of the material cluster and comparing it with the preset first grayscale threshold and second grayscale threshold, the dark area and the bright area in the image can be accurately identified. This detailed distinction helps to make the subsequent contrast adjustment more accurate. By using the first grayscale value (dark area grayscale value) and the second grayscale value (bright area grayscale value) to calculate the contrast adjustment coefficient, it can be ensured that the contrast of the adjusted image is significantly improved while maintaining the details. This adjustment method is more flexible and accurate than simple linear stretching or histogram equalization. The global image of the material cluster after contrast adjustment is visually clearer, and the details of the dark and bright parts are retained and enhanced. This is crucial for subsequent image processing tasks such as material contour recognition, because a clear image can significantly improve the accuracy and efficiency of recognition. By replacing the original unadjusted image, it is ensured that the subsequent processing steps (such as material contour recognition) are all based on the optimized image, thereby improving the stability and reliability of the entire processing flow. The contrast adjustment coefficient calculation formula in this technical solution takes into account multiple factors, including the number of second grayscale values, the number of first grayscale values, the median value of the grayscale value, and the central grayscale value of the global image. This comprehensive consideration enables the algorithm to adapt to images under different lighting conditions, different material types, and different shooting angles, enhancing the versatility and adaptability of the algorithm. Through the adjusted contrast numerical formula, it can be ensured that the adjusted image contrast is within a reasonable range, avoiding over-enhancement or under-enhancement, thereby ensuring the stability and consistency of image processing. Although the technical solution involves multiple steps and calculations in the contrast adjustment process, the entire processing process can still maintain a high efficiency due to the use of an efficient grayscale value extraction and comparison algorithm, as well as a reasonable contrast adjustment coefficient calculation formula. This is especially important for material recognition systems that require real-time processing.

[0103] In summary, this technical solution has achieved remarkable technical results in terms of contrast optimization, image quality improvement, algorithm adaptability enhancement, and processing efficiency improvement. These effects work together in the processing flow of the global image of the material cluster, providing strong support for subsequent tasks such as material contour recognition.

[0104] In another embodiment, the second machine vision recognition module is used to perform a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; recognize the global image of the sorting area to obtain material type distribution feature information in the sorting executable area, including:

[0105] Synchronously performing RGB machine vision acquisition and near-infrared spectrum vision acquisition on the sorting executable area to obtain an RGB global image and a near-infrared spectrum global image of the sorting executable area;

[0106] The material outline is identified on the RGB global image to obtain the position information and shape and size information of all the materials in the sorting executable area; the near-infrared reflectivity is identified on the near-infrared spectrum global image to obtain the material type information of all the materials in the sorting executable area; based on the position information, the shape and size information and the material type information, the shape and size, material type and position mapping relationship information of the materials in the sorting executable area is generated, which is used as the material type distribution feature information in the sorting executable area;

[0107] The material sorting grabbing path determination module is used to determine the material sorting grabbing path of the sorting executable area based on the material type distribution characteristic information, including:

[0108] Based on the material type distribution characteristic information, all materials in the sorting executable area are divided into several material clusters; all materials under each material cluster can be sorted and grasped with the same grasping action; based on the location information of all materials under each material cluster, the sorting and grasping path of all materials under each material cluster is determined.

[0109] The beneficial effects of the above embodiments are that the types, shapes and sizes of materials in each sorting executable area are different, and the sorting and grabbing methods of materials of different types, shapes and sizes are different. For example, materials containing iron components can be sorted and grabbed by magnetic suction, while materials formed by fragile materials such as glass need to be sorted and grabbed by a manipulator with a protective layer such as silica gel; in addition, materials with larger volumes need to be sorted and grabbed by a large-volume manipulator, while materials with smaller volumes need to be sorted and grabbed by a small-volume manipulator. In order to uniformly sort and grab materials with the same or similar material and external size attributes in the sorting executable area, RGB machine vision acquisition and near-infrared spectral vision acquisition are performed simultaneously on the sorting executable area to obtain an RGB global image and a near-infrared spectral global image of the sorting executable area. The RGB visible light image of the material can reflect the appearance of the material, and the absorption rate of near-infrared light by different materials is different, so the near-infrared spectral reflectance corresponding to the near-infrared images of different materials is also different. Therefore, the RGB global image and the near-infrared spectral global image of the sorting executable area are visually collected, and the shape, size and material of all materials in the sorting executable area can be accurately distinguished and identified. Specifically, the material shape contour is identified on the RGB global image to obtain the position information and shape and size information of all materials in the sorting executable area; the near-infrared reflectance is identified on the near-infrared spectral global image to obtain the material type information of all materials in the sorting executable area; based on the position information, the shape and size information and the material type information, the shape and size, material type and position mapping relationship information of the materials in the sorting executable area is generated, so as to compare and characterize the shape, size, material type and existence position of each material in the sorting executable area. In addition, based on the material type distribution characteristic information, all materials in the sorting executable area are divided into several material clusters, so that all materials under each material cluster can be sorted and grasped with the same grasping action, thereby realizing unified sorting and grasping of all materials under each material cluster, and the location information of all materials under each material cluster is used to determine the sorting and grasping path of all materials under each material cluster, thereby ensuring orderly sorting and grasping of all materials under each material cluster.

[0110] In another embodiment, the third machine vision recognition module is used to perform a third machine vision acquisition of the material sorting operation in the sorting executable area based on the material sorting grabbing path to obtain a dynamic image of the material sorting operation, including:

[0111] Based on the material sorting grabbing path, the material sorting operation in the sorting executable area is dynamically tracked and visually captured to obtain a dynamic image of the material sorting operation;

[0112] The material sorting action adjustment module is used to adjust the material sorting action state based on the material sorting operation dynamic image, including:

[0113] The dynamic image of the material sorting operation is analyzed to determine the relative distance and relative orientation information between the sorting and grasping robot and the material to be sorted and grasped; based on the relative distance and relative orientation information, the moving speed and sorting and grasping action posture of the sorting and grasping robot relative to the material to be sorted and grasped are adjusted.

[0114] The beneficial effect of the above embodiment is that the material sorting and grabbing path is used to navigate the path of the manipulator for sorting and grabbing different materials in sequence in the sorting executable area. In order to ensure that the manipulator accurately grabs the material during the movement along the material sorting and grabbing path, the material sorting operation in the sorting executable area is dynamically tracked and visually captured to obtain a dynamic image of the material sorting operation, and then the dynamic image of the material sorting operation is analyzed to determine the relative distance and relative orientation information between the sorting and grabbing manipulator and the material to be sorted and grabbed; and based on the relative distance and relative orientation information, the manipulator can accurately grab the material in the sorting executable area. According to the relative position information, the moving speed and sorting and grabbing posture of the sorting and grabbing robot arm relative to the material to be sorted and grabbed are adjusted. When the relative distance is less than the preset distance threshold, it indicates that the robot arm and the material to be sorted and grabbed are close enough. At this time, the moving speed of the robot arm relative to the material to be sorted and grabbed should be reduced to avoid collision between the robot arm and the material to be sorted and grabbed. According to the relative position information, the sorting and grabbing posture of the robot arm relative to the material to be sorted and grabbed is adjusted to ensure that the robot arm can stably grab the corresponding material and effectively avoid the material from falling.

[0115] See also Figure 2 As shown, an embodiment of the present application provides a material detection and sorting method based on machine vision. The material detection and sorting method based on machine vision includes:

[0116] Performing a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; performing material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of the materials in the material cluster; locating all sorting executable areas in the material cluster based on the spatial distribution feature information of the materials;

[0117] Performing a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; identifying the global image of the sorting area to obtain material type distribution characteristic information in the sorting executable area; and determining a material sorting grabbing path for the sorting executable area based on the material type distribution characteristic information;

[0118] Based on the material sorting grabbing path, the material sorting operation in the sorting executable area is collected by a third machine vision to obtain a dynamic image of the material sorting operation; based on the dynamic image of the material sorting operation, the material sorting action state is adjusted.

[0119] The beneficial effects of the above embodiments are as follows: the machine vision-based material detection and sorting method uses machine vision to collect global images of material clusters and obtain spatial distribution feature information of materials within the material cluster, thereby locating all sorting executable areas within the material cluster, accurately limiting the execution space range of material sorting, and avoiding repeated visual recognition of large-volume material clusters to increase the detection workload; machine vision collects global images of the sorting executable area and obtains material type distribution feature information within the sorting executable area, thereby determining the material sorting grabbing path, and providing accurate navigation for material sorting in the sorting executable area; based on the material sorting grabbing path, machine vision collects dynamic images of material sorting operations, thereby adjusting the material sorting action state, and being able to perform material sorting and grabbing in an orderly manner within the sorting executable area, thereby achieving unified machine vision recognition and sorting of materials with the same attributes, and improving material sorting efficiency and accuracy.

[0120] In another embodiment, a first machine vision acquisition is performed on a material cluster to obtain a global image of the material cluster; a material contour is identified on the global image of the material cluster to obtain spatial distribution feature information of materials in the material cluster; and based on the spatial distribution feature information of the materials, all sorting executable areas in the material cluster are located, including:

[0121] Based on the spatial boundary position of the material cluster, the plane area where the material cluster is located is divided into a number of visual acquisition sub-areas with regular shapes; based on the boundary position of each visual acquisition sub-area, all visual acquisition sub-areas are sequentially subjected to machine vision acquisition to obtain a number of material cluster sub-images, and all material cluster sub-images are spliced ​​into a global image of the material cluster;

[0122] Perform pixel edge sharpening and pixel contour recognition processing on the global image of the material cluster to obtain edge contour information of all materials in the global image of the material cluster; based on the edge contour information, obtain the spatial gap size information between any two adjacent materials in the material cluster, and use it as the spatial distribution feature information of the materials in the material cluster;

[0123] Based on the spatial distribution characteristic information of the material, the spatial overlapping status of all materials in the material cluster is analyzed to obtain the spatial overlapping area information of all materials in the material cluster, so as to locate all sorting executable areas in the material cluster.

[0124] The beneficial effect of the above-mentioned embodiment is that in the material detection and sorting scenario, it is necessary to accurately detect and sort each material inside a large-volume material cluster, so as to correctly transfer each sorted material to the corresponding production process. The material cluster contains a huge number of materials and different types of materials, and the distribution position and overlapping and interlacing of different types of materials in the material set are not the same. If the material cluster is globally machine-visually identified every time the material detection and sorting are performed, the recognition workload will increase and the real-time performance of the detection and sorting will be reduced. In order to perform small-scale machine-visual identification on the material cluster, it is necessary to divide the material cluster into machine-visual identification areas in advance, so that each time the IoT detection and sorting is performed, only the materials in the small area need to be machine-visually identified, which effectively reduces the workload of visual identification. In order to first perform global machine-visual identification on the material cluster, based on the spatial boundary position of the material cluster, the plane area where the material cluster is located is divided into several visual acquisition sub-areas with regular shapes, so that all visual acquisition sub-areas are sequentially machine-visually acquired, and several material cluster sub-images corresponding to all visual acquisition sub-areas are obtained, so that all material cluster sub-images are spliced ​​into a global image of the material cluster, and the material cluster is globally visually represented. Then, the global image of the material cluster is subjected to pixel edge sharpening and pixel contour recognition processing to obtain the edge contour information of all materials in the global image of the material cluster, which is the edge contour information of the outer surface of each material. The edge contours of any two adjacent materials in the material cluster are compared to determine the spatial gap size information between any two adjacent materials, thereby comprehensively identifying the size of the interval between adjacent materials in the material cluster. When some materials in the material cluster overlap with each other, it will cause great difficulty in identifying and operating the robot to detect and sort these materials, which is not conducive to the robot to quickly detect and sort out the corresponding materials. In order to select areas with larger spatial gaps between materials in the material cluster to avoid interference in material visual recognition for sorting operations, the spatial overlapping status of all materials in the material cluster is analyzed based on the spatial gap size information between any two adjacent materials in the material cluster to obtain the spatial overlapping area information of all materials in the material cluster. If the average spatial overlapping area between materials in a certain area in the material cluster is less than the preset area threshold, the corresponding area is determined as the sorting executable area in the material cluster, and the detection and sorting operations of the robot are limited to the sorting executable area to avoid global visual recognition of the material cluster and affect the material sorting efficiency.

[0125] In another embodiment, performing a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; identifying the global image of the sorting area to obtain material type distribution feature information in the sorting executable area; and determining a material sorting path in the sorting executable area based on the material type distribution feature information, including:

[0126] Synchronously performing RGB machine vision acquisition and near-infrared spectrum vision acquisition on the sorting executable area to obtain an RGB global image and a near-infrared spectrum global image of the sorting executable area;

[0127] The material outline is identified on the RGB global image to obtain the position information and shape and size information of all the materials in the sorting executable area; the near-infrared reflectivity is identified on the near-infrared spectrum global image to obtain the material type information of all the materials in the sorting executable area; based on the position information, the shape and size information and the material type information, the shape and size, material type and position mapping relationship information of the materials in the sorting executable area is generated, which is used as the material type distribution feature information in the sorting executable area;

[0128] Based on the material type distribution characteristic information, all materials in the sorting executable area are divided into several material clusters; all materials under each material cluster can be sorted and grasped with the same grasping action; based on the location information of all materials under each material cluster, the sorting and grasping path of all materials under each material cluster is determined.

[0129] The beneficial effects of the above embodiments are that the types, shapes and sizes of materials in each sorting executable area are different, and the sorting and grabbing methods of materials of different types, shapes and sizes are different. For example, materials containing iron components can be sorted and grabbed by magnetic suction, while materials formed by fragile materials such as glass need to be sorted and grabbed by a manipulator with a protective layer such as silica gel; in addition, materials with larger volumes need to be sorted and grabbed by a large-volume manipulator, while materials with smaller volumes need to be sorted and grabbed by a small-volume manipulator. In order to uniformly sort and grab materials with the same or similar material and external size attributes in the sorting executable area, RGB machine vision acquisition and near-infrared spectral vision acquisition are performed simultaneously on the sorting executable area to obtain an RGB global image and a near-infrared spectral global image of the sorting executable area. The RGB visible light image of the material can reflect the appearance of the material, and the absorption rate of near-infrared light by different materials is different, so the near-infrared spectral reflectance corresponding to the near-infrared images of different materials is also different. Therefore, the RGB global image and the near-infrared spectral global image of the sorting executable area are visually collected, and the shape, size and material of all materials in the sorting executable area can be accurately distinguished and identified. Specifically, the material shape contour is identified on the RGB global image to obtain the position information and shape and size information of all materials in the sorting executable area; the near-infrared reflectance is identified on the near-infrared spectral global image to obtain the material type information of all materials in the sorting executable area; based on the position information, the shape and size information and the material type information, the shape and size, material type and position mapping relationship information of the materials in the sorting executable area is generated, so as to compare and characterize the shape, size, material type and existence position of each material in the sorting executable area. In addition, based on the material type distribution characteristic information, all materials in the sorting executable area are divided into several material clusters, so that all materials under each material cluster can be sorted and grasped with the same grasping action, thereby realizing unified sorting and grasping of all materials under each material cluster, and the location information of all materials under each material cluster is used to determine the sorting and grasping path of all materials under each material cluster, thereby ensuring orderly sorting and grasping of all materials under each material cluster.

[0130] In another embodiment, based on the material sorting grabbing path, a third machine vision acquisition is performed on the material sorting operation of the sorting executable area to obtain a dynamic image of the material sorting operation; based on the dynamic image of the material sorting operation, the material sorting action state is adjusted, including:

[0131] Based on the material sorting grabbing path, the material sorting operation in the sorting executable area is dynamically tracked and visually captured to obtain a dynamic image of the material sorting operation;

[0132] The dynamic image of the material sorting operation is analyzed to determine the relative distance and relative orientation information between the sorting and grasping robot and the material to be sorted and grasped; based on the relative distance and relative orientation information, the moving speed and sorting and grasping action posture of the sorting and grasping robot relative to the material to be sorted and grasped are adjusted.

[0133] The beneficial effect of the above embodiment is that the material sorting and grabbing path is used to navigate the path of the manipulator for sorting and grabbing different materials in sequence in the sorting executable area. In order to ensure that the manipulator accurately grabs the material during the movement along the material sorting and grabbing path, the material sorting operation in the sorting executable area is dynamically tracked and visually captured to obtain a dynamic image of the material sorting operation, and then the dynamic image of the material sorting operation is analyzed to determine the relative distance and relative orientation information between the sorting and grabbing manipulator and the material to be sorted and grabbed; and based on the relative distance and relative orientation information, the manipulator can accurately grab the material in the sorting executable area. According to the relative position information, the moving speed and sorting and grabbing posture of the sorting and grabbing robot arm relative to the material to be sorted and grabbed are adjusted. When the relative distance is less than the preset distance threshold, it indicates that the robot arm and the material to be sorted and grabbed are close enough. At this time, the moving speed of the robot arm relative to the material to be sorted and grabbed should be reduced to avoid collision between the robot arm and the material to be sorted and grabbed. According to the relative position information, the sorting and grabbing posture of the robot arm relative to the material to be sorted and grabbed is adjusted to ensure that the robot arm can stably grab the corresponding material and effectively avoid the material from falling.

[0134] In general, the machine vision-based material detection and sorting system and method uses machine vision to collect global images of material clusters and obtain spatial distribution feature information of materials within the material clusters, thereby locating all sorting executable areas within the material clusters, accurately limiting the execution space range of material sorting, and avoiding repeated visual recognition of large-volume material clusters to increase the detection workload; machine vision collects global images of sorting executable areas and obtains material type distribution feature information within the sorting executable areas, thereby determining material sorting grabbing paths, and providing accurate navigation for material sorting in the sorting executable areas; based on the material sorting grabbing paths, machine vision collects dynamic images of material sorting operations, thereby adjusting the material sorting action status, and being able to perform material sorting and grabbing in an orderly manner within the sorting executable areas, thereby achieving unified machine vision recognition and sorting of materials with the same attributes, and improving material sorting efficiency and accuracy.

[0135] The above is only a specific implementation of the present invention, and any other improvements made based on the concept of the present invention are considered to be within the protection scope of the present invention.

Claims

1. A material detection and sorting system based on machine vision, characterized in that: include: A first machine vision recognition module is used to perform a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; perform material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of materials within the material cluster; A sorting executable area positioning module, used to locate all sorting executable areas in the material cluster based on the material spatial distribution feature information; A second machine vision recognition module is used to perform a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; recognize the global image of the sorting area to obtain material type distribution feature information in the sorting executable area; A material sorting and grabbing path determination module, used to determine the material sorting and grabbing path of the sorting executable area based on the material type distribution characteristic information; A third machine vision recognition module is used to perform a third machine vision acquisition on the material sorting operation in the sorting executable area based on the material sorting grabbing path to obtain a dynamic image of the material sorting operation; The material sorting action adjustment module is used to adjust the material sorting action state based on the material sorting operation dynamic image.

2. The material detection and sorting system based on machine vision according to claim 1, characterized in that: The first machine vision recognition module is used to perform a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; perform material contour recognition on the global image of the material cluster to obtain material spatial distribution feature information within the material cluster, including: Based on the spatial boundary position of the material cluster, the plane area where the material cluster is located is divided into a number of visual acquisition sub-areas with regular shapes; based on the boundary position of each visual acquisition sub-area, all visual acquisition sub-areas are sequentially subjected to machine vision acquisition to obtain a number of material cluster sub-images, and all material cluster sub-images are spliced ​​into a global image of the material cluster; Perform pixel edge sharpening and pixel contour recognition processing on the global image of the material cluster to obtain edge contour information of all materials in the global image of the material cluster; based on the edge contour information, obtain the spatial gap size information between any two adjacent materials in the material cluster, and use it as the spatial distribution feature information of the materials in the material cluster; The sorting executable area positioning module is used to locate all sorting executable areas in the material cluster based on the material spatial distribution feature information, including: Based on the material spatial distribution characteristic information, the spatial overlapping status of all materials in the material cluster is analyzed to obtain the spatial overlapping area information of all materials in the material cluster, so as to locate all sorting executable areas in the material cluster.

3. The material detection and sorting system based on machine vision as claimed in claim 2, characterized in that: Before performing material contour recognition on the material cluster global image, performing contrast adjustment on the material cluster global image includes: Extracting the grayscale value corresponding to each pixel point contained in the material cluster global map; Compare the grayscale value corresponding to each pixel with a preset first grayscale threshold and a second grayscale threshold; Retrieving the grayscale value corresponding to the pixel point lower than the first grayscale threshold as the first grayscale value; Retrieving the grayscale value corresponding to the pixel point that is higher than the second grayscale threshold as the second grayscale value; Obtaining a contrast adjustment coefficient using the first grayscale value and the second grayscale value; The contrast adjustment coefficient is obtained by the following formula: Wherein, S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; F 02i represents the gray value of the i-th second gray value; F 01i represents the gray value of the i-th first gray value; F 02z represents the grayscale value middle value of m second grayscale values; F 01z represents the middle value of the grayscale value of n first grayscale values; F z Represents the central gray value of the global graph of material clusters; F 01 and F 02 Represents the first grayscale threshold and the second grayscale threshold Using the contrast adjustment coefficient to adjust the contrast of the material cluster global map, obtain the material cluster global map after contrast adjustment, and replace the original unadjusted material cluster global map; Among them, the adjusted contrast value is obtained by the following formula: Among them, D t represents the contrast value after adjustment; D0 represents the contrast value before adjustment; S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; k represents the total number of pixels contained in the global map of the material cluster.

4. The material detection and sorting system based on machine vision according to claim 1, characterized in that: The second machine vision recognition module is used to perform second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; The global image of the sorting area is identified to obtain the material type distribution characteristic information in the sorting executable area, including: Synchronously performing RGB machine vision acquisition and near-infrared spectrum vision acquisition on the sorting executable area to obtain an RGB global image and a near-infrared spectrum global image of the sorting executable area; Perform material contour recognition on the RGB global image to obtain the position information and shape and size information of all materials in the sorting executable area; perform near-infrared reflectivity recognition on the near-infrared spectral global image to obtain the material type information of all materials in the sorting executable area; based on the position information, the shape and size information and the material type information, generate the shape and size, material type and position mapping relationship information of the materials in the sorting executable area, and use this as the material type distribution feature information in the sorting executable area; The material sorting grabbing path determination module is used to determine the material sorting grabbing path of the sorting executable area based on the material type distribution characteristic information, including: Based on the material type distribution characteristic information, all materials in the sorting executable area are divided into several material clusters; wherein all materials under each material cluster can be sorted and grasped with the same grasping action; based on the location information of all materials under each material cluster, the sorting and grasping path of all materials under each material cluster is determined.

5. The material detection and sorting system based on machine vision according to claim 1, characterized in that: The third machine vision recognition module is used to perform a third machine vision acquisition on the material sorting operation in the sorting executable area based on the material sorting grabbing path to obtain a dynamic image of the material sorting operation, including: Based on the material sorting grabbing path, dynamically track and visually collect the material sorting operation in the sorting executable area to obtain a dynamic image of the material sorting operation; The material sorting action adjustment module is used to adjust the material sorting action state based on the material sorting operation dynamic image, including: The dynamic image of the material sorting operation is analyzed to determine the relative distance and relative orientation information between the sorting and grabbing robot and the materials to be sorted and grabbed; based on the relative distance and relative orientation information, the moving speed and sorting and grabbing action posture of the sorting and grabbing robot relative to the materials to be sorted and grabbed are adjusted.

6. A material detection and sorting method based on machine vision, characterized in that: include: Performing a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; performing material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of materials within the material cluster; Based on the spatial distribution characteristic information of the materials, locate all sorting executable areas within the material cluster; Performing a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; identifying the global image of the sorting area to obtain material type distribution characteristic information in the sorting executable area; and determining a material sorting grabbing path in the sorting executable area based on the material type distribution characteristic information; Based on the material sorting grabbing path, the material sorting operation in the sorting executable area is collected by a third machine vision to obtain a dynamic image of the material sorting operation; based on the dynamic image of the material sorting operation, the material sorting action state is adjusted.

7. The material detection and sorting method based on machine vision as claimed in claim 6, characterized in that: Performing a first machine vision acquisition on the material cluster to obtain a global image of the material cluster; performing material contour recognition on the global image of the material cluster to obtain spatial distribution feature information of materials within the material cluster; Based on the spatial distribution characteristic information of the material, all sorting executable areas in the material cluster are located, including: Based on the spatial boundary position of the material cluster, the plane area where the material cluster is located is divided into a number of visual acquisition sub-areas with regular shapes; based on the boundary position of each visual acquisition sub-area, all visual acquisition sub-areas are sequentially subjected to machine vision acquisition to obtain a number of material cluster sub-images, and all material cluster sub-images are spliced ​​into a global image of the material cluster; Perform pixel edge sharpening and pixel contour recognition processing on the global image of the material cluster to obtain edge contour information of all materials in the global image of the material cluster; based on the edge contour information, obtain the spatial gap size information between any two adjacent materials in the material cluster, and use it as the spatial distribution feature information of the materials in the material cluster; Based on the material spatial distribution characteristic information, the spatial overlapping status of all materials in the material cluster is analyzed to obtain the spatial overlapping area information of all materials in the material cluster, so as to locate all sorting executable areas in the material cluster.

8. The material detection and sorting method based on machine vision as claimed in claim 7, characterized in that: Before performing material contour recognition on the material cluster global image, performing contrast adjustment on the material cluster global image includes: Extracting the grayscale value corresponding to each pixel point contained in the material cluster global map; Compare the grayscale value corresponding to each pixel with a preset first grayscale threshold and a second grayscale threshold; Retrieving the grayscale value corresponding to the pixel point lower than the first grayscale threshold as the first grayscale value; Retrieving the grayscale value corresponding to the pixel point that is higher than the second grayscale threshold as the second grayscale value; Obtaining a contrast adjustment coefficient using the first grayscale value and the second grayscale value; The contrast adjustment coefficient is obtained by the following formula: Wherein, S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; F 02i represents the gray value of the i-th second gray value; F 01i represents the gray value of the i-th first gray value; F 02z represents the grayscale value middle value of m second grayscale values; F 01z represents the middle value of the grayscale value of n first grayscale values; F z Represents the central gray value of the global graph of material clusters; F 01 and F 02 Represents the first grayscale threshold and the second grayscale threshold Using the contrast adjustment coefficient to adjust the contrast of the material cluster global map, obtain the material cluster global map after contrast adjustment, and replace the original unadjusted material cluster global map; Among them, the adjusted contrast value is obtained by the following formula: Among them, D t represents the contrast value after adjustment; D0 represents the contrast value before adjustment; S represents the contrast adjustment coefficient; m represents the number of second grayscale values; n represents the number of first grayscale values; k represents the total number of pixels contained in the global map of the material cluster.

9. The material detection and sorting method based on machine vision as claimed in claim 6, characterized in that: Performing a second machine vision acquisition on the sorting executable area to obtain a global image of the sorting area; identifying the global image of the sorting area to obtain material type distribution feature information in the sorting executable area; Determining a material sorting path in the sorting executable area based on the material type distribution characteristic information includes: Synchronously performing RGB machine vision acquisition and near-infrared spectrum vision acquisition on the sorting executable area to obtain an RGB global image and a near-infrared spectrum global image of the sorting executable area; Perform material contour recognition on the RGB global image to obtain the position information and shape and size information of all materials in the sorting executable area; perform near-infrared reflectivity recognition on the near-infrared spectral global image to obtain the material type information of all materials in the sorting executable area; based on the position information, the shape and size information and the material type information, generate the shape and size, material type and position mapping relationship information of the materials in the sorting executable area, and use this as the material type distribution feature information in the sorting executable area; Based on the material type distribution characteristic information, all materials in the sorting executable area are divided into several material clusters; wherein all materials under each material cluster can be sorted and grasped with the same grasping action; based on the location information of all materials under each material cluster, the sorting and grasping path of all materials under each material cluster is determined.

10. The material detection and sorting method based on machine vision according to claim 6, characterized in that: Based on the material sorting grabbing path, a third machine vision acquisition is performed on the material sorting operation in the sorting executable area to obtain a dynamic image of the material sorting operation; Based on the material sorting operation dynamic image, adjusting the material sorting action state includes: Based on the material sorting grabbing path, dynamically track and visually collect the material sorting operation in the sorting executable area to obtain a dynamic image of the material sorting operation; The dynamic image of the material sorting operation is analyzed to determine the relative distance and relative orientation information between the sorting and grabbing robot and the materials to be sorted and grabbed; based on the relative distance and relative orientation information, the moving speed and sorting and grabbing action posture of the sorting and grabbing robot relative to the materials to be sorted and grabbed are adjusted.

Images