Full-view-angle recognition structure and method for color sorter

Abstract

The invention discloses a full-view recognition structure and method for a color sorter, and the method comprises the steps: respectively arranging three groups of ultra-depth-of-field cameras around a point A, and carrying out the front-side, right-upper and rear-side imaging of a material at the point A; a group of super depth-of-field cameras are arranged around the point B, and the point B carries out right lower imaging on the material; and then panoramic image synthesis is carried out based on imaging right above, front and back sides and right below the material, and the quality of the material is judged based on the panoramic image. The complete 360-degree surface image of the material is synthesized by the images of the four groups of cameras through a space-time correlation and panoramic fusion algorithm. According to the method, the detection blind area caused by shadow, shielding and visual angle limitation is fundamentally eliminated, so that various defects such as surface scars, mildew and bud eyes cannot be hidden, the sorting precision and the rejection rate of unqualified materials are greatly improved, and the quality consistency of final products is effectively guaranteed.

Description

Technical Field

[0001] This invention relates to the field of color sorter material identification technology, and in particular to a color sorter full-view identification structure and method. Background Technology

[0002] In the field of agricultural product processing and material sorting, automated sorting of large, elliptical materials (such as potatoes and carrots) is a key step in improving production efficiency and ensuring product quality. As industries such as food processing, warehousing, and logistics increasingly demand higher precision and efficiency in material sorting, automated sorting equipment has gradually replaced manual sorting and become the mainstream application solution.

[0003] However, current conventional elliptical large material sorting equipment faces significant technical bottlenecks in actual operation, namely the blind spot problem caused by shadow interference. Due to the irregular curved surface structure of elliptical large materials, shadows easily form in the contact area between the material and the conveyor (such as a belt) and in the transition area of ​​the material's curved surface under the single-view or limited-view recognition mode of traditional sorting equipment. This prevents the recognition lens from effectively capturing the sides, back, and bottom areas of the material. This blind spot directly leads to incomplete detection of key indicators such as surface defects (such as scratches, spots, and damage), size specifications, and morphological integrity of the material. Consequently, the rejection rate of unqualified materials is low, and a large amount of material that does not meet quality requirements is mixed into qualified products. This fails to meet the downstream manufacturers' requirements for consistent material quality and also increases the cost of subsequent manual re-inspection, reducing overall production efficiency.

[0004] Therefore, it is necessary to develop a technical solution that can overcome the limitations of shadow interference and achieve full-view, blind-angle-free recognition of large elliptical materials within a reasonable cost range. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide a full-view recognition structure and method for color sorters, solving the problem of blind spots caused by shadow interference in existing elliptical large material sorting equipment, and achieving full-view recognition of elliptical large materials without blind spots.

[0006] The objective of this invention can be achieved through the following technical solution: a color sorter full-view recognition structure, comprising:

[0007] Three sets of ultra-depth-of-field cameras 1, 2 and 3 are set up above point A, and above front and above rear respectively to image the material at point A from the front, directly above and rear sides.

[0008] A set of super depth-of-field cameras 4 are set below and behind point B on the material throwing path to image the material directly below point B.

[0009] A panoramic image is synthesized based on imaging from directly above, front and back, and directly below the material, and the quality of the material is judged based on the panoramic image.

[0010] As a further embodiment of the present invention, the imaging distances of the materials by the ultra-depth-of-field cameras 1, 2, 3 and 4 are the same, wherein the ultra-depth-of-field camera 2 uses a reflected light path to image the material at point A.

[0011] As a further embodiment of the present invention, points A and B are provided with a super depth-of-field camera correction device.

[0012] As a further embodiment of the present invention, supplementary lights are respectively provided between point A and ultra-depth-of-field cameras 1, 2 and 3, and supplementary lights are provided between point B and ultra-depth-of-field camera 4.

[0013] As a further embodiment of the present invention, the material conveyor belt is provided with positioning ribs.

[0014] A color sorter full-view recognition method, the method comprising the following steps:

[0015] S1. Based on the stable material conveying of the conveyor belt;

[0016] S2. When the material is transported to point A, images are captured from the top and front and back sides of the material using ultra-depth-of-field cameras 1, 2 and 3 respectively.

[0017] S3. When the material is thrown to point B, an image is captured directly below the material using the ultra-depth-of-field camera 4.

[0018] S4. Based on imaging from directly above, front and back sides, and directly below the material, perform panoramic image synthesis;

[0019] S5. Based on panoramic images, identify the quality of materials, determine the quality of materials, and remove inferior materials.

[0020] As a further embodiment of the present invention, the panoramic image synthesis in step S4 includes the following steps:

[0021] S41. After standardizing the material images, extract key feature points from each image.

[0022] S42. Image the same material at points A and B;

[0023] S43. Imaging the material from the top and front and back sides is performed for viewpoint registration. Key feature points of the overlapping area are matched and determined. Based on the spatial transformation relationship of the three-view imaging, the images are stitched together to form an upper fused view.

[0024] S44. Register the upper fused view with the image directly below the material, match and determine the key feature points of the overlapping area, and stitch them together to form a panoramic fused view.

[0025] S45. Based on key feature points of the panoramic fusion view, material quality identification is performed.

[0026] As a further embodiment of the present invention, the S42 association of the same material imaging at points A and B includes: stamping the same timestamp T1 and position code ID1 on the material imaging at point A, and stamping the same timestamp T2 and position code ID2 on the material imaging at point B; establishing a material motion model based on timestamps T1 and T2, and position codes ID1 and ID2, and associating the same material imaging at points A and B based on the motion model.

[0027] As a further embodiment of the present invention, the association of the same material imaging at points A and B further includes: based on an image analysis neural network, identifying the imaging edge features on the front and back sides of the material and the imaging edge features directly below the material, performing feature matching, and associating the same material imaging at points A and B based on the feature matching results.

[0028] The beneficial effects of this invention are:

[0029] 1. This invention utilizes three sets of ultra-depth-of-field cameras positioned at point A above to collaboratively image the material, covering the top and both sides. Simultaneously, a third camera is positioned at point B along the material's trajectory below, specifically to capture the approximately 80° bottom edge of the material—a traditional blind spot for identification. The images from these four cameras are then combined using a spatiotemporal correlation and panoramic fusion algorithm to create a complete 360° surface image of the material. This solution fundamentally eliminates detection blind spots caused by shadows, occlusions, and viewing angle limitations, making surface scars, mold, buds, and other defects readily apparent. This significantly improves the sorting accuracy and rejection rate of substandard materials, effectively ensuring the consistency of the final product's quality.

[0030] 2. This invention utilizes a super-depth-of-field camera with the same shooting distance, supplemented by precise lighting and correction devices, ensuring consistency in scale, color, and sharpness across multiple viewpoints, thus laying a solid foundation for image stitching and fusion. When associating images of points A and B, a dual verification mechanism based on a spatiotemporal motion model and neural network image features can be employed. This hardware-software combined approach ensures that even if materials roll or shift during movement, images taken at different times can be accurately attributed to the same material, guaranteeing the high reliability of panoramic synthesis.

[0031] 3. This invention fully considers the actual needs of industrial applications, and its structural design combines innovation and practicality. The application of a reflective light path saves installation space; the conveyor belt positioning ribs effectively prevent material rolling and improve imaging quality; the independent camera calibration device facilitates daily maintenance and calibration, ensuring the long-term stability of the system. The entire system achieves full automation from image acquisition, transmission, processing to sorting execution, meeting the production line's requirements for high-efficiency, high-cycle continuous operation, and has high engineering application value. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the full-view recognition structure of the color sorter of the present invention;

[0033] Figure 2 This is a flowchart illustrating the color sorter's full-view recognition method of the present invention.

[0034] 1. Ultra-depth-of-field camera 1; 2. Ultra-depth-of-field camera 2; 3. Ultra-depth-of-field camera 3; 4. Ultra-depth-of-field camera 4; 5. Fill light. Detailed Implementation

[0035] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0036] Example 1:

[0037] like Figure 1 As shown, this embodiment discloses a full-view recognition structure for a color sorter. Three sets of ultra-depth-of-field cameras 1, 2 and 3 are respectively set above point A of the color sorter, and above the front and above the rear sides to image the material at point A from the front, directly above and rear sides.

[0038] Three sets of ultra-depth-of-field cameras capture image information of the material at point A from different angles. The ultra-depth-of-field camera 2 (2) directly above can clearly capture the features of the material at its top, while the ultra-depth-of-field camera (1) on the front side above and the ultra-depth-of-field camera (3) on the back side above can capture the features of the material on both the front and back sides. The ultra-depth-of-field cameras have high resolution and accurate color recognition capabilities, and can accurately record the color, shape, texture and other features of the material.

[0039] A set of ultra-depth-of-field cameras 4 (4) is also set below and behind point B on the material throwing path to image the material directly below. When the material is at point A, ultra-depth-of-field cameras 1 and 3 are responsible for identifying the left and right sides of the material, and ultra-depth-of-field camera 2 (2) identifies the top. At this time, only the bottom 80° range of the material is not identified. When the material moves to point B, ultra-depth-of-field camera 4 (4) below and behind will image the material to identify the remaining 80° blind spot.

[0040] Each ultra-depth-of-field camera is equipped with an advanced image transmission system, which can image the material from directly above, front and back, sides and directly below, and can quickly and stably transmit the captured image data to the color sorter's processing system for subsequent processing.

[0041] Furthermore, such as Figure 1 As shown, the imaging distances of the super depth-of-field cameras 1, 2, 3 and 4 are the same. Under the same signal trigger, super depth-of-field cameras 1, 2 and 3 can image simultaneously. When super depth-of-field camera 4 (4) is associated with the imaging of super depth-of-field cameras 1, 2 and 3, the influence of imaging distance does not need to be considered. This makes the integration of image data in the processing system more efficient and accurate.

[0042] At the instant the same signal is triggered, ultra-depth-of-field cameras 1, 2, and 3 simultaneously capture images of the material at different locations on point A. Because the shooting distances are consistent, the image information acquired by these cameras exhibits a high degree of consistency in size, proportions, and perspective. This eliminates the need for complex distance corrections and scale adjustments after the processing system receives this image data, facilitating image stitching and fusion.

[0043] When the super depth-of-field camera 4 (4) images the remaining 80° blind spot of the material at point B, since it does not need to consider the influence of imaging distance when associated with the imaging of super depth-of-field cameras 1, 2 and 3, it is convenient to associate the image taken at point B with the image taken at point A. At the same time, the same shooting distance also ensures that the clarity and quality of the images from each camera are highly consistent, reducing problems such as image blurring and distortion caused by differences in shooting distance.

[0044] Furthermore, the ultra-depth-of-field camera 2 (2) uses a reflected light path to image the material at point A. Increasing the reflected light path can make the imaging distances of ultra-depth-of-field cameras 1, 2 and 3 the same, saving layout space.

[0045] Furthermore, points A and B are equipped with ultra-depth-of-field camera calibration devices. The calibrated ultra-depth-of-field camera can better image the material at point A or B, improving imaging accuracy and stability. The calibration device at point A is temporary; it is installed when the equipment is stopped for ultra-depth-of-field camera calibration, and removed when the equipment is restarted, without affecting the conveyor belt's operation. The calibration device at point B is permanent and can be implemented using a reflector or a stainless steel reflector plate.

[0046] The camera's various parameters are precisely calibrated using a calibration device, such as lens distortion correction and color balance adjustment. Lens distortion correction effectively prevents image distortion, ensuring accurate representation of the material's true shape. Color balance adjustment guarantees more realistic and accurate colors in the captured image, preventing color deviations from affecting subsequent identification of material color characteristics.

[0047] The calibration device also optimizes the focusing accuracy of the ultra-depth-of-field camera. In the actual working environment of a color sorter, the position and state of the materials may change, which requires the camera to focus quickly and accurately.

[0048] Furthermore, supplementary lights (5) are respectively installed between point A and ultra-depth-of-field cameras 1, 2 and 3, and supplementary lights (5) are installed between point B and ultra-depth-of-field camera 4 (4). By setting supplementary lights (5), shadows during material imaging can be eliminated, and the imaging brightness and quality can be improved, which is convenient for subsequent analysis using the processing system.

[0049] The uniformity of the fill light (5) is carefully designed to ensure that all parts of the material receive stable and consistent illumination, avoiding imaging differences caused by uneven illumination. Different fill lights (5) adopt different illumination intensities and angles according to their positions and the shooting requirements of the corresponding ultra-depth-of-field cameras. The fill lights (5) corresponding to ultra-depth-of-field cameras 1, 2 and 3 at point A will adjust the illumination angle according to the shooting range of these cameras and the movement trajectory of the material, so that the light can cover the surface of the material to the greatest extent and reduce imaging dead angles.

[0050] Furthermore, the conveyor belt is equipped with positioning ribs, which can be used to fix the position of the material and prevent it from rolling along the conveyor belt, thus improving imaging quality.

[0051] Example 2:

[0052] Based on the color sorter's full-view recognition structure of Embodiment 1, this embodiment discloses a color sorter's full-view recognition method, the method comprising the following steps:

[0053] S1. Based on the stable conveyor belt transport of materials.

[0054] By setting ribs on the conveyor belt and adjusting parameters such as conveyor belt speed and throwing angle, the movement and position of materials on the conveyor belt can be precisely controlled, making it easier for the ultra-depth-of-field camera to capture images of materials from various angles more stably and accurately.

[0055] S2. When the material is transported to point A, images are captured from the top and front and back sides of the material using ultra-depth-of-field cameras 1, 2 and 3 respectively.

[0056] Ultra-depth cameras 1, 2, and 3 utilize their high resolution and ultra-depth characteristics to quickly and accurately capture image information of the material from directly above and from both sides. Combined with the illumination intensity and angle settings of the supplementary light (5), the light can be evenly illuminating the surface of the material, making the images captured by the cameras clear and with high contrast, fully showcasing the features and details of the material surface.

[0057] Meanwhile, the system performs preliminary preprocessing on the images captured by the camera, including noise removal, brightness and contrast adjustment, to improve image quality. This preprocessed image data is then transmitted to the processing system in real time, preparing it for subsequent analysis and recognition.

[0058] S3. When the material is thrown to point B, the image is captured directly below the material by the ultra-depth camera 4 (4).

[0059] The super depth-of-field camera 4 (4) also quickly and accurately acquires images directly below the material and performs preprocessing thanks to its excellent high resolution and super depth-of-field performance.

[0060] S4. Perform panoramic image synthesis based on imaging from directly above, front and back sides, and directly below the material.

[0061] First, the material images are standardized, and key feature points are extracted from each image. Feature extraction can be performed based on image analysis neural networks (such as CNN networks) to extract key feature points of the material, such as spots, edges, corners, textures, etc. These key feature points are used as feature identifiers of the material.

[0062] Then, correlate the images of the same material at points A and B. Correlating the images of the same material at points A and B is one of the key points in panoramic image synthesis. There are two methods for correlating the images of the same material:

[0063] One approach is to perform spatiotemporal encoding on the images of points A and B. When the material passes through point A, the photoelectric sensor is triggered. Ultra-depth cameras 1, 2, and 3 are exposed and captured almost simultaneously, obtaining images of the front, top, and rear sides of the material, respectively. The system assigns the same timestamp T1 and the material's position code ID1 on the conveyor belt to these images. When the material passes through point B, the electrical sensor is triggered, and ultra-depth camera 4 (4) captures an image of the material directly below. The system assigns a timestamp T2 and the corresponding material position code ID2 to this image. At this point, the system has obtained four local images of the same material at different times T1 and T2. Based on the timestamps T1 and T2, and the position codes ID1 and ID2, the system, combined with parameters such as conveyor belt speed and throwing angle, establishes a material motion model. Based on the motion model, the images of the same material at points A and B are linked together.

[0064] Another approach is based on image analysis neural networks (such as CNNs), which identify and match the edge features of the images on the front and back sides of the material with the edge features of the image directly below it. The system compares the edge features of the front and back sides and the image directly below one by one, calculating the similarity between them. When the similarity reaches a set threshold, these images are determined to belong to the same material. To improve the accuracy of the matching, other features such as the material's texture and color can be combined for a comprehensive judgment, making the feature matching more precise. Based on the feature matching results, images of the same material at points A and B are associated.

[0065] The two methods described above can be used in combination to mutually correct each other, thereby improving the accuracy and reliability of imaging correlation for the same material. In real-world industrial scenarios, factors such as the material's motion state and lighting conditions are complex and variable, and a single correlation method may have certain limitations. Combining these two methods can fully leverage their respective advantages. The method based on the material motion model can quickly establish preliminary correlations from the macroscopic laws of material motion; while the feature matching method based on image analysis neural networks can perform fine verification and correction of the correlation results at the microscopic level of image features.

[0066] Then, images are taken from the top and front and back sides of the material for viewpoint registration. Key feature points in the overlapping areas are matched and determined. Based on the spatial transformation relationship of the three-view imaging, the images are stitched together to form the upper fused view.

[0067] The system registers the three images at time T1. Since these three images were taken simultaneously and have adjacent viewpoints, their features have a large number of overlapping areas. By matching the feature points of these overlapping areas, the spatial transformation relationship between the three viewpoints can be accurately calculated, and they can be initially stitched together into an upper fused view of about 280°.

[0068] Then, the upper fused view is registered with the image directly below the material, the key feature points of the overlapping area are matched and determined, and the images are stitched together to form a panoramic fused view.

[0069] The upper fused view is registered with the material image at time T2. In the overlapping area of ​​the images, a fusion algorithm is used to smooth the transition, eliminating the seams, brightness differences and ghosting, and generating a visually seamless 360° panoramic fused view.

[0070] Finally, material quality identification is performed based on key feature points of the panoramic fusion view.

[0071] S5. Based on panoramic images, identify the quality of materials, determine the quality of materials, and remove inferior materials.

[0072] Based on key feature points of the panoramic fusion view, material quality identification can be performed. It can be based on image recognition algorithms combined with panoramic fusion view to comprehensively analyze various indicators of materials such as color, shape, size, scars, buds, and mold. According to preset standards, the material grade can be quickly determined. The system generates instructions containing material ID, category and location information, which are sent to the rejection structure to remove unqualified products at the accurate location and time.

[0073] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0074] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

Claims

1. A color sorter's all-view recognition structure, characterized in that, include: Three sets of ultra-depth-of-field cameras 1, 2 and 3 are set up above point A, and above front and above rear respectively to image the material at point A from the front, directly above and rear sides. A set of super depth-of-field cameras 4 are set below and behind point B on the material throwing path to image the material directly below point B. A panoramic image is synthesized based on imaging from directly above, front and back, and directly below the material, and the quality of the material is judged based on the panoramic image.

2. The structure according to claim 1, characterized in that, The ultra-depth-of-field cameras 1, 2, 3 and 4 image the material at the same shooting distance. Among them, ultra-depth-of-field camera 2 uses a reflected light path to image the material at point A.

3. The structure according to claim 1, characterized in that, Points A and B are equipped with ultra-depth-of-field camera correction devices.

4. The structure according to claim 1, characterized in that, A supplementary light is installed between point A and ultra-depth-of-field cameras 1, 2 and 3, and a supplementary light is installed between point B and ultra-depth-of-field camera 4.

5. The structure according to claim 1, characterized in that, The material conveyor belt is equipped with positioning ribs.

6. A color sorter full-view recognition method, applied to the structure described in any one of claims 1 to 5, characterized in that, The method includes the following steps: S1. Based on the stable material conveying of the conveyor belt; S2. When the material is transported to point A, images are captured from the top and front and back sides of the material using ultra-depth-of-field cameras 1, 2 and 3 respectively. S3. When the material is thrown to point B, an image is captured directly below the material using the ultra-depth-of-field camera 4. S4. Based on imaging from directly above, front and back sides, and directly below the material, perform panoramic image synthesis; S5. Based on panoramic images, identify the quality of materials, determine the quality of materials, and remove inferior materials.

7. The method according to claim 6, characterized in that, The panoramic image synthesis in step S4 includes the following steps: S41. After standardizing the material images, extract key feature points from each image. S42. Image the same material at points A and B; S43. Imaging the material from the top and front and back sides is performed for viewpoint registration. Key feature points of the overlapping area are matched and determined. Based on the spatial transformation relationship of the three-view imaging, the images are stitched together to form an upper fused view. S44. Register the upper fused view with the image directly below the material, match and determine the key feature points of the overlapping area, and stitch them together to form a panoramic fused view. S45. Based on key feature points of the panoramic fusion view, material quality identification is performed.

8. The method according to claim 7, characterized in that, The S42 association of imaging the same material at points A and B includes: Image the material at point A and assign the same timestamp T1 and location code ID1. Image the material at point B and assign the same timestamp T2 and location code ID2. Based on timestamps T1 and T2, and location codes ID1 and ID2, establish a material motion model. Based on the motion model, associate the images of the same material at points A and B.

9. The method according to claim 7, characterized in that, The imaging of the same material at points A and B also includes: Based on image analysis neural networks, feature matching is performed on the imaging edge features on the front and back sides of the material and the imaging edge features directly below the material. Based on the feature matching results, the imaging of the same material at points A and B is associated.

Images