Photoelectric color selection method and device for waxy corn ears based on three-eye vision

Abstract

The application discloses a kind of based on three vision's waxy corn ear photoelectric color selection method and device, three industrial cameras sequentially collect the original image of corn ear, the original image is preprocessed to obtain preprocessed image, the feature vector of each preprocessed image is extracted, whether corn ear is detected to exist mildew or moth-eaten defect;Three preprocessed images corresponding to defect-free corn ear are cylindrical development, feature matching, translation registration and fusion processing, to obtain panoramic image covering full surface, then comprehensive grading is carried out.The application collects the full surface image of waxy corn ear under the premise of not extra occupying production time, and then realizes the rapid detection and sorting of the multi-dimensional parameters of waxy corn ear, effectively solves the problem of uneven product quality detection classification caused by subjective factors such as uneven subjective experience level of workers, and the lag problem that traditional mechanical equipment can only identify single maturity parameter or size parameter.

Description

Technical Field

[0001] This invention belongs to the field of corn ear quality detection technology, specifically relating to a photoelectric color sorting method and device for waxy corn ears based on trinocular vision. Background Technology

[0002] As an important specialty economic crop in my country, waxy corn satisfies consumers' demand for healthy and novel agricultural products due to its nutritional value and unique purple-white appearance. In industrial processing, waxy corn typically undergoes harvesting, hulling and silk removal, color sorting, washing, vacuum packaging, and sterilization before being sold. The color sorting stage is crucial; first, defective ears are removed, then maturity is determined by the proportion of the distinctive purple color on the surface, and finally, multiple appearance indicators are considered for sorting. This stage is key to determining the added value of waxy corn.

[0003] The utility model disclosed in CN220716811U is a color sorting device for the maturity of waxy corn ears. It collects images of corn ears within a transparent pipe and then classifies them into mature and substandard categories based on whether the purple percentage on the ear surface is greater than 40%. While this method significantly improves efficiency compared to traditional manual sorting, it has the following drawbacks: First, it is functionally limited, only able to perform preliminary sorting based on the percentage of purple on the corn ear surface, unable to remove defective ears, and unable to detect basic physical appearance qualities such as ear length and thickness, making it difficult to meet the processing requirements of grading fresh corn ears by quality after washing and then vacuum packaging. Second, the conveying method is a free-fall pipe structure, which has poor stability and is prone to clogging in the pipe when processing corn ears of different sizes, making it difficult to guarantee the accuracy of image acquisition.

[0004] In existing technologies, there are two main methods for color sorting waxy corn ears: one relies on manual subjective evaluation, and the other is automated color sorting based on machine vision. The first method is not only heavily influenced by human subjective experience, but also suffers from time-consuming and inaccurate processing when grading large batches of corn ears. The second method often uses a monocular camera, which takes a long time for the corn ears to pass through the image acquisition area, consuming additional production time and significantly impacting production line efficiency. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a trinocular vision-based photoelectric color sorting method and device for glutinous corn ears that is efficient, stable, and can achieve full surface detection without additional production cycle time.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A photoelectric color sorting method for waxy corn ears based on trinocular vision includes the following steps: Transmission and transportation steps: The corn ears are transported through the transmission module, and the corn ears rotate around their own central long axis during the transportation process; Image acquisition steps: Three industrial cameras are arranged at equal intervals along the conveying direction. When the corn ears arrive at the acquisition station directly below the three industrial cameras in sequence, the three industrial cameras acquire the original images of the corn ears in sequence. The conveying speed of the drive wheel and the arrangement spacing of adjacent industrial cameras are adjusted so that the roller drives the corn ears to rotate 120° around their own central long axis within the acquisition interval between two adjacent industrial cameras. The three original images cover the entire surface of the corn ears, providing high-precision original data for subsequent statistical analysis of the proportion of purple pixels to classify maturity.

[0007] Preprocessing steps: The original image is preprocessed to obtain a preprocessed image; Defect detection steps: Extract the feature vector of each preprocessed image, input the feature vector into a pre-trained binary classification support vector machine model (SVM), and detect whether there are mold or insect defects in the corn ears; if at least one preprocessed image detects a defect, the corn ear is determined to be a defective product and the defect is removed. The feature vector is formed by concatenating the HSV color histogram of the preprocessed image with the LBP rotation-invariant texture features. The binary classification support vector machine model adopts the radial basis kernel function and is trained offline with images of healthy corn ears as positive samples and images of ears labeled as moldy or insect-infested as negative samples.

[0008] SVM achieves accurate data classification by constructing an optimal separating hyperplane in a high-dimensional feature space, maximizing the margin between samples of different classes.

[0009] Panoramic stitching steps: Perform cylindrical unfolding, feature matching, translation registration and fusion processing on three preprocessed images corresponding to defect-free corn ears to obtain a panoramic image covering the entire surface; Color sorting and grading steps: The maturity level of corn ears is determined based on the proportion of purple pixels on the surface of the corn ears, and the corn ears are comprehensively graded in combination with ear length, ear thickness, fullness and uniformity.

[0010] Furthermore, in the transmission and transportation step, the transmission module includes a drive belt, multiple rollers, and a rack; the multiple rollers are fixed at equal intervals on the drive belt along the conveying direction, and gears are fitted on both sides of the shaft end of each roller. The rack is horizontally fixed on the frame along the conveying direction and meshes with the gears. When the drive belt drives the rollers forward, the meshing of the gears and racks drives the rollers to rotate synchronously around their own axes. The corn ears supported in the V-shaped gap between adjacent rollers move forward and rotate synchronously with the rollers. This structure is simple and reliable, and image acquisition is completed synchronously during the normal material conveying process without occupying additional production cycles, greatly improving the production line operating efficiency.

[0011] In the image acquisition step, the PLC integrated control module detects the arrival status of corn ears through the through-beam photoelectric sensor on the feeding side, assigns a unique material identifier to the corn ear and records the start time, calculates the triggering sequence of the three cameras in combination with the preset conveying speed, and binds the three acquired original images with the same material identifier to achieve accurate matching between the image and the single ear.

[0012] Furthermore, the preprocessing step specifically includes: Brightness normalization: Brightness correction is performed on the original image to obtain a brightness normalized image; Denoising: The three channels of the brightness-normalized image are filtered separately using a 5×5 Gaussian kernel to obtain the denoised image; Contrast enhancement: The denoised image is converted to the HSV color space, and the contrast of the luminance V channel is enhanced using the CLAHE algorithm to obtain an enhanced image; Binarization: The enhanced image is binarized using a fixed threshold method to obtain a binarized image; Morphological processing: The binarized image is subjected to erosion and dilation operations in sequence to remove isolated noise and fill tiny holes, resulting in a morphologically optimized image; Target segmentation: The main region of the corn ear in the morphologically optimized image is extracted by connected component analysis to obtain a preprocessed image.

[0013] Further, in the panoramic stitching step: cylindrical unfolding, the preprocessed image is rolled out along the length of the corn ear to obtain an unfolded image; feature matching, SIFT feature points and 128-dimensional descriptors are extracted from the overlapping area of ​​two adjacent unfolded images, and matching is performed in the ear thickness direction to obtain a set of constrained matching points; translation registration, the optimal translation amount of adjacent unfolded images is calculated based on the set of constrained matching points, and coordinate transformation is performed on the next unfolded image to align the next unfolded image with the previous unfolded image in space to obtain a registered image; fusion, a linear gradient weight map based on pixel distance is constructed to smoothly transition the stitching gaps to obtain a panoramic image.

[0014] Furthermore, in the color sorting and grading step, the panoramic image is converted to the HSV color space, the hue H channel is extracted, and the purple and white segmentation threshold is calculated using the OTSU algorithm. Based on the proportion of purple pixels, the maturity is divided into three categories: immature, mature, and overripe. At the same time, the ear length, ear diameter, fullness, and uniformity of the corn ears in the panoramic image are extracted. The immature and overripe corn ears are directly classified into the second level. The corn ears rated as mature are further classified into special grade or first grade based on the ear length, ear diameter, fullness, and uniformity.

[0015] The fullness is the ratio of the area of ​​the main body of the corn ear to the area of ​​the smallest circumscribed rectangle. The uniformity is calculated as follows: after segmenting the sticky kernels, the centroid coordinates of each kernel are extracted. After clustering by row, the centroids of each row of kernels are fitted with least squares straight lines. The standard deviation σ and mean μ of the root mean square error (RMSE) of each row are calculated. The uniformity is evaluated by using CV = σ / μ. The smaller the CV, the more uniform the ear rows are.

[0016] This invention prioritizes hygienic and healthy processing as the primary standard in the fresh food industry. It first eliminates defective waxy corn ears, ensuring the necessary conditions for subsequent color sorting and grading. Furthermore, it uses maturity as the primary grading indicator, combined with size, plumpness, and uniformity for comprehensive grading. This solves the problems of high error rates in manual sorting and limited coverage of corn quality testing due to the single standard of traditional mechanical sorting, thus meeting market demands for product consistency and high-quality grading.

[0017] A photoelectric color sorting device for waxy corn ears based on trinocular vision is also provided to implement the above-described method, comprising: The transmission module is used to transport corn ears and drive the corn ears to rotate around their own central long axis; The tri-lens acquisition module, fixedly mounted above the transmission module, includes three industrial cameras arranged at equal intervals along the conveying direction to acquire raw images of corn ears. It overcomes the inherent defects of monocular and binocular vision systems in terms of circumferential blind spots, providing complete raw data for subsequent maturity determination and improving the accuracy and reliability of subsequent quality grading.

[0018] An industrial control computer, which is connected to the three-eye acquisition module, is configured to perform the preprocessing, defect detection, panoramic stitching and color sorting steps in the above-described method, and generate a grade control signal carrying material identification. The PLC integrated control module is connected to the three-eye acquisition module and the industrial control electromechanical system respectively. It is used to detect the position status of corn ears, trigger camera acquisition, receive level control signals and calculate the push rod action delay. The graded execution module, located downstream of the transmission module, includes three sets of push rod assemblies and four collection boxes. The PLC integrated control module drives the push rods at the corresponding workstations to push the corn ears into the corresponding collection boxes according to the graded control signals.

[0019] The PLC integrated control module and the transmission module are linked and calibrated to ensure that the corn ears complete a 120° rotation within the acquisition interval of two adjacent industrial cameras. After the industrial computer completes the defect detection, if it determines that the product is unqualified, it directly sends a rejection signal to the PLC integrated control module, without having to perform subsequent panoramic stitching and grading processes, thus releasing the image storage space of the unqualified product in advance.

[0020] By using three industrial cameras to capture images along the conveying direction for panoramic stitching, this method reduces the amount of image data and computational overhead required for inspection compared to extracting keyframes from the video stream and then stitching the images. It also avoids keyframe selection distortion caused by rotation and geometric drift issues accumulated during multi-frame stitching. Furthermore, by sequentially capturing three images during the corn cob conveying process, the inspection workflow is integrated into the normal conveying cycle, without additional time commitment to the production line, thus increasing production line throughput.

[0021] Furthermore, each push rod assembly includes a push rod and a drive motor. The drive motor is electrically connected to the PLC integrated control module and is used to receive execution instructions from the PLC integrated control module to drive the corresponding push rod to push the corn ears into the corresponding collection box.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention can collect full-surface images of waxy corn ears without taking up additional production time, thereby realizing rapid detection and sorting of multi-dimensional parameters (defects, maturity, ear length, ear diameter, fullness, uniformity) of waxy corn ears. It effectively solves the problem of uneven product quality detection and classification caused by subjective factors such as the uneven level of workers' subjective experience, as well as the lag problem of traditional mechanical equipment that can only identify a single maturity parameter or size parameter, filling the market gap for accurate grading of waxy corn ears. Attached Figure Description

[0023] Figure 1 is a schematic diagram of the physical structure of the photoelectric color sorting device for waxy corn ears based on trinocular vision according to the present invention. Detailed Implementation

[0024] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is used. It is also intended to include plural forms; furthermore, it should be understood that when the terms "comprising" and / or "package" are used in this specification... When "includes", it indicates the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] In this disclosure, terms such as "upper," "lower," "front," and "horizontal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are merely relational terms determined for the convenience of describing the structural relationship of each component or element, and do not specifically refer to any component or element. They should not be construed as limiting this disclosure.

[0027] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "set," "connection," and "linkage" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0028] With the popularization of healthy eating concepts, fresh corn, rich in dietary fiber, anthocyanins, various vitamins, and low in fat, has transformed from a traditional whole grain into a fashionable and healthy food consumed at all times of the day. In particular, glutinous corn, with its unique purple-white appearance and sweet-and-glutinous texture, is highly favored in the high-end market. However, if harvested glutinous corn is processed into vacuum packaging or quick-freezing at uneven maturity, it will severely lower the grade of the entire batch, leading to returns and damage to brand reputation.

[0029] To address the aforementioned technical problems, this invention provides a photoelectric color sorting method for waxy corn ears based on trinocular vision, which specifically includes the following steps: Step S1: The corn ears are conveyed through the transmission module, and the corn ears rotate around their own central long axis during the conveying process; Step S2: Image acquisition. Three industrial cameras are arranged at equal intervals along the conveying direction. When the corn ears arrive at the acquisition station directly below the three industrial cameras in sequence, the three industrial cameras acquire the original images of the corn ears in sequence. Adjust the conveying speed of the drive wheel and the arrangement spacing of adjacent industrial cameras so that the roller drives the corn ears to rotate 120° around their own central long axis within the acquisition interval between two adjacent industrial cameras. Three original images cover the entire surface of the corn ears. In this embodiment of the invention, the driving belt conveying speed of the transmission module is: V belt The roller diameter is D roll The number of teeth on the roller shaft end gear is Z The rack module is m The diameter of the corn ear is D corn The industrial camera spacing is L cam ,but Roller rotational angular velocity W roll = (rad / s) Corn ear rotation angular velocity W corn = W roll = (rad / s) Industrial camera interval time Δt = (s) Corn ear rotation angle θ= Δt W corn = = (rad) By adjusting V belt and L cam Make Δt W corn =120°, which ensures that the three original images cover the entire surface of the ear at equal angles around the circumference. For example, take D corn =50mm D roll =40mm Z =51、 m =3、 V belt =0.15m / s, L cam =200mm, after calibration, it can achieve stable 120° rotation interval data acquisition.

[0030] Meanwhile, in the image acquisition step, the PLC integrated control module detects the position of the corn ears through the through-beam photoelectric sensor on the feeding side, assigns a unique material identifier to the corn ear and records the start time t0, and triggers the second and third industrial cameras sequentially at t1=t0+Δt and t2=t0+2Δt. The three original images are bound to the same identifier to achieve precise matching between the image and the corn ear; Δt is the action delay time calculated based on the start time t0 recorded by the material identifier, the conveying speed and the distance between each push rod station and the acquisition station.

[0031] Step S3: Preprocessing, the original image is preprocessed to obtain a preprocessed image.

[0032] The preprocessing includes steps S301-S306.

[0033] Step S301: Brightness normalization. Homomorphic filtering is used to correct the brightness of the original image, separating the illumination component and the reflection component, suppressing the shadow on the ear surface and enhancing the grain texture. In this embodiment, the low-frequency cutoff is set to 0.05, the high-frequency cutoff to 0.4, the illumination component attenuation coefficient to 0.5, and the reflection component enhancement coefficient to 1.5, highlighting the boundary features of the purple and white areas.

[0034] Step S302: Denoising, filtering the three channels of the brightness normalized image to obtain a denoised image; A Gaussian filter of size (5, 5) is applied to the three channels of the brightness-normalized image to obtain a denoised image. Step S303: Enhance contrast. Convert the denoised image to the HSV color space and use the CLAHE algorithm (e.g., clipLimit=2.0, tileGridSize=(8,8)) to enhance the contrast of the luminance V channel to obtain the enhanced image; Step S304: Binarization, the enhanced image is binarized using a fixed threshold method to obtain a binarized image; Since the black background has an extremely low V value in the HSV color space, while the corn kernels have a significantly higher V value, a suitable fixed threshold (such as V=25) is selected to separate the background from the corn foreground, thus obtaining a binarized image.

[0035] Step S305: Morphological processing, the binarized image is subjected to erosion and dilation operations in sequence to remove isolated noise and fill small holes, so as to obtain a morphologically optimized image; At this point, the morphologically optimized image has a smooth and continuous outline edge, fully preserving the overall morphological features of the ear of corn, and can be directly used to extract the main area of ​​the corn ear.

[0036] Step S306: Target segmentation, extracting the main region of the corn ear in the morphologically optimized image through connected component analysis to obtain the preprocessed image; Connectivity analysis was performed using the `cv2.connectedComponentsWithStats` function in OpenCV. The connected component with the largest area was retained as the main region of the corn ear. A binary mask was generated using the pixel positions of this connected component, and a bitwise AND operation was performed with the denoised image to extract a preprocessed image containing only the main region of the corn ear. This target segmentation step can eliminate background interference and provide a clean target region for subsequent feature extraction and classification.

[0037] Step S4: Defect detection, continuously determining whether defects are detected in the corn ears; Defects refer to mold and insect infestation. The specific method for defect detection is to extract the feature vector of each preprocessed image and input the feature vector into a pre-trained binary support vector machine (SVM) model to detect mold and insect infestation.

[0038] If the preprocessed image detects at least one of mold and insect infestation, the output is "Yes". If at least one preprocessed image is detected as "Yes", the corn ear is classified as a substandard corn ear, and the process proceeds to step S5. Otherwise, the process proceeds to step S6.

[0039] The binary support vector machine (SVM) employs an RBF kernel. The input feature vector is constructed by concatenating the HSV color histogram and LBP texture features of the preprocessed image: each of the H, S, and V channels is quantized into 16 bins, resulting in a total of 48-dimensional color features; the LBP operator uses a radius of 1 and 8 neighborhoods to extract 59-dimensional rotation-invariant texture features, for a total of 107-dimensional feature vectors. The model is pre-trained offline: positive samples are images of healthy ears of fruit without mold or insect damage, while negative samples are images of ears of fruit marked with mold or insect damage. The penalty factor C and kernel bandwidth γ are optimized through 5-fold cross-validation. After training, the parameters are fixed for online detection.

[0040] Step S6: Panoramic stitching. Perform cylindrical unfolding, feature matching, translation registration and fusion processing on the three preprocessed images corresponding to the defect-free corn ears to obtain a panoramic image covering the entire surface.

[0041] Panoramic stitching includes the following steps S601-S604.

[0042] Step S601: Cylindrical unfolding, the preprocessed image is projected onto a cylindrical surface along the length of the corn cob to obtain the unfolded image; Because industrial cameras, when acquiring images of corn ears, experience perspective distortion due to lens distortion and the curvature of the ears themselves, the overlapping areas of adjacent images exhibit nonlinear displacement in space. Therefore, by mapping the curved surface to a plane through cylindrical projection, the nonlinear displacement of the overlapping area is simplified to a linear translation in the direction of ear thickness, thereby significantly improving the robustness and computational efficiency of feature matching.

[0043] Step S602: Feature matching, extract SIFT feature points and 128-dimensional descriptors from the overlapping area of ​​two adjacent unfolded images, and perform matching in the ear-thickness direction to obtain a set of constrained matching points; By understanding the geometric relationships between adjacent overlapping regions under different unfolded diagrams, we can constrain the feature matching direction, reduce computational effort, and improve matching accuracy.

[0044] Step S603: Translation registration. Calculate the optimal translation amount of adjacent images based on the constraint matching point set, and perform coordinate transformation on the next unfolded image to align it spatially with the previous image to obtain the registered image. Step S604: Fusion, constructing a linear gradient weight map based on pixel distance, smoothly transitioning the stitching gaps to obtain a panoramic image.

[0045] Step S7: Color sorting and grading. The maturity level of the corn ear is determined by the proportion of purple pixels on the surface of the corn ear, and the corn ear is comprehensively scored and graded by combining the ear length, ear thickness, fullness and uniformity.

[0046] The specific method is as follows: Maturity grading involves converting the initial image to the HSV color space, extracting the hue (H) channel value, and using the OTUS algorithm to calculate the optimal segmentation threshold between purple and white. Based on the proportion of purple pixels, corn ears are divided into three categories: immature, mature, and overripe. In the HSV space, the hue H value corresponding to the purple seed coat of waxy corn ranges from 270° to 330°. The OTUS algorithm is used to adaptively determine the purple-white segmentation threshold within this range, and the proportion of purple pixels in the main area of ​​the ear is statistically analyzed as a quantitative basis for maturity measurement.

[0047] Size detection involves extracting the minimum bounding rectangle of the main corn ear region from the target segmentation step, and calculating the ear length and ear diameter based on the pre-calibrated pixel equivalent K. To obtain the accurate size of the corn ear, the camera needs to be pixel-calibrated, that is, the actual physical size corresponding to each pixel in the image needs to be determined. After calibration, the pixel equivalent K millimeters per pixel is obtained. Therefore, the ear length is equal to the length of the minimum bounding matrix multiplied by K, and the ear diameter is equal to the height of the minimum bounding matrix multiplied by K.

[0048] Fullness detection is performed by calculating the ratio of the area of ​​the main corn cob region obtained in the target segmentation step to the area of ​​the minimum bounding rectangle, and then obtaining the fullness index. Uniformity detection involves segmenting adhered grains, clustering the centroid coordinates of each grain, and then performing least-squares linear fitting. The coefficient of variation (CV) of each row is used to evaluate the uniformity of the entire ear.

[0049] The specific formula for calculating CV is: CV = σ / μ. Here, σ is the standard deviation of the RMSE (root mean square error of the vertical distance from each point to the fitted line of its row) for each row, and μ is the mean of the RMSE for each row. The smaller the CV, the straighter and more uniform the ear rows are.

[0050] The comprehensive scoring standard is as follows: maturity is the primary indicator. Ears with less than 40% purple pixels are considered immature, those between 40% and 70% are considered mature, and those above 70% are considered overripe. Immature and overripe corn ears are directly classified as Grade II and will not participate in further grading. For mature ears, a weighted score is applied based on three factors: size (ear length and diameter), fullness, and uniformity. For size, the score is calculated based on the deviation of ear length and diameter from the baseline values; the smaller the deviation, the higher the score. Fullness score is determined by the fullness index; the closer the ratio is to 1, the higher the score. Uniformity score is assessed based on the coefficient of variation (CV) of the fitted lines of the kernel centroids in each row; the smaller the CV, the more uniform the ear rows, and the higher the score. The three scores are summed according to preset weights to obtain the comprehensive total score. A fixed threshold of T1 is set for Special Grade, and T2 is set for Grade I (T1 > T2). A total score ≥ T1 is rated Special Grade, T2 ≤ total score < T1 is rated Grade I, and a total score < T2 is rated Grade II.

[0051] Step S5: Defect removal. If at least one preprocessed image is found to have a defect, it is considered that a defect has been detected. The corn ears are classified as unqualified corn ears and fall into the unqualified collection box at the end of the grading execution module along with the conveyor belt.

[0052] This step, which occurs before the panoramic stitching in step S6, allows for the early removal of defective corn ears, freeing up storage space for the images of the substandard corn ears and improving system detection efficiency.

[0053] Combination Figure 1 The embodiment of the present invention also provides a photoelectric color sorting device for waxy corn ears based on trinocular vision, including a transmission module, a trinocular acquisition module, an industrial control computer, a PLC integrated control module, a graded execution module, and a material detection unit.

[0054] The transmission module is used to transport corn ears and drive the corn ears to rotate around their own central long axis; The three-eye acquisition module is fixedly mounted above the transmission module and includes three industrial cameras 5 arranged at equal intervals along the conveying direction for acquiring raw images of corn ears. The industrial control computer communicates with the three-eye acquisition module and the PLC integrated control module to preprocess the original image to obtain a preprocessed image. Then, defective corn ears are removed by defect detection. The defect-free preprocessed image is then stitched together in a panoramic view to obtain the initial image. The initial image is then subjected to maturity detection, and a comprehensive score is given based on ear length, ear diameter, fullness, and uniformity. The corn ears are then classified into special grade, first grade, and second grade, and a grade control signal is sent to the PLC integrated control module. The PLC integrated control module, with a programmable logic controller (PLC) as its core, is electrically or communicatively connected to the three-eye acquisition module, the industrial computer, and the graded execution module. It is used to detect the arrival status of corn ears and trigger the three-eye acquisition module to acquire images, receive the graded control signals output by the industrial computer, and output execution instructions to the graded execution module according to the control signals. The PLC calculates the action delay time Δt based on the grade signals, the start time t0 recorded by the material identification, the conveying speed, and the distance of each pusher station from the acquisition station.

[0055] The graded execution module, located downstream of the transmission module, includes three sets of push rod assemblies 6 and four collection boxes 7. Each set of push rod assemblies 6 includes a push rod and a drive motor. The drive motor is electrically connected to the PLC integrated control module and is used to receive execution commands from the PLC integrated control module to drive the corresponding push rod to push the corn ears into the collection box.

[0056] The transmission module includes a drive belt 2, multiple rollers 3, and a rack 4. The multiple rollers 3 are fixed at equal intervals on the drive belt 2 along the conveying direction. Each roller 3 has a gear mounted on both sides of its shaft end. The rack 4 is horizontally fixed on the frame 1 along the conveying direction and meshes with the gear. When the drive belt drives the rollers forward, the meshing of the gear and the rack drives the rollers to rotate synchronously around their own axis. The corn ears supported in the V-shaped gap between adjacent rollers 3 move forward synchronously with the rollers and rotate.

[0057] The three adjacent rollers form a V-shape, which can support the corn ears without the need for a separate tray or clamp, thus fully utilizing the functions of the device's structural components.

[0058] By meshing gears mounted on both ends of the roller 3 with a rack 4 located inside the frame along the conveying direction, the roller 3 can stably rotate around its own axis while revolving with the drive pulley 2. Compared to passive roller rotation schemes driven by friction, this method is more reliable, less prone to slippage, and allows for flexible adjustment of the roller's rotation speed by changing gear parameters. The structure is simple and easy to manufacture.

[0059] The material detection unit employs through-beam photoelectric sensors, positioned on both sides of the transmission module at the feed point. These sensors identify the position of the corn ears and output a position trigger signal to the PLC integrated control module. Upon receiving the position trigger signal, the PLC integrated control module assigns a unique material identifier to the corn ear and records the start time. It then calculates a fixed timing sequence based on preset parameters, binding all three acquired original images to the same material identifier for the corn ear. This triggers a camera trigger signal to the three-lens acquisition module, sequentially triggering the three industrial cameras to complete image acquisition. This ensures accurate correspondence between the image and the individual ear, avoiding image confusion under continuous conveying conditions and providing a data foundation for subsequent matching of materials and grading results.

[0060] The overall process is as follows: Corn ears are fed individually and evenly into the V-shaped gap formed by adjacent rollers 3, entering the transmission module and moving forward along the conveying direction with the drive belt 2. Simultaneously, gears on both sides of the roller 3 mesh with the rack 4 inside the frame 1, causing the roller 3 and the supported corn ears to rotate around their own central axis. When the corn ear reaches the material detection unit on the feeding side, the through-beam photoelectric sensor detects its arrival and outputs a trigger signal to the PLC integrated control module. Upon receiving the signal, the PLC integrated control module assigns a unique material identifier to the corn ear and records the start time. It then calculates the timing sequence based on preset parameters and sequentially sends camera trigger signals to the three industrial cameras of the three-eye acquisition module. When the corn ear reaches the acquisition station directly below the three industrial cameras, it completes a 120° rotation. The three cameras sequentially acquire three original images covering the entire surface of the corn ear, bind the images to the corresponding material identifiers, and upload them to the industrial control computer. The industrial control computer receives the three original images bound to the corresponding material identifiers. After the images are processed, all original images are preprocessed to obtain preprocessed images, followed by a defect detection step. If at least one preprocessed image is found to have surface defects, the corn ear is determined to be unqualified and is sent back to the PLC integrated control module along with the material identifier. The PLC integrated control module does not trigger the corresponding push rod action, and the corn ear continues to be conveyed forward by the transmission module, eventually falling into the end collection box of the grading execution module. If no defects are detected in any of the preprocessed images, the color sorting and grading step is entered. The defect-free preprocessed images are stitched together in a panoramic view to obtain the initial image. The maturity detection is completed, and a comprehensive score is made based on the characteristics of ear length, ear diameter, fullness, and uniformity. According to the score results, the corn ears are divided into special grade, first grade, and second grade, and a grade control signal carrying the corresponding material identifier is generated and sent back to the PLC integrated control module. After receiving the grade control signal, the PLC integrated control module calculates the push rod action delay based on the preset conveying speed and push rod station spacing when the corn ear moves to the corresponding grade grading station. It then outputs an execution command to the drive motor of the corresponding push rod assembly, driving the push rod to push the corn ear into the collection box of the corresponding grade, thus completing the full-process color sorting and grading of a single corn ear.

[0061] In summary, this embodiment addresses the multidimensional needs of grading waxy corn ears by constructing a photoelectric color sorting method and device for intelligent analysis and detection. This not only effectively replaces traditional sorting to reduce costs but also improves sorting efficiency, providing a new path for standardized and intelligent agricultural product processing and having practical application value for the development of modern agricultural equipment.

[0062] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the figures shown are only one embodiment of the present invention; the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present invention, such designs should fall within the protection scope of the present invention.

Claims

1. A photoelectric color sorting method for waxy corn ears based on trinocular vision, characterized in that, Includes the following steps: Transmission and transportation steps: The corn ears are transported through the transmission module, and the corn ears rotate around their own central long axis during the transportation process; Image acquisition steps: Three industrial cameras are arranged at equal intervals along the conveying direction. When the corn ears arrive at the acquisition station directly below the three industrial cameras in sequence, the three industrial cameras acquire the original images of the corn ears in sequence. Adjust the conveying speed of the drive wheel and the arrangement spacing of adjacent industrial cameras so that the roller drives the corn ears to rotate 120° around their own central long axis within the acquisition interval between two adjacent industrial cameras. Three original images cover the entire surface of the corn ears. Preprocessing steps: The original image is preprocessed to obtain a preprocessed image; Defect detection steps: Extract the feature vector of each preprocessed image, input the feature vector into a pre-trained binary classification support vector machine model, and detect whether there are mold or insect defects in the corn ears; if at least one preprocessed image detects a defect, the corn ear is determined to be a defective product and is removed. Panoramic stitching steps: Perform cylindrical unfolding, feature matching, translation registration and fusion processing on three preprocessed images corresponding to defect-free corn ears to obtain a panoramic image covering the entire surface; Color sorting and grading steps: The maturity level of corn ears is determined based on the proportion of purple pixels on the surface of the corn ears, and the corn ears are comprehensively graded in combination with ear length, ear thickness, fullness and uniformity.

2. The method according to claim 1, characterized in that, In the transmission and transportation step, the transmission module includes a drive belt, multiple rollers and a rack; multiple rollers are fixed at equal intervals on the drive belt along the conveying direction, and gears are fitted on both sides of the shaft end of each roller. The rack is fixed horizontally on the frame along the conveying direction and meshes with the gears. When the drive belt drives the rollers to convey forward, the meshing of the gears and racks drives the rollers to rotate synchronously around their own axes. The corn ears supported in the V-shaped gap between adjacent rollers move forward and rotate synchronously with the rollers. In the image acquisition step, the PLC integrated control module detects the arrival status of corn ears through the through-beam photoelectric sensor on the feeding side, assigns a unique material identifier to the corn ears and records the start time, calculates the trigger sequence of the three cameras in combination with the preset conveying speed, and binds the three acquired original images with the same material identifier.

3. The method according to claim 1, characterized in that, The preprocessing steps specifically include: Brightness normalization: Brightness correction is performed on the original image to obtain a brightness normalized image; Denoising: The three channels of the brightness-normalized image are filtered separately using a 5×5 Gaussian kernel to obtain the denoised image; Contrast enhancement: The denoised image is converted to the HSV color space, and the contrast of the luminance V channel is enhanced using the CLAHE algorithm to obtain an enhanced image; Binarization: The enhanced image is binarized using a fixed threshold method to obtain a binarized image; Morphological processing: The binarized image is subjected to erosion and dilation operations in sequence to remove isolated noise and fill tiny holes, resulting in a morphologically optimized image; Target segmentation: The main region of the corn ear in the morphologically optimized image is extracted by connected component analysis to obtain a preprocessed image.

4. The method according to claim 1, characterized in that, The feature vector is formed by concatenating the HSV color histogram of the preprocessed image with the LBP rotation-invariant texture features. The binary classification support vector machine model adopts the radial basis kernel function and is trained offline with images of healthy corn ears as positive samples and images of ears labeled as moldy or insect-infested as negative samples.

5. The method according to claim 1, characterized in that, In the panoramic stitching step: cylindrical unfolding, the preprocessed image is projected and unfolded along the length of the corn ear to obtain an unfolded image; feature matching, SIFT feature points and 128-dimensional descriptors are extracted from the overlapping area of ​​two adjacent unfolded images, and matching is performed in the ear thickness direction to obtain a set of constrained matching points; translation registration, the optimal translation amount of adjacent unfolded images is calculated based on the set of constrained matching points, and coordinate transformation is performed on the next unfolded image to align the next unfolded image with the previous unfolded image in space to obtain a registered image; By fusing the data, a linear gradient weight map based on pixel distance is constructed, and the stitching gaps are smoothly joined to obtain a panoramic image.

6. The method according to claim 1, characterized in that, In the color sorting and grading step, the panoramic image is converted to the HSV color space, the hue H channel is extracted, and the purple and white segmentation threshold is calculated using the OTSU algorithm. Based on the proportion of purple pixels, the maturity is divided into three categories: immature, mature, and overripe. At the same time, the ear length, ear diameter, fullness, and uniformity of the corn ears in the panoramic image are extracted. The immature and overripe corn ears are directly classified into the second level. The corn ears rated as mature are further weighted and comprehensively scored based on the ear length, ear diameter, fullness, and uniformity, and the corn ears are classified into special grade or first grade.

7. The method according to claim 1, characterized in that, The fullness is the ratio of the area of ​​the main body region of the corn ear to the area of ​​the smallest circumscribed rectangle; the uniformity is calculated as follows: after segmenting the sticky kernels, the centroid coordinates of each kernel are extracted, and after clustering by row, the centroids of each row of kernels are fitted with least squares straight lines. The standard deviation σ and mean μ of the root mean square error (RMSE) of each row are calculated. CV = σ / μ is used as the uniformity evaluation index. The smaller the CV, the more uniform the ear rows are.

8. A photoelectric color sorting device for waxy corn ears based on trinocular vision, characterized in that, Applied to implement the method of any one of claims 1 to 7, comprising: The transmission module is used to transport corn ears and drive the corn ears to rotate around their own central long axis; The three-lens acquisition module, mounted above the transmission module, includes three industrial cameras arranged at equal intervals along the conveying direction, used to acquire raw images of corn ears. An industrial control computer is connected to a three-eye acquisition module. The industrial control computer is equipped with an image preprocessing unit, a defect detection unit, a panoramic stitching unit, and a color sorting and grading unit, which are used to generate grade control signals carrying material identification. The PLC integrated control module is connected to the three-eye acquisition module and the industrial control electromechanical system respectively. It is used to detect the position status of corn ears, trigger camera acquisition, receive level control signals and calculate the push rod action delay. The graded execution module, located downstream of the transmission module, includes three sets of push rod assemblies and four collection boxes. The PLC integrated control module drives the push rods at the corresponding workstations to push the corn ears into the corresponding collection boxes according to the graded control signals.

9. The apparatus according to claim 8, characterized in that, After the industrial control computer completes the defect detection, if it determines that the product is unqualified, it will directly send a rejection signal to the PLC integrated control module and will no longer perform panoramic stitching and color sorting grading on the ear of fruit.

10. The apparatus according to claim 8, characterized in that, Each push rod assembly includes a push rod and a drive motor. The drive motor is electrically connected to a PLC integrated control module and is used to receive execution instructions from the PLC integrated control module to drive the corresponding push rod to push the corn ears into the corresponding collection box.

Images