A kiwifruit grading method based on machine vision and multi-feature fusion, a sorting device and a control method

Abstract

The application discloses a kiwi fruit grading method based on machine vision and multi-feature fusion, a sorting device and a control method. The sorting device comprises an image processing device, a controller, a visual sensor and a plurality of fruit outlet boxes. When the kiwi fruit reaches the detection station, the controller sends a trigger signal to the visual sensor to perform image acquisition. The image processing device receives image data, sequentially performs image preprocessing, kiwi fruit target area extraction, multi-dimensional feature extraction and multi-feature fusion grading, and generates corresponding grading signals sent to the controller. A plurality of trays are arranged on the annular conveying track and move along the track in a circulating manner. In the image acquisition stage, the controller drives the tray to rotate one revolution to collect the full-color image data of the kiwi fruit. In the sorting and placing stage, the controller drives the tray to move to the station corresponding to the fruit outlet box according to the grading signal and performs a turnover action, realizing the automatic, multi-index and high-precision sorting of the kiwi fruit, and significantly improving the sorting efficiency and product consistency.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sorting equipment technology, specifically relating to a kiwifruit grading method, sorting device and control method based on machine vision and multi-feature fusion. Background Technology

[0002] Post-harvest sorting of kiwifruit is a crucial step in enhancing its commercial value. Traditional sorting relies primarily on manual labor, using visual inspection and touch to determine the size, color, and surface defects of the fruit for grading. This method suffers from problems such as inconsistent sorting standards, low efficiency, high labor intensity, high costs, and susceptibility to fruit damage due to human error. With rising labor costs and increasing market demands for consistent fruit quality, the drawbacks of traditional sorting methods are becoming increasingly apparent.

[0003] In recent years, although some automated sorting equipment based on mechanical (such as sieve size) or weight sensors has emerged, these devices are limited in function and can usually only sort based on a single indicator (such as actual weight or maximum diameter), making it difficult to make accurate and comprehensive judgments on the complex appearance quality of kiwifruit, such as color and surface defects. Summary of the Invention

[0004] The purpose of this invention is to provide a kiwifruit grading method, sorting device and control method based on machine vision and multi-feature fusion, which can integrate multiple visual information to realize automated, multi-index and high-precision sorting of kiwifruit, significantly improving sorting efficiency and product consistency.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A machine vision-based kiwifruit sorting device includes a circular conveyor track on a workbench and a sorting device. The sorting device includes an image processing unit, a controller, a vision sensor located above the detection section of the circular conveyor track for collecting data on the size, color, and surface defects of the kiwifruit, and multiple fruit exit boxes arranged along the sorting path of the kiwifruit on one side of the workbench, each fruit exit box corresponding to a kiwifruit grading level. The controller is electrically connected to the vision sensor and sends a trigger signal to the vision sensor when the kiwifruit arrives at the detection station to initiate image acquisition. The vision sensor is communicatively connected to the image processing unit, which receives image data from the vision sensor, analyzes the size, color, and surface defects of the kiwifruit, and generates corresponding grading signals. The image processing unit is also communicatively connected to the controller and sends the grading signals to the controller. Multiple trays are set on a circular conveyor track by a slider assembly, and each tray is electrically connected to the controller. During the image acquisition stage, the controller drives the tray to rotate around its own axis once to acquire color image data of the kiwifruit from all directions. During the sorting and delivery stage, the controller drives the tray to move to the corresponding fruit box position according to the grading signal and performs a flipping action to make the kiwifruit fall into the fruit box.

[0006] Furthermore, the image processing device is configured to perform multi-dimensional feature analysis on the acquired images, specifically including: determining the projected outline of the fruit through edge detection based on the acquired images, and calculating its equivalent diameter according to the projected outline to characterize the fruit size; analyzing the color distribution characteristics of the peel in a preset color space to assess the fruit maturity; and detecting whether there are bruises, scars or disease-related defects on the fruit surface through texture feature extraction and defect recognition models.

[0007] Furthermore, an arched vision sensor cover is provided on the detection section of the workbench that covers the circular conveyor track. The inner side of the vision sensor cover is equipped with a vision sensor and a ring-shaped LED shadowless supplementary light source, and the inner wall is coated with a light-absorbing or matte coating. The vision sensor includes one or more sets of high-resolution color CCD / CMOS cameras for collecting color and texture information of the kiwi fruit surface and 3D depth cameras for acquiring three-dimensional point cloud data of the kiwi fruit.

[0008] Furthermore, the slider assembly includes a slider slidably disposed on an annular conveying track, a support rod vertically disposed on the slider, a rotating component electrically connected to the controller being mounted on the upper end of the support rod, the tray being fixed on the rotating component, and the bearing surface of the tray being a concave curved surface with its edge height higher than the central area.

[0009] Furthermore, a fruit guiding mechanism is provided on the workbench corresponding to each fruit dispensing box. The fruit guiding mechanism includes fixed blocks arranged opposite each other, and a rotating shaft is rotatably installed between the fixed blocks. One end of the rotating shaft is fixedly connected to the output end of the motor. A guide plate is installed on the rotating shaft. The front end of the guide plate extends to the fruit dispensing station, and its end is connected to a tray extending to the corresponding fruit dispensing box.

[0010] Furthermore, the guide plate has a V-shaped groove on its guide surface, and the opening size of the V-shaped groove is configured according to the size of the fruit corresponding to the fruit box. The surface of the tray is provided with an anti-slip coating or flexible material.

[0011] Furthermore, outside the feeding station of the circular conveyor track, there are also feeding devices for batch extraction of kiwifruit and conveying them to the feeding station, and unloading guide devices for separating individual kiwifruit, adjusting their posture, and placing them one by one into a tray.

[0012] This invention also provides a machine vision-based kiwi fruit sorting control method, comprising the following steps: Step 1: Start the circular conveyor track. Through the feeding equipment and the fruit unloading guide equipment, the kiwifruit to be sorted are transported one by one in an orderly manner and loaded onto the empty pallets. Step 2: The tray with the kiwifruit placed on it moves at a constant speed along the circular conveyor track. When the tray carrying the kiwifruit enters the inspection station, the controller triggers the vision sensor to start image acquisition and simultaneously drives the tray to rotate around its own axis once. During the rotation, the vision sensor takes continuous or multi-angle fixed-point pictures of the kiwifruit to obtain its all-round image data. Step 3: The captured image data is transmitted in real time to the image processing device for image processing and analysis, and generates corresponding grading signals, including the target fruit box number, grading level, rotating component flip angle and data acquisition timestamp, and sends them to the controller; when the tray carrying the identified kiwifruit moves to the station of the target fruit box, the controller drives the tray to perform a flipping action, and at the same time sends a drive signal to the motor, which drives the guide plate to rotate through the rotating shaft, so that the tray is aligned with the entrance of the target fruit box, so as to guide the kiwifruit to fall into the designated fruit box; Step 4: After the sorting action is completed, the guide plate is reset, the tray returns to a horizontal state and continues to move upstream with the track to prepare to receive the next fruit to be sorted. This cycle is repeated to achieve continuous automated operation.

[0013] This invention also provides a kiwifruit grading method based on machine vision and multi-feature fusion, comprising the following steps: Step 1: Image Preprocessing Step 1.1: Normalize the original image to unify the image size and pixel value range under different acquisition conditions. The normalization calculation formula is as follows: in, For the original image in coordinates Pixel value at that location, The maximum pixel value of the original image. The minimum pixel value of the original image. These are the normalized pixel values, ranging from 0 to 1. Step 1.2: Apply Gaussian filtering to the normalized image to eliminate Gaussian noise introduced by the image sensor or environment. The Gaussian rate formula is: in, For Gaussian kernel function, The standard deviation is the Gaussian kernel value, ranging from 1.0 to 1.5. k The half-width of the Gaussian kernel is [value], and the specification is [value]. ; Step 1.3: Use the grayscale world method to correct the illumination, eliminating color and texture analysis errors caused by uneven illumination. The calculation formula is as follows: in c It is the primary color channel (R / G / B). It is the global pixel value of the channel. In the gray-world method, it is the intermediate gray level. For an 8-bit image, the value is 128. Step 2: Extraction of the target region of the kiwi fruit Step 2.1, Color Space Conversion The RGB three-channel values ​​are mapped to HSV color attributes, and the ripeness of kiwifruit is analyzed using HSV. First calculate the lightness. Take the maximum value of the RGB three channels, and use the brightest channel to determine the overall brightness of the color, with saturation set to... in This represents the difference between the maximum and minimum values ​​of the three channels. The larger the value, the greater the difference between the RGB three channels, and the more vibrant the color. Calculate hue again H like H <0, then Correct negative numbers. H It only reflects the color type and is not affected by light or the reflection of the fruit peel; Finally, threshold segmentation is performed to limit the range of parameter values ​​for different varieties. H : H 1 -H 2, S : S 1- S 2, V : V 1- V 2. The segmentation formula is: in This is a mask image, where 1 represents the kiwi fruit area and 0 represents the background; Step 2.2, Morphological Processing Define a A structuring element SE, where all elements are 1, is used to traverse each pixel in the image. By determining the intersection relationship between the structuring element and the kiwi mask, the shape of the target region is changed. First, perform a dilation operation on each pixel in the mask image. If a pixel or one of its neighboring pixels has a value of 1, then that pixel is set to 1. The dilated image is as follows: The pixel and its 8 neighborhoods are "framed" with a 3×3 structuring element SE. If any of the neighborhoods is a kiwi region, the current pixel is marked as a kiwi region. Then perform the etching operation. The image after etching is as follows: The pixel and its 8 neighborhoods are "framed" with a 3×3 structuring element. Only if all the neighborhoods are kiwi areas will the current pixel be retained as a kiwi area; otherwise, it will be set as the background. Step 3: Multidimensional Feature Extraction Step 3.1: Calculate the equivalent diameter using edge detection and contour projection to characterize the fruit's size. For images that have undergone Gaussian filtering and illumination correction, horizontal edges are detected using the x-direction gradient operator. Detecting vertical edges using the y-direction gradient operator The gradient magnitude is This characterizes edge strength and reflects whether a pixel is an edge; the larger the value, the more obvious the edge. The gradient direction is It represents the direction of the edge and reflects the angle of the edge; The gradient direction θ is quantized into four directions: 0°, 45°, 90°, and 135° for each pixel. Compare the gradient magnitudes of the two pixels before and after it in the gradient direction. If the current pixel has the largest magnitude, it is set as an edge; otherwise, it is set as a non-edge, thus changing the kiwi outline from a blurry wide band to a clear thin line 1 pixel wide. A dual threshold screening method is used, with 30% of the maximum gradient magnitude set as the high threshold. low threshold Pixels with amplitude values ​​greater than the high threshold are considered strong edges and retained directly, while pixels with amplitude values ​​lower than the low threshold are considered noise and excluded. Pixels between the low and high thresholds are retained as weak edges only if the pixel is connected to a strong edge. Finally, the projected outline of the kiwi fruit was obtained. The area is accurately calculated using Green's formula, which is: ; Photograph a standard template of known size, calculate the number of pixels on the standard template, and find the ratio of the actual size of the standard template to the number of pixels. The actual projected area of ​​the kiwifruit is The equivalent diameter is If the diameter is less than 20mm or greater than 100mm, it is considered an abnormality and the image needs to be re-acquired; if the contour breakage is greater than 5%, the equivalent diameter is directly estimated using the bounding rectangle of the mask. Step 3.2: Assess maturity based on statistical characteristics of the HSV color space. First, calculate the mean of H. The normalized maturity score is: limited , The closer the value is to 1, the higher the maturity level. Step 3.3: Extract surface defect features of kiwifruit based on gray-level co-occurrence matrix (GLCM). GLCM is calculated on the grayscale image of the fruit region, and four core texture features are extracted: Contrast, Energy, Entropy, and Correlation. in Here is the probability matrix of GLCM. and The mean and standard deviation of the grayscale values; Defect identification was performed using a MobileNetV2 convolutional neural network, with the input being a cropped image of the fruit region (224). 224), the output is a defect evaluation. , Deemed to be without defects Deemed a minor defect. Deemed a serious defect; Step 4: Multi-feature fusion and hierarchical classification First, normalize the dimensions to The defects were corrected to The smaller the flaw, the higher the value; The formula for calculating the overall score is: The weight ; Step 5: Output the grading results: It is a first-grade fruit. It is a second-grade fruit. It is a third-grade fruit. It is an external result.

[0014] Compared with the prior art, the present invention has the following beneficial effects: This invention automates the entire process of feeding, inspection, and sorting through a circular conveyor track and multiple continuously operating trays. Compared to manual sorting, efficiency is increased several times, and labor intensity is significantly reduced. Utilizing visual sensors, multi-dimensional information such as fruit size, color (ripeness), and surface defects (e.g., bruises, diseases) can be simultaneously acquired. By driving the trays to rotate, the visual sensors can capture 360-degree images of the fruit, eliminating blind spots and ensuring more thorough identification of surface defects. This results in more comprehensive and accurate sorting, enhancing the scientific rigor of product grading. The entire sorting process combines optical visual inspection and mechanical guidance, avoiding rigid collisions and compression with the fruit, effectively reducing mechanical damage during sorting.

[0015] Furthermore, this invention guides the fruit through precisely positioned guide channels, eliminating "delivery" errors caused by tray positioning errors or fruit rolling, ensuring that each fruit enters the correct grade box. Each fruit exit box is equipped with an independent guide channel structure, reducing the manufacturing cost and complexity of the equipment, while also reducing potential failure points. The smooth curved surface of the guide channel and the flexible tray can effectively guide and buffer the falling fruit, minimizing mechanical damage to the kiwifruit during the sorting process and ensuring the commercial value of the fruit.

[0016] Furthermore, by setting up an arched vision sensor cover with an integrated uniform light source, ambient light interference and reflections are effectively avoided, providing a stable and consistent shooting environment for the vision sensor and ensuring image quality. The tray, which is high around the edges and low in the middle, effectively restricts the rolling of the kiwi fruit during movement and rotation, ensuring that it is always in the optimal center of the camera's field of view, thus improving the accuracy of image analysis and size measurement.

[0017] This invention employs a hierarchical control architecture centered on a controller. An image processing device generates grading signals based on data collected by a vision sensor. The controller receives these signals and, through asynchronous commands, drives the tray, motor, and conveying system to perform corresponding actions. Each execution unit (such as the tray flipping mechanism and the guide vane rotation mechanism) is independently controlled and does not obstruct others, achieving decoupling of control tasks. The grading logic is stored parametrically in the controller, supporting dynamic configuration of fruit size thresholds, color judgment standards, and defect tolerance levels, adapting to different sorting needs without hardware modifications. This architecture facilitates expansion by adding new sensor types, increasing the number of grading levels, or combining arbitrary grading parameters. Attached Figure Description

[0018] Figure 1 This is a first structural schematic diagram of the present invention; Figure 2 This is a schematic diagram of the second structure of the present invention; Figure 3 This is a schematic diagram of the connection structure between the tray and the slider assembly in this invention; Figure 4 This is a schematic diagram of the fruit guiding mechanism in this invention; Figure 5 This is a flowchart illustrating a hierarchical method based on multi-feature fusion and machine vision.

[0019] In the diagram: 11-Workbench; 12-Circular conveyor track; 13-Slider; 14-Support rod; 15-Rotating component; 16-Pattern; 17-Feeding equipment; 18-Fruit unloading guide equipment; 2-Sorting device; 21-Image processing device; 22-Vision sensor cover; 23-Vision sensor; 24-Fruit discharge box; 25-Controller; 26-Fixing block; 261-Rotating shaft; 262-Guide plate; 263-Motor; 264-Pattern. Detailed Implementation

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 , 2 As shown in this embodiment, a machine vision-based kiwifruit sorting device includes a circular conveyor track 12 and a sorting device 2 mounted on a workbench 11. The sorting device 2 includes an image processing device 21, a controller 25, a vision sensor 23 located above the detection section of the circular conveyor track 12, and multiple fruit exit boxes 24 arranged along the sorting path of the kiwifruit on one side of the workbench 11. Each fruit exit box 24 corresponds to a kiwifruit grading level and is used to classify and collect fruits of the corresponding grade. The controller 25 is electrically connected to the vision sensor 23 and sends a trigger signal to the vision sensor 23 when the kiwifruit arrives at the detection station to initiate image acquisition. The vision sensor 23 collects appearance feature data of the kiwifruit, including color, size, and surface defects. The vision sensor 23 is communicatively connected to the image processing device 21, which receives the image data from the vision sensor 23, analyzes the size, color, and surface defects of the kiwifruit, and generates corresponding grading signals. The image processing device 21 is communicatively connected to the controller 25 and sends the grading signals to the controller 25.

[0022] To achieve accurate grading of kiwifruit, the image processing device 21 is configured to perform multi-dimensional feature analysis on the acquired images. This grading method is based on machine vision and multi-feature fusion, such as... Figure 5 As shown, it includes the following steps: Step 1: Image Preprocessing Step 1.1: Normalize the original image to unify the image size and pixel value range under different acquisition conditions. The normalization calculation formula is as follows: in, For the original image in coordinates Pixel value at that location, The maximum pixel value of the original image. The minimum pixel value of the original image. These are the normalized pixel values, ranging from 0 to 1. Step 1.2: Apply Gaussian filtering to the normalized image to eliminate Gaussian noise introduced by the image sensor or environment. The Gaussian rate formula is: in, For Gaussian kernel function, The standard deviation is the Gaussian kernel value, ranging from 1.0 to 1.5. k The half-width of the Gaussian kernel is [value], and the specification is [value]. ; Step 1.3: Use the grayscale world method to correct the illumination, eliminating color and texture analysis errors caused by uneven illumination. The calculation formula is as follows: in c It is the primary color channel (R / G / B). It is the global pixel value of the channel. In the gray-world method, it is the intermediate gray level. For an 8-bit image, the value is 128. Step 2: Extraction of the target region of the kiwi fruit Step 2.1, Color Space Conversion The RGB three-channel values ​​are mapped to HSV color attributes, and the ripeness of kiwifruit is analyzed using HSV. First calculate the lightness. Take the maximum value of the RGB three channels, and use the brightest channel to determine the overall brightness of the color, with saturation set to... in This represents the difference between the maximum and minimum values ​​of the three channels. The larger the value, the greater the difference between the RGB three channels, and the more vibrant the color. Calculate hue again H like H <0, then Correct negative numbers. H It only reflects the color type and is not affected by light or the reflection of the fruit peel; Finally, threshold segmentation is performed to limit the range of parameter values ​​for different varieties. H : H 1 -H 2, S :S 1- S 2, V : V 1- V 2. The segmentation formula is: in This is a mask image, where 1 represents the kiwi fruit area and 0 represents the background; Step 2.2, Morphological Processing Define a A structuring element SE, where all elements are 1, is used to traverse each pixel in the image. By determining the intersection relationship between the structuring element and the kiwi mask, the shape of the target region is changed. First, perform a dilation operation on each pixel in the mask image. If a pixel or one of its neighboring pixels has a value of 1, then that pixel is set to 1. The dilated image is as follows: The pixel and its 8 neighborhoods are "framed" with a 3×3 structuring element SE. If any of the neighborhoods is a kiwi region, the current pixel is marked as a kiwi region. Then perform the etching operation. The image after etching is as follows: The pixel and its 8 neighborhoods are "framed" with a 3×3 structuring element. Only if all the neighborhoods are kiwi areas will the current pixel be retained as a kiwi area; otherwise, it will be set as the background. Step 3: Multidimensional Feature Extraction Step 3.1: Calculate the equivalent diameter using edge detection and contour projection to characterize the fruit's size. For images that have undergone Gaussian filtering and illumination correction, horizontal edges are detected using the x-direction gradient operator. Detecting vertical edges using the y-direction gradient operator The gradient magnitude is This characterizes edge strength and reflects whether a pixel is an edge; the larger the value, the more obvious the edge. The gradient direction is It represents the direction of the edge and reflects the angle of the edge; The gradient direction θ is quantized into four directions: 0°, 45°, 90°, and 135° for each pixel. Compare the gradient magnitudes of the two pixels before and after it in the gradient direction. If the current pixel has the largest magnitude, it is set as an edge; otherwise, it is set as a non-edge, thus changing the kiwi outline from a blurry wide band to a clear thin line 1 pixel wide. A dual threshold screening method is used, with 30% of the maximum gradient magnitude set as the high threshold. low threshold Pixels with amplitude values ​​greater than the high threshold are considered strong edges and retained directly, while pixels with amplitude values ​​lower than the low threshold are considered noise and excluded. Pixels between the low and high thresholds are retained as weak edges only if the pixel is connected to a strong edge. Finally, the projected outline of the kiwi fruit was obtained. The area is accurately calculated using Green's formula, which is: ; Photograph a standard template of known size, calculate the number of pixels on the standard template, and find the ratio of the actual size of the standard template to the number of pixels. The actual projected area of ​​the kiwifruit is The equivalent diameter is If the diameter is less than 20mm or greater than 100mm, it is considered an abnormality and the image needs to be re-acquired; if the contour breakage is greater than 5%, the equivalent diameter is directly estimated using the bounding rectangle of the mask. Step 3.2: Assess maturity based on statistical characteristics of the HSV color space. First, calculate the mean of H. The normalized maturity score is: limited , The closer the value is to 1, the higher the maturity level. Step 3.3: Extract surface defect features of kiwifruit based on gray-level co-occurrence matrix (GLCM). GLCM is calculated on the grayscale image of the fruit region, and four core texture features are extracted: Contrast, Energy, Entropy, and Correlation. in Here is the probability matrix of GLCM. and The mean and standard deviation of the grayscale values; Defect identification was performed using a MobileNetV2 convolutional neural network, with the input being a cropped image of the fruit region (224). 224), the output is a defect evaluation. , Deemed to be without defects Deemed a minor defect. Deemed a serious defect; Step 4: Multi-feature fusion and hierarchical classification First, normalize the dimensions to The defects were corrected to The smaller the flaw, the higher the value; The formula for calculating the overall score is: The weight ; Step 5: Output the grading results: It is a first-grade fruit. It is a second-grade fruit. It is a third-grade fruit. It is an external result.

[0023] This multi-feature fusion identification method significantly improves the accuracy and automation of sorting compared to single-indicator sorting.

[0024] Multiple trays 16 are arranged on the circular conveyor track 12, moving cyclically along the track via a slider assembly. The bearing surface of each tray 16 is a concave curved surface, with its edge height higher than the center area. Each tray 16 is electrically connected to a controller 25. During the image acquisition stage, the controller 25 drives the tray 16 to rotate around its own axis once to acquire color image data of the kiwifruit from all directions. During the sorting and placement stage, the controller 25 drives the tray 16 to move to the corresponding fruit box 24 according to the grading signal and performs a flipping action.

[0025] like Figure 3 As shown, the slider assembly includes a slider 13 that is slidably disposed on the annular conveying track 12. A support rod 14 is vertically disposed on the slider 13. A rotating component 15 that is electrically connected to the controller 25 is mounted on the upper end of the support rod 14. The tray 16 is fixed on the rotating component 15.

[0026] Preferably, the vision sensor cover 22 has an arched structure, fixed to the workbench 11 and covering the detection section of the annular conveyor track 12. A vision sensor 23 and an annular LED shadowless supplementary light source are installed on the inner side of the vision sensor cover 22, and the inner wall is coated with a light-absorbing or matte coating to reduce ambient light reflection, thus forming a closed or semi-closed imaging chamber, providing a stable and standardized image acquisition environment for the vision sensor 23. Specifically, the vision sensor 23 includes one or more sets of high-resolution color CCD / CMOS cameras and a 3D depth camera. The color CCD / CMOS camera is used to acquire the color and texture information of the kiwi fruit surface, and the 3D depth camera is used to acquire the three-dimensional point cloud data of the kiwi fruit. The controller 25 obtains the size, color, and surface defects of the kiwi fruit based on the three-dimensional point cloud data.

[0027] A fruit guiding mechanism is provided on the workbench 11 at the position corresponding to each fruit outlet box 24, such as Figure 4As shown, the fruit guiding mechanism includes fixed blocks 26 arranged opposite each other, with a rotating shaft 261 rotatably mounted between the fixed blocks 26. One end of the rotating shaft 261 is fixedly connected to the output end of the motor 263. A guide plate 262 is mounted on the rotating shaft 261, with its front end extending to the fruit discharge station and its end connected to a tray 264 extending to the corresponding fruit discharge box 24. The guide plate 262 has a V-shaped groove on its guiding surface, and the opening size of the V-shaped groove is configured according to the size of the fruit corresponding to the fruit discharge box 24. In the default state, the guide plate 262 is kept in the retracted position to avoid interfering with the normal conveying of the tray 16. The surface of the tray 264 is provided with an anti-slip coating or flexible material to increase the friction between the tray and the kiwifruit and prevent displacement during the flipping process.

[0028] Outside the feeding station of the circular conveyor track 12, there are also a feeding device 17 and a fruit unloading guide device 18. The feeding device 17 is used to extract kiwifruit in batches and transport them to the feeding station, while the fruit unloading guide device 18 is used to separate individual kiwifruit and adjust their posture, and place them one by one into the tray 16.

[0029] A machine vision-based method for sorting and controlling kiwifruit includes the following steps: Step 1: Start the circular conveyor track 12. Through the feeding device 17 and the fruit unloading guide device 18, the kiwifruit to be sorted are transported one by one in an orderly manner and loaded onto the empty pallet 16. Because the bearing surface of tray 16 is a concave curved surface, and its edge height is higher than the central area, the kiwi fruit will naturally stabilize in the center of tray 16 after being placed in it, preventing it from rolling during subsequent movement.

[0030] Step 2: The tray 16 with the kiwifruit placed on it moves at a constant speed along the circular conveyor track 12. When the tray 16 carrying the kiwifruit enters the inspection station, the controller 25 triggers the vision sensor 23 to start image acquisition and synchronously drives the tray 16 to rotate around its own axis once. During the rotation, the vision sensor 23 continuously takes pictures of the kiwifruit or takes pictures from multiple angles and fixed points to obtain its all-round image data. Step 3: The captured image data is transmitted in real time to the image processing device 21 for image processing and analysis, and generates corresponding grading signals, including the target fruit box number, grading level, rotating component flip angle and data acquisition timestamp, and sends them to the controller 25; When the tray 16 carrying the identified kiwifruit moves to the station of the target fruit box 24, the controller 25 drives the tray 16 to perform a flipping action, and at the same time sends a drive signal to the motor 263, which drives the guide plate 262 to rotate through the rotating shaft 261, so that the tray 264 is aligned with the entrance of the target fruit box 24, so as to guide the kiwifruit to fall into the designated fruit box 24; Step 4: After the sorting action is completed, the guide plate 262 is reset, the tray 16 returns to a horizontal state and continues to move upstream with the track to prepare to receive the next fruit to be sorted. This cycle repeats to achieve continuous automated operation.

Claims

1. A machine vision-based kiwi fruit sorting device, characterized in that, The system includes a circular conveyor track (12) and a sorting device (2) set on a workbench (11). The sorting device (2) includes an image processing device (21), a controller (25), a vision sensor (23) located above the detection section of the circular conveyor track (12) for collecting the size, color and surface defects of kiwifruit, and multiple fruit exit boxes (24) set on one side of the workbench (11) and arranged along the sorting path of the kiwifruit. Each fruit exit box (24) corresponds to a kiwifruit grading level. The controller (25) is electrically connected to the vision sensor (23) and is used to send a trigger signal to the vision sensor (23) when the kiwifruit arrives at the detection station to start image acquisition. The vision sensor (23) is communicatively connected to the image processing device (21). The image processing device (21) receives image data from the vision sensor (23), analyzes the size, color and surface defects of the kiwifruit, and generates corresponding grading signals. The image processing device (21) is communicatively connected to the controller (25) and sends the grading signals to the controller (25). Multiple trays (16) are set on the circular conveyor track (12) by a slider assembly and move cyclically along the track. Each tray (16) is electrically connected to the controller (25). During the image acquisition stage, the controller (25) drives the tray (16) to rotate around its own axis once to acquire color image data of the kiwifruit from all directions. During the sorting and delivery stage, the controller (25) drives the tray (16) to move to the corresponding fruit box (24) according to the grading signal and performs a flipping action so that the kiwifruit falls into the fruit box (24).

2. The kiwi fruit sorting device based on machine vision according to claim 1, characterized in that, The image processing device (21) is configured to perform multi-dimensional feature analysis on the acquired image, specifically including: determining the projected outline of the fruit by edge detection based on the acquired image, and calculating its equivalent diameter according to the projected outline to characterize the fruit size; analyzing the color distribution characteristics of the peel in a preset color space to evaluate the fruit maturity; and detecting whether there are bruises, scars or disease-related defects on the fruit surface by texture feature extraction and defect recognition model.

3. The kiwi fruit sorting device based on machine vision according to claim 1, characterized in that, An arched vision sensor cover (22) is provided on the detection section of the workbench (11) and covering the circular conveyor track (12). A vision sensor (23) and a ring-shaped LED shadowless light source are installed on the inner side of the vision sensor cover (22), and the inner wall is coated with a light-absorbing or matte coating. The vision sensor (23) includes one or more sets of high-resolution color CCD / CMOS cameras for collecting the color and texture information of the kiwi fruit surface and a 3D depth camera for acquiring the three-dimensional point cloud data of the kiwi fruit.

4. A kiwi fruit sorting device based on machine vision according to claim 1, characterized in that the slider assembly includes a slider (13) slidably disposed on an annular conveying track (12), a support rod (14) is vertically disposed on the slider (13), a rotating component (15) electrically connected to the controller (25) is mounted on the upper end of the support rod (14), the tray (16) is fixed on the rotating component (15), and the bearing surface of the tray (16) is a concave curved surface, the edge height of which is higher than the central area.

5. A kiwi fruit sorting device based on machine vision according to any one of claims 1-4, characterized in that a fruit guiding mechanism is provided on the workbench (11) corresponding to each fruit box (24), the fruit guiding mechanism includes fixed blocks (26) arranged opposite to each other, a rotating shaft (261) is rotatably installed between the fixed blocks (26), one end of the rotating shaft (261) is fixedly connected to the output end of the motor (263); a guide plate (262) is installed on the rotating shaft (261), the front end of the guide plate (262) extends to the fruit outlet station, and its end is connected to a tray (264) extending to the corresponding fruit box (24).

6. A kiwi fruit sorting device based on machine vision according to claim 5, characterized in that the guide plate (262) is provided with a V-shaped groove on the guide surface, the opening size of the V-shaped groove is configured according to the fruit size corresponding to the fruit box (24), and the tray (264) is provided with an anti-slip coating or flexible material on its surface.

7. A kiwi fruit sorting device based on machine vision according to claim 6, characterized in that, outside the feeding station of the circular conveying track (12), there is a feeding device (17) for batch extraction of kiwi fruit and conveying to the feeding station, and a fruit unloading guide device (18) for separating individual kiwi fruit and adjusting their posture and placing them one by one into a tray (16).

8. The device according to claim 7 implements a machine vision-based kiwi fruit sorting control method, characterized in that, Includes the following steps: Step 1: Start the circular conveyor track (12), and through the feeding equipment (17) and the fruit unloading guide equipment (18), the kiwifruit to be sorted are transported one by one in an orderly manner and loaded onto the empty pallet (16); Step 2: The tray (16) with the kiwifruit placed on it moves at a constant speed along the circular conveyor track (12). When the tray (16) carrying the kiwifruit enters the inspection station, the controller (25) triggers the vision sensor (23) to start image acquisition and synchronously drives the tray (16) to rotate around its own axis. During the rotation, the vision sensor (23) takes continuous or multi-angle fixed-point pictures of the kiwifruit to obtain its all-round image data. Step 3: The captured image data is transmitted in real time to the image processing device (21) to process and analyze the image and generate corresponding grading signals, including the target fruit box number, grading level, rotating part flip angle and data acquisition timestamp, and sent to the controller (25); When the tray (16) carrying the identified kiwifruit moves to the station of the target fruit box (24), the controller (25) drives the tray (16) to perform a flipping action, and at the same time sends a drive signal to the motor (263), which drives the guide plate (262) to rotate through the rotating shaft (261), so that the tray (264) is aligned with the entrance of the target fruit box (24) to guide the kiwifruit to fall into the designated fruit box (24). Step 4: After the sorting action is completed, the guide plate (262) is reset, the tray (16) returns to the horizontal state and continues to move upstream with the track to prepare to receive the next fruit to be sorted. This cycle repeats to achieve continuous automated operation.

9. A kiwifruit grading method based on machine vision and multi-feature fusion, applied to the image processing device (21) of the sorting device as described in claim 1, characterized in that, Includes the following steps: Step 1: Image Preprocessing Step 1.1: Normalize the original image to unify the image size and pixel value range under different acquisition conditions. The normalization calculation formula is as follows: in, For the original image in coordinates Pixel value at that location, The maximum pixel value of the original image. The minimum pixel value of the original image. These are the normalized pixel values, ranging from 0 to 1. Step 1.2: Apply Gaussian filtering to the normalized image to eliminate Gaussian noise introduced by the image sensor or environment. The Gaussian rate formula is: in, For Gaussian kernel function, The standard deviation is the Gaussian kernel value, ranging from 1.0 to 1.

5. k The half-width of the Gaussian kernel is [value], and the specification is [value]. ; Step 1.3: Use the grayscale world method to correct the illumination, eliminating color and texture analysis errors caused by uneven illumination. The calculation formula is as follows: in c It is the primary color channel (R / G / B). It is the global pixel value of the channel. In the gray-world method, it is the intermediate gray level. For an 8-bit image, the value is 128. Step 2: Extraction of the target region of the kiwi fruit Step 2.1, Color Space Conversion The RGB three-channel values ​​are mapped to HSV color attributes, and the ripeness of kiwifruit is analyzed using HSV. First calculate the lightness. Take the maximum value of the RGB three channels, and use the brightest channel to determine the overall brightness of the color, with saturation set to... in This represents the difference between the maximum and minimum values ​​of the three channels. The larger the value, the greater the difference between the RGB three channels, and the more vibrant the color. Calculate hue again H like H <0, then Correct negative numbers. H It only reflects the color type and is not affected by light or the reflection of the fruit peel; Finally, threshold segmentation is performed to limit the range of parameter values ​​for different varieties. H : H 1 -H 2, S : S 1- S 2, V : V 1- V 2. The segmentation formula is: in This is a mask image, where 1 represents the kiwi fruit area and 0 represents the background; Step 2.2, Morphological Processing Define a A structuring element SE, where all elements are 1, is used to traverse each pixel in the image. By determining the intersection relationship between the structuring element and the kiwi mask, the shape of the target region is changed. First, perform a dilation operation on each pixel in the mask image. If a pixel or one of its neighboring pixels has a value of 1, then that pixel is set to 1. The dilated image is as follows: The pixel and its 8 neighborhoods are "framed" with a 3×3 structuring element SE. If any of the neighborhoods is a kiwi region, the current pixel is marked as a kiwi region. Then perform the etching operation. The image after etching is as follows: The pixel and its 8 neighborhoods are "framed" with a 3×3 structuring element. Only if all the neighborhoods are kiwi areas will the current pixel be retained as a kiwi area; otherwise, it will be set as the background. Step 3: Multidimensional Feature Extraction Step 3.1: Calculate the equivalent diameter using edge detection and contour projection to characterize the fruit's size. For images that have undergone Gaussian filtering and illumination correction, horizontal edges are detected using the x-direction gradient operator. Detecting vertical edges using the y-direction gradient operator The gradient magnitude is This characterizes edge strength and reflects whether a pixel is an edge; the larger the value, the more obvious the edge. The gradient direction is It represents the direction of the edge and reflects the angle of the edge; The gradient direction θ is quantized into four directions: 0°, 45°, 90°, and 135° for each pixel. Compare the gradient magnitudes of the two pixels before and after it in the gradient direction. If the current pixel has the largest magnitude, it is set as an edge; otherwise, it is set as a non-edge, thus changing the kiwi outline from a blurry wide band to a clear thin line 1 pixel wide. A dual threshold screening method is used, with 30% of the maximum gradient magnitude set as the high threshold. low threshold Pixels with amplitude values ​​greater than the high threshold are considered strong edges and retained directly, while pixels with amplitude values ​​lower than the low threshold are considered noise and excluded. Pixels between the low and high thresholds are retained as weak edges only if the pixel is connected to a strong edge. Finally, the projected outline of the kiwi fruit was obtained. The area is accurately calculated using Green's formula, which is: ; Photograph a standard template of known size, calculate the number of pixels on the standard template, and find the ratio of the actual size of the standard template to the number of pixels. The actual projected area of ​​the kiwifruit is The equivalent diameter is If the diameter is less than 20mm or greater than 100mm, it is considered an abnormality and the image needs to be re-acquired; if the contour breakage is greater than 5%, the equivalent diameter is directly estimated using the bounding rectangle of the mask. Step 3.2: Assess maturity based on statistical characteristics of the HSV color space. First, calculate the mean of H. The normalized maturity score is: limited , The closer the value is to 1, the higher the maturity level. Step 3.3: Extract surface defect features of kiwifruit based on gray-level co-occurrence matrix (GLCM). GLCM is calculated on the grayscale image of the fruit region, and four core texture features are extracted: Contrast, Energy, Entropy, and Correlation. in Here is the probability matrix of GLCM. and The mean and standard deviation of the grayscale values; Defect identification was performed using a MobileNetV2 convolutional neural network, with the input being a cropped image of the fruit region (224). 224), the output is a defect evaluation. , Deemed to be without defects Deemed a minor defect. Deemed a serious defect; Step 4: Multi-feature fusion and hierarchical classification First, normalize the dimensions to The defects were corrected to The smaller the flaw, the higher the value; The formula for calculating the overall score is: The weight ; Step 5: Output the grading results: It is a first-grade fruit. It is a second-grade fruit. It is a third-grade fruit. It is an external result.

Images