Fruit and vegetable picking robot and target identification picking method thereof
By integrating the walking mechanism, rotating disc and five-axis robotic arms, combined with multiple image recognition algorithms, the challenges of identification and positioning in fruit and vegetable picking are solved, and efficient and accurate fruit and vegetable picking is achieved, reducing the rate of error picking and adapting to complex environments. ability.
Patent Information
- Application Number
- CN202411086325.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-12-03
AI Technical Summary
Existing fruit and vegetable picking robots face challenges brought about by differences in color, shape and texture when identifying and positioning different types of fruits and vegetables, resulting in inefficient recognition and high mis-picking rates.
Fruit and vegetable picking robots that integrate walking mechanism, rotating disc and five-axis robotic arms, combine multiple image recognition algorithms, such as instance segmentation algorithms based on color features, shape features and deep convolutional neural networks, through sparse convolution kernels and superpixels Segmentation technology to achieve accurate identification and positioning of a variety of fruits and vegetables.
It significantly improves the picking efficiency and recognition accuracy, reduces the mis-picking of immature or damaged fruits and vegetables, enhances the robot's adaptability in complex field environments, and reduces the efficiency reduction caused by fatigue.
Smart Images

Figure CN119054511A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field related to fruit and vegetable picking robots, and in particular to a fruit and vegetable picking robot and a target recognition and picking method thereof. Background Art
[0002] In the agricultural field, fruit and vegetable picking is a labor-intensive task that is arduous and time-consuming. The traditional manual picking method is not only inefficient, but also faces problems of labor shortage and rising costs. With the continuous development of modern agricultural technology, automation and intelligence have become an inevitable trend in the development of agricultural mechanization. Therefore, the development of an efficient and intelligent fruit and vegetable picking robot is of great significance to improving agricultural production efficiency and reducing labor costs.
[0003] Existing fruit and vegetable picking robots mostly use machine vision technology to identify and locate target fruits. Machine vision technology collects fruit and vegetable images through cameras, and uses image processing algorithms to identify and locate the fruits in the images, thereby guiding the robot to perform precise picking operations. However, different types of fruits and vegetables have significant differences in color, shape, texture, etc., which poses a great challenge to image recognition algorithms. Summary of the invention
[0004] In order to fill the gap in the market, the present invention provides a fruit and vegetable picking robot and a target recognition and picking method thereof.
[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a fruit and vegetable picking robot, comprising a walking mechanism, a rotating disk and a five-axis robotic arm, a storage box is placed in front of the bottom plate of the walking mechanism, and a storage box is arranged upward on the rear end box of the walking mechanism, a rotating disk is fixed in the middle of the walking mechanism, and a five-axis robotic arm is installed above the rotating disk, and a picking structure is installed at the front end of the five-axis robotic arm, a shearing structure is arranged at the front end of the picking structure, and a clamping assembly is installed above the shearing structure, the clamping assembly includes a clamping plate, a rotating shaft is installed below the clamping plate, and an elastic pad is installed above the rotating shaft.
[0006] Furthermore, a camera is installed below the picking structure, and an independent camera is installed on the rotating disk.
[0007] Furthermore, a disc spring is installed at the connection between the clamping assembly and the picking structure, and two ends of the disc spring are respectively connected to the picking structure and the clamping assembly.
[0008] An image recognition algorithm for a fruit and vegetable picking robot comprises the following steps: Step 1, image acquisition: acquiring images of target fruits and vegetables through a camera on the robot;
[0009] Step 2: Preprocessing: Preprocess the collected images, including denoising, filtering, etc., to improve image quality;
[0010] Step 3: Color space conversion: Convert the image in RGB color space to a color space that is more suitable for processing, such as HSV, CIELAB, etc., to enhance the distinguishability of color features;
[0011] Step 4: Feature extraction: Extract the color, shape, and texture features of fruits and vegetables according to the algorithm type;
[0012] Step 5: Identify candidate regions: Use superpixel segmentation and other methods to segment the image into multiple candidate regions to reduce the amount of computation required for subsequent processing; identify candidate regions that may contain fruits and vegetables;
[0013] Step 6: Feature classification: Classify the features in the candidate area to distinguish fruits from other objects. Linear classifiers, support vector machines, and deep learning methods can be used for classification.
[0014] Step 7: Target positioning: According to the classification results, determine the position of the fruit in the image. For apples, the exact position of the fruit can be determined by methods such as circle fitting. For irregularly shaped fruits and vegetables such as cucumbers, more complex positioning algorithms may be required.
[0015] Step 8: De-noising and post-processing: De-noise the positioning results to remove misidentified noise points; perform necessary post-processing, such as contour smoothing, size correction, etc., to improve positioning accuracy;
[0016] Step 9: Result output: The recognition result is output to the control system of the picking robot to guide the robotic arm to perform picking operations.
[0017] Furthermore, the algorithm can realize the recognition and positioning of a variety of fruits and vegetables. The specific algorithm includes the following three main modes:
[0018] Recognition algorithm based on color features: For fruits with significant differences in surface color and background, the algorithm analyzes the discrimination degree of different color factors in the fruit image and selects color factors that reflect the difference between the fruit and the background to form a multi-channel image; further, the algorithm uses sparse convolution kernels to comprehensively utilize pixel information within the neighborhood range to achieve segmentation of multi-channel images; sparse convolution kernels sample neighborhood pixels at intervals to reduce repeated operations in the convolution process while ensuring segmentation effects, thereby improving operation speed;
[0019] Recognition algorithm based on shape features: For fruits with uneven color distribution and easily affected by light, but whose shapes are not sensitive to light, the algorithm first determines the candidate area based on the color and texture features of the fruit, and then further identifies and locates the fruit within the candidate area based on the shape features of the fruit; the algorithm uses multi-scale sliding windows and directional gradient histogram operators to describe shape features, and uses a support vector machine classifier for shape recognition, effectively improving the recognition and positioning accuracy of the fruit;
[0020] Instance segmentation algorithm based on DCNN deep convolutional neural network: For fruits with similar color to branches and leaves and special shapes, such as cucumbers, the algorithm uses the Mask RCNN framework based on DCNN for identification and positioning; this framework can perform pixel-level segmentation to meet the requirements of high-precision positioning of fruits; according to the color and shape characteristics of cucumbers, the algorithm customizes the framework structure and anchor frame parameters, integrates artificial features into the framework, and improves the recognition and positioning accuracy as well as the efficiency of the algorithm operation.
[0021] Furthermore, the algorithm also includes adopting a method based on multi-channel image construction to combine the R, G, and B basic color components of the original image with the newly added nonlinear component S2 into a four-channel image to improve image recognition accuracy; using the LDA linear discriminant analysis algorithm to determine the elements in the convolution kernel of size 3*3*4, and training the sample image blocks through a linear classifier to make the convolution operation more accurate in classifying fruit pixels and non-fruit pixels.
[0022] Furthermore, the algorithm also includes the introduction of sparse convolution kernel design, which reduces the computational complexity of convolution operations and improves the algorithm processing speed by reducing repeated calculations of image pixels; the element spacing distribution of the sparse convolution kernel ensures that a high image recognition accuracy is maintained while reducing the computational complexity.
[0023] Furthermore, the algorithm also includes the use of superpixel segmentation technology to improve the accuracy of edge preservation in the fruit area through iterative optimization and enhanced connectivity processing; setting different superpixel block numbers K values and adjusting the image segmentation effect according to actual needs to achieve the best fruit recognition effect.
[0024] Furthermore, the algorithm also includes integrating a color-based salient target detection method, combined with the HSI color space component, to improve the recognition efficiency of target fruits and vegetables; applying the OTSU automatic threshold segmentation method to segment the salient map to achieve effective separation of the foreground and background in the fruit and vegetable image.
[0025] Furthermore, the algorithm also includes the use of a frequency-domain-based visual saliency target detection method to generate a frequency-coordinated visual saliency map to enhance the target fruit and vegetable features in the image; and combines the PCNN pulse-coupled neural network for image segmentation to improve the ability to recognize fruits and vegetables in complex backgrounds.
[0026] Compared with the prior art, the beneficial effects of the present invention are: by integrating the walking mechanism, the rotating disk and the five-axis robotic arm, the automation and intelligence of fruit and vegetable picking are realized, and the picking efficiency is significantly improved. Compared with manual picking, the robot can work continuously, reducing the problem of efficiency reduction due to fatigue. The design of the five-axis robotic arm enables the robot to flexibly adjust the picking posture to adapt to the picking needs of fruits and vegetables of different types and locations. At the same time, the cooperation of the walking mechanism and the rotating disk also enhances the robot's adaptability in complex field environments.
[0027] The fruit and vegetable picking robot of the present invention is also equipped with an advanced image recognition algorithm, which can accurately identify and locate the target fruits and vegetables, reducing the mistaken picking of immature or damaged fruits and vegetables. First, according to the color and shape characteristics of fruits and vegetables, a multi-color channel image is constructed, and a sparse convolution kernel is designed to make full use of the color information of each pixel and its neighboring pixels. Then, combined with a linear classifier and morphological denoising technology, accurate recognition and positioning of fruits in the candidate area are achieved. In addition, by modifying the aspect ratio of the anchor frame and reducing the number of FPN branches and other optimization measures, the recognition accuracy and efficiency of the fruit are improved.
[0028] During the picking process, robots replace humans to enter orchards or farmlands, reducing potential harm to pickers due to environmental factors and improving operational safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a three-dimensional schematic diagram of the structure of the present invention;
[0030] Figure 2 A schematic diagram of the installation and clamping assembly of the structure of the present invention;
[0031] Figure 3 Schematic diagram of the clamping assembly of the structure of the present invention.
[0032] In the figure: 1. walking mechanism; 2. rotating disk; 3. storage box; 4. five-axis robot arm; 5. picking structure; 6. clamping assembly; 61. clamping plate; 62. elastic pad; 63. rotating shaft; 7. camera; 8. shearing structure. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] Example 1: Please refer to Figure 1-3The present invention provides a technical solution: a fruit and vegetable picking robot, comprising a walking mechanism 1, a rotating disk 2 and a five-axis robotic arm 4, a storage box 3 is placed in front of a bottom plate of the walking mechanism 1, and a storage box 3 is arranged upward on the rear end box of the walking mechanism 1, a rotating disk 2 is fixed in the middle of the walking mechanism 1, and a five-axis robotic arm 4 is installed above the rotating disk 2, and a picking structure 5 is installed at the front end of the five-axis robotic arm 4, a shearing structure 8 is arranged at the front end of the picking structure 5, and a clamping assembly 6 is installed above the shearing structure 8, the clamping assembly 6 includes a clamping plate 61, and a rotating shaft 63 is installed below the clamping plate 61, and an elastic pad 62 is installed above the rotating shaft 63.
[0035] A camera 7 is installed below the picking structure 5 , and an independent camera is installed on the rotating disk 2 .
[0036] A disc spring is installed at the connection between the clamping assembly 6 and the picking structure 5, and the two ends of the disc spring are respectively connected to the picking structure 5 and the clamping assembly 6.
[0037] The walking mechanism 1 controls the wheels of the walking mechanism to move forward, backward, turn, etc. through the motor drive system, so as to realize the flexible movement of the robot in complex terrain. The rotating disk 2 is driven by the motor to rotate, so that the five-axis robot arm 4 can rotate 360 degrees around the center of the rotating disk, and flexibly adjust the picking angle and direction. The five-axis robot arm 4 realizes the flexible movement and positioning of the picking structure in three-dimensional space through the coordinated movement of multiple joints. Each joint is driven by a motor. By controlling the rotation angle and speed of the motor, the precise control of the robot arm is realized. The picking structure 5 drives the shearing structure 8 through the motor to perform the shearing action. At the same time, the clamping component 6 realizes the clamping and fixation of fruits and vegetables through the cooperation of the rotating shaft 63 and the elastic pad 62, ensuring that the fruits and vegetables will not fall or be damaged during the shearing process.
[0038] The clamping plate 61 is connected to the picking structure through a rotating shaft 63. During the clamping process, the clamping plate 61 can be rotated and opened without causing damage to the fruits and vegetables. The elastic pad provides appropriate buffering and clamping force to ensure that the fruits and vegetables remain stable during the picking process; the camera 7 captures the image of the fruits and vegetables through an optical lens, and transmits the image data to the image recognition system for processing and analysis, thereby achieving accurate identification and positioning of the fruits and vegetables. The disc spring transmits and disperses force between the clamping component and the picking structure through its elastic deformation ability.
[0039] When the robot starts working, the walking mechanism 1 drives the robot to move to the vicinity of the target fruits and vegetables; the camera 7 collects images of the target fruits and vegetables, and processes and analyzes them through an image recognition algorithm to determine the position and characteristics of the fruits and vegetables; the five-axis robotic arm 4 adjusts its position and posture according to the recognition results, so that the picking structure 5 and the shearing structure 8 are aligned with the target fruits and vegetables; the clamping assembly 6 opens the clamping plate 61, contacts and fixes the fruits and vegetables through the elastic pad 62; the shearing structure 8 performs a shearing action to complete the picking of the fruits and vegetables; the picked fruits and vegetables are placed in the storage box 3, and the robot continues to perform the next picking task.
[0040] An image recognition algorithm for a fruit and vegetable picking robot includes the following steps: Step 1: Image acquisition: Step 1: Image acquisition
[0041] First, the camera 7 installed on the fruit and vegetable picking robot collects the target fruit and vegetable images in the orchard or vegetable field in real time, and the image information captured by the camera is transmitted to the image recognition processing system;
[0042] Step 2: Preprocessing
[0043] The collected images are preprocessed, mainly including denoising and filtering operations. Denoising is to remove noise interference caused by environmental factors in the image; filtering is used to smooth the image and improve the overall quality of the image, laying a good foundation for subsequent processing.
[0044] Step 3: Color space conversion
[0045] The preprocessed images are converted from RGB color space to color spaces that are more suitable for feature analysis. These color spaces can better distinguish color features, enhance the contrast between fruits and vegetables and the background, and facilitate subsequent feature extraction.
[0046] Step 4: Feature extraction
[0047] According to the needs of specific algorithms, the color, shape, texture and other features of fruits and vegetables are extracted from the converted color space. These features will serve as the basis for subsequent classification and positioning.
[0048] Step 5: Candidate region identification
[0049] The image is segmented into multiple candidate regions using superpixel segmentation technology. This step can reduce the amount of computation required for subsequent processing and identify candidate regions that may contain fruits and vegetables. Superpixel segmentation technology improves the accuracy of edge preservation in the fruit region through iterative optimization and enhanced connectivity processing.
[0050] Step 6: Feature Classification
[0051] Classify the features in the candidate area to distinguish fruits from other non-target objects. The classification process can be performed using linear classifiers, support vector machines, or deep learning methods. For fruits and vegetables in complex backgrounds, deep learning methods such as DCNN deep convolutional neural networks can provide more accurate classification results.
[0052] Step 7: Target positioning
[0053] According to the classification results, the specific position of the fruit in the image is determined. For fruits with regular shapes, accurate positioning can be achieved through methods such as circle fitting; while for fruits and vegetables with irregular shapes, more complex positioning algorithms are required.
[0054] Step 8: Denoising and post-processing
[0055] The positioning results are denoised to remove noise points caused by misidentification. At the same time, necessary post-processing is performed, such as contour smoothing and size correction, to improve positioning accuracy and image quality.
[0056] Step 9: Output results
[0057] The final recognition result is output to the control system of the fruit and vegetable picking robot; the control system guides the five-axis robot arm 4 and the picking structure 5 to perform precise picking operations based on the recognition result.
[0058] The algorithm can realize the recognition and positioning of a variety of fruits and vegetables. The specific algorithm includes the following three main modes:
[0059] Recognition algorithm based on color features: For fruits with significant differences in surface color and background, the algorithm analyzes the discrimination degree of different color factors in the fruit image and selects color factors that reflect the difference between the fruit and the background to form a multi-channel image; further, the algorithm uses sparse convolution kernels to comprehensively utilize pixel information within the neighborhood range to achieve segmentation of multi-channel images; sparse convolution kernels sample neighborhood pixels at intervals to reduce repeated operations in the convolution process while ensuring segmentation effects, thereby improving operation speed;
[0060] Recognition algorithm based on shape features: For fruits with uneven color distribution and easily affected by light, but whose shapes are not sensitive to light, the algorithm first determines the candidate area based on the color and texture features of the fruit, and then further identifies and locates the fruit within the candidate area based on the shape features of the fruit; the algorithm uses multi-scale sliding windows and directional gradient histogram operators to describe shape features, and uses a support vector machine classifier for shape recognition, effectively improving the recognition and positioning accuracy of the fruit;
[0061] Instance segmentation algorithm based on DCNN deep convolutional neural network: For fruits with similar color to branches and leaves and special shapes, such as cucumbers, the algorithm uses the Mask RCNN framework based on DCNN for identification and positioning; this framework can perform pixel-level segmentation to meet the requirements of high-precision positioning of fruits; according to the color and shape characteristics of cucumbers, the algorithm customizes the framework structure and anchor frame parameters, integrates artificial features into the framework, and improves the recognition and positioning accuracy as well as the efficiency of the algorithm operation.
[0062] The algorithm also includes a method based on multi-channel image construction to combine the R, G, and B basic color components of the original image with the newly added nonlinear component S2 into a four-channel image to improve image recognition accuracy; the LDA linear discriminant analysis algorithm is used to determine the elements in the convolution kernel of size 3*3*4, and the sample image blocks are trained through a linear classifier to make the convolution operation more accurate in classifying fruit pixels and non-fruit pixels.
[0063] The algorithm also includes the introduction of sparse convolution kernel design, which reduces the amount of computation required for convolution operations and improves the algorithm processing speed by reducing repeated calculations of image pixels. The element spacing distribution of the sparse convolution kernel ensures that a high image recognition accuracy rate is maintained while reducing the amount of computation.
[0064] The algorithm also includes the use of superpixel segmentation technology to improve the accuracy of edge preservation in the fruit area through iterative optimization and enhanced connectivity processing; setting different superpixel block numbers K values and adjusting the image segmentation effect according to actual needs to achieve the best fruit recognition effect.
[0065] The algorithm also includes integrating a color-based salient target detection method and combining it with the HSI color space component to improve the recognition efficiency of target fruits and vegetables; applying the OTSU automatic threshold segmentation method to segment the salient map to achieve effective separation of the foreground and background in fruit and vegetable images.
[0066] The algorithm also includes the use of a frequency-domain-based visual saliency target detection method to generate a frequency-coordinated visual saliency map to enhance the target fruit and vegetable features in the image; and combines the PCNN pulse-coupled neural network for image segmentation to improve the ability to recognize fruits and vegetables in complex backgrounds.
[0067] The algorithm also includes the introduction of a histogram statistical feature extraction method, using the LBP local binary pattern operator to describe the texture of the image, enhancing the robustness of the algorithm, and further improving the comprehensiveness and accuracy of fruit and vegetable identification by fusing multiple feature information such as color, texture, and shape.
[0068] Embodiment 2: Based on Embodiment 1, this embodiment further optimizes the image recognition algorithm of the fruit and vegetable picking robot and demonstrates its specific application in the recognition of different fruit and vegetable varieties.
[0069] Multi-feature fusion:
[0070] In the feature extraction stage, in addition to color, shape and texture features, deep features are also introduced. Through pre-trained deep convolutional neural networks such as ResNet and VGG, high-level semantic information of the image is extracted and combined with traditional features to form multi-feature fusion to improve recognition accuracy.
[0071] Adaptive threshold segmentation: In view of the impact of illumination changes on image quality, an adaptive threshold segmentation method is used to dynamically adjust the segmentation threshold to ensure that the foreground and background can be effectively separated under different illumination conditions.
[0072] In the candidate region identification and feature classification stage, contextual information is introduced, such as the position of fruits and vegetables on the tree or vine, and the color distribution of the surrounding environment. By constructing a conditional CRF or graph neural network GNN model, the robustness and accuracy of recognition are improved.
[0073] Apple ID:
[0074] For brightly colored and regular-shaped fruits such as apples, the algorithm mainly relies on color and shape features for recognition. Through sparse convolution kernels and circle fitting algorithms, the position of the apple can be quickly located and the fruit area can be accurately segmented.
[0075] Cucumber Identification:
[0076] The color of cucumbers is similar to that of branches and leaves and their shape is special, so the Mask RCNN framework based on DCNN is used for identification and positioning. Through customized network structure and anchor frame parameters, high-precision pixel-level segmentation of cucumbers is achieved. At the same time, combined with superpixel segmentation technology, the accuracy of maintaining cucumber edges is improved.
[0077] Strawberry identification:
[0078] Strawberries are small in size, diverse in color, and easily obscured. The algorithm uses a multi-scale sliding window and HOG feature descriptor combined with a support vector machine (SVM) classifier for recognition. By introducing deep features and multi-feature fusion, the recognition rate of strawberries in complex backgrounds is improved.
[0079] In order to verify the effectiveness of the optimized image recognition algorithm, we conducted field tests in multiple orchards. The experimental results show that the optimized algorithm is superior to the traditional algorithm in terms of recognition accuracy, positioning accuracy and robustness. In particular, the algorithm can still maintain stable and efficient recognition performance under conditions of large lighting changes and complex backgrounds.
[0080] The final recognition result is output to the control system of the fruit and vegetable picking robot. The control system guides the five-axis robot arm 4 and the picking structure 5 to perform precise picking operations according to the recognition result.
[0081] Embodiment 3: Based on the inheritance of Embodiment 1 and Embodiment 2, this embodiment introduces deep learning and multi-feature fusion technology to further improve the accuracy and robustness of the image recognition algorithm of the fruit and vegetable picking robot.
[0082] Introduction of deep learning framework: Application of DCNN (deep convolutional neural network): Deep convolutional neural networks (such as ResNet and VGG) are used as the main framework for image recognition. These networks can automatically extract high-level features in images through multi-layer convolution and pooling operations, and effectively deal with complex and changeable fruit and vegetable images.
[0083] Transfer learning: Use models pre-trained on large datasets (such as ImageNet) for transfer learning. By fine-tuning network parameters to adapt them to the specific task of fruit and vegetable recognition, training time can be reduced and recognition efficiency can be improved.
[0084] Multi-feature fusion strategy: Fusion of color features and shape features: Combining the advantages of the recognition algorithm based on color features and the recognition algorithm based on shape features, the features extracted by the two are fused. By weighting or concatenating, the color features and shape features are combined into a comprehensive feature vector for subsequent classification and positioning.
[0085] Supplementation of texture features: For some fruits and vegetables with obvious surface texture features, such as the seed points on the surface of strawberries, texture feature extraction methods (such as gray-level co-occurrence matrix and local binary pattern) are introduced to further enrich the feature representation and improve the recognition accuracy.
[0086] Algorithm Flow
[0087] Image acquisition: As in the first embodiment, the image of the target fruit or vegetable is acquired by the camera 7 on the robot.
[0088] Preprocessing: Perform denoising, filtering and other preprocessing operations on the collected images to improve image quality.
[0089] Feature extraction:
[0090] Use DCNN to automatically extract high-level features of images.
[0091] Combine traditional methods to extract color, shape and texture features.
[0092] These features are fused to form a comprehensive feature vector.
[0093] Candidate region identification: Use superpixel segmentation or region proposal network (RPN) methods to identify candidate regions that may contain fruits and vegetables from the image.
[0094] Feature classification and target positioning:
[0095] Use deep learning classifiers (such as fully connected layers and softmax classifiers) to classify the features within the candidate regions and distinguish fruits from other objects.
[0096] According to the classification results, the exact position of the fruit in the image is determined by combining shape fitting, contour detection and other methods.
[0097] Denoising and post-processing: Denoise the positioning results to remove misidentified noise points; perform necessary post-processing, such as contour smoothing, size correction, etc., to improve positioning accuracy.
[0098] Result output: The recognition result is output to the control system of the picking robot to guide the robotic arm to perform picking operations.
[0099] A large number of experiments were conducted in an actual orchard environment to verify the effectiveness and superiority of the image recognition algorithm proposed in this embodiment. Compared with traditional methods and single feature recognition algorithms, this algorithm shows significant improvements in recognition accuracy, robustness, and processing speed. At the same time, by adjusting the network structure and parameters, it can further adapt to the recognition needs of fruits and vegetables of different types and growth environments.
Claims
1. A fruit and vegetable picking robot, comprising a walking mechanism (1), a rotating disk (2) and a five-axis mechanical arm (4), characterized in that: A storage box (3) is placed in front of the bottom plate of the walking mechanism (1), and a storage box (3) is arranged upward on the rear end box of the walking mechanism (1); a rotating disk (2) is fixed in the middle of the walking mechanism (1), and a five-axis robot arm (4) is installed above the rotating disk (2), and a picking structure (5) is installed at the front end of the five-axis robot arm (4); a shearing structure (8) is arranged at the front end of the picking structure (5), and a clamping assembly (6) is installed above the shearing structure (8); the clamping assembly (6) includes a clamping plate (61), and a rotating shaft (63) is installed below the clamping plate (61), and an elastic pad (62) is installed above the rotating shaft (63).
2. A fruit and vegetable picking robot according to claim 1, characterized in that: A camera (7) is installed below the picking structure (5), and an independent camera is installed on the rotating disk (2).
3. The fruit and vegetable picking robot according to claim 2, characterized in that: A disc spring is installed at the connection between the clamping assembly (6) and the picking structure (5), and the two ends of the disc spring are respectively connected to the picking structure (5) and the clamping assembly (6).
4. An image recognition algorithm for a fruit and vegetable picking robot according to any one of claims 1 to 3, characterized in that: The method comprises the following steps: Step 1, image acquisition: acquiring images of target fruits and vegetables through a camera (7) on the robot; Step 2: Preprocessing: Preprocess the collected images, including denoising, filtering, etc., to improve image quality; Step 3: Color space conversion: Convert the image in RGB color space to a color space that is more suitable for processing, such as HSV, CIELAB, etc., to enhance the distinguishability of color features; Step 4: Feature extraction: Extract the color, shape, and texture features of fruits and vegetables according to the algorithm type; Step 5: Identify candidate regions: Use superpixel segmentation and other methods to segment the image into multiple candidate regions to reduce the amount of computation required for subsequent processing; identify candidate regions that may contain fruits and vegetables; Step 6: Feature classification: Classify the features in the candidate area to distinguish fruits from other objects. Linear classifiers, support vector machines, and deep learning methods can be used for classification. Step 7: Target positioning: According to the classification results, determine the position of the fruit in the image. For apples, the exact position of the fruit can be determined by methods such as circle fitting. For irregularly shaped fruits and vegetables such as cucumbers, more complex positioning algorithms may be required. Step 8: De-noising and post-processing: De-noise the positioning results to remove misidentified noise points; perform necessary post-processing, such as contour smoothing, size correction, etc., to improve positioning accuracy; Step 9: Result output: The recognition result is output to the control system of the picking robot to guide the robotic arm to perform picking operations.
5. The image recognition algorithm of the fruit and vegetable picking robot according to claim 4, characterized in that: The algorithm can realize the recognition and positioning of a variety of fruits and vegetables. The specific algorithm includes the following three main modes: Recognition algorithm based on color features: For fruits with significant differences in surface color and background, the algorithm analyzes the discrimination degree of different color factors in the fruit image and selects color factors that reflect the difference between the fruit and the background to form a multi-channel image; further, the algorithm uses sparse convolution kernels to comprehensively utilize pixel information within the neighborhood range to achieve segmentation of multi-channel images; sparse convolution kernels sample neighborhood pixels at intervals to reduce repeated operations in the convolution process while ensuring segmentation effects, thereby improving operation speed; Recognition algorithm based on shape features: For fruits with uneven color distribution and easily affected by light, but whose shapes are not sensitive to light, the algorithm first determines the candidate area based on the color and texture features of the fruit, and then further identifies and locates the fruit within the candidate area based on the shape features of the fruit; the algorithm uses multi-scale sliding windows and directional gradient histogram operators to describe shape features, and uses a support vector machine classifier for shape recognition, effectively improving the recognition and positioning accuracy of the fruit; Instance segmentation algorithm based on DCNN deep convolutional neural network: For fruits with similar color to branches and leaves and special shapes, such as cucumbers, the algorithm uses the Mask RCNN framework based on DCNN for identification and positioning; this framework can perform pixel-level segmentation to meet the requirements of high-precision positioning of fruits; according to the color and shape characteristics of cucumbers, the algorithm customizes the framework structure and anchor frame parameters, integrates artificial features into the framework, and improves the recognition and positioning accuracy as well as the efficiency of the algorithm operation.
6. The image recognition algorithm of the fruit and vegetable picking robot according to claim 5, characterized in that: The algorithm also includes a method based on multi-channel image construction to combine the R, G, B basic color components of the original image with the newly added nonlinear component S2 into a four-channel image to improve the image recognition accuracy; The LDA linear discriminant analysis algorithm is used to determine the elements in the convolution kernel of 3*3*4 size, and the sample image blocks are trained by a linear classifier to make the convolution operation more accurate in classifying fruit pixels and non-fruit pixels.
7. The image recognition algorithm of the fruit and vegetable picking robot according to claim 5, characterized in that: The algorithm also includes the introduction of sparse convolution kernel design, which reduces the amount of computation required for convolution operations and improves the algorithm processing speed by reducing repeated calculations of image pixels. The element spacing distribution of the sparse convolution kernel ensures that a high image recognition accuracy rate is maintained while reducing the amount of computation.
8. The image recognition algorithm of the fruit and vegetable picking robot according to claim 5, characterized in that: The algorithm also includes the use of superpixel segmentation technology to improve the accuracy of edge preservation in the fruit area through iterative optimization and enhanced connectivity processing; setting different superpixel block numbers K values and adjusting the image segmentation effect according to actual needs to achieve the best fruit recognition effect.
9. The image recognition algorithm of the fruit and vegetable picking robot according to claim 5, characterized in that: The algorithm also includes integrating a color-based salient target detection method and combining it with the HSI color space component to improve the recognition efficiency of target fruits and vegetables; applying the OTSU automatic threshold segmentation method to segment the salient map to achieve effective separation of the foreground and background in fruit and vegetable images.
10. The image recognition algorithm of the fruit and vegetable picking robot according to claim 5, characterized in that: The algorithm also includes using a frequency-domain-based visual saliency target detection method to generate a frequency-coordinated visual saliency map to enhance the target fruit and vegetable features in the image; Combined with PCNN pulse coupled neural network for image segmentation, the ability to recognize fruits and vegetables in complex backgrounds is improved.
Citation Information
Patent Citations
Orchard intelligent system for fruit automatic identification, classification and picking
CN109220226A
Intelligent picking method and device based on neural network, equipment and storage medium
CN112712128A
Apple identification method fusing fruit features and deep convolutional neural network
CN113255434A
Hot pepper picking manipulator based on visual identification and end executing mechanism thereof
CN116276976A
Fruit picking system based on ai technology
CN117044497A
Cited By
Image recognition method for automatic pepper picking machine
CN120014471A
An image recognition method for a chili automatic picking machine
CN120014471B
Efficient eggplant picking and sorting device and picking and sorting method thereof
CN120283544A
Method and device for predicting productivity of fruits in greenhouse, electronic equipment and readable storage medium
CN121147911A
