Vibration type blueberry harvester control system and method based on feedback control

Abstract

The invention belongs to the field of automatic picking, provides a vibration type blueberry harvester control system and method based on feedback control, and integrates machine vision and blueberry picking technologies. The camera is installed above the conveyor belt and collects images of the conveyor belt in real time, and machine vision utilizes a high-precision algorithm to accurately recognize blueberries and calculate the proportion of impurities, so that the speed of the conveyor belt is adjusted, and then the impurity processing time of the fan is reserved. The camera is installed above the blueberry harvesting disc, blueberry images in the harvesting disc are collected in real time, the proportion of immature fruits is judged through the convolutional neural network, and when the proportion is too large, the amplitude and frequency of the vibration picking device are reduced so as to reduce the falling rate of the immature fruits. In addition, the system also has remote monitoring and fault self-diagnosis functions, so that the management personnel can conveniently monitor the operation condition in real time.

Description

Technical Field

[0001] The invention belongs to the technical field of automatic picking, and in particular relates to a control system and method of a vibration type blueberry harvester based on feedback control. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of blueberry planting, blueberry picking has become a time-consuming and laborious part of blueberry production. Due to the characteristics of small fruit, high yield and strong seasonality, the traditional manual picking method is not only inefficient but also costly. In recent years, some blueberry harvesters have been introduced, but most of them have problems such as low intelligence, low picking efficiency and high fruit damage rate. Therefore, it is particularly important to develop an efficient and intelligent blueberry harvesting intelligent control system.

[0004] For the existing fruit harvester structure, the reference application number is 202211227359.3, and the name is a patent for a traction-type liftable V-shaped vibrating small berry harvester, which discloses that its harvesting part is mainly composed of a gantry frame, a shrub guide device, a vibrating picking device, a conveying device, and other parts. The main mechanism for generating vibration is the vibrating picking device, including an inertial vibration power device, a vibrating shaft assembly, and a vibrating disk assembly; the harvester relies heavily on fixed frequency operation, resulting in unstable harvesting performance on fruit trees of different shapes. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a control system and method for a vibration blueberry harvester based on feedback control. The present invention adopts multi-information fusion in the blueberry harvester to optimize the parameters of various parts of the blueberry harvester, uses sensors to capture the specific conditions of the blueberry harvester when working, and feeds back to the decision-making system. The decision-making system accurately controls the motor to make the blueberry harvester reach a certain position.

[0006] According to some embodiments, a first solution of the present invention provides a control method of a vibration-type blueberry harvester based on feedback control, which adopts the following technical solution: A control method for a vibration blueberry harvester based on feedback control, comprising: Obtain fruit tree images for preprocessing, fit the crown curve based on the preprocessed fruit tree images and determine the height and angle of the fruit trees, and adjust the harvesting angle of the harvester according to the height and angle of the fruit trees; Determining initial values ​​of the amplitude and frequency of a vibration picking device in the harvester according to a fixed voltage of the harvester, thereby performing vibration harvesting on blueberries; Obtaining and preprocessing the blueberry image in the fruit collection device, and adjusting the amplitude and frequency of the vibration picking device in the harvester according to the proportion of unripe fruits in the preprocessed blueberry image; Obtaining and preprocessing a blueberry image of a conveyor device in a harvester, and adjusting a running speed of a conveyor belt according to a level of impurity in the preprocessed blueberry image of the conveyor device; After obtaining and preprocessing the harvested fruit tree image, the initial values ​​of the amplitude and frequency of the vibration picking device when harvesting the next fruit tree are adjusted according to the proportion of ripe fruits in the preprocessed harvested fruit tree image.

[0007] Furthermore, the fruit tree image is obtained and preprocessed, and the crown curve is fitted based on the preprocessed fruit tree image to determine the height and angle of the fruit tree, specifically: Obtain the fruit tree image and perform grayscale processing to generate a grayscale image; The real center line of the fruit tree in the grayscale image is fitted using Hough transformation to obtain the fitted center line of the fruit tree; determining the angle of the fruit tree based on the slope of the fitted centerline of the fruit tree and the slope of the true centerline of the fruit tree; The lateral position deviation of the fruit tree is calculated based on the data of the true center line of the fruit tree, and the maximum value of the lateral position deviation of the fruit tree is taken as the height of the fruit.

[0008] Furthermore, the harvesting angle of the harvester is adjusted according to the height and angle of the fruit tree, specifically: The picking frame in the harvester is adjusted to rise or fall based on the height of the fruit tree to match the height of the blueberry plant; wherein the picking frame is a V-shaped frame composed of two vibrating picking devices; According to the angle of the fruit tree, the upper fixed position is changed to adjust the inclination angle of the vibration picking device so that the inclination angle of the vibration picking device matches the growth direction of the blueberry fruit.

[0009] Furthermore, the amplitude and frequency of the vibration harvesting device in the harvester are adjusted according to the proportion of unripe fruits in the preprocessed blueberry image, specifically: Use the pre-trained image object recognition classification model to determine the proportion of unripe fruits in the pre-processed blueberry image; If the proportion of unripe fruits is greater than the set threshold, the amplitude and frequency of the vibrating picking device in the harvester are reduced to reduce the rate of unripe fruit falling off.

[0010] Furthermore, the pre-trained image target recognition and classification model is used to determine the proportion of unripe fruits in the pre-processed blueberry image, specifically: The deep learning-based object detection network is used as an image object recognition and classification model to identify and classify the processed blueberry images; According to the identification and classification results, the number of mature classification results and immature classification results is determined, and the proportion of immature classification results to all identification and classification results is calculated to obtain the proportion of immature fruits.

[0011] Furthermore, the operation speed of the conveyor belt is adjusted according to the impurity rate in the pre-processed blueberry image of the conveyor device, specifically: Based on deep learning algorithm segmentation, the proportion of impurities in the pre-processed conveyor blueberry image was identified; If the impurity ratio is greater than the set impurity ratio threshold, the conveyor belt speed is reduced.

[0012] Furthermore, the deep learning algorithm-based segmentation identifies the proportion of impurities in the pre-processed blueberry image of the conveyor, specifically: Performing feature extraction on the preprocessed blueberry image of the conveyor device to obtain a blueberry mask image and an impurity mask image; Calculate the number of pixels based on the blueberry mask image and the impurity mask image to obtain the impurity mask image area and the blueberry mask image area; Based on the area of ​​the impurity mask image divided by the sum of the area of ​​the impurity mask image and the area of ​​the blueberry mask image, the proportion of the impurity mask image, that is, the proportion of impurities in the preprocessed blueberry image of the conveyor device, is obtained.

[0013] According to some embodiments, the second solution of the present invention provides a control system of a vibration-type blueberry harvester based on feedback control, which adopts the following technical solution: A control system for a vibration-type blueberry harvester based on feedback control, comprising: A harvesting angle adjustment module is configured to obtain a fruit tree image for preprocessing, fit a crown curve based on the preprocessed fruit tree image and determine the height and angle of the fruit tree, and adjust the harvesting angle of the harvester according to the height and angle of the fruit tree; A vibration picking device setting module is configured to determine the initial values ​​of the amplitude and frequency of the vibration picking device in the harvester according to the fixed voltage of the harvester, so as to perform vibration harvesting on blueberries; A dynamic adjustment module for a vibrating picking device is configured to obtain and pre-process the blueberry image in the fruit collecting device, and adjust the amplitude and frequency of the vibrating picking device in the harvester according to the proportion of unripe fruits in the pre-processed blueberry image; The conveyor device dynamic adjustment module is configured to obtain and pre-process the blueberry image of the conveyor device in the harvester, and adjust the running speed of the conveyor device according to the impurity rate of the pre-processed blueberry image of the conveyor device; The vibration picking device refresh module is configured to obtain the harvested fruit tree image and preprocess it, and then adjust the initial values ​​of the amplitude and frequency of the vibration picking device when harvesting the next fruit tree according to the proportion of ripe fruits in the preprocessed harvested fruit tree image.

[0014] Furthermore, the harvest angle adjustment module utilizes a camera disposed on the outside of the blueberry harvester to obtain an image of each fruit tree during the picking process of the blueberry harvester.

[0015] Furthermore, the dynamic adjustment module of the vibration picking device uses a camera disposed above the fruit collecting device to obtain an image of blueberries in the fruit collecting device; The conveyor device dynamic adjustment module uses a camera arranged above the conveyor device to obtain a blueberry image of the conveyor device.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention detects the number of blueberries harvested in the fruit collection device in real time to determine the picking efficiency of the vibration picking device and the proportion of immature fruits. When it is greater than a threshold, the amplitude and frequency of the vibration picking device are reduced to reduce the shedding rate of immature fruits. The number of pulses of the motor is used to accurately control the amplitude and frequency of the vibration picking device to improve the picking efficiency. In addition, the fruit trees that have been harvested are detected in real time. When the proportion of mature fruits on the fruit trees after harvest exceeds a certain threshold, the initial values ​​of the amplitude and frequency of the vibration picking device are increased before the next fruit tree is picked to improve the net collection rate of the fruit.

[0017] The present invention adopts multi-information fusion in the blueberry harvester to optimize the parameters of various parts of the blueberry harvester, uses sensors to capture the specific conditions of the harvester when working, and feeds back to the decision-making system, and the decision-making system accurately controls the motor to make the harvester reach a certain position.

[0018] The present invention uses a camera to collect blueberry images on a conveyor in real time, identifies the blueberry images through a deep learning segmentation algorithm and calculates the proportion of impurities, sets a threshold, and if the proportion of impurities exceeds the threshold, reduces the speed of the conveyor, increases the time for the fan of the cleaning device to remove impurities, and slightly reduces the amplitude and speed of the vibrating picking device. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0020] Figure 1 This is a flow chart of a control method of a vibration-type blueberry harvester based on feedback control in Embodiment 1 of the present invention; Figure 2This is a schematic diagram of the angle of a blueberry tree fitted by Hough transform in the first embodiment of the present invention; Figure 3 This is a schematic diagram of the blueberry recognition effect of the image target recognition and classification model in the first embodiment of the present invention; Figure 4 is a schematic diagram of the overall structure of the blueberry harvester in the second embodiment of the present invention; Figure 5 This is the signal processing process of the execution model in the second embodiment of the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0023] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0025] Embodiment 1 like Figure 1 As shown, this embodiment provides a control method for a vibration blueberry harvester based on feedback control, comprising: Obtain fruit tree images for preprocessing, fit the crown curve based on the preprocessed fruit tree images and determine the height and angle of the fruit trees, and adjust the harvesting angle of the harvester according to the height and angle of the fruit trees; Determining initial values ​​of the amplitude and frequency of a vibration picking device in the harvester according to a fixed voltage of the harvester, thereby performing vibration harvesting on blueberries; Obtaining and preprocessing the blueberry image in the fruit collection device, and adjusting the amplitude and frequency of the vibration picking device in the harvester according to the proportion of unripe fruits in the preprocessed blueberry image; Obtaining and preprocessing a blueberry image of a conveyor device in a harvester, and adjusting the running speed of the conveyor device according to the impurity rate in the preprocessed blueberry image of the conveyor device; After obtaining and preprocessing the harvested fruit tree image, the initial values ​​of the amplitude and frequency of the vibration picking device when harvesting the next fruit tree are adjusted according to the proportion of ripe fruits in the preprocessed harvested fruit tree image.

[0026] The obtaining of the fruit tree image for preprocessing, fitting the crown curve based on the preprocessed fruit tree image and determining the height and angle of the fruit tree, specifically includes: Obtain the fruit tree image and perform grayscale processing to generate a grayscale image; The real center line of the fruit tree in the grayscale image is fitted using Hough transformation to obtain the fitted center line of the fruit tree; determining the angle of the fruit tree based on the slope of the fitted centerline of the fruit tree and the slope of the true centerline of the fruit tree; The lateral position deviation of the fruit tree is calculated based on the data of the true center line of the fruit tree, and the maximum value of the lateral position deviation of the fruit tree is taken as the height of the fruit.

[0027] The classic Hough transform is used to fit the center line of the blueberry branch, and the evaluation index is the center fitting time. , angular deviation and lateral position deviation , the calculation method is as follows Figure 2 As shown, the specific calculation formula is: ; ; ; Where: Indicates the first True center line; Represents the total number of center lines in the blueberry tree image: Indicates the number of images; Indicates The fitting time of the true center line, in ms; Indicates the image The slope of the true center line of the blueberry tree; Indicates the image The slope of the fitted center line of the blueberry tree; Indicates the image The horizontal position deviation of the blueberry tree, in pixels.

[0028] It will be appreciated that the above angular deviations are taken as the angles of the fruit trees.

[0029] Adjust the harvesting angle of the harvester according to the height and angle of the fruit trees, specifically: The picking frame in the harvester is adjusted to rise or fall based on the height of the fruit tree to match the height of the blueberry plant; wherein the picking frame is a V-shaped frame composed of two vibrating picking devices; According to the angle of the fruit tree, the upper fixed position is changed to adjust the inclination angle of the vibration picking device so that the inclination angle of the vibration picking device matches the growth direction of the blueberry fruit.

[0030] After obtaining the parameters related to the height and angle of the fruit tree, the harvester will start the up and down lifting motors of the lifting system to raise or lower the V-shaped frame composed of two vibrating picking devices - the picking frame. By adjusting the height of the picking frame to match the height of the blueberry plant, it can reduce fruit loss and improve picking efficiency.

[0031] The harvester's picking frame is V-shaped, with an adjustable servo installed at the bottom. The angle information transmitted can adjust the picking angle and adjust the angle of the vibrating picking device to match the growth direction of the blueberry fruit.

[0032] The amplitude and frequency of the vibration picking device in the harvester are adjusted according to the proportion of unripe fruits in the pre-processed blueberry image, specifically: Use the pre-trained image object recognition classification model to determine the proportion of unripe fruits in the pre-processed blueberry image; If the proportion of unripe fruits is greater than the set threshold, the amplitude and frequency of the vibrating picking device in the harvester are reduced to reduce the rate of unripe fruit falling off.

[0033] The pre-trained image target recognition and classification model is used to determine the proportion of unripe fruits in the pre-processed blueberry image. The model recognition effect is as follows: Figure 3 As shown, specifically: 1. Image Preprocessing Obtain a large number of blueberry image samples, ensure that the blueberry images meet the model input requirements, and perform preprocessing, which usually includes adjusting the image size, normalizing pixel values, etc.

[0034] Resize images: Resize images to the required size of 640x640 pixels for model input.

[0035] Normalization: Scale pixel values ​​to between 0 and 1, or the model is trained to use a scale of [0, 255] to [-1, 1].

[0036] After preprocessing, a blueberry image sample dataset is obtained.

[0037] 2. Model selection or fine-tuning The deep learning target detection network (YOLO) is used as the image target recognition and classification model.

[0038] The blueberry image sample dataset is labeled using the labels of ripe blueberries and unripe blueberries, and the labeled blueberry image sample dataset is divided into a training set, a test set, and a validation set; The training set is used for model training to fine-tune the parameters of the image object recognition and classification model to adapt to the current blueberry image classification, and the validation set and test set are used for verification and testing, and finally a trained image object recognition and classification model is obtained to distinguish between ripe and unripe blueberries.

[0039] 3. Object Detection and Classification If there are many blueberries in the image and they are unevenly distributed, use the object detection model YOLO to locate each blueberry and classify its ripeness.

[0040] 4. Calculation of results Calculate the percentage of unripe blueberries based on the model output.

[0041] For each detected blueberry, its classification result (ripe or unripe) was recorded, and the proportion of unripe blueberries to all detected blueberries was calculated.

[0042] The operation speed of the conveyor device is adjusted according to the impurity rate in the pre-processed blueberry image of the conveyor device, specifically: Based on deep learning algorithm segmentation, we identified the proportion of impurities in the pre-processed conveyor belt blueberry images; If the proportion of impurities is greater than the set threshold, the conveyor belt speed is reduced.

[0043] The deep learning algorithm-based segmentation identifies the proportion of impurities in the pre-processed conveyor belt blueberry image, specifically: Performing feature extraction on the preprocessed blueberry image of the conveyor device to obtain a blueberry mask image and an impurity mask image; Calculate the number of pixels based on the blueberry mask image and the impurity mask image to obtain the impurity mask image area and the blueberry mask image area; Based on the area of ​​the impurity mask image divided by the sum of the area of ​​the impurity mask image and the area of ​​the blueberry mask image, the proportion of the impurity mask image, that is, the proportion of impurities in the preprocessed blueberry image of the conveyor device, is obtained.

[0044] 1. Data Preparation Collect data: Collect images of the conveyor containing blueberries and trash. These images should be representative, covering different lighting conditions, blueberry distribution, and trash types.

[0045] Annotate the data: Annotate the images to differentiate between blueberries and impurities. This usually involves creating polygons or rectangular boxes to label each object.

[0046] Data enhancement: In order to increase the generalization ability of the model, the image can be rotated, scaled, flipped, etc.

[0047] Preprocessing: resize the image, normalize pixel values, and possibly perform other enhancement or filtering operations.

[0048] 2. Model selection and training Select model: For the image segmentation task, select the YOLO deep learning model.

[0049] Train the model: Train the model using the labeled dataset. This may require tuning the model architecture, hyperparameters, and loss function to achieve optimal performance.

[0050] Validate model: Evaluate the performance of the model on the validation set to ensure that it can accurately segment blueberries and impurities.

[0051] 3. Reasoning Loading model: In the inference phase, load the trained model.

[0052] Preprocess input images: Perform the same preprocessing steps as in training on the input images.

[0053] Perform segmentation: Use the model to segment the input image and obtain the mask of blueberries and impurities.

[0054] 4. Calculation of results Calculate the impurity area: Based on the segmentation results, calculate the total area of ​​the impurity region. This can be achieved by counting the pixels in the impurity mask.

[0055] Calculate total area: Similarly, calculate the total area of ​​the entire image (or the total area of ​​the blueberries and debris, if only relative proportions are needed).

[0056] Calculate the impurity ratio: Divide the impurity area by the total area to get the impurity ratio.

[0057] First, the blueberry images on the conveyor need to be preprocessed. Preprocessing steps may include image denoising, grayscale, binarization, morphological processing, etc. to improve image quality and segmentation effect. These steps help reduce noise and interference in the image, making blueberries and impurities more clearly distinguishable.

[0058] Deep learning model selection and training, using deep learning model for image segmentation. The model has powerful feature extraction and segmentation capabilities, and can accurately identify blueberries and impurities. In the model training stage, a large amount of labeled data is required for training. The labeled data should include the accurate location and category information of blueberries and impurities. Through training, the model can learn the characteristics of blueberries and impurities, and can accurately segment them in new images. It can be understood that the deep learning model here performs image segmentation, and a common model that can achieve image segmentation in the prior art can be used.

[0059] Image Segmentation and Impurity Identification: After the model is trained, it can be used to segment the preprocessed conveyor belt blueberry image. The segmentation result will include a mask image of blueberries and impurities, where each pixel is assigned a class label (blueberry or impurity). By counting the number of pixels in the impurity mask image, the total area of ​​the impurities can be obtained. At the same time, the total area of ​​the entire image can also be calculated. The impurity percentage can be calculated by dividing the impurity area by the total area.

[0060] Result verification and optimization: Finally, the segmentation results need to be verified and optimized. The verification step includes checking the accuracy, completeness, and consistency of the segmentation results. If errors or deficiencies are found in the segmentation results, the model can be further optimized and adjusted to improve the accuracy of the segmentation effect and impurity ratio.

[0061] Embodiment 2 This embodiment provides a control system for a vibration-type blueberry harvester based on feedback control, comprising: A harvesting angle adjustment module is configured to obtain a fruit tree image for preprocessing, fit a crown curve based on the preprocessed fruit tree image and determine the height and angle of the fruit tree, and adjust the harvesting angle of the harvester according to the height and angle of the fruit tree; A vibration picking device setting module is configured to determine the initial values ​​of the amplitude and frequency of the vibration picking device in the harvester according to the fixed voltage of the harvester, so as to perform vibration harvesting on blueberries; A dynamic adjustment module for a vibrating picking device is configured to obtain and pre-process the blueberry image in the fruit collecting device, and adjust the amplitude and frequency of the vibrating picking device in the harvester according to the proportion of unripe fruits in the pre-processed blueberry image; The conveyor device dynamic adjustment module is configured to obtain and pre-process the blueberry image of the conveyor device in the harvester, and adjust the running speed of the conveyor device according to the impurity rate of the pre-processed blueberry image of the conveyor device; The vibration picking device refresh module is configured to obtain the harvested fruit tree image and preprocess it, and then adjust the initial values ​​of the amplitude and frequency of the vibration picking device when harvesting the next fruit tree according to the proportion of ripe fruits in the preprocessed harvested fruit tree image.

[0062] The harvest angle adjustment module uses a camera arranged on the outside of the blueberry harvester to obtain an image of each fruit tree during the picking process of the blueberry harvester.

[0063] The dynamic adjustment module of the vibration picking device uses a camera arranged above the fruit collecting device to obtain an image of blueberries in the fruit collecting device; The conveyor device dynamic adjustment module uses a camera arranged above the conveyor device to obtain a blueberry image of the conveyor device.

[0064] It should be noted that the harvester control system described in this embodiment controls a blueberry harvester based on the harvester structure disclosed in the patent with application number 202211227359.3, and the name is a traction-type liftable V-shaped vibrating small berry harvester. The overall structure of the harvester structure is based on a gantry frame as the main structure, and the overall power supply device is a lithium battery to power the servo motor, and the PLC, single-chip microcomputer and other control devices are powered by the inverter. In addition, when the machine is harvesting, the brush steering speed, beating efficiency, conveying speed and other parameters can be adjusted in a small range according to the changes in the physical parameters of the external environment.

[0065] Overall mechanical Figure 4 As shown, the working principle is as follows: When harvesting blueberries, the "riding ridge" method is used to move forward at a uniform speed along the soil ridges where blueberries are planted. The blueberry plants enter the brushing and beating area along the center of the gantry; the lithium battery powers the stepper motor, and the inertial vibration power device-stepper motor drives the vibration picking device, which drives the vibration disk assembly on the vibration shaft assembly to perform rotational reciprocating vibration, and drives the shrub branches to shake through the vibration dial on the vibration disk to harvest the fruit. After being picked, the blueberry fruits fall onto the lateral conveyor device, and the conveyor device and the lifting and traveling system enter the fruit collecting basket to complete the blueberry mechanical harvesting task. Blueberry plants are shrubs. During the vibration process, the vibration picking device vibrates multiple main branches and side branches. Among them, the movement of the fruit and the fruit stalk during the vibration process is simplified to a swinging motion.

[0066] In order to achieve rapid blueberry picking, the control system workflow is as follows during the development of the blueberry harvester intelligent control system: (1) Control the blueberry harvester to move forward at a constant speed; (2) The infrared sensor installed on the blueberry harvester body will detect the position of the fruit trees. After the harvester stops moving at the picking point on the ground, the fruit tree image is collected through an external camera, and the image is gray-processed to generate a grayscale image. Then, the Hough change and Canny edge detection algorithms are used to fit the straight lines in the fruit tree image, fit the crown curve and determine its height and angle. After obtaining the relevant data, the posture angle of the blueberry harvester is adjusted through the hydraulic device to adjust to the blueberry harvester posture; (3) inputting a fixed voltage to drive the motor so that it outputs a fixed frequency and amplitude, and the vibration picking device vibrates the fruit; (4) The vibration picking device vibrates the fruit, and the fruit is knocked off the branch and falls into the fruit collecting device at the lower end. The camera fixed above the fruit collecting device is used to collect the real-time blueberry image in the fruit collecting device to determine the proportion of unripe fruits. When the proportion is too large, the amplitude and frequency of the vibration picking device are reduced to reduce the rate of unripe fruit shedding. The amplitude and speed of the vibration picking device are automatically adjusted according to the artificial neural network to complete the mechanical harvesting action according to the set time; (5) The harvested blueberries are placed in the conveying device. The fan of the cleaning device will clean and remove impurities such as leaves and branches of the blueberries on the conveying device. At the same time, a fixed camera is placed above the conveying device to collect images of the blueberries on the conveying device in real time and determine the impurity content. When the impurity content is too high, the input voltage of the conveying device drive motor is reduced, the running speed of the conveying device is reduced, and the fan in the cleaning device is used to increase the time for processing the blueberries on the conveying device, thereby removing impurities; (6) The cleaned blueberries are transferred to the fruit collection basket. After completing the operation of one tree, the external camera acquires images of the harvested fruit trees and uses the detection algorithm for real-time detection. When the proportion of ripe fruits on the fruit trees after harvesting is too high, the amplitude and initial frequency of the vibration picking device are increased when operating the next tree to improve the clean fruit collection rate.

[0067] 1. System architecture The control system consists of hardware and software. The hardware includes machine vision devices, intelligent sensor modules, controllers, and the structure of the harvester disclosed in the existing patents mentioned above; the software includes image processing algorithms, control system software, multi-information fusion technology, artificial neural network technology, etc.

[0068] 2. Machine vision device The fruit tree images are collected by an external camera, and the fruit tree images are gray-processed to generate gray-scale images. Then, the Hough change and Canny edge detection algorithms are used to fit the straight lines in the fruit tree images, fit the crown curve and judge its height and angle. After obtaining relevant data, the harvesting angle of the harvester is adjusted through the hydraulic device. After completing the operation of a tree, the external camera acquires the image of the harvested fruit tree, and uses the detection algorithm for real-time detection. When the proportion of mature fruits on the fruit trees after harvesting is too high, the amplitude and frequency of the vibration picking device are increased in the next operation to improve the clean fruit collection rate.

[0069] In addition, a camera is installed above the fruit collecting device to capture real-time images of blueberries in the fruit collecting device to determine the proportion of immature fruits. When the proportion is too large, the amplitude and frequency of the vibration picking device are reduced to reduce the rate of immature fruit shedding. The amplitude and speed of the vibration picking device are automatically adjusted according to the artificial neural network to complete the mechanical harvesting action according to the set time.

[0070] The camera on the conveyor collects and determines the impurities of blueberries on the conveyor in real time. The deep learning algorithm is used to segment and identify the blueberry image of the conveyor and determine the proportion of impurities on the conveyor. If the proportion is relatively high, the speed of the conveyor is reduced to give the fan of the cleaning device sufficient time to remove impurities, thereby improving the impurity removal rate.

[0071] 3. Execution module like Figure 5 As shown, the execution module collects the motor output signal of the vibration picking device, performs noise reduction processing on it, and obtains the noise reduction signal of the collected amplitude; collects the speed signal of the vibration picking device; integrates the noise reduction signal and the speed signal to obtain the real-time signal of the vibration picking device; The obtained signal is subjected to noise reduction processing to obtain a real-time and accurate noise reduction signal, a preset magnetoelectric Hall sensor is used to collect analog signals, and the obtained analog signals are subjected to analog-to-digital conversion to obtain the speed signal of the motor.

[0072] Discretize the analog signal to obtain a discrete signal of the analog signal; sample the analog signal to obtain a sampled signal of its discrete signal; encode the sampled signal to obtain the encoded signal, determine that the encoded signal is the speed signal, normalize the above signal to obtain a normalized speed signal; generate a real-time signal of the device using the normalized noise reduction signal, the speed signal and a preset integration weight. Generate a signal difference value based on the real-time signal and the preset signal threshold. Subtract the threshold from the real-time signal to obtain a signal difference value. Generate a cumulative error and a predicted error change rate using the signal difference value. Generate an adjustment value of the harvester based on the cumulative error.

[0073] The adjustment difference of the harvester is generated by using the preset adjustment algorithm and the system difference value, and the harvester is adjusted according to the adjustment difference. The preset adjustment function is: ; In the formula, is the given adjustment value of the system, is the proportionality coefficient, is the integration coefficient, is the differential coefficient, is the cumulative error of the harvesting device, is the rate of change of the error of the prediction device.

[0074] 4. Conveying device The camera collects the image data of the blueberries on the conveyor and sets the threshold. The proportion of unripe fruits is determined. When the proportion is too large, the motor voltage input is reduced, thereby reducing the speed of the conveyor and increasing the time for the fan in the cleaning device to process the blueberries on the conveyor, thereby removing impurities. Therefore, the conveyor speed data should be collected in real time, and the output data of the conveyor drive motor should be filtered. The exponential moving average filter (EMA) is used to filter the data signal, which can respond to the change of the actual force more quickly while ensuring data smoothness.

[0075] The calculation formula of EMA is as follows: ; in, represents the filtered value, Represents the data after EMA filtering at the previous moment, Indicates the current moment data. According to the response speed and noise suppression, the moving average filter is evaluated at different smoothing factors. to determine the best tactile sensor for the end effector. value.

[0076] In this embodiment, when the staff needs to control the blueberry harvester, a control signal can be sent through the remote control device. After the controller receives the control signal, the staff operates according to the remote control device to control the controller to move the blueberry harvester on the ground. At the same time, when the blueberry harvester fails, the controller will detect the corresponding functions of each module, and finally feed back the detection data to the remote control device, so that the staff can perform timely maintenance and debugging to improve the working efficiency of the blueberry harvester.

[0077] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A control method for a vibration blueberry harvester based on feedback control, characterized in that: include: Obtain fruit tree images for preprocessing, fit the crown curve based on the preprocessed fruit tree images and determine the height and angle of the fruit trees, and adjust the harvesting angle of the harvester according to the height and angle of the fruit trees; Determining initial values ​​of the amplitude and frequency of a vibration picking device in the harvester according to a fixed voltage of the harvester, thereby performing vibration harvesting on blueberries; Obtaining and preprocessing the blueberry image in the fruit collection device, and adjusting the amplitude and frequency of the vibration picking device in the harvester according to the proportion of unripe fruits in the preprocessed blueberry image; Obtaining and preprocessing a blueberry image of a conveyor device in a harvester, and adjusting a running speed of a conveyor belt according to a level of impurity in the preprocessed blueberry image of the conveyor device; After obtaining and preprocessing the harvested fruit tree image, the initial values ​​of the amplitude and frequency of the vibration picking device when harvesting the next fruit tree are adjusted according to the proportion of ripe fruits in the preprocessed harvested fruit tree image.

2. The control method of the vibration blueberry harvester based on feedback control according to claim 1, characterized in that: The obtaining of the fruit tree image for preprocessing, fitting the crown curve based on the preprocessed fruit tree image and determining the height and angle of the fruit tree, specifically includes: Obtain the fruit tree image and perform grayscale processing to generate a grayscale image; The real center line of the fruit tree in the grayscale image is fitted using Hough transformation to obtain the fitted center line of the fruit tree; determining the angle of the fruit tree based on the slope of the fitted centerline of the fruit tree and the slope of the true centerline of the fruit tree; The lateral position deviation of the fruit tree is calculated based on the data of the true center line of the fruit tree, and the maximum value of the lateral position deviation of the fruit tree is taken as the height of the fruit.

3. The control method of the vibration blueberry harvester based on feedback control according to claim 1, characterized in that: Adjust the harvesting angle of the harvester according to the height and angle of the fruit trees, specifically: The picking frame in the harvester is adjusted to rise or fall based on the height of the fruit tree to match the height of the blueberry plant; wherein the picking frame is a V-shaped frame composed of two vibrating picking devices; According to the angle of the fruit tree, the upper fixed position is changed to adjust the inclination angle of the vibration picking device so that the inclination angle of the vibration picking device matches the growth direction of the blueberry fruit.

4. The control method of the vibration blueberry harvester based on feedback control according to claim 1, characterized in that: The step of adjusting the amplitude and frequency of the vibration harvesting device in the harvester according to the proportion of unripe fruits in the preprocessed blueberry image is as follows: Use the pre-trained image object recognition classification model to determine the proportion of unripe fruits in the pre-processed blueberry image; If the proportion of unripe fruits is greater than the set threshold, the amplitude and frequency of the vibrating picking device in the harvester are reduced to reduce the rate of unripe fruit falling off.

5. The control method of the vibration blueberry harvester based on feedback control according to claim 4, characterized in that: The pre-trained image object recognition classification model is used to determine the proportion of unripe fruits in the pre-processed blueberry image, specifically: The deep learning-based object detection network is used as an image object recognition and classification model to identify and classify the processed blueberry images; According to the identification and classification results, the number of mature classification results and immature classification results is determined, and the proportion of immature classification results to all identification and classification results is calculated to obtain the proportion of immature fruits.

6. The control method of the vibration blueberry harvester based on feedback control according to claim 1, characterized in that: The operation speed of the conveyor belt is adjusted according to the impurity rate in the pre-processed blueberry image of the conveyor device, specifically: Based on deep learning algorithm segmentation, the proportion of impurities in the pre-processed conveyor blueberry image was identified; If the impurity ratio is greater than the set impurity ratio threshold, the conveyor belt speed is reduced.

7. The control method of a vibration blueberry harvester based on feedback control according to claim 6, characterized in that: The deep learning algorithm-based segmentation identifies the proportion of impurities in the pre-processed blueberry image of the conveyor, specifically: Performing feature extraction on the preprocessed blueberry image of the conveyor device to obtain a blueberry mask image and an impurity mask image; Calculate the number of pixels based on the blueberry mask image and the impurity mask image to obtain the impurity mask image area and the blueberry mask image area; Based on the area of ​​the impurity mask image divided by the sum of the area of ​​the impurity mask image and the area of ​​the blueberry mask image, the proportion of the impurity mask image, that is, the proportion of impurities in the preprocessed blueberry image of the conveyor device, is obtained.

8. A vibration blueberry harvester control system based on feedback control, characterized in that: include: A harvesting angle adjustment module is configured to obtain a fruit tree image for preprocessing, fit a crown curve based on the preprocessed fruit tree image and determine the height and angle of the fruit tree, and adjust the harvesting angle of the harvester according to the height and angle of the fruit tree; A vibration picking device setting module is configured to determine the initial values ​​of the amplitude and frequency of the vibration picking device in the harvester according to the fixed voltage of the harvester, so as to perform vibration harvesting on blueberries; A dynamic adjustment module for a vibrating picking device is configured to obtain and pre-process the blueberry image in the fruit collecting device, and adjust the amplitude and frequency of the vibrating picking device in the harvester according to the proportion of unripe fruits in the pre-processed blueberry image; The conveyor device dynamic adjustment module is configured to obtain and pre-process the blueberry image of the conveyor device in the harvester, and adjust the running speed of the conveyor device according to the impurity rate of the pre-processed blueberry image of the conveyor device; The vibration picking device refresh module is configured to obtain the harvested fruit tree image and preprocess it, and then adjust the initial values ​​of the amplitude and frequency of the vibration picking device when harvesting the next fruit tree according to the proportion of ripe fruits in the preprocessed harvested fruit tree image.

9. The feedback-control-based vibration blueberry harvester control system according to claim 8, characterized in that: The harvest angle adjustment module uses a camera arranged on the outside of the blueberry harvester to obtain an image of each fruit tree during the picking process of the blueberry harvester.

10. The feedback-control-based vibration blueberry harvester control system according to claim 8, characterized in that: The dynamic adjustment module of the vibration picking device uses a camera arranged above the fruit collecting device to obtain an image of blueberries in the fruit collecting device; The conveyor device dynamic adjustment module uses a camera arranged above the conveyor device to obtain a blueberry image of the conveyor device.

Images