Greenhouse fruit and vegetable picking and carrying system and method based on intelligent algorithm joint control

Abstract

This invention discloses a greenhouse fruit and vegetable harvesting and transportation system based on intelligent algorithm control, which enables precise harvesting location, automatic connection of harvesting and transportation processes, and allows the overhead crane to respond to dynamic harvesting position commands and operational needs. The system includes an intelligent harvesting unit, a greenhouse overhead crane unit, and an algorithm control unit. The intelligent harvesting unit identifies and locates mature target fruits and vegetables, generates overhead crane movement commands, harvesting commands, and transportation commands, and executes the harvesting actions. The greenhouse overhead crane unit is movably arranged on a track on the top of the greenhouse and carries the intelligent harvesting unit, moving the intelligent harvesting unit in response to overhead crane movement commands or transportation commands. The algorithm control unit receives and forwards the overhead crane movement commands or transportation commands sent by the intelligent harvesting unit to the greenhouse overhead crane unit, and provides feedback on the operating status of the greenhouse overhead crane unit to the intelligent harvesting unit. A method for harvesting and transporting greenhouse fruits and vegetables using the above-described greenhouse fruit and vegetable harvesting and transportation system is also disclosed.

Description

Technical Field

[0001] This invention relates to the field of intelligent agricultural equipment technology, and in particular to a greenhouse fruit and vegetable harvesting and handling system and method based on intelligent algorithm-based joint control. Background Technology

[0002] Greenhouse fruit and vegetable cultivation is playing an increasingly important role in the agricultural industry due to its high added value and off-season supply. Currently, the harvesting process mainly relies on ground-based mobile intelligent harvesters or manually assisted positioning equipment. However, the complex terrain within greenhouses (such as furrows, irrigation pipes, and supports) makes ground-based mobile equipment prone to jamming and lacking in positioning accuracy, limiting the harvester's positioning and making it difficult to adapt to different row and plant spacing arrangements. Furthermore, the greenhouse overhead cranes used for transport operate independently of the ground-based harvesters, requiring manual coordination between harvesting and transport, resulting in low automation and limited efficiency in fruit and vegetable harvesting and transport. In addition, traditional overhead crane control systems in greenhouses typically use a single-track preset mode, limiting their function to simple material hoisting. They lack a collaborative linkage logic with intelligent harvesters, failing to dynamically adjust based on the fruit and vegetable's growth position and harvesting progress. This makes it difficult to respond to the harvester's dynamic position commands and operational needs, hindering the full realization of the intelligent harvester's advantages, resulting in high overall operating costs and hindering large-scale promotion. Summary of the Invention

[0003] Therefore, it is necessary to address the above-mentioned shortcomings by providing a greenhouse fruit and vegetable harvesting and transportation system and method based on intelligent algorithm-based joint control, which can accurately locate the harvest, automatically connect the harvesting and transportation process, and enable the overhead crane to respond to dynamic harvesting position commands and operational needs.

[0004] A greenhouse fruit and vegetable harvesting and transportation system based on intelligent algorithm-controlled joint control includes: The intelligent harvesting unit is used to identify and locate mature target fruits and vegetables, generate crane movement instructions, harvesting instructions and transfer instructions, and execute fruit and vegetable harvesting actions; The greenhouse crane unit is used to be movably arranged on the track on the top of the greenhouse and to carry the intelligent harvesting unit, and to drive the intelligent harvesting unit to move in response to the crane movement command or transfer command; The algorithm control unit is used to receive and forward the crane movement instructions or transfer instructions sent by the intelligent harvesting unit to the greenhouse crane unit, and to feed back the operating status of the greenhouse crane unit to the intelligent harvesting unit so that the intelligent harvesting unit can optimize the movement path according to the operating status.

[0005] In one embodiment, the intelligent harvesting unit includes: The visual recognition module is used to scan the work area and identify mature target fruits and vegetables; The location positioning module is used to determine the operation coordinates by combining greenhouse layout data with real-time positioning information; The operation decision module is used to design the operation sequence and crane movement path according to the distribution density of fruits and vegetables, generate the crane movement instructions and send them to the algorithm control unit. The crane movement instructions include target position coordinates, movement speed and attitude adjustment parameters. The harvesting execution module is used to respond to harvesting commands and execute harvesting actions after the greenhouse crane unit is in place.

[0006] In one embodiment, the visual recognition module scans the work area and identifies mature target fruits and vegetables, including: Photograph fruits and vegetables and output RGB format images; Calculate the normalized values ​​of the RGB three channels of the image, as well as the maximum and minimum values ​​of the normalized values; Calculate and determine the ripeness of fruits and vegetables based on the image's hue, saturation, and brightness.

[0007] In one embodiment, the gripping part of the picking execution module is equipped with a pressure sensor for detecting the gripping force, and the gripping force threshold is different for different fruits and vegetables.

[0008] In one embodiment, the calibration of the work coordinates by the position positioning module includes: obtaining fused work coordinates by fusing the visual coordinates of the image collected by the visual recognition module, the pressure data uploaded by the pressure sensor, the displacement sensor data and encoder data of the position positioning module through multi-sensor fusion.

[0009] In one embodiment, the job decision module designs the job sequence and the crane movement path, and generates the crane movement instructions including: Select fruits and vegetables to be harvested based on their ripeness; Based on the fusion operation coordinates, the initial coordinates of the harvesting execution module, and the visually estimated coordinates, calculate the distance from each fruit and vegetable to be harvested to the initial position of the harvesting execution module; The work sequence and crane movement path are planned according to the distances from smallest to largest.

[0010] In one embodiment, the greenhouse crane unit includes a crane arranged on the track, a walking drive module for driving the crane to move longitudinally and laterally along the greenhouse track, a lifting adjustment module for controlling the lifting and lowering of the picking execution module, and a clamping and fixing module for clamping the picking execution module; the walking drive module and the lifting adjustment module receive instructions sent by the algorithm control unit and control the movement and lifting of the picking execution module, and the walking drive module and the lifting adjustment module send operation parameters to the algorithm control unit to provide feedback on the crane's operating status.

[0011] In one embodiment, the drive unit of the picking execution module adopts PID control; the picking execution module dynamically adjusts the servo motor angle based on the fruit and vegetable diameter and clamping force feedback to adaptively clamp the fruit and vegetables.

[0012] In one embodiment, the greenhouse fruit and vegetable harvesting and transportation system further includes a fruit and vegetable transfer unit, which is used to receive the harvested fruits and vegetables and, driven by the greenhouse crane unit, transfer the fruits and vegetables from the working position to the designated collection point in the greenhouse.

[0013] This invention also discloses a method for harvesting and transporting greenhouse fruits and vegetables based on intelligent algorithm-based joint control. This method is implemented using the aforementioned greenhouse fruit and vegetable harvesting and transporting system, and includes the following steps: S1. Identify and locate mature target fruits and vegetables, and generate crane movement instructions; S2. Control the crane to move along the track to above the target area according to the crane movement command, adjust the harvester to the preset working posture, and send back the harvester positioning signal; S3. Harvest ripe target fruits and vegetables and generate transfer instructions; S4. Control the overhead crane to move to the designated collection point and drop the harvested ripe target fruits and vegetables, output the transfer completion signal, and move the overhead crane to the next target operation position; S5. Repeat steps S1-S4 until the entire area is completed.

[0014] The intelligent algorithm-based greenhouse fruit and vegetable harvesting and transportation system and method of this invention integrates an intelligent harvesting unit mounted on a greenhouse crane unit, enabling aerial mobile installation of the intelligent harvesting unit. This eliminates the impact of complex terrain within the greenhouse on the positioning of the intelligent harvesting unit. The algorithm-controlled crane unit controls the crane unit's movements, allowing the intelligent harvesting unit to precisely move to a preset harvesting position, achieving precise fruit and vegetable harvesting. The integration of the intelligent harvesting unit with the crane unit enables coordinated control between the two. Controlling the crane unit's operation via the algorithm-controlled crane unit allows for the transfer of fruits and vegetables after harvesting, achieving automatic connection between harvesting and transfer processes, improving the automation level and efficiency of fruit and vegetable harvesting and transfer operations. Furthermore, the algorithm-controlled crane unit feeds back the crane unit's operating status to the intelligent harvesting unit, enabling it to perform harvesting operations after precise positioning. This responsiveness to dynamic harvesting location commands and operational needs improves harvesting accuracy and efficiency, reduces operating costs, and facilitates large-scale promotion. Attached Figure Description

[0015] Figure 1 This is a block diagram of a greenhouse fruit and vegetable harvesting and transportation system in one embodiment of the present invention; Figure 2This is a schematic diagram of the harvester in one embodiment of the present invention; Figure 3 This is a block diagram showing the module connection of the intelligent harvesting unit in one embodiment of the present invention; Figure 4 This is a flowchart illustrating the process of identifying ripe fruit and obtaining its coordinates in one embodiment of the present invention; Figure 5 This is a flowchart of the core board main program in one embodiment of the present invention. Detailed Implementation

[0016] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0017] Example 1 This invention aims to address the problems of limited positioning of intelligent harvesting machines, poor coordination with transport equipment, and insufficient automation in existing greenhouse fruit and vegetable harvesting operations. It provides a greenhouse fruit and vegetable harvesting and transport system based on intelligent algorithm-driven joint control. Through deep integration of the intelligent harvesting machine algorithm and the overhead crane control system, the system enables the overhead crane to precisely carry the harvesting equipment to the target work location, quickly transport the harvested fruit, improve the automation and intelligence level of greenhouse fruit and vegetable harvesting operations, reduce reliance on manual labor, and increase overall operational efficiency. For details, please refer to [link to relevant documentation]. Figure 1The greenhouse fruit and vegetable harvesting and transportation system of this embodiment includes an intelligent harvesting unit 1, a greenhouse crane unit 2, and an algorithm control unit 3. The intelligent harvesting unit 1 is used to identify and locate mature target fruits and vegetables, generate crane movement instructions, harvesting instructions, and transfer instructions, and execute the harvesting actions. In other words, the intelligent harvesting unit 1 serves both as a fruit and vegetable maturity identification and location unit and as an instruction generation and harvesting action execution unit. The greenhouse crane unit 2 is movably arranged on the track 10 at the top of the greenhouse and carries the intelligent harvesting unit 1. It moves the intelligent harvesting unit 1 in response to crane movement instructions or transfer instructions. In other words, the greenhouse crane unit 2 serves as the operating carrier for harvesting and transportation, carrying the intelligent harvesting unit 1 to achieve its positional movement. The algorithm control unit 3 receives and forwards crane movement or transfer instructions sent by the intelligent harvesting unit 1 to the greenhouse crane unit 2, and feeds back the operating status of the greenhouse crane unit 2 to the intelligent harvesting unit 1. This allows the intelligent harvesting unit 1 to optimize its movement path based on the operating status. Essentially, the algorithm control unit 3 acts as the central linkage between the intelligent harvesting unit 1 and the greenhouse crane unit 2. It is signal-connected to both units. On one hand, it receives corresponding instructions from the intelligent harvesting unit 1 to control the actions of the greenhouse crane unit 2, enabling the intelligent harvesting unit 1 to reach the appropriate location for harvesting and transferring fruits and vegetables. On the other hand, the algorithm control unit 3 also feeds back the crane's operating status to the intelligent harvesting unit 1, allowing it to perform harvesting operations upon reaching the designated location. This achieves two-way data communication and closed-loop control, ensuring the accuracy and timeliness of instruction execution. The crane's operating status includes movement progress, whether it is in position, and whether there is a malfunction. In this way, the intelligent harvesting unit 1 can dynamically adjust its movement commands based on the feedback of the crane's operating status. If it encounters an obstacle, the intelligent harvesting unit 1 will automatically optimize its movement path to ensure the smooth progress of the operation.

[0018] The intelligent harvesting unit 1 is used to identify the location of target fruits and vegetables, calibrate the operation coordinates, generate harvesting instructions, and simultaneously output crane movement instructions to the algorithm control unit 3. After the crane arrives, it performs precise harvesting operations and outputs transfer instructions to the crane after harvesting. Specifically, the intelligent harvesting unit 1 includes a visual recognition module, a location positioning module, an operation decision module, and a harvesting execution module. The visual recognition module scans the operation area and identifies mature target fruits and vegetables; the location positioning module calibrates the operation coordinates by combining greenhouse layout data and real-time positioning information; the operation decision module designs the operation sequence and crane movement path based on the distribution density of fruits and vegetables, generates crane movement instructions, and sends them to the algorithm control unit 3. The crane movement instructions include target location coordinates, movement speed, and attitude adjustment parameters; the harvesting execution module responds to the harvesting instructions and executes the harvesting action after the greenhouse crane unit 2 is in place. In this embodiment, the visual recognition module includes a high-definition industrial camera and an infrared supplementary lighting component to adapt to the low-light environment inside the greenhouse and achieve accurate acquisition of fruit and vegetable images. The positioning module integrates GPS and inertial navigation positioning components, with a positioning accuracy error of ≤±5mm, enabling precise positioning of fruits and vegetables. The operation decision module is an embedded chip equipped with a core control algorithm, possessing rapid computing and command output capabilities.

[0019] It should be noted that in this embodiment, the harvesting execution module and the greenhouse crane unit 2 together form a harvesting machine for harvesting and transporting fruits and vegetables. The greenhouse crane unit 2 includes a crane arranged on track 10, a walking drive module for driving the crane to move longitudinally and laterally along the greenhouse track, a lifting adjustment module for controlling the lifting and lowering of the harvesting execution module, and a clamping and fixing module for holding the harvesting execution module. The walking drive module and the lifting adjustment module receive instructions sent by the algorithm control unit 3 and control the movement and lifting of the harvesting execution module. Furthermore, the walking drive module and the lifting adjustment module send operation parameters to the algorithm control unit 3 to provide feedback on the crane's operating status. For details, please refer to [link to relevant documentation]. Figure 2The overhead crane is arranged on at least one track 10 extending along the X-axis. In this embodiment, to improve the stability of the crane's operation, two parallel tracks 10 are arranged along the X-axis, which together support the crane. The crane adopts a lightweight steel structure design to meet the load-bearing requirements of the greenhouse roof. The roof of the greenhouse is covered with longitudinal and transverse tracks to cover the entire working area; the tracks 10 are reserved with positioning scales to help improve positioning accuracy. The overhead crane includes a crane body 21 that can travel along the X-axis on track 10, and a mounting frame 22 arranged on the crane body 21 and movable in a horizontal plane along a direction perpendicular to the X-axis (i.e., the Y-axis direction). First rollers 211 are mounted on both sides of the crane body 21 and roll into contact with the two side tracks 10. The crane body 21 also has two parallel Y-axis tracks 212 extending along the Y-axis direction. Second rollers 221 are mounted on both sides of the mounting frame 22 and roll into contact with the two Y-axis tracks 212. The travel drive module uses a servo motor and a high-precision transmission mechanism, such as a reducer, to ensure smooth movement and accurate positioning. The travel drive module includes a first power motor 213, a first transmission mechanism 214 driven by the first power motor 213, a second power motor 222, and a second transmission mechanism 223 driven by the second power motor 222. The first transmission mechanism 214 is driven by at least one first roller 211, and the second transmission mechanism 223 is driven by at least one second roller 221. The first transmission mechanism 214 is a chain drive mechanism, belt drive mechanism, or gear drive mechanism, and the second transmission mechanism 223 is a chain drive mechanism, belt drive mechanism, gear drive mechanism, or gearbox. The lifting adjustment module includes a Z-axis bracket 224 fixed on the mounting frame 22 and extending vertically, a third power motor 225 fixed on the mounting frame 22, a third transmission mechanism (not shown) driven by the third power motor 225 and the Z-axis bracket 224, an X-axis bracket 226 slidably disposed on the Z-axis bracket 224, and a lifting drive component (not shown) for driving the X-axis bracket 226 to move vertically up and down on the Z-axis bracket 224. When the third power motor 225 is working, it drives the Z-axis bracket 224 to rotate around the axis of the Z-axis bracket 224 through the third transmission mechanism, so that the X-axis bracket 226 swings in the horizontal plane. The lifting drive component drives the X-axis bracket 226 to move up and down, so as to realize the raising or lowering of the picking execution module. The lifting drive uses an electric push rod with adjustable stroke to accommodate fruits and vegetables of varying heights. The clamping and fixing module stably holds or secures the harvesting module, responding precisely to commands from the algorithm control unit 3 and moving to the target work position. The clamping and fixing module employs a flexible clamping structure to prevent damage to the harvester and also features a quick-switching function, compatible with both the harvesting module and transport containers.The clamping and fixing module includes a clamping bracket 227 slidably mounted on the X-axis bracket 226, a fourth power motor 228 fixed on the X-axis bracket 226, a fourth transmission mechanism 229 driven by the fourth power motor 228 to allow the clamping bracket 227 to slide along the Y-axis on the X-axis bracket 226, a pneumatic gripper 230 arranged on the clamping bracket 227, and a rotary drive component capable of driving the pneumatic gripper 230 to rotate. The pneumatic gripper 230 is used to grip the intelligent picking unit 1 or the transfer container. In this embodiment, the third transmission mechanism is a gear transmission mechanism, which includes at least two gears with different numbers of teeth that mesh successively, so that the Z-axis bracket 224 rotates at a lower speed; the fourth transmission mechanism 229 can be a belt transmission mechanism or other transmission mechanism capable of converting rotation to sliding; the rotary drive component can be a servo motor.

[0020] In this way, during the harvesting and transportation of fruits and vegetables, the walking drive module and the lifting adjustment module receive instructions sent by the algorithm control unit 3, which can control the position of the harvesting execution module in the X-axis, Y-axis and Z-axis directions, and make the harvesting execution module swing within a certain range to expand its harvesting range; through the clamping and fixing module, the position of the harvesting execution module in the Y-axis direction can be further adjusted, and through the cooperation of the pneumatic gripper 230 and the rotary drive component, the harvesting execution module can grasp and pull the fruits and vegetables to pick the mature fruits and vegetables from the plant, thereby completing the entire process of harvesting fruits and vegetables.

[0021] It should be noted that during the harvesting operation, the harvesting execution module can either harvest a single fruit or vegetable and then transfer it to the designated collection point, or harvest multiple fruits and vegetables consecutively and then transfer them to the designated collection point. Furthermore, in this embodiment, the greenhouse fruit and vegetable harvesting and transportation system also includes a fruit and vegetable transfer unit 4. This unit 4 is used to receive the harvested fruits and vegetables and, driven by the greenhouse crane unit 2, transfers them from the working position to the designated collection point in the greenhouse. Specifically, the fruit and vegetable transfer unit 4 includes a transfer container detachably connected to the pneumatic gripper 230 for receiving the fruits and vegetables, and a positioning docking module for fixing the transfer container. Thus, after the harvesting operation is completed, the crane unit switches to transfer mode, the pneumatic gripper 230 switches to grip the transfer container, and the positioning docking module fixes the container, transferring the harvested fruit from the working position to the designated collection point in the greenhouse. During the transfer process, the algorithm control unit 3 controls the operating parameters of the greenhouse crane unit 2 to synchronously adjust the running speed of the transfer container and avoid fruit and vegetable damage.

[0022] The algorithm-controlled unit 3 employs an industrial-grade controller, including a data receiving module, an instruction parsing module, and a collaborative control module. The data receiving module supports wireless communication and establishes real-time data communication with the operation decision module of the intelligent harvesting unit 1, receiving position and operation instructions output by the operation decision module. This ensures real-time data transmission with the harvesting execution module and the overhead crane, with an instruction parsing latency of ≤100ms, guaranteeing the continuity of collaborative operations. The instruction parsing module parses instructions and converts them into executable operating parameters for the overhead crane. The collaborative control module communicates with the greenhouse overhead crane unit 2 and the fruit transfer unit, while simultaneously feeding back the overhead crane's operating status to the harvesting execution module of the intelligent harvesting unit 1, achieving bidirectional data exchange and closed-loop control, ensuring the accuracy and timeliness of instruction execution.

[0023] When harvesting and transporting fruits and vegetables in the greenhouse, the overhead crane's track 10 is laid along the length and width of the greenhouse to ensure that all planting ridges are covered. The harvesting execution module is initially placed in the pneumatic gripper 230, and the transfer container is pre-placed in the collection area at the corner of the greenhouse. After communication pairing is completed, it can be put into use.

[0024] It should be noted that during harvesting, since the visual recognition module, position positioning module, operation decision module, and harvesting execution module are integrated into one unit, the intelligent harvesting unit 1 is actually clamped by the pneumatic gripper 230. Please refer to [link / reference]. Figure 3 In this embodiment, the visual recognition module integrates a camera module and a cache chip. It acquires fruit and vegetable images through visual detection and achieves data collaboration through dual-machine serial communication between the baseboard controller and the core board (i.e., the main controller). A pressure sensor is also provided to communicate with the core board. The position positioning module is equipped with a capacitive displacement sensor and an encoder. The capacitive displacement sensor is connected to the core board sequentially through a signal conditioning circuit and an A / D conversion circuit. The operation decision module acts as the decision layer, using an STM32F103ZET6 chip as the core board. Based on an improved YOLOv5-SC model, it realizes fruit and vegetable detection and harvesting planning. Simultaneously, the operation decision module enables the harvesting execution module to accurately harvest fruits and vegetables through target detection, motion planning, and force control. Target detection enhances feature extraction, motion planning controls the trajectory of the harvesting execution module by controlling the operation of the greenhouse crane unit 2, and force control adjusts the clamping force of the harvesting execution module on the fruits and vegetables. The picking execution module, as the execution layer, consists of three-axis motors (i.e., X-axis motor, Y-axis motor, and Z-axis motor) and servo motors. It performs closed-loop control through the three-axis motors and servo motors. The X-axis motors and Z-axis motors can provide positioning feedback, the Y-axis motor has a speed and position closed loop, and the servo motors can provide clamping force feedback. That is, a pressure sensor is installed at the servo motor.

[0025] In this embodiment, the visual recognition module relies on the OV7725 camera module for image acquisition. This module contains an OV7725 CMOS image sensor and a 12MHz active crystal oscillator, outputting RGB565 format images. Under the control of the baseboard controller, image data is stored in the AL422B FIFO buffer chip via the SCCB interface for buffering, and then transmitted to the core board via serial port. The acquisition cycle is 0.04s, ensuring real-time performance. Experiments show that the module has a field of view of 60° at a distance of 1m, covering a picking area with a diameter of 1.7m, meeting the localized picking needs of a single fruit tree.

[0026] In one embodiment, the visual recognition module scans the work area and identifies mature target fruits and vegetables, including: Photograph fruits and vegetables and output RGB format images.

[0027] Calculate the normalized values ​​of the RGB three channels of the image, as well as the maximum and minimum values ​​of the normalized values.

[0028] Calculate and determine the ripeness of fruits and vegetables based on the image's hue, saturation, and brightness.

[0029] Specifically, the key to image preprocessing is the conversion between RGB and HSV color spaces to reduce lighting interference. Before conversion, the normalized values ​​of the RGB three channels of the RGB format image must be calculated. Among them, R... n =R / 255, G n =G / 255, B n =B / 255, where R, G, and B are the pixel values ​​of the RGB channels (0-255), R n G n B n The values ​​are normalized (0-1). The maximum value (maxValue) and minimum value (minValue) are then calculated as follows: maxValue=max(R n G n B n (1) minValue=min(R n G n B n (2) The calculation of hue (H), saturation (S), and brightness (V) is performed scene by scene: When maxValue = minValue, H = 0∘ (grayscale); When maxValue=R n hour, ; When maxValue=G n hour, ; When maxValue=B n hour, ; If H is negative, then H = H + 360∘ (values ​​range from 0 to 360°).

[0030] (3) The values ​​of S and V range from 0 to 1. The recognition threshold for red ripe fruit is set to (H>300∘ or H<60∘) and V>0.25. When the difference between the highest and lowest columns of the red pixel area exceeds 50, it is judged as a complete fruit.

[0031] Furthermore, maturity identification is based on the HSV channel and is divided into the following three levels: Maturity: (H>330∘ or H<30∘) and V>0.3; Semi-ripe: (H>300∘ or H<60∘) and 0.2 <V≤0.3; Immature: Other combinations.

[0032] The performance of the target detection process was compared between the traditional HSV method and the improved YOLO algorithm. The experimental data are shown in the table below.

[0033] Visual perception algorithm performance comparison table

[0034] As can be seen, the YOLOv5-SC algorithm, due to its attention mechanism, achieves an accuracy of 90.3% in occluded scenes, with a detection time of 108ms and a model weight of 6.69MB, making it suitable for embedded deployment and significantly superior to the traditional HSV method.

[0035] It should be noted that during the process of identifying ripe fruits and vegetables, their visual coordinates are also acquired to provide data for location positioning. For details, please refer to [link to relevant documentation]. Figure 4The visual recognition module first defines variables and checks for frame interruption updates. If no interruption occurs, the process ends. If interruption occurs, it checks if the number of rows has reached 240. If the number of rows reaches 240, it proceeds to calculate the center coordinates of the red fruit. If the number of rows has not reached 240, it checks if the number of columns has reached 320. If the number of columns reaches 320, it increments the row count and proceeds to calculate the center coordinates of the red fruit. If the number of columns has not reached 320, it resets the read pointer and reads data, extracts the RGB color components of the image, converts them to the HSV color model, and checks if they are red pixels. If they are not red pixels, it proceeds to calculate the red pixel. If they are red pixels, it increments the row count and checks if the row count is 1. If not, it proceeds to calculate the red pixel. If the row count is 1, the current column value is assigned the lowest column value of the red fruit, and the red pixel calculation process begins. The red pixel calculation process includes checking if the row count is 0. If not, increment the column count and row count by 1 sequentially, then proceed to the calculation of the red fruit's center coordinates. If the row count is 0, assign the current column value to the highest column value of the red fruit, and check if the difference between the highest and lowest column values ​​is greater than 50. If not, increment the column count and row count by 1 sequentially, then proceed to the calculation of the red fruit's center coordinates. If yes, scan column by column from lowest to highest, setting both the x and y coordinates of the red pixel to 1, then increment the column count and row count by 1 sequentially, then proceed to the calculation of the red fruit's center coordinates. The calculation of the red fruit's center coordinates involves scanning column by column from smallest to largest to obtain the minimum y coordinate of the red fruit; then scanning column by column from largest to smallest to obtain the maximum y coordinate of the red fruit; then scanning row by row from smallest to largest to obtain the minimum x coordinate of the red fruit; finally, scanning row by row from largest to smallest to obtain the maximum x coordinate of the red fruit, calculating the center coordinates of the red fruit based on the above coordinate values, clearing the frame interruption flag, and ending the process.

[0036] When sensing the clamping force using a pressure sensor mounted on the servo motor, the clamping force of the servo motor can be adjusted based on the type of fruit or vegetable identified. The gripping part (i.e., the servo motor) of the harvesting execution module is equipped with a pressure sensor for detecting the clamping force. Different fruits and vegetables have different clamping force thresholds. By detecting and adjusting the servo motor's clamping force, damage or fruit drop can be prevented. Specifically, the clamping force sensing module consists of a pressure sensor, a signal conditioning circuit, and an A / D conversion circuit. The pressure sensor is a strain gauge type, and its output voltage has a linear relationship with the clamping force. (4) In the formula, =0.02V / N (sensitivity) =0.1V (zero drift voltage). The clamping force is N. The output voltage is (V).

[0037] The signal conditioning circuit will Magnified 50 times, voltage after magnification : (5) Right now Simplify force calculations, after magnification The range is 0-3.3V, matching the input of an A / D conversion circuit.

[0038] The A / D conversion uses the ADS1118 16-bit converter, with SPI communication protocol and conversion accuracy of [missing information]. After reading the digital value D, the core board calculates the clamping force: (6) The value ranges from 0 to 65535 to ensure the force calculation accuracy is ±0.1N.

[0039] The clamping force thresholds for citrus fruits, apples, and peaches were set at 12.3N, 15.7N, and 10.9N, respectively. 100 clamping actions were tested, and the results are shown in the table below.

[0040] Test table of clamping damage rate of different fruit types

[0041] It is evident that by using the clamping force threshold of this scheme to control the clamping force of different fruits and vegetables, the damage rate is less than 2%, with an average damage rate of 1.7%, which meets the harvesting requirements.

[0042] The location positioning module calibrates the work coordinates by: using the visual coordinates of the image acquired by the visual recognition module, the pressure data uploaded by the pressure sensor, and the displacement sensor and encoder data from the location positioning module, and then performing multi-sensor fusion to obtain the fused work coordinates. In this embodiment, visual, displacement, and encoder data are integrated through multi-sensor fusion to improve positioning accuracy, and a weighted fusion algorithm is used. The fused coordinates of the picking execution module in the X-axis and Y-axis directions are as follows: (7) (8) In the formula, , For the merged coordinates, , For visual coordinates, For displacement sensor data, For encoder data, the values ​​of the aforementioned weight data such as 0.3, 0.4, 0, 6, and 0.7 were determined experimentally to ensure that visual data dominates positioning, while displacement and encoder data correct for errors.

[0043] The positioning error before and after fusion was tested, and the results are shown in the table below.

[0044] Multi-sensor fusion positioning error test table (unit: mm)

[0045] As can be seen, after fusion, the horizontal error is reduced to 1.3mm and the vertical error is 0.9mm, and the positioning accuracy is significantly improved, meeting the precise docking requirements of the harvesting execution module and fruits and vegetables during harvesting.

[0046] When making harvesting decisions, the harvesting decision algorithm implements a "detection-identification-ranking" function. Based on YOLOv5-SC, it combines HSV maturity assessment, resulting in a clear decision-making logic. The YOLOv5-SC algorithm for object detection incorporates SE and CA attention mechanisms. SE enhances channel features through compression-excitation, while CA captures spatial coordinate features. The loss function uses VarifocalLoss. (9) In the formula, N is the number of samples. To predict the probability of positive samples, γ=2 enhances attention to difficult-to-distinguish samples and improves the detection capability of occluded scenes.

[0047] The task decision module designs the task sequence and crane movement path, and generates crane movement instructions, including: Select fruits and vegetables to be harvested based on their ripeness.

[0048] Based on the fusion operation coordinates, the initial coordinates of the harvesting execution module, and the visually estimated coordinates, the distance from each fruit and vegetable to be harvested to the initial position of the harvesting execution module is calculated.

[0049] Plan the work sequence and crane movement path according to the distance from smallest to largest.

[0050] The distance d from the fruit / vegetable to be harvested to the initial position of the harvesting execution module is calculated as follows: (10) In the formula, The initial coordinates for the harvesting execution module, Based on visual estimation, prioritize picking ripe fruits that are close by.

[0051] The accuracy of maturity recognition was tested, and the results are shown in the table below.

[0052] Fruit Maturity Identification Accuracy Test Table

[0053] It can be seen that the accuracy rate of mature fruit is 98.0%, and the overall accuracy rate is 94.2%, which meets the requirements for decision ranking.

[0054] In one embodiment, the drive unit of the picking execution module adopts PID control. Specifically, the drive unit of the picking execution module is a robotic arm for clamping servo motors, and its movement is controlled by PID to achieve precise control of motor position and speed. Taking the X-axis as an example, the PID formula and parameters are clearly defined.

[0055] The input to the PID controller is the position deviation. Output PWM duty cycle D(t): (11) In the formula, (Proportion coefficient) (Integral coefficient) (Differential coefficient), D(t) takes values ​​from 0 to 100%.

[0056] The relationship between motor speed n and PWM duty cycle is as follows: (12) X-axis and Z-axis max =150r / min, Y-axis n max =120r / min, to ensure that the movement speed matches the harvesting efficiency.

[0057] The motion control accuracy was tested by setting different target positions, and the results are shown in the table below.

[0058] Robotic arm motion control accuracy test table (unit: mm)

[0059] It can be seen that the average errors of the X, Y, and Z axes are 1.2mm, 1.0mm, and 0.8mm, respectively, which meet the positioning requirements of the robot arm.

[0060] The harvesting module dynamically adjusts the servo motor angle based on feedback from the fruit / vegetable diameter and clamping force to adaptively clamp the fruit / vegetable. Specifically, the fruit / vegetable diameter d is estimated from the visual bounding box width W as follows: (13) In the formula, 0.012mm / pixel is the camera calibration coefficient (1m distance), and W is the bounding box width (pixels), ensuring a diameter estimation error of ±0.5mm.

[0061] The relationship between the servo motor rotation angle \theta and the diameter d is as follows: (14) In the formula, 2.5° / mm is the proportionality coefficient, 30° is the initial rotation angle, and θ ranges from 30° to 150° to match fruits and vegetables of different diameters.

[0062] The adaptive clamping effect was tested, and the results are shown in the table below.

[0063] Picking Execution Module Adaptive Clamping Test Table

[0064] It can be seen that the clamping force fluctuation range is less than 0.6N, ensuring stable clamping without damaging the fruit.

[0065] Please see Figure 5 When the core board of the intelligent harvesting unit 1 is working, after power-on, it sequentially performs hardware initialization, GPIO configuration, and timer TIM2 initialization. The LCD flashes every 1 second. Then, timers TIM1 and TIM4 are configured for PWM output, and timer TIM3 channels 3 and 4 are configured for input capture. Digital-to-analog parameters and serial communication parameters are also set. Next, the X-axis and Y-axis camera offsets are acquired, the LCD is initialized, initial values ​​for the area are set, and the buzzer is controlled. After the power-on interface is displayed, matrix keyboard key recognition and independent key recognition are performed to select the area setting, manual harvesting, and automatic harvesting modes. After the LCD display, the process exits.

[0066] In this solution, the 3D model design of the intelligent harvesting unit 1 and the motion simulation under different working conditions enable designers to identify design flaws at the virtual prototype stage, allowing for timely optimization and modification of the design scheme. This ensures the performance of the intelligent harvesting unit 1. Furthermore, stress analysis of key components optimizes its structure, improving its performance and lifespan. The intelligent harvesting unit 1 is integrated with the STM32F103ZET6 core board as the control center, enabling data interaction and collaborative control among modules. The baseboard controller controls the OV7725 camera to acquire images via the SCCB interface. After HSV conversion and YOLOv5-SC algorithm recognition of the fruit, the coordinate data is transmitted to the core board via serial port. The core board reads pressure data from the ADS1118 via SPI, combines it with data from the capacitive displacement sensor and encoder, and obtains the target coordinates through multi-sensor fusion. A timer outputs a PWM signal to control the three-axis motor and servo motor to complete the harvesting action. The software control adopts a modular design. The baseboard program includes subroutines for image acquisition and fruit recognition, while the core board program includes subroutines for motion control and picking decision-making, ensuring clear software logic and strong maintainability.

[0067] In this study, a citrus orchard was selected, and three scenarios were set up: no obstruction, obstruction by branches and leaves (30% obstruction rate), and dense fruit (15 fruits / ㎡). Each scenario was tested 20 times, and key indicators were recorded. The results are shown in the table below.

[0068] Overall performance test table

[0069] The results showed that the machine performed best in unobstructed scenarios, with a recognition accuracy of 98.3% and a picking time of 4.7 seconds. Even in complex scenarios, the recognition accuracy remained above 88%, the damage rate was below 2.5%, and the positioning error was less than 1.5 mm, meeting orchard requirements. Compared to traditional mechanical harvesters, the machine's performance was significantly improved. In continuous operation tests, the equipment ran without failure for 2 hours, with an average efficiency of 28 harvesters per hour, higher than manual harvesting, verifying the equipment's stability and efficiency.

[0070] In addition, in practical applications, a remote monitoring module can be added to receive operation data through a mobile terminal, enabling remote start / stop and operation status viewing; it can also be connected to the greenhouse IoT system to optimize the timing of harvesting operations by combining data such as temperature, humidity, and light.

[0071] The above-mentioned greenhouse fruit and vegetable harvesting and transportation system can achieve at least the following beneficial effects: 1) Improved operational efficiency: The entire process of picking and transporting is automated without the need for manual intervention. Compared with the traditional manual assistance mode, the operational efficiency is increased by more than 50%, and a single device can cover a larger operating area.

[0072] 2) Precise and reliable positioning: Relying on the intelligent harvester algorithm and the servo drive of the crane, the positioning accuracy is high and can be adapted to the operation needs of different fruit and vegetable planting densities and different plant heights, increasing the harvesting success rate to over 98%.

[0073] 3) High adaptability: The lightweight crane design is compatible with various greenhouse structures, and the track 10 can be laid flexibly. It can be modified according to the existing greenhouse layout without large-scale reconstruction, reducing promotion costs; at the same time, it is compatible with the harvesting and transportation needs of various fruits and vegetables (strawberries, tomatoes, cucumbers, etc.).

[0074] 4) Reduced losses: The crane operates smoothly, and the algorithm adjusts the speed during the transfer process to avoid fruit collision and loss. The fruit integrity rate is 15%-20% higher than that of traditional transfer methods.

[0075] 5) High level of intelligence: Two-way data communication enables closed-loop control, which can automatically avoid obstacles, optimize the operation path, and has autonomous operation capability, greatly reducing reliance on manual labor and saving labor costs.

[0076] Example 2 This invention also discloses a method for harvesting and transporting greenhouse fruits and vegetables based on intelligent algorithm-based joint control. It is implemented using the greenhouse fruit and vegetable harvesting and transporting system of Example 1. The method includes the following steps: S1. Identify and locate mature target fruits and vegetables, and generate crane movement instructions.

[0077] S2. Control the crane to move along track 10 to above the target area according to the crane movement command, adjust the harvester to the preset working posture, and send back the harvester positioning signal.

[0078] S3. Harvest the target ripe fruits and vegetables and generate transfer instructions.

[0079] S4. Control the overhead crane to move to the designated collection point and drop the harvested mature target fruits and vegetables, output the transfer completion signal, and move the overhead crane to the next target operation position.

[0080] S5. Repeat steps S1-S4 until the entire area is completed.

[0081] The following section describes the specific methods of greenhouse fruit and vegetable harvesting and transportation systems, using the system's workflow as an example.

[0082] During initialization, the greenhouse crane unit 2 and the intelligent harvesting unit 1 first undergo power-on self-tests. The algorithm control unit 3 establishes a two-way communication connection with the greenhouse crane unit 2 and the intelligent harvesting unit 1. The operation decision module of the intelligent harvesting unit 1 stores basic data on the greenhouse fruit and vegetable planting layout. At this time, the crane resets to its initial position and stands by. Subsequently, the intelligent harvesting unit 1 scans the operation area through the visual recognition module to identify mature target fruits and vegetables. The position positioning module calibrates the precise coordinates of the operation. The operation decision module combines the coordinate information to generate crane movement instructions (including longitudinal and lateral movement distances and lifting heights) and sends them to the algorithm control unit 3, completing the identification of target fruits and vegetables and the generation of instructions. After the algorithm control unit 3 parses the instructions, it converts the parameters into crane drive signals. The walking drive module drives the crane along track 10 to move above the target area. The lifting adjustment module drives the intelligent harvesting unit 1 (or harvesting execution module) held by the pneumatic gripper 230 to descend vertically until the intelligent harvesting unit 1 reaches the optimal working posture. After the greenhouse crane unit 2 completes its positioning, it sends a positioning signal back to the intelligent harvesting unit 1, achieving precise positioning of the crane. After receiving the positioning signal, the intelligent harvesting unit 1 starts the harvesting execution module to complete the harvesting of the target fruits and vegetables. The harvested fruits and vegetables are temporarily stored in the harvester's buffer structure or directly placed into the matching transfer container. After harvesting, a transfer instruction is generated and sent to the algorithm control unit 3 to achieve fixed-point harvesting. The algorithm control unit 3 parses the transfer instruction, the crane's lifting adjustment module drives the intelligent harvesting unit 1 (or transfer container) to rise, the walking drive module drives the crane along the optimal trajectory to the designated collection point, the lifting adjustment module descends to complete the placement of the fruits and vegetables, and sends a transfer completion signal after placement, realizing the transfer of fruits and vegetables. After receiving the feedback signal that the transfer is completed, the overhead crane resets to the next target operation position to stand by. The intelligent harvesting unit 1 continues to identify the next batch of target fruits and vegetables, repeating the above process until the operation of the entire area is completed.

[0083] It can be understood that the greenhouse fruit and vegetable harvesting and transportation method in this embodiment is actually the workflow of the greenhouse fruit and vegetable harvesting and transportation system. Its specific algorithms and control processes for mature fruit and vegetable identification, target fruit and vegetable positioning, instruction generation, positioning signal feedback, harvesting action execution, and transfer operation are completely consistent with the algorithms and control processes of the greenhouse fruit and vegetable harvesting and transportation system in Embodiment 1 when performing the above actions. For details, please refer to the relevant description in Embodiment 1.

[0084] This embodiment breaks through the traditional independent control mode of overhead cranes and establishes a joint control mechanism led by the intelligent harvesting machine algorithm. Using the core algorithm of the intelligent harvesting machine as the command output, it directly commands the greenhouse overhead crane unit 2 to complete position movement and attitude adjustment, achieving precise placement and delivery of the harvesting equipment. This breaks down the barriers to collaboration between equipment and realizes the linkage control of harvesting and transportation. It integrates three core functions: "point-to-point delivery of harvesting equipment, coordinated completion of harvesting operations, and immediate transfer of harvested fruits." Relying on algorithmic linkage, it achieves seamless connection of the entire operation process, eliminating the need for manual intervention in the connection links and realizing integrated automated operation of harvesting and transportation. Based on the precise coordinates output by the visual recognition and position calibration algorithm of the intelligent harvesting unit 1, the greenhouse overhead crane unit 2 responds in real time and dynamically adjusts its running trajectory, overcoming the limitations of traditional preset trajectories, adapting to the differences in the growth positions of different fruits and vegetables, and meeting the positioning accuracy requirements for refined harvesting.

[0085] The intelligent algorithm-based greenhouse fruit and vegetable harvesting and transportation system and method of this invention integrates an intelligent harvesting unit 1 mounted on a greenhouse crane unit 2, enabling aerial mobile installation of the intelligent harvesting unit 1. This eliminates the influence of complex terrain within the greenhouse on the positioning of the intelligent harvesting unit 1. The algorithm-based control unit 3 controls the movement of the greenhouse crane unit 2, allowing the intelligent harvesting unit 1 to move precisely to a preset harvesting position, achieving precise harvesting of fruits and vegetables. The mounting of the intelligent harvesting unit 1 on the greenhouse crane unit 2 enables coordinated control between the two. By controlling the operation of the greenhouse crane unit 2 through the algorithm-based control unit 3, fruits and vegetables can be transferred after harvesting, achieving automatic connection between the harvesting and transfer processes, improving the automation level and operational efficiency of fruit and vegetable harvesting and transfer operations. The algorithm-based control unit 3 feeds back the operating status of the greenhouse crane unit 2 to the intelligent harvesting unit 1, allowing the intelligent harvesting unit 1 to perform harvesting operations after precise positioning. This achieves responsiveness to dynamic harvesting position commands and operational needs, improving harvesting accuracy and efficiency, reducing operating costs, and facilitating large-scale promotion.

[0086] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A greenhouse fruit and vegetable harvesting and transportation system based on intelligent algorithm-based joint control, characterized in that, include: The intelligent harvesting unit is used to identify and locate mature target fruits and vegetables, generate crane movement instructions, harvesting instructions and transfer instructions, and execute fruit and vegetable harvesting actions; The greenhouse crane unit is used to be movably arranged on the track on the top of the greenhouse and to carry the intelligent harvesting unit, and to drive the intelligent harvesting unit to move in response to the crane movement command or transfer command; The algorithm control unit is used to receive and forward the crane movement instructions or transfer instructions sent by the intelligent harvesting unit to the greenhouse crane unit, and to feed back the operating status of the greenhouse crane unit to the intelligent harvesting unit so that the intelligent harvesting unit can optimize the movement path according to the operating status.

2. The greenhouse fruit and vegetable harvesting and transportation system according to claim 1, characterized in that, The intelligent harvesting unit includes: The visual recognition module is used to scan the work area and identify mature target fruits and vegetables; The location positioning module is used to determine the operation coordinates by combining greenhouse layout data with real-time positioning information; The operation decision module is used to design the operation sequence and crane movement path according to the distribution density of fruits and vegetables, generate the crane movement instructions and send them to the algorithm control unit. The crane movement instructions include target position coordinates, movement speed and attitude adjustment parameters. The harvesting execution module is used to respond to harvesting commands and execute harvesting actions after the greenhouse crane unit is in place.

3. The greenhouse fruit and vegetable harvesting and transportation system according to claim 2, characterized in that, The visual recognition module scans the work area and identifies mature target fruits and vegetables, including: Photograph fruits and vegetables and output RGB format images; Calculate the normalized values ​​of the RGB three channels of the image, as well as the maximum and minimum values ​​of the normalized values; Calculate and determine the ripeness of fruits and vegetables based on the image's hue, saturation, and brightness.

4. The greenhouse fruit and vegetable harvesting and transportation system according to claim 2, characterized in that, The grasping part of the harvesting execution module is equipped with a pressure sensor for detecting the clamping force, and the clamping force threshold is different for different fruits and vegetables.

5. The greenhouse fruit and vegetable harvesting and transportation system according to claim 4, characterized in that, The calibration of the operation coordinates by the positioning module includes: obtaining the fused operation coordinates by fusing the visual coordinates of the image collected by the visual recognition module, the pressure data uploaded by the pressure sensor, the displacement sensor data and encoder data of the positioning module through multi-sensor fusion.

6. The greenhouse fruit and vegetable harvesting and transportation system according to claim 5, characterized in that, The operation decision module designs the operation sequence and the crane movement path, and generates the crane movement instructions, including: Select fruits and vegetables to be harvested based on their ripeness; Based on the fusion operation coordinates, the initial coordinates of the harvesting execution module, and the visually estimated coordinates, calculate the distance from each fruit and vegetable to be harvested to the initial position of the harvesting execution module; The work sequence and crane movement path are planned according to the distances from smallest to largest.

7. The greenhouse fruit and vegetable harvesting and transportation system according to claim 4, characterized in that, The greenhouse crane unit includes a crane arranged on the track, a walking drive module for driving the crane to move longitudinally and laterally along the greenhouse track, a lifting adjustment module for controlling the lifting and lowering of the picking execution module, and a clamping and fixing module for holding the picking execution module. The walking drive module and the lifting adjustment module receive instructions sent by the algorithm control unit and control the movement and lifting of the picking execution module. The walking drive module and the lifting adjustment module send operation parameters to the algorithm control unit to provide feedback on the crane's operating status.

8. The greenhouse fruit and vegetable harvesting and transportation system according to claim 7, characterized in that, The drive unit of the picking execution module adopts PID control; the picking execution module dynamically adjusts the servo motor angle based on the fruit and vegetable diameter and clamping force feedback to adaptively clamp the fruit and vegetables.

9. The greenhouse fruit and vegetable harvesting and transportation system according to claim 1, characterized in that, The greenhouse fruit and vegetable harvesting and transportation system also includes a fruit and vegetable transfer unit, which is used to receive the harvested fruits and vegetables and, driven by the greenhouse crane unit, transfer the fruits and vegetables from the working position to the designated collection point in the greenhouse.

10. A method for harvesting and transporting greenhouse fruits and vegetables based on intelligent algorithm-based joint control, characterized in that, The greenhouse fruit and vegetable harvesting and transportation system described in any one of claims 1-9 is adopted, and the greenhouse fruit and vegetable harvesting and transportation method includes the following steps: S1. Identify and locate mature target fruits and vegetables, and generate crane movement instructions; S2. Control the crane to move along the track to above the target area according to the crane movement command, adjust the harvester to the preset working posture, and send back the harvester positioning signal; S3. Harvest ripe target fruits and vegetables and generate transfer instructions; S4. Control the overhead crane to move to the designated collection point and drop the harvested ripe target fruits and vegetables, output the transfer completion signal, and move the overhead crane to the next target operation position; S5. Repeat steps S1-S4 until the entire area is completed.

Images