Automatic imitation tea leaf collection method and device based on point cloud and color information

Abstract

To provide a method for automatically collecting imitation tea leaves based on point groups and color information, and a device therefor.SOLUTION: A surface of a tea leaf is scanned by an RGB-D camera to obtain a depth image. With a preliminary collection cutting depth standard s obtained by fitting of depth information of the surface of the tea leaf in each sampling region in a collection object region, a ratio P of raw leaf pixel extracted by a depth image of the surface of the tea leaf, and a depth dispersion average value Var in a vertical direction of the tea raw leaf as input, and with an optimum cutter imitation collection cutting depth standard as output, a BP neural network model is established, and an actual control variable of the imitation collection cutting depth of both ends of the cutter is obtained using a linear self-interference prevention control method. The cutting depth of both ends of the cutter is automatically adjusted by each actual control variable, and automatic imitation tea leaf collection is performed for the surface of the tea leaf. By controlling the posture of the cutter, the cutter imitates the surface of a tea ridge from three positions, achieves covering for the surface of the tea leaf, and improves efficiency and quality of imitation collection of a large amount of tea.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention belongs to the agricultural machinery technical field, and relates to the application of a combination of image processing technology and computer control technology in the field of tea picking machinery, and more particularly to a method and device for automatically imitation tea leaf picking based on point cloud and color information. [Background technology]

[0002] In recent years, China's tea production has developed rapidly, with increasing area and production volume. However, although China's tea industry is large, it is neither strong nor sophisticated, and the scientific and technological content and mechanization level of tea products are low, which has affected the sustainable and healthy development of the tea industry. The currently mainly adopted artificial plucking method for plucking fresh leaves before they are first made into tea products is time-consuming, labor-intensive, and inefficient. In addition, the quality of the fresh leaves is uneven due to differences in the physical strength, skill, and experience of the tea pluckers, so it is only suitable for plucking small batches of high-quality tea with one bud and one leaf or one bud and two leaves. Conventional tea plucking machines on the market are mainly used for plucking large quantities of tea leaves with relatively low requirements for fresh leaves. A single-person handheld tea plucking machine uses an artificial handheld device to pick new, tender tea leaves from the tips of tea plants, which improves the efficiency of tea leaf collection compared with manual plucking. However, the overall cost of manual plucking is high, which increases the production cost of tea leaves, and the time available for plucking tea leaves is limited, making manual plucking less efficient and significantly limiting the tea leaf production volume.

[0003] To improve tea harvesting efficiency, several automated imitation tea harvesting devices have been put into practical use. For example, patent publication number CN113039936A uses an ultrasonic distance measurement automatic tea harvesting method, but because ultrasonic waves can only measure point-to-point, it is greatly affected by the pitch of the tea leaves and is not effective for harvesting sparse tea leaf surfaces (referring to the surface of the tea plant crown). Other imitation tea harvesting devices, such as those using 2D-LiDAR distance measurement, use 2D-LiDAR point clouds to fit the cutter imitation harvesting cutting depth standard. However, because the amount of tea leaf surface information obtained by 2D-LiDAR is limited, it is greatly affected by the growth density and growth vigor of the fresh leaves, resulting in large deviations in the estimated imitation harvesting cutting depth standard.

[0004] Tea leaves are often planted in hilly and mountainous areas, where ditch interference is significant. To achieve good harvesting results, the control system must overcome the interference effects of ditch interference. Traditional PID and its improved algorithms are prone to overshoot or insufficient response when faced with interference. Self-anti-interference controllers have the characteristic of estimating the total disturbance of the system, but extended state observers lack estimation accuracy when interference changes rapidly. Although compensation can be achieved using a disturbance mathematical model, the unevenness of tea fields makes it difficult to establish a disturbance model, resulting in reduced anti-interference control performance.

[0005] To address the above problems, an automatic imitation harvesting method that can adapt to different tea leaf growth conditions and has strong anti-interference control capabilities is urgently needed, which can improve the efficiency and quality of large-scale imitation tea harvesting. Summary of the Invention

[0006] The objective of the present invention is to provide an automatic imitation tea leaf picking method and apparatus based on point cloud and color information in response to the shortcomings of the prior art.

[0007] The present invention provides an automatic imitation tea leaf collection method based on point cloud and color information, and the specific steps are as follows: In step 1, the lifting device and cross member are attached to the body frame of the moving device, and the lifting device drives the cross member to move up and down. Cutter assembly 1 and cutter assembly 2 are symmetrically attached to the cross member, each including a fixed frame, an upper cutter, and a lower cutter. In cutter assembly 1 or cutter assembly 2, the upper cutter and lower cutter are arranged parallel to each other and spaced apart on the fixed frame, and each is driven to reciprocate by drive member 1 or drive member 2 attached to the fixed frame. One end located inside the fixed frame of cutter assembly 1 or cutter assembly 2 is hingedly connected to the center of the cross member, and one end located outside is hingedly connected to the push rods of attitude adjustment cylinder 1 and attitude adjustment cylinder 2, respectively. The cylinder bodies of attitude adjustment cylinder 1 and attitude adjustment cylinder 2 are hingedly connected to both ends of the cross member. Ducts 1 and 2 are attached above the two upper cutters on the two fixed frames, respectively, and are connected to blower 2 and blower 1, respectively, attached to the cross member. Collection bags are attached to the rear openings on both fixed frames. The tail plate is hingedly connected to a location on the cross member behind cutter assembly 1 and cutter assembly 2, and is driven to rotate by two telescopic cylinders attached to the cross member. In the initial state, the piston rods of the two telescopic cylinders are retracted, and the tail plate is in a stored state. The RGB-D camera and accelerometer are both attached to the vehicle frame by sensor holders.

[0008] In step 2, the controller controls the two telescopic cylinders to synchronously drive the tail plate to rotate rearward to the deployed state, the lifting device drives the cross member to lift and lower, and further lifts and lowers cutter assembly 1, cutter assembly 2, attitude adjustment cylinder 1, attitude adjustment cylinder 2, duct 1, duct 2 and the tail plate, so that each cutter of cutter assembly 1 and cutter assembly 2 lifts and lowers to a preset initial height position and is positioned above the surface of the tea leaves. Then, the moving device drives the body frame to move through the tea furrows and start the mass automatic tea imitation picking operation.

[0009] In step three, during the collection process, the surface of the tea leaves is scanned with an RGB-D camera to obtain depth images.

[0010] In step four, surface depth information of the target area is extracted from the depth image, and outliers are removed using a radial filtering method to obtain a surface depth information image of the target area. An accelerometer is used to obtain the vertical acceleration of the RGB-D camera, and a Kalman filter algorithm is used to measure the surface depth information of the target area. The surface depth information image of the target area is then recursively corrected by combining the vertical acceleration of the RGB-D camera. Next, a preset width range area located at the center of the target area's surface and preset width range areas located at both ends are used as three sampling areas, and a cutter pre-collection cutting depth criterion s is fitted for each sampling area using a RANSAC algorithm.

[0011] In step five, the RGB information of the surface of each sampling area in the target area is extracted from the depth image to obtain an RGB information image of the tea leaf surface of each sampling area in the target area. In the depth information of the RGB information image of the tea leaf surface of each sampling area in the target area, pixels located in the upper predetermined range are directly classified as fresh leaves, and pixels in the lower predetermined range are directly classified as background. For the remaining image pixels, color components with large differences between the fresh leaves and the background are selected as the basis for division, and the remaining image pixels are divided into two cluster types, background and fresh leaves, using a variable threshold K-means clustering algorithm. Then, the proportion P of fresh leaf pixels is calculated.

[0012] In step six, a depth information image of the tea leaf surface in each sampling area of ​​each collection target area is obtained based on the surface depth information of each collection target area, and the vertical depth variance average value Var of the raw tea leaves is extracted from the depth information image of the tea leaf surface in each sampling area.

[0013] In step seven, a BP neural network model is constructed to optimize the optimal cutter-imitating harvesting cutting depth criterion S for each sampling area, and then trained and validated.

[0014] In step 8, during the harvesting process, the cutter preliminary harvesting cutting depth standard s for each sampling area in each harvesting target area is fitted in real time, and the raw leaf pixel proportion P on the surface of the tea leaves and the vertical depth variance average value Var for each sampling area in each harvesting target area are obtained, and input into the trained BP neural network model to output the optimal cutter-imitated harvesting cutting depth standard S for each sampling area in each harvesting target area in real time, and the encoder built into the lifting motor of the lifting device is used to obtain in real time the position information of each cutter in cutter assembly 1 and cutter assembly 2 positioned at one end of the inside, and the encoder built into attitude adjustment cylinder 1 and attitude adjustment cylinder 2 is used to obtain the position information of each cutter in cutter assembly 1 and cutter assembly 2 positioned at one end of the outside, respectively.

[0015] In step 9, based on the position information of each cutter and the optimal cutter imitation harvesting cutting depth criterion S of the corresponding sampling area output from the BP neural network model, a linear anti-self-interference control method is adopted to obtain the actual control amount of the imitation harvesting cutting depth at both ends of each cutter.

[0016] In step ten, the lifting device, and the attitude adjusting cylinders 1 and 2 automatically adjust the cutting depth of both ends of each cutter based on the actual control amount u corresponding to the real-time value, thereby realizing automatic imitation tea picking on the surface of the picking target area.

[0017] In step 11, when performing automatic imitation tea picking, the driving members 1 and 2 drive the corresponding upper cutters and lower cutters to reciprocate relative to each other, cutting the roots of the fresh tea leaves, and simultaneously control the operation of the blowers 1 and 2, so that the air blown from the blowers 1 and 2 is blown above the cutters of the cutter assembly 1 and the cutter assembly 2 through the ducts 2 and 1, respectively, and the cut fresh tea leaves are blown into the fixed frames and further into the collecting bags to collect the picked fresh tea leaves.

[0018] Preferably, the remaining image pixels in step 5 are divided into two cluster types, background and green leaves, by variable threshold K-means clustering algorithm, and the specific steps are as follows: (1) For the remaining image pixel samples, the initial value of the color space distance threshold is set, and the maximum number of iterations of the K-means clustering algorithm is set. Then, two pixel samples are randomly selected, and the pixel with the smallest value of the base color component is set as the initial background cluster center, and the other pixel is set as the initial leaf cluster center. (2) Using the K-means clustering algorithm, the remaining image pixels are divided into two cluster types: background and fresh leaves. Then, it is determined whether the color space Euclidean distance between the background cluster center and the fresh leaf cluster center is greater than the color space distance threshold. If the color space Euclidean distance between the background cluster center and the fresh leaf cluster center is greater than the color space distance threshold, the division is completed; otherwise, step (3) is performed. (3) Determine whether the number of iterations is greater than the set maximum number of iterations. If it is greater than the maximum number of iterations, decrease the color space distance threshold and return to step (2). Otherwise, for the cluster type with the highest degree of variance, the pixel with the maximum value of the splitting basis color component is set as the new center of the green leaf cluster, and for the cluster type with the highest degree of variance, the pixel with the minimum value of the splitting basis color component is set as the new center of the background cluster, and return to step (2).

[0019] Preferably, the specific steps of the above step 6 are as follows: (1) Based on the surface depth information of the sampling area, obtain the surface depth information image of the tea leaves in each sampling area within the sampling area, divide the surface depth information image of the tea leaves in each sampling area into m areas based on equal area, and calculate the vertical depth variance of the point cloud in each area, as follows:

number

number

[0020] Preferably, the specific steps of the above step 7 are as follows: (1) Select multiple different tea leaf surfaces, and for each tea leaf surface, obtain the vertical depth variance average value Var, the raw leaf pixel proportion P, and the cutter pre-collection cutting depth criterion s of each sampling area within each collection target area, and artificially determine the optimal cutter-imitated collection cutting depth criterion S for each sampling area, thereby establishing a data set including the vertical depth variance average value Var, the raw leaf pixel proportion P, the cutter pre-collection cutting depth criterion s, and the optimal cutter-imitated collection cutting depth criterion S for each sampling area. (2) A BP neural network model is constructed, taking the average vertical depth variance Var, the proportion of live leaf pixels P, and the cutter pre-harvesting cutting depth criterion s for each sampling area as inputs, and the optimal cutter-imitating harvesting cutting depth criterion S for each sampling area as output. The dataset is divided into a test set and a validation set, and the BP neural network model is trained and validated.

[0021] Preferably, in step 9 above, the specific steps for obtaining the actual control amount of the cutter-simulated harvesting cutting depth using the linear anti-self-interference control method are as follows: (1) The difference value h and the rate of change v between the two end positions of each cutter and the optimal cutter-imitating cutting depth criterion of the corresponding sampling area output from the BP neural network model are used as state variables, and the dynamic control equation for the controlled object is established:

number

number

number

number

number

[0022] Preferably, the RGB-D camera is replaced with a module consisting of a 3D radar and an RGB camera. The 3D radar scans the surface of the tea leaves to obtain depth information, and the RGB camera scans the surface of the tea leaves to obtain RGB information, and a depth image is obtained using the depth information and RGB information.

[0023] The automatic imitation tea leaf harvesting device based on point cloud and color information of the present invention includes a moving device, a lifting device, an imitation harvesting device, and a sensor assembly.

[0024] The counterfeit collection device includes a tail plate, a cross member, a blower 1, a blower 2, a position adjustment cylinder 1, a duct 1, a cutter assembly 1, a cutter assembly 2, a duct 2, and a position adjustment cylinder 2. The moving device drives the body frame to move it. The cross member and the body frame form a vertical slide pair, and the lifting device installed on the body frame drives the lifting. Cutter assembly 1 includes a fixed frame, an upper cutter, and a lower cutter. Both have openings at the front and rear positions on the fixed frame, and the upper and lower cutters are arc-shaped cutters that are parallel and stacked one above the other, form a horizontal slide pair with the position located at the front opening of the fixed frame, and are driven to reciprocate by driving member 1. The structure of cutter assembly 2 is identical to that of cutter assembly 1, and both the upper and lower cutters of cutter assembly 2 are driven to reciprocate by driving member 2. Cutter assembly 1 and cutter assembly 2 are arranged symmetrically, and one end located inside the fixing frame of cutter assembly 1 and cutter assembly 2 is both connected to the middle of the cross member by a hinge. Attitude adjustment cylinder 1 and attitude adjustment cylinder 2 are located above cutter assembly 1 and cutter assembly 2, respectively, and the cylinder bodies of attitude adjustment cylinder 1 and attitude adjustment cylinder 2 are connected to both ends of the cross member by a hinge, and the push rods of attitude adjustment cylinder 1 and attitude adjustment cylinder 2 are connected to one end located outside the fixing frame of cutter assembly 2 and cutter assembly 1 by a hinge. Blowers 1 and 2 are both fixed to the cross member. The duct 1 and duct 2 are fixed to the fixing frames of cutter assembly 1 and cutter assembly 2, respectively, and are located above the upper cutters of cutter assembly 1 and cutter assembly 2, respectively, and the intake ports of duct 1 and duct 2 communicate with the exhaust ports of blower 2 and blower 1 via elastic hose 2 and elastic hose 1, respectively, and a plurality of exhaust holes are opened in duct 1 and duct 2 at positions close to the corresponding upper cutters, all of which are spaced apart along the axial direction.The tail plate is disposed behind the cutter assembly 1 and the cutter assembly 2, and both ends of the tail plate are connected to both ends of the cross member by hinges, respectively, and are rotated synchronously by two telescopic cylinders.

[0025] The sensor assembly includes an RGB-D camera, an accelerometer, and a sensor holder, and both the RGB-D camera and the accelerometer are fixed to the vehicle frame by the sensor holder.

[0026] The present invention has the following beneficial effects: 1. The present invention uses a lifting device to control the overall height of each cutter, and also controls the height of the inner end of each cutter, and uses posture adjusting cylinder 2 and posture adjusting cylinder 1 to control the height of the outer end of each cutter in cutter assembly 1 and cutter assembly 2, respectively, thereby realizing control of the posture of each cutter. Each cutter in cutter assembly 1 and cutter assembly 2 can simulate the surface of the tea ridges from two positions, and further achieve complete coverage of the tea leaf surface, improving the efficiency and quality of mass tea simulation harvesting. 2. This invention uses an RGB-D camera to scan the tea leaf surface and extract a depth image of the tea leaf surface (including depth information and RGB image information). It uses an accelerometer to capture the vertical acceleration of the RGB-D camera and a Kalman filter algorithm to recursively correct the depth image of the tea leaf surface by combining the vertical acceleration of the RGB-D camera. It then uses the depth information of the tea leaf surface to fit a cutter pre-harvesting cutting depth criterion for each sampling area within the target area. Next, the growth density is estimated using the horizontal variance of the depth information of the tea leaf surface, and the tea leaf surface depth image is segmented using a variable threshold K-means algorithm to extract the live leaf pixel proportion for each sampling area. Finally, the cutter pre-harvesting cutting depth criterion, growth density, and live leaf pixel proportion parameters are combined to establish an optimal cutter imitation harvesting cutting depth criterion estimation model. This has advantages over simply relying on RGB camera images or 3D LiDAR point clouds to estimate the imitation harvesting cutting depth criterion, and can realize the harvesting of tea leaves with different growth stages. The sampling frequency of the accelerometer is several times that of the RGB-D camera, and the accelerometer can capture the vertical acceleration of the RGB-D camera to correct the surface depth information image of the tea leaves, greatly improving the measurement accuracy of imitation collection. The RGB-D camera can also be replaced with an assembly consisting of a 3D radar and an RGB camera, where the 3D radar scans the surface of the tea leaves to obtain depth information, and the RGB camera scans the surface of the tea leaves to obtain RGB information. 3. During the imitation harvesting process, it is desirable for the cutter to cut and harvest the fresh tea leaves according to the optimal cutter imitation harvesting cutting depth standard. However, under the uneven interference conditions of the ridge and furrow undulations in hilly and mountainous tea fields, the uneven interference from the ridge and furrow undulations is transmitted to the end cutter through the tea plucking device, causing cutter vibration and seriously affecting the quality of imitation harvesting of fresh leaves. This invention provides for the input of the disturbance measurement signal from the ridge and furrow undulations into a linear anti-self-interference control system, and the filtered disturbance measurement signal is used as a known disturbance. The improved design of the extended state observer allows the extended state observer to improve the estimation accuracy and convergence speed of the rapidly changing total disturbance by introducing the measurement value of the known disturbance, thereby improving the robustness and dynamic response performance of the control system. This invention has relatively strong anti-interference control ability during the harvesting process. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a schematic diagram of the overall structure of the present invention; [Figure 2] 1 is a structural schematic diagram of a moving device according to the present invention; [Figure 3] 1 is a structural schematic diagram of a rope-driven lifting device and an imitation collecting device according to the present invention; [Figure 4] 1 is a structural schematic diagram of a rope-driven lifting device according to the present invention; [Figure 5] 1 is a structural perspective view of a counterfeit collection device according to the present invention; [Figure 6] FIG. 2 is a front view of the imitation collection device of the present invention. [Figure 7] 3 is a cross-sectional view of an upper cutter, a lower cutter, and a first driving member or a second driving member in the present invention. FIG. [Figure 8] 3 is a structural schematic diagram of an upper cutter and a lower cutter according to the present invention. FIG. [Figure 9] 1 is a structural schematic diagram of a sensor assembly according to the present invention; [Figure 10] FIG. 1 is a schematic diagram of Kalman filter fusion in the present invention. [Figure 11] 1 is a flowchart of the automatic imitation tea harvesting method of the present invention. [Figure 12] FIG. 2 is a block diagram of a cutter cutting depth control system according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0028] The invention will now be further explained with reference to the drawings. As shown in FIG. 1, the automatic imitation tea leaf harvesting device based on point cloud and color information of the present invention includes a moving device 1, a lifting device 2, an imitation harvesting device 3, and a sensor assembly 4.

[0029] As shown in Figures 5, 6 and 8, the counterfeit collection device 3 includes a tail plate 301, a cross member 303, a blower 1 305, a blower 2 306, a position adjustment cylinder 1 309, a duct 1 310, a cutter assembly 1 311, a cutter assembly 2 313, a duct 2 314 and a position adjustment cylinder 2 315. The movement device 1 drives the body frame to move. The cross member 303 and the body frame form a vertical sliding pair, and the lifting and lowering is driven by a lifting device 2 provided on the body frame. The cutter assembly 1 311 includes a fixed frame, an upper cutter 322 and a lower cutter 323. The fixed frame has openings at its front and rear locations, and both upper cutter 322 and lower cutter 323 are arc-shaped cutters. Upper cutter 322 and lower cutter 323 are arranged parallel to each other and stacked one on top of the other. They form a horizontal sliding pair with the location at the front opening on the fixed frame, and both are driven to reciprocate by driving member 1 319. The location at the rear opening on the fixed frame is used for attaching collection bags. Cutter assembly 2 313 has a structure identical to that of cutter assembly 1 311, and both upper cutter 322 and lower cutter 323 of cutter assembly 2 313 are driven to reciprocate by driving member 2 317. Cutter assembly 1 311 and cutter assembly 2 313 are arranged symmetrically, and the inner ends of cutter assembly 1 311 and cutter assembly 2 313 on the fixed frame are both hingedly connected to the middle of cross member 303. Attitude adjustment cylinder one 309 and attitude adjustment cylinder two 315 are located above cutter assembly one 311 and cutter assembly two 313, respectively, and the cylinder bodies of attitude adjustment cylinder one 309 and attitude adjustment cylinder two 315 are hingedly connected to both ends of cross member 303, and the push rods of attitude adjustment cylinder one 309 and attitude adjustment cylinder two 315 are hingedly connected to one end located outside the fixing frames of cutter assembly two 313 and cutter assembly one 311, respectively. Blower one 305 and blower two 306 are both fixed to cross member 303.Duct one 310 and duct two 314 are fixed to the fixed frames of cutter assembly one 311 and cutter assembly two 313, respectively, and are located above the upper cutters 322 of cutter assembly one 311 and cutter assembly two 313, and the intake ports of duct one 310 and duct two 314 are connected to the exhaust ports of blower two 306 and blower one 305 via telescopic hose two 308 and telescopic hose one 304, respectively, and a plurality of exhaust holes are opened in duct one 310 and duct two 314 near the corresponding upper cutters 322, spaced apart along the axial direction. Tail plate 301 is installed behind cutter assembly one 311 and cutter assembly two 313, and both ends of tail plate 301 are connected to both ends of the cross member by hinges, and are rotated synchronously by two telescopic cylinders. The tail plate 301 is used to support the collecting bag, and in the initial state, the piston rods of the two telescopic cylinders are retracted and the tail plate 301 is in the retracted state.

[0030] As shown in FIG. 9, the sensor assembly 4 includes an RGB-D camera 403, an accelerometer 402, and a sensor holder 401, and the RGB-D camera 403 and the accelerometer 402 are both fixed to the vehicle frame by the sensor holder 401.

[0031] In a preferred embodiment, as shown in FIG. 2, the moving device 1 employs a crawler-type traveling mechanism 101, with a crawler-type traveling mechanism 101 installed on each side of the body frame. The two crawler-type traveling mechanisms 101 drive the body frame to move, and are powered by battery compartments 107 and 108, respectively, installed in the housings of the two crawler-type traveling mechanisms 101.

[0032] More preferably, the vehicle body frame includes pillar one 102, pillar two 103, pallet one 104, and pallet two 105. Pallet one 104 and pallet two 105 are arranged horizontally and parallel to each other, with pallet one 104 located in front of pallet two 105. The two front corners of pallet one 104 are fixed to the housings of the two crawler-type traveling mechanisms 101 via pillars one 102 and two 103. The two rear corners are fixed to the two front corners of pallet two 105 via two connecting pillars, and the two rear corners of pallet two 105 are fixed to the housings of the two crawler-type traveling mechanisms 101 via pillars three and four, and the height of pallet one 104 is greater than that of pallet two 105.

[0033] More preferably, connecting member one 302 and connecting member two 307 are fixed to both ends of cross member 303. Connecting member one 302 and connecting member two 307 form sliding pairs with upright pillar one 102 and upright pillar two 103, respectively, and rollers are hingedly connected to connecting member one 302 and connecting member two 307. The two rollers and upright pillar one 102 and upright pillar two 103 form rolling friction pairs, respectively. Both ends of the tail plate are hingedly connected to connecting member one and connecting member two.

[0034] 3 and 4, the lifting device 2 includes a rotating shaft, fixed pulley one 201, fixed pulley two 202, fixed pulley three 211, wire rope one 203, wire rope two 210, take-up reel one 218, take-up reel two 217, and lifting motor 209. The rotating shaft is horizontal, forms a rotating pair with the body frame, and is driven by lifting motor 209. Fixed pulley one 201 and fixed pulley two 202 are both located on one side of the rotating shaft and form a rotating pair with the body frame, with fixed pulley two 202 being closer to the rotating shaft than fixed pulley one 201. Fixed pulley three 211 is located on the other side of the rotating shaft and form a rotating pair with the body frame. The rotational central axes of fixed pulley one 201, fixed pulley two 202, and fixed pulley three 211 are all parallel to the central axis of the rotating shaft. Take-up reel one 218 and take-up reel two 217 are both fixed to the rotating shaft and are both installed coaxially with the rotating shaft, one end of wire rope one 203 is wound around take-up reel one 218 and fixed to take-up reel one 218, the other end passes around fixed pulley two 202 and fixed pulley one 201 in turn and passes through a hole opened in the body frame and is fixed to one end of cross member 303. One end of wire rope two 210 is wound around take-up reel two 217 and fixed to take-up reel two 217, the other end passes around fixed pulley three 211 and passes through another hole opened in the body frame and is fixed to the other end of cross member 303. The winding direction of wire rope 1 203 around winding reel 1 218 is the same as the winding direction of wire rope 2 210 around winding reel 2 217, ensuring that both ends of cross member 303 rise and fall simultaneously.

[0035] More preferably, lead screw slider module 1 204 is provided at a position located between the rotation shaft and fixed pulley 2 202 on the body frame, and lead screw slider module 2 212 is provided at a position located between the rotation shaft and fixed pulley 3 211. Lead screw slider module 1 204 and lead screw slider module 2 212 are used to uniformly wind wire rope 1 203 and wire rope 2 210 around take-up reel 1 218 and take-up reel 2 217, respectively. The bases of lead screw slider module 1 204 and lead screw slider module 2 212 are both fixed to the body frame. The lead screws of lead screw slider module 1 204 and lead screw slider module 2 212 are both parallel to the rotation shaft and are connected to both ends of the rotation shaft via belt transmission mechanism 1 and belt transmission mechanism 2, respectively. The sliders of lead screw slider module one 204 and lead screw slider module two 212 are respectively provided with guide sheave group one 205 and guide sheave group two 213. Guide sheave group one 205 and guide sheave group two 213 each consist of two guide sheaves spaced apart, a guide sheave and a corresponding slider form a rotating pair, and the rotation center axis of the guide sheave is perpendicular to the horizontal plane. Wire rope one 203 and wire rope two 210 pass through the middle positions of the two guide sheaves of guide sheave group one 205 and guide sheave group two 213, respectively, with wire rope one 203 running around the two guide sheaves of guide sheave group one 205 and wire rope two 210 running around the two guide sheaves of guide sheave group two 213.

[0036] More preferably, belt transmission mechanism 1 includes synchronous pulley 1 214, synchronous belt 1 215, and synchronous pulley 2 216. Synchronous pulley 216 and synchronous pulley 1 214 are connected by synchronous belt 1 215 and are fixed to the lead screw optical axis segment and rotation axis of lead screw slider module 1 204, respectively.

[0037] More preferably, belt transmission mechanism 2 includes synchronous pulley 3 206, synchronous belt 207, and synchronous pulley 4 208. Synchronous pulley 4 208 and synchronous pulley 3 206 are connected by synchronous belt 2 207 and are fixed to the lead screw optical axis segment and rotation axis of lead screw slider module 2 212, respectively.

[0038] 7 and 8, drive member-319 includes a drive shaft 320, an upper eccentric wheel 321, a lower eccentric wheel 325, and a mounting bracket. The mounting bracket is fixed to the fixed frame of cutter assembly-311, and the outer ends of upper cutter 322 and lower cutter 323 of cutter assembly-311 are both located within the mounting bracket. The drive shaft and mounting bracket form a rotating pair and are driven to rotate by drive motor-318. Upper eccentric wheel 321 and lower eccentric wheel 325 are stacked parallel to each other within the mounting bracket and are both fixed to the drive shaft. They are located within waist-shaped holes-1 326 and waist-shaped holes-2 324, respectively, which are opened in upper cutter 322 and lower cutter 323 of cutter assembly-311, and form a cam pair with waist-shaped holes-1 326 and waist-shaped holes-2 324, respectively. The structure of drive member 2 317 is identical to that of drive member 1 319, and the mounting bracket of drive member 2 317 is fixed to the fixed frame of cutter assembly 2 313. The drive shaft is driven to rotate by drive motor 2 316. The outer ends of upper cutter 322 and lower cutter 323 of cutter assembly 2 313 are both located within the mounting bracket of drive member 2 317. Upper eccentric wheel 321 and lower eccentric wheel 325 are located within waist-shaped hole 1 326 and waist-shaped hole 2 324, respectively, opened in upper cutter 322 and lower cutter 323 of cutter assembly 2 313, and form a cam pair with waist-shaped hole 1 326 and waist-shaped hole 2 324, respectively, opened in upper cutter 322 and lower cutter 323 of cutter assembly 2 313. The mounting phase difference between upper eccentric wheel 321 and the corresponding lower eccentric wheel 325 is 180°.

[0039] In a preferred embodiment, a leaf separating plate 312 is fixed to the cross member 303 at a position midway between cutter assembly 2 313 and cutter assembly 1 311. The leaf separating plate 312 separates the tea leaves located in the middle of the tea ridge during the automated imitation harvesting process and moves to both sides to cut the tea leaves located in the middle of the tea ridge.

[0040] Among them, the first attitude adjustment cylinder 309, the second attitude adjustment cylinder 315, the telescopic cylinder, the lift motor 209, the first drive motor 318, and the second drive motor 316 are all controlled by a controller 106 provided on the second pallet 105.

[0041] As shown in Figure 11, the method for automatically imitation tea leaf collection based on point cloud and color information of the present invention is specifically as follows: In step 1, a collection bag is attached to each fixed frame at a location corresponding to the rear opening.

[0042] In step 2, the controller 106 controls the two telescopic cylinders to synchronously drive the tail plate 301 to rotate backward to the deployed state, the lifting device 2 drives the cross member 303 to lift and lower, and then the entire imitation collecting device 3 is lifted and lowered, so that each cutter is lifted and lowered to a preset initial height position and positioned above the surface of the tea leaves. Then, the moving device 1 drives the lifting device 2, the imitation collecting device 3 and the sensor assembly 4 to move through the tea ridges and start the large-scale automated imitation tea picking operation.

[0043] In step 3, when performing automated imitation harvesting of large quantities of tea, each cutter cuts the tea leaves completely so that they can cut along the base of the tea leaves, and therefore the base of the tea leaves is set as the optimal cutter imitation harvesting cutting depth reference position.In the actual imitation harvesting process, since the sensor cannot directly sense the imitation harvesting cutting depth reference position, the present invention obtains the optimal cutter imitation harvesting cutting depth reference position by performing algorithm processing on the sensor detection information, and scans the surface of the tea leaves with the RGB-D camera 403 to obtain a depth image (including depth information and RGB information).

[0044] In step four, assuming that the tea leaves within the scanning range of the RGB-D camera 403 have the same leaf length and growth density, depth information of the tea leaf surface in the target area is extracted from the depth image, and outliers are removed using a radius filtering method to obtain a depth information image of the tea leaf surface in the target area. Because outdoor RGB-D cameras are sensitive to natural light when acquiring depth information, time-of-flight measurement (ToF) is used to acquire depth information. The sampling frequency of depth information images is typically below 50 Hz, and the measurement accuracy is typically 2-3 cm, which cannot meet the measurement accuracy requirements of imitation harvesting. The sampling frequency of the accelerometer 402 is several times higher than that of the RGB-D camera. Therefore, the present invention acquires the vertical acceleration of the RGB-D camera via the accelerometer 402 while moving, and uses a Kalman filter algorithm to measure depth information of the tea leaf surface in the target collection area, combining the vertical acceleration state of the RGB-D camera to recursively correct the depth information image of the tea leaf surface in the target collection area, as shown in Figure 10, where k represents time. Next, a 10 cm wide area located at the center of the tea leaf surface in the target collection area and 10 cm wide areas located at both ends are set as three sampling areas, and a cutter preliminary collection cutting depth reference s is fitted to each sampling area using the RANSAC algorithm (the fitted s is basically a horizontal line segment, but if s is a line segment with a small inclination angle, s can be replaced with a line segment passing through the midpoint of s to be used as the cutter preliminary collection cutting depth reference). In this case, if the growth density on the surface of the tea leaves is high, the expected cutter preliminary collection cutting depth standard s value is likely to be high or, conversely, low; and if the raw tea leaves are long, the expected cutter preliminary collection cutting depth standard s value is likely to be high or, conversely, low.

[0045] In step five, RGB information on the surface of each sampling area in the target area is extracted from the depth image to obtain an RGB information image of the tea leaf surface in each sampling area in the target area. Under natural conditions, fresh leaves on the tea leaf surface occlude each other, and the colors of old leaves and fresh leaves are similar, making it impossible to directly obtain the length of the fresh leaves. However, statistical analysis shows that there is a positive correlation between the proportion of fresh leaf pixels and the length of fresh leaves after image segmentation. This invention uses the proportion of fresh leaf pixels to represent the length of fresh leaves. Based on the spatial distribution characteristics of fresh leaves, pixels located in the upper predetermined range of depth information in the RGB information image of the tea leaf surface in each sampling area in the target area are directly classified as fresh leaves, and pixels located in the lower predetermined range are directly classified as background. For the remaining image pixels, a color component with a large difference between the fresh leaves and the background is selected as the basis for segmentation (e.g., the G color component). The remaining image pixels are then segmented into two cluster types, background and fresh leaves, using a variable threshold K-means clustering algorithm, and the proportion P of fresh leaf pixels is then calculated. The specific steps of the variable threshold K-means clustering algorithm are as follows: (1) For the remaining image pixel samples, the initial value of the color space distance threshold is set, and the maximum number of iterations of the K-means clustering algorithm is set. Then, two pixel samples are randomly selected, and the pixel with the smallest value of the base color component is set as the initial background cluster center, and the other pixel is set as the initial leaf cluster center. (2) Using the K-means clustering algorithm, the remaining image pixels are divided into two cluster types: background and fresh leaves. Then, it is determined whether the color space Euclidean distance between the background cluster center and the fresh leaf cluster center is greater than the color space distance threshold. If the color space Euclidean distance between the background cluster center and the fresh leaf cluster center is greater than the color space distance threshold, the division is completed; otherwise, step (3) is performed. (3) Determine whether the number of iterations is greater than the set maximum number of iterations. If it is greater than the maximum number of iterations, decrease the color space distance threshold and return to step (2). Otherwise, set the pixel with the maximum value of the splitting basis color component in the cluster type with the highest degree of variance as the new center of the green leaf cluster, and set the pixel with the minimum value of the splitting basis color component in the cluster type with the highest degree of variance as the new center of the background cluster, and return to step (2). When pixel segmentation is performed on the RGB information image of the tea leaf surface in each sampling area in the next collection target area, the current color space distance threshold is used as the initial value of the color space distance threshold of the K-means clustering algorithm to improve segmentation efficiency.

[0046] In step six, a depth information image of the tea leaf surface in each sampling area of ​​each sampling area is obtained based on the surface depth information of each sampling area, and the average variance value of the vertical depth of the tea leaves (the vertical projection of the distance from the RGB-D camera) that characterizes the growth density of the fresh tea leaves is extracted from the depth information image of the tea leaf surface in each sampling area. When the growth density of the fresh tea leaves is high, the leaf pitch is small, and conversely, the leaf pitch is large. Because the leaf pitch is large, the laser beam of the RGB-D camera is likely to pass through the fresh leaf layer and reach the bottom, resulting in a sparse distribution of depth information. Conversely, it is likely to be reflected by the fresh leaves on the surface, resulting in a dense distribution of depth information. Therefore, we propose to divide the depth information image of the tea leaf surface in each sampling area into multiple areas, and then represent the growth density of the fresh tea leaves using the average value of the vertical depth variance of the point cloud in each area. This method has strong real-time capabilities and low computational complexity. The specific steps are as follows: (1) Based on the surface depth information of the sampling area, obtain the surface depth information image of the tea leaves in each sampling area within the sampling area, divide the surface depth information image of the tea leaves in each sampling area into m areas based on equal area, and calculate the vertical (Z-axis direction) depth variance of the point cloud (all pixel points within the area) within each area, as follows:

number

number

[0047] In step seven, a BP neural network model is constructed to optimize the optimal cutter-simulated harvesting depth criterion S for each sampling area, and then trained and validated. The specific steps are as follows: (1) Select multiple different tea leaf surfaces, and for each tea leaf surface, obtain the average vertical depth variance Var, the fresh leaf pixel proportion P, and the cutter pre-collection cutting depth criterion s for each sampling area within each target area. Then, artificially determine the optimal cutter-imitated cutting depth criterion S for each sampling area, thereby establishing a data set including the average vertical depth variance Var, the fresh leaf pixel proportion P, the cutter pre-collection cutting depth criterion s, and the optimal cutter-imitated cutting depth criterion S for each sampling area. When selecting tea leaf surfaces, it is necessary to cover the surfaces of four types of tea leaves: dense long-leaf type, dense short-leaf type, sparse long-leaf type, and sparse short-leaf type. Based on the fresh leaf pixel proportion P and the average vertical depth variance Var, the tea leaf surfaces are divided into dense long-leaf type, dense short-leaf type, sparse long-leaf type, and sparse short-leaf type, and the Var limits for dense and sparse, and the P limits for long and short leaves are preset. (2) A BP neural network model was constructed, taking the average vertical depth variance Var, the proportion of live leaf pixels P, and the cutter pre-harvesting cutting depth criterion s for each sampling area as inputs, and the optimal cutter-imitating harvesting cutting depth criterion S for each sampling area as output. The dataset was divided into a test set and a validation set in a ratio of 7:3, and the BP neural network model was trained and validated.

[0048] In step eight, during the actual imitation harvesting process, the cutter preliminary harvesting cutting depth standard s for each sampling area in each target harvesting area is fitted in real time, and the fresh leaf pixel proportion P on the tea leaf surface and the vertical depth variance average Var for each sampling area in each target harvesting area are obtained, and input into the trained BP neural network model to output the optimal cutter imitation harvesting cutting depth standard S for each sampling area in the target harvesting area in real time. Furthermore, the encoder built into the lifting motor of the lifting device 2 obtains in real time the position information of the inner ends of the cutters in cutter assembly one 311 and cutter assembly two 313, and the encoders built into attitude adjustment cylinder one 309 and attitude adjustment cylinder two 315 obtain the position information of the outer ends of the cutters in cutter assembly one 311 and cutter assembly two 313, respectively. Note that the position information of the upper cutter 322 and the lower cutter 323 refers to the position information of the corresponding ends of the mating surfaces of the upper cutter 322 and the lower cutter 323, and the position information of the upper cutter 322 and the lower cutter 323 is considered to be identical.

[0049] In step 9, based on the position information of each cutter and the optimal cutter imitation harvesting cutting depth standard S of the corresponding sampling area output from the BP neural network model, a linear anti-self-interference control method is adopted to obtain the actual control amount of the imitation harvesting cutting depth at both ends of each cutter. During the actual imitation harvesting process, due to the influence of the undulations of the furrow topography, the arc surface of each cutter does not overlap with the surface of the tea leaves, and there will be an error between the cutting depth at both ends of each cutter and the corresponding optimal cutter imitation harvesting cutting depth standard S. The present invention controls the overall height of each cutter using lifting device 2, and also controls the height of the inner ends of cutter assembly 1 311 and cutter assembly 2 313, and adjusts the height of the outer ends of cutter assembly 1 311 and cutter assembly 2 313 using attitude adjustment cylinder 2 315 and attitude adjustment cylinder 1 309, respectively. Assuming that the cutting depth at a certain end of each cutter is H, and the corresponding control target is HS=0, i.e., the cutter end is aligned with the surface of the tea leaves, and the objective of harvesting the tea leaves along the corresponding optimal cutter-simulated harvesting cutting depth reference position S is achieved. However, interference effects such as model uncertainty, sensor measurement noise, and uneven ridge undulations are a major challenge to high-precision, fast-response cutter control. Therefore, the present invention adopts a linear anti-self-interference control method to provide decision information for each cutter-simulated harvesting cutting depth control, and introduces known disturbance measurements to improve the estimation accuracy and convergence speed of the extended state observer for rapidly changing disturbances. As shown in Figure 12, the specific steps for obtaining the actual control variable of the cutter-simulated harvesting cutting depth using the linear anti-self-interference control method are as follows: (1) The difference value h and the rate of change v between the two end positions of each cutter and the optimal cutter-imitating cutting depth criterion of the corresponding sampling area output from the BP neural network model are used as state variables, and the dynamic control equation for the controlled object is established:

number

number

number

number

number

[0050] In step 10, the lifting device 2, the attitude adjusting cylinder 2 315 and the attitude adjusting cylinder 1 309 automatically adjust the cutting depth at both ends of each cutter based on the actual control amount u corresponding to real time, so that each cutter can cut as close as possible to the base of the tea leaves, thereby realizing automatic imitation tea picking on the surface of the picking area.

[0051] In step eleven, when performing automated mass tea picking, drive member one 319 and drive member two 317 drive corresponding upper cutter 322 and lower cutter 323 to move back and forth relative to each other, cutting the bases of the fresh tea leaves and realizing the picking of the fresh tea leaves. At the same time, they control the operation of blower one 305 and blower two 306, and the air blown out from blower one 305 and blower two 306 is blown over the cutters of cutter assembly one 311 and cutter assembly two 313 via duct two 314 and duct one 310, respectively, and the cut tea leaves are blown into each fixed frame and further into a collecting bag to collect the picked fresh tea leaves.

Claims

1. A method for automatically harvesting tea leaves based on point cloud and color information, specifically, a lifting device and a cross member mounted on a body frame of a moving device, the lifting device driving the cross member to move up and down; a cutter assembly 1 and a cutter assembly 2 symmetrically mounted on the cross member, each of the cutter assembly 1 and the cutter assembly 2 including a fixed frame, an upper cutter, and a lower cutter; the upper cutter and the lower cutter in the cutter assembly 1 or the cutter assembly 2 are arranged on the fixed frame parallel to each other and spaced apart from each other, and are driven to reciprocate by a driving member 1 or a driving member 2 mounted on the fixed frame; one end of the cutter assembly 1 or the cutter assembly 2 positioned inside the fixed frame is hingedly connected to the center of the cross member, and one end of the cutter assembly 1 or the cutter assembly 2 positioned outside is hingedly connected to the push rods of an attitude adjustment cylinder 1 and an attitude adjustment cylinder 2, respectively; Step 1: The cylinder bodies of the attitude adjustment cylinders 1 and 2 are connected to both ends of the cross member by hinges; ducts 1 and 2 are attached to the two fixed frames above the two upper cutters, respectively, and the ducts 1 and 2 are connected to blowers 2 and 1 attached to the cross member, respectively; collection bags are attached to the rear openings on both fixed frames; a tail plate is connected to a location on the cross member behind cutter assembly 1 and cutter assembly 2 by hinges, and the tail plate is driven to rotate by two telescopic cylinders attached to the cross member; in an initial state, piston rods of the two telescopic cylinders are retracted and the tail plate is in a stored state; and an RGB-D camera and an accelerometer are both attached to the body frame by sensor holders; Step 2: the controller controls the two telescopic cylinders to synchronously drive the tail plate to rotate rearward to the deployed state, the lifting device drives the cross member to lift and lower, and further lifts and lowers the cutter assembly 1, the cutter assembly 2, the attitude adjustment cylinder 1, the attitude adjustment cylinder 2, the duct 1, the duct 2 and the tail plate, so that the cutters of the cutter assembly 1 and the cutter assembly 2 lift and lower to a preset initial height position and are positioned above the surface of the tea leaves, and then the moving device drives the body frame to move through the tea furrows and start the picking operation of automatically picking a large amount of tea leaves; During the collection process, the surface of the tea leaves is scanned with an RGB-D camera to obtain depth images. Step three and Step four: extract surface depth information, which is the position information of the surface of the tea leaves in the target area, from the depth image, and remove outliers using a radius filtering method to obtain a surface depth information image of the target area; obtain the vertical acceleration of the RGB-D camera using an accelerometer; use a Kalman filter algorithm to obtain the surface depth information of the target area as measurement data, and combine the vertical acceleration state of the RGB-D camera to recursively correct the surface depth information image of the target area; then, select a preset width range area located at the center of the surface of the target area and preset width range areas located at both ends as three sampling areas; and use a RANSAC algorithm to fit a cutter pre-collecting cutting depth standard s for each sampling area. Step 5: extracting RGB information of the surface of each sampling area in the collection target area from the depth image, obtaining RGB information images of the tea leaf surface of each sampling area in the collection target area, directly classifying pixels located in the upper predetermined range of depth information in the RGB information images of the tea leaf surface of each sampling area in the collection target area as raw leaves, and directly classifying pixels located in the lower predetermined range as background, and selecting color components with a large difference between the raw leaves and the background as the basis for division for the remaining image pixels, and dividing them into two cluster types, background and raw leaves, using a variable threshold K-means clustering algorithm, and then calculating the proportion P of raw leaf pixels; Step 6: obtaining a depth information image of the tea leaf surface in each sampling area of ​​each sampling area based on the surface depth information of each sampling area, and extracting a depth variance average value Var of the raw tea leaves in the vertical direction from the depth information image of the tea leaf surface in each sampling area; Step seven: construct a BP neural network model to optimize the cutter sampling cutting depth criterion S, which is the optimal cutter cutting position information for each sampling area, and then train and validate it; Step 8: in the harvesting process, the cutter preliminary harvesting cutting depth criterion s of each sampling area in each harvesting target area is fitted in real time, the raw leaf pixel ratio P of the tea leaf surface and the vertical depth variance average value Var of each sampling area in each harvesting target area are obtained, and input into the trained BP neural network model, and the optimal cutter harvesting cutting depth criterion S of each sampling area in each harvesting target area is output in real time, and the encoder built in the lifting motor of the lifting device is used to obtain in real time the position information of each cutter in cutter assembly 1 and cutter assembly 2 positioned at one end inside, and the encoder built in the attitude adjustment cylinder 1 and attitude adjustment cylinder 2 is used to obtain the position information of each cutter in cutter assembly 1 and cutter assembly 2 positioned at one end outside, respectively; Step 9: According to each cutter position information and the optimal cutter cutting depth standard S of the corresponding sampling area output from the BP neural network model, a linear anti-self-interference control method is adopted to obtain the actual control amount of the cutting depth at both ends of each cutter; Step 10: The lifting device, the first attitude adjusting cylinder, and the second attitude adjusting cylinder automatically adjust the cutting depth of each cutter according to the actual control amount u corresponding to the real time, so as to automatically pick tea leaves from the picking target area, thereby realizing automatic tea picking. and step 11: when automatically harvesting tea leaves, driving member 1 and driving member 2 drive the corresponding upper cutter and lower cutter to move back and forth relative to each other, cutting the roots of the raw tea leaves, and at the same time controlling the operation of blower 1 and blower 2, so that the air blown out from blower 1 and blower 2 is blown above each cutter of cutter assembly 1 and cutter assembly 2 through duct 2 and duct 1, respectively, so that the cut tea leaves are blown into each fixed frame and further into a collection bag, and the picked tea leaves are collected.

2. In step 5, the remaining image pixels are divided into two cluster types, background and live leaves, using a variable threshold K-means clustering algorithm. The specific steps are as follows: 、 For the remaining image pixel samples, an initial value of the color space distance threshold is set, and the maximum number of iterations of the K-means clustering algorithm is set. Then, two pixel samples are randomly selected, and the pixel with the smallest value of the splitting basis color component is set as the initial background cluster center, and the other pixel is set as the initial leaf cluster center. Step (1); Step (2): Divide the remaining image pixels into two cluster types, namely, background and green leaf, by using a K-means clustering algorithm, and then determine whether the color space Euclidean distance between the background cluster center and the green leaf cluster center is greater than the color space distance threshold. If the color space Euclidean distance between the background cluster center and the green leaf cluster center is greater than the color space distance threshold, the division is completed; otherwise, perform step (3); 2. The method of claim 1, further comprising: determining whether the number of iterations is greater than the set maximum number of iterations; if the number of iterations is greater than the set maximum number of iterations, decreasing the color space distance threshold and returning to step (2); if not, determining a pixel having the maximum value of the splitting basis color component in the cluster type with the greatest degree of dispersion as a new center of the green leaf cluster, determining a pixel having the minimum value of the splitting basis color component in the cluster type with the greatest degree of dispersion as a new center of the background cluster, and returning to step (2).

3. The specific steps of step 6 are: According to the surface depth information of the sampling area, a surface depth information image of the tea leaves in each sampling area is obtained, and the surface depth information image of the tea leaves in each sampling area is divided into m areas based on equal areas, and the vertical depth variance of the point cloud in each area is calculated, as follows: [Equation 1] Calculate the average depth variance of the vertical distances of the point clouds of m regions in each sampling region, as follows: [Equation 2] The collection method according to claim 1, comprising:

4. The specific steps of step seven are: Select a plurality of different tea leaf surfaces, and for each tea leaf surface, obtain the vertical depth variance average value Var, the raw leaf pixel ratio P, and the cutter pre-collection cutting depth criterion s of each sampling area in each target collection area, and artificially determine the optimal cutter collection cutting depth criterion S of each sampling area, thereby obtaining the vertical depth variance average value Var, the raw leaf pixel ratio P, the cutter pre-collection cutting depth criterion s, and the optimal cutter collection cutting depth criterion S of each sampling area. Step (1) establishing a data set including a degree criterion S; The automatic tea leaf harvesting method based on point cloud and color information as described in claim 1 further includes step (2) of constructing a BP neural network model, taking the average vertical depth variance Var of each sampling area, the proportion of live leaf pixels P, and the cutter pre-collection cutting depth criterion s as inputs, and the optimal cutter collection cutting depth criterion S of each sampling area as output, dividing the dataset into a test set and a validation set, and training and validating the BP neural network model.

5. The specific steps of obtaining the actual control amount of the cutter picking cutting depth using the linear anti-self-interference control method in step 9 are as follows: The difference value h and the difference value change rate v between the two end positions of each cutter and the optimal cutter sampling cutting depth criterion of the corresponding sampling area output from the BP neural network model are used as state variables, and a dynamic control equation for the controlled object is established: [Equation 3] Since the dynamic model of the controlled plant is a second-order model, the state space of the third-order extended state observer is constructed as follows: [Equation 4] A linear self-interference prevention system controller is designed, which is as follows: First, establish the following equation: [Equation 5] Then, design a preliminary output value of the controller control variable: [Equation 6] Finally, the actual control quantity of the linear anti-self-interference system controller is designed: [Equation 7] 2. The method for automatically picking tea leaves based on point cloud and color information according to claim 1.

6. A collection method according to any one of claims 1 to 5, characterized in that the RGB-D camera is replaced with a module consisting of a three-dimensional radar and an RGB camera, the three-dimensional radar scans the surface of the tea leaves to obtain depth information, the RGB camera scans the surface of the tea leaves to obtain RGB information, and a depth image is obtained using the depth information and RGB information.

7. An apparatus for automatically harvesting tea leaves based on point cloud and color information according to any one of claims 1 to 5, comprising: a moving device, a lifting device, a collecting device, and a sensor assembly; The collecting device includes a tail plate, a cross member, a blower 1, a blower 2, a position adjustment cylinder 1, a duct 1, a cutter assembly 1, a cutter assembly 2, a duct 2, and a position adjustment cylinder 2; the moving device drives the body frame to move; the cross member and the body frame form a vertical sliding pair, and the lifting device installed on the body frame drives the lifting; the cutter assembly 1 includes a fixed frame, an upper cutter, and a lower cutter, both of which are open at positions located in front and rear of the fixed frame; the upper cutter and the lower cutter are both arc-shaped cutters; the upper cutter and the lower cutter are arranged parallel and stacked one above the other, both of which form a horizontal sliding pair with the position located in the front opening of the fixed frame, and both of which are driven to reciprocate by driving member 1; the structure of the cutter assembly 2 is completely identical to that of the cutter assembly 1, and the upper cutter and the lower cutter of the cutter assembly 2 are both driven to reciprocate by driving member 2; the cutter assembly 1 and the cutter assembly 2 are symmetrical. and the ends of the first and second cutter assemblies located inside the fixed frames are both connected to the middle of the cross member by hinges, the first and second attitude adjustment cylinders are located above the first and second cutter assemblies, respectively, and the cylinder bodies of the first and second attitude adjustment cylinders are connected to both ends of the cross member by hinges, and the push rods of the first and second attitude adjustment cylinders are connected to the second and third cutter assemblies, respectively. the first and second blowers are hinged to one end of the first cutter assembly fixed frame located outside the first cutter assembly, the first blower and the second blower are both fixed to the cross member, the first duct and the second duct are fixed to the first cutter assembly fixed frame and are located above the upper cutters of the first cutter assembly and the second cutter assembly, respectively, and the intake ports of the first duct and the second duct communicate with the exhaust ports of the second blower and the first blower via the second and second elastic hoses, respectively;The device is characterized in that the duct 1 and the duct 2 each have a plurality of exhaust holes spaced apart along the axial direction at positions close to the corresponding upper cutter, the tail plate is provided behind the cutter assembly 1 and the cutter assembly 2, and both ends of the tail plate are hingedly connected to both ends of the cross member and are rotated synchronously by two telescopic cylinders, and the sensor assembly includes an RGB-D camera, an accelerometer, and a sensor holder, and the RGB-D camera and the accelerometer are both fixed to the body frame by the sensor holder.

Images