Intelligent pepper picking system and picking method

Abstract

The invention discloses an intelligent hot pepper picking system and method, and the system comprises a visual positioning device which is used for recognizing the coordinate position of hot peppers, a hot pepper picking device which is used for picking the hot peppers according to the position coordinates of the hot peppers, and a control device. The pepper picking device comprises a base, a mechanical arm arranged on the base, a picker arranged at the tail end of the mechanical arm and a picking driving mechanism used for driving the mechanical arm to move in the X-axis direction and the Y-axis direction. The picking device comprises a support, a fixed cutter arranged on the support, a movable cutter and a shearing driving mechanism used for driving the movable cutter to swing so as to be matched with the fixed cutter to complete shearing and clamping actions. The fixed cutter is fixed on the bracket; the movable cutter is rotationally connected to the support and located above the fixed cutter. The visual positioning device comprises a binocular camera arranged on a base of the mechanical arm. The intelligent hot pepper picking system can achieve accurate picking of hot peppers.

Description

Technical Field

[0001] The invention belongs to the field of pepper picking, and in particular relates to an intelligent pepper picking device and a picking method. Background Art

[0002] In the process of pepper cultivation, the picking stage is the most time-consuming and labor-intensive stage. The quality of picking directly affects the yield, storage, and processing of peppers. Therefore, it is crucial to reduce picking costs, save human resources, and improve picking efficiency.

[0003] Among them, the invention patent application with application publication number CN117918119A discloses a "pepper picking machine", which includes a support frame, a walking device, a mechanical arm device and a driving device. The walking device is arranged on the part of the support frame; the mechanical arm device includes a mechanical arm and a clamping claw structure arranged at the end of the mechanical arm, and the mechanical arm includes a rigid component and a flexible component, the rigid component is rotatably arranged on the support frame, and the flexible component is connected between the rigid component and the clamping claw structure; the driving device is transmission-connected to the clamping claw structure and the flexible component to drive the clamping claw structure to open or close, and drive the flexible component to bend. The mechanical arm of the above-mentioned pepper picking machine includes a rigid component and a flexible component, so that the flexible component can bend under the driving action of the driving device, and at the same time drive the clamping claw structure to open or close, so as to pick peppers as needed. Avoid the phenomenon that the degree of freedom is too large due to all flexible components, which causes the mechanical arm to twist into a ball when the driving mechanical arm rotates.

[0004] The invention patent application with application publication number CN116472868A discloses "a wheeled automatic pepper picking robot", which includes a mobile chassis, a body and other components. The body is fixed on the top surface of the mobile chassis, the grabbing device and the collection box are installed on the top of the body, the laser radar and the display are fixed on the front end of the body, the depth machine is installed on the grabbing device, the lower end of the mobile chassis is fixed with a steering wheel and a driving wheel, the steering wheel is connected to a steering device, the driving wheel is connected to a driving device, and the grabbing device, the display, the laser radar, the depth camera, the steering device, and the driving device are all connected to the control system. The control system can realize the coordinated work of the mobile chassis and the grabbing device; thereby realizing autonomous grabbing, precise picking, and automatic collection of peppers in a complex environment without affecting the quality of the peppers.

[0005] However, the above-mentioned pepper picking machine has a small picking accuracy and picking range due to its structural limitations; and the above-mentioned wheeled pepper automatic picking robot has a small working space range because it is not equipped with guide rails. Summary of the invention

[0006] In order to overcome the deficiencies in the prior art, the present invention provides an intelligent pepper picking system, which can achieve accurate picking of peppers.

[0007] The second object of the present invention is to provide an intelligent pepper picking method for an intelligent pepper picking system.

[0008] The technical solution of the present invention to solve the above technical problems is:

[0009] An intelligent pepper picking system comprises a visual positioning device for identifying the coordinate position of peppers, a pepper picking device and a control device for picking the peppers according to the position coordinates of the peppers, wherein the pepper picking device comprises a base, a mechanical arm arranged on the base, a picker arranged at the end of the mechanical arm, and a picking drive mechanism for driving the mechanical arm to move along the X-axis and Y-axis directions; the picker comprises a bracket, a fixed knife arranged on the bracket, a movable knife, and a shearing drive mechanism for driving the movable knife to swing so as to cooperate with the fixed knife to complete a shearing action, wherein the fixed knife is fixed on the bracket; the movable knife is rotatably connected to the bracket and is located above the fixed knife; the visual positioning device comprises a binocular camera arranged on the base of the mechanical arm.

[0010] Preferably, the binocular camera is used to obtain the first relative position coordinates of the pepper stalk relative to the binocular camera, and transmit the first relative position coordinates to the control device. The control device calculates the third relative position coordinates of the pepper stalk relative to the base of the robotic arm based on the first relative position coordinates and the second relative position coordinates of the binocular camera and the base of the robotic arm.

[0011] Preferably, the robotic arm is a six-degree-of-freedom robotic arm.

[0012] Preferably, the X-axis driving mechanism and the Y-axis driving mechanism both adopt a driving method combining a motor and a screw transmission mechanism.

[0013] Preferably, the X-axis driving mechanism and the Y-axis driving mechanism both further include a guiding mechanism, and the guiding mechanism is in the form of a combination of a slide rail and a slider, or in the form of a combination of a guide rod and a guide sleeve.

[0014] Preferably, the shearing drive mechanism comprises an electric push rod; the electric push rod is connected to the movable knife via a connecting rod, one end of the connecting rod is rotatably connected to the telescopic rod of the electric push rod, and the other end is rotatably connected to the movable knife.

[0015] An intelligent pepper picking method comprises the following steps:

[0016] Step S1: Perform hand-eye calibration to obtain the rotation transformation matrix between the image coordinate system of the binocular camera and the robotic arm coordinate system;

[0017] Step S2: The binocular camera identifies the pepper, processes the collected pepper image, and obtains the image coordinates of the pepper stalk in the pepper image;

[0018] Step S3: Calculate the three-dimensional coordinates of the pepper stalk in the robot arm coordinate system according to the rotation transformation matrix obtained in step S1 and the image coordinates of the pepper stalk obtained in step S2;

[0019] Step S4: According to the three-dimensional coordinates of the pepper stalk obtained in step S3, the robotic arm drives the picker to pick the peppers;

[0020] Step S5: Repeat steps S2 to S4 until the binocular camera no longer recognizes the peppers, and the picking work is completed.

[0021] Preferably, in step S2, the process of processing the pepper image includes:

[0022] Step 201: Train the YOLOv10 model using the pepper image collected by the binocular camera, and construct a YOLOv10 training model in ONNX format;

[0023] Step 202: preprocessing the collected pepper image, sending the preprocessed pepper image to the YOLOv10 training model to perform forward reasoning, and parsing its output from the inferred vector to obtain the best reasoning result;

[0024] Step 203: Based on the best inference result, the GrabCut algorithm is executed on the pepper image to generate a valid foreground mask; the partial differentials of the connected areas of the foreground mask are continuously solved from bottom to top, the maximum partial differential value is found and its direction is determined to obtain the approximate two-dimensional coordinates of the pepper stalk, and the approximate two-dimensional coordinates are offset corrected to determine the final two-dimensional coordinates of the pepper stalk;

[0025] Step 204: loading the archived intrinsic parameter matrix and extrinsic parameter matrix of the binocular camera; after inputting the pepper images of the left eye and the right eye of the binocular camera, performing monocular distortion correction on the pepper images through the intrinsic parameter matrix and the extrinsic parameter matrix, and then performing binocular epipolar stereo correction, so that the same object in the pepper images of the left eye and the right eye is in the same line; calculating the disparity map of the corrected pepper image relative to the left eye of the binocular camera in the current binocular camera field of view through the global stereo matching algorithm; filling the holes in the disparity map by adopting the multi-level mean filtering method; after the filling is completed, the depth map matrix of the current pepper image relative to the left eye of the binocular camera is obtained through the relationship between disparity and depth;

[0026] Step 205: Project the obtained two-dimensional coordinates of the pepper stalk into the depth map matrix to obtain the three-dimensional coordinates of the pepper stalk in the depth map, and calculate the three-dimensional coordinates of the pepper stalk in the pepper fruit in the robot arm coordinate system through the rotation transformation matrix obtained in step S1.

[0027] Preferably, in step S201, more than 1200 color pepper images including various weather conditions, lighting conditions and target posture conditions are used to train the YOLOv10 model to obtain a YOLOv10 training model in ONNX format.

[0028] Preferably, in step S202, the preprocessing process is: scaling the pepper image captured by the binocular camera using an image reduction algorithm; and then normalizing the scaled pepper image.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] 1. The intelligent pepper picking system of the present invention can quickly identify the position coordinates of the pepper (stalk), thereby realizing accurate picking of the pepper, thereby reducing the breakage rate of the pepper picking.

[0031] 2. The intelligent pepper picking system of the present invention can effectively improve the efficiency of pepper picking, reduce the cost of pepper picking, and has a good market application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 and Figure 2 Schematic diagram of the structure of the intelligent pepper picking system of the present invention from two different perspectives.

[0033] Figure 3 This is a schematic diagram of the structure of the picker.

[0034] Figure 4 The figure is a flow chart of the intelligent pepper picking method of the present invention.

[0035] Figure 5 This is the logic block diagram of the image processing program.

[0036] Figure 6 The logical block diagram for performing forward reasoning.

[0037] Figure 7 The logic block diagram for performing foreground and background binary separation.

[0038] Figure 8 For the pepper image.

[0039] Fig. 9 is the foreground mask.

[0040] Fig.10The logic block diagram for performing binocular stereo depth matching.

[0041] Fig.11 is the disparity map. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0043] See also Figure 1-Figure 3 The intelligent pepper picking system of the present invention comprises a mobile body 1, a visual positioning device arranged on the mobile body 1 for identifying the coordinate position of peppers, a pepper picking device and a control device for picking the peppers according to the position coordinates of the peppers, wherein the pepper picking device comprises a base, a mechanical arm 3 arranged on the base, a picker 4 arranged at the end of the mechanical arm 3, and a picking drive mechanism 2 for driving the mechanical arm 3 to move along the X-axis and Y-axis directions; the picker 4 comprises a bracket 401, a fixed knife 406 arranged on the bracket 401, a movable knife 405, and a device for driving the movable knife 405 to swing so as to cooperate with the fixed knife 406 to complete the picking. A shearing drive mechanism for cutting action, wherein the fixed knife 406 is fixed on the bracket 401; the movable knife 405 is rotatably connected to the bracket 401 and is located above the fixed knife 406; the visual positioning device includes a binocular camera arranged on the base of the robotic arm 3; the binocular camera is used to obtain the first relative position coordinates of the pepper stalk relative to the binocular camera, and transmit the first relative position coordinates to the control device, and the control device calculates the third relative position coordinates of the pepper stalk relative to the base of the robotic arm 3 according to the first relative position coordinates and the second relative position coordinates of the binocular camera and the base of the robotic arm 3.

[0044] In this embodiment, the binocular camera is installed in front of the base of the robot arm 3 for identifying peppers.

[0045] See also Figure 1-Figure 3 The shearing drive mechanism includes an electric push rod 402; the electric push rod 402 and the movable knife 405 are connected via a connecting rod 404, one end of the connecting rod 404 is rotatably connected to the telescopic rod of the electric push rod 402, and the other end is rotatably connected to the movable knife 405; the electric push rod 402 drives the telescopic rod 403 to extend or retract, thereby driving the movable knife 405 to rotate around its rotation fulcrum to complete the shearing action.

[0046] In this embodiment, elastic clamping blocks 407 are provided at the lower ends of the movable knife 405 and the fixed knife 406. When the movable knife 405 and the fixed knife 406 cooperate to cut the pepper stalk, the elastic clamping blocks 407 at the lower ends of the movable knife 405 and the fixed knife 406 clamp the pepper stalk of the pepper, thereby realizing integrated cutting and clamping.

[0047] See also Figure 1-Figure 3 , the robot arm 3 is a six-degree-of-freedom robot arm; the X-axis drive mechanism and the Y-axis drive mechanism both adopt a driving method combining a motor and a screw transmission mechanism; in addition, in order to ensure the motion accuracy, the X-axis drive mechanism and the Y-axis drive mechanism also include a guide mechanism, and the guide mechanism adopts a combination of a slide rail and a slider, or a combination of a guide rod and a guide sleeve.

[0048] See also Figure 4 The intelligent pepper picking method of the present invention comprises the following steps:

[0049] Step S1: Perform hand-eye calibration to obtain the rotation transformation matrix between the image coordinate system of the binocular camera and the robotic arm coordinate system;

[0050] Step S2: The binocular camera identifies the pepper, processes the collected pepper image, and obtains the image coordinates of the pepper stalk in the pepper image;

[0051] Step S3: Calculate the three-dimensional coordinates of the pepper stalk in the robot arm coordinate system according to the rotation transformation matrix obtained in step S1 and the image coordinates of the pepper stalk obtained in step S2;

[0052] Step S4: According to the three-dimensional coordinates of the pepper stalk obtained in step S3, the robotic arm drives the picker to pick the peppers;

[0053] Step S5: Repeat steps S2 to S4 until the binocular camera no longer recognizes the peppers, and the picking work is completed.

[0054] The program used in the intelligent pepper picking method of the present invention is written in a mixture of Python and C++, uses OpenCV4.9.0 as the back-end computer algorithm library, uses the ONNX target recognition and detection model from YOLOv10 as the reasoning basis, uses ROS-noetic as the communication framework, and uses StereolabsZEDSDK as the camera driver and front-end library.

[0055] In step S1, before starting work, the hand-eye calibration of the intelligent pepper picking system of the present invention is required; in this embodiment, the calibration method of "eye outside hand" is adopted: assuming that the transformation matrix of the camera coordinate system relative to the robot arm coordinate system is Use the camera to take n images, for each image there is:

[0056]

[0057] Where: is an unknown quantity; It can be determined by the posture parameters of the robot itself; Can be determined by camera calibration;

[0058] For each image, let:

[0059]

[0060] Solve the matrix equation:

[0061] A i X=XB i , i∈[1,n-1];

[0062] The rotation transformation matrix can be calculated. Through the rotation transformation matrix obtained above, the three-dimensional coordinates of the pepper in the robotic arm coordinate system, that is, the position where the picker at the end of the robotic arm should reach, can be calculated.

[0063] See also Figure 5 In step S2, the image processing flow mainly consists of six parts, namely program initialization, forward reasoning, foreground and background binary separation, binocular stereo matching, point cloud reading and message sending / visualization. These six main parts are connected block by block in sequence, and the selection structure is interpolated to judge and filter level by level. The internal codes are implemented in sequence or in parallel.

[0064] In the "Program Initialization" process, first load the necessary dependent modules, such as OpenCV, ONNXRuntime, and ROS; then load the deep learning model trained by YOLOv10 into the memory to facilitate the subsequent forward reasoning; then start the ROS node and prepare to publish the ROS topic message; finally set the startup parameters of the binocular camera and start the binocular camera. Each of the above steps has a self-checking and error handling process, using Python object references combined with try...except code blocks for checking. If an error occurs in one step, it is determined that the program is not initialized correctly and then exits the program immediately; if it is initialized correctly, it will officially enter the core loop process of the program.

[0065] The present invention uses a YOLOv10 training model based on the PyTorch deep learning framework, uses more than 1,200 pepper images, including color images of various weather conditions, lighting conditions, and target postures, to train the model, uses NVIDIA GPU to accelerate the training, and finally obtains a YOLOv10 training model file in ONNX format, which will be used in the subsequent reasoning process.

[0066] See also Figure 6 The first few steps of the "Perform forward inference" process will be handled by the Ultralytics module. To ensure stability and compatibility, the program will scale the original 1280×720 color pepper image from the stereo camera to a 720×720 pepper image using a local mean image reduction algorithm and normalize it. Let the size transformation matrix be C. After that, the pepper image is sent to the ONNX model trained from YOLOv10 for forward inference, and its output is parsed from the inferred vector; since the pepper image has been scaled and normalized before, it is necessary to reverse the mapping of these output data on the source pepper image, that is, to obtain the original inference vector and C. -1 Finally, the Ultralytics module obtains a format (c x , c y , w, h), each item in the tuple represents the center coordinates of the detection frame of the pepper target on the binocular camera screen (c x , c y ) and width and height (w, h). Since there may be multiple available data on a pepper image, it is necessary to verify the availability of these data, that is, to verify the rationality of the coordinate size of each detection box, and its center coordinate (c x , c y ) must not exceed the width and height of the pepper image, and (w, h) must not extend beyond the pepper image area. Assume that the size of the pepper image is (p x , p y ), then it should satisfy:

[0067] 0≤c x ≤p x And 0≤c y ≤p y ;

[0068] 0≤c x +w≤p x And 0≤c y +w≤p y ;

[0069] Next, we manually perform non-maximum suppression and optimal target screening on the data set (accuracy), filter out the detection frames at the edge of the region where the detection frames overlap, and retain the detection frames with the highest confidence in the central region. The center coordinates and width and height ROI of the detection frame in the central region are defined as The default confidence level thres = 0.65.

[0070] After the above post-processing, the best inference result can be obtained, and the best inference result and its related values ​​are retained. bestBbox is stored in the computer memory for subsequent binary separation of foreground and background.

[0071] See also Figure 7 In the process of "performing foreground and background binary separation", part of it is necessary to use OpenCV4.9.0 as the image processing algorithm library, and part of it is necessary to use the self-developed algorithm for accurate two-dimensional positioning of peppers (stalks), specifically:

[0072] First, initialize the parameters required by the GrabCut algorithm, such as the number of iterations k and the offset correction parameter ω, and input the image of the binocular camera. Then input the detection frame of the target object to be extracted from the foreground, which is obtained from the previous step of "performing forward reasoning", that is, The effect is as follows Figure 8 shown.

[0073] The GrabCut algorithm is iterated k times to generate a valid foreground image. The GrabCut algorithm is a composite iterative algorithm, which iterates and calls the following operations: Gaussian Mixture Model, foreground estimation and background color distribution; constructs a Markov Random Field with pixel labels, constructs the foreground and background, and applies Graph Cut Optimization to achieve the final segmentation effect.

[0074] After obtaining the foreground image, the image matrix can be processed in the next step, that is, obtaining a valid foreground mask, and processing the foreground to pure white in the grayscale image, and the background to pure black in the grayscale image. The effect is as follows: Fig. 9 shown.

[0075] Then, the distance L between the left and right endpoints (e1, e2) in the same row in the white part of the mask matrix is ​​used as the independent variable, starting from the bottom of the connected area bottom h starts at the top top h, solve the distance length d on the row segment L About line segment d h The maximum absolute value of the differential of :

[0076]

[0077] The row where the absolute value of the differential is the largest, the row is at the top of the connected mask, and the distance from the top is decreasing. At this time, i should satisfy:

[0078]

[0079] The approximate coordinates of the pepper stalk can be obtained by taking the perpendicular bisector of the left and right endpoints of the mask in this row, and the pose in the image is compensated and corrected to determine the final two-dimensional coordinates of the pepper stalk on the source image (t x , t y ), which satisfies:

[0080]

[0081] The coordinate value and the binary image of the foreground mask are retained for use in other subsequent processes.

[0082] See also Fig.10 The "Perform Stereo Depth Matching" process mainly relies on the Global Stereo Matching (SGBM) algorithm in OpenCV4.9.0 and the previously archived stereo camera calibration parameters, specifically:

[0083] First, load the archived stereo camera's intrinsic parameter matrix M CamL 、M CamR 、V DistL 、V DiskR And the external parameter matrices R, T, E, F, Q. The reading here is not directly read and loaded from the file, but should be pre-loaded when the previous program is initialized as a whole. After inputting the left and right eye images of the binocular camera, use the above-mentioned internal and external parameter matrices to perform monocular distortion correction on the images, and then perform binocular epipolar line stereo correction, so that the same object in the left and right eye images is basically in the same line, which is convenient for subsequent matching. Next, initialize the necessary parameters of the global stereo matching algorithm, and then pass the previously corrected binocular image along with the parameters to calculate the disparity map of the image relative to the left eye of the binocular camera in the current binocular camera field of view, such as Fig.11 shown.

[0084] Due to factors such as uneven light or differences in the surface material of objects, the disparity map may have many low-reliability holes. In order to fill the holes, the program uses a self-written multi-level mean filtering method to fill them. Assume that the hole area Ω = {(x, y) | D(x, y) = 0}, where D(x, y) = 0 indicates that the depth information is missing. Define the local window W r (x, y) = {(x′, y′)||x′-x|≤r,||y′-y||≤r}, where r is the window radius.

[0085] For each scale r, calculate the local mean:

[0086]

[0087] After multiple iterations from 1 to R using different scales for accumulation or weighted averaging, the final result of multi-level filtering is:

[0088]

[0089] Predict the hole disparity and fill it evenly with the disparity of the surrounding objects, that is, execute:

[0090]

[0091] After reliable filling is completed, the conversion formula between parallax and depth is derived using the geometric relationship of parallel binocular vision Where f is the focal length, B is the baseline distance, and D is the target disparity; the depth map matrix of the current picture relative to the left eye is obtained; then after the depth map matrix is ​​averaged and other data processing is performed, the deep copy retains a visual grayscale depth map image for subsequent visualization / analysis.

[0092] Obviously, the size of the depth map matrix should be close to the size of the left and right monocular images input by the stereo camera. d ,h d ) does not meet the left / right monocular image size (p x ±5px, p y ±5px), it is considered unqualified and the depth map or stereo camera original image should be obtained again.

[0093] The process of "performing high-dimensional point mapping acquisition" is relatively simple, which is the search and acquisition of matrix data. Through simple matrix operations and pointer / subscript operator operations, the precise coordinates of the pepper stem (t x , t y ) is projected into the depth map matrix and the three-dimensional coordinates (t rx , t ry , t rz ). Then, the rotation transformation matrix of the binocular camera-robotic arm base obtained by hand-eye calibration is The three-dimensional coordinates (t rx , t ry , t rz ) is transformed into the coordinates of the pepper stem of the pepper fruit in the robot arm coordinate system (t bx , t by , t bz ); Obviously, the depth of the coordinate point needs to meet the requirements, otherwise it is an invalid coordinate point:

[0094] 0≤t bz ≤2×10 4 ;

[0095] After checking that all the above data analysis and processing processes are valid, the target's 3D point data can be packaged as a topic message object in ROS and broadcast at a frequency of 2-3Hz using the ROS-noetic framework. At the same time, the binocular camera source image of the pepper target, the foreground mask grayscale image separated by foreground and background binarization, and the depth map are refreshed and displayed on the computer monitor, and important parameters are refreshed and displayed in the console for users to observe or developers to debug. After completing the above process, a new round of loop is automatically performed to start the analysis and recognition of the target in the next frame of the pepper image.

[0096] Unless the user manually confirms to exit the program or the program process is not killed, the program will loop, continuously reading images and performing analysis and calculations.

[0097] The binocular camera in the present invention adopts Stereolabs ZED binocular camera. By default, it can obtain 3840×108030fps, 2560×72030fps, VGA100fps and other specifications of color 8bitRGB format binocular merged images; taking 2560×72030fps as an example, after cropping, the left and right color images of 1280×72030fps can be obtained respectively; in addition, the camera baseline is 120mm long, the depth map is 1280×72030fps32bit, with F / 2.0 aperture, the camera can automatically white balance, automatically adjust exposure and other functions. Compared with RGB-D, TOF and other depth cameras, this camera is more suitable for field operations or outdoor scenes.

[0098] The above is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited to the above content. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An intelligent pepper picking system, characterized in that: The invention comprises a visual positioning device for identifying the coordinate position of peppers, a pepper picking device and a control device for picking the peppers according to the position coordinates of the peppers, wherein the pepper picking device comprises a base, a mechanical arm arranged on the base, a picker arranged at the end of the mechanical arm, and a picking drive mechanism for driving the mechanical arm to move along the X-axis and Y-axis directions; the picker comprises a bracket, a fixed knife arranged on the bracket, a movable knife, and a shearing drive mechanism for driving the movable knife to swing so as to cooperate with the fixed knife to complete a shearing action, wherein the fixed knife is fixed on the bracket; the movable knife is rotatably connected to the bracket and is located above the fixed knife; the visual positioning device comprises a binocular camera arranged on the base of the mechanical arm.

2. The intelligent pepper picking system according to claim 1 is characterized in that: The binocular camera is used to obtain the first relative position coordinates of the pepper stalk relative to the binocular camera, and transmit the first relative position coordinates to the control device. The control device calculates the third relative position coordinates of the pepper stalk relative to the base of the robotic arm based on the first relative position coordinates and the second relative position coordinates of the binocular camera and the base of the robotic arm.

3. The intelligent pepper picking system according to claim 1 is characterized in that: The robotic arm is a six-degree-of-freedom robotic arm.

4. The intelligent pepper picking system according to claim 1, characterized in that: The X-axis driving mechanism and the Y-axis driving mechanism both adopt a driving method combining a motor and a screw transmission mechanism.

5. The intelligent pepper picking system according to claim 4 is characterized in that: The X-axis driving mechanism and the Y-axis driving mechanism both further include a guide mechanism, and the guide mechanism is in the form of a combination of a slide rail and a slider, or in the form of a combination of a guide rod and a guide sleeve.

6. The intelligent pepper picking system according to claim 4, characterized in that: The shearing drive mechanism comprises an electric push rod; the electric push rod is connected to the movable knife via a connecting rod, one end of the connecting rod is rotatably connected to the telescopic rod of the electric push rod, and the other end is rotatably connected to the movable knife.

7. An intelligent pepper picking method for the intelligent pepper picking device according to any one of claims 1 to 6, characterized in that: The following steps are involved: Step S1: Perform hand-eye calibration to obtain the rotation transformation matrix between the image coordinate system of the binocular camera and the robotic arm coordinate system; Step S2: The binocular camera identifies the pepper, processes the collected pepper image, and obtains the image coordinates of the pepper stalk in the pepper image; Step S3: Calculate the three-dimensional coordinates of the pepper stalk in the robot arm coordinate system according to the rotation transformation matrix obtained in step S1 and the image coordinates of the pepper stalk obtained in step S2; Step S4: According to the three-dimensional coordinates of the pepper stalk obtained in step S3, the robotic arm drives the picker to pick the peppers; Step S5: Repeat steps S2 to S4 until the binocular camera no longer recognizes the peppers, and the picking work is completed.

8. The intelligent pepper picking method according to claim 7, characterized in that: In step S2, the process of processing the pepper image includes: Step 201: Train the YOLOv10 model using the pepper image collected by the binocular camera, and construct a YOLOv10 training model in ONNX format; Step 202: preprocessing the collected pepper image, sending the preprocessed pepper image to the YOLOv10 training model to perform forward reasoning, and parsing its output from the inferred vector to obtain the best reasoning result; Step 203: Based on the best inference result, the GrabCut algorithm is executed on the pepper image to generate a valid foreground mask; the partial differentials of the connected areas of the foreground mask are continuously solved from bottom to top, the maximum partial differential value is found and its direction is determined to obtain the approximate two-dimensional coordinates of the pepper stalk, and the approximate two-dimensional coordinates are offset corrected to determine the final two-dimensional coordinates of the pepper stalk; Step 204: loading the archived intrinsic parameter matrix and extrinsic parameter matrix of the binocular camera; after inputting the pepper images of the left eye and the right eye of the binocular camera, performing monocular distortion correction on the pepper images through the intrinsic parameter matrix and the extrinsic parameter matrix, and then performing binocular epipolar stereo correction, so that the same object in the pepper images of the left eye and the right eye is in the same line; calculating the disparity map of the corrected pepper image relative to the left eye of the binocular camera in the current binocular camera field of view through the global stereo matching algorithm; filling the holes in the disparity map by adopting the multi-level mean filtering method; after the filling is completed, the depth map matrix of the current pepper image relative to the left eye of the binocular camera is obtained through the relationship between disparity and depth; Step 205: Project the obtained two-dimensional coordinates of the pepper stalk into the depth map matrix to obtain the three-dimensional coordinates of the pepper stalk in the depth map, and calculate the three-dimensional coordinates of the pepper stalk in the pepper fruit in the robot arm coordinate system through the rotation transformation matrix obtained in step S1.

9. The intelligent pepper picking method according to claim 8, characterized in that: In step S201, more than 1200 color pepper images including various weather conditions, lighting conditions and target postures are used to train the YOLOv10 model to obtain a YOLOv10 training model in ONNX format.

10. The intelligent pepper picking method according to claim 9, characterized in that: In step S202, the preprocessing process is: using an image reduction algorithm to scale the pepper image collected by the binocular camera; and then normalizing the scaled pepper image.

Images