Apple picking robot based on visual guidance and picking method

Abstract

The invention relates to the field of agricultural automation equipment, in particular to an apple picking robot based on visual guidance and a picking method.A camera is used for recognizing apples and determining position information, a control system controls a movable chassis to move in a large range according to the position information, and height and angle adjustment is achieved in combination with a lifting column and a small arm; the apples are picked through the fruit picking opening and conveyed to the collecting frame through the net belt. The labor intensity of manual picking is greatly reduced, and the picking efficiency is improved. And then real-time pose information of the apples is obtained through the camera, and the mechanical arm is guided to adjust the action to complete tracking and grabbing of the apples, so that accurate and efficient picking work is realized, the labor intensity of manual picking is reduced, and the picking efficiency is improved. The robot can automatically recognize the target apples and pick the apples, and the picking efficiency and productivity are improved.

Description

Technical Field

[0001] The present invention relates to the field of agricultural automation equipment, and in particular to an apple picking robot and a picking method based on vision guidance. Background Art

[0002] In the fruit and vegetable picking season, the picking period is often short, so the picking work needs to be completed in a short time after the fruits and vegetables mature. At present, manual picking is still the main method for picking apples. Traditional manual picking has the disadvantages of high cost, low efficiency, and high labor intensity. At the same time, with the acceleration of urbanization, there is a serious shortage of labor in rural areas, and there are many safety hazards in the picking process for the elderly in rural areas.

[0003] In recent years, with the rapid development of robot technology, agricultural robots have gradually entered the field of agricultural production, effectively solving the above-mentioned problem of manual picking, liberating labor and improving labor productivity. However, when picking apples in windy and rainy weather, the apples will keep shaking. The existing apple picking robots may have problems such as misgrasping, empty grasping, and collision during the picking process, resulting in large errors when the robot arm grasps, affecting the efficiency of picking. Summary of the invention

[0004] In view of the problem in the prior art that the picking efficiency of apple picking robots is affected by grasping errors, the present invention provides an apple picking robot and a picking method based on vision guidance.

[0005] The present invention is achieved through the following technical solutions: An apple picking robot based on vision guidance comprises a robot body, a visual recognition system, a visual servo system and a control system. The robot body comprises a base, a lifting column, a horizontal rotating motor, a vertical rotating motor, a mechanical arm, a fruit picking port and a hose. The lifting column is fixedly mounted on the robot base, the horizontal rotating motor is fixedly mounted on the top of the lifting column, the vertical rotating motor is fixedly mounted on the top of the horizontal rotating motor, the output shaft of the horizontal rotating motor is perpendicular to the output shaft of the vertical rotating motor, the output shaft of the vertical rotating motor is connected to the mechanical arm, the fruit picking port and the hose are both arranged at the end of the mechanical arm, and the hose is located below the fruit picking port. The visual recognition system and the visual servo system are both arranged at the end of the mechanical arm, the output ends of the visual recognition system and the visual servo system are connected to the input end of the control system, and the output end of the control system is connected to the control end of the mechanical arm.

[0006] Preferably, the lifting column includes a fixed tube, a lifting motor, a lifting screw and a lifting telescopic tube. The bottom of the fixed tube is fixedly connected to the base, the shell of the lifting motor is fixedly connected to the base, the output shaft of the lifting motor is vertically upward, the output shaft of the lifting motor is connected to one end of the lifting screw, the lifting telescopic tube is slidably arranged inside the fixed tube, and the lifting telescopic tube is threadedly connected to the lifting screw.

[0007] Preferably, the housing of the horizontal rotary motor is fixed to the inner top of the lifting and telescopic tube, the output shaft of the horizontal rotary motor passes through the top of the lifting and telescopic tube and is connected to the rotating pan head, and the vertical rotary motor is installed on the rotating pan head; Two spaced-apart parallel support seats are arranged on the rotating pan-tilt platform, one end of the mechanical arm is rotatably arranged between the two support seats through a rotating shaft, the shell of the vertical rotating motor is fixedly connected to any support seat, and the output shaft of the vertical rotating motor is fixedly connected to the rotating shaft.

[0008] Preferably, the end of the robotic arm includes a forearm fixing tube, a forearm telescopic screw, a forearm telescopic tube and a forearm telescopic motor. One end of the forearm is rotatably connected to the support seat, the forearm telescopic motor is fixed inside the forearm fixing tube, the output shaft of the forearm telescopic motor is fixedly connected to the forearm telescopic screw, the forearm telescopic tube is threadedly connected to the forearm telescopic screw, and the fruit picking port is arranged at the other end of the forearm telescopic tube away from the forearm telescopic motor.

[0009] Preferably, the fruit picking opening includes a bracket and two relatively parallel cylinders, the bracket is fixedly connected to one end of the telescopic arm of the forearm away from the telescopic motor of the forearm, and the output ends of the two cylinders are provided with arc-shaped clamps, and the apples are fixed by the relative movement of the clamps.

[0010] A picking method for an apple picking robot based on vision guidance comprises the following steps: S1, the robot moves to the apple tree, performs target detection through the visual recognition system, obtains the apple tracking template in the system, and sends its estimated position and posture to the control system, which drives the robot arm to reach the specified area according to the target estimated position and posture; S2, after the robotic arm is in place, the improved SiamRPN++ tracking model in the visual servo system is used to visually track the apple to be grasped to obtain real-time pose information; S3, the conversion of the apple coordinates from the visual coordinate system to the robotic arm coordinate system is realized through hand-eye calibration; S4, according to the posture error, the robot arm is adjusted in real time to complete the precise tracking of apples and pick them. The picked apples are transported to subsequent equipment through the hose at the picking port.

[0011] Preferably, in S2, the improved SiamRPN++ tracking model includes a feature extraction network, a target prediction network and a background feature perception module. The feature extraction network uses a five-stage ResNet-50 as the feature extraction backbone network, and introduces a SimAM parameter-free attention module to focus on extracting the target picking apple features; the target prediction network is used to predict the position and posture of the apple. After receiving the target features, the network is further processed by fusing the three classification branches with the regression branch to obtain the target confidence and the target bounding box; the background feature perception module uses an apple occlusion detection method based on the confidence to detect whether the target is occluded, and defines the confidence threshold To distinguish whether the target is blocked; in each tracking cycle, calculate the confidence mean, when the mean is lower than the threshold , occlusion occurs.

[0012] Preferably, the SimAM parameter-free attention module combines the spatial attention mechanism and the channel attention mechanism, and enhances the features of the last three stages obtained by the feature extraction network in SiamRPN++ by SimAM, so as to better focus on the apple target without introducing additional parameters.

[0013] Preferably, the RPN module includes a classification branch and a regression branch, and the target prediction network converts the template image features and search graph features After changing the channel dimension through the convolution layer, it is used as the input of the two branches, and correlation calculation is performed on the two branches respectively. By training the data and minimizing the loss function, the corresponding target prediction network can be obtained. By weighting and combining the prediction maps of the three networks, each prediction area corresponds to the confidence of the position, and the area with the best confidence score is used as the final tracking area.

[0014] Preferably, the background feature perception module includes initialization, feature update and feature perception; Among them, initialization is in the initial stage of target tracking. The algorithm first locks the initial frame, which is recorded as ; Extract the global scale-invariant feature transformation feature point set in the frame and record the target position in the initial frame; For the moving apple, the algorithm determines whether to update the current feature by evaluating the richness of feature points and the position change of the bounding box. Feature perception is an algorithm that matches the current feature information with the previously stored feature information to determine the overall movement trend and spatial position change of the feature.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses an apple picking robot based on vision guidance. The robot uses a camera to identify apples and determine the position information. The control system controls the mobile chassis to move over a large range according to the position information. The lifting column and the forearm are combined to adjust the height and angle. The apples are picked using the fruit picking opening and transported to the collection frame through the mesh belt. This invention greatly reduces the labor intensity of manual picking and improves the efficiency of picking. After that, the real-time posture information of the apple is obtained through the camera, and the robot arm is guided to adjust the movement to complete the tracking and grasping of the apple, thereby achieving accurate and efficient picking work, reducing the labor intensity of manual picking and improving the efficiency of picking. The robot can automatically identify the target apples and pick them, thereby improving the picking efficiency and productivity.

[0016] The invention discloses a picking method of an apple picking robot based on vision guidance, which is a precise tracking algorithm guided by vision in real time. The SiamRPN++ algorithm of the visual servo system is used to visually track the apples to be grasped to obtain real-time posture information, thereby improving the accuracy and efficiency of picking.

[0017] Furthermore, the apple's position is updated in real time through vision, and an apple tracking model based on SiamRPN++ is constructed. The SimAM self-attention mechanism is introduced to improve the tracker's ability to extract apple features. An apple background feature perception algorithm is proposed to effectively deal with the problem of dynamic interference in the visual tracking process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is an overall structural diagram of an apple picking robot based on vision guidance of the present invention; Figure 2 is a cross-sectional view of an apple picking robot based on vision guidance of the present invention; Figure 3 The present invention is a flowchart of a picking method of an apple picking robot based on vision guidance.

[0019] In the figure, 1, base; 2, lifting column; 21, lifting telescopic tube; 22, lifting motor; 23, lifting screw; 3, horizontal rotation motor; 4, vertical rotation motor; 5, mechanical arm; 51, small arm telescopic tube; 52, small arm telescopic motor; 53, small arm telescopic screw; 6, camera; 7, fruit picking port; 8, hose; 9, bracket; 10, cylinder; DETAILED DESCRIPTION The present invention is further described in detail below in conjunction with specific embodiments, which are intended to explain the present invention rather than to limit it.

[0020] The present invention discloses an apple picking robot based on visual guidance, referring to Figure 1 , including robot body, visual recognition system, visual servo system and control system.

[0021] The robot body includes a base, a lifting column, a horizontal rotating motor, a vertical rotating motor, a robotic arm, a fruit picking port and a hose. The lifting column is fixedly installed on the robot base. The lifting column includes a fixed tube, a lifting motor, a lifting screw and a lifting telescopic tube. The bottom of the fixed tube is fixedly connected to the base, the shell of the lifting motor is fixedly connected to the base, the output shaft of the lifting motor is vertically upward, the output shaft of the lifting motor is connected to one end of the lifting screw, the lifting telescopic tube is slidably arranged inside the fixed tube, and the lifting telescopic tube is threadedly connected to the lifting screw.

[0022] The horizontal rotary motor is fixedly mounted on the top of the lifting column, and the vertical rotary motor is fixedly mounted on the top of the horizontal rotary motor, and the output shaft of the horizontal rotary motor is perpendicular to the output shaft of the vertical rotary motor. Specifically, the housing of the horizontal rotary motor is fixed to the inner top of the lifting telescopic tube, the output shaft of the horizontal rotary motor passes through the top of the lifting telescopic tube and is connected to the rotating pan-tilt head, and the vertical rotary motor is mounted on the rotating pan-tilt head; Two spaced-apart parallel support seats are arranged on the rotating pan-tilt platform, one end of the mechanical arm is rotatably arranged between the two support seats through a rotating shaft, the shell of the vertical rotating motor is fixedly connected to any support seat, and the output shaft of the vertical rotating motor is fixedly connected to the rotating shaft.

[0023] The output shaft of the vertical rotating motor is connected to the mechanical arm. The end of the mechanical arm includes a forearm fixing tube, a forearm telescopic screw, a forearm telescopic tube and a forearm telescopic motor. One end of the forearm is rotatably connected to the support seat. The forearm telescopic motor is fixed inside the forearm fixing tube. The output shaft of the forearm telescopic motor is fixedly connected to the forearm telescopic screw. The forearm telescopic tube is threadedly connected to the forearm telescopic screw. The fruit picking port is set at the other end of the forearm telescopic tube away from the forearm telescopic motor. Among them, the horizontal rotating motor, the vertical rotating motor, the lifting motor and the forearm telescopic motor all use stepper motors.

[0024] The fruit picking port and the hose are both arranged at the end of the mechanical arm, and the hose is located below the fruit picking port; specifically, the fruit picking port includes a bracket and two relatively parallel cylinders, the bracket is fixedly connected to one end of the telescopic arm of the forearm away from the telescopic motor of the forearm; the output ends of the two cylinders are provided with arc-shaped clamps, and the apples are fixed by the relative movement of the clamps.

[0025] The visual recognition system and the visual servo system are both arranged at the end of the mechanical arm, the output ends of the visual recognition system and the visual servo system are connected to the input end of the control system, and the output end of the control system is connected to the control end of the mechanical arm.

[0026] The present invention discloses an apple picking robot based on vision guidance, which adjusts the working height of the robot by vertically lifting and lowering a lifting and telescopic tube through a lifting motor and a lifting screw. The horizontal rotating motor is used to drive the robot body to rotate in a horizontal plane, and the vertical rotating motor is used to realize the rotation of a mechanical arm in a vertical plane. The mechanical arm drives the forearm telescopic tube through the telescopic motor to realize the telescopic extension of the mechanical arm so as to approach the apple picking area. The visual recognition system and the visual servo system are used to capture the position information of the apples in real time and send the information to the control system. The control system is used to receive the apple position information and drive the mechanical arm to perform precise picking according to the received information.

[0027] The present invention provides a picking method of an apple picking robot based on vision guidance, referring to Figure 3 , including the following steps: S1, the robot moves to the apple tree, performs target detection through the visual recognition system, obtains the apple tracking template in the system, and sends its estimated pose to the control system, which drives the robotic arm to reach the specified area according to the target estimated pose.

[0028] S2, after the robotic arm is in place, the improved SiamRPN++ tracking model in the visual servo system is used to visually track the apple to be grasped to obtain real-time pose information.

[0029] Among them, the improved SiamRPN++ tracking model includes a feature extraction network, a target prediction network and a background feature perception module. The feature extraction network uses a five-stage ResNet-50 as the feature extraction backbone network, and introduces the SimAM parameter-free attention module to focus on extracting the target picking apple features; the target prediction network is used to predict the position and posture of the apple. After receiving the target features, the network is further processed by fusing the three classification branches with the regression branch to obtain the target confidence and the target bounding box; the background feature perception module uses the confidence-based apple occlusion detection method to detect whether the target is occluded and defines the confidence threshold To distinguish whether the target is blocked; in each tracking cycle, calculate the confidence mean, when the mean is lower than the threshold , occlusion occurs.

[0030] Specifically, the SimAM parameter-free attention module combines the spatial attention mechanism and the channel attention mechanism, and the energy function formula is as follows: (1) in, and The target neuron and other neurons for the input features; is the spatial index, is the response value at the corresponding position, and is the weight and bias transformation, is the regularization coefficient, is the linear change on the neuron, is the number of neurons on the channel. By calculating the variance of all neurons With the mean The minimum energy function can be obtained: (2) From formula (2), we can see that the lower the energy, the more different the neurons in area t are from other areas. The scaling algorithm is used to optimize the features: (3) Where E is the total number of channels and spatial dimensions. , the sigmoid function is used to limit the larger value in E, X is the original feature, The optimized features are obtained by enhancing the feature extraction network in SiamRPN++ with SimAM, which can better focus on the apple target without introducing additional parameters.

[0031] The RPN module includes a classification branch and a regression branch. The target prediction network transforms the template image features into and search graph features After changing the channel dimension through the convolution layer, it is used as the input of the two branches, and the correlation calculation is performed on the two branches respectively. Specifically: (4) (5) in, For correlation calculation, Contains a 2k channel, indicating whether the anchor point at the corresponding position is activated. Contains 4k channel vectors, representing the distance between the measured anchor box and the corresponding actual box. When training the network with multiple anchor points, the classification loss uses cross entropy loss and regresses the smoothed L1 loss with normalized coordinates. The normalized distance is smoothed by the L1 loss: (6) in, is the difference between the predicted value and the true value, is a parameter that controls the smoothness. Finally, the loss function is: (7) in, is the cross entropy loss function, which is used to measure the consistency between the target predicted by the model and the target of Apple. is a regression loss function, which is used to measure the difference between the location parameters predicted by the model and the true value. The formula is as follows: (8) (9) in, is the true label, is the probability that the model predicts it to be an apple. is the difference between the predicted location parameter and the true value.

[0032] By training the data and minimizing the loss function, the corresponding target prediction network can be obtained. By weighting and combining the prediction maps of the three networks, each prediction area corresponds to the confidence of the location, and the area with the best confidence score is used as the final tracking area.

[0033] In the process of visual tracking of apples, branches and leaves may cause some occlusion of the apple target. When the target is occluded, the tracker will lose some target features, resulting in tracking failure. To solve this problem, a background feature perception module is designed to correct the pose when the target is occluded. A confidence-based apple occlusion detection method is proposed to detect whether the target is occluded and define the confidence threshold. To distinguish whether the target is blocked; in each tracking cycle, calculate the confidence mean, when the mean is lower than the threshold , occlusion occurs.

[0034] The background feature perception module includes initialization, feature update and feature perception; Among them, initialization is in the initial stage of target tracking. The algorithm first locks the initial frame, which is recorded as ; Extract the global scale-invariant feature transformation feature point set in the frame and record the target position in the initial frame.

[0035] Feature update is for the moving apple. The algorithm determines whether to update the current feature by evaluating the richness of feature points and the position change of the bounding box. The update criteria are: (10) in, is the current frame at time t, and its background feature point set is , the target box position is , is the number of feature sets, is a Boolean function that returns True when the target box is at the edge of the image. To adjust the parameters, it is necessary to adjust according to the on-site situation. Only when the current feature point richness exceeds the product of the adjustment parameter and the number of stored feature points, and the target is not currently at the edge of the image, update the current feature point and target frame position.

[0036] When the apple is blocked, only a few of its feature points are visible or completely invisible, but there is still a lot of background feature information around it. Feature perception is an algorithm that matches the current feature information with the previously stored feature information to determine the overall movement trend and spatial position change of the feature.

[0037] Assume that the feature point set at time t is , the currently stored historical feature point set is , for each feature point, get its feature descriptor vector .for Each feature point in Found Corresponding Satisfy ,in is the Euclidean distance between vectors. The smallest distance means the two feature points are the most matched. ,have , where H is the homography matrix. The coordinates of the stored target box in the historical frame are ,in is the coordinate of the upper left corner of the target, is the width and height of the target box, the corner points can be converted into a point set in the homogeneous coordinate system: (11) Use the obtained homography matrix to transform the point set: Transformed_points=H (12) The transformed point set represents the position of the target in the latest frame, and the minimum rectangular box surrounding these points is obtained. To ensure the accuracy and continuity of target tracking, calculate the updated target box center coordinates and original width and height: (13) The obtained box is the target box obtained by the background perception module, through which the target position where visual tracking fails can be corrected.

[0038] S3, through hand-eye calibration, realize the conversion of apple coordinates from the visual coordinate system to the robotic arm coordinate system; ensure that the robotic arm can accurately grasp the apple.

[0039] S4, according to the posture error, the robot arm is adjusted in real time to complete the precise tracking of apples and pick them. The picked apples are transported to subsequent equipment through the hose at the picking port.

[0040] The picking method of the apple picking robot based on vision guidance of the present invention first obtains the initial posture and tracking template of the target to be grasped through the visual recognition module, and the control system performs strategy allocation to assign the apple to the corresponding robotic arm for grasping; when the target apple enters the grasping work area of ​​the robotic arm, the single target tracking model constructed based on the twin network obtains the real-time posture information of the apple, and adjusts the robotic arm movement in real time to complete the grasping.

[0041] The specific process is as follows: take images of fruits and vegetables at the current position to be picked, perform target detection on the fruit tree images in the visual recognition system, obtain the apple tracking template and estimate its posture and send it to the control system. The control system drives the robotic arm to reach the specified area according to the estimated posture of the target. When the robotic arm reaches the position, the SiamRPN++ algorithm of the visual servo system is used to visually track the apples to be picked to obtain real-time posture information; then the hand-eye calibration is used to realize the conversion of the apple coordinates from the visual coordinate system to the robotic arm coordinate system; finally, the robotic arm is adjusted in real time according to the posture error to complete the precise tracking of the apples, and the robotic arm is driven to perform precise picking; after the picking port picks the target apple, the apple is sent to the storage box through the mesh belt. By integrating advanced visual recognition, visual servo and control systems, efficient and accurate apple picking is achieved, and the intensity of manual labor is reduced.

[0042] The above description is only a preferred embodiment of the present invention and is not intended to impose any limitation on the technical solution of the present invention. Those skilled in the art should understand that, without departing from the spirit and principles of the present invention, the technical solution can also be subjected to several simple modifications and substitutions, and these modifications and substitutions are also within the scope of protection covered by the claims.

Claims

1. A vision-guided apple picking robot, characterized in that: It includes a robot body, a visual recognition system, a visual servo system, and a control system. The robot body includes a base, a lifting column, a horizontal rotating motor, a vertical rotating motor, a mechanical arm, a fruit picking port, and a hose. The lifting column is fixedly installed on the robot base, the horizontal rotating motor is fixedly installed on the top of the lifting column, the vertical rotating motor is fixedly installed on the top of the horizontal rotating motor, the output shaft of the horizontal rotating motor is perpendicular to the output shaft of the vertical rotating motor, the output shaft of the vertical rotating motor is connected to the mechanical arm, the fruit picking port and the hose are both arranged at the end of the mechanical arm, and the hose is located below the fruit picking port; The visual recognition system and the visual servo system are both arranged at the end of the mechanical arm, the output ends of the visual recognition system and the visual servo system are connected to the input end of the control system, and the output end of the control system is connected to the control end of the mechanical arm.

2. The vision-guided apple picking robot according to claim 1, characterized in that: The lifting column includes a fixed tube, a lifting motor, a lifting screw and a lifting telescopic tube. The bottom of the fixed tube is fixedly connected to the base, the shell of the lifting motor is fixedly connected to the base, the output shaft of the lifting motor is vertically upward, the output shaft of the lifting motor is connected to one end of the lifting screw, the lifting telescopic tube is slidably arranged inside the fixed tube, and the lifting telescopic tube is threadedly connected to the lifting screw.

3. The vision-guided apple picking robot according to claim 2, characterized in that: The housing of the horizontal rotary motor is fixed to the inner top of the lifting and telescopic tube, the output shaft of the horizontal rotary motor passes through the top of the lifting and telescopic tube and is connected to the rotating pan head, and the vertical rotary motor is installed on the rotating pan head; Two spaced-apart parallel support seats are arranged on the rotating pan-tilt platform, one end of the mechanical arm is rotatably arranged between the two support seats through a rotating shaft, the shell of the vertical rotating motor is fixedly connected to any support seat, and the output shaft of the vertical rotating motor is fixedly connected to the rotating shaft.

4. The vision-guided apple picking robot according to claim 3, characterized in that: The end of the robotic arm includes a forearm fixing tube, a forearm telescopic screw, a forearm telescopic tube and a forearm telescopic motor. One end of the forearm is rotatably connected to the support seat, the forearm telescopic motor is fixed inside the forearm fixing tube, the output shaft of the forearm telescopic motor is fixedly connected to the forearm telescopic screw, the forearm telescopic tube is threadedly connected to the forearm telescopic screw, and the fruit picking port is arranged at the other end of the forearm telescopic tube away from the forearm telescopic motor.

5. The vision-guided apple picking robot according to claim 4, characterized in that: The fruit picking opening includes a bracket and two relatively parallel cylinders. The bracket is fixedly connected to one end of the telescopic arm of the arm away from the telescopic motor of the arm. The output ends of the two cylinders are provided with arc-shaped clamps, and the apples are fixed by the relative movement of the clamps.

6. A method for picking apples using a vision-guided apple picking robot according to any one of claims 1 to 5, characterized in that: The following steps are involved: S1, the robot moves to the apple tree, performs target detection through the visual recognition system, obtains the apple tracking template in the system, and sends its estimated position and posture to the control system, which drives the robot arm to reach the specified area according to the target estimated position and posture; S2, after the robotic arm is in place, the improved SiamRPN++ tracking model in the visual servo system is used to visually track the apple to be grasped to obtain real-time pose information; S3, the conversion of the apple coordinates from the visual coordinate system to the robotic arm coordinate system is realized through hand-eye calibration; S4, according to the posture error, the robot arm is adjusted in real time to complete the precise tracking of apples and pick them. The picked apples are transported to subsequent equipment through the hose at the picking port.

7. The picking method according to claim 6, characterized in that: In S2, the improved SiamRPN++ tracking model includes a feature extraction network, a target prediction network, and a background feature perception module. The feature extraction network uses a five-stage ResNet-50 as the feature extraction backbone network, and introduces the SimAM parameter-free attention module to focus on extracting the target apple picking features; the target prediction network is used to predict the position and posture of the apple. After receiving the target features, the network is further processed by fusing the three classification branches with the regression branch to obtain the target confidence and target bounding box; The background feature perception module uses the confidence-based apple occlusion detection method to detect whether the target is occluded and defines the confidence threshold. To distinguish whether the target is blocked; in each tracking cycle, calculate the confidence mean, when the mean is lower than the threshold , occlusion occurs.

8. The picking method according to claim 7, characterized in that: The SimAM parameter-free attention module combines the spatial attention mechanism and the channel attention mechanism. By enhancing the feature extraction network in SiamRPN++ with SimAM, the features of the last three stages can be obtained, which can better focus on the apple target without introducing additional parameters.

9. The picking method according to claim 7, characterized in that: The RPN module includes a classification branch and a regression branch. The target prediction network transforms the template image features into and search graph features After changing the channel dimension through the convolution layer, it is used as the input of the two branches, and correlation calculation is performed on the two branches respectively. By training the data and minimizing the loss function, the corresponding target prediction network can be obtained. By weighting and combining the prediction maps of the three networks, each prediction area corresponds to the confidence of the position, and the area with the best confidence score is used as the final tracking area.

10. The picking method according to claim 7, characterized in that: The background feature perception module includes initialization, feature update and feature perception; Among them, initialization is in the initial stage of target tracking. The algorithm first locks the initial frame, which is recorded as ; Extract the global scale-invariant feature transformation feature point set in the frame and record the target position in the initial frame; For the moving apple, the algorithm determines whether to update the current feature by evaluating the richness of feature points and the position change of the bounding box. Feature perception is an algorithm that matches the current feature information with the previously stored feature information to determine the overall movement trend and spatial position change of the feature.

Images