A Deep Learning-Based Method for Identifying and Locating Small Watermelons in Vertical Cultivation Mode

Abstract

This invention relates to a method for identifying and locating small watermelons in a three-dimensional cultivation model based on deep learning. It identifies the target by establishing a first-level SSD target detection model based on deep learning and a second-level target centroid extraction model based on pseudo-color depth image segmentation, marking the centroid points of the detected targets, i.e., generalized points of interest; determining the coordinates of the picking point in the image coordinate system, and then determining the coordinates of the picking point in the camera coordinate system, thus completing the watermelon identification and location. This invention can meet the needs of real-time detection of small watermelons during actual harvesting operations, with fast identification and detection speed; it realizes real-time detection and identification location functions for small watermelon harvesting operations, and solves the problem of poor robustness in target information acquisition under greenhouse three-dimensional cultivation environments.

Description

Technical Field

[0001] This invention belongs to the fields of automated harvesting robot technology development for greenhouse fruits and vegetables and computer vision technology, specifically involving a method for identifying and locating small watermelons in a three-dimensional cultivation mode based on deep learning. Background Technology

[0002] my country is the world's largest watermelon grower and consumer. In recent years, small watermelons have gained popularity, with individual fruits typically weighing 1-2 kg and about 15 cm in diameter. They are aesthetically pleasing, sweet, have a short cultivation cycle, and are highly adaptable to different production conditions. Their price is one to two times higher than that of ordinary watermelons, resulting in significantly higher production efficiency. Greenhouse vertical cultivation uses trellises or hanging vines, allowing the vines to grow vertically, maximizing space utilization, ensuring even light penetration, good ventilation, and superior quality. Currently, the production of small watermelons in my country still relies heavily on manual labor, resulting in high harvesting costs, harsh harvesting environments, limited working space, and high labor intensity. Therefore, accelerating the automation, intelligentization, and precision harvesting of small watermelons in greenhouse environments is of paramount importance.

[0003] In recent years, watermelon detection and recognition algorithms based on traditional visual algorithms have been proposed. However, in greenhouse vertical cultivation, problems such as changes in light intensity, similarity between fruit and background, and overlapping of target occlusions lead to slightly poor detection results. To improve the robustness and accuracy of acquiring information for small watermelon harvesting and thus increase the efficiency of automated harvesting, traditional algorithms are gradually being replaced by deep learning detection algorithms. SSD is a one-stage object detection algorithm that uses VGG16 as the backbone network and adds convolutional layers to obtain more feature layers for detection. It has a fast detection speed and high accuracy, making it suitable for detecting small watermelons in greenhouse vertical cultivation. Summary of the Invention

[0004] To address the aforementioned technical problems, the purpose of this invention is to provide a method for identifying and locating small watermelons under a three-dimensional cultivation model based on deep learning, which meets the need for real-time detection of small watermelons during actual harvesting operations, and features accurate identification and location with fast detection speed.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for identifying and locating small watermelons in a three-dimensional cultivation model based on deep learning, comprising the following steps:

[0007] S1. Establish a first-level SSD target detection model based on deep learning and a second-level target centroid extraction model based on pseudo-color depth image segmentation;

[0008] S2. Input an image to be detected containing one or more detection targets into the first-level SSD object detection model to perform first-level object detection and obtain a detection image containing one or more best prediction boxes.

[0009] S3. Render the image to be detected into a pseudo-color depth map; match the pseudo-color depth map with the detection image containing the best prediction box obtained in step S2 to obtain a pseudo-color depth map containing the best prediction box; filter out the background information outside the best prediction box, and extract the RGB image and the pseudo-color depth map within the best prediction box.

[0010] S4. Input the RGB image within the best predicted bounding box and the pseudo-color depth map within the best predicted bounding box into the second-level target centroid extraction model, and mark the length of the major axis y of the elliptical contour. b And the centroid of the detection target, i.e., the generalized point of interest;

[0011] The coordinates of the centroid of the detection target are:

[0012]

[0013] In the formula, x i y i Here, are the x and y coordinates of the pixel within the connected region i containing the centroid of the detected target, respectively. , where are the average values ​​of the horizontal and vertical coordinates of the pixels within the connected region i containing the centroid of the target, i.e., the coordinates of the centroid of the target; n is the number of pixels within the connected region i containing the centroid of the target.

[0014] S5. Align the generalized interest points marked in step S4 with the center point of the best prediction box in the detection image obtained in step S2, determine the coordinates of the picking point in the image coordinate system, and then determine the coordinates of the picking point in the camera coordinate system to complete the watermelon recognition and localization.

[0015] The coordinates of the picking point in the image coordinate system are:

[0016]

[0017] In the formula, u p v p These are the x and y coordinates of the picking point in the image coordinate system, respectively. b The length of the major axis of the elliptical profile; To detect the coordinates of the target centroid;

[0018] The coordinates of the picking point in the camera coordinate system are:

[0019]

[0020] In the formula, (x p yp , z p (x) represents the coordinates of the picking point in the camera coordinate system. b Let be the radius of the major axis of the elliptical profile in the image coordinate system. y τ The y-coordinate of the point of tangency between the tangent plane formed by the best prediction bounding box and the detected target in the camera coordinate system. d represents the pixel depth value in camera coordinates of the depth image captured by a stereo depth camera, obtained using stereo ranging theory; r represents the radius of the detected target. y r The y-coordinate of the pixel on the right border of the best prediction box; y l This represents the y-axis coordinate of the pixel on the left border of the best prediction box.

[0021] Step S1 includes the following steps:

[0022] S1.1. Collect multiple images of small watermelons, and create a primary detection sample library of small watermelons under the greenhouse three-dimensional cultivation mode after the image size is uniformly set to 512×512 pixels; mark the feature information of small watermelons in each image of the primary detection sample library, including the location of the target area, the size of the target area and the target category;

[0023] S1.2 Establish the first-level SSD object detection network. The first-level SSD object detection model is trained based on the first-level detection sample library obtained in step S1.1. The first-level SSD object detection model uses VGG16 as the backbone network and adds convolutional layers to obtain deeper semantic information.

[0024] S1.3. Convert the images in the primary detection sample library into pseudo-color depth maps as the secondary detection sample library; establish a secondary target centroid extraction network, and train the secondary target centroid extraction model based on the secondary detection sample library.

[0025] In step S1.1, data augmentation methods are used to improve the robustness of the final trained model and reduce overfitting during the training process.

[0026] In step S1.1, the processed sample data is randomly divided into a training set, a test set, and a validation set, with a sample set ratio of 6:2:2. The training sample set is used to train the network model, the test sample set is used to evaluate the model, and the validation set is used to optimize hyperparameters during the training process.

[0027] In step S1.2, the PRC curve is used as an evaluation index to calculate the average precision AP; the optimal parameters are selected based on the PRC curve, and the optimal first-level SSD target detection model is trained.

[0028] AP=∫0 1 PR·dR

[0029] Where P represents accuracy. R is the recall rate. dR represents the differential of R; T p F represents the number of samples that were correctly classified as positive. p F represents the number of samples incorrectly classified as positive. N This represents the number of samples that were incorrectly classified as negative.

[0030] Step S2 includes the following steps:

[0031] S2.1 After converting the size of the image to be detected to 512×512 pixels, it is input into the first-level SSD object detection model. The forward propagation result of the network yields six feature maps with different side lengths, namely 38, 19, 10, 5, 3 and 1.

[0032] S2.2 Construct three envelope boxes with different aspect ratios for each point on each feature map, with aspect ratios of 1:1, 6:5 and 3:2 respectively;

[0033] S2.3 Obtain the predicted bounding boxes at different scales of all feature layers. First, filter out the predicted bounding boxes with low confidence. Then, use the non-maximum suppression method to suppress some overlapping boxes and output the best predicted bounding box.

[0034] Step S4 includes the following steps:

[0035] S4.1 Perform temporal filtering, hole filling, HSV thresholding, and noise removal on the pseudo-color depth image in sequence, and extract the white region.

[0036] S4.2. Based on the white area extracted in step S4.1, perform ROI extraction on the RGB image, and then sequentially perform RGB channel separation, ultra-green feature extraction and grayscale conversion, and Otsu's threshold segmentation processing to extract the black area.

[0037] S4.3. Extract the ROI from the black region extracted in step S4.2, and then perform grayscale conversion, removal of small connected regions, and elliptical contour fitting to obtain the major axis length y of the elliptical contour. b And the coordinates of the centroid, i.e., the coordinates of the generalized point of interest.

[0038] Obtain the coordinates (x, y) of the picking point in the camera coordinate system. p y p , z pAfter that, based on the transformation matrix and translation matrix obtained from the hand-eye calibration, the coordinates of the picking point in the robotic arm coordinate system are calculated, and these coordinates are used as the target point for the robotic arm's movement to carry out the picking operation.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] The target recognition method proposed in this invention can meet the needs of real-time detection of small watermelons during actual harvesting operations, and the recognition and detection speed is fast; it realizes the real-time detection and identification positioning function of small watermelon harvesting operations, and solves the problem of poor robustness in acquiring target information in greenhouse three-dimensional cultivation environment.

[0041] This invention can automatically identify and locate small watermelons based on their phenotypic characteristics such as color, texture, and shape.

[0042] This invention can adapt to the growth environment of small watermelons harvested in a greenhouse three-dimensional cultivation environment, and has a strong generalization ability: it has a strong filtering ability for stem and leaf shading, background interference, etc.

[0043] This invention combines an SSD deep learning detection model with visual technology based on pseudo-color depth image segmentation to identify the target, mark the centroid of the target (i.e., the generalized point of interest), determine the coordinates of the picking point in the image coordinate system, and then determine the coordinates of the picking point in the camera coordinate system to complete watermelon identification and localization. This solves the problem of traditional visual algorithms' poor ability to handle complex background information and addresses the issue of poor bounding box effect when applying the SSD target detection model to greenhouse agriculture environmental prediction.

[0044] This invention is lightweight and highly portable, and can be implemented on multiple operating platforms, making it possible to achieve intelligent and precise harvesting. Attached Figure Description

[0045] Figure 1 The flowchart shows the small melon recognition and localization detection method based on deep learning and computer vision of the present invention.

[0046] Figure 2 This is a schematic diagram of the image acquisition system used in an embodiment of the present invention;

[0047] Figure 3 This is a first-level SSD object detection model network based on deep learning;

[0048] Figure 4 This is a schematic diagram of rendering a pseudo-color depth image according to an embodiment of the present invention;

[0049] Figures 5a to 5f This is a schematic diagram of the generalized interest point calibration process of the second-level target centroid extraction model based on pseudo-color depth image segmentation of the present invention.

[0050] Figure 6 This is a flowchart of the second-level target centroid extraction model based on pseudo-color depth image segmentation of the present invention;

[0051] Figure 7 This is a schematic diagram illustrating the determination of the harvesting point in this invention. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] like Figure 1 As shown, a method for identifying and locating small watermelons in a three-dimensional cultivation model based on deep learning includes the following steps:

[0054] S1. Establish a first-level SSD object detection model based on deep learning and a second-level object centroid extraction model based on pseudo-color depth image segmentation; the specific steps are as follows:

[0055] S1.1. Collect multiple images of small watermelons, and create a primary detection sample library of small watermelons under the greenhouse three-dimensional cultivation mode after the image size is uniformly set to 512×512 pixels; mark the feature information of small watermelons in each image of the primary detection sample library, including the location of the target area, the size of the target area and the target category;

[0056] In one embodiment of the present invention, a system is established as follows: Figure 2 The illustrated small-scale watermelon image acquisition system uses a depth camera mounted on a chassis between watermelon plants via a tripod, approximately 600mm from the plants. The depth camera acquires 2000 target images, each 1280×720 pixels in size, to create a dataset of small watermelon fruit samples representing different growth postures, lighting conditions, and varieties, using the widely adopted PASCAL VOC2007 dataset format. Preferably, the depth camera is an Intel RealSense D435.

[0057] In one embodiment of the present invention, data augmentation methods are used to improve the robustness of the final trained model and reduce overfitting during training. These methods include contrast enhancement or reduction, overall brightening or darkening of the image, rotation (±5°, ±10°), noise addition, and Gaussian noise addition.

[0058] In one embodiment of the present invention, the processed sample data is randomly divided into a training set, a test set, and a validation set, with a sample set ratio of 6:2:2. The training sample set is used to train the network model, the test sample set is used to evaluate the model, and the validation set is used to optimize hyperparameters during training.

[0059] In one embodiment of the invention, the image size is converted to 512×512 pixels. During training, the image is first frozen and then unfrozen to rapidly decrease the loss value until the loss function curve converges, at which point training ends. Appropriate hyperparameters are adjusted as follows: Batch_size is 16 when frozen and 4 when unfrozen; momentum is 0.9; learning rate is 0.001 when frozen and 0.0001 when unfrozen; IOU is 0.5; and weight decay coefficient is 0.0005.

[0060] S1.2. Establish the first-level SSD object detection network. Train the first-level SSD object detection model based on the first-level detection sample library obtained in step S1.1. The first-level SSD object detection model uses VGG16 as the backbone network, but due to the complex detection background and occlusion phenomenon, convolutional layers are added to obtain deeper semantic information to improve the detection effect. Figure 3 This is a diagram of the SSD network model structure.

[0061] In one embodiment of the present invention, the PRC curve is used as an evaluation index to calculate the average precision AP; the optimal parameters are selected based on the PRC curve to train the optimal first-level SSD target detection model.

[0062] AP=∫0 1 PR·dR

[0063] Where P represents accuracy. R is the recall rate. dR represents the differential of R; T p F represents the number of samples that were correctly classified as positive. p F represents the number of samples incorrectly classified as positive. N This represents the number of samples that were incorrectly classified as negative.

[0064] S1.3. Convert the images in the primary detection sample library into pseudo-color depth maps as the secondary detection sample library; establish a secondary target centroid extraction network, and train the secondary target centroid extraction model based on the secondary detection sample library.

[0065] S2. Input an image to be detected containing one or more detection targets into the first-level SSD object detection model to perform first-level object detection and obtain a detection image containing one or more best prediction boxes.

[0066] S2.1 After converting the size of the image to be detected to 512×512 pixels, it is input into the first-level SSD object detection model. The forward propagation result of the network yields six feature maps with different side lengths, namely 38, 19, 10, 5, 3 and 1.

[0067] S2.2 Construct three envelope boxes with different aspect ratios for each point on each feature map, with aspect ratios of 1:1, 6:5 and 3:2 respectively;

[0068] S2.3 Obtain the predicted bounding boxes at different scales of all feature layers. First, filter out the predicted bounding boxes with low confidence. Then, use the non-maximum suppression method to suppress some overlapping boxes and output the best predicted bounding box.

[0069] like Figure 3 As shown, in one embodiment of the present invention, feature maps of conv4_3, conv7, conv8_2, conv9_2, conv10_2, and conv11_2 corresponding to side lengths of 38, 19, 10, 5, 3, and 1 are extracted for regression and classification. For each predicted bounding box in the extracted six layers of feature maps, a confidence score (i.e., class score) and a bounding box offset localization are calculated. Predicted bounding boxes are sampled uniformly across different layers. A CNN is used to extract features for classification and regression, generating a confidence score for each target class in each predicted bounding box. Wherein:

[0070] The first step is to determine which predicted bounding boxes in the labeled image dataset match the labeled bounding boxes. Essentially, this involves using these predicted bounding boxes to predict the target represented by each labeled bounding box, and then training the network accordingly. Therefore, the real purpose of this step is to construct positive and negative samples. Simply put, for any predicted bounding box, if there is a matching labeled bounding box, it is a positive sample; otherwise, it is a negative sample. For each labeled bounding box, it is first matched with the predicted bounding box that has the highest Intersection over Union (IoU). This ensures that each labeled bounding box corresponds to at least one predicted bounding box, meaning that each target can be detected by at least one box. Secondly, for the remaining unmatched predicted bounding boxes, they are matched together as long as there is an IoU greater than 0.5 with any labeled bounding box. After these two steps, all predicted bounding boxes are divided into two categories: those matching labeled bounding boxes are positive samples, those not matching are negative samples, and those not matching any labeled bounding box can only match the background and are classified as negative samples.

[0071] S3. Render the image to be detected as a pseudo-color depth map, such as... Figure 4 As shown; the pseudo-color depth map is matched with the detection image containing the best prediction box obtained in step S2 to obtain a pseudo-color depth map containing the best prediction box, as shown. Figure 5a As shown; background information other than the best predicted bounding box is filtered out, such as... Figure 5b As shown, extract the RGB image within the best prediction box and the pseudo-color depth map within the best prediction box;

[0072] S4. Input the RGB image within the best prediction box and the pseudo-color depth map within the best prediction box into the second-level target centroid extraction model, and mark the centroid points of the detected targets, i.e., the generalized interest points.

[0073] S4.1. Perform temporal filtering, hole filling, HSV thresholding, and noise removal on the pseudo-color depth image sequentially, and extract the white region, as follows: Figure 5c As shown;

[0074] S4.2. Based on the white region extracted in step S4.1, perform ROI extraction on the RGB image, and then sequentially perform RGB channel separation, super-green feature extraction and grayscale conversion, and Otsu's threshold segmentation processing to extract the black region, such as... Figure 5d As shown;

[0075] S4.3. Extract the ROI from the black region extracted in step S4.2, and then perform grayscale conversion, removal of small connected regions, and elliptical contour fitting to obtain the major axis length y of the elliptical contour. b And the coordinates of the centroid, i.e., the coordinates of the generalized point of interest, such as Figure 5f As shown;

[0076] The coordinates of the centroid of the detection target are:

[0077]

[0078] In the formula, x i y i Here, are the x and y coordinates of the pixel within the connected region i containing the centroid of the detected target, respectively. , where are the average values ​​of the horizontal and vertical coordinates of the pixels within the connected region i containing the centroid of the target, i.e., the coordinates of the centroid of the target; n is the number of pixels within the connected region i containing the centroid of the target.

[0079] like Figure 6As shown, in one embodiment of the present invention, noise reduction processing is performed on the pseudo-color depth map within the optimal prediction box, and a morphological hole-filling method is established. The OpenCV `findcontours()` function is used to discriminate the contours and fill the holes. The red region is extracted from the processed pseudo-color depth image. The OpenCV `cv2.MORPH_OPEN()` function is used to remove noise from the extracted image region, filtering out small holes inside foreground objects or small black dots on objects. The white region is further extracted. The OpenCV `cv.inRange()` function is used to obtain the ROI region's mask, and the OpenCV `cv.bitwise()` function is used to extract the ROI region, i.e., the image mask of the small watermelon region, and the mask is matched to the corresponding region in the RGB image. Background information such as withered branches and shadows of the small watermelon fruit is suppressed through RGB image channel separation, grayscale extraction, and super-green feature extraction. Then, the grayscale-processed black region is extracted, and the ROI region is further extracted and grayscale-processed. The `skimage.morphology.remove_small_objects()` function (which removes small regions) is used to detect and remove discrete small connected regions from the image. The `fitEllipse()` function from OpenCV is used to fit an ellipse to the outer contour of a small watermelon and mark its centroid.

[0080] S5, such as Figure 7 As shown, the generalized interest points marked in step S4 are aligned with the center point of the best prediction box in the detection image obtained in step S2 to determine the coordinates of the picking point in the image coordinate system, and then the coordinates of the picking point in the camera coordinate system are determined to complete the watermelon recognition and localization.

[0081] The coordinates of the picking point in the image coordinate system are:

[0082]

[0083] In the formula, u p v p These are the x and y coordinates of the picking point in the image coordinate system, respectively. b The length of the major axis of the elliptical profile; To detect the coordinates of the target centroid;

[0084] The coordinates of the picking point in the camera coordinate system are:

[0085]

[0086] In the formula, (x p y p , z p(x) represents the coordinates of the picking point in the camera coordinate system. b Let be the radius of the major axis of the elliptical profile in the image coordinate system. y τ The y-coordinate of the point of tangency between the tangent plane formed by the best prediction bounding box and the detected target in the camera coordinate system. d represents the pixel depth value in camera coordinates of the depth image captured by a stereo depth camera, obtained using stereo ranging theory; r represents the radius of the detected target. y r The y-coordinate of the pixel on the right border of the best prediction box; y l This represents the y-axis coordinate of the pixel on the left border of the best prediction box.

[0087] The coordinates (x, y) of the picking point in the camera coordinate system are obtained using the method described above. p y p , z p Then, based on the transformation matrix and translation matrix obtained from the hand-eye calibration, the coordinates of the picking point in the robotic arm coordinate system are calculated, and these coordinates are used as the target point for the robotic arm's movement to carry out the picking operation.

Claims

1. A deep learning-based small watermelon recognition and positioning method in a three-dimensional cultivation mode, characterized in that, Comprising the following steps: S1, establishing a first level SSD target detection model based on deep learning and a second level target centroid extraction model based on pseudo-color depth image segmentation; S2, inputting a to-be-detected image containing one or more detection targets into the first level SSD target detection model for first level target detection to obtain a detection image containing one or more optimal prediction boxes; S3, rendering the to-be-detected image into a pseudo-color depth image; matching the pseudo-color depth image with the detection image containing the optimal prediction box obtained in step S2 to obtain a pseudo-color depth image containing the optimal prediction box; filtering out background information other than the optimal prediction box to extract the RGB image within the optimal prediction box and the pseudo-color depth image within the optimal prediction box; S4, input the RGB image in the best prediction box and the pseudo-color depth map in the best prediction box into the second-level target centroid extraction model, mark the long axis length y of the elliptical contour b and detect the target centroid point, i.e. the generalized interest point; The detection target centroid point coordinates are: In the formula, x i , y i are respectively the horizontal and vertical coordinates of the pixel in the connected region i where the center point of the detection target is located, are respectively the average values of the horizontal and vertical coordinates of the pixel in the connected region i where the center point of the detection target is located, that is, the coordinates of the center point of the detection target; n is the number of pixels in the connected region i where the center point of the detection target is located. S5, aligning the generalized interest points marked in step S4 with the center points of the optimal prediction boxes in the detection image obtained in step S2 to determine the coordinates of the picking point in the image coordinate system and then determine the coordinates of the picking point in the camera coordinate system, thereby completing the watermelon recognition and positioning; The coordinates of the picking point in the image coordinate system are: In the formula, u p , v p are the horizontal and vertical coordinates of the picking point in the image coordinate system, respectively, y b is the length of the major axis of the elliptical contour. is the coordinate of the center of the detected target. The coordinates of the picking point in the camera coordinate system are: wherein (x p , y p , z p ) is the coordinate of the picking point in the camera coordinate system, x b is the long axis radius of the ellipse contour in the image coordinate system, y τ is the coordinate value of the y-axis of the tangent point of the tangent plane formed by the best prediction frame and the detection target in the camera coordinate system, d is the pixel depth value of the depth image photographed by the binocular depth camera in the camera coordinate system measured using the binocular ranging theory; r is the radius of the detection target, y r is the coordinate value of the y-axis of the pixel point on the right side frame of the best prediction frame; and y l is the coordinate value of the y-axis of the pixel point on the left side frame of the best prediction frame.

2. The deep learning-based small watermelon recognition and positioning method in a three-dimensional cultivation mode according to claim 1, characterized in that, The step S1 comprises the following steps: S1.1, collecting multiple small watermelon target images, creating a first level detection sample library of small watermelons in a greenhouse stereoscopic cultivation mode after unifying the image size to 512*512 pixels; labeling the feature information of the small watermelons in each image in the first level detection sample library, wherein the feature information includes the position of the target region, the size of the target region and the target category; S1.2, establishing a first level SSD target detection network, training the first level SSD target detection model according to the first level detection sample library obtained in step S1.1; using VGG16 as the backbone network of the first level SSD target detection model and adding new convolutional layers to obtain deeper semantic information; S1.3, converting the images in the first level detection sample library into pseudo-color depth images as a second level detection sample library; establishing a second level target centroid extraction network and training the second level target centroid extraction model according to the second level detection sample library.

3. The deep learning-based small watermelon recognition and positioning method in a three-dimensional cultivation mode according to claim 2, characterized in that, In the step S1.1, a data enhancement method is used to improve the robustness of the final trained model and reduce the overfitting problem in the training process.

4. The deep learning-based small watermelon recognition and positioning method in a three-dimensional cultivation mode according to claim 2, characterized in that, In the step S1.1, the processed sample data is randomly divided into a training set, a test set and a validation set, and the sample set ratio is 6:2:2; the training sample set is used for training the network model, the test sample set is used for evaluating the model, and the validation set is used for optimizing the hyperparameters in the training process.

5. The deep learning-based small watermelon recognition and positioning method in a three-dimensional cultivation mode according to claim 2, characterized in that, In the step S1.2, the PRC curve is used as an evaluation index to calculate the average precision AP; the optimal parameters are selected according to the RPC curve to train the optimal first level SSD target detection model; wherein P is the accuracy, R is the recall, dR represents the differential of R; T p is the number of positive samples that are correctly classified; F p is the number of positive samples that are incorrectly classified; F N is the number of negative samples that are incorrectly classified.

6. The deep learning-based small watermelon recognition and positioning method in a three-dimensional cultivation mode according to claim 1, characterized in that, The step S2 comprises the following steps: S2.1, inputting the to-be-detected image with a size of 512*512 pixels into the first level SSD target detection model, and obtaining six feature maps with different side lengths through network forward propagation, wherein the side lengths of the feature maps are 38, 19, 10, 5, 3 and 1 respectively; S2.2, construct three different aspect ratio envelope boxes on each point on each feature map respectively, and the aspect ratios of the envelope boxes are 1:1, 6:5 and 3:2 respectively; S2.3, obtain the prediction boxes under different scales of all feature layers, first filter out the prediction boxes with too low confidence, and then suppress the partially overlapped boxes through the non-maximum suppression method to output the best prediction box.

7. The deep learning-based small watermelon recognition and positioning method in a three-dimensional cultivation mode according to claim 1, characterized in that, The step S4 comprises the following steps: S4.1, sequentially performing time filtering, hole filling, HSV threshold segmentation and noise removal processing on the pseudo-color depth image, and extracting a white region; S4.2, performing ROI extraction on the RGB image according to the white region extracted in step S4.1, and then performing RGB channel separation, super green feature extraction grayscale and Otsu threshold segmentation processing to extract a black region. S4.3, the black region extracted in step S4.2 is subjected to ROI extraction, and sequentially subjected to graying, small connected region removal and ellipse contour fitting processing to obtain the long axis length y of the ellipse contour b and the centroid point coordinates, i.e. the generalized interest point coordinates.

8. The deep learning-based small watermelon recognition and positioning method in a three-dimensional cultivation mode according to claim 1, characterized in that, Obtaining the coordinates (x p , y p , z p ) of the picking point in the camera coordinate system, the conversion matrix and the translation matrix obtained according to the hand-eye calibration are used to calculate the coordinates of the picking point in the robot coordinate system, and the coordinates are used as the motion target point of the robot to perform the picking operation.

Images