Citrus picking path planning method for low computing resource equipment

Abstract

The invention provides a low computing resource equipment-oriented citrus picking path planning method. The method comprises the following steps: S1, collecting and preprocessing data; S2, designing a citrus fruit detection process; S3, carrying out knowledge distillation on DINO; according to the method, grid modeling is carried out on the orchard by adopting the A * algorithm, and different weights are given to the grids according to the fruit density, so that the robot preferentially passes through a high-density fruit area during path planning, invalid movement is reduced, and the picking efficiency is improved. According to the method, the picking path is optimized by further introducing the genetic algorithm, unnecessary path bending is reduced, the path length and the picking priority of a high-density area are balanced, the picking robot can execute the picking task with a better path, and the overall operation efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural automation, and particularly relates to a citrus picking path planning method for low-computation-resource equipment. Background Art

[0002] With the rapid development of intelligent agriculture, unmanned agricultural machinery and equipment are increasingly widely used in farmland environments. The citrus picking task in orchard environments is an important research direction for automated fruit harvesting. Traditional manual picking methods not only have a high labor intensity but also low efficiency, making it difficult to meet the production requirements of large-scale orchards. Therefore, how to achieve efficient and accurate citrus picking has become an urgent problem to be solved in the field of intelligent agriculture. However, citrus picking robots still face many technical challenges in practical applications, including insufficient fruit detection accuracy, excessive consumption of computing resources, and inefficient picking path planning. These problems seriously affect the overall performance of the intelligent picking system.

[0003] First of all, the accuracy of fruit detection directly affects the execution effect of the picking task. The orchard environment is complex, with citrus fruits varying in shape, size, and color. Moreover, the fruits are often blocked or partially hidden by branches and leaves. At the same time, some fruits are highly similar to the background (such as leaves) in terms of color and texture features, making it difficult for traditional feature extraction-based object detection algorithms to effectively identify fruits. In recent years, methods based on deep learning have made significant progress in object detection tasks. Among them, the DETR (Detection Transformer) model based on the Transformer architecture performs well in fruit detection tasks in complex environments due to its global information perception ability. However, the DETR model has a large computational amount and a large number of parameters, making it difficult to directly deploy on resource-constrained unmanned agricultural machinery equipment and difficult to meet the requirements of real-time detection. To solve this problem, the present invention adopts knowledge distillation technology, using the computationally complex DETR model as the teacher model, and through the knowledge distillation method, effectively transferring its knowledge to a lightweight student model, enabling the student model to maintain a high detection accuracy while significantly reducing the computational cost, thereby achieving efficient and accurate fruit detection.

[0004] Secondly, in addition to fruit detection, the planning of the picking path is also crucial for improving the picking efficiency. Most traditional path planning methods are based on fixed preset paths or simple obstacle avoidance strategies. However, these methods often fail to consider the spatial distribution characteristics of fruits, resulting in the need for the picking robot to move over long distances, increasing the time and energy consumption costs and reducing the overall operation efficiency. Moreover, the orchard environment is highly dynamic and uncertain, with uneven fruit distribution density. The robot needs to complete as many picking tasks as possible within a limited time. Therefore, it is difficult to achieve efficient picking simply by relying on traditional path planning methods. To solve this problem, the present invention uses the A* algorithm to perform grid modeling on the orchard and assigns different weights to the grids according to the fruit density, enabling the robot to preferentially pass through areas with high fruit density during path planning, thereby reducing ineffective movement and improving the picking efficiency. In addition, the genetic algorithm is further introduced to optimize the picking path to reduce unnecessary path bends and balance the path length and the picking priority of high-density areas, enabling the picking robot to execute the picking task with a better path and improving the overall operation efficiency. Summary of the Invention

[0005] The object of the present invention is to provide a citrus picking path planning method for low-computing-resource equipment to improve the visual information processing performance on agricultural edge devices, optimize the picking path, and improve the operation efficiency.

[0006] The technical solution adopted by the present invention is a citrus picking path planning method for low-computing-resource equipment, and the steps include:

[0007] Step S1: Data collection and preprocessing;

[0008] Step S2: Design the citrus fruit detection process;

[0009] Step S3: Perform knowledge distillation on DINO;

[0010] Step S4: Plan the picking path of citrus;

[0011] Step S5: Deploy the trained citrus picking model and the path planning algorithm to the agricultural picking equipment.

[0012] Further, the specific steps of S1 are as follows:

[0013] S11: Collect pictures of citrus fruits from immature to mature;

[0014] S12: Perform data augmentation processing on the original image; [[ID=3,3]]

[0015] S13: Divide the preprocessed data set into a training set, a validation set, and a test set.

[0016] Further, the specific process of citrus fruit detection designed in S2 is as follows:

[0017] S21. Read the citrus fruit image preprocessed in S12 and perform normalization. The formula is as follows:

[0018]

[0019] where I(x, y) represents the original pixel value of the image at position (x, y), and I norm (x, y) represents the pixel value of the normalized image at position (x, y), and the value range is [0, 1];

[0020] Then adjust the size of the normalized image I norm The specific formula is as follows:

[0021] I resize = Resize(I norm , H, W)

[0022] where I resize represents the image after size adjustment, Resize(·) represents the image size adjustment operation, H represents the height of the image after size adjustment, and W represents the width of the image after size adjustment;

[0023] S22. Input the image I resize after size adjustment into the ResNet backbone network for feature extraction, and extract the deep features of the image, that is:

[0024] F = ResNet(I resize ; θ)

[0025] where ResNet(·) represents the forward propagation operation using ResNet as the backbone network, θ is the network weight, and F represents the deep feature map extracted from the image;

[0026] S23. Flatten the extracted deep feature map F. First, flatten F into a one-dimensional sequence Then define a fixed positional encoding and add it to the flattened features, and then extract the global feature Y through multiple Transformer encoders. The specific formula is as follows:

[0027] X = Flatten(F), X = {x1, x2, …, x i , …, x N}

[0028] X pos = X + P

[0029] Y = TransformerEncoder(X pos )

[0030] where Flatten(·) represents the flattening operation, X represents the one-dimensional sequence after flattening, x i represents the feature value at the i-th position, N = H × W, d represents the number of channels, represents the set of real numbers, N represents the total number of positions, H represents the height of the resized image, W represents the width of the resized image, P represents the position encoding matrix, each row of which corresponds to the encoding of a specific position, TransformerEncoder(·) represents the encoder processing after multi-layer self-attention and feed-forward network operations, and X pos represents the feature after adding the position encoding;

[0031] S24, target query and decoding.

[0032] Furthermore, the specific steps of knowledge distillation in S3 are as follows:

[0033] S31: Select DINO-ResNet50 as the teacher model, train the teacher model so that it learns rich feature information on a large-scale dataset, and fine-tune it on the citrus dataset to obtain the optimized ability for citrus fruit detection;

[0034] S32, train the student model based on the output of the teacher model, and transfer the target matching strategy, feature information, and spatial features of the teacher model to the student model through the knowledge distillation method.

[0035] Furthermore, in S32, the specific steps of training the student model are as follows:

[0036] S32a, for an input image, define the query set Q T of the teacher model as the prior knowledge for its object detection, and use it as an additional query input to the student model. Given the teacher query Q T , obtain a stable target matching query from the teacher model For each GT (Ground Truth) object y i , the teacher model assigns a target matching query directly used for calculating the loss in the student model training. Finally, the loss of the teacher assignment module is:

[0037]

[0038] where L QueryKD (·) represents the loss of the teacher assignment module, y represents the GT label set, Denote the query prediction result in the student model that matches the $i$-th GT target, $M$ represents the total number of matched GT targets, $L$ match $(·)$ represents the bipartite matching cost, $y$ i represents the $i$-th GT target;

[0039] S32b transfers the shallow and deep feature knowledge of the teacher model to the student model, optimizes the loss function, and the loss function for mimicking the spatial features of the teacher model is as follows:

[0040]

[0041] where 1, $L$ represents the number of layers of the distilled features selected by the encoder, $F$ T ∈ ℝ H×W×d and $F$ S ∈ ℝ H×W×d represent the encoder output features of the teacher model and the student model respectively, $H$ represents the height of the resized image, $W$ represents the width of the resized image, and $d$ is the number of channels of the features.

[0042] Furthermore, in the above-mentioned S4, the specific steps for planning the citrus picking path are as follows:

[0043] S41 uses the knowledge-distilled DINO model to detect and locate the fruits. The position of each detected fruit is $(x, y)$, where $x$ and $y$ represent the two-dimensional coordinates of the fruit in the image. The detection result is represented by the bounding box generated by the model, and the center point of the bounding box is the current position of the fruit. For each image, the model outputs a set $C$ containing all the fruit information, specifically as follows:

[0044] $C = \{(x_1, y_1), (x_2, y_2),..., (x$ n , $y$ n )\}$

[0045] where $n$ represents the total number of fruits in the image;

[0046] S42 analyzes the fruit distribution in different regions through the fruit density estimation algorithm. Each region is a grid with a size of $G$, and the density calculation formula is:

[0047]

[0048] where $Density(·)$ represents the fruit density of the region, $n$ represents the total number of fruits in the image, $u$ represents the $u$-th fruit target, is the indicator function, and when the fruit position $(x, y)$ falls within the grid $G$ u inside, returns 1, otherwise returns 0;

[0049] S43: Employ the A* path planning algorithm. According to the fruit distribution and density, initially calculate an optimal picking path. Divide the picture area into multiple grids, set the weight of each grid according to the fruit density. The goal of the path planning algorithm is to start from the starting point, pass through multiple grids, and finally reach the target area with the minimum total weight of the path, that is:

[0050]

[0051] where w k is the weight of the k-th grid on the path, D k is the movement cost of the k-th grid, min represents taking the minimum value, m is the total number of grids passed on the path, ò is a very small positive number to prevent division by zero, Density(·) represents the fruit density in the area, and G k represents the k-th grid area;

[0052] S44. Use the genetic algorithm to optimize the picking path. The optimization objective function is:

[0053]

[0054] where F represents the optimization objective function, d k is the movement distance between grids in the path (such as the distance from G k-1 to G k ), λ is a regulation factor used to balance the weight of the path length and the priority picking of high-density areas, min represents taking the minimum value, v is the number of target grids to be picked in the path, ò is a very small positive number to prevent division by zero, Density(·) represents the density of the fruit in the area, and G k represents the k-th grid.

[0055] The beneficial effects of the present invention are:

[0056] 1. The present invention optimizes the distillation process of the DINO model through the teacher query assignment distillation method, significantly reducing the demand for computing resources while ensuring the target detection accuracy, enabling it to operate efficiently on low-computing-power agricultural equipment.

[0057] 2. The present invention optimizes the path by combining the A* algorithm and the genetic algorithm, improving the working efficiency of the picking robot, reducing the redundancy of the movement path, and improving the picking accuracy in the orchard environment.

[0058] 3. The present invention enhances the robustness of citrus detection in complex environments, optimizes the intelligent picking path planning, making it more suitable for agricultural edge devices, and providing a more efficient solution for intelligent agricultural operations. Description of the Drawings

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0060] Figure 1 is the flowchart of the present invention.

[0061] Figure 2 is the architecture diagram of the citrus picking path planning and deployment system of the present invention. Detailed implementation manners

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0063] Embodiment

[0064] As Figures 1-2 shown, the embodiment of the present invention provides a citrus picking path planning method for low-computing-resource equipment. The steps include:

[0065] Step S1: Data collection and preprocessing. The specific steps are as follows:

[0066] S11, collect pictures of citrus fruits from immature to mature to comprehensively capture the appearance characteristics of citrus fruits at different growth stages. In the data collection stage of the present invention, multiple collections were carried out in the orchard to ensure that citrus data under different varieties and environmental conditions were covered. After obtaining the image data of the citrus orchard environment, use the COCO (Common Objects in Context) format to mark the category and position tags of the data, save the annotation file in JSON format, save the object category identifier to the category_id field, and save the upper left coordinates, width, and height information of the object bounding box to the bbox field to ensure the accurate positioning of the fruit target. The image data covers different maturity stages, occlusion situations, and lighting conditions to enhance the generalization ability of the model;

[0067] S12. Perform data augmentation on the original image to obtain a preprocessed image dataset, so as to reduce the impact of illumination changes, occlusion problems, and different shooting angles on the number of fruits. The methods for processing the original image include: color balance, adjustment of brightness change, enhancement of sharpness, color adjustment, horizontal and vertical shearing, horizontal and vertical translation (including annotation boxes), and rotation operations;

[0068] S13. Divide the preprocessed dataset into a training set, a validation set, and a test set according to the quantity ratio of 7:1:2. This ratio can make the best use of limited data resources on the premise of ensuring the training effect and the accuracy of model validation.

[0069] Step S2: Design the citrus fruit detection process based on DINO (DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection), which is specifically as follows:

[0070] S21. Read the preprocessed citrus fruit image in step S12 and perform normalization processing to map the pixel values to the range of [0, 1]. The goal of the normalization operation is to eliminate the influence of differences in acquisition devices and illumination conditions on the pixel value scale between different images, and ensure the numerical stability in subsequent model training. The normalization formula is as follows:

[0071]

[0072] where I(x, y) represents the original pixel value of the image at the position (x, y), and I norm (x, y) represents the pixel value of the normalized image at the position (x, y), and its value range is [0, 1].

[0073] Then adjust the size of the normalized image I norm to ensure that all input images have a unified size for subsequent batch processing. The specific formula is as follows:

[0074] I resize = Resize(I norm , H, W)

[0075] where I resize represents the image after size adjustment, Resize(·) represents the image size adjustment operation, H represents the height of the image after size adjustment, and W represents the width of the image after size adjustment.

[0076] S22. Take the image I resizeIt is input into the ResNet backbone network for feature extraction to extract the deep features of the image. The size of the feature map is gradually reduced through convolutional layers and pooling layers while retaining key semantic information, that is:

[0077] F = ResNet(I resize ; θ)

[0078] where ResNet(·) represents the forward propagation operation using ResNet as the backbone network, θ is the network weight, and F represents the deep feature map extracted from the image.

[0079] S23. Flatten the extracted deep feature map F. First, flatten F into a one-dimensional sequence Then, to preserve spatial information, define a fixed positional encoding and add it to the flattened features, and then extract the global feature Y through a multi-layer Transformer (Multi-layer Transformer Encoder) encoder. The specific formula is as follows:

[0080] X = Flatten(F), X = {x1, x2, …, x i , …, x N}

[0081] X pos = X + P

[0082] Y = TransformerEncoder(X pos )

[0083] where Flatten(·) represents the flattening operation, X represents the flattened one-dimensional sequence, x i represents the feature value at the i-th position, N = H × W, d represents the number of channels, represents the set of real numbers, N represents the total number of positions, H represents the height of the resized image, W represents the width of the resized image, P represents the positional encoding matrix, each row of which corresponds to the encoding of a specific position, and TransformerEncoder(·) represents the encoder processing after multi-layer self-attention and feed-forward network operations, and X pos represents the feature after adding the positional encoding.

[0084] In the encoder stage, each Transformer layer contains self-attention and a feed-forward neural network (Feed-Forward Network, FFN) to capture long-range feature dependencies.

[0085] S24, Target Query and Decoder. In the target query stage, DeNoising Queries are introduced to enable the model to converge faster. The specific steps are as follows:

[0086] S24a, Initialize the query vectors (Object Queries), where each query vector corresponds to a possible fruit target.

[0087] S24b, Introduce a denoising mechanism. Introduce the positions of some correct targets and the positions of some perturbed targets as additional prior information to reduce the uncertainty of target matching. The correct target positions refer to target candidates that are very close to or directly correspond to the true positions of the actual objects in the image, and the perturbed targets refer to those target candidates obtained by applying certain random disturbances to the correct target positions.

[0088] S24c, Cross-Attention. The target query and the feature map perform cross-attention decoding to generate the final target prediction. Each query vector is decoded into where, represents the predicted class, represents the predicted bounding box. Then, the bipartite matching loss is used to optimize the target assignment (the matching relationship between the target and the prediction) to ensure that the model still maintains high accuracy in the Anchor-Free detection framework.

[0089] The bipartite matching loss means that in the training stage, the principle of label assignment is to minimize the matching cost between the model prediction and the groundtruth (GT), and the optimal bipartite matching is obtained through the Hungarian algorithm.

[0090] Step S3: Perform knowledge distillation on DINO to compress the DINO model from a larger teacher model into a lightweight student model to reduce the computational resource requirements, enable it to adapt to the deployment of unmanned agricultural machinery equipment, achieve the lightweight of the large model, and ensure its effectiveness in the citrus fruit detection task. The specific steps are as follows:

[0091] S31: Select DINO-ResNet50 as the teacher model, train the teacher model to learn rich feature information on a large-scale dataset, and perform fine-tuning on the citrus dataset to obtain the optimized ability for citrus fruit detection;

[0092] S32. Train the student model (DINO-ResNet18) based on the output of the teacher model. Through the knowledge distillation method, transfer the target matching strategy, feature information, and spatial features of the teacher model to the student model, enabling it to maintain a high detection accuracy while reducing the computational cost. The specific steps are as follows:

[0093] S32a. For an input image, define the query set Q of the teacher model T as the prior knowledge for its object detection and use it as an additional query input to the student model. Given the teacher query Q T , obtain a stable target matching query from the teacher model to guide the target matching process of the student model. For each GT (GroundTruth) object y i , the teacher model assigns a target matching query directly used for the student model training to calculate the loss. Finally, the loss of the teacher assignment module is:

[0094]

[0095] where L QueryKD (·) represents the loss of the teacher assignment module, y represents the GT label set, represents the query prediction result matching the i-th GT object in the student model, M represents the total number of matching GT objects, and L match (·) represents the bipartite matching cost, and y i represents the i-th GT object.

[0096] S32b. Transfer the shallow and deep feature knowledge of the teacher model to the student model to improve the feature extraction ability of the student model. This method optimizes the loss function to encourage the student model to learn more expressive attention maps in the shallow layer and obtain richer semantic information in the deep layer, thereby improving the accuracy of feature representation. The loss function for mimicking the spatial features of the teacher model is:

[0097]

[0098] where 1, L represent the number of layers of the distilled features selected by the encoder, F T ∈R H×W×d and F S ∈R H×W×d represent the encoder output features of the teacher model and the student model respectively, H represents the height of the resized image, W represents the width of the resized image, and d is the number of channels of the features.

[0099] Step S4. Plan the picking path to improve the overall efficiency of the picking operation. The specific steps are as follows:

[0100] S41. Use the DINO model after knowledge distillation to detect and locate the fruits, and determine their exact positions in the image. The position of each detected fruit is (x, y), where x and y represent the two-dimensional coordinates of the fruit in the image. The detection result is represented by the bounding box generated by the model, and the center point of the bounding box is the current position of the fruit. For each image, the model outputs a set C containing all fruit information, which is as follows:

[0101] C = {(x1, y1), (x2, y2),..., (x n , y n )}

[0102] Among them, n represents the total number of fruits in the image.

[0103] S42. Analyze the fruit distribution in different regions through the fruit density estimation algorithm to provide reference information for path optimization, so as to improve the efficiency of path planning. Optimize the picking order according to the fruit density. Assume that each region is a grid with a grid size of G, and the density calculation formula is:

[0104]

[0105] Among them, Density(·) represents the fruit density of the region, n represents the total number of fruits in the image, u represents the u-th fruit target, is the indicator function. When the fruit position (x, y) falls within the grid G u inside, returns 1, otherwise returns 0.

[0106] Through density estimation, divide the fruit area into high-density areas and low-density areas. Set a threshold, mark the areas with fruit density greater than the threshold as "high-density areas", and mark the areas with fruit density lower than the threshold as "low-density areas".

[0107] S43: Adopt the A* path planning algorithm. According to the fruit distribution and density, initially calculate an optimal picking path. Divide the picture area into multiple grids, and set the weight of each grid according to the fruit density. The greater the weight, the higher the fruit density. The goal of the path planning algorithm is to start from the starting point, pass through multiple grids, and finally reach the target area, and the total weight of the path is the smallest, that is:

[0108]

[0109] Among them, w k is the weight of the k-th grid on the path, D kis the moving cost of the k-th grid, min represents taking the minimum value, m is the total number of grids passed on the path, ò is a very small positive number to prevent division by zero, Density(·) represents the fruit density of the area, and G k represents the k-th grid area.

[0110] S44. Use the genetic algorithm to optimize the picking path to reduce redundant movements and ensure that fruits in high-density areas are picked first. The optimization objective function is:

[0111]

[0112] where F represents the optimization objective function, and d k is the moving distance between grids in the path (such as the distance from G k-1 to G k ), λ is a regulation factor used to balance the weight of the path length and the priority picking of high-density areas, min represents taking the minimum value, v is the number of target grids to be picked in the path, ò is a very small positive number to prevent division by zero, Density(·) represents the density of fruits in the area, and G k represents the k-th grid.

[0113] Step S5. Deploy the trained citrus picking model and the path planning algorithm to agricultural picking equipment with low computing resources. The specific steps are as follows:

[0114] S51. Select a deep learning inference framework suitable for low-computing-resource devices. In this embodiment, TensorFlowLite is selected as the inference framework, which is optimized for mobile and embedded devices and can run efficiently in a low-power environment.

[0115] S52. Convert the trained citrus fruit detection model and the path planning algorithm (the detection model of DINO-ResNet18 and the picking path planning algorithm) into the TensorFlow Lite format. The conversion process includes freezing the model graph, quantization, and optimization processing to ensure that the model occupies less memory and computing resources during the inference process.

[0116] S53. Use the inference hardware of the edge computing device. For the citrus picking equipment with low computing resources, select an ARM architecture single-board computer equipped with a dedicated NPU or a low-power GPU to achieve efficient inference in the edge computing environment and accelerate the real-time execution of the model.

[0117] S54. Deploy the converted model and the inference framework to agricultural equipment, and integrate the converted TensorFlowLite model into the embedded system of the agricultural equipment. The embedded system collects orchard images through a high-definition camera, and is equipped with a distance sensor and other positioning devices to determine the position and distance of citrus fruits, ensuring that the target fruit and its environmental information can be judged in real time during the detection and path planning processes.

[0118] S55. Integrate sensors and data streams into the inference system. For the citrus picking task, integrate the data streams from cameras, GPS, IMU (Inertial Measurement Unit), and other environmental sensors, enabling the inference system to receive and process real-time data, so that the model can not only perform fruit detection, but also generate accurate picking paths by combining position information and respond in real time to the dynamic changes in the orchard.

[0119] S56. Conduct real-time inference tests in the actual orchard environment. Deploy the equipment to the actual orchard environment to carry out real-time inference tests, evaluate the detection accuracy, path planning effect, and response speed of the model on a low-computing resource platform. According to the test feedback, make necessary adjustments and optimizations to the detection threshold, path planning parameters, and sensor fusion strategy to adapt to actual picking scenarios such as different lighting, occlusion, and complex backgrounds.

[0120] S57. Deploy a remote monitoring and update mechanism. Through Internet of Things (IoT) technology, achieve remote monitoring of the citrus picking equipment and collection of model performance data, establish a remote update mechanism, and timely push model optimization patches and system updates to ensure that the equipment always maintains the best state during long-term operation and can quickly respond to orchard environment changes or new task requirements.

[0121] The present invention proposes a citrus picking path planning method for low-computing resource equipment. By combining the DINO object detection model and knowledge distillation technology, it realizes efficient and accurate citrus fruit detection on edge devices. At the same time, by integrating the A* algorithm and genetic algorithm to optimize the picking path, the operation efficiency and path rationality are effectively improved. The purpose of the present invention is to provide a visual recognition and path planning solution with both accuracy and efficiency for resource-constrained picking equipment.

[0122] Experimental verification

[0123] Table 1 Detection results of different models

[0124]

[0125] Among them, AP is the average precision, and there is also AP 50 , AP 75 , and values such as AP 50Indicates the AP measurement value when the IoU threshold (the IoU threshold (Intersection over Union Threshold) is a criterion used to measure the overlap degree between the prediction result and the ground truth annotation in tasks such as object detection and image segmentation, and is usually used as the basis for determining whether the prediction is correct) is 0.5, AP 75 Indicates the measurement value when the IoU threshold is 0.75, AP S Indicates the AP measurement value of the target bounding box with a pixel area less than 322. FPS represents the number of frames that the model can process per second during the inference process, reflecting the real-time ability of the model. Params represents the total number of trainable parameters in the model, indicating the size and complexity of the model. FLOPS refers to the number of floating-point operations executed during the inference process, representing the computational cost.

[0126] During the training process of the student model, the present invention maintains the optimal weights of the teacher network and uses DINO (ResNet-18) as the student model. To verify the effect of the present invention, the student network DINO (ResNet-18) is compared with the teacher network DINO (ResNet-50), and is also compared with traditional CNN models (such as Faster-RCNN, YOLOv5x) and some latest DETR-like models. The experimental results show that compared with traditional CNN models, the method of the present invention performs better in terms of performance. And after knowledge distillation, the model performance exceeds Grounding DINO, and compared with the model without distillation, the method of the present invention improves by 8.3 APs. The present invention also tests the running speed of these models on an RTX3060 graphics card, and the results show that our solution reaches 4.2 frames per second in terms of running efficiency.

[0127] Each embodiment in this specification is described in a related manner. For the same or similar parts between the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts can refer to the partial description of the method embodiment.

[0128] The above is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A citrus picking path planning method for equipment with low computing resources, characterized in that the steps Including: Step S1: Data collection and preprocessing; Step S2: Design the citrus fruit detection process; Step S3: Perform knowledge distillation on DINO; Step S4, Plan the picking path of citrus; Step S5, Deploy the trained citrus picking model and path planning algorithm to agricultural picking equipment.

2. The citrus fruit picking path planning method for low-computation-resource equipment according to claim 1, characterized in that The specific steps of S1 are as follows: S11, Collect pictures of citrus fruits from immature to mature; S12, Perform data augmentation on the original image; S13, Divide the preprocessed dataset into a training set, a validation set, and a test set.

3. A citrus picking path planning method for low-computation-resource equipment according to claim 1, characterized in that The citrus fruit detection process designed in S2 is specifically as follows: S21, Read the preprocessed citrus fruit image in S12 and perform normalization processing. The formula is as follows: Among them, I(x, y) represents the original pixel value of the image at the position (x, y), and I norm (x, y) represents the pixel value of the normalized image at the position (x, y), and the value range is [0, 1]; Then adjust the normalized image I norm with the following specific formula: I resize = Resize(I norm , H, W) Among them, I resize represents the resized image, Resize(·) represents the image resizing operation, H represents the height of the resized image, and W represents the width of the resized image; S22, input the resized image I resize into the ResNet backbone network for feature extraction to extract the deep features of the image, i.e.: F = ResNet(I resize ; θ) Among them, ResNet(·) represents using ResNet as the backbone network for forward propagation operation, θ is the network weight, and F represents the deep feature map extracted from the image; S23. Flatten the extracted deep feature map F, first flatten F into a one-dimensional sequence Then define a fixed positional encoding And add it to the flattened features, and then extract the global feature Y through a multi-layer Transformer encoder. The specific formula is as follows: X = Flatten(F), X = {x1, x2, …, x i , ···, x N} X pos = X + P Y = TransformerEncoder(X pos ) Among them, Flatten(·) represents the flattening operation, X represents the one-dimensional sequence after flattening, and x i represents the eigenvalue at the i-th position, N = H × W, d represents the number of channels, represents the set of real numbers, N represents the total number of positions, H represents the height of the image after size adjustment, W represents the width of the image after size adjustment, P represents the position encoding matrix, each row of which corresponds to the encoding of a specific position, and TransformerEncoder(·) represents the encoder processing after multi-layer self-attention and feed-forward network operations, X pos represents the feature after adding the position encoding; S24, Target query and decoding.

4. A citrus fruit picking path planning method for low-computation-resource equipment according to claim 1, characterized in that The specific steps of knowledge distillation in S3 are as follows: S31: Select DINO-ResNet50 as the teacher model, train the teacher model to learn rich feature information on a large-scale dataset, and perform fine-tuning on the citrus dataset to obtain the optimization ability for citrus fruit detection; S32, Train the student model based on the output of the teacher model. Through the knowledge distillation method, transfer the target matching strategy, feature information, and spatial features of the teacher model to the student model.

5. A citrus picking path planning method for low-computation-resource equipment according to claim 4, characterized in that In S32 shown, the specific steps of training the student model are as follows: S32a. Define the query set Q of the teacher model for an input image T as prior knowledge for its object detection and use it as additional queries input to the student model. Given the teacher queries Q T , obtain a stable object matching query from the teacher model For each GT object y i , the teacher model assigns an object matching query directly used for calculating the loss in the student model training. Finally, the loss of the teacher assignment module is: Among them, L QueryKD (·) represents the loss of the teacher assignment module, y represents the set of GT labels, represents the query prediction result in the student model that matches the i-th GT target, M represents the total number of matching GT targets, L match (·) represents the bipartite matching cost, y i represents the i-th GT target; S32b, Transfer the shallow and deep feature knowledge of the teacher model to the student model, optimize the loss function, and the loss function imitating the spatial features of the teacher model is as follows: Among them, 1. L represents the number of layers of the distilled features selected by the encoder, F T ∈R H×W×d and F S ∈R H×W×d respectively represent the encoder output features of the teacher model and the student model. H represents the height of the resized image, W represents the width of the resized image, and d is the number of channels of the features.

6. The citrus fruit picking path planning method for low-computation-resource equipment according to claim 1, wherein, In S4, the specific steps of planning the citrus picking path are as follows: S41, Use the DINO model after knowledge distillation to detect and locate the fruits. The position of each detected fruit is (x, y), where x and y represent the two-dimensional coordinates of the fruit in the image. The detection result is represented by the bounding box generated by the model, and the center point of the bounding box is the current position of the fruit. For each image, the model outputs a set C containing all fruit information, specifically as follows: C = {(x1, y1), (x2, y2),..., (x n , y n )} Among them, n represents the total number of fruits in the image; S42, Analyze the fruit distribution in different regions through the fruit density estimation algorithm. Each region is a grid with a grid size of G, and the density calculation formula is: Among them, Density(·) represents the density of fruits in the region, n represents the total number of fruits in the image, and u represents the u-th fruit target. is an indicator function. When the fruit position (x, y) falls within the grid G u inside, returns 1, otherwise returns 0. S43: Adopt the A* path planning algorithm. According to the fruit distribution and density, initially calculate an optimal picking path. Divide the picture area into multiple grids, set the weight of each grid according to the fruit density. The goal of the path planning algorithm is to start from the starting point, pass through multiple grids, and finally reach the target area, and the total weight of the path is the smallest, that is: Among them, w k is the weight of the k-th grid on the path, D k is the movement cost of the k-th grid, min represents taking the minimum value, m is the total number of grids passed on the path, ò is a very small positive number to prevent division by zero, Density(·) represents the regional fruit density, and G k represents the k-th grid area; S44, Use the genetic algorithm to optimize the picking path. The optimization objective function is: Among them, F represents the optimization objective function, and d k is the moving distance between grids in the path (such as the distance from G k-1 to G k . λ is the adjustment factor used to balance the weight of the path length and the priority picking in the high-density area. min means taking the minimum value. v is the number of target grids to be picked in the path. ò is a very small positive number to prevent division by zero. Density(v) represents the density of fruits in the area. G k represents the k-th grid.

Images