Sugarcane planting quality real-time detection method and system based on deep learning

Abstract

The invention discloses a sugarcane planting quality real-time detection method based on deep learning, and the method comprises the following steps: calibrating the internal and external parameters of a depth camera, and calculating the position and posture of an object in a world coordinate system, so as to obtain the relative position information of the depth camera; a depth camera is used for collecting sugarcane seed images after seed falling of a sugarcane seed machine, and the collected images are preprocessed; the YOLOV8-OBB network model is improved, a fusion extensible residual error DWR attention mechanism is introduced, and an improved YOLOV8-OBB network model is obtained; training the improved YOLOV8-OBB model based on the preprocessed sugarcane seed image to obtain a sugarcane seed quality target detection model; and deploying the sugarcane seed quality target detection model to an AI computing platform NVIDIA JETSON ORIN NX, and operating the target detection model to detect the seed falling quality of the sugarcane planter in real time. According to the method, the seed falling quality of the sugarcane planter is detected in real time, the sowing mode is adjusted according to the detection result, precise planting is achieved, and the sugarcane field planting quality is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent agricultural machinery, and in particular to a real-time detection method and system for sugarcane planting quality based on deep learning. Background Art

[0002] At present, the main method of mechanical planting of sugarcane in Guangxi is still semi-mechanized manual planting and whole-pole real-time cutting. The planting methods include single-row top-joined strip planting, double-row top-joined strip planting, double-row herringbone strip planting, etc. These sowing methods have the characteristics of large seeding volume and high labor intensity of manual seeding. Therefore, Guangxi is currently vigorously promoting the new agricultural technology of horizontal planting of pre-cut double-bud segments of sugarcane. The horizontal planting of double rows in one ridge has high yield per unit area, strong resistance to lodging, good ventilation between sugarcane seeds, and helps to reduce the phenomenon of missing and few seedlings. The pre-cut sugarcane horizontal planter developed with this technology has the characteristics of low labor intensity, high planting efficiency, high seedling rate, stable root system and resistance to lodging, and high yield per mu. However, due to the hilly terrain and uneven slopes in Guangxi, the lateral seed placement during the planting process is unstable. The dropped sugarcane seeds will tilt forward and backward, have height differences up and down, and have uneven spacing between seeds. The forward and backward tilting will occupy nutrients and cause poor development. The height difference will lead to uneven development height and poor resistance to lodging. The uneven spacing between seeds and inconsistent development will affect the harvesting efficiency and the subsequent sugarcane seed germination rate and sugarcane field yield.

[0003] In order to reduce or even avoid problems such as reduced sugarcane field production caused by seeding problems, it is crucial to monitor the quality of the planter's seeding process. The existing pre-cut sugarcane horizontal planter does not have a seeding quality monitoring function, and the planter driver cannot know the quality of the seeded sugarcane. Once the seeding device has structural damage, the seeding quality cannot be known in time, which will greatly waste sugarcane seeds, fields and manpower. In addition, seeding quality monitoring can also provide important reference indicators for subsequent field yield estimation, while improving the intelligence level of the sugarcane horizontal planter.

[0004] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0005] The purpose of the present invention is to provide a real-time detection method and system for sugarcane planting quality based on deep learning, so as to overcome the problem that the existing equipment cannot timely know the sowing quality during sowing, which will greatly cause the waste of sugarcane seeds, fields and manpower.

[0006] To achieve the above object, the present invention provides a real-time detection method for sugarcane planting quality based on deep learning, comprising the following steps:

[0007] (1) Calibrate the internal and external parameters of the depth camera and calculate the position and posture of the object in the world coordinate system to obtain the relative position information of the depth camera;

[0008] (2) Using a depth camera to collect images of sugarcane seeds after the sugarcane seeding machine drops the seeds, and preprocessing the collected images;

[0009] (3) Improve the YOLOV8-OBB network model and introduce the fused expandable residual DWR attention mechanism to obtain the improved YOLOV8-OBB network model;

[0010] (4) The improved YOLOV8-OBB model is trained based on the preprocessed sugarcane seed images to obtain a sugarcane seed quality target detection model;

[0011] (5) The sugarcane seed quality target detection model is deployed on the AI ​​computing platform NVIDIA JETSON ORIN NX, and the target detection model is run to perform real-time detection of the seed quality of the sugarcane planter.

[0012] Preferably, in the above technical solution, step (1) calibrates the internal and external parameters of the depth camera by using a standard calibration plate, wherein the internal parameters include focal length, principal point position, and distortion parameters, and the external parameters are the rotation and translation matrices of the camera relative to the world coordinate system. The calibrated parameters are used to reconstruct the position and posture of the object in the world coordinate system, thereby accurately obtaining the relative position information of the camera relative to the object being measured.

[0013] Preferably, in the above technical solution, the method of calibrating the internal and external parameters of the depth camera in step (1) includes:

[0014] 1) Prepare a standard black and white checkerboard pattern as a calibration plate;

[0015] 2) Use a depth camera to capture images of the checkerboard pattern from different angles and distances to ensure that the entire effective field of view is covered;

[0016] 3) Automatically detect the chessboard corner positions on each image. Corner detection can use the findChessboardCorners function in the OpenCV library or other applicable methods;

[0017] 4) According to the detected corner point positions and the known world coordinates (i.e. the real positions of the corner points on the calibration pattern), the camera's intrinsic parameter matrix is ​​solved using the least squares method or other optimization algorithms. The intrinsic parameters include focal length (f_x, f_y), principal point (c_x, c_y), radial distortion coefficient (k_1, k2, ...) and tangential distortion coefficient (p_1, p_2);

[0018] 5) Define the coordinates of each corner point on the chessboard in the world coordinate system as the world coordinates of the calibration pattern;

[0019] 6) Match the corner point positions in the image with the corner point positions in the world coordinate system;

[0020] 7) Using the known intrinsic parameter matrix, image coordinates, and world coordinates, the rotation and translation matrices are calculated by solving the perspective n-point problem (PnP) to obtain the extrinsic parameters of the camera.

[0021] The internal parameter matrix is:

[0022]

[0023] Where f x is the horizontal focal length, f y is the vertical focal length, c x is the x-coordinate position in the image, c y is the y coordinate position in the image.

[0024] Distortion coefficient:

[0025] r 2 =x 2 +y 2

[0026] x distorted =x(1+k1r 2 +k2r 4 +…)+2p1xy+p2(r 2 +2x 2 )

[0027] y distorted =y(1+k1r 2 +k2r 4 +…)+2p2xy+p1(r 2 +2y 2 )

[0028] Where r 2 is the square of the radius, that is, the square of the distance from the point to the center of the image, k1, k2, k3 are radial distortion coefficients, which are used to describe the radial distortion that gradually increases from the center to the edge of the image, and p1, p2 are tangential distortion coefficients, which are used to describe the tangential distortion caused by the camera sensor not being parallel to the lens optical axis.

[0029] Preferably, in the above technical solution, step (2) of preprocessing the image is to perform enhancement preprocessing on the collected image and use X-AnyLabeling to mark the sugarcane species in the image with a rotating box.

[0030] Preferably, in the above technical solution, step (2) image enhancement preprocessing is to use a Mosaic data enhancement method to splice the images, enhance the background information in the images, and obtain an enhanced image.

[0031] The rotation frame annotation is to use annotation software to annotate the sugarcane species in the image with an OBB frame with an angle, and one OBB frame can be marked with (x c ,y c , w, h, θ) to represent, (x c ,y c ) are the coordinates of the center point of the rectangle, w is the width of the rectangle, h is the height of the rectangle, and θ is the rotation angle of the rectangle relative to the horizontal axis.

[0032] Preferably, in the above technical solution, in step (2), the image information of sugarcane seeds collected by the depth camera includes images of the sugarcane seeds falling to the ground and background information, the background includes various landforms, and the collected types of fallen seeds include images of sugarcane seeds with uniform planting and images of sugarcane seeds with chaotic planting.

[0033] Preferably, in the above technical solution, the method of improving the YOLOV8-OBB network model in step (3) includes:

[0034] 1) Improve the YOLOv8-OBB model by introducing the fused expandable residual DWR attention mechanism and the attention weight w i The calculation formula is as follows:

[0035]

[0036] Among them, q i and k i They are query vector and key vector, q i and k i is calculated from the input feature map and used to determine the attention weight. k is the dimension of the key vector, divided by This is to scale the dot product value to prevent the softmax function from saturating. i represents the attention weight of the i-th position, which determines the importance of the position. The similarity between the query vector and the key vector is calculated. The larger the dot product, the higher the correlation. The attention weight of each position is calculated by the softmax function, so that the sum of all attention weights is 1, so that the value vector can be weighted and summed according to the weight.

[0037] 2) The DWR attention mechanism enhances the model’s ability to capture the morphological features of fallen seeds by weighting the important areas in the input image;

[0038] 3) Add the DWR module based on YOLOv8-OBB. This module uses an expandable residual structure to learn the complex morphological features of fallen seeds, which improves the robustness and accuracy of the model. By adjusting the network architecture of YOLOv8-OBB and integrating the DWR module, the effective extraction and detection of fallen seed morphological features are achieved. The improved model shows higher accuracy and faster speed in the task of detecting fallen seed morphological features, and is suitable for real-time monitoring of sugarcane planting quality.

[0039] Preferably, in the above technical solution, the method of obtaining the seed morphological feature recognition network model in step (4) comprises:

[0040] 1) Use the preprocessed image dataset to train the improved YOLOv8-OBB model;

[0041] 2) The dataset contains images of sugarcane seeds with uniform and disordered seeding, as well as corresponding label information;

[0042] 3) Use cross-validation methods to evaluate the performance of the model and optimize the model training process by adjusting hyperparameters;

[0043] 4) Use the loss function to measure the difference between the model prediction results and the true label, and update the model weights through the back-propagation algorithm to minimize the loss;

[0044] 5) Finally, a trained sugarcane seed quality target detection model is obtained.

[0045] Preferably, in the above technical solution, step (5) comprises:

[0046] 1) Convert the trained sugarcane seed quality target detection model into a format suitable for the NVIDIA JETSON ORIN NX platform;

[0047] 2) Use the TensorRT toolkit to optimize the model and generate a TensorRT engine file for efficient execution;

[0048] 3) Deploy the TensorRT optimized model on the NVIDIA JETSON ORIN NX platform and configure the necessary operating environment;

[0049] 4) Test the performance of the deployed model in real-time detection of seed placement quality of sugarcane planters;

[0050] 5) Adjust the model configuration to adapt to different lighting and environmental conditions to ensure the accuracy and stability of the detection results.

[0051] A real-time detection system for sugarcane planting quality based on deep learning, comprising: an identification unit, which includes the above-mentioned sugarcane seed quality target detection model; an image recognition device, which includes the above-mentioned depth camera and obtains image information in real time; a working unit, which is a planting machine; and a control unit, which is data-connected with the identification unit, the image recognition device and the working unit, the control unit receives data from the image recognition unit, makes judgments based on the data from the identification unit, and controls the operation of the working unit.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] (1) The real-time detection method of sugarcane planting quality based on deep learning in the present invention significantly improves the detection accuracy of the YOLOv8-OBB model for sugarcane planting quality by introducing the fused expandable residual DWR attention mechanism. This improvement enables the model to more accurately identify the morphological characteristics of sugarcane plants, especially the morphological changes after the sugarcane seeds fall to the ground, thereby improving the accuracy of detection. The DWR attention mechanism enhances the model's ability to capture the morphological characteristics of sugarcane by weighting the important areas in the input image. This helps the model focus more effectively on the key parts of the sugarcane seeds, improving the accuracy and robustness of detection.

[0054] (2) The model of the present invention is specially designed to effectively cope with complex planting environments and different soil factors, including uniform and chaotic planting of sugarcane seeds. This enables the model to maintain good performance under a variety of different planting conditions. Through careful tuning of the model, including hyperparameter adjustment and data set enhancement, it is ensured that the model can maintain stable performance in the face of light changes and background interference.

[0055] (3) The present invention significantly improves the processing speed of the model on edge devices by deploying the model on the NVIDIA Jetson Orin NX platform and using TensorRT for acceleration. This enables the model to achieve real-time detection in a resource-constrained environment, reducing latency and improving the response speed of the overall system.

[0056] (4) The detection system of the present invention can automatically identify the quality of sugarcane seeds, reduce the need for manual inspection, reduce labor costs and improve work efficiency. By achieving real-time monitoring, the system can promptly discover problems that occur during the planting process, such as uneven seeding of sugarcane seeds or improper planting density, so that measures can be quickly taken to make adjustments and improve the overall quality of sugarcane planting. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flow chart of the real-time detection method of sugarcane planting quality based on deep learning according to the present invention.

[0058] Figure 2 The present invention uses a depth camera to collect morphological images of sugarcane seeds of different planting in the field under different lighting and soil conditions.

[0059] Figure 3 This is a schematic diagram of using X-Anylabeling software to calibrate the OBB rotation frame of the collected sugarcane seed images.

[0060] Figure 4 It is the design diagram of the DWR module of the present invention.

[0061] Figure 5 It is a schematic diagram of the network model structure in which the DWR module is embedded in YOLOV8-OBB in the present invention.

[0062] Figure 6 It is a schematic diagram of the effect of detecting and positioning the planting machine for sugarcane seeds planting quality of the present invention.

[0063] Figure 7 This is a schematic diagram of the actual field detection effect test. DETAILED DESCRIPTION

[0064] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0065] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0066] like Figures 1 to 7 As shown, a method and system for real-time detection of sugarcane planting quality based on deep learning according to a specific embodiment of the present invention comprises the following steps:

[0067] Step 1: Calibrate the internal and external parameters of the depth camera to calculate the position and posture of the object in the world coordinate system to obtain the relative position information of the depth camera. Specifically include:

[0068] (1) Select a calibration plate with known geometric features, such as a chessboard, a circular array, etc.

[0069] (2) Take multiple images containing the calibration plate at different positions and angles to ensure that the entire effective field of view is covered.

[0070] (3) Use image processing algorithms (such as Harris corner detection, Shi-Tomasi corner detection, etc.). Corner detection can use the findChessboardCorners function in the OpenCV library or other applicable methods.

[0071] (4) Based on the detected corner point positions and the known world coordinates (i.e., the actual positions of the corner points on the calibration pattern), the camera's intrinsic parameter matrix is ​​solved using the least squares method or other optimization algorithms. The intrinsic parameters include focal length (f_x, f_y), principal point (C_x, c_y), radial distortion coefficients (k_1, k_2, ...) and tangential distortion coefficients (p_1, p_2).

[0072] Extract the feature points on the calibration plate (such as the corner points of the chessboard) from the acquired depth image, and use the extracted feature point coordinates and the actual size of the calibration plate to solve the camera's intrinsic parameter matrix K through a nonlinear optimization algorithm (such as the calibrateCamera function in OpenCV). The matrix includes the focal length f x and f y , principal point position c x and c y , the offset s between pixels.

[0073] (5) Using the extracted feature point coordinates and the actual size of the calibration plate, the camera's extrinsic parameter matrix is ​​solved through a nonlinear optimization algorithm. The extrinsic parameter matrix consists of a rotation matrix and a translation vector. Once the extrinsic parameters are obtained, the depth information obtained by the depth camera can be converted into position information in three-dimensional space.

[0074] Define the coordinates of each corner point on the chessboard in the world coordinate system as the world coordinates of the calibration pattern; match the corner point positions in the image with the corner point positions in the world coordinate system; use the known intrinsic parameter matrix, image coordinates and world coordinates to calculate the rotation and translation matrices by solving the perspective n-point problem (PnP), thereby obtaining the extrinsic parameters of the camera.

[0075] The internal parameter matrix is:

[0076]

[0077] Where f x is the horizontal focal length, f y is the vertical focal length, c x is the x-coordinate position in the image, c y is the y coordinate position in the image.

[0078] Distortion coefficient:

[0079] r 2 =x2 +y 2

[0080] x distorted =x(1+k1r 2 +k2r 4 +…)+2p1xy+p2(r 2 +2x 2 )

[0081] y distorted =y(1+k1r 2 +k2r 4 +…)+2p2xy+p1(r 2 +2y 2 )

[0082] Where r 2 is the square of the radius, that is, the square of the distance from the point to the center of the image, k1, k2, k3 are radial distortion coefficients, which are used to describe the radial distortion that gradually increases from the center to the edge of the image, and p1, p2 are tangential distortion coefficients, which are used to describe the tangential distortion caused by the camera sensor not being parallel to the lens optical axis.

[0083] Step 2: Use a depth camera to collect images of sugarcane seeds after the sugarcane seeding machine drops the seeds, and pre-process the collected images;

[0084] Reference Figure 2 and Figure 3 , use the depth camera to collect the sugarcane seed image after the sugarcane planter drops the seeds, pre-process the collected image to obtain the processed image, and use the X-Anylabeling software to calibrate the OBB rotation frame of the collected image. The OBB is usually defined by the center point position, width, height and rotation angle, which can be expressed as (x c ,y c , w, h, θ) to represent, (x c ,y c ) are the coordinates of the center point of the rectangle, w is the width of the rectangle, h is the height of the rectangle, and θ is the rotation angle of the rectangle relative to the horizontal axis.

[0085] Step 3: Improve the YOLOV8-OBB network model by introducing the fused expandable residual DWR attention mechanism to obtain the improved YOLOV8-OBB network model.

[0086] Methods for improving the YOLOV8-OBB network model include:

[0087] (1) Improve the YOLOv8-OBB model by introducing the fused expandable residual DWR attention mechanism and the attention weight w i The calculation formula is as follows:

[0088]

[0089] Among them, q i and k i They are query vector and key vector, q i and k i is calculated from the input feature map and used to determine the attention weight. k is the dimension of the key vector, divided by This is to scale the dot product value to prevent the softmax function from saturating. i represents the attention weight of the i-th position, which determines the importance of the position. The similarity between the query vector and the key vector is calculated. The larger the dot product, the higher the correlation. The attention weight of each position is calculated by the softmax function, so that the sum of all attention weights is 1, so that the value vector can be weighted and summed according to the weight.

[0090] (2) The DWR attention mechanism enhances the model’s ability to capture seed morphological features by weighting important areas in the input image.

[0091] (3) Add the DWR module based on YOLOv8-OBB. This module uses an expandable residual structure to learn the complex morphological features of fallen seeds, which improves the robustness and accuracy of the model. By adjusting the network architecture of YOLOv8-OBB and integrating the DWR module, the effective extraction and detection of fallen seed morphological features are achieved. The improved model shows higher accuracy and faster speed in the task of detecting fallen seed morphological features, and is suitable for real-time monitoring of sugarcane planting quality.

[0092] Reference Figure 4 and Figure 5 The improved YOLOv8-OBB model further improves the detection performance of the model by introducing the fused expandable residual DWR attention mechanism to replace the original C2F structure. This structure innovatively decomposes the original single-step feature processing into two steps: regional residualization and semantic residualization.

[0093] Specifically, regional residualization focuses on the differences in local features under different receptive fields, while semantic residualization aims to improve the model's ability to extract global semantic information. Through these two steps, the model can capture multi-scale contextual information more accurately. In the fusion process, multiple outputs are first aggregated layer by layer. After all feature maps are connected, batch normalization (BN) is performed to reduce the offset problem of feature distribution in different layers. Subsequently, the features are integrated through point-wise convolution to generate the final residual information. This residual can not only effectively fuse multi-scale features, but also improve the expressiveness of features at a lower computational cost. Finally, the model adds the residual to the input features to form a more expressive feature representation. In terms of lightweight design, this improved YOLOv8-OBB model adopts an optimized YOLO backbone network, which significantly reduces the model's memory usage by reducing the number of parameters and calculations, while maintaining high detection accuracy and speed. The specific lightweight strategy includes the simplification of convolutional layers and feature extraction modules, as well as the application of weight sharing technology, which further reduces redundant calculations. In addition, by using multi-scale feature fusion technology, the model can effectively fuse feature information from different scales, enhance the detection ability of small targets and complex scenes, and make up for the accuracy loss that may be caused by the lightweight process. The improved YOLOv8-OBB model has significant advantages in multi-scale feature extraction and context information capture, especially in complex agricultural scenes such as sugarcane planting. It has the characteristics of high recognition rate, fast recognition speed, and strong feature extraction ability, and can complete target detection tasks more accurately and efficiently.

[0094] Step 4: Train the improved YOLOV8-OBB model based on the preprocessed sugarcane seed images to obtain the sugarcane seed quality target detection model.

[0095] A convolutional neural network is used to iteratively train the calibrated training set images and validation set images to obtain the optimal weight file and the seed quality target detection model. The network model is evaluated at the same time. The indicators used to evaluate the network model include accuracy, recall rate and mean average precision, among which the accuracy of the network model is required to be ≥90%.

[0096] The method for obtaining the seed morphological feature recognition network model includes:

[0097] (1) Use the preprocessed image dataset to train the improved YOLOv8-OBB model;

[0098] (2) The dataset contains images of sugarcane seeds with uniform and disordered seeding, as well as corresponding label information;

[0099] (3) Use cross-validation methods to evaluate model performance and optimize the model training process by adjusting hyperparameters;

[0100] (4) Use the loss function to measure the difference between the model prediction results and the true label, and update the model weights through the back-propagation algorithm to minimize the loss;

[0101] (5) Finally, a trained sugarcane seed quality target detection model is obtained.

[0102] The present invention uses accuracy (Precision), recall rate (Recal1), and mean average precision (meanAveragePrecision, mAP) as evaluation indicators.

[0103] Precision is the detection accuracy, which means the ratio of the number of correctly predicted samples to the actual number of correct samples. TP is the number of correctly detected samples, and FP is the number of falsely detected samples.

[0104]

[0105] Recall represents the recall rate, which is the ratio of the number of correct samples predicted by the model to the total number of samples, where TP represents the number of positive samples predicted as positive samples. , FN represents the number of positive samples predicted as negative samples.

[0106]

[0107] mAP stands for mean average precision, which is the sum of the average precision of all categories divided by the average precision of all categories in the data set. The computer formula is shown in formula (9). The value of AP is equal to the area of ​​the curve enclosed by Precision and Recal1.

[0108]

[0109] Step 5: Deploy the sugarcane seed quality target detection model on the AI ​​computing platform NVIDIA JETSON ORIN NX, and run the target detection model to perform real-time detection of the seed quality of the sugarcane planter.

[0110] The specific steps include:

[0111] (1) Convert the trained sugarcane seed quality target detection model into a format suitable for the NVIDIA JETSON ORIN NX platform;

[0112] (2) Use the TensorRT toolkit to optimize the model and generate a TensorRT engine file for efficient execution;

[0113] (3) Deploy the TensorRT optimized model on the NVIDIA JETSON ORIN NX platform and configure the necessary operating environment;

[0114] (4) Test the performance of the deployed model in real-time detection of seed placement quality of sugarcane planters;

[0115] (5) Adjust the model configuration to adapt to different lighting and environmental conditions to ensure the accuracy and stability of the detection results.

[0116] Reference Figure 7 , the improved YOLOv8-OBB model was deployed on the edge computing platform NVIDIA Jetson OrinNX to detect the seeding quality of sugarcane planters in real time. NVIDIA Jetson Orin NX is an embedded platform designed for edge AI computing. It has the characteristics of high performance and low power consumption, and is particularly suitable for running complex deep learning models in resource-constrained environments. The platform integrates an efficient GPU architecture and AI accelerator, which can significantly reduce power consumption while ensuring high computing power, making it an ideal choice for agriculture, automation and other fields.

[0117] A real-time detection system for sugarcane planting quality based on deep learning, comprising: an identification unit, which includes the above-mentioned sugarcane seed quality target detection model; an image recognition device, which includes the above-mentioned depth camera and obtains image information in real time; a working unit, which is a planting machine; and a control unit, which is data-connected with the identification unit, the image recognition device and the working unit, the control unit receives data from the image recognition unit, makes judgments based on the data from the identification unit, and controls the operation of the working unit.

[0118] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A real-time detection method for sugarcane planting quality based on deep learning, characterized in that: The following steps are involved: (1) Calibrate the internal and external parameters of the depth camera and calculate the position and posture of the object in the world coordinate system to obtain the relative position information of the depth camera; (2) Using a depth camera to collect images of sugarcane seeds after the sugarcane seeding machine drops the seeds, and preprocessing the collected images; (3) Improve the YOLOV8-OBB network model and introduce the fused expandable residual DWR attention mechanism to obtain the improved YOLOV8-OBB network model; (4) The improved YOLOV8-OBB model is trained based on the preprocessed sugarcane seed images to obtain a sugarcane seed quality target detection model; (5) The sugarcane seed quality target detection model is deployed on the AI ​​computing platform NVIDIA JETSON ORIN NX, and the target detection model is run to perform real-time detection of the seed quality of the sugarcane planter.

2. The real-time detection method for sugarcane planting quality based on deep learning according to claim 1 is characterized in that: Step (1) calibrates the internal and external parameters of the depth camera as follows: use a standard calibration plate to calibrate the internal and external parameters of the depth camera, the internal parameters include focal length, principal point position, distortion parameters, and the external parameters are the rotation and translation matrices of the camera relative to the world coordinates.

3. The real-time detection method for sugarcane planting quality based on deep learning according to claim 2 is characterized in that: The method of calibrating the internal and external parameters of the depth camera in step (1) includes: 1) Prepare a standard black and white checkerboard pattern as a calibration plate; 2) Use a depth camera to capture images of the checkerboard pattern from different angles and distances; 3) Automatically detect the positions of checkerboard corner points on each image; 4) According to the detected corner point positions and the known world coordinates, the camera's intrinsic parameter matrix is ​​solved using the least squares method or other optimization algorithms. The intrinsic parameters include focal length (f_x, f_y), principal point (c_x, c_y), radial distortion coefficient (k_1, k_2, ...) and tangential distortion coefficient (p_1, p_2); 5) Define the coordinates of each corner point on the chessboard in the world coordinate system as the world coordinates of the calibration pattern; 6) Match the corner point positions in the image with the corner point positions in the world coordinate system; 7) Using the known intrinsic parameter matrix, image coordinates and world coordinates, the rotation and translation matrices are calculated by solving the perspective n-point problem to obtain the extrinsic parameters of the camera. The internal parameter matrix is: Where f x is the horizontal focal length, f y is the vertical focal length, c x is the x-coordinate position in the image, c y is the y coordinate position in the image. Distortion coefficient: r 2 ＝r 2 +y 2 x distorted =x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 ) <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> distorted <h2 style=";text-align:left;direction:ltr"> =y(1+k1r<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +k2r<h2 style=";text-align:left;direction:ltr"> 4 <h2 style=";text-align:left;direction:ltr"> +…)+2p2xy+p1(r<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +2y<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> ) Where r 2 is the square of the radius, that is, the square of the distance from the point to the center of the image, k1, k2, k3 are radial distortion coefficients, which are used to describe the radial distortion that gradually increases from the center to the edge of the image, and p1, p2 are tangential distortion coefficients, which are used to describe the tangential distortion caused by the camera sensor not being parallel to the lens optical axis.

4. The real-time detection method for sugarcane planting quality based on deep learning according to claim 1 is characterized in that: Step (2) preprocesses the image by performing enhanced preprocessing on the collected image and using X-AnyLabeling to mark the sugarcane species in the image with a rotating box.

5. The method for real-time detection of sugarcane planting quality based on deep learning according to claim 4, characterized in that: Step (2) image enhancement preprocessing is to use the Mosaic data enhancement method to splice the image, enhance the background information in the image, and obtain an enhanced image; The rotation frame annotation is to use annotation software to annotate the sugarcane species in the image with an OBB frame with an angle, and one OBB frame can be marked with (x c ,y c , w, h, θ) to represent, (x c ,y c ) are the coordinates of the center point of the rectangle, w is the width of the rectangle, h is the height of the rectangle, and θ is the rotation angle of the rectangle relative to the horizontal axis.

6. The real-time detection method for sugarcane planting quality based on deep learning according to claim 1, characterized in that: Step (2) The depth camera collects image information of sugarcane seeds, including images of sugarcane seeds falling to the ground and background information. The background includes various landforms. The collected types of fallen seeds include images of sugarcane seeds with uniform planting and images of sugarcane seeds with disordered planting.

7. The real-time detection method for sugarcane planting quality based on deep learning according to claim 1, characterized in that: Step (3) The method for improving the YOLOV8-OBB network model includes: 1) Improve the YOLOv8-OBB model by introducing the fused expandable residual DWR attention mechanism and the attention weight w i The calculation formula is as follows: Among them, q i and k i are the query vector and the key vector, q i and k i is calculated from the input feature map and used to determine the attention weight; d k is the dimension of the key vector, divided by It is to scale the value of the dot product to prevent the softmax function from saturating; w i Represents the attention weight of the i-th position, through the dot product operation The similarity between the query vector and the key vector is calculated, and the attention weight of each position is calculated through the softmax function so that the sum of all attention weights is 1. The value vector is weighted and summed according to the weight; 2) The DWR attention mechanism weights the important areas in the input image; 3) Add the DWR module based on YOLOv8-OBB.

8. The method for real-time detection of sugarcane planting quality based on deep learning according to claim 1, characterized in that: The method of obtaining the seed morphological feature recognition network model in step (4) includes: 1) Use the preprocessed image dataset to train the improved YOLOv8-OBB model; 2) The dataset contains images of sugarcane seeds with uniform and disordered seeding, as well as corresponding label information; 3) Use cross-validation methods to evaluate the performance of the model and optimize the model training process by adjusting hyperparameters; 4) Use the loss function to measure the difference between the model prediction results and the true label, and update the model weights through the back-propagation algorithm to minimize the loss; 5) Finally, a trained sugarcane seed quality target detection model is obtained.

9. The method for real-time detection of sugarcane planting quality based on deep learning according to claim 1, characterized in that: Step (5) comprises: 1) Convert the trained sugarcane seed quality target detection model into a format suitable for the NVIDIA JETSON ORIN NX platform; 2) Use the TensorRT toolkit to optimize the model and generate a TensorRT engine file for efficient execution; 3) Deploy the TensorRT optimized model on the NVIDIA JETSON ORIN NX platform and configure the necessary operating environment; 4) Test the performance of the deployed model in real-time detection of seed placement quality of sugarcane planters; 5) Adjust the model configuration to adapt to different lighting and environmental conditions to ensure the accuracy and stability of the detection results.

10. A real-time detection system for sugarcane planting quality based on deep learning, characterized in that: include: An identification unit, comprising the sugarcane seed quality target detection model according to claim 1; An image recognition device, comprising the depth camera of claim 1, for acquiring image information in real time; A working unit, which is a planting machine; as well as The control unit is data-connected with the recognition unit, the image recognition device and the working unit. The control unit receives the data of the image recognition unit, makes judgments based on the data of the recognition unit, and controls the work of the working unit.

Images