A device and method for detecting corn inter-row diseases based on deep learning

Abstract

The present invention discloses a device and method for detecting corn interrow pests based on deep learning, belonging to the technical field of target detection and image segmentation methods. The present invention specifically comprises the following steps: capturing corn leaf images with a camera, annotating diseased areas in the images, and training to obtain a weight file; loading the weight file into a deep learning algorithm to obtain a target detection network and an image segmentation network; capturing omnidirectional video images of corn interrows using a camera; sending the video to a Jeston nano edge server, running the target detection network and image segmentation network on the Jeston nano edge server to obtain pest and disease areas in the image; transmitting the results to a mobile application, which allows real-time monitoring of corn interrow disease conditions, obtaining disease grades, and determining appropriate pesticide concentrations. The present invention solves the backward manual prevention methods used in traditional agricultural production, saves labor costs, and improves work efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and image segmentation methods, and in particular to a device and method for detecting three-dimensional diseases between corn rows based on deep learning. Background Art

[0002] Diseases are a significant factor affecting crop growth, and disease detection has always been a crucial task in agricultural production. Traditionally, disease detection in agricultural production has relied on manual labor. However, with the development of modern agriculture, this manual approach is no longer suitable for modern agriculture, resulting in long processing times, high labor costs, and low efficiency. However, intelligent detection systems based on artificial intelligence (AI) avoid the drawbacks of manual detection and play a crucial role in modern agricultural production management.

[0003] Compared to deep learning image classification, object detection not only locates the specific coordinates of an object in an image but also identifies its class. Image segmentation can accurately segment the specific shape of a disease, revealing its details. Therefore, the application of object detection and image segmentation in agricultural disease detection facilitates the precise location and segmentation of diseased areas, thereby improving agricultural productivity. For these reasons, the present invention proposes a deep learning-based device and method for corn inter-row three-dimensional disease detection. Summary of the Invention

[0004] The purpose of the present invention is to solve the backward manual prevention methods in traditional agricultural production, save labor costs, and improve work efficiency, and propose a deep learning-based three-dimensional corn inter-row disease detection device and method.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A deep learning-based three-dimensional disease detection device for corn rows includes a loading platform and a camera device, the loading platform includes a first frame and a second frame, a GNSS antenna is fixedly installed at the center of the upper surface of the first frame, a battery and a camera device are provided on both sides of the GNSS antenna, the battery and the camera device are both fixedly installed on the upper surface of the first frame, the camera device includes an upper camera, a left camera, and a right camera, and a Jetson Nano edge server is also fixedly installed on one side of the camera device; the second frame is fixedly connected to the bottom end of the first frame, the front end of the second frame is fixedly connected to a laser radar, the four corners of the bottom surface of the second frame are fixedly connected to straight rods, the bottom end of the straight rods is fixedly installed with a steering motor, the bottom end of the steering motor is fixedly connected to a connecting bearing, a DC motor is fixedly connected to the connecting bearing, and a wheel is fixedly connected to the output shaft of the DC motor.

[0007] A method for detecting corn inter-row diseases based on deep learning specifically includes the following steps:

[0008] S1. Use a camera to take pictures of corn;

[0009] S2, label the corn pictures taken in S1 and divide them into training set and test set;

[0010] S3. Input the training set and test set obtained in S2 into the target detection network to obtain the target detection weight file;

[0011] S4, using the weight file of the target detection network obtained in S3 to train the weight file of the image segmentation network;

[0012] S5. Load the trained image segmentation network weight file into the deep learning algorithm for video detection;

[0013] S6. Obtain all-round and all-angle video image information between corn rows through the upper, left, and right cameras of the camera device, send the video image information to three Jetson Nano edge servers, and run a deep learning algorithm on the Jetson Nano edge servers to perform disease detection on the video image information;

[0014] S7. The detection results obtained in S6 are transmitted to the corresponding mobile application, the detection results are analyzed, the disease level is classified, and the required pesticide concentration for spraying is determined.

[0015] Preferably, when taking pictures of corn mentioned in S1, in order to better adapt to the real environment, the following requirements are specifically included:

[0016] A1. Collect image information under different lighting conditions, including overexposure, normal, and dim;

[0017] A2. The pictures taken must include all parts of the corn;

[0018] A3. The proportion of healthy and diseased people in the pictures is balanced.

[0019] Preferably, the image annotation mentioned in S2 specifically includes the following steps:

[0020] B1. Use image enhancement methods to perform operations such as cropping, flipping, and mirroring on the training set to expand the data;

[0021] B2. Use LabelImg software to perform data annotation, including coarse-grained target box annotation and fine-grained image segmentation pixel annotation. The target box annotation is used to mark the diseased area and generate an .xml file. The .xml file contains: category, horizontal coordinate center of the labeled object, vertical coordinate center of the labeled object, width, and height. The fine-grained image pixel annotation is used to mark the target box generated by target detection and generate a .json file.

[0022] B3. Divide the images into training set, validation set and test set with a specific ratio of 7:2:1.

[0023] Preferably, the step of obtaining the target detection weight file mentioned in S3 specifically includes the following steps:

[0024] C1. Put the training set images into the target detection algorithm in batches for training, and obtain the training weight file and the change curve of each parameter during the training process;

[0025] C2. Improve the model convergence speed by adjusting hyperparameters such as learning rate and batch size during training;

[0026] C3. Put the weight file into the target detection algorithm to detect the test set and observe the detection results;

[0027] C4. If the weight file fails the test, re-create and expand the training set, and repeat C1 to C3 operations until the weight file passes the test.

[0028] Preferably, the training of the image segmentation network weight file mentioned in S4 specifically includes the following steps:

[0029] D1. Pass the training set through the target detection network to obtain the target frame of the diseased area;

[0030] D2. Send the target box obtained in D1 to the image segmentation network for training to obtain the training weight file and the change curve of each parameter during the training process;

[0031] D3. Observe whether the change curve obtained in D2 conforms to the training rules and whether the training accuracy meets the requirements;

[0032] D4. Put the weight file into the image segmentation network to test the test set data and observe the detection results;

[0033] D5. If the weight file fails the test, re-create and expand the training set, and repeat D1 to D4 until the weight file passes the test.

[0034] Preferably, the weight file loading of the image segmentation network mentioned in S5 specifically includes the following steps:

[0035] E1. Install the Linux system on the Jeston nano edge server and configure the environment required for the deep learning algorithm;

[0036] E2. Import the weight file and deep learning algorithm into the Jeston nano edge server and control the operation of the algorithm through remote services.

[0037] Preferably, the video image information acquisition and disease detection mentioned in S6 specifically include the following steps:

[0038] F1. Use the high-definition camera on the detection vehicle to capture corn video. The camera is connected to the Jeston nano edge server via a USB interface, enabling the Jeston nano edge server to control the camera and transmit the video back to the Jeston nano edge server.

[0039] F2: Run the object detection network and image segmentation network on the Jeston nano edge server to detect the diseased area in the video.

[0040] Compared with the existing technology, the present invention provides a device and method for detecting corn inter-row diseases based on deep learning, which has the following beneficial effects:

[0041] This invention can automatically detect and identify disease types, meticulously segmenting specific disease areas. It provides an integrated service from data acquisition to disease detection, and finally visualization and analysis of results on an Android app. Based on a deep learning algorithm and implemented using a deep learning framework, this invention is easily scalable and applicable to users. It addresses the outdated manual prevention methods used in traditional agricultural production, saving labor costs and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a schematic diagram of the corn inter-row disease recognition process of a corn inter-row three-dimensional disease detection method based on deep learning proposed by the present invention;

[0043] Figure 2 This is a schematic diagram of the corn disease detection process of a deep learning-based corn inter-row three-dimensional disease detection method proposed by the present invention;

[0044] Figure 3 This is a schematic diagram of the mobile application interface of the deep learning-based corn inter-row disease detection method proposed in the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of a deep learning-based corn inter-row disease detection device proposed by the present invention;

[0046] Figure 5 This is a front view of a deep learning-based corn inter-row disease detection device proposed by the present invention;

[0047] Figure 6 This is a top view of the deep learning-based corn inter-row three-dimensional disease detection device proposed by the present invention. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0049] Example 1:

[0050] See also Figure 1-6 , a corn row three-dimensional disease detection device based on deep learning, including a loading platform 3 and a camera device 9, the loading platform 3 includes a first frame 31 and a second frame 32, a GNSS antenna 2 is fixedly installed at the center of the upper surface of the first frame 31, a battery 1 and a camera device 9 are provided on both sides of the GNSS antenna 2, the battery 1 and the camera device 9 are both fixedly installed on the upper surface of the first frame 31, the camera device 9 includes an upper camera 91, a left camera 92, and a right camera 93, and a jetson nano edge server 10 is also fixedly installed on one side of the camera device 9; the second frame 32 is fixedly connected to the bottom end of the first frame 31, the front end of the second frame 32 is fixedly connected to the laser radar 4, the four corners of the bottom surface of the second frame 32 are fixedly connected to a straight rod 8, the bottom end of the straight rod 8 is fixedly installed with a steering motor 7, the bottom end of the steering motor 7 is fixedly connected to a connecting bearing, the connecting bearing is fixedly connected to a DC motor, and the output shaft of the DC motor is fixedly connected to a wheel 6.

[0051] A method for detecting corn inter-row diseases based on deep learning specifically includes the following steps:

[0052] S1, use the camera device 9 to take pictures of corn;

[0053] When taking pictures of corn mentioned in S1, in order to better adapt to the real environment, the following requirements are included:

[0054] A1. Collect image information under different lighting conditions, including overexposure, normal, and dim;

[0055] A2. The pictures taken must include all parts of the corn;

[0056] A3. The proportion of healthy and diseased people in the pictures is balanced;

[0057] S2, label the corn pictures taken in S1 and divide them into training set and test set;

[0058] The image annotation mentioned in S2 specifically includes the following steps:

[0059] B1. Use image enhancement methods to perform operations such as cropping, flipping, and mirroring on the training set to expand the data;

[0060] B2. Use LabelImg software to perform data annotation, including coarse-grained target box annotation and fine-grained image segmentation pixel annotation. Target box annotation is used to label the diseased area and generate an .xml file. The .xml file contains: category, horizontal coordinate center of the labeled object, vertical coordinate center of the labeled object, width, and height. Fine-grained image pixel annotation is used to label the target box generated by target detection and generate a .json file.

[0061] B3. Divide the images into training set, validation set and test set with a specific ratio of 7:2:1;

[0062] S3. Input the training set and test set obtained in S2 into the target detection network to obtain the target detection weight file;

[0063] Obtaining the target detection weight file mentioned in S3 specifically includes the following steps:

[0064] C1. Put the training set images into the target detection algorithm in batches for training, and obtain the training weight file and the change curve of each parameter during the training process;

[0065] C2. Improve the model convergence speed by adjusting hyperparameters such as learning rate and batch size during training;

[0066] C3. Put the weight file into the target detection algorithm to detect the test set and observe the detection results;

[0067] C4. If the weight file fails the test, re-create and expand the training set, and repeat C1 to C3 until the weight file passes the test;

[0068] S4, using the weight file of the target detection network obtained in S3 to train the weight file of the image segmentation network;

[0069] The training of the image segmentation network weight file mentioned in S4 specifically includes the following steps:

[0070] D1. Pass the training set through the target detection network to obtain the target frame of the diseased area;

[0071] D2. Send the target box obtained in D1 to the image segmentation network for training to obtain the training weight file and the change curve of each parameter during the training process;

[0072] D3. Observe whether the change curve obtained in D2 conforms to the training rules and whether the training accuracy meets the requirements;

[0073] D4. Put the weight file into the image segmentation network to test the test set data and observe the detection results;

[0074] D5. If the weight file fails the test, recreate and expand the training set, and repeat D1 to D4 until the weight file passes the test;

[0075] S5. Load the trained image segmentation network weight file into the deep learning algorithm for video detection;

[0076] The weight file of the image segmentation network mentioned in S5 is loaded, which specifically includes the following steps:

[0077] E1. Install the Linux system on the Jeston nano edge server and configure the environment required for the deep learning algorithm;

[0078] E2, import the weight file and deep learning algorithm into the Jeston nano edge server and control the operation of the algorithm through remote services;

[0079] S6. Obtain all-round and all-angle video image information between corn rows through the upper camera 91, left camera 92, and right camera 93 of the camera device 9, send the video image information to three Jetson Nano edge servers 10, and run a deep learning algorithm on the Jetson Nano edge servers 10 to perform disease detection on the video image information;

[0080] The video image information acquisition and disease detection mentioned in S6 specifically include the following steps:

[0081] F1. Use the high-definition camera on the detection vehicle to capture corn video. The camera is connected to the Jeston nano edge server via a USB interface, enabling the Jeston nano edge server to control the camera and transmit the video back to the Jeston nano edge server.

[0082] F2: Run the object detection network and image segmentation network on the Jeston nano edge server to detect the diseased area in the video;

[0083] S7. The detection results obtained in S6 are transmitted to the corresponding mobile application, the detection results are analyzed, the disease level is classified, and the required pesticide concentration for spraying is determined.

[0084] This invention can automatically detect and identify disease types, meticulously segmenting specific disease areas. It provides an integrated service from data acquisition to disease detection, and finally visualization and analysis of results on an Android app. Based on a deep learning algorithm and implemented using a deep learning framework, this invention is easily scalable and applicable to users. It addresses the outdated manual prevention methods used in traditional agricultural production, saving labor costs and improving work efficiency.

[0085] The above are only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for detecting corn inter-row diseases based on deep learning, characterized in that: The method is based on a device for detecting three-dimensional diseases between corn rows. The device comprises a loading platform (3) and a camera device (9). The loading platform (3) comprises a first frame (31) and a second frame (32). A GNSS antenna (2) is fixedly mounted at the center of the upper surface of the first frame (31). A battery (1) and a camera device (9) are provided on both sides of the GNSS antenna (2). The battery (1) and the camera device (9) are both fixedly mounted on the upper surface of the first frame (31). The camera device (9) comprises an upper camera (91), a left camera (92), and a right camera (93). A jetson is also fixedly mounted on one side of the camera device (9). nano edge server (10); the second frame (32) is fixedly connected to the bottom end of the first frame (31); the front end of the second frame (32) is fixedly connected to a laser radar (4); the four corners of the bottom surface of the second frame (32) are fixedly connected to straight rods (8); the bottom end of the straight rod (8) is fixedly mounted with a steering motor (7); the bottom end of the steering motor (7) is fixedly connected to a connecting bearing; the connecting bearing is fixedly connected to a DC motor; the output shaft of the DC motor is fixedly connected to a wheel (6); The method specifically comprises the following steps: S1, using a camera device (9) to take pictures of corn; S2, label the corn pictures taken in S1 and divide them into training set and test set; S3. Input the training set and test set obtained in S2 into the target detection network to obtain the target detection weight file, which specifically includes the following steps: C1. Put the training set images into the target detection algorithm in batches for training, and obtain the training weight file and the change curve of each parameter during the training process; C2. Improve the model convergence speed by adjusting hyperparameters such as learning rate and batch size during training; C3. Put the weight file into the target detection algorithm to detect the test set and observe the detection results; C4. If the weight file fails the test, re-create and expand the training set, and repeat C1 to C3 until the weight file passes the test; S4. Use the weight file of the target detection network obtained in S3 to train the weight file of the image segmentation network, specifically including the following steps: D1. Pass the training set through the target detection network to obtain the target frame of the diseased area; D2. Send the target box obtained in D1 to the image segmentation network for training to obtain the training weight file and the change curve of each parameter during the training process; D3. Observe whether the change curve obtained in D2 conforms to the training rules and whether the training accuracy meets the requirements; D4. Put the weight file into the image segmentation network to test the test set data and observe the detection results; D5. If the weight file fails the test, recreate and expand the training set, and repeat D1 to D4 until the weight file passes the test; S5. Load the trained image segmentation network weight file into the deep learning algorithm for video detection; S6, obtaining all-round and all-angle video image information between corn rows through the upper camera (91), the left camera (92), and the right camera (93) of the camera device (9), sending the video image information to three Jetson Nano edge servers (10), running a deep learning algorithm on the Jetson Nano edge servers (10), and performing disease detection on the video image information; S7. The detection results obtained in S6 are transmitted to the corresponding mobile application, the detection results are analyzed, the disease level is classified, and the required pesticide concentration for spraying is determined.

2. The method for detecting corn inter-row diseases based on deep learning according to claim 1, characterized in that: When taking pictures of corn mentioned in S1, in order to better adapt to the real environment, the following requirements are specifically included: A1. Collect image information under different lighting conditions, including overexposure, normal, and dim; A2. The pictures taken must include all parts of the corn; A3. The proportion of healthy and diseased people in the pictures is balanced.

3. The method for detecting corn inter-row diseases based on deep learning according to claim 1, characterized in that: The image annotation mentioned in S2 specifically includes the following steps: B1. Use image enhancement methods to perform operations such as cropping, flipping, and mirroring on the training set to expand the data; B2. Use LabelImg software to perform data annotation, including coarse-grained target box annotation and fine-grained image segmentation pixel annotation. The target box annotation is used to mark the diseased area and generate an .xml file. The .xml file contains: category, horizontal coordinate center of the labeled object, vertical coordinate center of the labeled object, width, and height. The fine-grained image pixel annotation is used to mark the target box generated by target detection and generate a .json file. B3. Divide the images into training set, validation set and test set with a specific ratio of 7:2:

1.

4. The method for detecting corn inter-row diseases based on deep learning according to claim 1, characterized in that: The weight file of the image segmentation network mentioned in S5 is loaded, specifically including the following steps: E1. Install the Linux system on the Jeston nano edge server and configure the environment required for the deep learning algorithm; E2. Import the weight file and deep learning algorithm into the Jeston nano edge server and control the operation of the algorithm through remote services.

5. The method for detecting corn inter-row diseases based on deep learning according to claim 1, characterized in that: The video image information acquisition and disease detection mentioned in S6 specifically include the following steps: F1. Use the high-definition camera on the detection vehicle to shoot corn video. The camera is connected to the Jeston nano edge server through the USB interface, so that the Jeston nano edge server can control the camera and transmit the video back to the Jeston nano edge server. F2. Run the object detection network and image segmentation network on the Jeston nano edge server to detect the diseased area in the video.

Images