Accurate monitoring and identification system for diseases and insect pests

Abstract

The invention provides a pest and disease damage accurate monitoring and recognition system, and relates to the field of agricultural intelligent robots, the pest and disease damage accurate monitoring and recognition system comprises a searchlighting recognition system, a walking mechanism, a mechanical arm assembly, an electric water pump, a central control system, a liquid medicine box, a chassis and a sensing system.The specific recognition control method comprises the steps that 1, addition correction is conducted firstly, then binocular stereoscopic vision calibration is conducted, and a calibration result is obtained; 2, monitoring insect pests of the crops in real time by adopting an improved YOLOv8 algorithm, and identifying corresponding types and numbers of the insect pests; 3, acquiring three-dimensional space coordinates of the insect pest image through the insect pest image; and 4, controlling the dosage of the pesticide according to the three-dimensional space coordinates and the corresponding types and quantity of the insect pests. The system can carry out all-weather and all-directional insect pest recognition, is high in detection precision and high in real-time performance, and effectively improves the accuracy and timeliness of cotton bollworm pest prevention and control.

Description

[0001] The present application is a divisional application of patent application No. 2024105292279, entitled "Identification and control spraying robot for field insect pests and method thereof". TECHNICAL FIELD

[0002] The present application relates to the technical field of agricultural intelligent robots, in particular to a precision monitoring and identification system for plant diseases and insect pests. BACKGROUND

[0003] The cotton bollworm is a kind of agricultural pest that damages soybeans, peanuts, cotton, rice, wheat and other crops. The adult cotton bollworm is 15-20 mm long and 31-40 mm wide, and has strong phototaxis and chemotaxis. It hides in the leaf back, flower crown and other hidden places during the day, and starts to move at dusk, feeding on cotton flower buds, tender leaves and young bolls. The larvae feed on tender leaves, and start to bore into young bolls after two instars. The third and fourth instar larvae mainly damage buds and flowers, causing bud drop. The fifth and sixth instar larvae enter the feeding period and damage green bolls, large buds or flowers. Due to the diurnal habit of the adult cotton bollworm, it is difficult to monitor and detect the pest, and it is not easy to distinguish. It not only wastes a lot of manpower and resources, but also cannot realize real-time and accurate identification, thus slowing down the subsequent prevention and control of the cotton bollworm, causing irreversible damage to cotton or a large outbreak of the pest, resulting in reduced yield and economic benefits of cotton planting.

[0004] In the prior art, the monitoring of cotton bollworm in the field is mainly carried out by human identification and unmanned aerial vehicle identification. Human identification is accurate, but it consumes a lot of manpower (requires staff to be stationed in the field for a long time), has a long cycle, is greatly influenced by human subjectivity, and relies on the experience of staff, which limits its practical application. Unmanned aerial vehicle identification mainly relies on spectrum to identify pests, which can achieve large-area and rapid identification, but has a certain lag (i.e. the unmanned aerial vehicle can only determine the occurrence of pests by the characteristics of the plants, which has caused damage to the cotton plants), and cannot timely and quickly detect the early stage of the cotton bollworm, thus failing to achieve the purpose of accurate and early prevention and control. SUMMARY

[0005] In view of the problems in the prior art, the present application provides a precision monitoring and identification system for plant diseases and insect pests, which can identify pests all-weather and all-round, has high detection accuracy and strong real-time performance, thus avoiding the limitations of human identification and the lag of unmanned aerial vehicle identification, improving the accuracy and timeliness of cotton bollworm prevention and control, and avoiding large-area damage or yield reduction of cotton plants.

[0006] The object of the present application is achieved by the following technical solutions: A precision monitoring and identification system for pests and diseases includes a detection and identification system, a walking mechanism, a robotic arm assembly, an electric water pump, a central control system, a pesticide tank, a chassis, and a sensing system. The walking mechanism is located on the bottom of the chassis and is used to control the movement of the entire monitoring and identification system. A pesticide tank is fixedly installed on the front side of the chassis end face. Detection and identification systems are symmetrically arranged on both sides of the pesticide tank on the chassis end face. Two electric water pumps are symmetrically arranged on the rear side of the pesticide tank on the chassis end face, and each electric water pump is connected to the pesticide tank. Robotic arm assemblies are symmetrically arranged on the side of the chassis end face away from the pesticide tank on the side of the electric water pumps. The central control system is fixedly installed on the chassis end face between the two sets of robotic arm assemblies. The electric water pumps are connected to the bionic nozzle components at the ends of the corresponding robotic arm assemblies. A sensing system is located at the front end of the chassis. The detection and identification system, the walking mechanism, the robotic arm assembly, the electric water pump, and the sensing system are all electrically connected to the central control system.

[0007] Based on further optimization of the above scheme, the searchlight recognition system includes a first depth camera, a camera bracket, a system mounting base, a dust cover, and a first searchlight. The system mounting base is fixedly installed on the end face of the chassis and located on both sides of the liquid tank. Two first depth cameras are evenly arranged vertically on the side of the system mounting base away from the liquid tank, and a dust cover is installed around the outer ring of the first depth camera. The first searchlight is installed on the upper end of the side of the system mounting base away from the liquid tank. The robotic arm assembly includes a rotating base, a support platform, a servo motor, a first robotic arm, a linkage mechanism, a second robotic arm, a control motor, and a bionic nozzle component. The rotating base is rotatably connected to the end face of the chassis, and the support platform is fixedly mounted on the end face of the rotating base. The first robotic arm is rotatably mounted on the end face of the support platform, and the end of the first robotic arm away from the support platform is rotatably connected to the second robotic arm. The rear end of the second robotic arm (i.e., the end where the two sets of robotic arm assemblies are close to each other) is rotatably equipped with a linkage mechanism, and the end of the linkage mechanism away from the second robotic arm is rotatably connected to the rotating shaft of the first robotic arm. The servo motor is fixedly mounted on the support platform and is used to control the rotation of the first robotic arm. The end of the second robotic arm away from its corresponding linkage mechanism (i.e., the end where the two sets of robotic arm assemblies are far apart) is equipped with a bionic nozzle component via a fixed support, and a control motor is mounted on the fixed support corresponding to the bionic nozzle component. The biomimetic nozzle component includes a nozzle mounting plate, a nozzle, multiple sets of flanges, multiple ball cage drive joints, a connecting hose, and a drive wire. Multiple nozzles are evenly arranged on the nozzle mounting plate, and the nozzle mounting plate is connected to the fixed support via multiple sets of flanges spaced apart and connected to multiple ball cage drive joints. All connections to the nozzle mounting plate and the fixed support are flanges (i.e., the ball cage drive joints are located between two sets of flanges). A hose hole is opened on the outer ring of the flange corresponding to the nozzle and on the fixed support. One end of the connecting hose is connected to the corresponding nozzle. The other end passes through the hose holes on each flange and the fixed support, and is connected to the corresponding electric water pump. Four wire holes are opened on the outer ring of the hose holes on the flange, and corresponding wire holes are opened on the fixed support. Four control motors are installed on the fixed support, and four drive wires are corresponding to the control motors. One end of each drive wire is wound around the output shaft of the corresponding control motor, and the other end passes through the corresponding wire holes on the fixed support and each flange, and is fixedly connected to the flange near the nozzle mounting plate. The sensing system is used for road navigation and obstacle recognition for chassis movement, and includes a fixed block, a lidar, a second depth camera, and a second searchlight. The fixed block is fixedly installed at the front end of the chassis bottom and between the walking mechanisms. The lidar, second depth camera, and second searchlight are sequentially installed on the fixed block.

[0008] A method for identifying and controlling field pests, comprising using a spotlight identification system within the monitoring and identification system described above to identify field pests, including: Step 1: First, perform single-target calibration on the two first depth cameras to correct optical distortion during the imaging process. Then, perform binocular stereo vision calibration on the two first depth cameras to obtain the reprojection matrix, pixel distance, and the conversion relationship between real physical distance for binocular correction. Step 2: Based on different times and light intensities, turn on the searchlights in the searchlight recognition system to provide illumination for the corresponding first depth cameras. Use the improved YOLOv8 algorithm to monitor crop pests in real time and identify their corresponding species (e.g., cotton bollworm adults or larvae, other pests harmful to cotton, etc.) and quantities. Step 3: Obtain the two-dimensional coordinates of the pests from the pest images obtained in Step 2 and convert them into three-dimensional spatial coordinates. Step 4: Obtain the three-dimensional spatial coordinates of the intended depth point from Step 3 and convert them into three-dimensional spatial coordinates in the coordinate system of the robotic arm component to control the movement of the corresponding robotic arm component. Based on the pest species and quantities obtained in Step 2, control the electric water pump to spray the corresponding amount of pesticide.

[0009] Based on further optimization of the above scheme, the specific steps of performing single-target calibration on the two first depth cameras in step one to correct optical distortion during the depth camera imaging process are as follows: First, using a chessboard calibration board whose corner points are known in the world coordinate system, multiple images of the chessboard calibration board are taken using cameras, ensuring that the chessboard calibration board is at different angles and positions; then, a corner detection algorithm (e.g., the function cv2.findChessboardCorners in the OpenCV library for detecting chessboard corner points) is used to find the corner points of the chessboard calibration board in each image; finally, using the correspondence between feature points in the image and the world coordinate system, the Zhang Zhengyou calibration method (e.g., the function cv2.calibrateCamera in the OpenCV library for implementing the Zhang Zhengyou calibration method) is used to calculate the intrinsic parameter matrix and distortion coefficients of each first depth camera, where the intrinsic parameter matrix includes the principal point coordinates. （c x ,c y ） With focal length f x 、f y The distortion coefficients include the radial distortion coefficients. k 1 、 k 2 、k 3 and tangential distortion coefficient p 1 、p 2.

[0010] Based on further optimization of the above scheme, the step one of performing binocular stereo vision calibration on the two first depth cameras to obtain the reprojection matrix for binocular correction and the conversion relationship between pixel distance and real physical distance is specifically as follows: First, obtain a distorted image point through the intrinsic parameter matrix. （x d ,y d ） Normalized coordinate points （x n ,y n ） : ; Then, the distortion coefficients are used to obtain the distorted coordinates. （x c ,y c ） :

[0011] in: ; Finally, the normalized coordinates are transformed back to image coordinates. （x u ,y u ） :

[0012] Based on further optimization of the above scheme, the specific method for real-time monitoring of crop pests using the improved YOLOv8 algorithm in step two is as follows: First, the input pest image is preprocessed (e.g., resized, cropped) to adapt to the input size requirements of the model; then, basic features of the image (e.g., edges, textures, shapes, etc.) are identified through a series of convolutional layers (Conv), instance normalization, and activation functions (ELU) of the backbone network; next, the input image is processed by the C2f module to convert it into a series of features; then, the extracted features are further processed by the neck network; finally, the head detection network is used to detect targets in the image; specifically: ; In the formula: X Indicates the input image; F This represents the feature extraction function (composed of Backbone and Neck). PY represents the prediction function (i.e., the Head part); Y represents the model output, including the coordinates, size, confidence score, and class probability of the bounding box. Throughout the training process, the improved YOLOv8 algorithm uses a loss function to guide the learning process, specifically:

[0013] In the formula: This represents the model's predicted value. Y t Indicates the true label; This represents the total loss function of the model. , These represent the confidence loss for objects with and without targets, respectively. Indicates category loss; Indicates the bounding box loss; These represent the hyperparameters that balance different loss terms.

[0014] Based on further optimization of the above scheme, the specific method for identifying the corresponding pest types and quantities in step two is as follows: First, collect an image dataset containing two pests (such as adult and larval cotton bollworms, or pest A and pest B, etc.), denoted as... ,in, I i This process involves representing a single image and dividing the image dataset into training, validation, and test sets. A bounding box and category label are then added to each pest using an annotation tool. Next, an improved YOLOv8 model is trained using the labeled image dataset (i.e., the training set). During training, the YOLOv8 model learns how to detect and classify different pests. Finally, the trained YOLOv8 model is used for object detection, obtaining the bounding boxes and category predictions for each pest. For each detected object, the YOLOv8 model outputs a detection vector. ,in, Indicates the coordinates and dimensions of the bounding box. s The confidence score is represented by the bounding boxes. Then, non-maximum suppression is applied to the detected bounding boxes to remove overlapping bounding boxes and retain the highest confidence bounding boxes. Subsequently, the detection structure of the YOLOv8 model is used as input, and candidate regions between frames are associated with DeepSORT's motion information and appearance features to form a stable pest trajectory. t The set of successfully tracked trajectories is as follows: ; in, Indicates the first j The bounding box and category information of the trajectory within the time window; M t Indicates the number of currently active trajectories; ; In the formula: Indicates a point in time In the corresponding frame, the first k The coordinates of the detected bounding boxes; Indicates a point in time In the corresponding frame, corresponding to the bounding box Category confidence, This indicates the current time point, which is either the rightmost or latest time point in the time window being discussed. Indicates the length of the time window; Finally, for each trajectory Based on the category confidence of all the frames it contains C x By setting confidence thresholds for each pest category C d Determine the type of pest: like If the pests in the trajectory are in the corresponding category, then the pests belong to that category; otherwise, they do not. Analyze the categories of each trajectory and calculate the pests for each category separately. C a 、C b quantity N a 、N b : ; In the formula: This indicates an indicator function that takes the value 1 when the condition is true and 0 otherwise.

[0015] Further optimization of the above scheme, specifically the non-maximum suppression of the detected bounding boxes to remove overlapping bounding boxes and retain the bounding boxes with the highest confidence scores, involves: classifying all detected bounding boxes according to their confidence scores. s Sort from highest to lowest, if the bounding box is B i Then its corresponding confidence score is s i Select the bounding box with the highest confidence level. B h and their corresponding confidence scores s h Calculate separately B h With each subsequent bounding box B j ( j>1 intersection-union ratio IoU(Bh , B j ) : ; Preset crossover ratio threshold IoU d ， like Then the bounding box B j and B h If there is overlap, remove the overlapping bounding boxes, update the removed bounding boxes, and sort them; otherwise, retain them and perform the next bounding box verification until there are no more bounding boxes to process.

[0016] Based on further optimization of the above scheme, the specific method for converting the two-dimensional coordinates of the pest into three-dimensional spatial coordinates in step three is as follows: First, obtain the two-dimensional coordinates of the center point of the pest detection box in the pixel coordinate system. (x o ,y o ) : ; In the formula: (x r ,y r )、(x l ,y l ) These represent the coordinates of the top left and bottom right corners of the detection box, respectively. Then, by using the spatial mapping relationship between the pixel coordinate system and the camera coordinate system, the three-dimensional spatial coordinate transformation of the pest is completed:

[0017] In the formula: (x t ,y t ,z t ) Represents three-dimensional spatial coordinates; z de This represents the current depth value acquired by the camera. To mitigate the adverse effects of lighting conditions and other unfavorable factors in the field on camera depth value acquisition and to ensure the accuracy of camera depth values, the two-dimensional coordinates of the center point of the detection box are first acquired simultaneously. (x o ,y o )The depth values ​​of the point and its eight nearest neighbors are denoted as... z i ( i=1,2,…,9 Then, obtain the median value of the depth values ​​from the nine points. z mid excluding the two-dimensional coordinates of the center point of the detection box (x o ,y o ) The other eight depth values ​​besides the median value are respectively compared with the median value. z mid Subtract the values ​​to obtain the comparison values. z bi :

[0018] like If so, the corresponding depth value will be considered an outlier and removed. Finally, the remaining depth values ​​after outlier removal are compared with the two-dimensional coordinates of the detection box center point. (x o ,y o ) The corresponding depth values ​​are averaged to obtain the depth value of the current camera point. z de .

[0019] Based on further optimization of the above scheme, the proposed depth point of the pest in step four is a point that extends 3cm outward from the position of the innermost pest in the image.

[0020] The following are the technical effects of this invention: This invention system, through the coordinated operation of a detection and identification system, a walking mechanism, a robotic arm assembly, an electric water pump, a central control system, a pesticide tank, a chassis, and a sensing system, enables all-weather, real-time monitoring and identification of cotton field pests. This not only improves the real-time and timely nature of pest identification, preventing irreversible damage to plants due to delayed detection, but also ensures accurate identification of pests, enhances the accuracy of pest control (i.e., pesticide application), avoids waste and environmental pollution during pesticide application, and prevents insufficient pesticide application from failing to achieve pest control objectives. Ultimately, it precisely kills cotton bollworms in the field, ensuring the quality and yield of cotton cultivation.

[0021] Furthermore, this invention utilizes a biomimetic nozzle component and a lotus-pod-inspired multi-nozzle nozzle design to spray and control pests of different types and at different stages, offering wide applicability and strong practicality. Simultaneously, the biomimetic nozzle component can be precisely adjusted vertically and horizontally, enabling deep spraying and ensuring even coverage of the crop's upper, lower, and inner sides with pesticides, thus increasing the nozzle's coverage area and significantly improving spraying efficiency and uniformity. Regarding precise pest identification, this invention is based on an improved YOLOv8 image recognition algorithm, which not only efficiently and accurately identifies pests but also, through its integration with DeepSORT, acquires pest movement trajectories, thereby accurately determining the type and quantity of pests. This provides accurate target information for the biomimetic nozzle component, achieving precise and efficient pest eradication. Through integrated design and intelligent visual control technology, this invention significantly improves the efficiency and accuracy of agricultural spraying operations, enabling real-time monitoring of pests and diseases, thereby minimizing the impact of pests on crop cultivation and improving the economic benefits of cotton planting. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall structure of the monitoring and identification system in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the structure of the detection and identification system in the embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of the sensor system of the monitoring and identification system in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of the robotic arm component of the monitoring and identification system in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of the structure of the bionic nozzle component of the monitoring and identification system in an embodiment of the present invention.

[0027] Figure 6 This is a network structure diagram of YOLOv8 in an embodiment of the present invention.

[0028] The components include: 1. Searchlight recognition system; 11. First depth camera; 12. Camera bracket; 13. System mounting base; 14. Dust cover; 15. First searchlight; 2. Walking mechanism; 3. Robotic arm assembly; 31. Rotating base; 32. Support platform; 33. Servo motor; 34. First robotic arm; 35. Linkage mechanism; 36. Second robotic arm; 360. Fixed support; 37. Control motor; 38. Bionic nozzle component; 381. Nozzle mounting plate; 382. Nozzle; 383. Flange; 384. Ball cage transmission joint; 385. Connecting hose; 386. Drive wire; 4. Electric water pump; 5. Central control system; 6. Liquid tank; 7. Chassis; 8. Sensing system. Detailed Implementation The technical solutions in the embodiments of the present invention will be clearly and completely described below. In the following description, specific details such as specific system structures and technologies are presented for illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention.

[0029] Example 1: A precision monitoring and identification system for pests and diseases includes a detection and identification system 1, a walking mechanism 2, a robotic arm assembly 3, an electric water pump 4, a central control system 5, a pesticide tank 6, a chassis 7, and a sensing system 8; the walking mechanism 2 is mounted on the bottom surface of the chassis 7 and is used to control the movement of the entire monitoring and identification system; see also Figure 1 As shown, in this embodiment, the walking mechanism 2 adopts a tracked walking structure, and its specific structure and setting method are all existing conventional technologies. The front side of the chassis 7 end face (i.e....) Figure 1 The medicine tank 6 is fixedly installed on the lower right side (as shown). A detection and identification system 1 is symmetrically installed on both sides of the base 7 end face and on both sides of the medicine tank. Figure 2 As shown: The searchlight recognition system 1 includes a first depth camera 11, a camera bracket 12, a system mounting base 13, a dust cover 14, and a first searchlight 15. The system mounting base 13 is fixedly mounted on the end face of the chassis 7 and located on both sides of the liquid tank 6. Two first depth cameras 11 are evenly arranged vertically along the side of the system mounting base 13 away from the liquid tank 6 via the camera bracket 12 (see...). Figure 2 (As shown) A dust cover 14 is provided around the outer ring of the first depth camera 11, and a first searchlight 15 is provided on the upper side of the system mounting base 13 on the side away from the liquid tank 6. Two electric water pumps 4 are symmetrically arranged on the end face of the chassis 7 at the rear end of the liquid tank 6, and the two electric water pumps 4 are respectively connected to the liquid tank 6. Robotic arm assemblies 3 are symmetrically arranged on the end face of the chassis 7 on the side of the electric water pumps 4 away from the liquid tank 6, and a central control system 5 is fixedly installed on the end face of the chassis 7 between the two sets of robotic arm assemblies 6. Figure 1 As shown), the electric water pump 4 is connected to the bionic nozzle component 8 located at the end of the corresponding robotic arm assembly 3; as shown Figure 4As shown, specifically: the robotic arm assembly 3 includes a rotating base 31, a support platform 32, a servo motor 33, a first robotic arm 34, a linkage mechanism 35, a second robotic arm 36, a control motor 37, and a bionic nozzle component 38. The rotating base 31 is rotatably connected to the end face of the chassis 7, and the support platform 32 is fixedly mounted on the end face of the rotating base 31. The first robotic arm 34 is rotatably mounted on the end face of the support platform 32, and the end of the first robotic arm 34 away from the support platform 32 is rotatably connected to the second robotic arm 36. The rear end of the second robotic arm 36 (i.e., the end where the two sets of robotic arm assemblies 3 are close to each other) is rotatably equipped with the linkage mechanism 35, and the end of the linkage mechanism 35 away from the second robotic arm 36 is connected to the first robotic arm 36. A robotic arm 34 is rotatably connected to a pivot (it should be noted that the number of links in the linkage mechanism 35 is determined according to the actual situation; in this embodiment, two rotatably connected links are used, with the ends of the two links being rotatably connected to the second robotic arm 36 and the first robotic arm 34, respectively); a servo motor 33 is fixedly mounted on a support platform 32 to control the rotation of the first robotic arm 34; the end of the second robotic arm 36 furthest from its corresponding linkage mechanism 35 (i.e., the ends of the two sets of robotic arm assemblies 30 that are far apart) is connected to a bionic nozzle component 38 via a fixed support 360, and a control motor 37 is mounted on the fixed support 360 corresponding to the bionic nozzle component 38. (Refer to...) Figure 5 As shown: The bionic nozzle component 38 includes a nozzle mounting plate 381, nozzles 382, ​​multiple sets of flanges 383, multiple ball cage drive joints 384, a connecting hose 385, and a drive wire 386. Multiple nozzles 382 are evenly arranged on the nozzle mounting plate 381, and the nozzle mounting plate 381 is connected to the fixed support 360 via multiple sets of flanges 383 and multiple ball cage drive joints 384 spaced apart. All connections to the nozzle mounting plate 381 and the fixed support 360 are flanges 383 (i.e., the ball cage drive joint 384 is located between two sets of flanges 383, such as...). Figure 5 As shown, from the fixed support 360 to the nozzle mounting plate 381, the sequence is: flange 383 - ball cage drive joint 384 - flange 383 - ball cage drive joint 384 - flange 383 - ball cage drive joint 384 - flange 383; flange 383 corresponds to the nozzle 382 and is located on the outer ring of the ball cage drive joint 384, with a hose hole; the fixed support 360 also has a corresponding hose hole (e.g., Figure 5As shown, in this embodiment, there are four nozzles 382 on the same nozzle mounting plate 381, therefore there are four hose holes on the same flange 383 and four hose holes on the fixed support 360. One end of the connecting hose 385 (in this embodiment, there are four connecting hoses 385 on the same bionic nozzle component 38) is connected to the corresponding nozzle 382, ​​and the other end of the connecting hose 385 passes through the hose holes on each flange 383 and the hose holes on the fixed support 360 in sequence, and then connects to the corresponding electric water pump 4; flange 383 Four wire holes are provided on the outer ring of the hose opening, and corresponding wire holes are provided on the fixed support 360. Four control motors 37 are provided on the fixed support 360, and four drive wires 386 are provided corresponding to the control motors 37. One end of each drive wire 386 is wound around the output shaft of the corresponding control motor 37, and the other end passes through the corresponding wire holes on the fixed support 360 and the corresponding wire holes on each flange 383, and is then fixedly connected to the flange 383 near the nozzle mounting plate 381 (e.g., ...). Figure 5 (As shown). A sensor system 8 is installed at the front of the chassis 7, as shown. Figure 3 As shown: The sensing system 8 is used for road navigation and obstacle recognition as the chassis 7 moves (road navigation and obstacle recognition are performed using existing conventional methods, which are not specifically limited in this embodiment). It includes a fixed block 81, a lidar 82, a second depth camera 83, and a second searchlight 84. The fixed block 81 is fixedly installed at the front end of the bottom of the chassis 7 and located between the walking mechanisms 2. The lidar 82, the second depth camera 83, and the second searchlight 84 are sequentially installed on the fixed block 81. The searchlight recognition system, the walking mechanism, the robotic arm assembly, the electric water pump, and the sensing system are all electrically connected to the central control system.

[0030] Example 2: As a further optimization of the present application, a method for identifying and controlling field pests is provided, which uses a spotlight identification system in the monitoring and identification system described in Example 1 to identify field pests, including: Step 1: First, perform single-target calibration on both first depth cameras to correct optical distortion during the depth camera imaging process. Specifically: First, using a chessboard calibration board whose corner points are known in the world coordinate system, take multiple images of the chessboard calibration board using cameras, ensuring the board is at different angles and positions. Then, use a corner detection algorithm (e.g., the OpenCV function `cv2.findChessboardCorners` for detecting chessboard corners) to find the corner points of the chessboard calibration board in each image. Finally, using the correspondence between feature points in the image and the world coordinate system, calculate the intrinsic parameter matrix and distortion coefficients of each first depth camera using the Zhang Zhengyou calibration method (e.g., the OpenCV function `cv2.calibrateCamera`). The intrinsic parameter matrix includes the principal point coordinates. （c x ,c y ） With focal length f x 、f y The distortion coefficients include the radial distortion coefficients. k 1 、k 2 、k 3 and tangential distortion coefficient p 1 、p 2.

[0031] Then, the two first-depth depth cameras are calibrated using binocular stereo vision to obtain the reprojection matrix, the conversion relationship between pixel distance and real physical distance for binocular correction; The distance transformation relationship is as follows: First, obtain a distorted image point through the intrinsic parameter matrix. （x d ,y d ） Normalized coordinate points （x n ,y n ） : ; Then, the distortion coefficients are used to obtain the distorted coordinates. （x c ,y c ） :

[0032] in: ; Finally, the normalized coordinates are transformed back to image coordinates.（x u ,y u ） :

[0033] Step 2: Based on the time and light intensity, turn on the searchlights in the searchlight recognition system to provide illumination for the corresponding first depth camera. Use the improved YOLOv8 algorithm to monitor crop pests in real time and identify their corresponding species (e.g., cotton bollworm adults or larvae, other pests harmful to cotton, etc.) and quantities. The specific method for real-time monitoring of crop pests using the improved YOLOv8 algorithm is as follows: Figure 6 As shown, firstly, the input insect pest image is preprocessed (e.g., resized, cropped) to fit the model's input size requirements. Then, a series of convolutional layers (Conv), instance normalization, and activation functions (ELU) in the backbone network are used to identify basic image features (e.g., edges, textures, shapes). Next, the C2f module processes the input image, converting it into a series of features. This new feature map contains multiple features of the input image and can be used for subsequent processing and analysis. Then, the neck network further processes the extracted features, including more convolutional layers, pooling layers (Max Pool2d), and upsampling layers. These layers help the network fuse features at different scales to better identify targets in the image. Finally, the head detection network is used to detect targets in the image. Specifically, this is represented as follows: ; In the formula: X Indicates the input image; F This represents the feature extraction function (composed of Backbone and Neck). P Y represents the prediction function (i.e., the Head part); Y represents the model output, including the coordinates, size, confidence score, and class probability of the bounding box. Throughout the training process, the improved YOLOv8 algorithm uses a loss function to guide the learning process, specifically:

[0034] In the formula: This represents the model's predicted value. Y t Indicates the true label; This represents the total loss function of the model. , These represent the confidence loss for objects with and without targets, respectively. Indicates category loss; Indicates the bounding box loss; These represent the hyperparameters that balance different loss terms.

[0035] The specific method for identifying the corresponding types and quantities of pests is as follows: First, collect an image dataset containing two types of pests (such as adult and larval cotton bollworms, or pest A and pest B, etc. Since this embodiment targets two types of cotton pests, this scheme involves two types, but the entire technical approach is not limited to two types), denoted as... ,in, I i This represents a single image; the image dataset is divided into a training set, a validation set, and a test set (the ratio of training set, validation set, and test set is 7:2:1). Then, a bounding box and category label are added to each pest using an annotation tool (any annotation tool commonly used in the field is acceptable; this embodiment is not specifically limited). Next, the improved YOLOv8 model is trained using the labeled image dataset (i.e., the training set). During training, the YOLOv8 model learns how to detect and classify different pests. Afterward, the trained YOLOv8 model is used for object detection, obtaining the bounding box and category prediction of the pests. For each detected object, the YOLOv8 model outputs a detection vector. ,in, Indicates the coordinates and dimensions of the bounding box. s Indicates the confidence score; Next, non-maximum suppression is applied to the detected bounding boxes to remove overlapping bounding boxes and retain the bounding boxes with the highest confidence scores; specifically, all detected bounding boxes are sorted according to their confidence scores. s Sort from highest to lowest, if the bounding box is B i Then its corresponding confidence score is s i Select the bounding box with the highest confidence level. B h and their corresponding confidence scores s h Calculate separately B h With each subsequent bounding box B j ( j>1 intersection-union ratio IoU(B h , B j ) : ; Preset crossover ratio threshold IoUd ， like (In this embodiment) IoU d If the value is 0.5, then the bounding box B j and B h If there is overlap, remove the overlapping bounding boxes, update the removed bounding boxes, and sort them; otherwise, retain them and perform the next bounding box verification until there are no more bounding boxes to process.

[0036] Subsequently, the detection structure of the YOLOv8 model is used as input, and candidate regions between frames are associated with motion information and appearance features from DeepSORT to form a stable pest trajectory; t The set of successfully tracked trajectories is as follows: ; in, Indicates the first j The bounding box and category information of the trajectory within the time window; M t Indicates the number of currently active trajectories; ; In the formula: Indicates a point in time In the corresponding frame, the first k The coordinates of the detected bounding boxes; Indicates a point in time In the corresponding frame, corresponding to the bounding box Category confidence, This indicates the current time point, which is either the rightmost or latest time point in the time window being discussed. Indicates the length of the time window; Finally, for each trajectory Based on the category confidence of all the frames it contains C x By setting confidence thresholds for each pest category C d Determine the type of pest: like If the pests in the trajectory are in the corresponding category, then the pests belong to that category; otherwise, they do not. Analyze the categories of each trajectory and calculate the pests for each category separately. C a 、C b quantity N a 、N b : ; In the formula: This indicates an indicator function that takes the value 1 when the condition is true and 0 otherwise.

[0037] Step 3: Obtain the two-dimensional coordinates of the pests from the pest image obtained in Step 2, and convert them into three-dimensional spatial coordinates; specifically: First, obtain the two-dimensional coordinates of the center point of the pest detection box in the pixel coordinate system. (x o ,y o ) : ; In the formula: (x r ,y r )、(x l ,y l ) These represent the coordinates of the top left and bottom right corners of the detection box, respectively. Then, by using the spatial mapping relationship between the pixel coordinate system and the camera coordinate system, the three-dimensional spatial coordinate transformation of the pest is completed:

[0038] In the formula: (x t ,y t ,z t ) Represents three-dimensional spatial coordinates; z de This represents the current depth value acquired by the camera. To mitigate the adverse effects of lighting conditions and other unfavorable factors in the field on camera depth value acquisition and to ensure the accuracy of camera depth values, the two-dimensional coordinates of the center point of the detection box are first acquired simultaneously. (x o ,y o ) The depth values ​​of the point and its eight nearest neighbors are denoted as... z i ( i=1,2,…,9 Then, obtain the median value of the depth values ​​from the nine points. z mid excluding the two-dimensional coordinates of the center point of the detection box (x o ,y o )The other eight depth values ​​besides the median value are respectively compared with the median value. z mid Subtract the values ​​to obtain the comparison values. z bi : ; like If so, the corresponding depth value will be considered an outlier and removed. Finally, the remaining depth values ​​after outlier removal are compared with the two-dimensional coordinates of the detection box center point. (x o ,y o ) The corresponding depth values ​​are averaged to obtain the depth value of the current camera point. z de .

[0039] Step 4: Obtain the three-dimensional spatial coordinates of the intended penetration point (the intended penetration point is the point 3cm outward from the innermost insect position in the image) through Step 3, and convert it into three-dimensional spatial coordinates in the coordinate system of the robotic arm component (coordinate transformation can be performed using existing conventional calibration algorithms), control the movement of the corresponding robotic arm component; and control the electric water pump to spray the corresponding amount of pesticide according to the type and quantity of insect pests obtained in Step 2.

[0040] Example 3: As a further optimization of the solution in this application, based on the solution in Example 2, since crops are affected by field wind factors and their leaves are prone to displacement, leading to image acquisition and recognition errors, the leaf displacement is obtained during the image acquisition process. Complete position compensation during the spraying process:

[0041] In the formula: , These represent the offset of the blade in the horizontal and vertical directions, respectively. l Indicates the length of the blade; Indicates the angular offset.

[0042] In the formula, dt represents the test time interval; ; In the formula: m Indicates the weight of the blade; d Indicates the distance from the point of wind action to the crop stalk (i.e., cotton stalk); indicates air density; A Indicates the area affected by the wind; C dThis indicates the drag coefficient of the blade; v Indicates wind speed; Wind speed is collected by wind sensors installed in the fields. v With wind direction The data, combined with the wind-affected area identified by the first depth camera of the monitoring and identification system, A (i.e., the effective area of ​​the blade) to obtain the positional offset of the blade, thereby achieving compensation for the bionic nozzle component 38 during the spraying process.

Claims

1. A precise monitoring and identification system for pests and diseases, characterized in that: The system includes a searchlight recognition system, a walking mechanism, a robotic arm assembly, an electric water pump, a central control system, a liquid tank, a chassis, and a sensing system. The walking mechanism is located on the bottom surface of the chassis. A liquid tank is fixedly installed on the front side of the chassis end face. A searchlight recognition system is symmetrically installed on both sides of the liquid tank on the chassis end face. Two electric water pumps are symmetrically installed on the rear side of the liquid tank on the chassis end face, and each electric water pump is connected to the liquid tank. A robotic arm assembly is symmetrically installed on the side of the chassis end face away from the liquid tank on the side of the electric water pump. A central control system is fixedly installed on the chassis end face between the two sets of robotic arm assemblies. The electric water pumps are connected to the bionic nozzle components at the ends of the corresponding robotic arm assemblies. A sensing system is installed at the front end of the chassis. The searchlight recognition system, the walking mechanism, the robotic arm assembly, the electric water pump, and the sensing system are electrically connected to the central control system. The sensing system includes a fixed block, a lidar, a second depth camera, and a second searchlight. The fixed block is fixedly installed on the front end of the chassis bottom face, between the walking mechanisms. The lidar, the second depth camera, and the second searchlight are sequentially installed on the fixed block.

2. The precise monitoring and identification system for pests and diseases according to claim 1, characterized in that: The searchlight identification system includes a first depth camera, a camera bracket, a system mounting base, a dust cover, and a first searchlight. The system mounting base is fixedly installed on the end face of the chassis and located on both sides of the liquid tank. Two first depth cameras are evenly arranged on the side of the system mounting base away from the liquid tank along the vertical direction through the camera bracket, and a dust cover is installed around the outer ring of the first depth camera. The first searchlight is installed on the upper end of the side of the system mounting base away from the liquid tank.

3. The precise monitoring and identification system for pests and diseases according to claim 2, characterized in that: The robotic arm assembly includes a rotating base, a support platform, a servo motor, a first robotic arm, a linkage mechanism, a second robotic arm, a control motor, and a bionic nozzle component. The rotating base is rotatably connected to the end face of the chassis, and the support platform is fixedly mounted on the end face of the rotating base. The first robotic arm is rotatably mounted on the end face of the support platform, and the end of the first robotic arm away from the support platform is rotatably connected to the second robotic arm. The rear end of the second robotic arm is rotatably connected to the linkage mechanism, and the end of the linkage mechanism away from the second robotic arm is rotatably connected to the rotating shaft of the first robotic arm. The servo motor is fixedly mounted on the support platform and is used to control the rotation of the first robotic arm. The end of the second robotic arm away from its corresponding linkage mechanism is mounted on a fixed support, and a control motor is mounted on the fixed support corresponding to the bionic nozzle component. The bionic nozzle component includes a nozzle mounting plate, a nozzle, multiple sets of flanges, multiple ball cage transmission joints, a connecting hose, and a drive steel wire. The nozzle mounting plate is evenly distributed with... Multiple nozzles are mounted, and the nozzle mounting plate and the fixed support are connected to multiple ball cage drive joints via multiple sets of flanges spaced apart. All connections to the nozzle mounting plate and the fixed support are flanges. Each flange corresponds to a nozzle and has a hose hole on the outer ring of the ball cage drive joint. A corresponding hose hole is also provided on the fixed support. One end of the hose is connected to the corresponding nozzle, and the other end passes through the hose holes on each flange and the fixed support, and then connects to the corresponding electric water pump. Four wire holes are provided on the outer ring of the hose holes on each flange and the fixed support. Four control motors are mounted on the fixed support, and four drive wires are provided corresponding to the control motors. One end of each drive wire is wound around the output shaft of the corresponding control motor, and the other end passes through the corresponding wire holes on the fixed support and the corresponding wire holes on each flange, and then is fixedly connected to the flange near the nozzle mounting plate.

4. A precise pest and disease monitoring and identification system as described in claim 2 or 3, characterized in that: The specific identification and control method for field pests in this system includes: Step 1: First, perform single-target calibration on the two first depth cameras to correct optical distortion during the imaging process; then, perform binocular stereo vision calibration on the two first depth cameras to obtain the reprojection matrix used for binocular correction, and the conversion relationship between pixel distance and real physical distance; Step 2: According to different times and light intensities, turn on the searchlights in the searchlight identification system to provide illumination for the corresponding first depth cameras, and use the improved YOLOv8 algorithm to monitor crop pests in real time and identify their corresponding species and quantities; Step 3: Obtain the two-dimensional coordinates of the pests from the pest images obtained in Step 2, and convert them into three-dimensional spatial coordinates; Step 4: Obtain the three-dimensional spatial coordinates of the intended depth point from Step 3, convert them into three-dimensional spatial coordinates in the coordinate system of the robotic arm component, control the movement of the corresponding robotic arm component, and control the amount of pesticide sprayed by the electric water pump according to the corresponding pest species and quantities obtained in Step 2.

5. The precise monitoring and identification system for pests and diseases according to claim 4, characterized in that: The specific steps of performing single-target calibration on the two first depth cameras in step one to correct optical distortion during the depth camera imaging process are as follows: First, using a chessboard calibration board whose corner points are known in the world coordinate system, multiple images of the chessboard calibration board are taken using cameras, ensuring that the chessboard calibration board is at different angles and positions; then, a corner detection algorithm is used to find the corner points of the chessboard calibration board in each image; finally, using the correspondence between feature points in the image and the world coordinate system, the Zhang Zhengyou calibration method is used to calculate the intrinsic parameter matrix and distortion coefficients of each first depth camera, where the intrinsic parameter matrix includes the principal point coordinates. （c x ,c y ） With focal length f x 、f y The distortion coefficients include the radial distortion coefficients. k 1 、k 2 、k 3 and tangential distortion coefficient p 1 、 p 2.

6. The precise monitoring and identification system for pests and diseases according to claim 5, characterized in that: In step one, the two first depth cameras are calibrated using binocular stereo vision to obtain the reprojection matrix used for binocular correction, and the conversion relationship between pixel distance and real physical distance is as follows: First, obtain a distorted image point through the intrinsic parameter matrix. （x d ,y d ） Normalized coordinate points （x n ,y n ） : ； Then, the distortion coefficients are used to obtain the distorted coordinates. （x c ,y c ） : in: ; Finally, the normalized coordinates are transformed back to image coordinates. （x u ,y u ） : 。 7. The precise monitoring and identification system for pests and diseases according to claim 6, characterized in that: The specific steps in step two, which involve using the improved YOLOv8 algorithm for real-time monitoring of crop pests, are as follows: First, the input insect pest image is preprocessed to fit the input size requirements of the model; then, the basic features of the image are identified through a series of convolutional layers, batch normalization, and activation functions of the backbone network Backbon; after that, the input image is processed by the C2f module to convert it into a series of features. Next, the extracted features are further processed through the neck network; subsequently, the head detection network is used to detect targets in the image; specifically: ； In the formula: X Indicates the input image; F Represents the feature extraction function; P Y represents the prediction function; Y represents the model output, including the coordinates, size, confidence score, and class probability of the bounding box. Throughout the training process, the improved YOLOv8 algorithm uses a loss function to guide the learning process, specifically: In the formula: This represents the model's predicted value. Y t Indicates the true label; This represents the total loss function of the model. , These represent the confidence loss for objects with and without targets, respectively. Indicates category loss; Indicates the bounding box loss; These represent the hyperparameters that balance different loss terms.

8. The precise monitoring and identification system for pests and diseases according to claim 7, characterized in that: The specific steps for identifying the types and quantities of pests in step two are as follows: First, collect an image dataset containing two insect pests, denoted as . ,in, I i This represents a single image; the image dataset is divided into training, validation, and test sets, and then a labeling tool is used to add bounding boxes and category labels to each pest. Then, the improved YOLOv8 model was trained using the labeled image dataset. During the training process, the YOLOv8 model learned how to detect and classify different pests. Next, the trained YOLOv8 model was used for object detection to obtain the bounding boxes and class predictions of pests; for each detected object, the YOLOv8 model output a detection vector. ,in, Indicates the coordinates and dimensions of the bounding box. s Indicates the confidence score; Then, non-maximum suppression is applied to the detected bounding boxes to remove overlapping bounding boxes and retain the bounding boxes with the highest confidence. Subsequently, the detection structure of the YOLOv8 model is used as input, and candidate regions between frames are associated with motion information and appearance features from DeepSORT to form a stable pest trajectory; t The set of successfully tracked trajectories is as follows: ； in, Indicates the first j The bounding box and category information of the trajectory within the time window; M t Indicates the number of currently active trajectories; ； In the formula: Indicates at a point in time In the corresponding frame, the first k The coordinates of the detected bounding boxes; Indicates at a point in time In the corresponding frame, corresponding to the bounding box Category confidence, This indicates the current time point, which is either the rightmost or latest time point in the time window being discussed. Indicates the length of the time window; Finally, for each trajectory Based on the category confidence of all the frames it contains C x By setting confidence thresholds for each pest category C d Determine the type of pest: like If the pests in the trajectory are in the corresponding category, then the pests belong to that category; otherwise, they do not. Analyze the categories of each trajectory and calculate the pests for each category separately. C a 、C b quantity N a 、N b : ； In the formula: This indicates an indicator function that takes the value 1 when the condition is true and 0 otherwise.

9. The precise monitoring and identification system for pests and diseases according to claim 8, characterized in that: The specific method for converting the two-dimensional coordinates of the pests into three-dimensional spatial coordinates in step three is as follows: First, obtain the two-dimensional coordinates of the center point of the pest detection box in the pixel coordinate system. (x o ,y o ) : ； In the formula: (x r ,y r )、(x l ,y l ) These represent the coordinates of the top left and bottom right corners of the detection box, respectively. Then, by using the spatial mapping relationship between the pixel coordinate system and the camera coordinate system, the three-dimensional spatial coordinate transformation of the pest is completed: In the formula: (x t ,y t ,z t ) Represents three-dimensional spatial coordinates; z de This represents the depth value of the current point acquired by the camera; Simultaneously acquire the two-dimensional coordinates of the center point of the detection box (x o ,y o ) The depth values ​​of the point and its eight nearest neighbors are denoted as... z i ( i=1, 2,…,9 Then, obtain the median value of the depth values ​​from the nine points. z mid excluding the two-dimensional coordinates of the center point of the detection box (x o ,y o ) The other eight depth values ​​besides the median value are respectively compared with the median value. z mid Subtract the values ​​to obtain the comparison values. z bi : ； like If so, the corresponding depth value will be considered an outlier and removed. Finally, the remaining depth values ​​after outlier removal are compared with the two-dimensional coordinates of the detection box center point. (x o ,y o ) The corresponding depth values ​​are averaged to obtain the depth value of the current camera point. z de .

10. The precision monitoring and identification system for pests and diseases according to claim 8, characterized in that: Step four also includes compensation for the bionic nozzle component, specifically: during image acquisition, the offset of the blades is obtained. Complete position compensation during the spraying process: In the formula: , These represent the offset of the blade in the horizontal and vertical directions, respectively. l Indicates the length of the blade; Indicates the angular offset. In the formula, dt represents the test time interval; ； In the formula: m Indicates the weight of the blade; d Indicates the distance from the point of wind action to the crop stalk; indicates air density; A Indicates the area affected by the wind; C d This indicates the drag coefficient of the blade; v Indicates wind speed; Wind speed is collected by wind sensors installed in the fields. v With wind direction The data, combined with the wind-affected area identified by the first depth camera, A This allows for the acquisition of the blade's positional offset, thereby enabling compensation for the bionic nozzle components during the spraying process.

Images