Identification control spraying robot for field insect pests and method thereof

Abstract

The application provides a recognition control spraying robot for field insect pests, and relates to the field of intelligent agricultural robots, and comprises an exploring and identifying system (1), a walking mechanism (2), a mechanical arm assembly (3), an electric water pump (4), a central control system (5), a pesticide tank (6), a chassis (7) and a sensing system (8). The electric water pump (4) is communicated with a bionic spray head component (38) arranged at the tail end of the corresponding mechanical arm assembly (3). The exploring and identifying system (1), the walking mechanism (2), the mechanical arm assembly (3), the electric water pump (4) and the sensing system (8) are electrically connected with the central control system (5) respectively. The spraying robot can realize all-weather detection and recognition in the day and at night based on the machine vision technology, has high detection precision and strong timeliness. Meanwhile, the spraying robot can realize uniform spraying on crops, improve spraying efficiency and avoid waste of pesticide.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural intelligent robots, in particular to a recognition and control spraying robot for field insect pests and a method thereof. BACKGROUND

[0002] The cotton bollworm, also known as the cotton bollworm, is an agricultural pest that damages soybeans, peanuts, cotton, rice, wheat and other crops. The adult cotton bollworm hides during the day and comes out at night. It usually hides in the leaf back, flower crown and other hidden places during the day. It starts to move at dusk and even climbs to the liquid surface. It feeds on cotton flower buds, tender leaves and young bolls. It prefers to lay eggs on cotton with dense growth and many flower buds. The egg-laying sites are usually the tender tips and tender leaves of the young plants. The cotton bollworm larvae feed on tender leaves. After two generations, they start to eat young bolls. The third and fourth instar larvae mainly damage bolls and flowers, causing boll drop. The fifth and sixth instar larvae enter the feeding period and damage green bolls, large bolls or flowers. Therefore, the cotton bollworm can cause rapid boll drop, tender leaves to be full of holes and young bolls to be hollowed out, resulting in a reduction in cotton yield. The yield loss is generally 15-20%, and can be as high as 50% in severe cases.

[0003] Due to the diurnal and nocturnal habits of the cotton bollworm, it is difficult to identify and monitor it in real time and accurately, so timely prevention and control cannot be carried out, resulting in irreversible damage to the cotton caused by the cotton bollworm feeding on the plants. Currently, there are two ways to monitor field insect pests in cotton fields: one is human identification and judgment, but human identification requires workers to stay in the cotton field for a long time, which is extremely labor-intensive, has a long cycle and high labor cost. In addition, human identification is greatly affected by human factors, resulting in low identification accuracy. The second is to identify field insect pests by unmanned aerial vehicle spectrum, but the unmanned aerial vehicle identification has a certain lag, that is, when the unmanned aerial vehicle spectrum is detected, the cotton bollworm has already caused great damage to the cotton, affecting subsequent control measures. At the same time, the unmanned aerial vehicle spectrum identification cannot be used for real-time monitoring at night and during the day (the effect of unmanned aerial vehicle detection at night is significantly lower than that during the day), thus causing the cotton to be heavily damaged by the cotton bollworm and resulting in a reduction in cotton yield. In addition, the existing pesticide spraying equipment cannot achieve deep spraying in crops, resulting in uneven spraying, low spraying efficiency, easy waste of pesticide and poor control effect on the cotton bollworm. SUMMARY

[0004] In view of the problems existing in the prior art, the present application aims to provide a recognition and control spraying robot for field insect pests. The spraying robot can detect and identify day and night, has high detection accuracy and strong timeliness. At the same time, the spraying robot can achieve uniform spraying of crops, effectively improve the spraying efficiency and uniformity, avoid waste of pesticide and poor control effect on the cotton bollworm.

[0005] Another object of the present application is to provide a recognition control method for field insect pests.

[0006] The object of the present application is achieved by the following technical solutions:

[0007] A recognition control spraying robot for field insect pests comprises a search and identification system, a walking mechanism, a mechanical arm assembly, an electric water pump, a central control system, a liquid tank, a chassis and a sensing system. The walking mechanism is arranged on the bottom surface of the chassis and is used to control the walking of the whole spraying robot. The liquid tank is fixedly arranged on the front side of the end surface of the chassis. The search and identification system is symmetrically arranged on the end surface of the chassis and located on both sides of the liquid tank. Two electric water pumps are symmetrically arranged on the end surface of the chassis and located at the rear end of the liquid tank, and the two electric water pumps are respectively communicated with the liquid tank. The mechanical arm assembly is symmetrically arranged on the end surface of the chassis and located on the side of the electric water pump away from the liquid tank, and the central control system is fixedly arranged on the end surface of the chassis between the two mechanical arm assemblies. The electric water pump is communicated with the bionic nozzle component arranged at the end of the corresponding mechanical arm assembly. The sensing system is arranged on the front end of the chassis. The search and identification system, the walking mechanism, the mechanical arm assembly, the electric water pump and the sensing system are respectively electrically connected with the central control system.

[0008] Based on the further optimization of the above scheme, the search and identification system comprises a first depth camera, a camera support, a system fixing seat, a dust cover and a first search lamp. The system fixing seat is fixedly arranged on the end surface of the chassis and located on both sides of the liquid tank. Two first depth cameras are uniformly arranged on the side surface away from the liquid tank through the camera support in the vertical direction, and the dust cover is arranged on the outer circle of the first depth camera. The first search lamp is arranged on the upper end of the side surface away from the liquid tank.

[0009] Based on the further optimization of the above scheme, the mechanical arm assembly comprises a rotating base, a support table, a servo motor, a first mechanical arm, a connecting rod mechanism, a second mechanical arm, a control motor and a bionic nozzle component. The rotating base is rotatably connected with the end surface of the chassis, and the support table is fixedly arranged on the end surface of the rotating base. The first mechanical arm is rotatably arranged on the end surface of the support table, and one end of the first mechanical arm away from the support table is rotatably connected with the second mechanical arm. The connecting rod mechanism is rotatably arranged on the rear end of the second mechanical arm (i.e. one end of the two mechanical arm assemblies close to each other), and one end of the connecting rod mechanism away from the second mechanical arm is rotatably connected with the rotating shaft of the first mechanical arm. The servo motor is fixedly arranged on the support table and is used to control the rotation of the first mechanical arm. The bionic nozzle component is arranged on one end of the second mechanical arm away from the corresponding connecting rod mechanism (i.e. one end of the two mechanical arm assemblies away from each other) through a fixed support, and the control motor is arranged on the fixed support corresponding to the bionic nozzle component.

[0010] Based on the further optimization of the above scheme, the bionic nozzle component includes a nozzle setting plate, a nozzle, multiple sets of flanges, multiple ball cage transmission joints, a communication hose and a driving steel wire, multiple nozzles are uniformly arranged on the nozzle setting plate, and the nozzle setting plate and the fixed support are connected through multiple sets of flanges and multiple ball cage transmission joints arranged at intervals, and the flanges connected with the nozzle setting plate and the fixed support are flanges (i.e. the ball cage transmission joint is located between the two sets of flanges); the flanges correspond to the nozzles and are located outside the soft tube holes of the ball cage transmission joints, soft tube holes are correspondingly arranged on the fixed support, one end of the communication hose is communicated with the corresponding nozzle, and the other end of the communication hose (i.e. the communication hose) is communicated with the corresponding electric water pump after penetrating the soft tube holes on each flange and the soft tube holes on the fixed support; four steel wire holes are arranged on the outer circle of the flanges, and steel wire holes are correspondingly arranged on the fixed support, four control motors are arranged on the fixed support, and the driving steel wire is correspondingly arranged on the control motor, i.e. the driving steel wire is four, one end of the driving steel wire is wound on the output shaft of the corresponding control motor, and the other end of the driving steel wire (i.e. the driving steel wire) is sequentially penetrated through the corresponding steel wire holes on the fixed support and the corresponding steel wire holes on each flange, and is fixedly connected with the flange close to the nozzle setting plate.

[0011] Based on the further optimization of the above scheme, the sensing system is used for road navigation and obstacle identification as the chassis moves, and includes a fixed block, a laser radar, a second depth camera and a second searchlight, the fixed block is fixedly arranged on the bottom surface of the chassis at the front end and between the walking mechanisms, and the laser radar, the second depth camera and the second searchlight are sequentially arranged on the fixed block.

[0012] A recognition control method for field insect pests, which uses the search and recognition system in the spraying robot as described above to recognize field insect pests, comprising:

[0013] Step one, first, single target calibration is performed on the two first depth cameras to correct optical distortion in the imaging process of the depth camera; then, binocular stereo vision calibration is performed on the two first depth cameras to obtain a re-projection matrix for binocular correction and a conversion relationship between pixel distance and real physical distance;

[0014] Step two, according to the difference in time and light intensity, the searchlight in the search and recognition system is turned on to provide illumination for the corresponding first depth camera, the improved YOLOv8 algorithm is used to monitor the insect pests of crops in real time and identify the corresponding types (for example: cotton bollworm adults or larvae, other insect pests harmful to cotton, etc.) and quantities;

[0015] Step three, the two-dimensional coordinates of the insect pests are obtained from the insect pest images obtained in step two, and are converted into three-dimensional space coordinates;

[0016] Step four, the three-dimensional space coordinates of the proposed in-depth point obtained by step three are converted into three-dimensional space coordinates in the mechanical arm assembly coordinate system, the corresponding mechanical arm assembly is controlled to move, and the electric water pump is controlled to spray the corresponding amount of pesticide according to the corresponding species and quantity of the insect pests obtained in step two.

[0017] Based on the further optimization of the above scheme, the specific steps for correcting optical distortion in the depth camera imaging process in the step one of single target calibration for the two first depth cameras are as follows:

[0018] Firstly, a chessboard calibration board is used, the positions of the corner points of which in the world coordinate system are known, and multiple pictures of the chessboard calibration board are taken by the camera respectively, ensuring that the chessboard calibration board is at different angles and positions; then, the corner point detection algorithm (for example, the function cv2.findChessboardCorners in the OpenCV library for detecting chessboard corner points) is used to find the corner points of the chessboard calibration board in each picture; finally, the correspondence between the feature points in the picture and the world coordinate system is used, and the Zhang Zhengyou calibration method (for example, the function cv2.calibrateCamera in the OpenCV library for implementing the Zhang Zhengyou calibration method) is used to calculate the intrinsic matrix and distortion coefficient of each first depth camera, wherein the intrinsic matrix includes principal point coordinates （c x ,c y ） and focal length f x 、f y , and the distortion coefficient includes radial distortion coefficient k 1 、k 2 、k 3 and tangential distortion coefficient p 1 、p 2.

[0019] Based on the further optimization of the above scheme, the specific steps for correcting optical distortion in the depth camera imaging process in the step one of single target calibration for the two first depth cameras are as follows:

[0020] Firstly, the normalized coordinate point （x d ,y d ） of a distorted image point （x n ,y n ） is obtained through the intrinsic matrix:

[0021] ;

[0022] Then, the distortion coefficient is used to obtain the distorted coordinate points （x c ,y c ） :

[0023]

[0024] wherein: ;

[0025] Finally, the normalized coordinates are converted back to image coordinates （x u ,y u ） :

[0026]

[0027] Based on the further optimization of the above scheme, the specific method for real-time monitoring of crop pests in step two is as follows:

[0028] First, the input pest image is preprocessed (such as resizing, cropping, etc.) to adapt to the input size requirements of the model; then, the basic features (such as edges, textures, shapes, etc.) of the image are identified through a series of convolution layers (Conv), batch normalization (Instance Normalization) and activation functions (ELU) in the backbone network Backbon; then, the input image is processed through the C2f module to convert it into a series of features; then, the extracted features are further processed through the neck network Neck; then, the head network Head Detection is used to detect the targets in the image; the specific representation is as follows:

[0029] ;

[0030] wherein: X represents the input image; F represents the feature extraction function (composed of Backbone and Neck); P represents the prediction function (i.e. the Head part); Y represents the model output, including the coordinates, size, confidence and class probability of the bounding box, etc.

[0031] During the entire training process, the improved YOLOv8 algorithm uses a loss function to guide the learning process, which is as follows:

[0032]

[0033] wherein: a prediction value of the model, Y t a real label; a total loss function of the model, 、 confidence loss of objects with and without targets, respectively; a category loss; a bounding box loss; hyperparameters for balancing different loss terms, respectively.

[0034] Based on the further optimization of the above scheme, the specific method for identifying the corresponding species and quantity of the pest in step two is as follows:

[0035] First, collect an image dataset containing two pests (such as cotton bollworm adults, larvae, or pest A and pest B), denoted as wherein, I i denotes a single image; and divide the image dataset into a training set, a validation set, and a test set, and then use a labeling tool to add a bounding box and a category label for each pest;

[0036] Then, use the labeled image dataset (i.e., the training set) to train the improved YOLOv8 model. During the training process, the YOLOv8 model learns how to detect and classify different pests.

[0037] After that, use the trained YOLOv8 model for target detection to obtain the bounding box and category prediction of the pest. For each detection object, the YOLOv8 model outputs a detection vector wherein, denotes the coordinates and dimensions of the bounding box, s denotes the confidence score;

[0038] Then, perform non-maximum suppression on the detected bounding boxes to remove overlapping bounding boxes and retain the highest confidence bounding box.

[0039] Subsequently, use the detection structure of the YOLOv8 model as input to associate the candidate regions between frames through the motion information and appearance features of DeepSORT to form stable pest trajectories. The first t frame tracked successful trajectory set is:

[0040] ;

[0041] wherein, denotes the bounding box and category information of the j th trajectory within the time window; M t denotes the number of currently active trajectories;

[0042] ;

[0043] In the formula: represents the coordinates of the detected bounding box in the corresponding frame at time point ; k represents the class confidence corresponding to the bounding box in the corresponding frame at time point ; represents the current time point, i.e. the rightmost or latest time point in the time window being discussed; represents the length of the time window;

[0044] Finally, for each trajectory , the class confidence C of all the frames contained therein is determined according to the class confidence x of each frame in the trajectory. C d The pest class is determined by setting a confidence threshold for each pest class:

[0045] If , the pest of the trajectory belongs to the corresponding class; otherwise, it does not belong to the corresponding class.

[0046] The classes of each trajectory are counted, and the number of two types of pests C a 、C b N a 、N b is calculated respectively:

[0047] ;

[0048] In the formula: represents an indicator function, which takes a value of 1 when the condition is true, and a value of 0 otherwise.

[0049] Further optimization based on the above scheme is that the detected bounding boxes are subjected to non-maximum suppression to remove overlapping bounding boxes and retain the highest confidence bounding box, which is specifically:

[0050] All detected bounding boxes are sorted in descending order according to the confidence score s . If the bounding box is B i , the corresponding confidence score is s i ; the bounding box with the highest confidence B h and the corresponding confidence score s ​h , respectively calculate B h The intersection-over-union of each subsequent bounding box B j ( j>1 ) with the previous one IoU(B h , B j ) :

[0051] ;

[0052] A preset intersection-over-union threshold IoU d ， If , the bounding box B j and B h overlap, remove the overlapping bounding box, update the removed bounding box and sort; otherwise, keep and perform the next bounding box validation until there is no redundant bounding box to process.

[0053] Based on the further optimization of the above scheme, the method for converting the two-dimensional coordinates of the pest into three-dimensional space coordinates in step three is as follows:

[0054] First, obtain the two-dimensional coordinates of the pest detection frame center point in the pixel coordinate system (x o ,y o ) :

[0055] ;

[0056] In the formula: (x r ,y r )、(x l ,y l ) respectively represent the coordinate points of the top left corner and the bottom right corner of the detection frame;

[0057] Then, through the spatial mapping relationship between the pixel coordinate system and the camera coordinate system, the three-dimensional space coordinates of the pest are converted:

[0058]

[0059] In the formula: (x t ,y t ,zt ) represents three-dimensional space coordinates; z de represents the current point depth value acquired by the camera;

[0060] In order to reduce the adverse effects of adverse conditions such as light on the acquisition of camera depth values in the field environment, and to achieve the accuracy of camera depth values, first, the two-dimensional coordinates of the center point of the detection frame are acquired (x o ,y o ) and the depth values of the eight points around it are recorded as z i i=1,2,…,9 ; Then, the median value of the depth values of the nine points z mid , excluding the depth value of the two-dimensional coordinates of the center point of the detection frame (x o ,y o ) , is subtracted from the remaining eight depth values respectively, to obtain the comparison value z mid z bi :

[0061]

[0062] If , the corresponding depth value is regarded as an abnormal value and is removed;

[0063] Finally, the remaining depth values after removing the abnormal values are averaged with the corresponding depth values of the two-dimensional coordinates of the center point of the detection frame (x o ,y o ) to obtain the depth value of the current point of the camera z de .

[0064] Further optimization based on the above scheme, the proposed in-depth point of the pest in step four is a point extending 3cm outward from the position of the innermost pest in the image.

[0065] The technical effects possessed by the present application are as follows:

[0066] ​​The spraying robot of the application is equipped with a search and identification system, a walking mechanism, a mechanical arm assembly, an electric water pump, a central control system, a liquid tank, a chassis and a sensing system, realizes uninterrupted monitoring and identification of cotton pest damage throughout the day and night, and thus improves the timeliness and accuracy of pest management, avoids the irreversible damage to cotton crops caused by delayed pest treatment, and thus causes problems such as crop yield reduction and pest outbreak. At the same time, through accurate monitoring and identification of pests, the accurate delivery of pesticides can be realized, the use of chemical pesticides can be reduced, the waste of chemical pesticides can be avoided, the pollution of the ecological environment can be avoided, the quality and yield of cotton can be effectively ensured, and crop mutation caused by the abuse of chemical pesticides can be avoided. In addition, through the setting of the bionic spray head component and the design of the lotus-shaped multi-nozzle spray head mechanism, the bionic spray head component can be accurately adjusted in the vertical and horizontal directions and can realize deep spraying, so that the pesticide can uniformly cover the upper, lower and inner sides of the crops, the coverage range of the spray head is improved, and the spraying efficiency and uniformity are significantly improved.

[0067] For the identification of pests, the improved YOLOv8 algorithm is used to efficiently and accurately identify pests. At the same time, through the cooperation of YOLOv8 algorithm and DeepSORT, the movement track of pests is obtained, and the type and quantity of pests are accurately obtained, providing accurate target information for the bionic spray head component. The present application greatly improves the efficiency and accuracy of agricultural pesticide application through integrated design and intelligent visual control technology, realizes real-time monitoring of pests, and thus solves the influence of pests on crop planting to the greatest extent. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 It is a schematic diagram of the overall structure of the spraying robot in the embodiment of the application.

[0069] Figure 2 It is a schematic diagram of the structure of the search and identification system of the spraying robot in the embodiment of the application.

[0070] Figure 3 It is a schematic diagram of the structure of the sensing system of the spraying robot in the embodiment of the application.

[0071] Figure 4 It is a schematic diagram of the structure of the mechanical arm assembly of the spraying robot in the embodiment of the application.

[0072] Figure 5 It is a schematic diagram of the structure of the bionic spray head component of the spraying robot in the embodiment of the application.

[0073] Figure 6 It is a network structure diagram of YOLOv8 in the embodiment of the application.

[0074] The components include: 1. Searchlight recognition system; 11. First depth camera; 12. Camera bracket; 13. System mounting base; 14. Dust cover; 15. First searchlight; 2. Walking mechanism; 3. Robotic arm assembly; 31. Rotating base; 32. Support platform; 33. Servo motor; 34. First robotic arm; 35. Linkage mechanism; 36. Second robotic arm; 360. Fixed support; 37. Control motor; 38. Bionic nozzle component; 381. Nozzle mounting plate; 382. Nozzle; 383. Flange; 384. Ball cage transmission joint; 385. Connecting hose; 386. Drive wire; 4. Electric water pump; 5. Central control system; 6. Liquid tank; 7. Chassis; 8. Sensing system. Detailed Implementation

[0075] The technical solutions in the embodiments of the present invention will be clearly and completely described below. In the following description, specific details such as specific system structures and technologies are presented for illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention.

[0076] Example 1:

[0077] A field pest identification and control spraying robot includes a searchlight identification system 1, a walking mechanism 2, a robotic arm assembly 3, an electric water pump 4, a central control system 5, a pesticide tank 6, a chassis 7, and a sensing system 8; the walking mechanism 2 is mounted on the bottom surface of the chassis 7 and is used to control the movement of the entire spraying robot; see also Figure 1 As shown, in this embodiment, the walking mechanism 2 adopts a tracked walking structure, and its specific structure and setting method are all existing conventional technologies. The front side of the chassis 7 end face (i.e....) Figure 1 The medicine tank 6 is fixedly installed on the lower right side (as shown). A detection and identification system 1 is symmetrically installed on both sides of the base 7 end face and on both sides of the medicine tank. Figure 2 As shown: The searchlight recognition system 1 includes a first depth camera 11, a camera bracket 12, a system mounting base 13, a dust cover 14, and a first searchlight 15. The system mounting base 13 is fixedly mounted on the end face of the chassis 7 and located on both sides of the liquid tank 6. Two first depth cameras 11 are evenly arranged vertically along the side of the system mounting base 13 away from the liquid tank 6 via the camera bracket 12 (see...). Figure 2 (As shown) The first depth camera 11 is provided with a dust cover 14 on its outer ring, and the system mounting base 13 is located on the side away from the medicine tank 6 and the upper end of the first searchlight 15 is provided.

[0078] Two electric water pumps 4 are symmetrically arranged on the end face of the chassis 7 at the rear end of the liquid tank 6, and the two electric water pumps 4 are respectively connected to the liquid tank 6. Robotic arm assemblies 3 are symmetrically arranged on the end face of the chassis 7 on the side of the electric water pumps 4 away from the liquid tank 6, and a central control system 5 is fixedly installed on the end face of the chassis 7 between the two sets of robotic arm assemblies 6. Figure 1As shown), the electric water pump 4 is connected to the bionic nozzle component 8 located at the end of the corresponding robotic arm assembly 3; as shown Figure 4 As shown, specifically: the robotic arm assembly 3 includes a rotating base 31, a support platform 32, a servo motor 33, a first robotic arm 34, a linkage mechanism 35, a second robotic arm 36, a control motor 37, and a bionic nozzle component 38. The rotating base 31 is rotatably connected to the end face of the chassis 7, and the support platform 32 is fixedly mounted on the end face of the rotating base 31. The first robotic arm 34 is rotatably mounted on the end face of the support platform 32, and the end of the first robotic arm 34 away from the support platform 32 is rotatably connected to the second robotic arm 36. The rear end of the second robotic arm 36 (i.e., the end where the two sets of robotic arm assemblies 3 are close to each other) is rotatably equipped with the linkage mechanism 35, and the end of the linkage mechanism 35 away from the second robotic arm 36 is connected to the first robotic arm 36. A robotic arm 34 is rotatably connected to a pivot (it should be noted that the number of links in the linkage mechanism 35 is determined according to the actual situation; in this embodiment, two rotatably connected links are used, with the ends of the two links being rotatably connected to the second robotic arm 36 and the first robotic arm 34, respectively); a servo motor 33 is fixedly mounted on a support platform 32 to control the rotation of the first robotic arm 34; the end of the second robotic arm 36 furthest from its corresponding linkage mechanism 35 (i.e., the ends of the two sets of robotic arm assemblies 30 that are far apart) is connected to a bionic nozzle component 38 via a fixed support 360, and a control motor 37 is mounted on the fixed support 360 corresponding to the bionic nozzle component 38. (Refer to...) Figure 5 As shown: The bionic nozzle component 38 includes a nozzle mounting plate 381, nozzles 382, ​​multiple sets of flanges 383, multiple ball cage drive joints 384, a connecting hose 385, and a drive wire 386. Multiple nozzles 382 are evenly arranged on the nozzle mounting plate 381, and the nozzle mounting plate 381 is connected to the fixed support 360 via multiple sets of flanges 383 and multiple ball cage drive joints 384 spaced apart. All connections to the nozzle mounting plate 381 and the fixed support 360 are flanges 383 (i.e., the ball cage drive joint 384 is located between two sets of flanges 383, such as...). Figure 5 As shown, from the fixed support 360 to the nozzle mounting plate 381, the sequence is: flange 383 - ball cage drive joint 384 - flange 383 - ball cage drive joint 384 - flange 383 - ball cage drive joint 384 - flange 383; flange 383 corresponds to the nozzle 382 and is located on the outer ring of the ball cage drive joint 384, with a hose hole; the fixed support 360 also has a corresponding hose hole (e.g., Figure 5As shown, in this embodiment, the number of nozzles 382 on the same nozzle setting plate 381 is four, so the number of hose holes on the same flange plate 383 is four, and the number of hose holes on the fixed support 360 is four. One end of the communication hose 385 (in this embodiment, there are four communication hoses 385 for the same bionic nozzle component 38) is in communication with the corresponding nozzle 382, and the other end of the communication hose 385 is in communication with the corresponding electric water pump 4 after sequentially penetrating the hose holes on each flange plate 383 and the hose holes on the fixed support 360. Four steel wire holes are provided on the outer ring of the hose hole of the flange plate 383, and corresponding steel wire holes are provided on the fixed support 360. Four control motors 37 are provided on the fixed support 360, and the driving steel wires 386 are arranged corresponding to the control motors 37, i.e., there are four driving steel wires 386. One end of the driving steel wire 386 is wound around the output shaft of the corresponding control motor 37, and the other end of the driving steel wire 386 is fixedly connected to the flange plate 383 close to the nozzle setting plate 381 (as shown in Figure 5 ).

[0079] The chassis 7 is provided with a sensing system 8 at the front end, as shown in Figure 3 : The sensing system 8 is used for road navigation and obstacle identification of the chassis 7 movement (road navigation and obstacle identification are performed by using existing conventional methods, and in this embodiment, no specific limitation is made). The sensing system 8 includes a fixed block 81, a laser radar 82, a second depth camera 83, and a second searchlight 84. The fixed block 81 is fixedly arranged on the bottom surface of the chassis 7 at the front end and between the walking mechanisms 2. The laser radar 82, the second depth camera 83, and the second searchlight 84 are sequentially arranged on the fixed block 81. The search and identification system, the walking mechanism, the mechanical arm assembly, the electric water pump, and the sensing system are electrically connected to the central control system.

[0080] Embodiment 2

[0081] As a further optimization of the scheme of the present application, a recognition control method for field insect pests is provided. The search and identification system in the spraying robot as described in Embodiment 1 is used to identify field insect pests, which includes the following steps:

[0082] Step one, first, the two first depth cameras are individually calibrated for single target, to correct optical distortion in the depth camera imaging process. Specifically:

[0083] Firstly, a chessboard calibration plate whose corner points in the world coordinate system are known is used, and multiple pictures of the chessboard calibration plate are taken by the camera respectively, ensuring that the chessboard calibration plate is at different angles and positions; then, the corner points of the chessboard calibration plate in each picture are found by using a corner detection algorithm (for example: the function cv2.findChessboardCorners in the OpenCV library for detecting the corner points of the chessboard grid); finally, the correspondence between the feature points in the picture and the world coordinate system is used, and the Zhang Zhengyou calibration method (for example: the function cv2.calibrateCamera in the OpenCV library for implementing the Zhang Zhengyou calibration method) is used to calculate the intrinsic matrix and distortion coefficient of each first depth camera, wherein the intrinsic matrix includes the principal point coordinates （c x ,c y ） and focal length f x 、f y , and the distortion coefficient includes the radial distortion coefficient k 1 、k 2 、k 3and tangential distortion coefficient p 1 、p 2.

[0084] Then, binocular stereo vision calibration is performed on the two first depth cameras to obtain the re-projection matrix for binocular correction and the conversion relationship between the pixel distance and the real physical distance; specifically:

[0085] The conversion relationship of the distance is specifically:

[0086] Firstly, the normalized coordinate point （x d ,y d ） of a distorted image point （x n ,y n ） is obtained by using the intrinsic matrix

[0087] ;

[0088] Then, the distorted coordinate point （x c ,y c ） is obtained by using the distortion coefficient

[0089]

[0090] wherein: ;

[0091] Finally, the normalized coordinates are converted back to image coordinates （x u ,y u ） :

[0092]

[0093] Step two, according to the different time and light intensity, the corresponding searchlight recognition system is opened, and the searchlight is provided for the corresponding first depth camera to provide illumination, and the improved YOLOv8 algorithm is used to monitor the crop pests in real time and identify the corresponding types (such as cotton bollworm adults or larvae, other pests harmful to cotton, etc.) and quantity;

[0094] The specific method for real-time monitoring of crop pests by using the improved YOLOv8 algorithm is as follows:

[0095] As shown in Figure 6 , first, the input pest image is preprocessed (such as resizing, cropping, etc.) to adapt to the input size requirements of the model; then, the basic features (such as edges, textures, shapes, etc.) of the image are identified through a series of convolution layers (Conv), batch normalization (Instance Normalization) and activation functions (ELU) of the backbone network Backbon; then, the input image is processed through the C2f module to convert it into a series of features, and this new feature map contains multiple features of the input image, which can be used for subsequent processing and analysis; then, the extracted features are further processed through the neck network Neck, which includes more convolution layers, pooling layers (Max Pool2d) and up-sampling layers (Upsample), which help the network to fuse features of different scales to better identify the targets in the image; then, the head network Head Detection is used to detect the targets in the image; the specific representation is as follows:

[0096] ;

[0097] In the formula: X represents the input image; F represents the feature extraction function (composed of Backbone and Neck); P represents the prediction function (i.e. the Head part); Y represents the model output, including the coordinates, size, confidence and class probability of the bounding box, etc.

[0098] During the entire training process, the improved YOLOv8 algorithm uses a loss function to guide the learning process, which is as follows:

[0099]

[0100] In the formula: represents the predicted value of the model, Y t represents the true label; represents the total loss function of the model, , respectively represent the confidence loss of the target object and the non-target object; represents the category loss; represents the bounding box loss; respectively represent the hyperparameters for balancing different loss terms.

[0101] The specific method for identifying the corresponding species and quantity of pests is as follows:

[0102] First, collect an image dataset containing two types of pests (such as cotton bollworm adults, larvae, or pest A and pest B), denoted as wherein, I i represents a single image; and divide the image dataset into a training set, a validation set, and a test set (the ratio of the training set, the validation set, and the test set is 7:2:1), and then use a labeling tool (any commonly used labeling tool in the art can be used, and the present embodiment is not limited in this regard) to add a bounding box and a category label for each pest;

[0103] Then, use the labeled image dataset (i.e., the training set) to train the improved YOLOv8 model. During the training process, the YOLOv8 model learns how to detect and classify different pests.

[0104] After that, use the trained YOLOv8 model for target detection to obtain the bounding box and category prediction of the pests. For each detection object, the YOLOv8 model outputs a detection vector wherein, represents the coordinates and size of the bounding box, s represents the confidence score;

[0105] Then, perform non-maximum suppression on the detected bounding boxes to remove overlapping bounding boxes and retain the highest confidence bounding box. Specifically:

[0106] Sort all the detected bounding boxes in descending order of confidence scores s If the bounding box is B i , then its corresponding confidence score is s i; select the bounding box with the highest confidence B h and its corresponding confidence score s h , respectively B h and each subsequent bounding box B j ( j>1 ) of the bounding box IoU(B h , B j ) :

[0107] ;

[0108] a preset intersection-over-union threshold IoU d ， If (the intersection-over-union threshold in this embodiment IoU d is 0.5), the bounding box B j and the bounding box B h overlap, the overlapping bounding box is removed, the remaining bounding boxes are updated, and the bounding boxes are sorted; otherwise, the bounding box is retained and the next bounding box is verified, until there are no redundant bounding boxes to be processed.

[0109] Subsequently, the detection structure of the YOLOv8 model is taken as input, the candidate regions between frames are associated through the motion information and appearance features of DeepSORT, and a stable pest trajectory is formed; the first t frame tracked is also a successful trajectory set:

[0110] ;

[0111] wherein, denotes the bounding box and class information of the j th trajectory in the time window; M t denotes the number of currently active trajectories;

[0112] ;

[0113] In the formula: denotes the coordinates of the th detected bounding box in the corresponding frame at time point k ; denotes the class confidence corresponding to the bounding box in the corresponding frame at time point , represents the current time point, i.e. the rightmost or latest time point in the time window under discussion; represents the length of the time window;

[0114] Finally, for each trajectory , the category confidence of all frames contained therein C x , the confidence threshold of each pest category is set C d , and the pest category is determined:

[0115] If , the pest of the trajectory belongs to the corresponding category; otherwise, it does not belong to the corresponding category;

[0116] The categories of each trajectory are counted, and the number of two types of pests C a 、C b is calculated respectively N a 、N b :

[0117] ;

[0118] In the formula: represents the indicator function, which takes the value 1 when the condition is true, and 0 otherwise.

[0119] Step three, obtain the two-dimensional coordinates of the pest image obtained in step two, and convert them to three-dimensional space coordinates; specifically:

[0120] First, obtain the two-dimensional coordinates of the pest detection box center point in the pixel coordinate system (x o ,y o ) :

[0121] ;

[0122] In the formula: (x r ,y r )、(x l ,y l ) respectively represent the coordinates of the top left corner and the bottom right corner of the detection box;

[0123] Then, through the spatial mapping relationship between the pixel coordinate system and the camera coordinate system, the three-dimensional space coordinate conversion of the pest is completed:

[0124]

[0125] In the formula: (x t ,y t ,z t ) represents the three-dimensional space coordinates; z de represents the current point depth value obtained by the camera;

[0126] In order to reduce the adverse effects of adverse conditions such as light on the acquisition of camera depth value in the field environment, and realize the accuracy of camera depth value, first, the two-dimensional coordinates of the detection box center point (x o ,y o ) and the depth values of the eight points around it are obtained, denoted as z i i=1,2,…,9 ; Then, the median value of the depth values of the nine points z mid is obtained, and the two-dimensional coordinates of the detection box center point (x o ,y o ) are excluded from the remaining eight depth values, and the median value z mid is subtracted to obtain the comparison value z bi :

[0127] ;

[0128] If , the corresponding depth value is regarded as an abnormal value and is removed;

[0129] Finally, the remaining depth values after removing the abnormal values are averaged with the two-dimensional coordinates of the detection box center point (x o ,y o ) to obtain the depth value of the current point of the camera z de .

[0130] ​Step four, obtain the three-dimensional space coordinates of the proposed deep point (the proposed deep point is a point 3 cm outwardly extended from the position of the innermost pest in the image) by step three, and convert it into three-dimensional space coordinates under the mechanical arm assembly coordinate system (coordinate conversion can be carried out by using existing conventional calibration algorithm), control the corresponding mechanical arm assembly to move; and according to the pest corresponding species and quantity obtained in step two, control the electric water pump to spray the corresponding amount of pesticide.

[0131] Embodiment 3:

[0132] As a further optimization of the scheme of the present application, on the basis of the scheme of embodiment 2, since the crop is affected by the field wind factor, its leaf blade is easy to deviate, and then leads to image acquisition and recognition error, therefore, in the image acquisition process, the deviation of the leaf blade is obtained , and the position compensation in the spraying process is completed:

[0133]

[0134] In the formula: , respectively represent the deviation of the leaf blade in the horizontal direction and the vertical direction, l represents the length of the leaf blade;

[0135] represents the angle deviation,

[0136] In the formula, dt represents the test time interval;

[0137] ;

[0138] In the formula: m represents the weight of the leaf blade; d represents the distance from the wind action point to the crop stem (i.e. cotton rod); represents the air density; A represents the wind action area; C d represents the resistance coefficient of the leaf blade; v represents the wind speed;

[0139] By setting the wind detection sensor in the field, the data of the wind speed v and the wind direction are collected, combined with the wind action area A (i.e. the effective area of the leaf blade) obtained by the first depth camera of the spraying robot, the position deviation of the leaf blade is obtained, so as to realize the compensation of the bionic spray head component 38 in the spraying process.

Claims

1. An identification control method of an identification control spraying robot for field insect pests, characterized by: The spraying robot comprises a search and identification system, a walking mechanism, a mechanical arm assembly, an electric water pump, a central control system, a liquid tank, a chassis and a sensing system; the walking mechanism is arranged on the bottom surface of the chassis; the liquid tank is fixedly arranged on the front side of the end surface of the chassis; the search and identification system is symmetrically arranged on the end surface of the chassis and located on both sides of the liquid tank; two electric water pumps are symmetrically arranged on the end surface of the chassis and located at the rear end of the liquid tank, and the two electric water pumps are in communication with the liquid tank; the mechanical arm assembly is symmetrically arranged on the end surface of the chassis and located on the side of the electric water pump away from the liquid tank, and the central control system is fixedly arranged between the two sets of mechanical arm assemblies; the electric water pump is in communication with the bionic spray head component arranged at the end of the corresponding mechanical arm assembly; the sensing system is arranged on the front end of the chassis; the search and identification system, the walking mechanism, the mechanical arm assembly, the electric water pump and the sensing system are electrically connected with the central control system; the search and identification system comprises a first depth camera, a camera support, a system fixing seat, a dust cover and a first search lamp; the system fixing seat is fixedly arranged on the end surface of the chassis and located on both sides of the liquid tank; two first depth cameras are uniformly arranged on the vertical side of the system fixing seat located away from the liquid tank through the camera support, and the dust cover is arranged on the outer circle of the first depth camera; the first search lamp is arranged on the upper end of the system fixing seat located away from the liquid tank. The specific identification control method comprises: Step 1: First, single target calibration is performed on the two first depth cameras to correct optical distortion in the imaging process of the depth camera; then, binocular stereo vision calibration is performed on the two first depth cameras to obtain a re-projection matrix for binocular correction and a conversion relationship between pixel distance and real physical distance; Step 2: According to the difference in time and light intensity, the search and identification system is turned on to provide illumination for the corresponding first depth camera, and the improved YOLOv8 algorithm is used to monitor the crop pests in real time and identify the corresponding types and quantities of the pests; The specific method for monitoring crop pests in real time using the improved YOLOv8 algorithm is as follows: First, the input pest image is preprocessed to meet the input size requirements of the model; then, the basic features of the image are identified through a series of convolution layers, batch normalization and activation functions of the backbone network; then, the input image is processed through the C2f module to convert it into a series of features; then, the extracted features are further processed through the neck network; then, the head detection network is used to detect the targets in the image; the specific expression is as follows: ； In the formula: X represents an input image; F represents a feature extraction function; P represents a prediction function; Y represents a model output, including the coordinates, size, confidence and class probability of the bounding box; During the entire training process, the improved YOLOv8 algorithm uses a loss function to guide the learning process, which is as follows: In the formula: represents the prediction value of the model, Y t represents the real label; represents the total loss function of the model, , respectively represents the confidence loss of the target and non-target object; represents the category loss; represents the bounding box loss; respectively represents the hyperparameter for balancing different loss terms; The specific method for identifying the types and quantities of pests is as follows: First, an image dataset containing two kinds of insect pests, denoted as wherein, I i denotes a single image; and the image dataset is divided into a training set, a validation set, and a test set, and a labeling tool is used to add a bounding box and a class label for each insect pest; Then, the improved YOLOv8 model is trained using the labeled image dataset; during the training process, the YOLOv8 model learns how to detect and classify different pests; After that, the trained YOLOv8 model is used for target detection to obtain the boundary box and class prediction of the pest; for each detection object, the YOLOv8 model outputs a detection vector wherein, represents the coordinates and size of the boundary box, s represents the confidence score; Then, non-maximum suppression is performed on the detected bounding boxes to remove overlapping bounding boxes and retain the highest confidence bounding boxes. Subsequently, the detection structure of the YOLOv8 model is taken as input, the candidate regions between frames are associated through the motion information and appearance features of DeepSORT, and a stable pest trajectory is formed. t The successful trajectory set tracked by the frame is: ； wherein, representing the number of tracks in the time window; j representing the bounding box and class information of the track in the time window; M t representing the number of currently active tracks; ； In the formula: Indicates a point in time In the corresponding frame, the first k The coordinates of the detected bounding boxes; Indicates a point in time In the corresponding frame, corresponding to the bounding box Category confidence, This indicates the current time point, which is either the rightmost or latest time point in the time window being discussed. Indicates the length of the time window; Finally, for each trajectory , the class confidence of all frames it contains C x , by setting a confidence threshold for each pest class C d , the pest class is determined: If then the infestation of the trajectory belongs to the corresponding class; otherwise, it does not. Statistics of each trajectory class, respectively, calculate the number of two kinds of pests C a 、C b of quantity N a 、N b : ； In the formula: denotes an indicator function that takes the value 1 when the condition is true and 0 otherwise. Step three, through the pest image obtained in step two, obtain the two-dimensional coordinates of the pest, and convert it into three-dimensional space coordinates; Step four, through step three, obtain the three-dimensional space coordinates of the proposed in-depth point, and convert it into three-dimensional space coordinates under the coordinate system of the mechanical arm assembly, control the corresponding mechanical arm assembly to move; and according to the type and quantity of the pest obtained in step two, control the electric water pump to spray the corresponding amount of pesticide.

2. The identification control method of the identification control robot for field pest according to claim 1, characterized in that: The mechanical arm assembly comprises a rotating base, a support table, a servo motor, a first mechanical arm, a connecting rod mechanism, a second mechanical arm, a control motor and a bionic spray head component, the rotating base is rotatably connected with the end surface of the chassis, and the end surface of the rotating base is fixedly provided with the support table; the first mechanical arm is rotatably arranged at the end surface of the support table, and one end of the first mechanical arm away from the support table is rotatably connected with the second mechanical arm; the connecting rod mechanism is rotatably arranged at the rear end of the second mechanical arm, and one end of the connecting rod mechanism away from the second mechanical arm is rotatably connected with the rotating shaft of the first mechanical arm; the servo motor is fixedly arranged on the support table and used for controlling the rotation of the first mechanical arm; the bionic spray head component is arranged on one end of the second mechanical arm away from the corresponding connecting rod mechanism through a fixed support, and the control motor is arranged on the fixed support corresponding to the bionic spray head component.

3. The identification control method of the identification control robot for field pest according to claim 1 or 2, characterized in that: The bionic spray head component comprises a spray head setting plate, a spray head, a plurality of flange plates, a plurality of ball cage transmission joints, a communication hose and a driving steel wire, a plurality of spray heads are uniformly arranged on the spray head setting plate, and the spray head setting plate and the fixed support are connected through a plurality of flange plates and a plurality of ball cage transmission joints arranged at intervals; the flange plates connected with the spray head setting plate and the fixed support are all flange plates; the flange plates corresponding to the spray heads are provided with hose holes in the outer circle of the ball cage transmission joints, and the fixed support is provided with corresponding hose holes; one end of the communication hose is communicated with the corresponding spray head, and the other end of the communication hose is communicated with the corresponding electric water pump after penetrating the hose holes of each flange plate and the hose holes of the fixed support; four steel wire holes are arranged in the outer circle of the flange plates, and corresponding steel wire holes are arranged on the fixed support; the control motor arranged on the fixed support is four, and the driving steel wire is arranged corresponding to the control motor, that is, the driving steel wire is four; one end of the driving steel wire is wound on the output shaft of the corresponding control motor, and the other end of the driving steel wire is fixedly connected with the flange plate close to the spray head setting plate after penetrating the corresponding steel wire holes of the fixed support and the corresponding steel wire holes of each flange plate in sequence.

4. The identification control method of the identification control robot for field pest according to claim 1, characterized in that: The specific steps for correcting optical distortion in the imaging process of the depth camera in step one are as follows: Firstly, a chessboard calibration board whose corner points are known in the world coordinate system is used, and multiple pictures of the chessboard calibration board are taken by the camera respectively, ensuring that the chessboard calibration board is at different angles and positions; then, a corner point detection algorithm is used to find the corner points of the chessboard calibration board in each picture; finally, the corresponding relationship between the feature points in the picture and the world coordinate system is used, and the Zhang Zhengyou calibration method is used to calculate the intrinsic matrix and distortion coefficient of each first depth camera, wherein the intrinsic matrix includes principal point coordinates （c x ,c y ） and focal length f x 、f y , and the distortion coefficient includes radial distortion coefficient k 1 、k 2 、k 3 and tangential distortion coefficient p 1 、 p 2.

5. The identification control method of the identification control robot for field pest according to claim 4, characterized in that: The specific steps for correcting optical distortion in the imaging process of the depth camera in step one are as follows: First, a distorted image point is acquired by an internal reference matrix （x d ,y d ） of a normalized coordinate point （x n ,y n ） : ； Then, the distortion coefficient is used to obtain the distorted coordinate points （x c ,y c ） : wherein: ; Finally, the normalized coordinates are converted back to image coordinates （x u ,y u ） :

6. The identification control method of the identification control robot for field pest according to claim 5, characterized in that: The specific steps for correcting optical distortion in the imaging process of the depth camera in step one are as follows: all the detected bounding boxes are sorted according to the confidence scores s in descending order, if the bounding box is B i , the corresponding confidence score is s i ; the bounding box with the highest confidence score B h and the corresponding confidence score s h are selected B h and the intersection over union of each subsequent bounding box B j is calculated IoU(B h , B j ) : ； Pre-set intersection ratio threshold IoU d ， If , the bounding box B j and B h overlap, remove the overlapping bounding box, update the removed bounding box and sort; On the contrary, it is retained and the next boundary box verification is carried out, until there is no redundant boundary box to be processed.

7. The identification control method of the identification control robot for field pest according to claim 5, characterized in that: The specific method for converting the two-dimensional coordinates of the pest into three-dimensional space coordinates in step three is as follows: First, the two-dimensional coordinates of the center point of the pest detection frame in the pixel coordinate system are obtained (x o ,y o ) : ； In the formula: (x r ,y r )、(x l ,y l ) respectively represent the coordinate points of the upper left corner and the lower right corner of the detection frame; Then, through the space mapping relationship between the pixel coordinate system and the camera coordinate system, the three-dimensional space coordinate conversion of the pest is completed: In the formula: (x t ,y t ,z t ) represents a three-dimensional space coordinate; z de represents a current point depth value acquired by the camera; Synchronously acquire the two-dimensional coordinates of the center point of the detection frame (x o ,y o ) and the depth values of the eight points around it, denoted as z i ( i=1, 2,…,9 ); then, acquire the median value of the depth values of the nine points z mid , and the two-dimensional coordinates of the center point of the detection frame (x o ,y o ) , respectively subtract the remaining eight depth values from the depth value of the center point of the detection frame to obtain comparison values z mid z bi :​ ； If then the corresponding depth value is considered an outlier and is removed; Finally, the depth values remaining after removing outliers are averaged with the two-dimensional coordinates of the center point of the detection frame (x o ,y o ) The corresponding depth values are averaged to obtain the depth value of the current point of the camera z de .

Images