Plant disease and insect pest monitoring robot and application method thereof

Abstract

The invention discloses a plant disease and insect pest monitoring robot and an application method thereof, and relates to the technical field of intelligent agricultural robotics.The plant disease and insect pest monitoring robot comprises a robot body, and a moving assembly is arranged on one side of the robot body; the identification module is mounted at the top of the robot body; the insect catching mechanism and the spraying mechanism are both slidably mounted on the moving assembly. Wherein the identification module is used for identifying leaves with plant diseases and insect pests, calculating the distance between the leaves with the plant diseases and insect pests and the identification module, and determining the positions of the plant diseases and insect pests according to the distance; the moving assembly is used for sequentially moving the insect catching mechanism and the spraying mechanism to the positions where diseases and insect pests are located according to the distance, and deinsectization and pesticide spraying treatment are sequentially conducted on the leaves; according to the device, the automatic detection process can be achieved, the target position can be accurately recognized in the complex environment, and accurate deinsectization and pesticide spraying are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent agricultural robots, and in particular to a plant disease and insect pest monitoring robot and an application method thereof. Background Art

[0002] In recent years, fruit tree diseases and insect pests in many areas of my country have become more serious year by year. The spread of diseases and insect pests will directly affect the safety of trees and the quality of products, and may even cause consumer panic and affect industrial benefits.

[0003] With the continuous growth of industrialization and the advancement of agricultural modernization, agricultural spraying machines have gradually emerged, which have improved the shortcomings of low efficiency and uneven spraying of manual spraying; Patent CN115104588A proposes a spraying machine that can identify leaf pests and diseases and automatically spray liquid medicine. The recognition module is used to identify leaf pests and diseases in the farmland area, and different liquid medicines are used for spraying according to different types of pests and diseases; Patent CN110703661A discloses a crop pest and disease monitoring system, which uses drones to take aerial photos of crops, analyzes the aerial images, and uses robots to spray pesticides in the area when pests and diseases are detected.

[0004] However, the pest and disease monitoring methods currently used are mainly fixed-area monitoring, and pesticides are sprayed on a large scale in areas where pests and diseases occur. It is difficult to accurately locate the specific location where pests and diseases exist in a complex environment, and it is impossible to carry out precise pest control and spraying of pests and diseases. Summary of the invention

[0005] In view of the shortcomings of the prior art that most of the fixed monitoring stations are difficult to accurately locate in complex environments and cannot accurately disinfest and spray pests, the present invention proposes a plant pest and disease monitoring robot and an application method thereof. The leaves of the plant with pests and diseases are identified by an identification module, and the distance between the leaves with pests and diseases and the identification module is calculated to determine the location of the pests and diseases. According to the distance, the insect catching mechanism and the spraying mechanism are moved to the location of the pests and diseases in turn, and the leaves are disinfested and sprayed with pesticides in turn, thereby solving the problems existing in the prior art.

[0006] A plant pest monitoring robot, comprising:

[0007] A robot body, one side of which is provided with a moving component;

[0008] An identification module is installed on the top of the robot body;

[0009] The insect catching mechanism and the spraying mechanism are both slidably mounted on the moving assembly;

[0010] Among them, the recognition module is used to identify the leaves of the plant with pests and diseases, and calculate the distance between the leaves with pests and diseases and the recognition module, and determine the location of the pests and diseases based on the distance; the moving component is used to move the insect catching mechanism and the spraying mechanism to the location of the pests and diseases in turn according to the distance, and perform insect removal and pesticide spraying on the leaves in turn.

[0011] Furthermore, the recognition module adopts an openmv H7 plus camera.

[0012] Furthermore, the insect catching mechanism includes a vacuum pump, a filter screen and a suction cup; the vacuum pump is connected to the suction cup through a hose, the suction cup is provided with a filter screen, and a collecting bowl is provided under the suction cup.

[0013] Furthermore, a method for applying a plant pest monitoring robot comprises the following steps:

[0014] The fomo algorithm is used to identify the leaves of the plant with pests and diseases through the recognition module, and the pnp algorithm is used to calculate the distance between the leaves with pests and diseases and the recognition module; the location of the pests and diseases is determined according to the distance;

[0015] The insect catching mechanism is moved to the location of the pests and diseases by the moving component to eliminate the pests;

[0016] The spraying mechanism is moved to the top of the insect-free leaves by the moving assembly to spray the pesticide.

[0017] Furthermore, the method of using the fomo algorithm to identify leaves of plants with pests and diseases specifically includes the following steps:

[0018] Collecting image data of plants; the image data includes normal plants and plants with different types of pests and diseases;

[0019] Preprocessing the image data set; the preprocessing includes image enhancement, denoising and cropping;

[0020] Extracting features from the preprocessed image data set; the features include color features, texture features and shape features;

[0021] The extracted features are used as input to train the deep neural network model to build a pest and disease recognition model;

[0022] The image data to be detected is input into the pest and disease recognition model, and the recognition image of the pest and disease is output.

[0023] Furthermore, the method of using the pnp algorithm to calculate the distance between the leaf with pests and diseases and the identification module specifically includes the following steps:

[0024] Extract the characteristic points of leaf pests and diseases in the pest and disease recognition image;

[0025] Match the extracted feature points with the corresponding points in the 3D target model to solve the camera's position information;

[0026] Based on the camera's position information and the information of the three-dimensional target model, the posture information of the leaves with pests and diseases can be inferred, and then the location of the pests and diseases can be determined.

[0027] Furthermore, the insect catching mechanism is moved to the location of the pests by the moving component to remove the pests, and the pests are removed after the moving component is driven by the motor to reach the location of the pests on the leaves. The pest removal process specifically includes the following steps:

[0028] According to the signal of leaf pests and diseases recognized by the recognition module, a command is sent to the mobile component, so that the mobile component drives the insect catching mechanism to move to the location of the pests and diseases;

[0029] By driving the vacuum pump in the insect catching mechanism, the rotor in the vacuum pump rotates continuously, and the pumped gas is sucked into the space between the rotor and the pump housing from the air inlet, and then discharged through the exhaust port;

[0030] When the gas is discharged from the pump, a pressure difference is formed on both sides of the suction cup, and the pests are sucked onto the suction cup through the pressure difference;

[0031] Move the suction cup to the top of the collection box, then control the vacuum pump to close, so that the pests fall into the collection box, completing the pest control.

[0032] Furthermore, the method of moving the spraying mechanism to the top of the insect-free leaves to spray the pesticide by the moving assembly specifically includes the following steps:

[0033] Based on the recognition images of pests and diseases, a fuzzy algorithm is used to estimate the degree of plant pests and diseases and the corresponding amount of pesticide spraying;

[0034] Based on the estimation results, the spraying organization is controlled to spray pesticides at the locations where plant pests and diseases are located.

[0035] Furthermore, it also includes using a thin film pressure sensor to monitor the remaining amount of pesticide in the pesticide storage in the spraying mechanism in real time. By building a pressure sensor into the pesticide storage, when the liquid level is lower than the set value, the microcontroller sends a corresponding instruction to issue a refill warning.

[0036] The present invention provides a plant disease and insect pest monitoring robot and an application method thereof, which have the following beneficial effects:

[0037] The present invention adopts an identification module to identify the leaves of plants with pests and diseases and calculates the distance between the leaves with pests and diseases and the identification module, and then determines the location of the pests and diseases, and moves to the location of the pests and diseases through an insect catching mechanism and a spraying mechanism in turn, and performs pest control and pesticide spraying on the leaves in turn; the present invention can accurately identify the location of leaves with pests and diseases in a complex environment, discover pests and diseases in time, and flexibly adjust the moving components according to the pest and disease situation, control the spraying mechanism and the insect catching mechanism to achieve accurate pest control and pesticide spraying, reduce the damage of pests and diseases to crops, and improve the yield and quality of crops. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of the side structure of the plant protection robot in an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of the plant protection robot in an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of the structure of the XYZ three-axis mechanical mobile platform in an embodiment of the present invention.

[0041] In the attached figure: 1-camera, 2-insect trap, 3-Y-axis slide, 4-vacuum pump, 5-medicine box, 6-bracket, 7-chassis, 8-X-axis slide, 9-Z-axis slide, 10-collector, 11-sprayer, 12-water pump, 13-driver, 14-frame. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0043] The present invention proposes a plant pest monitoring robot, the robot body includes an identification module, a control device (driver 13), a propulsion device (robot chassis), a mobile component (XYZ three-axis mechanical mobile platform), an insect catching mechanism and a spraying mechanism; Figure 1 , 2As shown, the robot body is composed of a frame 14 and a robot chassis, the propulsion device includes a robot chassis (tracked chassis 7) and a track motor; the tracked chassis 7 is fixed to the bottom of the frame 14, and the track motor is used to drive the tracked chassis to move; the insect catching mechanism includes an XYZ three-axis mechanical mobile platform, a vacuum pump 4, an insect catcher 2 and a collector 10; the spraying mechanism includes a medicine box 5, a water pump 12 and a sprayer 11; wherein the two Z-axis slides 9 of the XYZ three-axis mechanical mobile platform are connected to one side of the frame 14 through aluminum profile angle brackets, and the Y-axis slide 3 is connected to the moving devices on the two Z-axis slides 9 at the same time, and the X-axis slide 8 is connected to the Y-axis slide 3, which can realize free movement in three-dimensional space, and the insect trap and sprayer installed on the X-axis slide are accurately moved to the location of the pests and diseases to realize the pest removal function. The collector 10 is placed below the insect trap 2 corresponding to the insect trap 2 to collect the captured pests; the identification camera 1 is installed on the top of the frame to monitor and identify the fruit tree pests and diseases and their locations. The medicine box 5 is placed on the other side of the frame 14 through the bracket 6 and is connected to the sprayer 11 through a hose.

[0044] (1) Control device: responsible for the management of machine objectives and functions, including the management system, information processing system, and information node system of wireless sensors.

[0045] (2) Propulsion device: A rubber crawler chassis is used for propulsion. Since crawler travel mechanisms are widely used in field operation vehicles such as engineering machinery and tractors, they can quickly adapt to various road conditions and have high passability. The crawler travel device consists of "four wheels and one belt" (driving wheel, supporting wheel, guide wheel, traction wheel and crawler), tensioning device and buffer spring, and travel mechanism. The crawler chassis drive design improves the stability and flexibility of the plant protection robot, and also improves the forward speed and carrying capacity, so that the operation can be completed more smoothly and efficiently. The machine moves along the designated route in the orchard. Considering the complexity of the field environment, the present invention adopts a solution that allows the chassis to move along the laid metal rails. The self-made circular induction coil is connected to the inductance digital converter LCD1000, and the circular induction coil is installed at the bottom of the chassis and placed vertically above the metal rail. When the rail deviates from the center position of the circular induction coil, the inductance of LCD1000 will change. By comparing the sensor measurement value with the preset center value, it can be determined whether the chassis deviates from the track, and adjustments can be made.

[0046] (3) Robotic arm: It includes three-coordinate telescopic robotic arm, control chip, drive circuit and other parts. It mainly obtains the position of the target object through the image information transmitted by the recognition system, and sucks the pests by driving the robotic arm. The visual robotic arm uses the intelligent recognition camera to identify the position of the pests on the leaves and take pictures. It corresponds to the pest monitoring library and identifies the pest species according to the pest color, size, shape and other characteristics, and transmits the data back to the background for statistics. The computer analyzes it and provides farmers with suggestions on prevention and eradication of pests through the supporting APP; the plant protection wizard can be equipped with different insecticide modules, such as installing an insect sucker under the visual robotic arm, whose powerful suction can accurately kill pests. Or install a pesticide nozzle under the visual robotic arm, and perform quantitative spraying of pesticides based on the data from the recognition camera to kill insects efficiently; the visual robotic arm samples and takes pictures of plant leaves, and the machine automatically recognizes the photos. It determines what disease the plant has and what fertilizer it lacks based on information such as leaf color and edge shape, and transmits the data back to the background, providing farmers with treatment plans or fertilizer ratios through the APP; the location of diseased plants is determined based on the location information transmitted by the recognition camera, and compound fertilizers or nutrients are sprayed on the fruit trees to supplement the elements that the plants lack.

[0047] (4) Spraying mechanism: water pump, water tank, nozzle and water pipe, the water tank is connected to the water pump, the water pump is connected to the water pipe, the water pipe is connected to the nozzle, and the water pump and the water pipe are connected to the control device. The spraying mechanism sprays pesticides according to the settings of the control device.

[0048] (5) The insect trapping mechanism consists of a vacuum pump, a hose, a filter, a suction cup, and a collector. After the high-definition camera identifies the pest, it sends a signal to the single-chip microcomputer, which sends instructions to the switch and the stepper motor of the XYZ three-axis synchronous belt pulley to move the mechanical platform to a place close to the pest. At the same time, it controls the switch of the vacuum pump to turn on. The rotor in the vacuum pump rotates continuously, and the pumped gas is sucked from the air inlet into the space v0 between the rotor and the pump housing, and then discharged through the exhaust port. Because the space v0 is fully closed after suction, there is no compression and expansion of the gas in the pump chamber. However, when the top of the rotor turns over the edge of the exhaust port and the space v0 is connected to the exhaust side, because the gas pressure on the exhaust side is higher, part of the gas will rush back into the space v0, causing the gas pressure to suddenly increase. When the rotor continues to run, the gas is discharged out of the pump to form a pressure difference on both sides of the suction cup, and the pest is pressed on the suction cup by the pressure difference. At the same time, due to the barrier of the filter, the pest cannot be sucked into the pipeline. After the suction is successful, the single-chip computer sends a signal to the stepper motor again, so that the XYZ three-axis mechanical platform moves to the top of the collection box, and then the single-chip computer sends a signal to the vacuum pump switch to turn off the vacuum pump, allowing the pests to fall into the collection box. Compared with traditional manual capture, the intelligent pest suction module has the characteristics of high efficiency, high precision and strong pertinence.

[0049] Start the plant protection robot, and power the whole device through the power management board. When the user chooses to use the autonomous operation series mode, the user only needs to set the operation content of the robot required by the user end. After confirming that it starts to perform the operation, the plant protection elf will run autonomously, and automatically cut off the power and stop working after completing the operation content, reducing unnecessary energy waste. When the user chooses to use the human-computer interaction series mode, the user can control the operation status of the plant protection elf by remote control with the assistance of the robot arm recognition camera, and control the robot to complete the operation accurately according to his own requirements. The mechanical part is mainly composed of XYZ three-axis mechanical movement module, pest suction module, pesticide spraying module and propulsion module. The XYZ three-axis mechanical movement module consists of four synchronous belt slides, which can realize the free movement of the mechanical platform in the three directions of XYZ. The pest suction module consists of a high-definition camera, a vacuum pump, a hose, a filter, a suction cup and a collection box. After identifying the pest, the device will automatically move to the location of the pest. After the suction cup absorbs the pest, it will be placed in the collection box, thereby realizing the pest removal function. The pesticide spraying module consists of a water pump, a nozzle and a medicine storage box, and can spray pesticides according to the condition of the leaves. The propulsion module is mainly composed of a crawler chassis. The entire mechanism is mounted on the crawler chassis to enable the entire mechanism to move in the target direction. The XYZ three-degree-of-freedom mechanical mobile platform consists of four synchronous belt linear module slides, such as Figure 3As shown. The X-axis synchronous belt slide is installed with two Z-axis synchronous belt slides through a bracket. One of the two Z-axis synchronous belt slides is equipped with a stepper motor as the active slide, and the other Z-axis synchronous belt slide is not equipped with a stepper motor and is used as a driven slide. The active slide and the driven slide are installed together on both sides of the X-axis slide, which can greatly improve the stability of the mechanical mobile platform. The Y-axis synchronous belt slide is installed with the X-axis synchronous belt slide through a bracket. The microcontroller sends a pulse signal to the stepper motor. The stepper motor drives the active wheel of the synchronous belt to rotate, and then the belt drives the slider on the linear guide to move back and forth, thereby realizing the free movement of the mechanical platform in the three directions of XYZ. The XYZ three-degree-of-freedom mechanical mobile platform has the characteristics of high precision, high speed, high versatility, and simple algorithm. Different modules can be installed on the platform to achieve different functions. The XYZ three-axis robotic arm equipped at the front end of this robot extracts the spatial position into three directions, and replaces the expensive and complex single multi-joint robotic arm with three simple and inexpensive mechanical slides. Compared with the traditional multi-joint robotic arm, it has the characteristics of simplicity, efficiency, fast positioning, and easy maintenance, and reduces the production cost, thereby reducing the cost of agricultural mechanization and alleviating the economic burden of farmers. The three-axis robotic arm simplifies the complex while ensuring smooth and accurate motion trajectory and flexible motion performance, and accurately and stably reaches the target position in a complex environment. The monitoring camera on the top of the machine uses the computer vision technology mentioned above, with accurate recognition and good vision. The insect trap on the robotic arm can accurately capture pests and effectively reduce the number of pests; the pesticide spraying mechanism next to it can accurately spray pesticides, reduce pesticide waste, and reduce pollution to the environment. This structural distribution method not only enables the robot to observe the growth of plants in all directions and detect pests and diseases in time, but also can flexibly adjust the posture of the robotic arm according to different pest and disease conditions to achieve accurate pest control and spraying. In addition, the robot can cruise autonomously to carry out continuous and comprehensive pest and disease monitoring of plants. Through the built-in intelligent algorithm, the robot can autonomously adjust its working strategy according to the pest and disease situation to achieve automated and intelligent management. In addition, the robot can also communicate with the remote monitoring center in real time to provide remote support and guidance to farmers. The plant pest and disease monitoring robot of the present invention can significantly improve agricultural production efficiency. Through automated and intelligent management methods, the workload of manual inspections and prevention and control is reduced, and the labor cost is reduced. At the same time, due to the robot's precise pest control and spraying, the prevention and control effect can be improved, and the damage to crops caused by pests and diseases can be reduced, thereby increasing the yield and quality of crops.

[0050] Based on the same inventive concept, the present invention proposes an application method of a plant pest monitoring robot, comprising the following steps:

[0051] Step 1. Intelligent identification technology for pests and diseases, and design of intelligent clear identification camera. Data collection is performed through camera identification. The camera used is openmv H7 plus camera, and the target is tracked through openmv's built-in fomo algorithm, and simple target distance positioning is performed through pnp, and control is performed through serial port communication; the specific steps include:

[0052] (1) Data collection: Collect a large number of image datasets containing different plant pests and diseases. These images should include normal plants and different types of plants with pests and diseases.

[0053] (2) Image preprocessing: The collected images are preprocessed, including image enhancement, denoising, cropping, etc. The purpose of preprocessing is to reduce noise, eliminate background interference, improve image quality, and make the image more suitable for subsequent feature extraction and classification.

[0054] (3) Feature extraction: Extract features from the preprocessed image to classify different types of pests and diseases. Commonly used feature extraction methods include color features, texture features, shape features, etc.

[0055] (4) Model training: The extracted features are used as the input of the model, and machine learning or deep learning algorithms are used to train the model, thereby achieving automatic classification of plant diseases. Commonly used machine learning algorithms include support vector machines (SVM), decision trees, random forests, etc. Commonly used deep learning algorithms include convolutional neural networks (CNN), recurrent neural networks (RNN), etc.

[0056] (5) Application deployment: Deploy the trained model to actual applications to detect pests and diseases on new images. Application deployment can be done in two ways: offline deployment and online deployment. Offline deployment is suitable for static image detection, while online deployment is suitable for real-time video stream detection.

[0057] Considering the efficiency of image processing, the system uses an openmv H7 plus camera for image acquisition. The size of the fruit tree is represented by the black and white ratio in the acquired image, and the fuzzy algorithm is used to obtain the amount of pesticide spraying. According to the solution results, the STM32 microcontroller is used to output an electrical signal to control the spraying mechanism to spray the fruit trees with an appropriate amount of pesticide. In addition, a thin film pressure sensor is used to monitor the remaining amount of pesticide in real time. The pesticide storage device has a built-in pressure sensor. When the liquid level is lower than the set value, the microcontroller sends a corresponding instruction, and the control system returns to the starting point to wait for manual loading. When the battery power is lower than the preset value, an electrical signal will be sent to the user to remind the user to replace the battery or charge it in time. The device is simple and easy to implement. At the same time, the nozzle can be changed in time according to different crop needs, so that one device can be used for multiple purposes; the unmanned vehicle is equipped with a detection and spraying mechanism to achieve precise spraying, greatly improve labor productivity, reduce labor costs, and reduce pesticide pollution and residues.

[0058] The use of computer vision technology for plant pest and disease detection has the following advantages: Compared with traditional manual detection methods, computer vision technology can realize the automated detection process, greatly improving efficiency. Computer vision technology can process and classify a large number of images in a short time, so it has a higher accuracy rate. Computer vision technology does not require sampling or harming plants, and will not affect the normal growth of plants. Collecting more image data sets can expand the application scope of the model and make the model more adaptable.

[0059] Example: The camera used is openmv H7 plus camera.

[0060] Use Faster R-CNN to achieve target detection: Perform relevant pre-training: ① Prepare relevant data sets. ② Build relevant models: Select a suitable convolutional neural network model and modify or customize it as needed. ③ Define loss function and optimizer: Select appropriate loss function and optimizer according to task requirements, such as cross entropy loss and stochastic gradient descent (SGD) optimizer. ④ Train model: Use the data set to train the model and record indicators such as loss and accuracy during training.

[0061] Target tracking is performed through the fomo algorithm that comes with openmv: The specific implementation process is: ① Take about 300-400 training images. The selected images must contain the objects that need to be identified. The training images include relevant images of various environments to facilitate the relevant complexity of the environment; ② Log in to the edge impulse website (Edge Impulse is an end-to-end development platform designed for embedded devices. It can help developers deploy machine learning models to low-power devices). Through the multiple machine learning algorithms provided on this website, it is faster and more convenient to train related models, and because the website supports openmv, it can increase the compatibility of openmv machine learning; ③ By loading the trained model, the target is tracked and classified, and its relevant 2D coordinates in the image are obtained, so that the chassis and mechanical claws can perform relevant direction tracking. Better to complete related tasks.

[0062] 3. Simple target distance positioning through PNP: In order to control costs in the early stage, the PNP algorithm is implemented on OpenMV to measure the target distance to save costs. Moreover, compared with laser ranging and ultrasonic ranging, the PNP algorithm is simple to implement and reduces the corresponding debugging cost. It has better characteristics when the ranging accuracy is not required. Implementation principle:

[0063] (1) Feature point extraction: Extract feature points that can represent objects from the image.

[0064] (2) Feature point matching: Match the extracted feature points with the corresponding points in the 3D target model.

[0065] (3) Solve the camera pose: Solve the camera pose information through the matching relationship of feature points, including the camera's rotation angle and translation vector.

[0066] (4) Posture inversion: Based on the camera pose and the information of the 3D target model, the object’s posture information is inferred, including the object’s rotation matrix and translation vector.

[0067] Since openmv does not have its own solvepnp library, it can only be implemented by self-programming. Since the python library of opencv is open source on github, the method adopted is to pull the relevant open source code from github and import it into the storage of openmv.

[0068] Implementation process:

[0069] (1) Camera calibration: The most commonly used Zhang Zhengyou calibration method is used for relevant calibration. The specific implementation method is to buy a chessboard drawing and extract the square data to calibrate the camera. The purpose of camera calibration is to obtain the camera intrinsic parameter matrix. The camera intrinsic parameter matrix can be extracted by writing existing source code.

[0070] (2) Measure the approximate size of the object to be measured.

[0071] (3) The 2D coordinates are converted into 3D coordinates through the PNP algorithm to achieve ranging.

[0072] 4. Control via serial communication

[0073] (1) Principle of serial communication:

[0074] ①First, STM32F103C8T6 and openmv need to be connected to the same serial communication line, including TX (transmit) and RX (receive) lines.

[0075] ② Before communication begins, you need to ensure that the serial port parameters between the two devices are the same, such as baud rate, number of data bits, check bit, and stop bit.

[0076] ③When STM32F103C8T6 wants to send data to openmv, it will write the data into the serial port send buffer. At this time, the TX line will be set high to indicate that the data is ready to be sent.

[0077] ④After receiving the change of TX line status, openmv will start reading the receive buffer to obtain data. When the reading is completed, openmv will send a confirmation signal to STM32F103C8T6 through the TX line to indicate that the data has been received.

[0078] When openmv sends data to STM32F 103C8T6, its working principle is similar to that of STM32F103C8T6 sending data to openmv, that is, openmv writes the data into the serial send buffer and waits for STM32F 103C8T6 to read and send a confirmation signal.

[0079] In short, serial communication is a hardware-based communication protocol that realizes data transmission between the two parties through hardware signals. STM32F 103C8T6 and openmv can realize data transmission through serial communication protocol, and need to cooperate in parameter setting, send buffer and receive buffer.

[0080] (2) Specific implementation process:

[0081] ① Connect the serial port pins (TX and RX) of stm32f103c8t6 and openmv, and make sure they use the same baud rate and communication parameters such as data bits, stop bits, and parity bits.

[0082] ② Initialize the USART serial port module in stm32f103c8t6 and configure the corresponding parameters. This may involve setting the GPIO pin as the UART function, and selecting parameters such as the baud rate, number of data bits, number of stop bits, and parity bits.

[0083] ③ In the code, use the USART of stm32f103c8t6 to send data to openmv, or receive data from openmv. This can be done using the send and receive functions of USART.

[0084] ④Configure the corresponding serial port parameters in openmv, and use the uart function of the pyb library in the code to send / receive data.

[0085] 5. Use classification algorithms to classify diseases:

[0086] (1) Data collection: First, we need to collect a large number of pictures of plant pests and diseases, including pictures of healthy plants and pictures of plants with pests and diseases. These pictures can be obtained through field photography, online searches, or from professional organizations.

[0087] (2) Data preprocessing: The collected images are preprocessed, including image scaling, cropping, enhancement, etc. Among them, scaling and cropping operations can unify the image size, and enhancement operations can increase the image contrast and saturation, thereby improving image quality and recognizability.

[0088] (3) Feature extraction: The preprocessed image is input into the convolutional neural network, and the feature information of the image is extracted using multi-layer convolution and pooling operations. The convolution operation can extract image features such as edges and textures, while the pooling operation can reduce the number of features and reduce computational complexity.

[0089] (4) Classification and recognition: The feature-extracted image is classified and recognized through a fully connected layer. The fully connected layer can map the image feature vector to the category and output the probability value of each category, thereby determining the category to which the image belongs.

[0090] (5) Model optimization: Adjust the parameters and train the convolutional neural network to optimize the model performance. Model optimization can be performed through cross-validation, learning rate adjustment, regularization and other methods to improve the accuracy and generalization ability of the model.

[0091] Step 2. Robotic arm sucks insects: The robotic arm can fully automatically and accurately identify the location of pests through the image information transmitted by the high-definition camera, and carry out pest control work in a targeted manner. This pest control mode can greatly shorten the time to achieve a good control effect. And it minimizes the pollution of fruit products. Specifically, it uses the stm32f103 microcontroller and the DM542 stepper motor driver to control the 42-phase stepper motor to drive the robotic arm. It is an efficient, accurate and reliable control method that can reach the location of pests and diseases more quickly and accurately.

[0092] Step 3. Intelligent spraying system technology: Through a special sensor module, the detection signal is transmitted to the control device according to the different heights of the crops, and the height of the spray head is adjusted in time, so that the corresponding drugs and nutrient solutions can be sprayed accurately and effectively. This allows the environment to develop sustainably and greenly, greatly reducing the harm of pesticides or nutrient solutions to the environment.

[0093] The L298N motor driver chip is a commonly used DC motor driver chip that can control the forward and reverse rotation and speed of the DC motor. It has a low-level logic input, simple control, and convenient circuit connection. The L298N motor driver chip consists of two H bridges, each of which consists of 4 MOS tubes. The output of the H bridge can be used to control the forward and reverse rotation and speed of the DC motor. The power supply voltage of the L298N motor driver chip is usually 5V to 35V.

[0094] A DC water pump is a device that converts electrical energy into mechanical energy to drive water flow. It is suitable for various water circulation systems, water flow control systems and other applications that require water flow drive. The DC water pump can be adjusted and controlled according to demand. Since the principle of spraying medicine is the same as that of water spraying and the experimental conditions are limited, a water pump is used here instead of a plant disease medicine pump as the control object and a technical analysis is carried out.

[0095] Control principle:

[0096] (1) Control principle of L298N motor driver chip: When the level of IN1 pin is different, the H-bridge will output current to the positive or negative pole of the motor through OUT1 pin, thereby controlling the start and stop of the water pump. Specifically, the positive pole of the water pump is connected to 5V, and the negative pole of the water pump is connected to the circuit controlled by IN1. When the IN1 pin is high level, the water pump stops; when the IN1 pin is low level, the water pump works.

[0097] (2) PWM control principle: PWM (Pulse Width Modulation) control is a common speed regulation method, which controls the power of the water pump by controlling the width of the pulse. Specifically, the duty cycle of the pulse signal output by PWM represents the power of the water pump. The larger the duty cycle, the greater the power of the water pump, and vice versa. PWM control can be achieved through the timer of STM32F103.

[0098] (3) Advantages of using STM32F103 to drive L298N to control the water pump:

[0099] High precision: The STM32F103 is a high-performance microcontroller that provides high-precision timing and control, enabling accurate control of the water pump.

[0100] Versatility: STM32F103 has multiple channels of PWM output, which can easily control the speed of the water pump, and has a rich peripheral interface to meet various application scenarios.

[0101] Low power consumption: STM32F103 uses low power consumption technology, which can effectively reduce system energy consumption and provide longer running time and more stable performance in water pump applications.

[0102] High cost-effectiveness: L298N is a cheap and readily available driver chip. When used with STM32F103, it can achieve a cost-effective and stable water pump control solution.

[0103] Step 4. Through the hardware circuit, the plant protection device is moved along the designated route in the orchard. Considering the complexity of the field environment, the solution of moving the chassis along the laid metal rails is adopted. By accurately detecting the metal rails, whether the chassis deviates from the track, adjustments are made so that the plant protection device can accurately avoid obstacles and reach the crops for operation. The chassis is moved along the laid metal rails, and the homemade circular induction coil is connected to the inductance digital converter LCD1000. The circular induction coil is installed at the bottom of the chassis and placed vertically above the metal rails. When the rail deviates from the center position of the circular induction coil, the inductance of LCD1000 will change. By comparing the sensor measurement value with the preset center value, it can be determined whether the chassis deviates from the track and make adjustments.

[0104] Experimental test:

[0105] Common recognition technology analysis is shown in Table 1:

[0106] Table 1 Common recognition technologies

[0107]

[0108]

[0109] The test environment is shown in Table 2:

[0110] Table 2 Test environment for recognition function

[0111]

[0112] Testing process: simulated fruit trees are placed randomly in the venue, disease characteristics are simulated with paint, and the camera is used to identify the location of the fruit trees and the pests and diseases. The present invention records the number of times the robot successfully identifies the position of the simulated fruit trees and the leaves with pests and diseases as reference data for the experimental test of the present invention, and improves the deficiencies reflected in the experimental process.

[0113] The test results are shown in Table 3:

[0114] Table 3 Identification function test results

[0115]

[0116] Test analysis: From the test data table of this intelligent autonomous camera recognition, it can be concluded that the Plant Protection Wizard can fully autonomously identify pests and diseases for farmers and conduct daily health checks on fruit trees, freeing up manpower. At the same time, the growth of fruit trees can be monitored in all directions without time difference, thereby improving product quality.

[0117] Robotic arm test: Use stm32f103 microcontroller and DM542 stepper motor driver to control 42-phase stepper motor to drive the robotic arm. Driven by the motor, the robotic arm will realize functions such as lifting and moving left and right. Through recognition technology and the movement of the chassis, the Plant Protection Wizard robot will stop near the target object to facilitate the operation of the robotic arm. The robotic arm will move accurately to perform sucker operations or spray pesticides and nutrient solutions on the pests or diseased leaves.

[0118] The test environment is shown in Table 4:

[0119] Table 4 Robotic arm test environment

[0120]

[0121] Test process: Place the pest location of the plastic fruit tree, start the robot, identify the camera for positioning, the chassis moves the plant protection robot to the designated location, the robot arm moves based on the pest identification by the camera, and reaches the location of the pest, the sucker on the robot arm sucks the pest, and the spraying mechanism sprays pesticides and nutrient solution on the pest leaves. Change the location of the pest leaves, record the number of times the robot arm accurately reaches the designated working position and performs the work content, and analyze the above data.

[0122] The test results are shown in Table 5:

[0123] Table 5 Robotic arm test results

[0124]

[0125] Test analysis: From the test data table of the movement of the mechanical arm of the fruit tree intelligent monitoring robot, it can be concluded that the Plant Protection Wizard can enable the mechanical arm to accurately reach the designated position within 5 minutes to facilitate the following work. Compared with manual labor, it saves time and effort. For places that are not convenient for manual operation, the use of the Plant Protection Wizard is safer and more efficient, reducing the huge cost caused by labor costs.

[0126] Drug spray test: Common diseases and treatment methods are as follows.

[0127] Table 6 Common diseases and treatment methods

[0128]

[0129]

[0130] The module includes a tank, a water pump, a nozzle and other structures. The storage tank is placed at the rear of the plant protection robot for spraying. The tank can be filled with pesticides, nutrient solution or compound fertilizer according to the specific needs of the user, and multiple tanks can be matched according to needs to make the machine more versatile. The overall design is reasonable, the tank is not easy to move and damage, and the pressure is high, the spraying efficiency is high, the time is long, and the effect is good; at the same time, a detachable filter is used, which can effectively remove impurities when absorbing water to prevent clogging of the water pipe.

[0131] The test environment is shown in Table 7:

[0132] Table 7 Spraying test environment

[0133]

[0134] Testing process:

[0135] Use gouache to paint on the branches and leaves of the plastic fruit trees according to the specific characteristics of different diseases, and apply color blocks of different colors and sizes to simulate fruit tree diseases. After the camera identifies the disease, the robotic arm moves to the location of the disease, and the water pump receives the command to spray the medicine.

[0136] The test results are shown in Table 8:

[0137] Table 8 Spraying test results

[0138]

[0139] Test analysis: From the test data table of the intelligent medicine box spraying medicine, it can be concluded that the plant protection wizard is easy for farmers to operate. They only need to place the robot in the orchard to control the robot to run flexibly and complete their target functions. There were cases where the spraying of medicine was not successful in the test. The reasons were that the recognition camera did not recognize the corresponding disease and the adjustment was not in place. The camera recognition accuracy and the flexibility of the robot arm operation can be improved in the future.

[0140] Trematode test: A mechanical arm can be equipped with a trematode to facilitate timely removal of pests after the camera finds pests. Due to the strong suction force, pests can be removed efficiently and accurately. The working principle of the trematode is to use an electric motor to drive the blades to rotate at high speed, generate negative air pressure in the sealed shell, and suck in external air from the suction port to create a vacuum in the caisson. The pests will be sucked into the caisson through the pipe. At the same time, a layer of filter is set in the box, and the pests will remain outside the filter. The filtered air will then enter the motor to protect the normal operation of the motor.

[0141] The test environment is shown in Table 9:

[0142] Table 9 Pest control test environment

[0143]

[0144] Test process: Use white fake insects on the branches and leaves of plastic fruit trees to simulate fruit tree pests, and test the first-generation prototype made by our team to remove pests from fruit trees. Place the robot next to the plastic fruit tree, start the robot, and start the recognition camera and mechanical arm. After the camera recognizes the pests, the mechanical arm moves to the location of the pests, and the insect sucker sucks the pests. The pest removal operation is completed with the coordination of multiple devices. Set the operation time to 5 minutes, record the number of recovered pests, and perform data analysis.

[0145] The test results are shown in Table 10: After 5 minutes of testing, the following test data table can be obtained.

[0146] Table 10 Insect removal test results

[0147]

[0148] Test analysis: From the test data table of the fruit tree intelligent monitoring robot in sucking out pests, it can be concluded that the Plant Protection Wizard can remove 8 pests in 1 minute. Compared with manual cleaning or spraying pesticides, using the Plant Protection Wizard is safer and more efficient, reducing the loss caused by labor costs and the environmental pollution caused by pesticides.

[0149] The present invention is highly integrated and provides two series for users to choose from: the first is autonomous operation. After the user sets the operation content and working time of the plant protection robot, the robot will operate autonomously, patrol regularly, and complete the vegetation monitoring work in the specified area and the operation requirements set by the user through intelligent path planning. It is fully autonomous and saves worry and effort; the second is the human-computer interaction mode. The user can control the operation status of the machine through remote control, and can increase the work of accurate insect and disease inspection, fertilizer application, pesticide spraying, etc. according to the needs, improve the efficiency of the machine, and more accurately complete the user's operation needs. Compared with traditional manual detection methods, computer vision technology can realize the automated detection process and greatly improve efficiency; computer vision technology can process and classify a large number of images in a short time, so it has a higher accuracy rate. At the same time, computer vision technology does not need to sample or damage plants, and will not affect the normal growth of plants. Collecting more image data sets can expand the application range of the model and make the model more adaptable.

[0150] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A plant pest monitoring robot, characterized in that: include: A robot body, one side of which is provided with a moving component; An identification module is installed on the top of the robot body; The insect catching mechanism and the spraying mechanism are both slidably mounted on the moving assembly; Among them, the recognition module is used to identify the leaves of the plant with pests and diseases, and calculate the distance between the leaves with pests and diseases and the recognition module, and determine the location of the pests and diseases based on the distance; the moving component is used to move the insect catching mechanism and the spraying mechanism to the location of the pests and diseases in turn according to the distance, and perform insect removal and pesticide spraying on the leaves in turn.

2. A plant pest monitoring robot according to claim 1, characterized in that: The recognition module adopts an openmv H7 plus camera.

3. A plant pest monitoring robot according to claim 2, characterized in that: The insect catching mechanism comprises a vacuum pump, a filter screen and a suction cup; the vacuum pump is connected to the suction cup through a hose, a filter screen is arranged on the suction cup, and a collecting box is arranged under the suction cup.

4. An application method of the plant pest monitoring robot according to any one of claims 1 and 3, characterized in that: The following steps are involved: The fomo algorithm is used to identify the leaves of the plant with pests and diseases through the recognition module, and the pnp algorithm is used to calculate the distance between the leaves with pests and diseases and the recognition module; the location of the pests and diseases is determined according to the distance; The insect catching mechanism is moved to the location of the pests and diseases by the moving component to eliminate the pests; The spraying mechanism is moved to the top of the insect-free leaves by the moving assembly to spray the pesticide.

5. The application method of the plant pest monitoring robot according to claim 4, characterized in that: The method of using the fomo algorithm to identify leaves of plants with pests and diseases specifically includes the following steps: Collecting image data of plants; the image data includes normal plants and plants with different types of pests and diseases; Preprocessing the image data set; the preprocessing includes image enhancement, denoising and cropping; Extracting features from the preprocessed image data set; the features include color features, texture features and shape features; The extracted features are used as input to train the deep neural network model to build a pest and disease recognition model; The image data to be detected is input into the pest and disease recognition model, and the recognition image of the pest and disease is output.

6. The application method of the plant pest monitoring robot according to claim 5, characterized in that: The pnp algorithm is used to calculate the distance between the leaf with pests and diseases and the identification module; The specific steps include: Extract the characteristic points of leaf pests and diseases in the pest and disease recognition image; Match the extracted feature points with the corresponding points in the 3D target model to solve the camera's position information; Based on the camera's position information and the information of the three-dimensional target model, the posture information of the leaves with pests and diseases can be inferred, and then the location of the pests and diseases can be determined.

7. The application method of the plant pest monitoring robot according to claim 6, characterized in that: The insect trapping mechanism is moved to the location of the pests and diseases by the mobile component to remove the pests, and the pest removal is performed after the mobile component is driven by the motor to reach the location of the pests and diseases on the leaves. The pest removal process specifically includes the following steps: According to the signal of leaf pests and diseases recognized by the recognition module, a command is sent to the mobile component, so that the mobile component drives the insect catching mechanism to move to the location of the pests and diseases; By driving the vacuum pump in the insect catching mechanism, the rotor in the vacuum pump rotates continuously, and the pumped gas is sucked into the space between the rotor and the pump housing from the air inlet, and then discharged through the exhaust port; When the gas is discharged from the pump, a pressure difference is formed on both sides of the suction cup, and the pests are sucked onto the suction cup through the pressure difference; Move the suction cup to the top of the collection box, then control the vacuum pump to close, so that the pests fall into the collection box, completing the pest control.

8. The application method of the plant pest monitoring robot according to claim 5, characterized in that: The method of moving the spraying mechanism to the upper part of the leaves after insecticide removal by the moving assembly to spray the pesticide specifically comprises the following steps: Based on the recognition images of pests and diseases, a fuzzy algorithm is used to estimate the degree of plant pests and diseases and the corresponding amount of pesticide spraying; Based on the estimation results, the spraying organization is controlled to spray pesticides at the locations where plant pests and diseases are located.

9. The application method of the plant pest monitoring robot according to claim 4, characterized in that: It also includes the use of a thin film pressure sensor to monitor the remaining amount of pesticide in the pesticide storage device in the spraying mechanism in real time. By building a pressure sensor into the pesticide storage device, when the liquid level is lower than the set value, the microcontroller sends a corresponding instruction to issue a refill warning.

Images