Method for automatically monitoring, identifying and preventing plant insect pests based on deep learning

Abstract

The invention discloses a deep learning-based plant insect pest automatic monitoring identification and prevention method, which comprises the following steps of: in an image acquisition and preprocessing stage, acquiring a multi-view image by optimizing camera parameters and preprocessing the multi-view image; in the plant crevasse identification and positioning link, multi-scale feature fusion identification is adopted, and a sub-pixel-level positioning algorithm and space geometric constraint optimization are improved; in the prevention and treatment process, a processing path is adaptively planned, processing parameters are adjusted in real time, and the processing stability is controlled; the post-processing detection comprises deep learning evaluation model construction, multi-modal data fusion detection and dynamic threshold adaptive adjustment detection. According to the method, the pest monitoring and recognition precision and the prevention and control effect are improved, pesticide pollution is reduced, the agricultural production efficiency and the resource utilization efficiency are improved, the method is suitable for various plants and pests, good universality and adaptability are achieved, and agricultural production is promoted to develop towards the precise, intelligent and environment-friendly direction.

Description

Technical Field

[0001] The present invention relates to the field of plant protection and automated control technology, and in particular to a method for automatic monitoring, identification and prevention of plant pests based on deep learning. Background Art

[0002] In modern agricultural production, insect pests have always been one of the important factors affecting crop yield and quality. Insect pests not only directly damage the leaves, stems, fruits and other parts of plants, resulting in obstruction of photosynthesis and loss of nutrients, but also may cause the spread of plant diseases, further aggravating the harm to plants. According to statistics, the annual crop losses caused by insect pests in the world amount to billions of dollars, which seriously threatens food security and sustainable agricultural development.

[0003] Traditional pest control mainly relies on manual inspections and chemical pesticide spraying. Manual inspections are inefficient, consume a lot of manpower and time, and are limited by the experience and expertise of inspectors, which can easily lead to missed inspections and misjudgments. The excessive use of chemical pesticides has brought about a series of environmental and ecological problems, such as pesticide residues polluting soil and water sources, killing beneficial insects and destroying the ecological balance, and pests developing resistance to pesticides, resulting in reduced control effects. In addition, traditional pesticide spraying methods often lack precision and cannot be targeted according to the actual situation of the pests, resulting in pesticide waste and unsatisfactory control effects.

[0004] With the development of science and technology, some automation technologies have begun to be applied in the field of pest control, but there are still many shortcomings. For example, when collecting plant pest images, the existing image acquisition system often cannot automatically adjust the camera parameters according to the actual situation of the plant breach, resulting in unstable image quality and affecting the subsequent pest identification accuracy. In terms of pest identification algorithms, the single feature recognition method is difficult to accurately distinguish different types of pests and plant breaches in complex environments, and the recognition accuracy needs to be improved. At the same time, in the prevention and control link, the processing path planning is not intelligent enough and cannot adapt to plant breaches of different shapes and sizes. The adjustment of processing parameters lacks real-time and accuracy, and it is difficult to ensure the consistency and stability of the processing effect.

[0005] In order to solve the problems of traditional pest control methods, intelligent control technology has become the future development direction. Using advanced sensor technology, image processing technology, artificial intelligence algorithm and automatic control technology, real-time monitoring, accurate identification and efficient control of plant pests can be achieved, which can not only improve the control effect, reduce the use of pesticides, reduce the impact on the environment, but also improve agricultural production efficiency and ensure the yield and quality of crops. The present invention came into being in this context, aiming to provide a more accurate, efficient and intelligent method for automatic monitoring, identification and control of plant pests. Summary of the invention

[0006] The present invention provides a method for automatic monitoring, identification and control of plant pests based on deep learning to solve the problems raised in the above-mentioned background technology.

[0007] The present invention provides a method for automatic monitoring, identification and prevention of plant pests based on deep learning, which comprises the following steps: S100, image acquisition and preprocessing; S200, plant breach identification and location; S300, spray pesticides on the pest areas; S400, post-processing detection; Preferably, the step S100, image acquisition and preprocessing, includes: S110, image acquisition optimization strategy; S120, image preprocessing; In the S100, image acquisition and preprocessing S110, and image acquisition optimization strategy, the number of cameras is assumed to be , No. The shooting angle of the camera is , the shooting distance is By adjusting the angle and distance of the camera, ensure that all areas of the plant's surface can be clearly photographed. The multi-view image acquisition formula is: (in Indicates The images captured by the camera, Indicates the surface characteristics of plant breaks) Then dynamically adjust the acquisition parameters: according to the lighting conditions and texture characteristics of the plant surface, dynamically adjust the camera exposure time in real time , Gain and aperture size , introduce a light sensor to detect the ambient light intensity , Texture analysis algorithm calculates the complexity of plant crack surface texture , dynamically adjust the acquisition parameters according to the following types:

[0008] in It is an empirical coefficient, which is determined by optimizing a large amount of experimental data.

[0009] In the S100, image acquisition and preprocessing, S120, image preprocessing, an improved LoG filter is firstly used to introduce an adaptive weight factor , according to the pixel The grayscale change rate of the surrounding pixels is used to adjust the filter weight, and the formula is improved to The grayscale change rate calculation method is: (in Pixel The gray value of is the neighborhood window size), weight factor ( is the weight adjustment coefficient).

[0010] Next, we use deep learning to further enhance the image: we build a convolutional neural network (CNN) model for image enhancement. The formula is: (in is the sample size, To truly enhance the image, The image is enhanced by the model prediction. The collected image is enhanced by the trained CNN model to improve the image clarity and contrast, making the plant breach features more obvious.

[0011] Then binarize the image and calculate the local average gray value of the image and local grayscale standard deviation ,by ( is the threshold adjustment parameter) as the adaptive threshold. For each pixel , if its gray value is greater than or equal to the threshold , it is assigned as the foreground (the part with the plant opening), otherwise it is assigned as the background. This method can automatically select the appropriate threshold according to the grayscale distribution characteristics of different areas of the image and effectively segment the plant opening from the background.

[0012] Finally, morphological gradient edge detection and optimization are performed. In the morphological operation, the morphological gradient of the image is first calculated, and the formula is: (in is the original image, is the structural element, represents the expansion operation, represents an erosion operation).

[0013] Preferably, the step S200, identifying and locating plant breaches, includes the following steps: S210, plant breach identification using multi-scale feature fusion; S220, constructing an improved sub-pixel level plant breach location algorithm; S230, performing plant breach location optimization based on spatial geometric constraints; As an embodiment of the present invention, in the plant breach identification and location S200, S210, using multi-scale feature fusion to identify plant breaches, an image pyramid is constructed for the preprocessed multi-view images to obtain image representations at different scales. Assume that the number of layers of the image pyramid is , No. The resolution of the layer image is , images of different scales are obtained through downsampling operations: ,in represents the downsampling function.

[0014] For each scale of image, local binary pattern (LBP) features and histogram of oriented gradients (HOG) features are extracted respectively. LBP features can effectively describe the local texture information of the image, while HOG features are sensitive to the edge and shape information of the image.

[0015] The LBP features and HOG features at different scales are fused to obtain the fused feature vector F. The fusion method uses the weighted summation strategy formula: ,in For the The weights of the layer features are determined experimentally according to the importance of different scale features for plant breach identification.

[0016] The support vector machine (SVM) classifier is used to classify the fused feature vector to determine whether there are plant gaps in the image. The decision function of SVM is: ,in is the number of training samples, is the Lagrange multiplier, is the sample label, is the kernel function (radial basis function can be selected , is the kernel parameter), The SVM classifier is trained by a large number of image samples containing plant breaks and without plant breaks, so that it can accurately identify plant breaks.

[0017] As an embodiment of the present invention, in the S200, the plant breach identification and positioning S220, and the construction of an improved sub-pixel plant breach positioning algorithm, on the basis of determining the laser stripe boundary points and calculating the geometric center method, a curve fitting technology is introduced to more accurately model the laser stripes, and a cubic spline curve is used to fit the upper boundary points of the laser stripes. and the lower boundary point , the cubic spline curve formula is: ( is the number of boundary points) Determine the curve coefficients by solving the system of equations , so that the curve can smoothly pass through the boundary points and more accurately describe the shape of the laser stripe.

[0018] Based on the fitted curve, recalculate the coordinates of the center point of the laser stripe. Assume that the coordinates of the point on the fitted curve are , then the horizontal coordinate of the center point: ( and For the fitting curve The value range on the axis), the vertical axis For sub-pixel positioning, the grayscale centroid method is further optimized. When the gradient information of the grayscale value of the neighboring pixels is introduced, the formula is improved to: ,in Pixel The gray gradient value at is obtained by calculating the difference of the gray values ​​of adjacent pixels.

[0019] As an embodiment of the present invention, in the plant breach identification and location S200 and plant breach location optimization based on spatial geometric constraints, the position information and calibration parameters of the camera during multi-view image acquisition are used to establish a spatial geometric model. The extrinsic parameter matrix of a camera is and translation vectors , the internal parameter matrix is For the plant break feature points identified in the image (in the image coordinate system), and transform it to the world coordinate system using the following formula: .

[0020] According to the spatial geometric constraints, the plant gap feature points located at different perspectives are matched and fused. Assume that the plant gap feature points identified at two perspectives are and , calculate the spatial distance between two points If the distance Less than the set threshold , then the two feature points are considered to belong to the same plant break, and they are fused to obtain a more accurate estimate of the plant break position. The fusion method adopts a weighted average strategy, and the fused plant break position ,in and is the weight, which is determined according to the positioning accuracy and reliability of the feature points (the feature points with high positioning accuracy have larger weights).

[0021] Preferably, the step S300 of spraying pesticides on the pest area comprises the following steps: S310, adaptive processing path planning; S320, intelligent processing parameters real-time adjustment; S330, process stability control; As an embodiment of the present invention, in the step S300 of spraying pesticides on the pest site, S310, and adaptive processing path planning, a three-dimensional model of the plant breach is constructed according to the plant breach shape and position information determined in the step of plant breach identification and positioning (S200). Assume that the boundary point coordinates of the plant breach in the world coordinate system are ( is the number of boundary points), and the cubic spline interpolation method is used to fit the surface curve of the plant breach to obtain the surface equation of the plant breach (in is a parameter variable, For The relevant coefficient function is obtained by solving the boundary point coordinates).

[0022] Based on the three-dimensional model of the plant breach, the processing path of the processing robot is planned. The spiral path planning strategy is adopted, starting from the center of the plant breach, and processing along the edge of the plant breach in a spiral manner. The spiral equation is (in is the helix radius, is the angle parameter, = ( ) is the pitch of the spiral. Adjust the parameters of the spiral according to the size and shape of the plant breach to ensure that the treatment path covers the entire plant breach area and that the spacing of the treatment paths is uniform to ensure the consistency of the treatment quality.

[0023] In the process of path planning, the robot's kinematic constraints and workspace limitations are considered. The robot kinematic model is introduced to calculate the robot joint angle corresponding to each path point. Assume that the robot's joint variables are ( is the number of robot joints), the position of the end effector in the world coordinate system is , through the forward kinematics equation ( The forward kinematics function is used to calculate the joint angles. At the same time, it is checked whether the path point is within the robot's workspace to prevent the robot from colliding or going beyond the workspace.

[0024] As an embodiment of the present invention, in the S300, the S320 of spraying pesticides on the pest site, and the real-time adjustment of intelligent processing parameters, the processing parameters are adjusted in real time according to the geometric characteristics of the plant breach and the plant properties. ,width and plant break plant types As an input parameter, establish the processing current , Processing voltage and processing speed Dynamic adjustment model. The neural network model is used for modeling. The neural network structure includes an input layer (plant breach depth, width, plant type), a hidden layer (multiple neurons, using nonlinear activation functions such as RelU function ) and output layer (processing current, voltage, speed).

[0025] The neural network model is trained by a large amount of experimental data, and the loss function in the training process adopts the mean square error (MSE) function (in is the number of training samples, is the actual processing parameter, is the treatment parameter predicted by the model). After training, the model can accurately predict the appropriate treatment parameters based on the plant breach characteristics and plant information obtained in real time.

[0026] As an embodiment of the present invention, in the S300, S330 of spraying pesticides on the pest area, and the stability control of the treatment process, the model predictive control (MPC) method is used to ensure the stability of the treatment process. A dynamic model of the treatment process is established, and the treatment current is ,Voltage , Processing speed As an input variable, the breach depth ,Width and handling deformation As the output variable. The model prediction equation is: (in is the time step, is the model coefficient matrix, obtained through experimental data identification).

[0027] Based on the set treatment quality goals (such as the desired depth, width and allowable treatment deformation), the MPC controller calculates the optimal control input sequence (treatment current, voltage, speed) for multiple time steps in the future. The optimization objective function is: (in For the prediction time domain, To control the time domain, is the predicted output variable, is the reference output value, By solving the optimization problem, the optimal control input at the current moment is obtained and applied to the processing robot.

[0028] Preferably, the post-processing detection in S400 comprises the following steps: S410, constructing a processing quality assessment model based on deep learning; S420, multimodal data fusion detection; S430, dynamic threshold adaptive adjustment detection; As an embodiment of the present invention, in the S400, the post-processing detection S410, and the construction of the processing quality assessment model based on deep learning, the cross entropy loss function is used for the classification task (in is the sample size, is the true label, is the probability value predicted by the model), and the mean square error (MSE) loss function is used for regression tasks (in is the true value, The model is trained by back propagation algorithm and the model parameters are optimized so that the model can accurately identify the quality status after processing.

[0029] As an embodiment of the present invention, in the S400, the post-processing detection S420, and the multimodal data fusion detection, an ultrasonic sensor is used to detect defects inside the breach to obtain internal structural information of the breach; a laser displacement sensor is used to measure the surface flatness and deformation of the processed plant breach.

[0030] The laser displacement sensor obtains the surface height data of the plant breach, and the deformation of the treatment is evaluated by calculating the change in surface height. Assume that the coordinates of the measurement point are ( is the number of measuring points), calculate the surface flatness (in is the average height value), processing deformation .

[0031] As an embodiment of the present invention, in the S400, the post-processing detection S430, and the dynamic threshold adaptive adjustment detection, the detection threshold is dynamically adjusted according to the actual parameters in the processing process and the plant characteristics of the plant breach, and a threshold adjustment model is established, with the processing parameters and plant characteristics as input and the detection value as output.

[0032] The present invention provides a method for automatic monitoring, identification and prevention of plant pests based on deep learning, which has the following beneficial effects: In the image acquisition and preprocessing stage, multi-view images are collected and preprocessed by optimizing camera parameters. In the plant breach identification and positioning link, multi-scale feature fusion recognition, improved sub-pixel positioning algorithm and spatial geometric constraint optimization are adopted. In the prevention and control process, the processing path is adaptively planned, the processing parameters are adjusted in real time and the processing stability is controlled. Post-processing detection includes building a deep learning evaluation model, multimodal data fusion detection and dynamic threshold adaptive adjustment detection. The present invention improves the accuracy of pest monitoring and identification and the prevention and control effect, reduces pesticide pollution, improves agricultural production efficiency and resource utilization efficiency, is applicable to a variety of plants and pests, has good versatility and adaptability, and promotes agricultural production to develop in the direction of precision, intelligence and environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0034] Figure 1 This is a framework diagram of the method for automatically processing, identifying and repairing plant breaches in the present invention; Figure 2 This is a flow chart of the plant breach automatic processing, identification and repair method of the present invention; The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0035] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the present invention. It should be understood that the specific implementation methods described here are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the embodiments of the specification.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0038] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0039] The present invention provides a method for automatic monitoring, identification and prevention of plant pests based on deep learning, which comprises the following steps: S100, image acquisition and preprocessing; S200, plant breach identification and location; S300, spray pesticides on the pest areas; S400, post-processing detection; As an embodiment of the present invention, S100, image acquisition and preprocessing, includes: S110, image acquisition optimization strategy; S120, image preprocessing; In S100, image acquisition and preprocessing, S110, image acquisition optimization strategy, multi-view acquisition fusion is first performed: multiple high-definition cameras are used to simultaneously shoot plant breaches from different angles to obtain multi-view images. Assume that the number of cameras is , No. The shooting angle of the camera is , the shooting distance is By adjusting the angle and distance of the camera, ensure that all areas of the plant's surface can be clearly photographed. The multi-view image acquisition formula is: (in Indicates The images captured by the camera, Indicates the surface characteristics of plant breaks) Then dynamically adjust the acquisition parameters: according to the lighting conditions and texture characteristics of the plant surface, dynamically adjust the camera exposure time in real time , Gain and aperture size , introduce a light sensor to detect the ambient light intensity , Texture analysis algorithm calculates the complexity of plant crack surface texture , dynamically adjust the acquisition parameters according to the following types:

[0040] in It is an empirical coefficient, which is determined by optimizing a large amount of experimental data.

[0041] In S100, image acquisition and preprocessing, S120, image preprocessing, the improved LoG filter is first used: in the traditional LoG filter formula Based on this, an adaptive weight factor is introduced , according to the pixel The grayscale change rate of the surrounding pixels is used to adjust the filter weight, and the formula is improved to The grayscale change rate calculation method is: (in Pixel The gray value of is the neighborhood window size), weight factor ( is the weight adjustment coefficient).

[0042] Secondly, we use deep learning to further enhance the image: we build a convolutional neural network (CNN) model for image enhancement. The network structure includes multiple convolutional layers, pooling layers, and activation function layers. The input image is extracted through the convolutional layer, downsampled through the pooling layer, and the activation function layer increases the nonlinear mapping capability. During the model training process, a large number of image samples containing plant cracks are used for supervised learning, and the loss function uses the mean square error (MSE) function, the formula is: (in is the sample size, To truly enhance the image, The image is enhanced by the model prediction. The collected image is enhanced by the trained CNN model to improve the image clarity and contrast, making the plant breach features more obvious.

[0043] Then the image is binarized, and the adaptive threshold segmentation method is used in the image binarization process. Calculate the local average gray value of the image and local grayscale standard deviation ,by ( is the threshold adjustment parameter) as the adaptive threshold. For each pixel , if its gray value is greater than or equal to the threshold , it is assigned as the foreground (the part with the plant opening), otherwise it is assigned as the background. This method can automatically select the appropriate threshold according to the grayscale distribution characteristics of different areas of the image and effectively segment the plant opening from the background.

[0044] Finally, morphological gradient edge detection and optimization are performed. In the morphological operation, the morphological gradient of the image is first calculated, and the formula is: (in is the original image, is the structural element, represents the expansion operation, Represents the erosion operation). The morphological gradient can highlight the edge of the object in the image. Then the gradient image is optimized, and the non-maximum suppression algorithm is used to remove the non-maximum points in the edge response, and only the local maximum value is retained as the final edge point, which further refines the edge of the plant gap and improves the edge positioning accuracy.

[0045] As an embodiment of the present invention, S200, identifying and locating plant breaches, includes the following steps: S210, plant breach identification using multi-scale feature fusion; S220, constructing an improved sub-pixel level plant breach location algorithm; S230, performing plant breach location optimization based on spatial geometric constraints; As an embodiment of the present invention, in S200, plant breach identification and location S210, using multi-scale feature fusion to identify plant breaches, an image pyramid is constructed for the preprocessed multi-view images to obtain image representations at different scales. Assume that the number of layers of the image pyramid is , No. The resolution of the layer image is , images of different scales are obtained through downsampling operations: ,in represents the downsampling function.

[0046] For each scale of image, local binary pattern (LBP) features and histogram of oriented gradients (HOG) features are extracted respectively. LBP features can effectively describe the local texture information of the image, while HOG features are sensitive to the edge and shape information of the image.

[0047] The LBP features and HOG features at different scales are fused to obtain the fused feature vector F. The fusion method uses the weighted summation strategy formula: ,in For the The weights of the layer features are determined experimentally according to the importance of different scale features for plant breach identification.

[0048] The support vector machine (SVM) classifier is used to classify the fused feature vector to determine whether there are plant gaps in the image. The decision function of SVM is: ,in is the number of training samples, is the Lagrange multiplier, is the sample label, is the kernel function (radial basis function can be selected , is the kernel parameter), The SVM classifier is trained by a large number of image samples containing plant breaks and without plant breaks, so that it can accurately identify plant breaks.

[0049] As an embodiment of the present invention, in S200, plant breach identification and positioning S220, constructing an improved sub-pixel plant breach positioning algorithm, on the basis of determining the boundary points of the laser stripes and calculating the geometric center method, a curve fitting technology is introduced to more accurately model the laser stripes, and a cubic spline curve is used to fit the upper boundary points of the laser stripes. and the lower boundary point , the cubic spline curve formula is: ( is the number of boundary points) Determine the curve coefficients by solving the system of equations , so that the curve can smoothly pass through the boundary points and more accurately describe the shape of the laser stripe.

[0050] Based on the fitted curve, recalculate the coordinates of the center point of the laser stripe. Assume that the coordinates of the point on the fitted curve are , then the horizontal coordinate of the center point: ( and For the fitting curve The value range on the axis), the vertical axis For sub-pixel positioning, the grayscale centroid method is further optimized. When the gradient information of the grayscale value of the neighboring pixels is introduced, the formula is improved to: ,in Pixel The gray gradient value at is obtained by calculating the difference of the gray values ​​of adjacent pixels. This improvement can more accurately consider the trend of pixel grayscale changes and improve sub-pixel positioning accuracy.

[0051] As an embodiment of the present invention, in S200, plant breach identification and positioning S230, plant breach positioning optimization based on spatial geometric constraints is performed, using the position information and calibration parameters of the camera during multi-view image acquisition to establish a spatial geometric model. The extrinsic parameter matrix of a camera is and translation vectors , the internal parameter matrix is For the plant break feature points identified in the image (in the image coordinate system), and transform it to the world coordinate system using the following formula: .

[0052] According to the spatial geometric constraints, the plant gap feature points located at different perspectives are matched and fused. Assume that the plant gap feature points identified at two perspectives are and , calculate the spatial distance between two points If the distance Less than the set threshold , then the two feature points are considered to belong to the same plant break, and they are fused to obtain a more accurate estimate of the plant break position. The fusion method adopts a weighted average strategy, and the fused plant break position ,in and is the weight, which is determined according to the positioning accuracy and reliability of the feature points (the feature points with high positioning accuracy have larger weights).

[0053] Through the above spatial geometric constraint optimization, the accuracy and stability of plant breach positioning can be effectively improved, the deviation caused by single-view positioning error can be reduced, and more accurate location information can be provided for subsequent spraying of pesticides on pest sites.

[0054] Through the above-mentioned innovative plant break identification and positioning method, combined with image acquisition and preprocessing steps, plant breaks on plant breaks can be identified more accurately and reliably and their position and shape can be precisely located, providing strong support for subsequent pesticide spraying at the pest sites and improving the performance and repair quality of the entire system.

[0055] As an embodiment of the present invention, S300, spraying pesticides on the pest area, includes the following steps: S310, adaptive spraying path planning; S320, intelligent spraying parameters real-time adjustment; S330, spraying process stability control; As an embodiment of the present invention, in S300, spraying pesticides on the pest site, in S310, adaptive spraying path planning, a three-dimensional model of the plant breach is constructed according to the plant breach shape and position information determined in the plant breach identification and positioning (S200) step. Assume that the boundary point coordinates of the plant breach in the world coordinate system are ( is the number of boundary points), and the cubic spline interpolation method is used to fit the surface curve of the plant breach to obtain the surface equation of the plant breach (in is a parameter variable, For The relevant coefficient function is obtained by solving the boundary point coordinates).

[0056] Based on the three-dimensional model of the plant gap, the spraying path of the spraying robot is planned. The spiral path planning strategy is adopted, starting from the center of the plant gap, and spraying along the edge of the plant gap in a spiral manner. The spiral equation is (in is the helix radius, is the angle parameter, = ( ) is the pitch of the spiral. Adjust the parameters of the spiral according to the size and shape of the plant break to ensure that the spray path covers the entire plant break area and the spacing of the spray path is uniform to ensure the consistency of the spraying quality.

[0057] In the process of path planning, the kinematic constraints and workspace limitations of the spraying robot are considered. The robot kinematic model is introduced to calculate the robot joint angle corresponding to each path point. Assume that the robot joint variables are ( is the number of robot joints), the position of the end effector in the world coordinate system is , through the forward kinematics equation ( The forward kinematics function is used to calculate the joint angles. At the same time, it is checked whether the path point is within the robot's workspace to prevent the robot from colliding or going beyond the workspace.

[0058] As an embodiment of the present invention, in S300, spraying pesticides on the pest site, S320, intelligent spraying parameter real-time adjustment, the spraying parameters are adjusted in real time according to the geometric characteristics of the plant breach and the plant properties. ,width and plant breach types As an input parameter, establish the spraying current , spraying voltage and spraying speed Dynamic adjustment model. The neural network model is used for modeling. The neural network structure includes an input layer (plant breach depth, width, plant type), a hidden layer (multiple neurons, using nonlinear activation functions such as RelU function ) and the output layer.

[0059] The neural network model is trained by a large amount of experimental data, and the loss function in the training process adopts the mean square error (MSE) function (in is the number of training samples, is the actual spraying parameter, is the spraying parameter predicted by the model). After training, the model can accurately predict the appropriate spraying parameters based on the plant breach characteristics and plant information obtained in real time.

[0060] As an embodiment of the present invention, in S300, spraying pesticides on the pest area, S330, spraying process stability control, the model predictive control (MPC) method is used to ensure the stability of the spraying process. A dynamic model of the spraying process is established, and the spraying current is ,Voltage , spraying speed As an input variable, the breach depth ,Width and spraying range As the output variable. The model prediction equation is: (in is the time step, is the model coefficient matrix, obtained through experimental data identification).

[0061] Based on the set spraying quality goals (such as the desired depth, width, and allowable amount of spraying range), the MPC controller calculates the optimal control input sequence for multiple future time steps. The optimization objective function is: (in For the prediction time domain, To control the time domain, is the predicted output variable, is the reference output value, is the change in control input). By solving the optimization problem, the optimal control input at the current moment is obtained and applied to the spraying robot.

[0062] After each control cycle, the model parameters and state variables are updated, and the model is corrected according to the actual measured quality data (depth, width, deformation) to improve the accuracy of model prediction, thereby achieving real-time optimization control of the spraying process and ensuring stable and reliable spraying quality.

[0063] Through the above-mentioned innovative method of spraying pesticides on pest-infested areas, which is closely connected with the steps of identifying and locating plant breaches, it is possible to adaptively plan the spraying path according to the specific circumstances of the plant breaches, adjust the spraying parameters in real time, and effectively control the stability of the spraying process, thereby improving the quality and efficiency of spraying pesticides on pest-infested areas.

[0064] As an embodiment of the present invention, S400, post-processing detection, includes the following steps: S410, constructing a processing quality assessment model based on deep learning; S420, multimodal data fusion detection; S430, dynamic threshold adaptive adjustment detection; As an embodiment of the present invention, in S400, in S410 of post-processing detection and in the construction of a processing quality assessment model based on deep learning, a large number of processed plant crack image samples are collected, and these images are annotated, and the annotated information includes detailed information such as the crack quality category and the location and size of the defect.

[0065] A convolutional neural network (CNN) model is constructed to process quality assessment. The network structure uses multiple layers of convolutional layers, pooling layers, and fully connected layers. The convolutional layer is used to extract the features of the image, the pooling layer performs downsampling to reduce the data dimension, and the fully connected layer is used for classification and regression tasks. The network structure can include 5 convolutional layers, with convolution kernel sizes of 3x3, 3x3, 5x5, 5x5, 3x3, and a step size of 1 or 2. Each convolutional layer is followed by a ReLu activation function and a 2x2 maximum pooling layer; the fully connected layer includes 3 hidden layers, with the number of nodes being 256, 128, and 64, respectively. The last output layer sets the corresponding number of nodes according to the classification task (qualified or defective type).

[0066] Define the loss function and use the cross entropy loss function for classification tasks (in is the sample size, is the true label, is the probability value predicted by the model), and the mean square error (MSE) loss function is used for regression tasks (such as defect location and size prediction) (in is the true value, The model is trained by back propagation algorithm and the model parameters are optimized so that the model can accurately identify the quality status after processing.

[0067] As an embodiment of the present invention, in S400, S420 of post-processing detection, and multimodal data fusion detection, an ultrasonic sensor is used to detect defects inside the breach to obtain internal structural information of the breach; a laser displacement sensor is used to measure the surface flatness and deformation of the processed plant breach.

[0068] For ultrasonic detection data, the presence of defects inside the breach and the location and size of the defects are determined by analyzing the amplitude and time delay of the echo signal. A mathematical model between the ultrasonic echo signal and the defect characteristics is established, such as using a signal processing algorithm to extract the characteristic parameters of the echo signal, and then using a pattern recognition method to determine the defect type.

[0069] The laser displacement sensor obtains the surface height data of the plant breach, and the deformation of the treatment is evaluated by calculating the change in surface height. Assume that the coordinates of the measurement point are ( is the number of measuring points), calculate the surface flatness (in is the average height value), processing deformation .

[0070] Image data, ultrasonic detection data and laser displacement sensor data are fused, and data layer fusion method is used to fuse different sensor data in the feature extraction stage. Ultrasonic detection data and laser displacement sensor data are converted into image form (such as mapping ultrasonic echo signal intensity to grayscale value and representing laser displacement data as height image), and then input into CNN model together with original image data for training and detection, so that the model can integrate multi-modal information for more accurate processing quality assessment.

[0071] As an embodiment of the present invention, in S400, S430 of post-processing detection, and dynamic and threshold adaptive adjustment detection, the detection threshold is dynamically adjusted according to the actual parameters in the processing process (such as processing current, voltage, speed, etc.) and plant breach characteristics, and a threshold adjustment model is established, with the processing parameters and plant characteristics as input and the detection value as output.

[0072] For the defect recognition threshold in image detection, the current is analyzed and processed ,Voltage Relationship with the formation of breach defects, establishing the threshold adjustment formula (in is a coefficient determined based on experimental data). When the processing current is large, the defect recognition threshold is appropriately increased to avoid misjudgment; when the voltage fluctuates greatly, the value is dynamically adjusted according to the voltage change to ensure that the defect can be accurately detected. Through the above innovative post-treatment detection method, closely combined with the step of spraying pesticides on the pest site, it is possible to comprehensively and accurately evaluate the treatment quality, timely discover treatment defects and provide feedback, and ensure the quality and safety of the plant after spraying pesticides on the pest site. At the same time, polymorphic data fusion and dynamic threshold adjustment technology improve the reliability and adaptability of the detection system to meet the treatment detection needs under different working conditions.

[0073] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for automatic monitoring, identification and prevention of plant pests based on deep learning, characterized in that: The following steps are involved: S100, image acquisition and preprocessing; S200, plant breach identification and location; S300, spray pesticides on the pest areas; S400, testing after processing.

2. The method for automatic monitoring, identification and prevention of plant pests based on deep learning according to claim 1, characterized in that: The S100, image acquisition and preprocessing, includes: S110, image acquisition optimization strategy; S120, image preprocessing; In the S110, image acquisition optimization strategy, the number of cameras is assumed to be , No. The shooting angle of the camera is , the shooting distance is By adjusting the angle and distance of the camera, we can ensure that all areas of the plant's surface can be clearly photographed. The multi-view image acquisition formula is: ,in, Indicates The images captured by the camera, Indicates the surface characteristics of plant breaches; Then dynamically adjust the acquisition parameters: according to the lighting conditions and texture characteristics of the plant surface, dynamically adjust the camera exposure time in real time , Gain and aperture size , introduce a light sensor to detect the ambient light intensity , Texture analysis algorithm calculates the complexity of plant crack surface texture , dynamically adjust the acquisition parameters according to the following types: in It is an empirical coefficient, which is determined by optimizing a large amount of experimental data; In the image preprocessing step S120, an improved LoG filter is first used to introduce an adaptive weight factor , according to the pixel The grayscale change rate of the surrounding pixels is used to adjust the filter weight, and the formula is improved to , the grayscale change rate calculation method is: ,in, Pixel The gray value of is the neighborhood window size, weight factor , is the weight adjustment coefficient; Secondly, we use deep learning to further enhance the image: we build a convolutional neural network model for image enhancement, the formula is: ,in is the sample size, To truly enhance the image, For the enhanced image predicted by the model, the collected image is enhanced through the trained CNN model to improve the clarity and contrast of the image and make the plant breach features more obvious; Then binarize the image and calculate the local average gray value of the image and local grayscale standard deviation ,by As the adaptive threshold, Adjust the parameters for the threshold; for each pixel , if its gray value is greater than or equal to the threshold , it is assigned as the foreground, i.e. the plant gap, otherwise it is assigned as the background. This method can automatically select the appropriate threshold according to the grayscale distribution characteristics of different areas of the image and effectively segment the plant gap and the background; Finally, morphological gradient edge detection and optimization are performed. In the morphological operation, the morphological gradient of the image is first calculated, and the formula is: ,in is the original image, is the structural element, represents the expansion operation, Represents an erosion operation.

3. The method for automatic monitoring, identification and prevention of plant pests based on deep learning according to claim 2, characterized in that: The S200, identifying and locating plant breaches, includes the following steps: S210, plant breach identification using multi-scale feature fusion; S220, constructing an improved sub-pixel level plant breach location algorithm; S230, performing plant breach location optimization based on spatial geometric constraints; In the above S210, in the plant breach recognition using multi-scale feature fusion, an image pyramid is constructed for the pre-processed multi-view images to obtain image representations at different scales. The number of layers of the image pyramid is set to , No. The resolution of the layer image is , images of different scales are obtained through downsampling operations: ,in represents the downsampling function; For each scale of image, local binary pattern features and directional gradient histogram features are extracted and represented as LBP features and HOG features respectively. LBP features can effectively describe the local texture information of the image, while HOG features are sensitive to the edge and shape information of the image. The LBP features and HOG features at different scales are fused to obtain the fused feature vector F. The fusion method adopts the weighted summation strategy formula: ,in For the The weights of the layer features are determined experimentally based on the importance of different scale features for plant breach identification; The support vector machine classifier is the SVM classifier. The support vector machine classifier is used to classify the fused feature vector to determine whether there are plant gaps in the image. The decision function of SVM is: ,in is the number of training samples, is the Lagrange multiplier, is the sample label, is the kernel function, and the radial basis function can be selected , is the kernel parameter, is a bias term; a large number of image samples containing plant breaks and no plant breaks are used to train the SVM classifier so that it can accurately identify plant breaks; In the above S220, in constructing the improved sub-pixel level plant breach positioning algorithm, on the basis of determining the boundary points of the laser stripes and calculating the geometric center method, a curve fitting technique is introduced to more accurately model the laser stripes, and a cubic spline curve is used to fit the upper boundary points of the laser stripes. and the lower boundary point , the cubic spline curve formula is: ,in, is the number of boundary points; Determine the curve coefficients by solving the system of equations , so that the curve can smoothly pass through the boundary points and more accurately describe the shape of the laser stripe; Based on the fitted curve, recalculate the coordinates of the center point of the laser stripe, and set the coordinates of the point on the fitting curve to , then the horizontal coordinate of the center point: ( and For the fitting curve The value range on the axis), the vertical axis ; For sub-pixel positioning, the grayscale centroid method is further optimized to calculate the correction value. When the gradient information of the grayscale value of the neighboring pixels is introduced, the formula is improved to: ,in Pixel The gray gradient value at is obtained by calculating the difference of the gray values ​​of adjacent pixels; In the above S230, in the optimization of plant breach positioning based on spatial geometric constraints, the position information and calibration parameters of the camera during multi-view image acquisition are used to establish a spatial geometric model. The extrinsic parameter matrix of a camera is and translation vectors , the internal parameter matrix is , for the plant break feature points identified in the image (in the image coordinate system), and transform it to the world coordinate system using the following formula: ; According to the spatial geometric constraints, the plant break feature points located at different perspectives are matched and fused. Assuming that the plant break feature points identified at two perspectives are and , calculate the spatial distance between two points , if the distance Less than the set threshold , then the two feature points are considered to belong to the same plant break, and they are fused to obtain a more accurate estimate of the plant break position. The fusion method adopts a weighted average strategy. The fused plant break position ,in and is the weight, which is determined according to the positioning accuracy and reliability of the feature points. The feature points with high positioning accuracy have larger weights.

4. The method for automatic monitoring, identification and prevention of plant pests based on deep learning according to claim 3 is characterized in that: The step S300 of spraying pesticides on the pest area includes the following steps: S310, adaptive processing path planning; S320, intelligent processing parameters real-time adjustment; S330, process stability control; In the step S310, adaptive processing path planning, a three-dimensional model of the plant breach is constructed according to the plant breach shape and position information determined in the step S200, plant breach identification and positioning, and the boundary point coordinates of the plant breach in the world coordinate system are set as ( is the number of boundary points), and the cubic spline interpolation method is used to fit the surface curve of the plant breach to obtain the surface equation of the plant breach ,in is a parameter variable, For The relevant coefficient function is obtained by solving the boundary point coordinates; Based on the three-dimensional model of the plant breach, the processing path of the processing robot is planned. The spiral path planning strategy is adopted. Starting from the center of the plant breach, the processing is carried out along the edge of the plant breach in a spiral line. The spiral line equation is: ,in is the helix radius, is the angle parameter, The pitch is adjusted according to the size and shape of the plant breach to ensure that the treatment path covers the entire plant breach area and the spacing of the treatment paths is uniform to ensure the consistency of the treatment quality; In the process of path planning, the robot's kinematic constraints and workspace limitations are considered, the robot kinematic model is introduced, and the robot joint angle corresponding to each path point is calculated. The robot's joint variables are set as ( is the number of robot joints), the position of the end effector in the world coordinate system is , through the forward kinematics equation ( is the forward kinematics function) to calculate the joint angles and, at the same time, check whether the path point is within the robot's workspace to prevent the robot from colliding or going beyond the workspace; In the S320, intelligent processing parameter real-time adjustment, the processing parameters are adjusted in real time according to the geometric characteristics of the plant breach and the plant attributes, and the depth of the plant breach is introduced. ,width and plant break plant types As an input parameter, establish the processing current , Processing voltage and processing speed The dynamic adjustment model is modeled using a neural network model. The neural network structure includes an input layer, a hidden layer, and an output layer. The input layer includes the depth, width, and plant type of the plant breach; the hidden layer includes multiple neurons and uses a nonlinear activation function, such as the RelU function. ;The output layer includes processing current, voltage, and speed; The neural network model is trained by a large amount of experimental data, and the loss function in the training process adopts the mean square error (MSE) function ,in is the number of training samples, is the actual processing parameter, The treatment parameters predicted by the model, after training, the model can accurately predict the appropriate treatment parameters based on the plant breach characteristics and plant information obtained in real time; In the above S330, in the stability control of the processing process, a model predictive control method is used to ensure the stability of the processing process, a dynamic model of the processing process is established, and the processing current is ,Voltage , Processing speed As an input variable, the breach depth ,Width and handling deformation As the output variable, the model prediction equation is: ,in is the time step, is the model coefficient matrix, obtained through experimental data identification; According to the set treatment quality goals, such as the desired depth, width and allowable treatment deformation, the MPC controller calculates the optimal control input sequence for multiple future time steps, including treatment current, voltage and speed. The optimization objective function is: ,in For the prediction time domain, To control the time domain, is the predicted output variable, is the reference output value, To control the change in input, the optimal control input at the current moment is obtained by solving the optimization problem and applied to the processing robot.

5. The method for automatic monitoring, identification and prevention of plant pests based on deep learning according to claim 4, characterized in that: The post-processing detection in S400 includes the following steps: S410, constructing a processing quality assessment model based on deep learning; S420, multimodal data fusion detection; S430, dynamic threshold adaptive adjustment detection; In the construction of the processing quality assessment model based on deep learning in S410, a cross entropy loss function is used for the classification task. ,in is the sample size, is the true label, is the probability value predicted by the model, and the mean square error loss function is used for regression tasks ,in is the true value, To predict the model value, the model is trained through the back propagation algorithm and the model parameters are optimized so that the model can accurately identify the quality status after processing; In the multimodal data fusion detection in S420, an ultrasonic sensor is used to detect defects inside the breach to obtain internal structural information of the breach; a laser displacement sensor is used to measure the surface flatness and deformation of the plant breach after treatment; The laser displacement sensor obtains the surface height data of the plant breach, and the deformation is evaluated by calculating the change in surface height. The coordinates of the measurement point are set as , Calculate the surface flatness for the number of measurement points ,in is the average height value, processing deformation .

Images