Method, system and equipment for detecting diseases and insect pests of street tree crowns and storage medium

Abstract

The invention discloses a street tree crown pest and disease detection method, system and device and a storage medium. The method comprises the following steps: acquiring image information of a street; wherein the image information is acquired by the unmanned aerial vehicle and carries lesion information of crowns on the border trees; preprocessing the image information to obtain a training sample; based on the training sample, training the initial YOLOv8 network model to obtain a trained pest and disease identification model; detecting the street image to be recognized by adopting the disease and pest recognition model to obtain a detection result; wherein the detection result comprises positioning information of the street image to be identified. According to the pest and disease damage identification model, large-batch street images can be detected, high-precision automatic identification and spatial positioning of infected street trees are realized, the identification time is saved, the troubleshooting cost is reduced, and the dynamic monitoring requirement of large-area urban street tree pest and disease damage is met.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, system, device and storage medium for detecting diseases and insect pests in crowns of roadside trees. Background Art

[0002] With the acceleration of urbanization, the importance of street trees as urban greening is also increasing. They not only bring huge social and economic benefits, but also provide vital ecosystem services to the ecosystem. From a social perspective, urban street trees not only improve the landscape value of the city, but also provide a healthier and more comfortable living environment for urban residents; in terms of economic benefits, urban street trees can improve the air quality of the city, reduce the concentration of heavy metals and harmful gases, and effectively reduce the urban heat island effect; street trees are also important providers of urban ecosystem services. They can mitigate climate change through carbon sequestration, alleviate extreme events such as urban floods, and protect urban biodiversity.

[0003] Many cities use plane trees as the backbone tree species of urban green space, but the canopy of plane trees is susceptible to the infection of pests and diseases such as powdery mildew and Corythucha ciliata. Powdery mildew is a fungal disease that infects repeatedly, mainly harming leaves, tender shoots and buds. In the early stage of onset, a white powder layer is produced on the front or back of the blade, which is gradually expanded, causing the tender leaves to be twisted and deformed. When serious, the leaf lesions heal each other and connect into pieces, spread to the whole blade, and make the blade turn yellow, wither, and fall off in advance. Corythucha ciliata parasitizes on the bottom surface of the blade, sucks the juice of the blade to feed, and the damage on the upper surface of the leaf is the most obvious. Initially, it is many very dense yellow-white spots, which gradually develop into a large piece of bronze color, and the back is covered with rusty brown insect feces and secretions, which inhibits vegetation photosynthesis, and finally causes the tree to weaken, the leaves to fall off or die, which brings an impact that cannot be ignored to the growth and development of street trees. Both of the above-mentioned powdery mildew pests and diseases are highly contagious. Once introduced into new areas, if timely measures are not taken, the diseases will spread rapidly on a block-by-block basis.

[0004] At present, the main monitoring methods for vegetation diseases and insect pests are manual field survey methods. The above methods are mainly based on visual judgment of the human eye, judging the type of disease according to the external symptoms of the leaves, the life cycle and transmission mode of the pathogen, and grading the health status of the trees. Although the traditional manual screening method has a high accuracy in identifying crown diseases and insect pests of street trees, it is costly, easy to miss, and requires high professional knowledge of the screening personnel; some complex samples need to be sent to the laboratory, which requires a large workload and identification cycle, resulting in certain limitations in large-scale monitoring and identification, making it difficult to fully grasp the situation of urban street tree diseases and insect pests. Summary of the invention

[0005] Based on this, it is necessary to provide a method, system, equipment and storage medium for detecting diseases and insect pests of street trees in response to the above-mentioned technical problems, so as to solve the problem that although the manual screening method in the existing technology has high accuracy in identifying diseases and insect pests of street trees, it is costly, easy to miss, and requires high professional knowledge of the screening personnel; some complex samples need to be sent to the laboratory, which requires a large workload and identification cycle, resulting in certain limitations in large-scale monitoring and identification, and it is difficult to fully grasp the disease and insect pest situation of urban street trees.

[0006] In a first aspect, the present application provides a method for detecting diseases and insect pests in crowns of roadside trees, comprising:

[0007] Obtaining image information of the street; wherein the image information is obtained by a drone and carries pathological information of the tree crowns on the roadside trees;

[0008] Preprocessing the image information to obtain training samples;

[0009] Based on the training samples, the initial YOLOv8 network model is trained to obtain a trained pest and disease recognition model;

[0010] The pest and disease identification model is used to detect the street image to be identified to obtain a detection result; wherein the detection result includes the positioning information of the street image to be identified.

[0011] In one embodiment, the preprocessing of the image information to obtain training samples includes:

[0012] Processing the image information to obtain an orthophoto;

[0013] annotating the orthophoto;

[0014] According to a preset ratio, the annotated orthophotos are divided into a training set, a validation set, and a test set to obtain initial training samples;

[0015] Cutting the initial training sample to obtain first data;

[0016] Data enhancement is performed on the first data to obtain the training samples.

[0017] In one embodiment, processing the image information to obtain an orthophoto includes:

[0018] The image information is geometrically corrected and positioned, and then the corrected image information is spliced ​​to obtain an orthophoto.

[0019] In one embodiment, the annotated orthophoto is divided into a training set, a validation set, and a test set according to a preset ratio, including:

[0020] For each of the orthophotos belonging to the same street, the annotated orthophotos are divided into a sub-training set, a sub-validation set, and a sub-test set according to a preset ratio;

[0021] Each of the sub-training sets, each of the sub-validation sets, and each of the sub-test sets constitutes the training set, the validation set, and the test set, respectively.

[0022] In one of the embodiments, the method of performing data enhancement on the first data includes: any one or more of a random rotation operation, a brightness transformation operation, a noise addition operation, a random cropping operation, an image flipping operation, and an image translation operation.

[0023] In one embodiment, the use of the pest and disease identification model to detect the street image to be identified includes:

[0024] Processing the street image to obtain a street orthophoto;

[0025] First, the spatial resolution of the street orthophoto is resampled to make it consistent with the spatial resolution of the training sample;

[0026] Then adjust the size of the street orthophoto, and the adjustment formula is shown as:

[0027]

[0028] Among them, pred R is the spatial resolution of the street orthophoto;

[0029] train R is the spatial resolution of the training samples;

[0030] pred(x) size is the size of the street orthophoto after adjustment in the x direction;

[0031] pred(x) input is the input size of the street orthophoto in the x direction;

[0032] train(x) input is the resampling size of the training sample in the x direction;

[0033] train(x) size is the input size of the training sample in the x direction;

[0034] pred(y) sizeis the size of the street orthophoto after adjustment in the x direction;

[0035] pred(y) input is the input size of the street orthophoto in the x direction;

[0036] train(y) input is the resampling size of the training sample in the x direction;

[0037] train(y) size is the input size of the training sample in the x direction.

[0038] In one embodiment, the method for obtaining the positioning information includes:

[0039] Constructing a plane coordinate system according to the positioning device of the UAV, and obtaining pixel coordinates of each detection frame based on the plane coordinate system;

[0040] Convert the pixel coordinates into the latitude and longitude coordinates of the detection frame to obtain positioning information;

[0041] The conversion formula is shown as:

[0042] Xgeo=GT(0)+X*GT(1)+Y*GT(2)

[0043] Ygeo=GT(3)+X*GT(4)+Y*GT(5)

[0044] Among them, Xgeo is the latitude and longitude coordinates of the detection frame in the x direction; Ygeo is the latitude and longitude coordinates of the detection frame in the y direction;

[0045] X is the pixel coordinate of the detection box in the x direction; Y is the pixel coordinate of the detection box in the y direction;

[0046] GT(0) is the longitude coordinate of the upper left corner of the street orthophoto; GT(3) is the latitude coordinate of the upper left corner of the street orthophoto;

[0047] GT(1) is the horizontal resolution of the street orthophoto; GT(5) is the vertical resolution of the street orthophoto;

[0048] GT(2) is the rotation coefficient of the street orthophoto in the x direction; GT(4) is the rotation coefficient of the street orthophoto in the y direction.

[0049] In a second aspect, the present application provides a roadside tree crown pest detection system, comprising:

[0050] An acquisition module is used to acquire image information of the street; wherein the image information is acquired by a drone and carries pathological information of the tree crowns on the roadside trees;

[0051] A processing module, used for preprocessing the image information to obtain training samples;

[0052] A training module, used to train the initial YOLOv8 network model based on the training samples to obtain a trained pest and disease recognition model;

[0053] The detection module is used to use the pest and disease identification model to detect the street image to be identified and obtain a detection result; wherein the detection result includes the positioning information of the street image to be identified.

[0054] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method steps described in any one of the first aspects are implemented.

[0055] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0056] The above-mentioned roadside tree crown pest detection method, system, device and storage medium have at least the following advantages:

[0057] After receiving street image information acquired by drones, this application preprocesses the image information to obtain a large number of diverse training samples. Based on the training samples, the YOLOv8 network model is trained to effectively improve the robustness of the model. The pest and disease recognition model trained with the above training samples and the YOLOv8 network model can detect a large number of street images, realize high-precision automatic recognition and spatial positioning of diseased roadside trees, save recognition time, reduce investigation costs, and meet the dynamic monitoring requirements of large-scale urban roadside tree pests. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 A diagram showing an application environment of a method for detecting pests and diseases in the crowns of roadside trees according to an embodiment;

[0061] Figure 2 A schematic diagram of a process for detecting pests and diseases in the crowns of roadside trees according to an embodiment;

[0062] Figure 3 is a flowchart of the steps of obtaining training samples in one embodiment;

[0063] Figure 4 A schematic diagram of the structure of an initial YOLOv8 network model in one embodiment;

[0064] Figure 5 A schematic diagram of the average accuracy of initial YOLOv8 model training in one embodiment;

[0065] Figure 6 A schematic diagram of the convergence speed of the initial YOLOv8 model in one embodiment;

[0066] Figure 7 A schematic diagram of a flow chart of a step of detecting a street image to be identified in one embodiment;

[0067] Figure 8 A schematic diagram of the detection results of a pest and disease identification model in one embodiment;

[0068] Fig. 9 A schematic diagram of a flow chart of a step of obtaining positioning information in an embodiment;

[0069] Fig.10 A schematic diagram of positioning result attributes in an embodiment;

[0070] Fig.11 It is a structural block diagram of a system for detecting pests and diseases of roadside tree crowns according to an embodiment;

[0071] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0072] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0073] While some exemplary embodiments of the present invention have been described for the purpose of illustration, it should be understood that the present invention may be implemented in other ways not specifically shown in the drawings.

[0074] The method for detecting roadside tree crown diseases and insect pests provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0075] The terminal 102 can send the image information of the street to the server 104, so that the server 104 pre-processes the image information after obtaining the image information to obtain a training sample; wherein the image information is obtained by the drone and carries the pathological information of the tree crown on the street tree. Then, based on the training sample, the initial YOLOv8 network model is trained to obtain a trained pest and disease recognition model; finally, the trained pest and disease recognition model is used to detect the street image to be identified to obtain the detection result; wherein the detection result includes the positioning information of the street image to be identified. In this way, the server 104 feeds back the detection result to the terminal 102.

[0076] After receiving the street image information obtained by the drone, the above-mentioned roadside tree crown pest detection method preprocesses the image information to obtain a large number of diverse training samples, and trains the YOLOv8 network model based on the training samples, which can effectively improve the robustness of the model. The pest identification model trained with the above-mentioned training samples and the YOLOv8 network model can detect a large number of street images, realize high-precision automatic identification and spatial positioning of diseased roadside trees, save identification time, reduce investigation costs, and meet the dynamic monitoring requirements of large-scale urban roadside tree pests.

[0077] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0078] In a feasible embodiment, the present application embodiment provides a method for detecting pests and diseases in the crown of roadside trees. Figure 1 The server 104 in FIG. 1 is used as an example for explanation.

[0079] See also Figure 2 , Figure 2 The flowchart of the method for detecting pests and diseases in the crown of roadside trees of this embodiment specifically includes the following steps:

[0080] Step S202, obtaining image information of the street; wherein the image information is obtained by a drone and carries pathological information of the tree crowns on the roadside trees.

[0081] Specifically, the image information is obtained by drones shooting the streets from the air. Before shooting, the flight data should be planned in advance, such as determining the flight time, flight area, flight altitude, overlap rate, flight path, etc. Among them, the flight area should be selected in areas with the characteristics of sycamore pests and diseases.

[0082] For example, the shooting period of this embodiment is from June of the current year to June of the next year; the flight altitude is defined as between 30-50m; the ground resolution is between 0.01 and 0.02m; clear and cloudless weather is selected, and the shooting time range is from 11 am to 16 pm when the sun is sufficient, so as to ensure that the street trees in the captured images have complete tree crowns and high clarity. In addition, the heading overlap rate of this embodiment is set to 70%, and the lateral overlap rate is set to 60% to ensure that there is sufficient overlap between adjacent images, thereby generating high-quality digital orthophotos.

[0083] The lesion information of the crown of street trees is used to characterize the powdery mildew and web bug diseases and pests of sycamore. The main manifestations are that powdery mildew mainly harms leaves, tender shoots and buds. In the early stage of the disease, a white powder layer is produced on the front or back of the leaves, which gradually expands, causing the tender leaves to twist and deform. In severe cases, the leaf spots heal each other and connect into pieces, spreading to the entire leaf, causing the leaves to turn yellow, wither, and fall prematurely. Web bugs parasitize on the underside of the leaves, sucking the sap of the leaves to feed. The damage on the upper surface of the leaves is most obvious. Initially, there are many very dense yellow-white spots, which gradually develop into large areas of bronze. The back is covered with rust-brown insect feces and secretions. The tree is weak, and the leaves fall or die.

[0084] Step S204: pre-process the image information to obtain training samples.

[0085] Specifically, the purpose of preprocessing image information is to remove poor quality and atypical pest and disease images, and obtain high-definition images with obvious pest and disease characteristics.

[0086] It should be understood that after obtaining the training samples, it is also necessary to compare the feature differences between different pest samples to build a unified tree pest labeling standard for subsequent labeling. When classifying street trees, only tree species and pest attributes are considered in this embodiment.

[0087] Step S206: Based on the training samples, the initial YOLOv8 network model is trained to obtain a trained pest and disease recognition model.

[0088] Specifically, the YOLOv8 network model usually includes an input end, a feature extraction network, a feature fusion network, a detection head, and an output end. The input end is used to receive input training samples; the feature extraction network is used to extract useful feature information from the input training samples; the feature fusion network is used to fuse the feature information of different levels and scales extracted by the feature extraction network; the detection head is used to predict the bounding box, category, and confidence of the target in the image based on the feature information output by the feature fusion network; the output end is used to post-process the results predicted by the detection head and output the final detection result.

[0089] Furthermore, the initial YOLOv8 network model in this embodiment also includes a positioning module, which is used to match the feature points in the input training sample with the known geographic coordinates to determine the geographic coordinates of the pixel points in the training sample, thereby obtaining the positioning information of the training sample.

[0090] Step S208, using the pest and disease recognition model to detect the street image to be identified, and obtain a detection result; wherein the detection result includes the positioning information of the street image to be identified.

[0091] Specifically, the street image to be identified is acquired by a drone and includes the tree crown images of multiple roadside trees. After the pest and disease identification model reads the input street image to be identified, it extracts its feature information, compares the extracted feature information with the features learned during training, and then classifies it according to the comparison results. Finally, the detection result is output according to the classification result. The monitoring result is the pest and disease category or probability distribution to which the identified tree trunk image belongs, as well as the positioning information of the tree trunk image.

[0092] After receiving the street image information obtained by the drone, the above-mentioned roadside tree crown pest detection method preprocesses the image information to obtain a large number of diverse training samples, and trains the YOLOv8 network model based on the training samples, which can effectively improve the robustness of the model. The pest identification model trained with the above-mentioned training samples and the YOLOv8 network model can detect a large number of street images, realize high-precision automatic identification and spatial positioning of diseased roadside trees, save identification time, reduce investigation costs, and meet the dynamic monitoring requirements of large-scale urban roadside tree pests.

[0093] See also Figure 3 Optionally, preprocess the image information to obtain training samples, including:

[0094] Step S302: Process the image information to obtain an orthophoto.

[0095] Specifically, orthophotos are remote sensing images with orthographic projection properties. The original images taken by drones are often distorted to varying degrees due to changes in the internal state of the sensor, the external environment, and the surface conditions during imaging. These distortions include geometric deformations caused by factors such as camera projection mode, posture, and terrain undulations. By performing a series of mathematical and geographic information processing on the original images, geometric deformations and distortions are eliminated, thereby generating an image that accurately reflects the true surface conditions.

[0096] Optionally, the image information is processed to obtain an orthophoto, including:

[0097] The image information is geometrically corrected and positioned, and then the corrected image information is spliced ​​to obtain an orthophoto.

[0098] Exemplarily, this embodiment uses the Pix4UAV algorithm to calculate the elevation value of each pixel of the original image information, locates the spatial posture and position of each image using aerial triangulation, generates three-dimensional point cloud data, and finally stitches the images to obtain orthophotos.

[0099] Step S304: labeling the orthophoto.

[0100] Specifically, the processed orthophotos are labeled based on the pre-built tree pest and disease labeling standards. Since the sample quality directly affects the recognition accuracy of the model, careful judgment and repeated confirmation are required during labeling to reduce random errors. For example, the present embodiment uses the LabelImg tool to feature label the original image to create a label data set for model training and verification.

[0101] Step S306 , dividing the annotated orthophoto into a training set, a validation set, and a test set according to a preset ratio to obtain initial training samples.

[0102] Specifically, the training set is used to train the initial model, that is, to let the model update its internal parameters by learning the data in the training set to minimize the loss function and optimize its prediction performance. The validation set is used to select and tune different model architectures, hyperparameters, etc. during the model training process, so as to monitor the performance of the model during the training process and avoid overfitting. The test set is used to evaluate the performance of the model in practical applications after the model training is completed; it provides an independent, unseen data set to test whether the model can accurately predict new data. To ensure the fairness and accuracy of model evaluation and selection, the training set, validation set, and test set should be independent of each other. Exemplarily, the annotated orthophotos are randomly divided in a ratio of 6:3:1, of which 60% is the training set, 30% is the validation set, and 10% is the test set.

[0103] Optionally, the annotated orthophotos are divided into a training set, a validation set, and a test set according to a preset ratio, including:

[0104] For each orthophoto belonging to the same street, the labeled orthophotos are divided into a sub-training set, a sub-validation set, and a sub-test set according to a preset ratio; each sub-training set, each sub-validation set, and each sub-test set constitutes a training set, a validation set, and a test set, respectively.

[0105] Specifically, there is a high overlap rate between adjacent image information obtained by drones. If all image information is directly divided, the same tree in the same area will be divided into both the training set and the validation set, which will cause serious overfitting of the model. Based on this, this embodiment first divides by street, and then divides the orthophoto of the same street according to a preset ratio to obtain a sub-training set, a sub-validation set, and a sub-test set for each street. Finally, all sub-training sets are combined to obtain a training set, similar to the validation set and the test set.

[0106] Step S308, cutting the initial training sample to obtain first data.

[0107] Specifically, due to computer memory limitations, deep learning algorithms cannot directly perform deep learning training on large images. Therefore, after obtaining street drone orthophotos, the images need to be reprocessed and cropped into a series of small images of preset sizes to construct an image data set, i.e., the first data. Exemplarily, using the above method, the final training set has 445 image samples, the test set has 216 image samples, and the validation set has 58 images. In the training set, there are 1,146 diseased sycamores, 456 healthy sycamores, and 758 other types of street trees.

[0108] Step S310: perform data enhancement on the first data to obtain training samples.

[0109] Specifically, the manifestations of pests and diseases in the real world are diverse, and existing training samples are difficult to express the complexity of pests and diseases in the real world. Obtaining enough disease samples requires a lot of manpower and material costs, which is extremely challenging. Therefore, the embodiment of the present application adopts a data enhancement method to generate diversified training samples, which can prevent the model from overfitting on the training data, thereby improving the performance of the model on unseen data, and then simulating different scenes in the real world, improving the robustness of the model, enabling the model to learn stronger and more extensive features, reducing the model's bias towards specific scenes or specific conditions, and better identifying various types of diseases.

[0110] Optionally, the first data is enhanced by any one or more of a random rotation operation, a brightness change operation, a noise addition operation, a random cropping operation, an image flip operation, and an image translation operation. After data enhancement, the number of sample data sets increases to 7 times the original number, with a total of 3115 training sets and 1512 validation sets.

[0111] Specifically, the steps of random rotation include: obtaining the width and height of the image, converting the rotation angle into radians, and then calculating the width and height of the rotated new image based on the rotation angle. Next, use the getRotationMatrix2D function of OpenCV to obtain the rotation matrix, calculate the translation from the center of the old image to the center of the new image, and update the translation in the rotation matrix. Then use the affine transformation function cv2.warpAffine to rotate the image to obtain the rotated image. Next, traverse each bounding box of the image, transform the four corner points of the bounding box into the new coordinate system through the rotation matrix, calculate and obtain the new bounding box coordinates, and finally return the rotated image and bounding box list. By randomly rotating the image, simulating its views at different angles, simulating the recognition ability of pests and diseases at different angles and directions, and allowing the model to adapt to the characteristics of pests and diseases at various angles.

[0112] The brightness transformation first defines an all-zero matrix with the same shape and data type as the input image, and then uses the cv2.addWeighted function to perform a weighted calculation on the image by multiplying the input image by the brightness adjustment coefficient alpha, plus the all-zero matrix multiplied by (1-alpha), and adding a constant offset (usually 0) to the result. This method can linearly adjust the brightness of the image to obtain a new image with adjusted brightness, so that the model can adapt to different lighting conditions and can still accurately identify pests and diseases in low or enhanced light conditions.

[0113] Adding noise usually refers to adding Gaussian noise to the input image. First, a random seed is generated to ensure the randomness of the noise, and then the mean and variance of the noise are set (usually the mean is 0 and the variance is 0.01). Then the standard deviation is calculated, and a noise matrix with the same shape as the image is generated. Each value in the matrix is ​​a Gaussian distribution value generated according to the set mean and standard deviation. Then this noise matrix is ​​added to the original image to obtain a new image after adding noise. Since the pixel value of the new image is between 0 and 1, it is finally multiplied by 255 and converted to an unsigned 8-bit integer type to obtain the final noisy image. Adding Gaussian noise to the image simulates the irregular features such as spots and particles that may appear on the surface of the disease in reality, so that the model can better identify the subtle changes of these diseases. In the detection of pests and diseases, the noise may be closer to some characteristics of the actual disease, rather than a simple degradation of image quality, so the robustness of the model in these cases is enhanced, so that the model can learn to accurately identify pests and diseases in a noisy environment, and the ability of the model to process low-quality images is enhanced.

[0114] Random cropping first defines a mask matrix with all 1s and the same shape as the input image. Then, for the specified number of crops, the center point of the cropped area is randomly selected in the image, and the coordinates of the cropped area are calculated to ensure that the cropped area is within the image range. Next, the intersection of the cropped area and each bounding box in the image is calculated. If the intersection of the cropped area and any bounding box exceeds the specified threshold, the cropped area is reselected until a suitable area is found. Finally, the value of the cropped area in the mask matrix is ​​set to 0, and the original image is multiplied by the mask matrix to obtain the cropped image. Through random cropping, occluding certain areas on the image can help simulate various obstacles or occlusions that may occur in the real environment. The model will pay more attention to the unoccluded areas in the image, thereby improving the situation of insufficient learning of few-sample disease features caused by imbalanced disease samples, and is more effective for diseases with obvious local features. The random cropping technique greatly enhances the robustness of the model when dealing with partially occluded or incomplete inputs.

[0115] Image flipping first makes a deep copy of the input image and gets the height and width of the image. Generate a random number and determine the flip type based on its value: if the random number is between 0 and 0.33, perform a vertical flip; if the random number is between 0.33 and 0.66, perform a horizontal flip; otherwise, perform a horizontal and vertical flip. Then, adjust the coordinates of the bounding box to fit the flipped image: for vertical flips, adjust the y coordinate; for horizontal flips, adjust the x coordinate; for horizontal and vertical flips, adjust both the x and y coordinates. Finally, return the flipped image and the updated bounding box list. By randomly flipping the image in multiple directions, the model can be adapted to different perspectives, and for some pests and diseases with strong symmetry, the recognition accuracy can be improved.

[0116] Image translation first obtains the width and height of the input image and initializes the minimum and maximum coordinates of all target boxes to determine the maximum translation distance. Then, the minimum and maximum coordinates of all bounding boxes in the input image are calculated to obtain the maximum left, right, up, and down distances. Next, the translation distances x and y are randomly generated within these distances and a translation matrix M is created. The image is translated using the cv2.warpAffine function to obtain the translated image. Finally, each bounding box is traversed, its coordinates are adjusted to reflect the translation effect, and the translated image and the updated bounding box list are returned. By randomly translating the image, the model can learn to correctly locate and identify the disease even when the location changes. In addition, translation can simulate the displacement that may occur during the camera's image capture process to enhance the model's robustness to pests and diseases in different locations and ensure that it can still be accurately identified when the location of the disease changes.

[0117] Furthermore, the image information is cleaned, format converted, resized and normalized to facilitate subsequent model training.

[0118] The above-mentioned method for detecting pests and diseases in the crowns of roadside trees processes the image information taken by the drone into an orthophoto to eliminate the distortion of the original image and obtain an image that can accurately reflect the actual situation of the surface; for the annotated orthophoto, it is first divided according to the street, and then the orthophotos belonging to the same street are divided, and finally the training set, verification set and test set are obtained. This can avoid the high overlap rate between adjacent image information, which will cause the same tree to be divided into the training set and verification set, thereby causing serious overfitting of the model; first, data enhancement of the data can generate diversified training samples to prevent the model from overfitting on the training data, thereby improving the performance of the model on unseen data, and then simulating different real-world scenarios to improve the robustness of the model, so that the model can learn stronger and broader features, reduce the model's bias towards specific scenarios or specific conditions, and better identify various types of diseases. The pest and disease identification model trained using the above training samples and the YOLOv8 network model can be used to detect large quantities of street images, realize high-precision automatic identification and spatial positioning of diseased street trees, save identification time, reduce investigation costs, and meet the requirements for dynamic monitoring of pests and diseases of street trees in large areas of cities.

[0119] Optionally, the embodiment of the present application trains the initial YOLOv8 network model based on the Pytorch framework, the hardware device is NVIDIA GeForce RTX 4090, and two GPUs are used for acceleration.

[0120] See also Figure 4Optionally, the initial YOLOv8 network model of the embodiment of the present application includes an input end, a feature extraction network, a feature fusion network, a detection head and an output end.

[0121] Exemplarily, the feature extraction network adopts the CSPDarknet53 network, including the C2f module and the FPN (FeaturePyramid Network) module. Among them, the introduced C2f module realizes more efficient feature transfer and fusion through partial connections across stages. The C2f module divides the input data into two branches for processing, one of which is directly passed to the output, and the other is processed by multiple Bottleneck modules in the C2f module. This branch design helps to increase the nonlinear ability and representation ability of the network. Different branches can focus on different types of disease characteristics, making the model more balanced in detecting various diseases, and improving the network's modeling ability for complex data. At the same time, the residual structure introduced in the Bottleneck module greatly reduces the computational complexity and the number of parameters, effectively reduces redundant calculations, improves the computational efficiency and feature expression ability of the model, and retains the model performance.

[0122] In addition, the C2f module performs feature transformation on the input data through two convolutional layers cv1 and cv2. These convolution operations help extract features of different levels and abstraction levels in the input data and capture the diverse disease features existing at different scales and levels. Finally, the C2f module helps to achieve feature fusion of different scales and enhance the detection capability of diseases of different scales by splicing features of different branches in the channel dimension.

[0123] The FPN module enhances the detection capability of small and large targets by constructing a top-down path and combining feature maps at different levels. This multi-scale feature fusion method makes YOLOv8 more robust when facing targets of different sizes and shapes, ensuring that the characteristics of pests and diseases from small to large can be effectively captured. In addition, similar to the C2f module, the cross-level feature sharing structure within FPN combines high-level and low-level features, integrates global context information and detail information, effectively takes into account the overall distribution and local characteristics of disease samples, and significantly improves the generalization ability of the model. The head network of YOLOv8 is responsible for decoding the feature map and generating the final detection result. This part includes multiple convolutional layers for predicting bounding boxes, category probabilities, and confidence scores. The improved loss function further optimizes the regression effect of the bounding box during training.

[0124] Optionally, the feature fusion network includes an average pooling module.

[0125] Specifically, the feature extraction network of the present application adopts the SPPF network (Spatial Pyramid Pooling-Fast), which is optimized on the basis of the SPP network (Spatial Pyramid Pooling), and achieves faster speed and fewer floating-point operations by reducing unnecessary calculations. At the same time, SPPF is also combined with the perceptual large kernel convolution UniRepLK to further improve the receptive field and detection accuracy of the model. It extracts multi-scale spatial information by dividing the feature map into grids of different sizes and performing a maximum pooling operation on each grid. This technology helps the model to better handle targets of different sizes and improve detection accuracy. However, before the target feature enters the SPPF network, it usually goes through multiple layers of convolution, resulting in the model usually lacking low-level semantic features in the feature fusion stage, resulting in insufficient learning.

[0126] Based on this, the embodiment of the present application introduces an average pooling module into the SPPF network, which enables the model to make corresponding decisions based on the high-level semantic information deep in the network and obtain global context information.

[0127] Optionally, the initial YOLOv8 network model of the embodiment of the present application further includes a core attention mechanism module. The detection head includes a large target prediction module and a medium target prediction module, and the core attention mechanism module is arranged before the large target prediction module and the medium target prediction module.

[0128] Specifically, the detection head includes a large target prediction module, a medium target prediction module and a small target prediction module for detecting targets of different sizes. The core attention mechanism module is set before the large target prediction module and the medium target prediction module. Due to the large size of these targets, the network needs a larger receptive field to cover the entire target to ensure full perception of the target as a whole. Therefore, the core attention mechanism can select a larger receptive field to help the network focus on the global features of large or medium targets. For small targets, the network often only needs a smaller receptive field, because the target itself occupies a small space, and an overly large receptive field may introduce redundant background information, which is not conducive to the detection of small targets. Therefore, it is not applied before the small target prediction module in the embodiment of the present application.

[0129] Furthermore, the kernel attention mechanism module includes Split operation, Fuse operation and Select operation to implement. Among them, due to the differences in the morphology of pests and diseases in the training samples and the different distance scales of pests and diseases displayed on the photos, in order to capture multi-scale feature information as much as possible, the Split operation processes the input features through convolution kernels of different sizes to capture multi-scale feature information. The purpose of the Fuse operation is to process the spliced ​​features and guide the selection operation through global information, so that the model can select the most appropriate kernel convolution according to the scale of the disease and effectively extract the features of diseases of different scales. The purpose of the Select operation is to dynamically select and fuse features of different scales according to the scale weights generated by the Fuse operation, and weight the sum of the weight of each scale with the corresponding result after the previous convolution to obtain different branch weight combinations. Combined with these features, the network can focus on the most effective feature scale at the moment, thereby optimizing the processing results.

[0130] At the same time, since the YOLOv8 network model is an end-to-end target detection model, after the core attention mechanism module is introduced into it, the addition of the fully connected layer will bring a huge number of parameters to the model, causing the model reasoning speed to slow down. The average pooling module can effectively alleviate the huge number of parameters of the fully connected layer, thereby reducing the black box operation of the fully connected layer and achieving a balance between the lightness and accuracy of the model.

[0131] Exemplarily, there are three types of targets to be identified in this application, including other tree species, diseased sycamore trees, and healthy sycamore trees. The detection head generates a large number of target frames in the input image information, and the NMS (Non-Maximum Suppression) algorithm is used to process these target frames, for example, to screen and sort the target frames according to their confidence and overlap, thereby selecting the best target frame from multiple candidate target frames to improve the accuracy and efficiency of model detection. Specifically, the NMS algorithm first sorts the bounding boxes in descending order according to their confidence, where confidence is usually a comprehensive indicator of classification probability and positioning accuracy; the bounding box with the highest confidence is considered to be the most likely to correctly detect the target, and then selects the bounding box with the highest confidence from the sorted list, marks it as selected, and adds it to the final detection result list. For each remaining bounding box, calculate its IOU (Intersection over Union) with the selected bounding box. In this process, if the IOU of a bounding box and the selected box exceeds the preset threshold, it is considered that the two boxes represent the same target, so according to the principle of low confidence, the low-confidence bounding box is eliminated. Finally, repeat the above steps until all bounding boxes have been checked and each target corresponds to an optimal bounding box.

[0132] During the processing, the identified target will be framed by boxes of various colors, and the corresponding confidence and coordinate information of the box will be stored. The higher the confidence, the higher the probability of being the target.

[0133] Through the above steps, the improvement of the YOLOv8 network model can be completed. Further, the steps of training the improved YOLOv8 network model include:

[0134] The training set is input into the original YOLOv8 network model after training and improvement. Under the constraint of the loss function, the original YOLOv8 network model is iteratively trained, and the model parameters are gradually adjusted until the preset requirements are met. Among them, the training set is 640×640 image samples and label samples. Specifically, 16 groups of image samples x and corresponding label samples y are randomly selected in each batch (batch size) in the training set, and enter each layer of the original YOLOv8 network model in turn for calculation until all training sets are traversed once, that is, a training round (epoch) is completed. The difference (loss) between the predicted box ypred of the model and the real box y of the data set is calculated using the loss function. The closer the loss value is to 0, the better the convergence effect of the model, and vice versa.

[0135] Furthermore, during the training process, the validation set is used to fine-tune the model, helping to select the best model hyperparameters and prevent model overfitting.

[0136] Furthermore, after the training is completed, the test set is used to evaluate the final performance of the model, and evaluation indicators such as accuracy, precision, recall, F1 value, mean square error, etc. are calculated to measure the model's generalization ability to new data.

[0137] It should be understood that during the above training process, the training samples may be adjusted as needed. For example, if the detection result of a certain category is not accurate, sample images that better reflect the characteristics of the category may be added in the next round of training.

[0138] Through the above steps, the model learns better features and parameters during the training process, gradually adapts to the training data, and the verification loss decreases rapidly in the early stage and then gradually stabilizes.

[0139] For example, after each round of training, the accuracy of the pest and disease identification model is evaluated. The evaluation indicators of this embodiment include precision P (Precision) and recall R (Recall), and the formula is as follows:

[0140]

[0141] The curve formed by Precision and Recall is called the PR curve. Based on the PR curve, by calculating the average value of the Precision value corresponding to each Recall value, a numerical evaluation index AP can be obtained to measure the detection ability of the trained model in the category of interest. However, the AP value calculation is only for one category. In order to achieve a comprehensive evaluation of multiple types of diseases, it is necessary to calculate the AP of all categories and take the average value, that is, mAP. mAP measures the detection ability of the trained model in all categories. Assuming there are k categories, the calculation formula of mAP is as follows.

[0142] AP = ∫ 0 1 P(R)dR

[0143]

[0144] Among them, TP is the number of samples that are actually positive and correctly classified as positive; FP is the number of samples that are actually negative and incorrectly classified as positive; FN is the number of samples that are actually positive and incorrectly classified as negative. In addition, in this experiment, the mAP threshold is selected as 0.5 for verification to achieve a balance between precision and recall, ensuring that the model can accurately identify and locate pest targets in the image, and ensuring the lowest missed detection rate and false detection rate as possible.

[0145] See also Figure 5 and Figure 6 , Figure 5 This is a diagram of the average accuracy of the initial YOLOv8 model training. Figure 6 The figure is a schematic diagram of the convergence speed of the initial YOLOv8 model; Epochs is the number of training times. By observing the changing trend of the curve, the training status of the model can be analyzed. It can be seen from the figure that the average detection accuracy of the model in the first 80 rounds of training has a fluctuating upward trend with the increase of training rounds, and it slowly rises and tends to be flat after 80 rounds; the convergence speed of the loss value first decreases rapidly with the increase of training rounds in the first 80 rounds, and then shows a gentle downward trend after 80 rounds. The training curve is relatively smooth and has low volatility, indicating that the model learning rate setting is relatively reasonable; the loss function can achieve fast convergence, indicating that the network has effectively learned the data set; at the same time, there is no loss increase trend, which effectively suppresses the occurrence of overfitting. In general, after 300 rounds of iterations, the network achieves the best parameter combination, the model is effectively trained, and the model training is completed.

[0146] Table 1 Model accuracy evaluation table

[0147] category quantity Overall accuracy Recall mAP50 All roadside trees 8309 0.736 0.719 0.767 Other tree species 2730 0.59 0.538 0.579 Infected sycamore 4305 0.777 0.92 0.907 Healthy plane tree 1274 0.842 0.698 0.815

[0148] From the experimental results in Table 1, it can be seen that it is feasible to use the trained deep network model for automatic recognition and extraction of street trees. When the mean average precision and class recognition accuracy are used for performance evaluation, all types of detection objects of the deep network model have high scores, which further illustrates that the trained deep network model has strong learning and generalization capabilities, and has strong feasibility and accuracy in the refined detection of urban street tree pests and diseases, providing an efficient intelligent interpretation method for future urban street tree census and real-time detection of pests and diseases.

[0149] In the above-mentioned method for detecting diseases and pests in the crown of roadside trees, the feature extraction network adopts the C2f module and the FPN module, which significantly improves the detection capability of various diseases and pests, while effectively reducing redundant calculations, improving the computational efficiency and feature expression capability of the model, and retaining the model performance; the feature fusion network adopts the SPPF network and introduces the average pooling module, which enables the model to make corresponding decisions based on the high-level semantic information in the deep layer of the network and obtain the global context information; at the same time, it can effectively alleviate the huge parameters of the fully connected layer, thereby reducing the black box operation of the fully connected layer and achieving a balance between the lightness and accuracy of the model.

[0150] See also Figure 7 Optionally, a pest and disease identification model is used to detect the street image to be identified, including:

[0151] Step S702: Process the street image to obtain a street orthophoto.

[0152] Step S704: first resample the spatial resolution of the street orthophoto to make it consistent with the spatial resolution of the training sample.

[0153] Step S706, adjusting the size of the street orthophoto.

[0154] Specifically, during the model training phase, the training samples are processed from the original images into orthophotos with a size of 2560×2560, then cropped on the orthophotos and resampled to a size of 640×640, and finally input into the model for training. For example, the spatial resolution of the training samples in this embodiment is set to 1.23 cm / pixel.

[0155] When using the trained pest and disease model for prediction, the following problems are found:

[0156] (1) When detecting each small image, the information in the edge area is significantly less than that in the middle area of ​​the image, resulting in the prediction results of the edge area being worse than those of the middle area.

[0157] (2) When predicting street orthophotos and counting street tree targets, the association information between each image is blocked and the contextual information of the same type of object is ignored. If a single street tree is split into two parts during cropping, it will cause incorrect detection results and duplicate statistics.

[0158] (3) The prediction effects of drone images with different spatial resolutions vary.

[0159] In order to solve the above problems and improve the detection efficiency of the model, this application directly predicts the input street orthophoto and adjusts its spatial resolution and size.

[0160] Exemplarily, the adjustment formula is shown as:

[0161]

[0162] Among them, pred R is the spatial resolution of the street orthophoto;

[0163] train R is the spatial resolution of the training samples;

[0164] pred(x) size is the size of the street orthophoto after adjustment in the x direction;

[0165] pred(x) input is the input size of the street orthophoto in the x direction;

[0166] train(x) input is the resampling size of the training sample in the x direction;

[0167] train(x) size is the input size of the training sample in the x direction;

[0168] pred(y) size is the size of the street orthophoto after adjustment in the x direction;

[0169] pred(y) input is the input size of the street orthophoto in the x direction;

[0170] train(y) input is the resampling size of the training sample in the x direction;

[0171] train(y) size is the input size of the training sample in the x direction.

[0172] See also Figure 8 , Figure 8The figure shows the results of using the pest identification model to detect pests and diseases on roadside trees in Qinglong Road, Shanghai in two seasons. It can be seen that the model has a high detection accuracy for diseased roadside trees, and the size and position of the detection frame are basically consistent with the corresponding roadside trees. Table 2 Figure 8 Statistical table of detection results of diseases and insect pests on roadside trees.

[0173] Table 2 Statistics of roadside tree pest and disease detection results

[0174]

[0175] The above-mentioned street tree crown pest and disease detection method uses a trained pest and disease recognition model to directly predict the input street orthophoto and adjust its spatial resolution and size, thereby solving the problems of poor prediction results in edge areas and incorrect and repeated detection results caused by cropping street orthophotos into small images. It also solves the problem of uneven prediction effects of drone images with different spatial resolutions, thereby improving the detection accuracy of the model.

[0176] See also Fig. 9 Optionally, the method for obtaining the positioning information includes:

[0177] Step S902: construct a plane coordinate system according to the positioning device of the drone, and obtain the pixel coordinates of each detection frame based on the plane coordinate system.

[0178] Step S904: convert the pixel coordinates into the latitude and longitude coordinates of the detection frame to obtain positioning information.

[0179] The conversion formula is shown as:

[0180] Xgeo=GT(0)+X*GT(1)+Y*GT(2)

[0181] Ygeo=GT(3)+X*GT(4)+Y*GT(5)

[0182] Among them, Xgeo is the latitude and longitude coordinates of the detection frame in the x direction; Ygeo is the latitude and longitude coordinates of the detection frame in the y direction;

[0183] X is the pixel coordinate of the detection box in the x direction; Y is the pixel coordinate of the detection box in the y direction;

[0184] GT(0) is the longitude coordinate of the upper left corner of the street orthophoto; GT(3) is the latitude coordinate of the upper left corner of the street orthophoto;

[0185] GT(1) is the horizontal spatial resolution of the street orthophoto; GT(5) is the vertical spatial resolution of the street orthophoto;

[0186] GT(2) is the rotation coefficient of the street orthophoto in the x direction; GT(4) is the rotation coefficient of the street orthophoto in the y direction.

[0187] Specifically, after the street image to be identified is processed into a street orthophoto, the pest and disease identification model identifies multiple targets on the street orthophoto, such as multiple roadside trees, and then uses a detection frame to mark each target, thereby obtaining multiple detection frames.

[0188] Furthermore, the positioning device of the drone usually refers to a GPS device, which can provide coordinate information for each pixel in the street orthophoto. The coordinate information is two-dimensional information. According to the boundary information of the detection frame, the coordinates of the pixel corresponding to the upper left corner of the detection frame and the coordinates of the pixel corresponding to the lower right corner are obtained, thereby obtaining the pixel coordinates of the detection frame. The pixel coordinates are then geometrically corrected to obtain the longitude and latitude coordinates.

[0189] See also Fig.10 , Fig.10 The following is a schematic diagram of the location result attributes, where the ID column indicates the number of the street tree; the Category column indicates the category of the detected street tree, which mainly includes three categories in this article, namely infected plane trees (diseased tree), healthy plane trees (health tree) and other species plane trees (other species tree); the Coordinate point1 column and the Coordinate point2 column are the GPS coordinates (CRS EPSG:4326-WGS 84-geographic coordinate system) of the upper left corner and lower right corner of the detection box, respectively. The Conf column indicates the confidence score in the range of [0,1]. In practical applications, more potential diseased street trees can be displayed by lowering the confidence score threshold.

[0190] It should be noted that the positioning information in this embodiment is the latitude and longitude coordinates of each detection frame. The positioning information is two-dimensional. In actual applications, it can be restored to three dimensions by adding height information.

[0191] The above-mentioned roadside tree crown pest and disease detection method can not only accurately identify the target, but also provide the positioning information of the target, meeting the dynamic monitoring requirements of large-scale urban roadside tree pests and diseases.

[0192] Optionally, the roadside tree crown pest detection method of this embodiment further includes:

[0193] Analyze the test results and generate an analysis report.

[0194] For example, the image information of all streets in the monitoring area can be input into the pest identification model to obtain the detection result, which is the pest category or probability distribution of each image information, as well as the positioning information. By analyzing the detection result, the distribution of pests and diseases on different streets and the severity of each type of pests and diseases can be located, providing an important basis for users to carry out subsequent tree maintenance and tree planning.

[0195] The above-mentioned method for detecting pests and diseases in the crowns of roadside trees processes the image information taken by the drone into an orthophoto to eliminate the distortion of the original image and obtain an image that can accurately reflect the actual situation of the surface; for the annotated orthophoto, it is first divided according to the street, and then the orthophotos belonging to the same street are divided, and finally the training set, verification set and test set are obtained. This can avoid the high overlap rate between adjacent image information, which will cause the same tree to be divided into the training set and verification set, thereby causing serious overfitting of the model; first, data enhancement of the data can generate diversified training samples to prevent the model from overfitting on the training data, thereby improving the performance of the model on unseen data, and then simulating different real-world scenarios to improve the robustness of the model, so that the model can learn stronger and broader features, reduce the model's bias towards specific scenarios or specific conditions, and better identify various types of diseases.

[0196] Furthermore, the feature extraction network adopts the C2f module and the FPN module, which significantly improves the detection ability of various pests and diseases, while effectively reducing redundant calculations, improving the computational efficiency and feature expression ability of the model, and retaining the model performance; the feature fusion network adopts the SPPF network and introduces the average pooling module, which enables the model to make corresponding decisions based on the high-level semantic information in the deep layer of the network and obtain global context information; at the same time, it can effectively alleviate the huge parameters of the fully connected layer, thereby reducing the black box operation of the fully connected layer and achieving a balance between the lightness and accuracy of the model.

[0197] The pest and disease recognition model trained using the above training samples and the original YOLOv8 network model can be used to detect large quantities of street images, realize high-precision automatic recognition and spatial positioning of diseased street trees, save recognition time, reduce investigation costs, and meet the dynamic monitoring requirements of large-scale urban street tree pests and diseases.

[0198] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0199] Based on the same inventive concept, the embodiment of the present application also provides a roadside tree crown pest detection system for implementing the roadside tree crown pest detection method provided above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model training system embodiments provided below can refer to the above limitations on the roadside tree crown pest detection method, and will not be repeated here.

[0200] See also Fig.11 In a feasible embodiment, the present application also provides a street tree crown pest and disease detection system, including: an acquisition module, a processing module, a training module and a detection module.

[0201] An acquisition module is used to acquire image information of the street; wherein the image information is acquired by a drone and carries pathological information of the tree crowns on the roadside trees;

[0202] A processing module is used to pre-process the image information to obtain training samples;

[0203] The training module is used to train the initial YOLOv8 network model based on the training samples to obtain a trained pest and disease recognition model;

[0204] The detection module is used to detect the street image to be identified by using the pest and disease identification model to obtain the detection result; wherein the detection result includes the positioning information of the street image to be identified.

[0205] Optionally, the processing module preprocesses the image information to obtain training samples, including: processing the image information to obtain orthophotos; annotating the orthophotos; dividing the annotated orthophotos into a training set, a validation set, and a test set according to a preset ratio to obtain initial training samples; cropping the initial training samples to obtain first data; and performing data enhancement on the first data to obtain training samples.

[0206] Optionally, the processing module divides the annotated orthophotos into a training set, a verification set, and a test set according to a preset ratio, including: for each orthophoto belonging to the same street, dividing each annotated orthophoto into a sub-training set, a sub-verification set, and a sub-test set according to a preset ratio; each sub-training set, each sub-verification set, and each sub-test set constitutes a training set, a verification set, and a test set, respectively.

[0207] Optionally, the processing module performs data enhancement on the first data in a manner including any one or more of a random rotation operation, a brightness transformation operation, a noise addition operation, a random cropping operation, an image flipping operation, and an image translation operation.

[0208] Optionally, the processing module is also used to perform processing such as cleaning, format conversion, size adjustment and normalization on the image information.

[0209] Optionally, the improved YOLOv8 network model includes an input end, a feature extraction network, a feature fusion network, a detection head and an output end; wherein the feature extraction network adopts a CSPDarknet53 network, including a C2f module and an FPN module; and the feature fusion network adopts an SPPF network, including an average pooling module. Then the training module trains the improved YOLOv8 network model according to the training samples, including: inputting the training set into the original YOLOv8 network model after training, iteratively training the original YOLOv8 network model, and gradually adjusting the model parameters until the preset requirements are met.

[0210] Furthermore, during the training process, the training module uses the validation set to fine-tune the model, helping to select the best model hyperparameters and prevent model overfitting.

[0211] Furthermore, after the training module is completed, the test set is used to evaluate the final performance of the model, and evaluation indicators such as accuracy, precision, recall, F1 value, mean square error, etc. are calculated to measure the model's generalization ability to new data.

[0212] Furthermore, during the training process, the training module can also adjust the training samples as needed.

[0213] Optionally, the detection module uses a pest and disease recognition model to detect the street image to be identified, including: processing the street image to obtain a street orthophoto; first resampling the spatial resolution of the street orthophoto to keep it consistent with the spatial resolution of the training sample; and then adjusting the size of the street orthophoto.

[0214] Optionally, the detection module obtains positioning information by: constructing a plane coordinate system according to the positioning device of the drone, and obtaining pixel coordinates of each detection frame based on the plane coordinate system; converting the pixel coordinates into latitude and longitude coordinates of the detection frame to obtain positioning information.

[0215] Optionally, the roadside tree trunk disease and insect pest detection system provided in the embodiment of the present application further includes: a parsing module.

[0216] The analysis module is used to analyze the detection results and generate analysis reports.

[0217] The above-mentioned street tree crown disease and pest detection system processes the image information taken by the drone into an orthophoto to eliminate the distortion of the original image and obtain an image that can accurately reflect the actual situation of the surface; for the annotated orthophoto, it is first divided according to the street, and then the orthophotos belonging to the same street are divided, and finally the training set, verification set and test set are obtained. This can avoid the high overlap rate between adjacent image information, which will cause the same tree to be divided into the training set and verification set, thereby causing serious overfitting of the model; the data enhancement of the first data can generate diversified training samples to prevent the model from overfitting on the training data, thereby improving the performance of the model on unseen data, and then simulating different real-world scenarios to improve the robustness of the model, so that the model can learn stronger and broader features, reduce the model's bias towards specific scenarios or specific conditions, and better identify various types of diseases.

[0218] Furthermore, the feature extraction network adopts the C2f module and the FPN module, which significantly improves the detection ability of various pests and diseases, while effectively reducing redundant calculations, improving the computational efficiency and feature expression ability of the model, and retaining the model performance; the feature fusion network adopts the SPPF network and introduces the average pooling module, which enables the model to make corresponding decisions based on the high-level semantic information in the deep layer of the network and obtain global context information; at the same time, it can effectively alleviate the huge parameters of the fully connected layer, thereby reducing the black box operation of the fully connected layer and achieving a balance between the lightness and accuracy of the model.

[0219] The pest and disease recognition model trained using the above training samples and the original YOLOv8 network model can be used to detect large quantities of street images, realize high-precision automatic recognition and spatial positioning of diseased street trees, save recognition time, reduce investigation costs, and meet the dynamic monitoring requirements of large-scale urban street tree pests and diseases.

[0220] Each module in the above-mentioned roadside tree crown pest detection system can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0221] In a feasible embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for detecting pests and diseases of street trees can be implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0222] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0223] In a feasible embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the steps in the method for detecting pests and diseases in the crown of roadside trees provided in the above embodiment.

[0224] In a feasible embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is used by a processor to execute the steps of the method for detecting pests and diseases in the crown of roadside trees provided in the above embodiment.

[0225] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0226] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0227] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

[0228] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for detecting diseases and insect pests in the crowns of roadside trees, characterized in that: include: Obtaining image information of the street; wherein the image information is obtained by a drone and carries pathological information of the tree crowns on the roadside trees; Preprocessing the image information to obtain training samples; Based on the training samples, the initial YOLOv8 network model is trained to obtain a trained pest and disease recognition model; The pest and disease identification model is used to detect the street image to be identified to obtain a detection result; wherein the detection result includes the positioning information of the street image to be identified.

2. The method according to claim 1, characterized in that The preprocessing of the image information to obtain training samples includes: Processing the image information to obtain an orthophoto; annotating the orthophoto; According to a preset ratio, the annotated orthophotos are divided into a training set, a validation set, and a test set to obtain initial training samples; Cutting the initial training sample to obtain first data; Data enhancement is performed on the first data to obtain the training samples.

3. The method according to claim 2, characterized in that The processing of the image information to obtain an orthophoto includes: The image information is geometrically corrected and positioned, and then the corrected image information is spliced ​​to obtain an orthophoto.

4. The method according to claim 2, characterized in that The orthophotos after annotation are divided into a training set, a validation set and a test set according to a preset ratio, including: For each of the orthophotos belonging to the same street, the annotated orthophotos are divided into a sub-training set, a sub-validation set, and a sub-test set according to a preset ratio; Each of the sub-training sets, each of the sub-validation sets, and each of the sub-test sets constitutes the training set, the validation set, and the test set, respectively.

5. The method according to claim 2, characterized in that The manner of performing data enhancement on the first data includes: any one or more of a random rotation operation, a brightness transformation operation, a noise addition operation, a random cropping operation, an image flipping operation, and an image translation operation.

6. The method according to claim 1, characterized in that The method of using the pest and disease identification model to detect the street image to be identified includes: Processing the street image to obtain a street orthophoto; First, the spatial resolution of the street orthophoto is resampled to make it consistent with the spatial resolution of the training sample; Then adjust the size of the street orthophoto, and the adjustment formula is shown as: Among them, pred R is the spatial resolution of the street orthophoto; train R is the spatial resolution of the training samples; pred(x) size is the size of the street orthophoto after adjustment in the x direction; pred(x) input is the input size of the street orthophoto in the x direction; train(x) input is the resampling size of the training sample in the x direction; train(x) size is the input size of the training sample in the x direction; pred(y) size is the size of the street orthophoto after adjustment in the x direction; pred(y) input is the input size of the street orthophoto in the x direction; train(y) input is the resampling size of the training sample in the x direction; train(y) size is the input size of the training sample in the x direction.

7. The method according to claim 1, characterized in that The method for obtaining the positioning information includes: Constructing a plane coordinate system according to the positioning device of the UAV, and obtaining pixel coordinates of each detection frame based on the plane coordinate system; Convert the pixel coordinates into the latitude and longitude coordinates of the detection frame to obtain positioning information; The conversion formula is shown as: Xgeo=GT(0)+X*GT(1)+Y*GT(2) Ygeo=GT(3)+X*GT(4)+Y*GT(5) Among them, Xgeo is the latitude and longitude coordinates of the detection frame in the x direction; Ygeo is the latitude and longitude coordinates of the detection frame in the y direction; X is the pixel coordinate of the detection box in the x direction; Y is the pixel coordinate of the detection box in the y direction; GT(0) is the longitude coordinate of the upper left corner of the street orthophoto; GT(3) is the latitude coordinate of the upper left corner of the street orthophoto; GT(1) is the horizontal resolution of the street orthophoto; GT(5) is the vertical resolution of the street orthophoto; GT(2) is the rotation coefficient of the street orthophoto in the x direction; GT(4) is the rotation coefficient of the street orthophoto in the y direction.

8. A roadside tree crown pest detection system, characterized in that: include: An acquisition module is used to acquire image information of the street; wherein the image information is acquired by a drone and carries pathological information of the tree crowns on the roadside trees; A processing module, used for preprocessing the image information to obtain training samples; A training module, used to train the initial YOLOv8 network model based on the training samples to obtain a trained pest and disease recognition model; The detection module is used to use the pest and disease identification model to detect the street image to be identified and obtain a detection result; wherein the detection result includes the positioning information of the street image to be identified.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images