Method and system for monitoring invasive plants outside road domain based on vehicle-mounted image

Abstract

The invention relates to the technical field of plant monitoring, and provides a method and a system for monitoring an invasive plant outside a road domain based on a vehicle-mounted image, and the method comprises the following steps: 1, collecting the image data of an alien invasive plant in the field; 2, performing classification according to biological characteristics of alien invasive plants in ecology, performing picture labeling according to minimum characteristic points, and making a training set and a verification set according to a certain proportion; and 3, putting into an optimized YOLOv8 algorithm to carry out model training, and improving the model identification accuracy through multiple times of training set optimization and material capacity addition. According to the invention, invasive plants outside the road region can be well monitored.

Description

Technical Field

[0001] The present invention relates to the technical field of plant monitoring, and in particular to a method and system for monitoring invasive alien plants on roads based on vehicle-mounted images. Background Art

[0002] Biological invasions have caused huge ecological impacts, health damage and economic losses, and have become a serious global environmental problem. In order to mitigate or minimize the potential harm of these biological invasions, effective real-time monitoring and timely control of alien invasive species are crucial. The target detection algorithm based on deep learning uses the Convolutional Neural Networks (CNN) feature learning method. This feature learning method can automatically discover the features required for detection and classification of targets, and at the same time, the original input information is converted into more abstract and higher-dimensional features through the convolutional neural network. This high-dimensional feature has strong feature expression ability and generalization, so its performance in complex scenarios is good and can meet most application requirements. The YOLO (You Only Look Once) series of algorithms are typical of One-Stage target detection algorithms. Considering that the application scenario of this design is fast and accurate alien plant detection, YOLOv8 will be selected to complete the model training. YOLOv8 inherits the excellent improvement strategies of earlier versions, such as no anchor box, decoupled head, training data enhancement, etc. At the same time, it replaces the C3 module of YOLOv5 with a C2f module with more residual connections, which improves the detection accuracy and provides more abundant hyperparameters and model structure options, making it easier to adjust and optimize the model in the application.

[0003] At present, some relevant units have used imaging equipment such as vehicle-mounted dashcams to monitor the road surface, and used deep learning target detection technology to identify some road surface diseases. This technology uses target detection algorithms and some road surface disease pictures (road cracks, potholes, etc.) to train the model. When the model reaches a certain accuracy, it can be put into road monitoring. The monitoring solution used is to install image acquisition devices at various angles in the car, similar to vehicle-mounted dashcams, which can collect detailed pictures of the entire road surface. When put into the recognition model, the road surface diseases existing in the section can be counted. In the field of plant recognition, there are already many mature plant recognition technologies on the market (such as flower companions, shape and color, etc.). They have a large number of plant pictures to train big data models, and the classification of plants in static shooting scenes has a certain accuracy.

[0004] However, existing plant identification technologies mainly serve to popularize plant classification for the general public, with a focus on ornamental plants or plants with special uses, etc. The main images to be identified are still, close-range handheld shots, which have good plant classification effects. However, there are some limitations in applying these technologies to invasive plant monitoring. Alien invasive plants usually grow among many plants. When multiple species exist in the picture at the same time, ordinary plant classification techniques are susceptible to significant interference. In particular, this design uses plants captured by on-board images for identification in order to improve the efficiency of invasive plant monitoring in road areas. Since the car is in motion, the detailed features of the plants are less and the angle and quality of the images are quite different from those of still shots. Existing plant classification technologies make it difficult to identify accurately. Summary of the invention

[0005] The present invention provides a method and system for monitoring invasive alien plants on roads based on vehicle-mounted images, which can solve the problem of difficulty in monitoring invasive alien plants on roads.

[0006] According to the present invention, a method for monitoring invasive alien plants on roads based on vehicle-mounted images comprises the following steps:

[0007] 1. Collect images of invasive alien plants in the wild;

[0008] Second, classify the invasive alien plants according to their biological characteristics and annotate the images according to the minimum feature points, and prepare training sets and validation sets according to a certain ratio;

[0009] 3. Invest in the optimized YOLOv8 algorithm for model training, and improve the model recognition accuracy through multiple training set optimizations and material capacity additions.

[0010] Preferably, in step one, the image data includes plant morphology images from various angles in different growth periods and under different environmental backgrounds.

[0011] Preferably, in step one, when collecting image data, the hardware end is a vehicle-mounted camera. According to the growth and distribution characteristics of invasive alien plants in the road area, the vehicle-mounted camera is set to a 45-degree angle with the roadside, with two camera positions, one above and one below. The dual camera positions take into account the monitoring of low herbs and tall shrubs to avoid missing the target.

[0012] Preferably, in step 2, in order to improve the recall rate and recognition accuracy of the model under vehicle-mounted conditions, images of invasive alien plants are collected from the vehicle-mounted perspective, and images under different weather conditions, different backgrounds, and different vehicle speeds are collected in field experiments to produce a training set.

[0013] As a preferred method, in step three, in the optimized YOLOv8 algorithm, the adaptive spatial feature fusion ASFF mechanism is introduced to optimize the YOLO algorithm detection head, and an additional small target detection layer is added to improve the detection capability of small leaf targets; in the feature fusion stage, the multi-scale expansion MSDA attention mechanism is introduced to help the model fuse features at different levels; an end-to-end defogging network AOD-Net is introduced before the feature extraction backbone network to enhance the detection capability of the model; based on the above optimization, in order to make the algorithm model lightweight, the SPDConv and Slim-Neck architectures are used to optimize the backbone network and feature fusion layer respectively, reducing the amount of calculation without losing the model learning ability and accuracy.

[0014] The present invention provides a road-area invasive alien plant monitoring system based on vehicle-mounted images, which adopts the above-mentioned road-area invasive alien plant monitoring method based on vehicle-mounted images.

[0015] The present invention designs a unique complete monitoring method and system based on the distribution characteristics and biological characteristics of invasive alien plants on roads. First, the YOLOv8 algorithm is improved in a targeted manner, and then a large amount of image feature information of invasive alien plants is collected through field experiments. After classification and labeling according to their biological characteristics, a unique recognition model is trained to make up for some gaps in the existing plant recognition software in the field of invasive alien plants. According to the special circumstances of on-board highway shooting and the growth distribution of invasive alien plants on roads, the shooting position is set and training materials under special viewing angles are collected, so that in on-board monitoring applications that are not supported by existing recognition software, there are better detection rates and recognition rates, and finally the problem of difficult monitoring of invasive alien plants on roads is solved.

[0016] The present invention can monitor and count the invasive alien plants on both sides of some roads (national roads, provincial roads, rural roads, expressways, etc.) in real time. The on-board imaging equipment can capture the species characteristics of roadside invasive alien plants under the specified parameters, and the relevant background personnel can accurately identify the species of plants and observe their growth conditions through pictures; at the same time, the performance of the YOLOv8 algorithm optimized by the present invention in the road invasive plant data set has been improved compared with the performance before optimization: in the validation set, both average accuracy indicators have been improved, mAP50 has been improved by 3%, and mAP50-95 has been improved by 7.8%. After light optimization, the floating point operation parameter GFLOPs has been reduced by 7 (from 51GFLOPs to 44GFLOPs). In the actual application test, the optimized algorithm also shows advantages: it has improved the detection ability of some small target leaves, and compared with the original model, it has improved the ability to distinguish the characteristics of airplane grass and purple stem eupatorium, which have similar leaves, and reduced some misidentification. The detection ability in some extreme environments (dark light, blur) has been improved.

[0017] With the help of the optimized YOLOv8 algorithm, a target detection and recognition model including common invasive plants on the road was trained. Compared with the common plant recognition software on the market, it has a better recognition rate for invasive alien plants. Taking the common invasive alien plants on the roadside, Eupatorium adenophorum and Eupatorium sphaerocephalum, as examples, the recognition rate of the model for the leaves of Eupatorium adenophorum and Eupatorium sphaerocephalum in the relatively complex validation set reached more than 80%, and it also had a good recognition rate in the field materials. The entire monitoring system can capture the characteristic information of invasive plants on the roadside and accurately identify them while the car is driving. It can detect and count invasive plants on the road in real time, reduce a lot of manpower field work, and can provide reasonable suggestions to relevant departments by monitoring the growth trend and spread rate of invasive alien plants, and take timely invasive plant prevention and control measures, which is of great significance to alleviate the harm of biological invasion. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of a method for monitoring invasive alien plants on roads based on vehicle-mounted images in an embodiment;

[0019] Figure 2 It is a schematic diagram of the detection results in the embodiment. DETAILED DESCRIPTION

[0020] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

[0021] Example

[0022] like Figure 1 As shown, this embodiment provides a method for monitoring invasive alien plants on roads based on vehicle-mounted images, which includes the following steps:

[0023] 1. Collect images of invasive alien plants in the wild; the images include plant morphology from various angles in different growth cycles and different environmental backgrounds.

[0024] When collecting image data, the hardware end is the vehicle-mounted camera equipment. According to the growth and distribution characteristics of alien invasive plants in the road area, the vehicle-mounted camera equipment is set to a 45-degree angle with the roadside, with two camera positions, one above and one below. The dual camera positions take into account the monitoring of low herbs and tall shrubs to avoid missing the target.

[0025] Second, accurately classify the alien invasive plants based on their biological characteristics and annotate the images based on the smallest feature points (such as flowers and leaves), and prepare training and verification sets in a certain ratio.

[0026] In order to improve the recall rate and recognition accuracy of the model when in vehicle conditions, we collected images of invasive alien plants from a vehicle perspective during field trips and collected images under different weather conditions, backgrounds, and vehicle speeds in field experiments to create a training set to improve the stability and accuracy of the vehicle-mounted invasive alien plant monitoring system. This allows the car to capture plant images and accurately identify species while driving.

[0027] 3. Invest in the optimized YOLOv8 algorithm for model training, and improve the model recognition accuracy through multiple training set optimizations and material capacity additions.

[0028] In the optimized YOLOv8 algorithm, the adaptive spatial feature fusion (ASFF) mechanism is introduced to optimize the YOLO algorithm detection head in view of the diverse shapes and sizes of invasive plant leaves from the vehicle's perspective, and an additional small target detection layer is added to improve the detection ability of small leaf targets; the multi-scale expansion (MSDA) attention mechanism is introduced in the feature fusion stage to help the model more effectively fuse features at different levels and improve the overall understanding of image content, so that the model has better performance in identifying invasive plants with complex backgrounds; an end-to-end defogging network AOD-Net is introduced before the feature extraction backbone network to enhance the model's detection ability for some blurry scenes such as rainy and foggy days or unclear pictures taken while the vehicle is driving; based on the above optimizations, in order to make the algorithm model lightweight, the SPDConv and Slim-Neck architectures with lower computational costs are used to optimize the backbone network and feature fusion layer respectively, reducing the amount of computation without losing the model's learning ability and accuracy, which is particularly beneficial for environments with limited computing resources.

[0029] The present embodiment provides a road-area invasive alien plant monitoring system based on vehicle-mounted images, which adopts the above-mentioned road-area invasive alien plant monitoring method based on vehicle-mounted images.

[0030] Effect test

[0031] The car was always traveling at a speed of about 40 yards on the test route, and a number of pictures were taken by smartphones and micro-single cameras. The main invasive alien plants on the road, Eupatorium adenophorum, were used as the main monitoring targets for system testing. To ensure the comprehensiveness of the test, 20 pictures of the road section under different light conditions, different vegetation densities and different picture qualities were extracted from all the materials and put into the recognition model for testing. The results showed that except for 2 pictures where there were no invasive plants, the remaining 18 pictures completed the species identification and statistics.

[0032] The test results are shown in the figure Figure 2As shown, the dark frame is the airplane grass, and the light frame is the purple loosestrife. The current monitoring system is easy to misidentify the airplane grass and purple loosestrife, and even misidentify the stone. The method proposed in this embodiment can accurately identify the airplane grass and purple loosestrife.

[0033] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. A method for monitoring invasive alien plants on roads based on vehicle-mounted images, characterized in that: The following steps are involved:

1. Collect images of invasive alien plants in the wild; Second, classify the invasive alien plants according to their biological characteristics and annotate the images according to the minimum feature points, and prepare training sets and validation sets according to a certain ratio; 3. Invest in the optimized YOLOv8 algorithm for model training, and improve the model recognition accuracy through multiple training set optimizations and material capacity additions.

2. The method for monitoring invasive alien plants on roads based on vehicle-mounted images according to claim 1, characterized in that: In step one, the image data includes plant morphology images from various angles under different growth periods and different environmental backgrounds.

3. The method for monitoring invasive alien plants on roads based on vehicle-mounted images according to claim 2, characterized in that: In step one, when collecting image data, the hardware end is the vehicle-mounted camera equipment. According to the growth and distribution characteristics of alien invasive plants in the road area, the vehicle-mounted camera equipment is set to a 45-degree angle with the roadside, with two camera positions, one above and one below. The dual camera positions take into account the monitoring of low herbs and tall shrubs to avoid missing the target.

4. The method for monitoring invasive alien plants on roads based on vehicle-mounted images according to claim 3 is characterized by: In step 2, in order to improve the recall rate and recognition accuracy of the model under vehicle-mounted conditions, images of invasive alien plants were collected from the vehicle-mounted perspective, and pictures under different weather conditions, different backgrounds, and different vehicle speeds were collected in field experiments to create a training set.

5. The method for monitoring invasive alien plants on roads based on vehicle-mounted images according to claim 4 is characterized by: In step three, in the optimized YOLOv8 algorithm, the adaptive spatial feature fusion ASFF mechanism is introduced to optimize the YOLO algorithm detection head, and an additional small target detection layer is added to improve the detection ability of small leaf targets; in the feature fusion stage, the multi-scale expansion MSDA attention mechanism is introduced to help the model fuse features at different levels; an end-to-end dehazing network AOD-Net is introduced before the feature extraction backbone network to enhance the detection ability of the model; based on the above optimization, in order to make the algorithm model lightweight, the SPDConv and Slim-Neck architectures are used to optimize the backbone network and feature fusion layer respectively, reducing the amount of calculation without losing the model learning ability and accuracy.

6. The road-area alien invasive plant monitoring system based on vehicle-mounted images is characterized by: It adopts the method for monitoring invasive alien plants on roads based on vehicle-mounted images as described in any one of claims 1-5.

Images