Green rural waste area rapid measurement method based on unmanned aerial vehicle

Abstract

The invention belongs to the technical field of rural environmental protection monitoring, and particularly relates to a green rural waste area rapid measurement method based on an unmanned aerial vehicle. A YOLO rural waste rapid detection model is established at an unmanned aerial vehicle end, so that waste in each frame of image of a green country can be rapidly identified and positioned by the unmanned aerial vehicle; a U-Net rural waste segmentation model is established, semantic segmentation is performed on the identified waste image, and intelligent and rapid outlining of the waste form is realized; according to the method, pixel-level calculation is carried out on the waste area by using a binary center skeleton and a horizontal direction search method, and the physical area of the green rural waste is calculated by calculating a pixel conversion coefficient, so that rapid measurement of the area of the green rural waste is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of rural environmental protection monitoring, and in particular relates to a method for quickly measuring green rural waste areas based on drones. Background Art

[0002] Traditionally, measuring the accumulation area of ​​rural waste (such as straw, construction waste, and domestic garbage) relies primarily on manual field surveys or satellite remote sensing technology. While manual field surveys are highly accurate, they are time-consuming and labor-intensive, requiring significant manpower and financial resources. Furthermore, field surveys struggle to dynamically monitor waste and lack timeliness. Remote sensing measurements are less accurate and can result in unrealistic color reproduction. Furthermore, the raw data collected through remote sensing mapping is prone to errors during processing, leading to significant discrepancies. Furthermore, the effectiveness of manual rural waste management is highly dependent on the supervisors' competence, sense of responsibility, and work attitude, leading to significant uncertainty. Summary of the Invention

[0003] In response to the above-mentioned shortcomings, a green rural waste area rapid measurement method based on drones is proposed. The invention aims to solve the technical problems of dynamic monitoring of existing rural waste and accurate calculation of the accumulation area.

[0004] A method for rapid measurement of waste areas in green villages based on drones, comprising the following steps:

[0005] A method for quickly measuring waste areas in green villages based on drones, the method comprising the following steps:

[0006] Step 1: Establish a YOLO rural waste rapid detection model, which can identify and locate waste and obtain waste images and corresponding distance data.

[0007] Step 2: Establish a U-Net rural waste segmentation model. The U-Net rural waste segmentation model can perform semantic segmentation on the waste image to obtain a waste segmentation image.

[0008] Step 3: The drone is equipped with the YOLO rural waste rapid detection model and the U-Net rural waste segmentation model to determine the rural surrounding environment and waste detection area, and formulate a drone flight plan for green rural waste detection.

[0009] Step 4: According to the developed drone flight plan, control the drone to fly to the designated waste detection area. When the YOLO Rural Waste Rapid Detection Model detects waste, it collects the waste image and the corresponding distance data d; establish the correspondence between the physical imaging coordinates and the pixel coordinates, and calculate the pixel conversion coefficient k, let k = dx = dy, where dx and dy represent the number of length units occupied by a pixel in the x-direction and y-direction, respectively.

[0010] Step 5: The U-Net rural waste segmentation model performs semantic segmentation on the waste image collected in step 4, automatically outlines the waste area, and obtains a waste segmentation image.

[0011] Step 6: Based on the waste segmentation image obtained in step 5, the waste segmentation image is binarized and the skeleton physical length L and skeleton physical width W are calculated using the binary center skeleton method and the horizontal search method. Combined with the distance data d and the pixel conversion coefficient k obtained in step 4, the waste physical area M is calculated. The calculation formula for the waste physical area M is:

[0012]

[0013] Where N is the number of pixels in the waste skeleton; k is the pixel conversion coefficient; d is the distance data; d1 and d2 are the number of pixels from each skeleton pixel to the boundary along the horizontal directions of 0° and 180° respectively; f is the focal length of the camera.

[0014] As a preferred option:

[0015] The YOLO rural waste rapid detection model established in step 1 uses self-photographed waste images, manually collected on-site waste images, and network-shared waste images as data sets for model training. The data sets are divided into training sets and validation sets, and the ratio of training sets to validation sets is 8:2; YOLOs.pt is used as the model pre-training weight; the training parameters are set to: learning rate = 0.01, momentum parameter = 0.8, weight decay = 0.0005, iteration limit = 1000, training batch = 8, number of first training rounds = 350, and training time is 4 hours.

[0016] The accuracy of the YOLO rural waste rapid detection model established in step 1 is not less than 92.28%, the recall rate is not less than 93.26%, and the average precision is not less than 95.63%.

[0017] The ratio of the training set and validation set in the dataset used by the U-Net rural waste segmentation model described in step 2 is 8:2; VGG is selected as the backbone network; the training parameters are set as: training batch = 8, learning rate = 0.01, momentum parameter = 0.8, the maximum number of training rounds is 160 rounds, and the training time is 3.5 hours.

[0018] The accuracy of the rural waste segmentation model of the U-Net described in step 2 is not less than 92.89%, the recall rate is not less than 94.36%, and the average precision is not less than 93.42%.

[0019] The specific calculation steps for the physical area of ​​waste in step 6 are:

[0020] (1) Binarize the waste segmentation image and find the waste center skeleton line in the waste segmentation image binary skeleton map based on the binary center skeleton method. Let the number of pixels of the waste center skeleton be N = ∑(w1+w2+w3...w n );

[0021] (2) Calculate the center skeleton length: If the pixel conversion coefficient is known to be k, then the physical length of the skeleton is L = N × k;

[0022] (3) Use the horizontal search method to measure each skeleton pixel w1, w2, w3...w n The number of pixels along the two horizontal directions of 0° and 180° to the boundary is d1 and d2. The number of pixels in the corresponding directions in the two horizontal directions is added together to obtain the pixel width of each skeleton pixel as d1+d2. The pixel width of the central skeleton is Skeleton physical width

[0023] (4) The physical area of ​​waste is

[0024] This invention can rapidly measure the accumulation area of ​​rural waste, enabling dynamic monitoring of rural waste and accurate calculation of accumulation area. The method of the present invention accurately meets the requirements for green waste treatment in beautiful rural transformation projects, significantly improves safety and inspection efficiency, and offers significant advantages in green rural waste management. The rural waste images identified and waste area calculated by the method of the present invention can provide real-time decision support for rural waste management and resource scheduling, and provide parameter support for green waste treatment technologies in beautiful rural transformation projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a method for rapidly measuring the area of ​​rural waste dumping based on a drone, provided by an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of a UAV test in an embodiment of the present invention;

[0027] Figure 3 It is an embodiment of the present invention that a drone collects images and data of rural waste;

[0028] Figure 4 The imaging size principle and conversion relationship of the drone camera in the embodiment of the present invention;

[0029] Figure 5 2. YOLO rural waste rapid detection model diagram according to an embodiment of the present invention;

[0030] Figure 6 This is a diagram of a U-Net rural waste segmentation model according to an embodiment of the present invention;

[0031] Figure 7 This is a flow chart for calculating the physical area of ​​rural waste in an embodiment of the present invention;

[0032] Figure 8 It is a point cloud image of rural waste collected by a drone in an embodiment of the present invention;

[0033] Figure 9 This is an example diagram of calculating the physical area of ​​rural waste in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0035] Step 1: Build the YOLO rural waste rapid detection model. The YOLO rural waste rapid detection model can identify and locate waste, obtaining waste images and distance data.

[0036] The process of establishing the YOLO rural waste rapid detection model is as follows:

[0037] The model training dataset consisted of self-photographed waste images, images collected on-site, and images shared online. Labelmg annotation software was used to manually annotate all waste information contained in these datasets. In this model, only the target category, Waste, was defined. The dataset was divided into a training set and a validation set with an 8:2 ratio to evaluate model performance and provide a reliable assessment of the model's generalization ability, facilitating timely parameter adjustments and optimizing the YOLO rural waste rapid detection model.

[0038] Model training was conducted on an Intel Core i9-13900kf processor, an NVIDIA GeForce RTX3090-24GB graphics card, and 32GB of RAM. YOLOs.pt was used as the pre-trained model weights to optimize training speed. The input image size of the dataset was 640×640 pixels. Training parameters were set as follows: learning rate = 0.01; momentum = 0.8; decay = 0.0005; burn in = 1000 (i.e., when the number of iterations exceeds 1000, the learning rate update method is changed); batch size = 8, and the number of first training rounds = 350. Python 3.8 and the PyCharm deep learning framework were used for development and design. Training took 4 hours, resulting in the YOLO rural waste rapid detection model and training parameters.

[0039] To ensure better waste identification effects, the accuracy, recall rate, and average precision of the trained YOLO rural waste rapid detection model should be no less than 92.28%, 93.26%, and 95.63%, respectively. When the accuracy, recall rate, and average precision are lower than the above values, the YOLO rural waste rapid detection model can be further optimized by expanding the data set, changing some training parameters, and increasing the number of training rounds to ensure that the model has better waste identification effects.

[0040] Step 2: Build a U-Net rural waste segmentation model. The U-Net rural waste segmentation model can perform semantic segmentation on waste images, automatically outline waste areas, and obtain waste segmentation images.

[0041] The process of building the U-Net rural waste segmentation model is as follows:

[0042] The waste images from the YOLO Rural Waste Rapid Detection Model can continue to be used as the dataset, or new waste images can be added to expand the dataset, still splitting the dataset into training and validation sets at an 8:2 ratio. The dataset was manually annotated using Labelme image annotation software, with only the "Waste" category set.

[0043] The model training was developed and designed using the same computer environment, programming language and PyCharm deep learning framework as the YOLO rural waste rapid detection model. The U-Net rural waste segmentation model used VGG as the backbone network to optimize the model training speed. The input image was 512*512 pixels, and the training parameters were set to: batch size (training batch) = 8; learning rate (learning rate) = 0.01; momentum (momentum parameter) = 0.8; the maximum number of training rounds was 160 rounds; the model training time was 3.5 hours, and the weight when the network output loss was the lowest was used to construct the U-Net rural waste segmentation model.

[0044] By analyzing the inference performance and model evaluation indicators of the U-Net network model, the trained U-Net rural waste segmentation model was optimized by increasing the number of training rounds and enhancing the data set. The enhanced U-Net network model was then trained, and the optimal weights obtained from the second training were used for model inference. Finally, the optimized U-Net rural waste segmentation model was used for inference. To ensure the segmentation effect, the accuracy, recall, and average precision of the U-Net rural waste segmentation model on the test set were preferably not less than 92.89%, 94.36%, and 93.42%, respectively. If the accuracy is lower than the above values, optimization is required, such as increasing the number of training rounds and increasing the data set, to ensure that the model has better inference effect, more refined waste segmentation, and the outlined shape is closer to the contour of real waste.

[0045] Step 3: The drone is equipped with the YOLO rural waste rapid detection model and the U-Net rural waste segmentation model to determine the rural surrounding environment and waste detection area, and formulate a drone flight plan for green rural waste detection.

[0046] Select the waste area of ​​the target village, determine the key waste storage area in the village based on the village drawings and previous waste location reports, survey the surrounding environment near the village, develop a drone flight plan for waste detection, and finally check the local drone flight policy and apply for flight approval from the air traffic management agency of the local Civil Aviation Administration in a timely manner, paying attention to complying with relevant laws, regulations and safety regulations.

[0047] Step 4: According to the developed flight plan, the drone is controlled to fly to the designated waste detection area. When the YOLO Rural Waste Rapid Detection Model detects waste, it collects waste images and corresponding distance data d. A correspondence between physical image coordinates and pixel coordinates is established, and the pixel conversion coefficient k is calculated. Using the drone's wireless transmission function and local storage on an SD card, the drone transmits the collected waste images and distance data d to the ground terminal and backs them up on the SD card.

[0048] The raw data used in the waste detection based on drone vision technology is mainly the image data collected by the drone. The images collected by the drone through the gimbal camera are based on the camera imaging principle. The following analysis includes the camera imaging principle, coordinate system conversion relationship, camera parameter acquisition, etc., to provide theoretical support for subsequent waste image processing and pixel calculation. The process is as follows:

[0049] Principles of UAV Visual Imaging

[0050] The principle of drone visual imaging is to map spatial points in the three-dimensional world to a two-dimensional image plane through a camera. The imaging process involves four coordinate systems and the conversion between them. The four coordinate systems are set as follows:

[0051] World Coordinate System: OXYZ

[0052] Physical imaging coordinate system: O′-x′-y′

[0053] Camera coordinate system: Oxy

[0054] Pixel coordinate system: Ouv

[0055] Point P[X, Y, Z] in the world coordinate system T , projected from the optical center O in the camera coordinate system to the physical imaging coordinate system as point P'[X', Y', Z'] T , which corresponds to the point [u, v] in the pixel coordinate system T , the process can be expressed as:

[0056]

[0057] Here the negative sign indicates the inverted image. To simplify the model, the physical imaging plane is considered to be placed in front of the camera, which gives:

[0058]

[0059] Assume that the pixel coordinates are scaled by α times in the u direction and β times in the v direction; the origin is translated by [c x , c y ] T Then the point P'[X', Y', Z'] and the point [u, v] in the pixel coordinate system are T The relationship is:

[0060]

[0061] Can be transformed into:

[0062]

[0063] Where, the unit of f is mm; the unit of α and β is mm / pixel; f x , f y The unit is pixel.

[0064] Normalize the coordinates, homogenize the pixel coordinates on the left, and write them in matrix form. The matrix composed of the quantities in the middle is called the intrinsic parameter matrix K of the camera.

[0065] Get the internal parameter matrix

[0066] Assume the image size [w, h], unit pixel; camera sensor size [m, n], unit mm; camera focal length f, unit mm; pixel unit length dx, dy, unit mm / pixel; internal parameter f x , f y , unit: pixel.

[0067]

[0068] f x Represents the component of the camera focal length f in the x direction, which is equivalent to quantifying the physical focal length f with the number of pixels in the x direction;

[0069] f y Represents the component of the camera focal length f in the y direction, which is equivalent to quantifying the physical focal length f with the number of pixels in the y direction;

[0070] The intrinsic parameters of the camera are fixed. We use the method of finding the intrinsic parameters using the known hardware parameters of the camera. Assuming the camera sensor size is m×n (mm) and the image size is b×h (pixel), the pixel unit length is:

[0071]

[0072] If dx=dy, the image pixel unit is a square, and f x =f y , unit: mm / pixel;

[0073] If dx≠dy, the image pixel unit is a rectangle, and f x ≠f y , unit: mm / pixel.

[0074] The pixel conversion coefficient k is calculated based on the above principle. The pixel unit length when dx = dy is the pixel conversion coefficient k. To achieve dx = dy and facilitate subsequent image processing, if dx ≠ dy, the image size and camera sensor size are adjusted until dx = dy, thus obtaining the pixel conversion coefficient k.

[0075] Step 5: The U-Net rural waste segmentation model performs semantic segmentation on the waste image collected in step 4, automatically outlines the waste area, and obtains a waste segmentation image.

[0076] Step 6: Based on the waste segmentation image obtained in step 5, the waste segmentation image is binarized and the skeleton physical length L and skeleton physical width W are calculated using the binary center skeleton method and the horizontal search method. Combined with the distance data d and the pixel conversion coefficient k obtained in step 4, the waste physical area M is calculated. The calculation formula for the waste physical area M is:

[0077]

[0078] Where N is the number of pixels in the waste skeleton; k is the pixel conversion coefficient; d is the distance data; d1 and d2 are the number of pixels from each skeleton pixel to the boundary along the horizontal directions of 0° and 180° respectively; f is the focal length of the camera.

[0079] The following are the specific steps for calculating the physical area of ​​waste based on drone vision:

[0080] (1) First, the waste segmentation image is binarized. Based on the binary center skeleton method, the waste center skeleton line is found in the binary skeleton image of the waste segmentation image. The number of pixels of the waste center skeleton is set to N = ∑(w1+w2+w3...w n ).

[0081] (2) Calculate the length of the central skeleton: If the pixel conversion coefficient is known to be k, then the physical length of the skeleton is L = N × k.

[0082] (3) Use the horizontal search method to measure each skeleton pixel w1, w2, w3...w n The number of pixels along the two horizontal directions of 0° and 180° to the boundary is d1 and d2. By adding the number of pixels in the corresponding directions in the two horizontal directions, it can be concluded that the pixel width of each skeleton pixel is d1+d2, and the pixel width of the central skeleton is Skeleton physical width

[0083] (4) The physical area of ​​waste is

[0084] The present invention is explained in more detail below through a specific example:

[0085] The Green Village project in this example is located in Huaiyin District, Jinan City. It runs from north to south and connects to the Qilu Yellow River Bridge to the north. It is 697.2m long and 68.8m wide, including roads, villages, green areas, and waste. The key areas of rural waste are as follows: Figure 2 shown.

[0086] The rapid measurement site of rural waste dumping area based on drone, as shown in the attached Figure 3 As shown in the figure, staff control the drone according to the flight plan to reach the rural waste detection area and collect waste images and distance data.

[0087] Calculate the pixel conversion coefficient k, select the drone's camera as a 1 / 1.7-inch CMOS sensor, select a sensor size of 7.4mm × 5.6mm, and select a fixed-size waste image size of 3700 × 2800 pixels. The pixel unit length is:

[0088]

[0089] Then dx=dy, the image pixel unit is a square, and f x =f y , then the pixel conversion coefficient k is 0.002mm / pixel.

[0090] When the YOLO rural waste rapid detection model on the drone side detects waste in the countryside, it triggers the shutter of the drone's gimbal camera to record the waste image of the current frame and its corresponding distance data.

[0091] The waste image is input into the U-Net rural waste segmentation model for semantic segmentation, which automatically outlines the waste area morphology and performs binarization processing to perform pixel-level measurement on the waste segmentation image.

[0092] The drone captured 23 images of waste in the countryside, saving both the images and the corresponding distance data. Semantic segmentation was then performed on the waste images, and the U-Net rural waste segmentation model was used to delineate the shapes of the waste. Ultimately, semantic segmentation of the 23 waste images was completed, resulting in a waste shape delineation image. Finally, the physical area of ​​each waste site was calculated.

[0093] The calculated results of the physical area of ​​waste were compared with the actual waste area measured by the instrument. All the results are listed in Table 1 below.

[0094] Table 1 Comparison of the method of the present invention and actual measurement results

[0095]

[0096] In summary, the present invention uses drones to collect graphics and data on rural waste, realizes the rapid measurement of waste area, and realizes the global information representation of green rural waste positioning, shaping, and quantitative calculation based on drones. Compared with manual on-site inspections, this method has higher waste detection accuracy and stability, and shows great advantages in the efficiency of calculating the physical area of ​​waste.

[0097] In the above description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed above.

Claims

1. A method for rapid measurement of green rural waste area based on drones, characterized in that: The method comprises the following steps: Step 1: Establish a YOLO rural waste rapid detection model, which can identify and locate waste and obtain waste images and corresponding distance data; Step 2: Establish a U-Net rural waste segmentation model. The U-Net rural waste segmentation model can perform semantic segmentation on the waste image to obtain a waste segmentation image; Step 3: The drone is equipped with the YOLO rural waste rapid detection model and the U-Net rural waste segmentation model to determine the rural surrounding environment and waste detection area, and develop a drone flight plan for green rural waste detection; Step 4: According to the developed drone flight plan, control the drone to fly to the designated waste detection area. When the YOLO rural waste rapid detection model detects waste, it collects waste images and corresponding distance data. ; Establish the correspondence between physical imaging coordinates and pixel coordinates, and calculate the pixel conversion coefficient ,let ,in 、 Indicates how many length units a pixel occupies in the x-direction and y-direction respectively; Step 5: The U-Net rural waste segmentation model performs semantic segmentation on the waste image collected in step 4, automatically outlines the waste area, and obtains a waste segmentation image; Step 6: Based on the waste segmentation image obtained in step 5, perform binarization on the waste segmentation image and calculate the physical length of the skeleton using the binary center skeleton method and horizontal search method. and the physical width of the skeleton , combined with the distance data obtained in step 4 and pixel conversion coefficients , calculate the physical area of ​​waste , waste physical area The calculation formula is: in, N is the number of pixels of the waste skeleton; is the pixel conversion coefficient; is the distance data; d 1 、 d 2 The number of pixels from each skeleton pixel to the boundary along the horizontal directions of 0° and 180° respectively; is the camera focal length.

2. The method for rapid measurement of green rural waste area based on drone according to claim 1 is characterized in that: The YOLO rural waste rapid detection model established in step 1 uses self-photographed waste images, manually collected waste images on site, and network-shared waste images as the model training dataset. The dataset is divided into a training set and a validation set, with a training set to validation set ratio of 8:

2. YOLOs.pt is used as the model pre-training weight; The training parameters are set as follows: learning rate = 0.01, momentum parameter = 0.8, weight decay = 0.0005, iteration limit = 1000, training batch size = 8, number of first training rounds = 350, and training time is 4 h.

3. The method for rapid measurement of green rural waste area based on drone according to claim 1 is characterized in that: The accuracy of the YOLO rural waste rapid detection model established in step 1 is not less than 92.28%, the recall rate is not less than 93.26%, and the average precision is not less than 95.63%.

4. The method for rapid measurement of waste areas in green villages based on drones according to claim 1, characterized in that: The ratio of the training set and validation set in the dataset used by the U-Net rural waste segmentation model described in step 2 is 8:2; VGG is selected as the backbone network; the training parameters are set as follows: training batch = 8, learning rate = 0.01, momentum parameter = 0.8, the maximum number of training rounds is 160 rounds, and the training time is 3.5 h.

5. The method for rapid measurement of green rural waste area based on drone according to claim 1 is characterized in that: The accuracy of the rural waste segmentation model of the U-Net described in step 2 is not less than 92.89%, the recall rate is not less than 94.36%, and the average precision is not less than 93.42%.

6. The method for rapid measurement of waste areas in green villages based on drones according to claim 1 is characterized in that: The specific calculation steps for the physical area of ​​waste in step 6 are: (1) Binarize the waste segmentation image and find the waste center skeleton line in the waste segmentation image binary skeleton map based on the binary center skeleton method. Let the number of pixels of the waste center skeleton be ; (2) Calculate the center skeleton length: The pixel conversion coefficient is known to be , then the physical length of the skeleton is L = N × k； (3) Use the horizontal search method to measure each skeleton pixel 、、... The number of pixels from the 0° and 180° horizontal directions to the boundary d 1 、 d 2 , add the number of pixel points in the corresponding directions in the two horizontal directions, and the pixel width of each skeleton pixel is obtained as d 1 + d 2 , then the pixel width of the central skeleton is ， Skeleton physical width ; (4) The physical area of ​​waste is .

Images