A method for quickly locating and verifying typical garbage on the coastline based on drone aerial photography technology

Abstract

The present invention discloses a method for quickly locating and checking typical garbage on a coastline based on drone aerial photography technology, comprising the following steps: selecting suitable weather conditions to periodically carry out drone aerial photography along the coastline according to meteorological and tidal forecasts, and collecting a collection of drone aerial photography photos of the coastline; acquiring the longitude and latitude information and time series information of the photos based on the photo collection to construct a coastline spatiotemporal image information sequence; establishing a rapid garbage target recognition algorithm according to the color, outline, and floating characteristics of typical garbage on the coastline, and performing operations on the coastline spatiotemporal image information sequence one by one to identify and mark the coastline spatiotemporal image information sequence, so as to obtain the longitude and latitude information of typical garbage targets in the coastline spatiotemporal image information sequence; dividing the coastline into sections, and using drones to carry out aerial photography inspections of each section, so as to quickly obtain the garbage distribution, size, and positioning information of each section, and uploading the summary report to the control system, and sending it to on-site cleaning personnel to achieve efficient coast cleaning.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) aerial photography garbage inspection, and in particular relates to a method for quickly locating and verifying typical garbage on a coastline based on UAV aerial photography technology. Background Art

[0002] The management of marine and coastal garbage is a global coastal ecological and environmental problem. Under the influence of ocean currents and waves, garbage from rivers entering the sea, garbage caused by human activities on the shore, and various types of garbage floating on the ocean will cause serious coastal garbage pollution. Not only will it damage the coastal ecological environment, but if it is not cleaned up in time, the microplastics formed by the gradual decomposition of marine garbage will also enter marine life and human body through the food chain, poisoning marine life and human health.

[0003] Timely cleaning of marine and coastal garbage is extremely important. However, given the long coastline and high dispersion of garbage, the efficiency of garbage distribution monitoring directly affects the efficiency of garbage cleaning. Traditional methods of monitoring marine and coastal garbage mainly rely on regular patrols and observations by coastal environmental protection personnel. These methods are highly subjective, time-consuming, have a small coverage area, are slow to inspect, have poor horizontal vision, are difficult to locate garbage, and leave traces when cleaning. In addition, high labor costs and repeated investment in personnel can easily lead to a lack of supervision, which cannot meet the needs of the long coastline for efficient environmental governance and ecological protection.

[0004] The garbage inspection method based on drone aerial photography has received widespread attention. Usually, binocular image reconstruction technology is used to reconstruct and locate garbage on the coastline. However, this requires a 70% image repetition rate between adjacent aerial images to achieve 3D reconstruction of garbage targets on the coastline. On the one hand, it causes slow inspection speed and low efficiency, and requires extremely strong computing power, which puts high demands on hardware equipment. The 3D reconstruction of the image takes a long time, and it takes a long time to reconstruct the image after each aerial photography. It lacks timeliness and is not conducive to promotion and application. Summary of the invention

[0005] The purpose of the present invention is to propose a method for quickly locating and verifying typical garbage on the coastline based on drone aerial photography technology, using an efficient image processing algorithm to quickly locate typical garbage targets in a spatiotemporal image sequence, and periodically compare and verify them, thereby solving the defects of traditional coastline garbage inspections, such as slow speed, complex garbage positioning algorithms, and difficult verification.

[0006] To achieve the above objectives, the present invention provides a method for quickly locating and checking typical garbage on the coastline based on drone aerial photography technology, comprising:

[0007] According to weather and tidal forecasts, select suitable weather conditions to periodically conduct drone aerial photography along the coastline and collect a collection of drone aerial photos of the coastline;

[0008] Based on the photo collection, the longitude and latitude information and time sequence information of the photos are obtained to construct a coastline spatiotemporal image information sequence;

[0009] According to the color, outline and floating characteristics of typical garbage on the coastline, a fast garbage target recognition algorithm is established, and the coastline spatiotemporal image information sequence is operated one by one to identify and mark, and the latitude and longitude information of typical garbage targets in the coastline spatiotemporal image information sequence is obtained;

[0010] The coastline is divided into sections, and drones are used to conduct aerial inspections of each section to quickly obtain information on the distribution, size, and location of garbage in each section. The information is uploaded to the control system in the form of a summary report and sent to on-site cleaning staff to achieve efficient coastal cleaning.

[0011] Optionally, obtaining a sequence of spatiotemporal image information of a coastline includes:

[0012] Firstly, the spatially continuous image information in the time series is obtained based on the latitude and longitude information and time series information in the photo collection;

[0013] Secondly, feature matching is performed on two adjacent images by extracting features from neighboring regions and transforming the image homogeneous matrix, and color correction is used to adjust the image color difference;

[0014] Finally, adjacent images are fused according to their overlapping positions to obtain a coastline spatiotemporal image information sequence.

[0015] Optionally, the image information is standardized, including: converting the RGB photo into an HSV photo by converting RGB to HSV, and standardizing the image information by using a Gamma correction method to reduce interference from local shadows and light in the image.

[0016] Optionally, the standardized image information is matched to obtain the portion of the image that meets the HSV threshold of typical garbage, including: setting the HSV threshold of typical garbage, adjusting the threshold according to the ecological features and coastal characteristics of the monitored area and the color of typical garbage, and increasing the matching degree.

[0017] Optionally, a portion of the image that meets the HSV threshold of typical garbage is selected, and a Harr feature pixel module difference method is used to obtain a fast and simple typical garbage feature edge structure.

[0018] Optionally, contour reconstruction is performed based on typical garbage feature edge structures, including: obtaining typical garbage size categories and positioning longitude and latitude of the reconstructed contour based on pixel position and quantity information in the aerial image in combination with longitude and latitude information.

[0019] Optionally, based on the typical garbage size categories and location longitude and latitude of the reconstructed contour, the coastline image spatiotemporal sequence atlas is processed with a rapid garbage identification algorithm to obtain the size information and location longitude and latitude information of the garbage distribution on the coastline, and determine the typical garbage size and location information.

[0020] Optionally, the information processed after each inspection will be archived in the system. After the next drone inspection, garbage target comparison will be automatically performed to verify the cleaning effect and obtain sanitary blind spots.

[0021] Technical effect of the invention: The invention discloses a method for quickly locating and verifying typical garbage on the coastline based on drone aerial photography technology. Drone aerial photography is used to realize rapid coastline inspection, and an efficient image processing algorithm is used to quickly locate typical garbage targets in spatiotemporal image sequences, and periodically compare and verify them, which solves the defects of slow traditional coastline garbage inspection, complex garbage positioning algorithm and difficult verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 This is a flow chart of a method for quickly locating and checking typical garbage on a coastline based on drone aerial photography technology according to an embodiment of the present invention;

[0024] Figure 2 The latitude and longitude information diagram of the randomly selected photo DJI_0051 taken on December 2, 2021 of the present invention;

[0025] Figure 3 This is a set of route images of a certain section of the coastline on December 2, 2021 of the present invention, numbered from DJI_0001 to DJI_0138;

[0026] Figure 4 For the present invention, the original photo of the coordinates (117.78875905648776, 39.05096952094477) was taken on May 30, 2021, and the frame selection image for quick garbage identification was performed. DETAILED DESCRIPTION

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] like Figure 1-4 As shown, this embodiment provides a method for quickly locating and checking typical garbage on the coastline based on drone aerial photography technology, including:

[0030] According to weather and tidal forecasts, select suitable weather conditions to periodically conduct drone aerial photography along the coastline and collect a collection of drone aerial photos of the coastline;

[0031] Based on the photo collection, the longitude and latitude information and time sequence information of the photos are obtained to construct a coastline spatiotemporal image information sequence;

[0032] According to the color, outline and floating characteristics of typical garbage on the coastline, a fast garbage target recognition algorithm is established, and the coastline spatiotemporal image information sequence is operated one by one to identify and mark, and the latitude and longitude information of typical garbage targets in the coastline spatiotemporal image information sequence is obtained;

[0033] The coastline is divided into sections, and drones are used to conduct aerial inspections of each section to quickly obtain information on the distribution, size, and location of garbage in each section. The information is uploaded to the control system in the form of a summary report and sent to on-site cleaning staff to achieve efficient coastal cleaning.

[0034] Further optimization schemes to obtain coastline spatiotemporal image information sequences include:

[0035] Firstly, the spatially continuous image information in the time series is obtained based on the latitude and longitude information and time series information in the photo collection;

[0036] Secondly, feature matching is performed on two adjacent images by extracting features from neighboring regions and transforming the image homogeneous matrix, and color correction is used to adjust the image color difference;

[0037] Finally, adjacent images are fused according to their overlapping positions to obtain a coastline spatiotemporal image information sequence.

[0038] Requirements for drone selection: The drone must be equipped with a gimbal camera stabilization device and a high-resolution camera, built-in GPS, Beidou and other high-precision positioning devices and data information transmission modules. When shooting the coastline, the positioning longitude and latitude information and shooting time information of the photos can be saved in real time in the aerial image, and transmitted back to the onshore control console in real time. The console will construct the image set into a coastline space-time image information sequence.

[0039] UAV flight photography requirements: The flight altitude and speed of the drone are related to the shooting interval of the gimbal camera and the camera performance parameters. To ensure high-definition aerial images, the flight speed and altitude must satisfy the object-image relationship formula: , is the camera field of view, The flight speed of the drone , is the height of the camera lens from the ground, The width of the camera sensor , the width is consistent with the flight direction of the drone, is the camera focal length , t is the shooting interval s of the drone camera. Since the camera focal length is much smaller than the height of the camera lens from the ground, It can be approximated as the flight height of the drone. Considering that the image spatiotemporal sequence involves the splicing of adjacent images, edge target feature comparison is required, so a 20% image overlap rate is retained. From the above, it can be obtained that the flight speed should satisfy: ; At the same time, the flight altitude should meet the following requirements: .

[0040] The fast coastline garbage identification and positioning algorithm used in the present invention is based on 2D image calculation and does not perform 3D landform reconstruction. Therefore, when shooting, only a 20% overlap rate is required between two adjacent pictures to achieve feature matching image stitching; when shooting, the posture of the drone and the gimbal camera is controlled so that the camera shooting direction is always kept perpendicular to the coastline plane.

[0041] Spatiotemporal image sequence stitching: After aerial photography, image information with continuous spatial distribution in the time series is obtained, and the adjacent area features are extracted and matched between two adjacent images. The images are matched in the form of image homogeneous matrix transformation. During the matching process, color correction is required to adjust the color difference between the two adjacent images due to exposure and other reasons. Finally, the adjacent images are fused according to the overlapping positions. This process can be used to obtain a time and space continuous image sequence. After completing the construction of the spatiotemporal sequence of coastline images, the image information at any position of the coastline and its previous and subsequent adjacent image information can be quickly found, realizing electronic inspection and facilitating cleaning and verification.

[0042] The scheme is further optimized to standardize the image information, including: converting RGB photos into HSV photos by converting RGB to HSV, and standardizing the image information by using a Gamma correction method to reduce the interference of local shadows and light in the image.

[0043] To further optimize the scheme, the standardized image information is matched to obtain the portion of the image that meets the HSV threshold of typical garbage, including: setting the HSV threshold of typical garbage, adjusting the threshold according to the ecological features and coastal characteristics of the monitored area and the color of typical garbage, and increasing the matching degree.

[0044] To further optimize the solution, the portion of the image that meets the HSV threshold of typical garbage is selected, and the Harr feature pixel module difference method is used to obtain a fast and simple typical garbage feature edge structure.

[0045] The scheme is further optimized to reconstruct the contour based on the characteristic edge structure of typical garbage, including: obtaining the typical garbage size category and positioning longitude and latitude of the reconstructed contour according to the pixel position and quantity information in the aerial image and the longitude and latitude information.

[0046] To further optimize the scheme, based on the typical garbage size categories and location longitude and latitude of the reconstructed contours, the coastline image spatiotemporal series atlas was processed with a garbage rapid identification algorithm to obtain the size information and location longitude and latitude information of the garbage distribution on the coastline, and to determine the typical garbage size and location information.

[0047] Further optimization plans also include archiving the information after each inspection in the system. After the next drone inspection, garbage target comparison will be automatically performed to verify the cleaning effect and identify sanitary blind spots.

[0048] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for quickly locating and checking typical garbage on the coastline based on drone aerial photography technology, characterized by: include: According to weather and tidal forecasts, select suitable weather conditions to periodically carry out drone aerial photography along the coastline, and collect a collection of drone aerial photos of the coastline; the image overlap rate of adjacent images is retained at 20%; the flight speed of the drone should meet the following requirements: ;in, is the flight speed of the drone, is the camera focal length, is the height of the camera lens from the ground, is the width of the camera’s photosensitive element, and t is the shooting interval of the drone’s camera; Based on the photo collection, the latitude and longitude information and time sequence information of the photos are obtained to construct a coastline spatiotemporal image information sequence; specifically, it includes: First, based on the latitude and longitude information and time series information in the photo collection, spatially continuously distributed image information in the time series is obtained, and the image information is standardized. The standardized image information is matched to obtain the part of the image that meets the typical garbage HSV threshold, including: Setting a typical garbage HSV threshold, adjusting the threshold according to the ecological features and coastal characteristics of the monitored area and the typical garbage color, and increasing the matching degree; selecting the portion of the image that meets the typical garbage HSV threshold, and using the Harr feature pixel module difference method to obtain a fast and simple typical garbage feature edge structure; wherein, standardizing the image information includes: using RGB to HSV conversion method to convert RGB photos into HSV photos, and using the Gamma correction method to standardize the image information to reduce the interference of local shadows and light in the image; Secondly, feature matching is performed on two adjacent images by extracting features from neighboring regions and transforming the image homogeneous matrix, and color correction is used to adjust the image color difference; Finally, the adjacent images are fused according to their overlapping positions to obtain a spatiotemporal image information sequence of the coastline; According to the color, outline and floating characteristics of typical garbage on the coastline, a fast garbage target recognition algorithm is established, and the coastline spatiotemporal image information sequence is operated one by one to identify and mark, and the latitude and longitude information of typical garbage targets in the coastline spatiotemporal image information sequence is obtained; The coastline is divided into sections, and drones are used to conduct aerial inspections of each section to quickly obtain information on the distribution, size, and location of garbage in each section. The information is uploaded to the control system in the form of a summary report and sent to on-site cleaning staff to achieve efficient coastal cleaning.

2. The method for quickly locating and checking typical coastline garbage based on drone aerial photography technology as claimed in claim 1 is characterized in that: The outline is reconstructed based on the characteristic edge structure of typical garbage, including: obtaining the typical garbage size category and positioning longitude and latitude of the reconstructed outline according to the pixel position and quantity information in the aerial image and the longitude and latitude information.

3. The method for rapid positioning and verification of typical coastline garbage based on drone aerial photography technology as claimed in claim 2 is characterized in that: Based on the typical garbage size categories and positioning longitude and latitude of the reconstructed contour, the coastline image spatiotemporal sequence atlas is processed with a garbage rapid identification algorithm to obtain the size information and positioning longitude and latitude information of the garbage distribution on the coastline, and to determine the typical garbage size and positioning information.

4. The method for rapid positioning and verification of typical coastline garbage based on drone aerial photography technology as claimed in claim 3 is characterized in that: It also includes that the information after each inspection will be archived in the system. After the next drone inspection, garbage target comparison will be automatically performed to verify the cleaning effect and obtain sanitary blind spots.

Images