Cyanobacterial bloom analysis method based on 720-degree panoramic photograph

Abstract

The invention relates to a cyanobacterial bloom analysis method based on a 720-degree panoramic photo, and belongs to the technical field of image processing. According to the method, shooting is carried out right above a cyanobacterial bloom distribution center point to obtain a 720-degree panoramic photo, the 720-degree panoramic photo is unfolded in a plane coordinate mode to carry out direction calibration, then a coordinate orientation grid is established, cyanobacterial blooms are identified, position statistics is carried out, and finally the cyanobacterial bloom area is calculated. The 720-degree panoramic picture is adopted for analysis, and the height angle is the lowest, so that the area calculation error is small, and the precision is high. And the efficiency of single investigation and analysis can be greatly improved.

Description

Technical Field

[0001] This invention relates to a method for analyzing cyanobacterial blooms based on 720-degree panoramic photographs, belonging to the field of image processing technology. Background Technology

[0002] In lake protection and management, analyzing the location and area of ​​cyanobacterial blooms has always been a key challenge. In recent years, the application of drone technology in surveys has greatly improved efficiency. However, accurate identification of the distribution range and area of ​​cyanobacterial blooms using drones mainly relies on drone photographs taken from a single angle combined with manual estimation. On the one hand, drone photographs are generally taken from a fixed direction or angle, limiting their effective coverage, and the analysis error increases with the distance between the target water area and the drone. On the other hand, the accuracy of manual surveys is often limited by the expertise and experience of the surveyors, and their efficiency is low. Furthermore, cyanobacterial blooms can change rapidly in a short period, making it difficult for the survey results to meet some time-sensitive requirements. Summary of the Invention

[0003] This invention addresses the problems of limited coverage, low accuracy, and low efficiency caused by the single shooting angle in existing drone-based cyanobacterial bloom surveys. It proposes a cyanobacterial bloom analysis method based on 720-degree panoramic photos to quickly and accurately obtain the distribution location and coverage area of ​​cyanobacterial blooms in the surveyed waters.

[0004] The present invention specifically adopts the following technical solution:

[0005] A method for analyzing cyanobacterial blooms based on 720-degree panoramic photographs includes the following steps:

[0006] S1, Input a 720-degree panoramic photo

[0007] The drone was used to take a 720-degree panoramic photo directly above the water surface, and the GPS coordinates and lake elevation information at the time of the drone's capture were used as input.

[0008] S2, Calibration Direction

[0009] The vertical center line AE of the 720-degree panoramic photo input for calibration is due north, the vertical edge lines A1E1 / A4E4 are due south, the vertical center line A2E2 between the vertical edge line A1E1 and the vertical center line AE is due west, the vertical center line A3E3 between the vertical center line AE and the vertical edge line A4E4 is due east, P1P4 is the horizontal line when the panoramic photo was taken, and its elevation is P_Ele. With P1P4 as the reference, the elevation angle is in the direction of A1A4, and the depression angle is in the direction of E1E4.

[0010] S3. Establish a coordinate orientation grid.

[0011] A coordinate grid is established at the input image resolution, the width of the grid is the width of the image, the height of the grid is the height of the image, each pixel point is a grid, and the position of each pixel point is marked according to the direction marking in S2;

[0012] S4, cyanobacterial bloom recognition

[0013] S4.1, identify the water surface and generate a water surface mask

[0014] The pixels below the P1P4 line are analyzed and classified as water, and a water surface mask is generated;

[0015] Take the water body where E1E4 is located as a seed point, and record its RGB values as R0, G0, and B0 respectively, search using the region growing method, complete the water surface recognition, and classify the pixel points as water surface under the following conditions:

[0016] Water: |R-R0|≤50 and |G-G0|≤50 and |B-B0|≤50

[0017] According to the above conditions, the pixels below the P1P4 line are analyzed and classified as water, and a water surface mask is generated.

[0018] S4.2, identify cyanobacterial bloom in the water surface range

[0019] Extract the cyanobacterial bloom in the water surface range, and generate a cyanobacterial bloom mask after analyzing and classifying the pixels in the water surface mask range as cyanobacterial bloom;

[0020] Extract the cyanobacterial bloom in the water surface range using the region growing method, select the point where the cyanobacterial bloom is located as the region growing seed point, record its RGB values as R1, G1, and B1 respectively, and complete the search of the cyanobacterial bloom in the water surface mask range using the 8-neighborhood method, and classify the pixel points as cyanobacterial bloom under the following conditions:

[0021] Algal_bloom: |R-R1|≤10 and |G-G1|≤10 and |B-B1|≤10

[0022] According to the above conditions, the pixels in the water surface mask range are analyzed and classified as cyanobacterial bloom, and a cyanobacterial bloom mask is generated.

[0023] S5 cyanobacterial bloom position calculation

[0024] Overlay the cyanobacterial bloom mask generated in S4 and the coordinate grid established in S3, and calculate the height angle and horizontal angle of the cyanobacterial bloom pixel points to determine the distribution of the cyanobacterial bloom at the unmanned aerial vehicle shooting point;

[0025] The degrees of the upper and lower boundary lines of the cyanobacterial bloom are calculated, wherein the degrees of the upper and lower boundary lines are denoted as height angle H H and H L , the value range is 0-90 degrees, and the degrees of the left and right boundary lines are denoted as horizontal angle L L and L R , the value range is -180-+180 degrees.

[0026] The average value H of the degrees of the cyanobacterial bloom in the vertical direction is calculated.

[0027] The average value L of the degrees of the cyanobacterial bloom in the horizontal direction is calculated.

[0028] (4) The position of the cyanobacterial bloom is represented as follows: taking the GPS coordinates when the unmanned aerial vehicle is shooting as a reference point, the center height angle is H, the height angle range is H L -H H , the center azimuth angle is L, and the azimuth angle range is L L -L R .

[0029] S6 cyanobacterial bloom area calculation

[0030]

[0031] wherein n is the total number of cyanobacterial bloom pixels, GSD i is the actual side length corresponding to the i th cyanobacterial bloom pixel.

[0032]

[0033] αi represents the height angle of the i th cyanobacterial bloom pixel in the coordinate azimuth grid, Lαi represents the distance of the plane from the target water area when the height angle is αi, s is the pixel size, f is the focal length, L_Ele is the lake water surface elevation, and P_Ele is the elevation when the unmanned aerial vehicle is shooting.

[0034] Beneficial effects:

[0035] 1. In the method, the cyanobacterial bloom is located directly below the shooting angle, the height angle is small, and the area calculation error is small. The 720-degree panoramic photo can cover the visual range of 180 degrees upward and downward and 360 degrees left and right of the unmanned aerial vehicle shooting point. Therefore, under the same maximum height angle limit, the coverage range is much larger than that of the conventional unmanned aerial vehicle single-angle photo, which can greatly improve the efficiency of single investigation and analysis.

[0036] 2、Since the photos of left and right 360-degree view range in the 720-degree panorama photo are continuously spliced, the problem that when a single-view photo of a conventional unmanned aerial vehicle cannot cover the target area of cyanobacterial bloom, but when multiple single-view photos of the unmanned aerial vehicle are used, the overlapping area of adjacent photos is difficult to accurately calculate the area of the overlapping area can be effectively solved.

[0037] 3、The method can be implemented by using a programming language such as Python, and there are a large amount of parallel computing contents in the calculation process, and GPU can be used to realize fast parallel computing, so that the input image can be processed quickly. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a 720-degree panorama photo unfolded in a plane coordinate;

[0039] Figure 2 is a 720-degree panorama photo direction calibration diagram;

[0040] Figure 3 is a water surface mask diagram (white for water surface and black for non-water surface);

[0041] Figure 4 is a cyanobacterial bloom mask diagram;

[0042] Figure 5 is a diagram of aligning the cyanobacterial bloom mask with the coordinate grid;

[0043] Figure 6 is a diagram of the spatial structure relationship when the unmanned aerial vehicle performs panoramic imaging;

[0044] Figure 7 is a diagram of the difference between the conventional unmanned aerial vehicle single-view shooting and the 720-degree panoramic shooting. DETAILED DESCRIPTION

[0045] The present application will be further described below in combination with the drawings and specific embodiments.

[0046] EMBODIMENT

[0047] S1 input 720-degree panorama photo

[0048] The 720-degree panorama photo with GPS coordinate information taken by the conventional unmanned aerial vehicle directly above the water surface, the GPS coordinates [longitude (P_Lon: Panoramic Longitude), latitude (P-Lat: Panoramic Latitude) and elevation (P_Ele: Panoramic Elevation)] when the unmanned aerial vehicle is shooting, and the lake water surface elevation (L_Ele: Lake Elevation) are taken as input information.

[0049] The GPS coordinates used by the drone during the 720-degree panoramic photo capture are stored in the photo's detailed information. The lake's elevation can be found through local management agencies or publicly available online resources.

[0050] S2 Calibration Direction

[0051] Unfold the 720-degree panoramic photo using planar coordinates. The unfolded 720-degree panoramic photo is as follows: Figure 1 As shown, its key points are as follows: Figure 2 As shown. In this panoramic view, A, A1, A2, A3, and A4 overlap at the center of the sky, while E, E1, E2, E3, and E4 overlap at the center directly below. P1 and P4 overlap.

[0052] The vertical center line AE of the 720-degree panoramic photo is calibrated to be due north. A1E1 (overlapping with A4E4) is due south. The vertical center line A2E2 between the vertical edge line A1E1 and the vertical center line AE is due west. The vertical center line A3E3 between the vertical center line AE and the vertical edge line A4E4 is due east. P1P4 is the horizon line when the panoramic photo was taken, and the elevation of this horizon line is P_Ele. Using this horizon line as a reference, the elevation angle towards A1A4 is the elevation angle, and the depression angle towards E1E4 is the depression angle.

[0053] S3 establishes a coordinate orientation grid.

[0054] A coordinate orientation grid is created based on the input image resolution. The width of the grid is the same as the image width, denoted as Width, and the height of the grid is the image height. Each pixel is used as a grid point, and the following steps are performed... Figure 2 The orientation calibration diagram shown calibrates the position of each pixel.

[0055] For example, the input image size is 21600 pixels × 10800 pixels. The width of this coordinate grid is 21600, and the height is 10800. Line E1E4 represents 0 degrees, line P1P4 represents 90 degrees, and line A1A4 represents 180 degrees. Distributing these 180 degrees evenly across the 10800 pixels of the image height, each pixel represents 180 / 10800 = 0.01667 degrees. Line AE is located due north and is denoted as 0 degrees. Centered on AE, the leftward direction (A1E1) is negative, and the rightward direction (A4E4) is positive; A1E1 is -180 degrees, and A4E4 is +180 degrees. Similarly, distributing this evenly across the 21600 pixels horizontally, each pixel in the width direction represents 360 / 21600 = 0.1667 degrees.

[0056] S4 Blue-green algal bloom identification

[0057] S4.1 Identify the water surface and generate a water surface mask

[0058] Since the UAV is taking pictures directly above the water body, the location of E1E4 is the water body at this time, and this location is taken as the seed point, and its RGB values are recorded as R0, G0, and B0, respectively. The tolerances of the three image bands are set as follows: red band (R) 50, green band (G) 30, and blue band (B) 50. (Since there may be plants along the shore of the water area, the tolerance of the green band is set to be small to improve accuracy. In actual work, the best tolerance can be determined by multiple attempts.) Search in 8-neighborhood mode to complete water surface identification. The conditions for classifying pixel points as water surface are:

[0059] Water: |R-R0|≤50 and |G-G0|≤50 and |B-B0|≤50

[0060] If the above conditions are not met, the pixel points are not classified as water surface.

[0061] After traversing and analyzing the pixels below the P1P4 line and classifying them as water surface according to the above conditions, a water surface mask can be generated. The water surface mask is shown in Figure 3 .

[0062] S4.3 Further identify blue-green algae blooms within the water surface range

[0063] Take the white mask as the identification boundary of blue-green algae blooms, and extract the blue-green algae blooms within the water surface range. The extraction method can use region growing or YOLOv5, DeepLab V3++, or other semantic segmentation algorithms. Taking region growing as an example, set the threshold of the three image bands to 10, manually select the blue-green algae bloom point, take this point as the region growing seed point, record its RGB values as R1, G1, and B1, respectively, and use the 8-neighborhood mode to complete the search of blue-green algae blooms within the water surface mask range. The conditions for classifying pixel points as blue-green algae blooms are:

[0064] Algal_bloom: |R-R1|≤10 and |G-G1|≤10 and |B-B1|≤10

[0065] After traversing and analyzing the pixels within the water surface mask range and classifying them as blue-green algae blooms according to the above conditions, a blue-green algae bloom mask can be generated. The blue-green algae bloom mask is shown in Figure 4 .

[0066] S5 Blue-green algae bloom position calculation

[0067] Align and superimpose Figure 4 with the coordinate grid established in S3 (as shown in Figure 5 ), and count Figure 4The elevation and horizontal angles of the pixels in the cyanobacterial bloom (white area in the diagram) can determine the location of the cyanobacterial bloom at the drone's shooting point.

[0068] The calculation steps are as follows:

[0069] (1) Calculate the degrees of the four boundary lines of the cyanobacterial bloom, including the top, bottom, left, and right. The degrees at the top and bottom boundary lines are denoted as the elevation angle H. H and H L The value ranges from 0 degrees to 90 degrees. The degree measures of the left and right boundaries are denoted as horizontal angle L according to their relationship with the coordinate grid. L and L R The value range is -180 degrees to +180 degrees.

[0070] (2) Calculate the average degree H of cyanobacterial blooms (i.e. all white pixels) in the vertical direction.

[0071] (3) Calculate the average degree L of cyanobacterial bloom (i.e. all white pixels) in the horizontal direction.

[0072] (4) The location of the cyanobacterial bloom can be represented as follows: with the coordinates [longitude (P_Lon: Panoramic Longitude), latitude (P-Lat: Panoramic Latitude), and elevation (P_Ele: Panoramic Elevation)] as the reference point, the central elevation angle is H, and the elevation angle range is: H L ~H H The center azimuth is L, and the azimuth range is: L L ~L R area.

[0073] S6 Blue-green algal bloom area calculation

[0074] like Figure 6 As shown. Point P is the intersection with... Figure 2 The P1P4 positions shown represent the same horizontal position as the drone when taking the 720-degree panoramic photo, i.e., its longitude (P_Lon), latitude (P_Lat), and elevation (P_Ele). Point E is directly below the drone's shooting point P, therefore its elevation angle is 0 degrees. Centered on point E, as the elevation angle gradually increases, the elevation angle corresponding to the 720-degree panoramic photo also gradually increases. S1 and S2 are located on the same horizontal line after the 720-degree panoramic photo is unfolded, and... Figure 5 S1 and S2 are shown as corresponding.

[0075] The elevation at point E is the lake surface elevation (L_Ele), and the length of PE is: P_Ele-L_Ele. That is, the height of the UAV from the lake surface. And ∠EPS1=∠EPS2=S1S2 is the height angle α in the coordinate azimuth grid. Then the real ground size corresponding to each pixel edge length (each pixel is regarded as a square) at the height angle α can be calculated according to the following formula:

[0076]

[0077] wherein the pixel size s and the focal length f are generally fixed parameters of the UAV lens and can be directly obtained through the device parameters. α is the distance of the plane from the target water area when the height angle is α, that is, the length of PS1. α The calculation formula is:

[0078]

[0079] According to the height angle of each pixel point identified as a blue-green algae bloom in the 720-degree panoramic image in the coordinate azimuth grid and the GPS coordinates [longitude (P_Lon), latitude (P-Lat) and elevation (P_Ele)] input in Step.1 when the UAV is shooting, the lake surface elevation (L_Ele), the real ground edge length corresponding to each blue-green algae bloom pixel can be calculated. These points are counted using the following formula, and the coverage area of the blue-green algae bloom can be obtained.

[0080]

[0081] wherein n is the total number of blue-green algae bloom pixels, GSD i is the real ground edge length corresponding to the i-th blue-green algae bloom pixel.

[0082] Since the UAV is shooting above the water surface when shooting the 720-degree panoramic photo, in theory, the height angle of the water surface coverage area is less than 90 degrees. When using the 720-degree panoramic photo to identify blue-green algae bloom, the best position of the UAV shooting point is directly above the center point of the blue-green algae bloom distribution. When the 720-degree panoramic photo is taken at this position and the area of the blue-green algae bloom is counted, since the height angle is the lowest, the area calculation error is small and the accuracy is high.

[0083] However, the conventional UAV blue-green algae bloom survey can only be shot in one direction when the UAV is shooting. Therefore, in order to reduce the error when shooting, the position shown in Figure 7 (a) is often used for shooting, and ∠F2PE is the maximum height angle, at which time the area calculation error is the largest. When the UAV 720-degree panoramic photo is used to survey the same piece of blue-green algae bloom at the same height, the position shown in Figure 7(b) the position shown, the maximum height angle at this time is ∠G2PE. It can be seen that in Figure 7 In (a), the actual area of the cyanobacterial bloom has a larger height angle in the UAV image, and the area calculation error is also larger. Conversely, Figure 7 In (b), since the cyanobacterial bloom is located directly below, the height angle is smaller, and the area calculation error is also smaller. This can improve the area calculation accuracy of cyanobacterial bloom investigation.

[0084] The specific embodiments of the application are described in detail above with reference to the accompanying drawings, but the application is not limited to the above-described embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the application.

Claims

1. A method for analyzing cyanobacterial blooms based on 720-degree panoramic photographs, characterized in that, Includes the following steps: S1, Input a 720-degree panoramic photo The drone was used to take a 720-degree panoramic photo directly above the water surface, and the GPS coordinates and lake elevation information at the time of the drone's capture were used as input. S2, Calibration Direction The vertical center line AE of the 720-degree panoramic photo input for calibration is due north, the vertical edge lines A1E1 / A4E4 are due south, the vertical center line A2E2 between the vertical edge line A1E1 and the vertical center line AE is due west, the vertical center line A3E3 between the vertical center line AE and the vertical edge line A4E4 is due east, P1P4 is the horizontal line when the panoramic photo was taken, and its elevation is P_Ele. With P1P4 as the reference, the elevation angle is in the direction of A1A4, and the depression angle is in the direction of E1E4. S3. Establish a coordinate orientation grid. A coordinate orientation grid is established based on the input image resolution. The width of the grid is the width of the image, and the height of the grid is the height of the image. Each pixel is used as a grid, and the position of each pixel is calibrated according to the orientation calibration in S2. S4, Blue-green algae bloom identification S4.1 Identify the water surface and generate a water surface mask. Perform traversal analysis and water surface classification on the pixels below P1P4 lines to generate a water surface mask; S4.2 Identifying cyanobacterial blooms within the water surface area The cyanobacterial blooms within the water surface area are extracted, and after traversing and analyzing the pixels within the water surface mask area and classifying the cyanobacterial blooms, a cyanobacterial bloom mask is generated. S5 Cyanobacterial Bloom Location Calculation The cyanobacterial bloom mask generated in S4 and the coordinate grid established in S3 are superimposed on each other. The elevation angle and horizontal angle of the cyanobacterial bloom pixels are counted to determine the location of the cyanobacterial bloom at the drone shooting point. S6 Blue-green algal bloom area calculation Where n is the total number of pixels in the cyanobacterial bloom, GSD i Let be the actual side length corresponding to the i-th pixel of the cyanobacterial bloom. αi represents the elevation angle of the i-th cyanobacterial bloom pixel in the coordinate grid, Lαi represents the distance between the aircraft and the target water area when the elevation angle is αi, s is the pixel size, f is the focal length, L_Ele is the lake surface elevation, and P_Ele is the elevation when the drone takes the picture.

2. The method for analyzing cyanobacterial blooms based on 720-degree panoramic photographs as described in claim 1, characterized in that, The water surface mask recognition process in S4 is as follows: Using the water body where E1E4 is located as the seed point, its RGB values ​​are denoted as R0, G0, and B0 respectively. A region growing algorithm is used to search for and identify the water surface. The criteria for classifying pixels as water surfaces are: Water: |R-R0|≤50 and |G-G0|≤50 and |B-B0|≤50 Based on the above conditions, after traversing and analyzing the pixels below the P1P4 line and classifying the water surface, a water surface mask is generated.

3. The method for analyzing cyanobacterial blooms based on 720-degree panoramic photographs as described in claim 1, characterized in that, The process for identifying cyanobacterial blooms in S4 is as follows: The region growing method was used to extract cyanobacterial blooms within the water surface area. The location of the cyanobacterial bloom was selected as the seed point for region growing, and its RGB values ​​were denoted as R1, G1, and B1, respectively. An 8-neighborhood approach was used to complete the search for cyanobacterial blooms within the water surface mask area. The criteria for classifying pixels as cyanobacterial blooms were: Algal_bloom: |R-R1|≤10 and |G-G1|≤10 and |B-B1|≤10 Based on the above conditions, after performing traversal analysis and classifying cyanobacterial blooms within the water surface mask area, a cyanobacterial bloom mask is generated.

4. The method for analyzing cyanobacterial blooms based on 720-degree panoramic photographs as described in claim 1, characterized in that, The calculation steps for the distribution of cyanobacterial blooms in S5 are as follows: (1) Calculate the degrees of the four boundary lines of the cyanobacterial bloom, including the elevation angle H at the top and bottom boundary lines. H and H L The value ranges from 0 degrees to 90 degrees, and the degree of the left and right boundaries is denoted as the horizontal angle L according to its relationship with the coordinate grid. L and L R The value range is -180 degrees to +180 degrees. (2) Calculate the average vertical degree H of cyanobacterial blooms; (3) Calculate the average degree L of cyanobacterial bloom in the horizontal direction; (4) The location of the cyanobacterial bloom is represented as follows: with the GPS coordinates taken by the drone as the reference point, the center elevation angle is H, and the elevation angle range is: H L ~H H The center azimuth is L, and the azimuth range is: L L ~L R area.

Images