An automatic monitoring method for cyanobacteria blooms combining AI and drones

Abstract

The present invention relates to an automatic monitoring method for cyanobacteria blooms combining AI and drone fields, and belongs to the field of image processing technology. The present invention first improves the existing model to more accurately identify cyanobacteria blooms, and then uses the information in the photo itself to directly calculate the area of ​​cyanobacteria blooms on a single drone photo, and finally comprehensively calculates the total area of ​​cyanobacteria blooms. The present invention combines the advantages of automatic photo collection of drone fields and the automatic recognition function of deep learning, and accurately calculates the actual size corresponding to each pixel in the photo, thereby realizing automatic monitoring of cyanobacteria blooms. Compared with the existing image recognition method, the present invention realizes a one-to-one correspondence between photos taken by drones and the actual size, so that the actual area of ​​cyanobacteria blooms can be quickly output using photos automatically acquired by drone fields.

Description

Technical Field

[0001] The invention relates to an automatic monitoring method for blue algae blooms combining AI and an unmanned aerial vehicle (UAV) field, and belongs to the technical field of image processing. Background Art

[0002] In the work of lake water environment protection and governance, the location and area changes of cyanobacteria blooms have always been the focus and difficulty of environmental monitoring. At present, the identification of the distribution range and area of ​​cyanobacteria blooms mainly relies on satellite remote sensing images combined with manual surveys. On the one hand, the acquisition of satellite images is easily affected by weather. When there is cloud cover above the target area, it is often difficult to obtain effective image data in time; on the other hand, the accuracy of manual surveys is often limited by the professionalism and experience of investigators, and the efficiency is low. The cyanobacteria blooms change rapidly, so that the survey results are difficult to meet some time-sensitive requirements. With the development of drone technology, drone fields with automatic flight functions have unparalleled advantages in rapid image acquisition in such application scenarios, and can obtain multispectral images of the target area by carrying multispectral sensors, thereby improving the accuracy of the survey. However, at present, the statistics of the area of ​​cyanobacteria blooms in automatic monitoring of cyanobacteria blooms based on drone fields are still not accurate enough.

[0003] There are two main methods for counting the area of ​​cyanobacteria blooms in existing drone photos: one is to use drone photos as a reference, mark the approximate range of cyanobacteria blooms on satellite images, and then count the area of ​​the marked range. This method mainly relies on human subjective judgment of the distribution range of cyanobacteria blooms, which is highly subjective and not very accurate. The second is to use multiple adjacent drone photos, combined with aerial photogrammetry technology (a technology in the field of surveying and mapping) to stitch drone photos into orthophotos, and then mark the range based on the stitched orthophotos, and then count the area of ​​cyanobacteria blooms. This method has a certain interval between the shooting time of adjacent drone photos, and the cyanobacteria bloom may have changed during this period, resulting in the misalignment of the stitching of the two photos, causing statistical errors. In addition, the existing image stitching technology is generally based on image feature points for stitching, and most of the water surface feature points are not obvious, so it often leads to the inability to stitch adjacent drone photos, generate orthophotos, and perform statistics. Summary of the invention

[0004] The present invention effectively connects the UAV airport with AI technology to realize automatic monitoring of cyanobacteria blooms and statistics of cyanobacteria bloom area. This achievement can be further applied to lake water environment protection and governance projects such as automatic monitoring of cyanobacteria blooms, calculation of the distribution range and area of ​​aquatic plants.

[0005] The present invention specifically adopts the following technical solutions:

[0006] An automatic monitoring method for cyanobacteria blooms combining AI and drones, comprising the following steps:

[0007] S1. Automatic cruise of drone airport

[0008] Use a drone with an automatic cruise function to automatically acquire photos of the target area. The photo information acquired by the drone includes the GPS coordinates of the photo shooting point, the flight altitude relative to the ground, the tilt angle of the drone when taking the photo, and other drone photo shooting status information.

[0009] S2. Automatic upload of video data

[0010] The photos taken by the drone are transmitted to the server for processing. The photos taken by the drone are automatically transmitted back to the server through the wireless network module. After the drone completes the cruise mission and lands at the airport, the image data can be automatically uploaded using the airport's network environment. The server can use its own server or cloud server, but the server should be equipped with computing resources that can perform reasoning normally.

[0011] S3, AI automatic recognition

[0012] S3.1. Identification of cyanobacterial blooms in drone photos

[0013] The server receives the photos automatically transmitted by the drone and uses the trained AI model to automatically identify the cyanobacteria blooms in each photo;

[0014] S3.1.1. Semantic segmentation model construction and training

[0015] Since the existing semantic segmentation model mainly processes conventional visible light, it is necessary to improve the existing semantic segmentation model to make it compatible with multispectral image data.

[0016] The specific improvement method is to judge the input image. If it is a 3-band color image, the model training is directly performed. If it is a 5-band multispectral image, the input image is integrated using the band integrator C. The function of the band integrator C is to integrate the input 5-band multispectral data into a 3-band image according to the weight parameter, and then use the integrated three-band image as the input of the semantic segmentation model. The calculation of each band in the band integrator C is as follows:

[0017] C 1 =0.3*Red+0.6*Green+0.1*Blue

[0018] C 2 =0.3*NIR+0.6*Green+0.1*Blue

[0019] C 3=0.3*RE+0.6*Green+0.1*Blue

[0020] Among them, Red, Green, Blue, NIR, and RE represent the red, green, blue, near infrared, and red edge bands respectively;

[0021] The improved model consists of a band integrator C and a basic model, among which the basic model can be selected from mainstream semantic segmentation models such as Unet and DeepLabV3+.

[0022] Create a cyanobacteria bloom sample label dataset D, use the cyanobacteria bloom sample dataset D to train a semantic segmentation model, and take the model training result with the highest accuracy in the model verification set as the final inference model IM (Inference model) for automatic identification of cyanobacteria blooms;

[0023] S3.1.2. Use the trained model to identify cyanobacteria blooms and obtain a binary image of the identification result

[0024] The server scans the returned drone airport image data and compares it with the database file by file. If the photo has been recorded in the database, the blue algae bloom recognition of the photo will be skipped and the next photo will be read. If the photo has not been recorded in the database, the inference model will be used to identify the blue algae bloom and the photo record will be added to the database.

[0025] S3.2. Area statistics of cyanobacteria blooms in drone photos

[0026] The area is calculated using the information in the photo itself. The specific calculation is as follows:

[0027] S3.2.1. Read the drone photo shooting status information and lens parameters

[0028] Read the relative flight altitude H recorded in the photo information when each photo is taken, the tilt angle θ of the lens at the moment the photo is taken, and determine the field of view angle α;

[0029] The field angle α is determined as follows:

[0030] A reference road was selected near the drone airport, and no drones were detected at flight altitudes of 30m, 60m, and 90m.

[0031] The human-machine lens is pointed vertically downward to take photos. The long side or short side of the photo is parallel to the road. The reference road is in the middle of the photo. Point A is the position of the drone. Line segment AO 1 The length of line segment AO is 90 meters. 2 The length of line segment AO is 60 meters. 3 The length is 30 meters, corresponding to the flight altitude of the drone at 90 meters, 60 meters and 30 meters respectively. Line segment B1 D 1 , B 2 D 2 and B 3 D 3 They represent the length of the straight road photographed by the drone at a flight altitude of 90 meters, 60 meters, and 30 meters respectively; the field of view angle α is calculated as follows:

[0032] .

[0033] S3.2.2. Calculate the ground resolution of each pixel in the photo. The specific calculation formula is:

[0034]

[0035] Among them, L is the distance between the calculated target point and the drone lens, s is the camera pixel size, f is the camera focal length, and s and f are fixed parameters of the drone lens;

[0036] S3.2.3. Calculate the pixel resolution corresponding to each pixel row in the photo

[0037] According to the relationship between the tilt angle θ of the lens at the moment of taking the photo and the field angle α of the photo that is consistent with the tilt direction, the change of ground resolution in the photo is divided into two cases, and it is assumed that the ground resolution corresponding to the same row of pixels is the same;

[0038] (1) When α>2θ, the photo is regarded as two isosceles trapezoids, one in front and one in back. The two isosceles trapezoids share the same short base EF. The long base of the front isosceles trapezoid is IJ and the height is OD, while the long base of the rear isosceles trapezoid is KL and the height is OB.

[0039] As the pixel row changes, the ground resolution changes along the long side according to the ratio k. The ratio k represents the increase in ground resolution corresponding to one pixel for each additional pixel row, which is calculated according to the following formula:

[0040]

[0041]

[0042] Among them, R 前 and R 后 They represent the number of pixel rows corresponding to the height of the front isosceles trapezoid and the rear isosceles trapezoid, that is, the number of pixels corresponding to the actual distances OD and OB in the photo, R 前 +R 后 = photo pixel length or photo pixel width;

[0043] The pixel row where the KL edge is located is the starting row, the pixel row where the IJ edge is located is the ending row, and the pixel row where the EF edge is located is the Mth row, SRPM is the pixel spatial resolution of the pixel row where the EF edge is located. At this time, the pixel resolution size of each row is SRP_R （i） Calculated according to the following formula:

[0044]

[0045] Among them, SRP_R (i) It indicates the real resolution of each pixel in the ith pixel row starting from the pixel row where the KL edge is located, SRP indicates the pixel spatial resolution of the pixel row where the EF edge is located, and Img_Width is the photo width;

[0046] (2) When α≤2θ, after the photos taken by the drone are mapped to the actual distance, an inverted isosceles trapezoid range is obtained. The long base of the inverted isosceles trapezoid is MN, the short base is GH, and the height of the inverted isosceles trapezoid is BD. At this time:

[0047]

[0048] Among them, R represents the number of pixel rows corresponding to the height of the isosceles trapezoid, that is, the number of pixels corresponding to the actual distance BD in the photo. The resolution of each pixel row is calculated according to the following formula:

[0049]

[0050] Among them, SRP_R (i) It represents the real resolution corresponding to each pixel in the i-th pixel row starting from the pixel row where the GH edge is located, and SRP represents the pixel spatial resolution of the pixel row where the GH edge is located;

[0051] S3.2.4. Use the binary image of the drone identification result to count the number of pixels with different resolutions and multiply them by the ground resolution corresponding to the pixel to obtain the actual area of ​​cyanobacteria bloom, which is calculated according to the following formula:

[0052]

[0053] Among them, Area_Sum represents the total area of ​​cyanobacteria bloom in the target photo. Indicates the resolution of each pixel in the i-th row of pixels. represents the number of pixels identified as cyanobacterial blooms in the i-th row;

[0054] S4. Output results report

[0055] The results report is output by integrating the original image taken by the drone, the automatic identification result of cyanobacteria bloom, the spatiotemporal information annotation map of the drone photos and the area statistics results.

[0056] The spatiotemporal information annotation map of drone photos is obtained through the following steps:

[0057] S4.1. Read the drone photo shooting status information

[0058] Read the shooting information recorded in the photos taken by the drone, including GPS location information and shooting time information;

[0059] S4.2. Generate point annotation map

[0060] Based on the GPS coordinate information read, the shooting point is superimposed on the satellite image base map according to its actual longitude and latitude, and a fixed-scale annotation map is generated with the annotation point as the center point;

[0061] S4.3. Overlaying time information on the annotation graph

[0062] Overlay the shooting time of drone photos on the annotation map as additional information.

[0063] Beneficial effects:

[0064] 1. The present invention combines the advantages of automatic photo collection of drone airports with the automatic recognition function of deep learning, and accurately calculates the actual size corresponding to each pixel in the photo, realizing automatic monitoring of cyanobacteria blooms. Compared with the existing image recognition method, the present invention realizes a one-to-one correspondence between the photos taken by drones and the actual size, so that the actual area of ​​cyanobacteria blooms can be quickly output using the photos automatically obtained by drone airports.

[0065] 2. The model used in this method can be flexibly adjusted, and the latest semantic segmentation results of deep learning can be conveniently combined with the present invention to improve the recognition accuracy of cyanobacterial blooms.

[0066] 3. This method can be implemented using programming languages ​​such as Python and JAVA. There are a lot of contents that can be calculated in parallel in the algorithm calculation process. GPU can be used to achieve fast parallel calculation, so the input image can be processed quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is the overall flow chart of the method of the present invention;

[0068] Figure 2 It is a schematic diagram of the relative flight altitude and lens tilt angle when the drone is shooting;

[0069] Figure 3 It is a schematic diagram of the field scene measurement of the UAV field of view;

[0070] Figure 4 This is a schematic diagram of the photos taken by the drone and the corresponding actual situation. DETAILED DESCRIPTION

[0071] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0072] Example

[0073] Taking the Caohai Lake at Dabokou, Dianchi Lake as an example, drone photos at an altitude of 80 meters were automatically acquired using the DJI Airport 2 drone (the photo name is: DJI_20240725093654_0008_V.jpeg), and automatically uploaded to the server. After using DeepLabV3+ as the basic model for recognition, the method of the present invention was used to calculate the area of ​​cyanobacteria blooms and automatically output the results report.

[0074] This method uses Python for programming during implementation (other programming languages ​​can also be used to implement this method). The specific implementation process is as follows Figure 1 The detailed steps are as follows:

[0075] S1. Automatic cruise of drone airport

[0076] Use a drone with an automatic cruise function to automatically obtain photos of the target area. The drone's cruise route is 100 meters along the coast of Dianchi Caohai. The cruise route is designed based on the general empirical law of the distribution of blue algae blooms over the years. The photo information obtained by the drone includes the latitude and longitude coordinates of the photo shooting point, the relative ground flight altitude of 80 meters, and the tilt angle of the drone when taking pictures is 45 degrees. This information will be synchronously recorded in the shooting status information of the drone photo when the drone is shooting.

[0077] S2. Automatic upload of video data

[0078] After the drone completes its cruise mission and lands at the airport, it uses the airport's network environment to automatically upload image data to the server.

[0079] S3, AI automatic recognition

[0080] The server receives the photos automatically transmitted by the drone, uses the trained DeepLabV3+ model to automatically identify the cyanobacteria bloom in each photo, and then uses the information in the photo itself to calculate the area. The specific calculation is as follows:

[0081] S3.1. Identification of cyanobacterial blooms in drone photos

[0082] S3.1.1. Semantic segmentation model construction and training

[0083] Since the existing semantic segmentation model mainly processes conventional visible light, it is necessary to improve the existing semantic segmentation model to make it compatible with multispectral image data. The specific improvement method is to judge the input image. If it is a 3-band color image, the model training is directly performed. If it is a 5-band multispectral image, the input image is integrated using the band integrator C. The function of the band integrator C is to integrate the input 5-band multispectral data into a 3-band image according to the weight parameters, and then use the integrated three-band image as the input of the semantic segmentation model. The calculation of each band in the band integrator C is as shown in equations (1) to (3):

[0084] C 1 =0.3*Red+0.6*Green+0.1*Blue (1)

[0085] C 2 =0.3*NIR+0.6*Green+0.1*Blue (2)

[0086] C 3 =0.3*RE+0.6*Green+0.1*Blue (3)

[0087] Among them, Red, Green, Blue, NIR, and RE represent the red, green, blue, near infrared, and red edge bands respectively.

[0088] The improved model is composed of a band integrator C and a basic model. In this embodiment, the basic model uses the DeepLabV3+ model, and other mainstream semantic segmentation models such as Unet can also be used.

[0089] Create a cyanobacterial bloom sample label dataset D. The label organization format is: (photo name, label file with the same name). For example, the label file corresponding to the photo P001.JPG is P001.JSON, where there are two label categories: CB and O). Among them, CB (Cyanobacterial Bloom) represents cyanobacterial bloom, and O (Other) represents non-cyanobacterial bloom.

[0090] The semantic segmentation model was trained using the cyanobacteria bloom sample dataset D, and the model training result with the highest accuracy in the model validation set was taken as the final inference model IM (Inference model) for automatic identification of cyanobacteria blooms.

[0091] S3.1.2. Use the trained model to identify cyanobacteria blooms and obtain a binary image of the identification results.

[0092] When the drone airport image data is transmitted back to the inference server, the automatic identification task T of cyanobacteria bloom is triggered. Task T first scans the image folder of the day and compares it with the database file by file. If the photo has been recorded in the database, the cyanobacteria bloom identification of the photo is skipped and the next photo is read. If the photo is not recorded in the database, the inference model IM is used to identify the cyanobacteria bloom and the photo record is added to the database. The model recognition result is stored as a binary image, where white represents cyanobacteria bloom and black represents non-cyanobacteria bloom.

[0093] S3.2. Area statistics of cyanobacteria blooms in drone photos

[0094] S3.2.1. Read the drone photo shooting status information and lens parameters.

[0095] The relative altitude H recorded in the photo information when the photo was taken is 80 meters (RelativeAltitude field in EXIF, value is 80.022), the tilt angle θ of the lens at the moment of photo shooting is 45° (GimbalPitchDegree field in EXIF, value is -45.00), and the field of view α is measured, including the long side field of view α1 and the short side field of view α2 of the lens. Figure 2 As shown in the figure, point A is the position of the drone when taking the photo, point O is the ground point corresponding to the direction of gravity from the drone lens, the length of the line segment OA is the relative flight altitude H at the moment the photo is taken, AC is the orientation of the drone lens, ∠OAC is the tilt angle θ of the lens at the moment the photo is taken, points B and D are the actual positions corresponding to the edges of the drone photo's field of view, and ∠BAC=∠DAC.

[0096] The long side field of view angle α1 and the short side field of view angle α2 of the drone lens are measured using a real scene according to the process of this method.

[0097] (1) Select a hardened road that is sufficiently flat and long near the drone airport.

[0098] (2) Using the hardened road as a reference, point the drone lens vertically downward at a flight altitude of 30 meters, 60 meters, and 90 meters, and take two photos respectively, with the long side of one photo parallel to the road and the short side of the other photo parallel to the road. At the same time, try to keep the road in the middle of each photo, and divide the photo into two parts evenly. Figure 3 As shown, point A is the position of the drone, and line segment AO 1 The length of line segment AO is 90 meters. 2 The length of line segment AO is 60 meters. 3 The length is 30 meters, corresponding to the flight altitude of the drone at 90 meters, 60 meters and 30 meters respectively. Line segment B 1 D1 , B 2 D 2 and B 3 D 3 They represent the length of the straight road photographed by the drone at a flight altitude of 90 meters, 60 meters, and 30 meters. The relationship between the two photos and the selected road is shown in the attached figure. Figure 3 Shown in the two pictures on the right.

[0099] (3) Measure the length of the road in each photo on the spot or use geographic information software on satellite images, and combine it with the corresponding flight altitude to calculate the long side field of view angle α1 and the short side field of view angle α2 of the drone lens according to formula (4). Figure 4 As shown, ∠B 1 AD 1 That is the field of view angle α.

[0100] (4)

[0101] When the straight road is parallel to the short side of the drone photo, ∠B 1 AD 1 is the long side field of view angle α1. When the straight road is parallel to the long side of the drone photo, ∠B 1 AD 1 It is the short side field angle α2. Generally speaking, the two field angles of the same size photos taken by the same model of drone lens are basically the same, and there is no need to repeat the measurement.

[0102] S3.2.2. Use the UAV’s flight altitude and shooting angle to calculate the ground resolution for each pixel in the corresponding photo.

[0103] The spatial resolution of the pixel (SRP) is calculated by the distance L between the target point and the drone lens, the camera pixel size s, and the camera focal length f. The specific calculation formula is:

[0104] (5)

[0105] Among them, the pixel size s and focal length f are generally fixed parameters of the drone lens, which can be directly obtained through the device parameters or can be converted to Consider it as a resolution parameter GSD_Ref, and obtain the value of GSD_Ref through actual measurement, that is:

[0106] (6)

[0107] Formula (5) can be rewritten as: , in the field photos taken by the drone vertically downward, select two feature points in the center of the photo, measure the field distance P between the two obvious feature points, and then use image viewing tools such as Photoshop to count the number of pixels N between the two obvious feature points. According to the rewritten formula (5), SRP can be calculated, and the result obtained based on formula (5) can be verified. At this time, since the drone shoots vertically downward, L is the flight altitude of the drone, and the resolution parameter GSD_Ref can be calculated using formula (6).

[0108] As attached Figure 4 As shown in (b1), , The SRP at point B can be calculated according to formula (5), and the SRP at point O can be calculated by substituting the OA length (i.e., the flight altitude of the UAV) into formula (5).

[0109] S3.2.3. Calculate the pixel resolution corresponding to each pixel row in the photo

[0110] As attached Figure 4 As shown in the figure, the change of ground resolution in the photo is divided into two cases according to the relationship between the tilt angle θ of the lens at the moment of photo shooting and the field angle α of the photo consistent with the tilt direction. In order to simplify the calculation, this method assumes that the ground resolution corresponding to the same row of pixels is the same, that is, the pixel rows parallel to the base of the isosceles trapezoid have the same ground resolution.

[0111] (1) When α>2θ, the correspondence between the photographed image and the actual situation is as shown in the attached figure. Figure 4 As shown in (a2), at this time, the photo can be regarded as two isosceles trapezoids, one in front and one in the back. These two isosceles trapezoids share the same short base EF, where the long base of the front isosceles trapezoid is IJ and the height is OD, and the long base of the rear isosceles trapezoid is KL and the height is OB. The change in ground resolution size corresponding to each pixel on the photo is shown in the attached figure. Figure 4 As shown in sub-figure (a3) ​​in the figure, the rule is based on the resolution at the short side EF. As the pixel row changes, its size changes along the long side according to the ratio k. The ratio k represents the increase in ground resolution corresponding to one pixel for each additional pixel row, which is calculated according to equations (7) and (8).

[0112] (7)

[0113] (8)

[0114] Among them, R 前 and R 后 Respectively represent the number of pixel rows corresponding to the height of the front isosceles trapezoid and the rear isosceles trapezoid, that is, the number of pixels corresponding to the actual distances OD and OB in the photo.前 +R 后 = Photo length in pixels (or Photo width in pixels).

[0115] The pixel row where the KL edge is located is the starting row (i.e., i=0), the pixel row where the IJ edge is located is the ending row (i.e., i=Img_Width, where Img_Width is the width of the photo), and the pixel row where the EF edge is located is the Mth row, SRP M is the pixel spatial resolution of the pixel row where the EF edge is located. At this time, the pixel resolution size of each row is SRP_R （i） It can be calculated according to formula (9).

[0116] (9)

[0117] Among them, SRP_R (i) It represents the actual resolution of each pixel in the ith pixel row starting from the pixel row where the KL edge is located, in centimeters per pixel. SRP represents the pixel spatial resolution of the pixel row where the EF edge is located.

[0118] (2) When α≤2θ, the corresponding situation between the photographed photo and the actual situation is as shown in the attached figure. Figure 4 As shown in the right figure, the ground resolution distribution corresponding to each pixel in the photo is as follows Figure 4 As shown in the sub-figure (b3) in the figure. At this time, after the photos taken by the drone are matched to the actual distance, an inverted isosceles trapezoid range is obtained. Among them, the long base of the inverted isosceles trapezoid is MN, the short base is GH, and the height of the inverted isosceles trapezoid is BD. At this time,

[0119] (10)

[0120] Where R represents the number of pixel rows corresponding to the height of the isosceles trapezoid, that is, the number of pixels corresponding to the actual distance BD in the photo. The resolution of each pixel row can be calculated according to formula (11).

[0121] (11)

[0122] Among them, SRP_R (i) It represents the actual resolution of each pixel in the i-th pixel row starting from the pixel row where the GH edge is located, in units of centimeters per pixel. SRP represents the pixel spatial resolution of the pixel row where the GH edge is located, that is, the SRP at point B.

[0123] S3.2.4. Using the binary image of the drone identification result, count the number of pixels with different resolutions and multiply them by the ground resolution corresponding to the pixel to obtain the actual area of ​​cyanobacteria bloom. Calculate according to formula (12).

[0124] (12)

[0125] Among them, Area_Sum represents the total area of ​​cyanobacteria blooms in the target photo, i represents the number of pixel rows, Indicates the resolution of each pixel in the i-th row of pixels. Indicates the number of pixels in the ith row that are identified as cyanobacteria blooms. Img_Width indicates the width of the photo.

[0126] Using the above method, the area of ​​cyanobacteria bloom corresponding to the DJI_20240725093654_0008_V.jpeg photo was calculated to be 119,565 square meters (0.1196 square kilometers).

[0127] S4. Output results report

[0128] After comprehensively expressing the original image taken by the drone, the automatic identification result of cyanobacteria blooms, the spatiotemporal information annotation map of the drone photos and related statistical results such as the area, the iText library interface is used to automatically output the results report in PDF format.

[0129] The spatiotemporal information annotation map of drone photos is obtained through the following steps:

[0130] S4.1. Read the drone photo shooting status information

[0131] Read the shooting information recorded in the photos taken by the drone, including GPS location information and shooting time information;

[0132] S4.2. Generate point annotation map

[0133] Based on the GPS coordinate information read, the shooting point is superimposed on the satellite image base map according to its actual longitude and latitude, and a fixed-scale annotation map is generated with the annotation point as the center point;

[0134] S4.3. Overlaying time information on the annotation graph

[0135] Overlay the shooting time of drone photos on the annotation map as additional information.

[0136] The specific embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. An automatic monitoring method for blue algae blooms combining AI and drones, characterized in that: The steps include: S1. Automatic cruise of drone airport Use a drone with an automatic cruise function to automatically obtain photos of the target area. The photo information obtained by the drone includes the GPS coordinates of the photo shooting point, the relative flying height to the ground, and the tilt angle of the drone when taking the photo; S2. Automatic upload of video data Photos taken by drones are transmitted to the server for processing; S3, AI automatic recognition S3.

1. Identification of cyanobacterial blooms in drone photos The server receives the photos automatically transmitted by the drone and uses the trained AI model to automatically identify the cyanobacteria blooms in each photo; S3.

2. Area statistics of cyanobacteria blooms in drone photos The area of ​​cyanobacteria bloom is calculated using the information in the photo itself. The specific calculation is as follows: S3.2.

1. Read the drone photo shooting status information and lens parameters Read the relative flight altitude H recorded in the photo information when each photo is taken, the tilt angle θ of the lens at the moment the photo is taken, and determine the field of view angle α; S3.2.

2. Calculate the ground resolution of each pixel in the photo. The specific calculation formula is: ； Among them, L is the distance between the target point and the drone lens, s is the camera pixel size, f is the camera focal length, and s and f are fixed parameters of the drone lens; S3.2.

3. Calculate the pixel resolution corresponding to each pixel row in the photo According to the relationship between the tilt angle θ of the lens at the moment of taking the photo and the field angle α of the photo that is consistent with the tilt direction, the change of ground resolution in the photo is divided into two cases, and it is assumed that the ground resolution corresponding to the same row of pixels is the same; (1) When α>2θ, the photo is regarded as two isosceles trapezoids, one in front and one in back. The two isosceles trapezoids share the same short base EF. The long base of the front isosceles trapezoid is IJ and the height is OD, while the long base of the rear isosceles trapezoid is KL and the height is OB. As the pixel row changes, the ground resolution changes along the long side according to the ratio k. The ratio k represents the increase in ground resolution corresponding to one pixel for each additional pixel row, which is calculated according to the following formula: ； ； Among them, R 前 and R 后 They represent the number of pixel rows corresponding to the height of the front isosceles trapezoid and the rear isosceles trapezoid, that is, the number of pixels corresponding to the actual distances OD and OB in the photo, R 前 +R 后 = photo pixel length or photo pixel width; The pixel row where the KL edge is located is the starting row, the pixel row where the IJ edge is located is the ending row, and the pixel row where the EF edge is located is the Mth row, SRP M is the pixel spatial resolution of the pixel row where the EF edge is located. At this time, the pixel resolution size of each row is SRP_R （i） Calculated according to the following formula: ； Among them, SRP_R (i) It indicates the real resolution of each pixel in the ith pixel row starting from the pixel row where the KL edge is located, SRP indicates the pixel spatial resolution of the pixel row where the EF edge is located, and Img_Width is the photo width; (2) When α≤2θ, after the photos taken by the drone are mapped to the actual distance, an inverted isosceles trapezoid range is obtained. The long base of the inverted isosceles trapezoid is MN, the short base is GH, and the height of the inverted isosceles trapezoid is BD. At this time: ； Among them, R represents the number of pixel rows corresponding to the height of the isosceles trapezoid, that is, the number of pixels corresponding to the actual distance BD in the photo. The resolution of each pixel row is calculated according to the following formula: Among them, SRP_R (i) It represents the real resolution corresponding to each pixel in the i-th pixel row starting from the pixel row where the GH edge is located, and SRP represents the pixel spatial resolution of the pixel row where the GH edge is located; S3.2.

4. Use the binary image of the drone identification result to count the number of pixels with different resolutions and multiply them by the ground resolution corresponding to the pixel to obtain the actual area of ​​cyanobacteria bloom, which is calculated according to the following formula: ； Among them, Area_Sum represents the total area of ​​cyanobacteria bloom in the target photo. Indicates the resolution of each pixel in the i-th row of pixels. represents the number of pixels identified as cyanobacterial blooms in the i-th row; S4. Output results report The results report is output by integrating the original image taken by the drone, the automatic identification result of cyanobacteria bloom, the spatiotemporal information annotation map of the drone photos and the area statistics results.

2. The method for automatic monitoring of blue algae blooms by combining AI and drones according to claim 1 is characterized in that: The identification of cyanobacteria blooms in drone photos specifically includes: S3.1.

1. Semantic segmentation model construction and training A band integrator C is added to the basic semantic segmentation model to integrate the 5-band multispectral images appearing in the input image. The calculation of each band in the band integrator C is as follows: C1=0.3*Red+0.6*Green+0.1*Blue C2=0.3*NIR+0.6*Green+0.1*Blue C3=0.3*RE+0.6*Green+0.1*Blue Among them, Red, Green, Blue, NIR, and RE represent the red, green, blue, near infrared, and red edge bands respectively; Create a cyanobacteria bloom sample label dataset D, use the cyanobacteria bloom sample dataset D to train a semantic segmentation model, and take the model training result with the highest accuracy in the model verification set as the final inference model for automatic identification of cyanobacteria blooms; S3.1.

2. Use the trained model to identify cyanobacteria blooms and obtain a binary image of the identification result The server scans the returned drone airport image data and compares it with the database file by file. If the photo has been recorded in the database, the blue algae bloom recognition of the photo will be skipped and the next photo will be read. If the photo has not been recorded in the database, the inference model will be used to identify the blue algae bloom and the photo record will be added to the database.

3. The method for automatic monitoring of blue algae blooms by combining AI and drones according to claim 1 is characterized in that: The field angle α is determined as follows: A reference road was selected near the drone airport, and no drones were detected at flight altitudes of 30m, 60m, and 90m. The drone lens is pointed vertically downward to take photos. The long side or short side of the photo is parallel to the road. The reference road is in the middle of the photo. Point A is the position of the drone. The length of line segment AO1 is 90 meters, the length of line segment AO2 is 60 meters, and the length of line segment AO3 is 30 meters, which correspond to the flight altitudes of the drone at 90 meters, 60 meters, and 30 meters, respectively. Line segments B1D1, B2D2, and B3D3 represent the lengths of straight roads photographed by the drone at flight altitudes of 90 meters, 60 meters, and 30 meters, respectively. The field of view angle α is calculated as follows: 。 4. The method for automatic monitoring of blue algae blooms by combining AI and drones according to claim 1, characterized in that: The spatiotemporal information annotation graph of the drone photo is obtained by the following steps: S4.

1. Read the drone photo shooting status information Read the shooting information recorded in the photos taken by the drone, including GPS location information and shooting time information; S4.

2. Generate point annotation map Based on the GPS coordinate information read, the shooting point is superimposed on the satellite image base map according to its actual longitude and latitude, and a fixed-scale annotation map is generated with the annotation point as the center point; S4.

3. Overlaying time information on the annotation graph Overlay the shooting time of drone photos on the annotation map as additional information.

Images