A method, equipment, medium, and product for unmanned detection of cloud and weather phenomena.

Abstract

This invention discloses an unmanned aerial vehicle (UAV) method, device, medium, and product for detecting clouds and weather phenomena, relating to the field of aircraft meteorological detection technology. The method includes: acquiring UAV images; determining the environmental state of the UAV based on the images; recording the current environmental state of the UAV when it is in the clouds, in clear sky, or above clouds, and storing each state in the UAV's storage device; calculating the cloud top height when the cloud top height calculation conditions are met; using a tolerance method, marking cloud locations with approximately the same grayscale, and calculating the corresponding geographical locations of each cloud location to obtain cloud coverage; obtaining cloud-like products based on the cloud top height and cloud coverage; storing the cloud top height, cloud coverage, and cloud-like products in the UAV's storage device; and obtaining weather phenomenon information based on the UAV images and storing the weather phenomenon information in the UAV's storage device. This invention can reduce meteorological detection costs and offers high mobility.

Description

Technical Field

[0001] This invention relates to the field of aircraft meteorological detection technology, and in particular to an unmanned detection method, equipment, medium and product for clouds and weather phenomena. Background Technology

[0002] Clouds play a crucial role in indicating and regulating weather and climate change. They are also a significant weather phenomenon affecting daily life and work, particularly a major limiting factor for aircraft takeoff and flight. Current cloud measurement methods mainly include conventional ground-based manual observation, ground-based remote sensing, balloon-based cloud measurement, airborne microwave radiometer cloud measurement, airborne passive remote sensing, and airborne active remote sensing. Conventional ground-based manual observation methods offer high spatiotemporal resolution but are highly subjective, generally only acquiring cloud base information and failing to identify multi-layered clouds. Accurate observation is also difficult at night and in situations with poor vertical visibility. Ground-based remote sensing methods utilize equipment such as radar, microwave radiometers, and all-sky imagers, offering high data quality and high local spatiotemporal resolution, but are expensive. Balloon-based cloud measurement offers long observation times but is difficult to operate, costly, has lower spatiotemporal resolution, and, due to the balloon's drift with the wind, cannot conduct long-term continuous observations of target clouds. Airborne microwave radiometers are a direct and reliable method for cloud measurement, allowing observations at any time and location, especially with commercial aircraft providing global coverage. However, their spatiotemporal resolution is relatively low, and data is limited by airspace and severe convective weather, while also being costly. Space-based passive remote sensing methods, employing visible light, infrared, and microwave passive remote sensing technologies, can accurately acquire cloud top information, offering wide spatial coverage, high precision, and global observation capabilities. However, they are ineffective at detecting low clouds and their inversion capabilities are limited by various atmospheric substances (such as aerosols and ice crystals) and Earth's surface characteristics. Space-based active remote sensing methods, using spaceborne radar and laser detection technologies, can detect the three-dimensional distribution of clouds globally, but are costly and cannot conduct long-term continuous observations of local target areas.

[0003] In summary, the above methods are suitable for continuous observation of target clouds in local areas, including conventional ground-based manual observation, ground-based remote sensing of clouds, and airborne microwave radiometer cloud measurement. However, these methods are costly and subject to significant influences from subjective factors, observation site selection, observation range, airport location, and personnel support, making them unsuitable for portable and mobile applications. Summary of the Invention

[0004] The purpose of this invention is to provide an unmanned detection method, equipment, medium, and product for cloud and weather phenomena, which can reduce the cost of meteorological detection and has high mobility.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] In a first aspect, the present invention provides an unmanned detection method for clouds and weather phenomena, the unmanned detection method for clouds and weather phenomena comprising:

[0007] Acquire drone images and drone image parameters; the drone images are video data files of two consecutive encoded frames; the drone image parameters include: shooting time, location, drone attitude, and optoelectronic mission payload attitude.

[0008] Based on the drone images, the environmental state of the drone is determined; the environmental state includes: in the clouds, clear sky, and above the clouds.

[0009] When the drone is in a cloud environment, the drone's current environment status is recorded as "in the cloud" and stored in the drone's storage device.

[0010] When the drone is in a clear sky environment, if the position of the cloud relative to the drone in the previous frame image is still clear sky, then the current environmental state of the drone is recorded as clear sky and stored in the drone's storage device; if the drone was under the cloud in the previous frame image, then the cloud base height is calculated and stored in the drone's storage device.

[0011] When the drone is in an environment above the clouds, determine whether the conditions for calculating the cloud top height are met, and obtain the judgment result.

[0012] If the judgment result is negative, then the current environmental state of the drone is recorded as "in the cloud" and stored in the drone's storage device.

[0013] If the judgment result is yes, then calculate the cloud top height.

[0014] Using a tolerance method, cloud locations with approximately the same gray level are marked, and the geographical location corresponding to each cloud location is calculated to obtain cloud coverage.

[0015] Based on the cloud top height and the cloud coverage, a cloud-shaped product is obtained, and the cloud top height, the cloud coverage, and the cloud-shaped product are stored in the drone storage device respectively.

[0016] Weather information is obtained from the drone images and stored in the drone's storage device.

[0017] Optionally, when the UAV is at the same flight altitude in two frames, the clouds are stationary, and the principal optical axis of the optoelectronic payload is perpendicular to the ground, the formula for calculating the cloud top height is:

[0018]

[0019] Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, and L is the UAV flight distance from time t1 to time t2.

[0020] Optionally, when the UAV is flying at different altitudes in the two images, the clouds are stationary, and the principal optical axis of the optoelectronic payload is perpendicular to the ground, the formula for calculating the cloud top height is:

[0021]

[0022] Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, and Δh is the difference in UAV flight altitude between time t1 and time t2.

[0023] Optionally, when the UAV flies at different altitudes in the two images, the principal optical axis of the optoelectronic payload is perpendicular to the ground, and the cloud has shifted, the formula for calculating the cloud top height is:

[0024]

[0025] Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, and S is the horizontal displacement of the cloud from time t1 to time t2.

[0026] Optionally, when the UAV is at the same flight altitude in two frames, the clouds are stationary, and the principal optical axis of the optoelectronic payload has a certain tilt angle, the formula for calculating the cloud top height is:

[0027]

[0028] r1 = F·tg(α1 + β1);

[0029]

[0030] r2=F·tg(α2+β2);

[0031] Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, g1 is the distance from the actual optical payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t1, α1 is the angle between the principal optical axis of the actual optical payload and the cloud edge at time t1, β1 is the angle between the principal optical axis of the actual optical payload and the principal optical axis of the rotated optical payload at time t1, g2 is the distance from the actual optical payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t2, α2 is the angle between the principal optical axis of the actual optical payload and the cloud edge at time t2, and β2 is the angle between the principal optical axis of the actual optical payload and the principal optical axis of the rotated optical payload at time t2.

[0032] Optionally, when the UAV flies at different altitudes in the two images, the clouds are stationary, and the principal optical axis of the optoelectronic payload has a certain tilt angle, the formula for calculating the cloud top height is:

[0033]

[0034] r1 = F·tg(α1+β1);

[0035]

[0036] r2=F·tg(α2+β2);

[0037] Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, g1 is the distance from the actual optical payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t1, α1 is the angle between the principal optical axis of the actual optical payload and the cloud edge at time t1, β1 is the angle between the principal optical axis of the actual optical payload and the principal optical axis of the rotated optical payload at time t1, g2 is the distance from the actual optical payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t2, α2 is the angle between the principal optical axis of the actual optical payload and the cloud edge at time t2, and β2 is the angle between the principal optical axis of the actual optical payload and the principal optical axis of the rotated optical payload at time t2.

[0038] Optionally, when the UAV flies at different altitudes in the two images, the clouds have shifted, and the principal optical axis of the optoelectronic payload has a certain tilt angle, the formula for calculating the cloud top height is:

[0039]

[0040] r1 = F·tg(α1+β1);

[0041]

[0042] r2=F·tg(α2+β2);

[0043] Wherein, H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, S is the horizontal displacement of the cloud from time t1 to time t2, g1 is the distance from the actual photoelectric payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t1, α1 is the angle between the principal optical axis of the actual photoelectric payload and the cloud edge at time t1, β1 is the angle between the principal optical axis of the actual photoelectric payload and the principal optical axis of the rotated photoelectric payload at time t1, g2 is the distance from the actual photoelectric payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t2, α2 is the angle between the principal optical axis of the actual photoelectric payload and the cloud edge at time t2, and β2 is the angle between the principal optical axis of the actual photoelectric payload and the principal optical axis of the rotated photoelectric payload at time t2.

[0044] Thirdly, the present invention provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the unmanned detection method for cloud and weather phenomena described in any of the above-mentioned methods.

[0045] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the unmanned detection method for cloud and weather phenomena described above.

[0046] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the unmanned detection method for cloud and weather phenomena described above.

[0047] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] This invention provides a method, device, medium, and product for unmanned cloud and weather phenomenon detection. Based on acquired drone images, the environmental state of the drone is determined. When the drone is in a cloud-covered environment, the cloud top height can be calculated. Using a tolerance method, cloud locations with approximately the same grayscale are marked, and the geographical locations corresponding to each cloud location are calculated to obtain cloud coverage. Based on the cloud top height and cloud coverage, a cloud-shaped product can be obtained. Additionally, weather phenomenon information can be obtained from the drone images. Currently, ordinary drone platforms and photoelectric imaging payloads are inexpensive, significantly reducing the cost of this invention compared to traditional cloud measurement methods. Furthermore, a drone platform and photoelectric imaging payload are very lightweight, allowing for observation by a single person with virtually no site limitations. Finally, by operating the drone platform, continuous, mobile tracking observation of target clouds can be conducted over extended periods, and the drone platform battery can be replaced to maintain continuous cloud observation. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is an application environment diagram of an unmanned detection method for cloud and weather phenomena according to an embodiment of the present invention.

[0051] Figure 2 This is a flowchart illustrating an unmanned detection method for clouds and weather phenomena provided in an embodiment of the present invention.

[0052] Figure 3 This is a schematic diagram of the imaging principle of an optoelectronic mission payload provided in an embodiment of the present invention.

[0053] Figure 4 This is a schematic diagram illustrating the principle of cloud top height calculation under ideal conditions, provided as an embodiment of the present invention.

[0054] Figure 5 A schematic diagram illustrating the principle of cloud top height calculation at different flight altitudes provided in an embodiment of the present invention.

[0055] Figure 6 This is a schematic diagram illustrating the principle of cloud top height calculation when clouds shift, as provided in an embodiment of the present invention.

[0056] Figure 7 This is a schematic diagram illustrating the principle of cloud top height calculation when the main optical axis has a certain tilt angle, as provided in an embodiment of the present invention.

[0057] Figure 8This is a flowchart of the cloud and weather phenomenon analysis subsystem provided in an embodiment of the present invention.

[0058] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] The unmanned detection method for clouds and weather phenomena provided in this invention can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send acquired drone images and drone image parameters to server 104. The drone image is a video data file image after encoding two consecutive frames; the drone image parameters include: shooting time, location, drone attitude, and optoelectronic payload attitude. After receiving the drone image and drone image parameters, server 104 determines the drone's environmental state based on the drone image. The environmental state includes: in the clouds, clear sky, and above the clouds. When the drone's environmental state is in the clouds, it records the current environmental state as "in the clouds" and stores it in the drone's storage device. When the drone's environmental state is clear sky, if the position of the clouds relative to the drone in the previous frame is still clear sky, it records the current environmental state as "clear sky" and stores it in the drone's storage device. The status is clear sky, stored in the drone's storage device; if the drone was below clouds in the previous frame, the cloud base height is calculated and stored in the drone's storage device; when the drone's environment is above clouds, it is determined whether the cloud top height calculation condition is met, and a judgment result is obtained; if the judgment result is negative, the current environment status of the drone is recorded as above clouds, and this status is stored in the drone's storage device; if the judgment result is positive, the cloud top height is calculated; using a tolerance method, cloud locations with approximately the same grayscale are marked, and the geographical location corresponding to each cloud location is calculated to obtain cloud coverage; based on the cloud top height and cloud coverage, cloud-like products are obtained, and the cloud top height, cloud coverage, and cloud-like products are stored in the drone's storage device respectively; based on the drone image, weather phenomenon information is obtained and stored in the drone's storage device. The server 104 can feed back the obtained cloud-like products and weather phenomenon information to the terminal 102. In addition, in some embodiments, the unmanned detection method for cloud and weather phenomena can also be implemented by the server 104 or the terminal 102 separately. For example, the terminal 102 can directly perform unmanned detection processing on the acquired drone images and drone image parameters, or the server 104 can obtain drone images and drone image parameters from the data storage system and perform unmanned detection processing on the drone images and drone image parameters.

[0062] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, and tablets. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0063] In one exemplary embodiment, such as Figure 2 As shown, an unmanned detection method for clouds and weather phenomena is provided. This method is executed by computer equipment, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment of the invention, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S1 to S10. Wherein:

[0064] S1: Acquire UAV images and UAV image parameters; the UAV images are video data file images encoded from two consecutive frames; the UAV image parameters include: shooting time, location, UAV attitude, and optoelectronic mission payload attitude.

[0065] S2: Determine the environmental state of the drone based on the drone image; the environmental state includes: in the clouds, clear sky, and above the clouds.

[0066] S3: When the drone is in a cloud environment, record the drone's current environment as being in a cloud and store this information in the drone's storage device.

[0067] S4: When the drone is in a clear sky environment, if the position of the cloud relative to the drone in the previous frame image is still clear sky, then record the current environment status of the drone as clear sky and store the current environment status of the drone as clear sky in the drone storage device; if the drone was under the cloud in the previous frame image, then calculate the cloud bottom height and store the cloud bottom height in the drone storage device.

[0068] S5: When the drone is in an environment above the clouds, determine whether the conditions for calculating the cloud top height are met, and obtain the judgment result.

[0069] S6: If the judgment result is negative, record the current environmental state of the drone as "in the cloud" and store the current environmental state of the drone as "in the cloud" in the drone storage device.

[0070] S7: If the judgment result is yes, then calculate the cloud top height.

[0071] S8: Using a tolerance method, mark cloud locations with approximately the same gray level, and calculate the geographical location corresponding to each cloud location to obtain cloud coverage.

[0072] S9: Based on the cloud top height and the cloud coverage, obtain the cloud-shaped product, and store the cloud top height, the cloud coverage, and the cloud-shaped product into the UAV storage device respectively.

[0073] S10: Obtain weather phenomenon information based on the drone image, and store the weather phenomenon information in the drone storage device.

[0074] By implementing steps S1 to S10 above, the present invention has the following beneficial effects:

[0075] First, the cost is low. Currently, ordinary UAV platforms and optoelectronic imaging payloads are inexpensive on the market, significantly reducing costs compared to traditional cloud measurement methods.

[0076] Secondly, it is highly mobile. A single UAV platform and optoelectronic imaging payload weigh very little, allowing a single person to conduct observations, and it is almost unrestricted by location.

[0077] Third, it can continuously track and observe. By operating the drone platform, it is possible to conduct continuous, mobile tracking and observation of target clouds for extended periods of time. The drone platform's battery can be replaced to maintain continuous cloud observation.

[0078] Once the UAV enters its cruise flight phase at an altitude of 5000m, it can detect the cloud top height and cloud cover of low and mid-level clouds, thereby generating cloud top height and cloud cover products within its flight path area. The imaging principle of the optoelectronic payload is described in [link to documentation]. Figure 3 .

[0079] Clouds at different altitudes may appear in the same position in an image, making it difficult to determine their height and location from a single frame. Two methods can be used to obtain the cloud top height and the location of the cloud boundary.

[0080] One approach is to use operator experience to interpret the data. When the drone flies directly above the cloud boundary, the location of the cloud boundary is determined by the drone's location information. Then, the operator interprets the video captured by the visible light payload based on experience, that is, estimates the cloud top height based on the cloud shape in the video.

[0081] Secondly, the cloud top height and cloud boundary position are calculated using the triangulation method in the binocular imaging system.

[0082] 1) Measurement of cloud top height under ideal conditions.

[0083] Assuming the UAV flies at the same altitude at times t1 and t2 (at time t2 the UAV is closer to the cloud), the cloud is stationary, and the principal optical axis of the optoelectronic payload is perpendicular to the ground, see... Figure 4 .

[0084] From the basic principle of similar triangles, we get:

[0085]

[0086] Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, and X is the horizontal distance between the UAV and the edge of the cloud at time t2.

[0087] After sorting, we get:

[0088]

[0089] As can be seen from formula (2), r1-r2 should not be too small (e.g., close to zero).

[0090] 2) The flight altitude is different at time t1 and time t2.

[0091] Assuming the UAV flies at different altitudes at times t1 and t2, the clouds are stationary, and the principal optical axis of the optoelectronic payload is perpendicular to the ground, see... Figure 5 .

[0092] From the basic principle of similar triangles, we get:

[0093]

[0094] Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, and X is the horizontal distance between the UAV and the edge of the cloud at time t2.

[0095] After sorting, we get:

[0096]

[0097] 3) The situation where the cloud shifted between time t1 and t2.

[0098] Clouds typically move and change under the influence of wind. Considering the situation where the cloud's position changed at times t1 and t2 (at time t2 the drone was closer to the cloud), the rest are the same as in case 2). Figure 6 .

[0099] From the basic principle of similar triangles, we get:

[0100]

[0101] Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, S is the horizontal displacement of the cloud from time t1 to time t2, and X is the horizontal distance between the UAV and the edge of the cloud at time t2.

[0102] After sorting, we get:

[0103]

[0104] 4) The main optical axis of the optoelectronic payload has a certain tilt angle.

[0105] Considering the case where the principal optical axis of the optoelectronic payload has a certain tilt angle, the imaging system is as follows: Figure 7 .in:

[0106]

[0107] r1=F·tg(α1+β1) (8);

[0108]

[0109] r2=F·tg(α2+β2) (10);

[0110] In the formula, g1 is the distance from the focal point of the actual optoelectronic payload along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t1, α1 is the angle between the principal optical axis of the actual optoelectronic payload and the edge of the cloud at time t1, β1 is the angle between the principal optical axis of the actual optoelectronic payload and the principal optical axis of the rotated optoelectronic payload at time t1, g2 is the distance from the focal point of the actual optoelectronic payload along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t2, α2 is the angle between the principal optical axis of the actual optoelectronic payload and the edge of the cloud at time t2, and β2 is the angle between the principal optical axis of the actual optoelectronic payload and the principal optical axis of the rotated optoelectronic payload at time t2.

[0111] By substituting r1 in formula (8) and r2 in formula (10) into formula (2), formula (4) or formula (6) respectively, the cloud top height under different assumptions can be calculated.

[0112] If β1 and β2 are large, they will have a significant impact on the calculation results of r1 and r2. Therefore, during flight, β1 = 0 and β2 = 0 should be maintained as much as possible.

[0113] When performing binocular ranging on objects with rigid boundaries, it is necessary to register the feature points (usually the boundary) of the two images. Since the edges of clouds are blurred, it is difficult to find feature points for registration, and r1 and r2 in the above formula cannot be given separately.

[0114] For formula (4), if Δh = 0 or r2 = 0 (i.e., the edge of the cloud is exactly at the center of the image when the UAV is flying horizontally or at time t2), then formula (4) can be simplified to formula (2), where r1-r2 is the displacement of the cloud edge. Therefore, in actual calculation, it is only necessary to obtain r1-r2, without having to find the feature points of the two images.

[0115] For formula (5), if S is negligible (i.e., the ground speed of the cloud is very small relative to the drone), it can also be simplified to formula (2).

[0116] 5) Cloud edge movement detection.

[0117] The edge of the cloud moves in the opposite direction relative to the drone, appearing as a continuous vertical downward motion in the video. In the field of computer vision, one effective method for detecting moving objects is optical flow. In this invention, optical flow is proposed to calculate the vertical motion pixel velocity at each point on the cloud edge.

[0118] Let the grayscale value of the observed object point at time t, position (x, y), be f(x, y, t). When the object point is displaced, at time (t + Δt), the object moves from (x, y) to (x + Δx, y + Δy). Since Δt is very small, we can approximate it as follows:

[0119] f(x,y,t)=f(x+Δx,y+Δy,t+Δt) (11);

[0120] Expanding the right side of the above equation using the tailor method and setting Δt→0, we have:

[0121]

[0122] Right now:

[0123] fxu+fyv+ft=0 (13);

[0124] In the formula,

[0125] In the above formula, f x f y f t The estimate is:

[0126]

[0127]

[0128] Therefore, the velocity (u, v) of the object satisfies:

[0129]

[0130] Where ε is the error estimate.

[0131] The optical flow field is obtained by applying smoothing constraints. The variational model is as follows:

[0132]

[0133] In the formula, E S Let be the variational optical flow energy function. λ weighting factor.

[0134] After discretization, the minimization model is:

[0135]

[0136] In the formula, e is the discretized variational optical flow energy function, and s ij c is a data term of the optical flow variational energy function. ij This is the smoothing term of the optical flow variational energy function.

[0137]

[0138] In the formula, u i,j v i,j Let u and v be the values ​​at position (i,j), respectively.

[0139] Therefore, the iterative process can be obtained:

[0140]

[0141] in, For u kl v kl The four-neighbor average. kl v kl Let u and v be the values ​​at positions (k, l), respectively.

[0142] The method for selecting moving points in an image is as follows: statistically analyze the gray-level distribution of two adjacent images, find the first maximum point in the histogram of each image, which corresponds to the gray level of the image background, and denote it as g1 and g2.

[0143] |I n -I n+1 |>|g n -g n+1 |+ε (24);

[0144] Then the pixel is considered as a point on the moving target, I iLet g be the gray level of the i-th image. i Let ε be the gray level of the background corresponding to the first maximum point in the histogram of the i-th image, and let ε be the error estimate, which is obtained based on the experimental results in the specific implementation.

[0145] The average velocity of all moving target points in the image can be used to estimate the velocity of the cloud. Multiplying this by the image capture interval gives the displacement r1-r2 of the cloud edge.

[0146] In summary, to automatically calculate the cloud top altitude within the flight path range, the following points should be considered:

[0147] a) During the UAV mission planning phase, the flight path and altitude should be planned based on the cloud coverage and cloud top height obtained from the satellite cloud image at that time.

[0148] b) Before the UAV reaches the edge of the cloud layer, the main optical axis of the optoelectronic payload should be adjusted to make it perpendicular to the ground.

[0149] c) When the drone reaches the edge of the cloud layer, it should try to maintain a level flight.

[0150] d) The cloud being detected should have a low ground speed, and the drone should have a high ground speed.

[0151] e) To reduce errors, the value should not be too small. It should also be able to be calculated continuously to obtain the average value.

[0152] After the cloud top height is measured, the location of the cloud edge can be further calculated. Using the "tolerance" method in image processing, "cloudy" location points with approximately the same gray level are marked, and their corresponding geographical locations are calculated one by one to generate cloud coverage products.

[0153] By continuously calculating cloud top height and cloud cover from continuous video or playback images, cloud top height and cloud cover products within the flight path area can be generated. Operators combine video and playback images to interpret and label cloud shapes within a defined geographical area, thereby generating cloud shape products within the flight path area.

[0154] In another exemplary embodiment of the present invention, such as Figure 8 As shown, the real-time monitoring subsystem can store video files in units when encoding and saving video. After the video files are stored, they are processed according to the flowchart below. This mainly includes:

[0155] A1: Separate the encoded video data file into two consecutive images.

[0156] A2: Based on the shooting time of the two images, by querying the drone's storage device, obtain parameters such as the shooting time, drone position, attitude, and electro-optical payload attitude. Through linear interpolation, obtain the drone position, attitude, and electro-optical payload attitude parameters that completely correspond to the shooting time. Calculate the geographical range of the electro-optical payload's field of view and store it in the drone's storage device.

[0157] A3: Based on two consecutive images, determine the environmental state of the drone, including clear sky, in the clouds, or above the clouds.

[0158] A31: If the drone is in the cloud, record the drone's current status and store it in the drone's storage device.

[0159] A32: If the drone is in the cloud, first determine whether the conditions for calculating the cloud top height are met. If they are met, calculate the cloud top height and cloud coverage, and obtain the cloud information through manual interpretation, and store it in the drone's storage device. If the conditions are not met, record the drone's current status and store it in the drone's storage device.

[0160] A33: If the current environment of the drone is clear sky, the position of the cloud relative to the drone is determined based on the previous frame image. If it is still clear sky, the current state is recorded as clear sky and stored in the drone's storage device. If the drone was below the cloud in the previous frame image, the cloud base height is obtained and stored in the drone's storage device.

[0161] A4: Based on two consecutive images, weather phenomenon information is obtained through manual interpretation and stored in the drone's storage device.

[0162] The present invention has the following beneficial effects:

[0163] First, the cost is low. Currently, ordinary UAV platforms and optoelectronic imaging payloads are inexpensive on the market, significantly reducing costs compared to traditional cloud measurement methods.

[0164] Secondly, it is highly mobile. A single UAV platform and optoelectronic imaging payload weigh very little, allowing a single person to conduct observations, and it is almost unrestricted by location.

[0165] Third, it can continuously track and observe. By operating the drone platform, it is possible to conduct continuous, mobile tracking and observation of target clouds for extended periods of time. The drone platform's battery can be replaced to maintain continuous cloud observation.

[0166] This invention also provides an application scenario in which the above-mentioned unmanned cloud and weather phenomenon detection method is applied. Specifically, the unmanned cloud and weather phenomenon detection method provided in this embodiment can be applied in an aircraft meteorological detection scenario. The aircraft meteorological detection scenario includes: an unmanned aerial vehicle (UAV) image acquisition stage, an environmental state determination stage, an environmental state recording stage, a cloud formation calculation stage, and a weather phenomenon information determination stage; based on the acquired UAV image, the environmental state of the UAV is determined; the UAV image is a video data file image after encoding two consecutive frames; the environmental state includes: in the clouds, clear sky, and above the clouds; when the UAV is in the clouds, the current environmental state of the UAV is recorded as in the clouds, and the current environmental state of the UAV in the clouds is stored in the UAV storage device; when the UAV is in the clear sky, if the position of the clouds relative to the UAV in the previous frame image is still clear sky, the current environmental state of the UAV in the clear sky is recorded as clear sky, and the current environmental state of the UAV in the clear sky is stored in the UAV storage device; if the previous frame image is .... If a drone is below a cloud in an image frame, the cloud base height is calculated and stored in the drone's storage device. If the drone is above a cloud, it is determined whether the cloud top height calculation condition is met, and a judgment result is obtained. If the judgment result is negative, the current environment state of the drone is recorded as above a cloud and stored in the drone's storage device. If the judgment result is positive, the cloud top height is calculated. Using a tolerance method, cloud locations with approximately the same grayscale are marked, and the geographical location corresponding to each cloud location is calculated to obtain cloud coverage. Based on the cloud top height and cloud coverage, a cloud-shaped product is obtained and stored in the drone's storage device. Weather phenomenon information is obtained from the drone image and stored in the drone's storage device.

[0167] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores UAV images and UAV image parameters. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements an unmanned cloud and weather phenomenon detection method.

[0168] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0169] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method embodiments.

[0170] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method embodiments.

[0171] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method embodiments.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0173] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0174] The databases involved in the various embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the various embodiments provided by this invention may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for unmanned detection of clouds and weather phenomena, characterized in that, The unmanned detection methods for clouds and weather phenomena include: Acquire drone images and drone image parameters; the drone images are video data file images encoded from two consecutive frames; the drone image parameters include: shooting time, location, drone attitude, and optoelectronic mission payload attitude. Based on the drone images, the environmental state of the drone is determined; the environmental state includes: in the clouds, clear sky, and above the clouds; When the drone is in a cloud environment, record the drone's current environment as being in a cloud and store this information in the drone's storage device. When the drone is in a clear sky environment, if the position of the cloud relative to the drone in the previous frame image is still clear sky, then the current environmental state of the drone is recorded as clear sky and stored in the drone's storage device; if the drone was under the cloud in the previous frame image, then the cloud bottom height is calculated and stored in the drone's storage device. When the drone is in an environment above the clouds, determine whether the conditions for calculating the cloud top height are met, and obtain the judgment result. If the judgment result is negative, then record the current environmental state of the drone as "in the cloud" and store the current environmental state of the drone as "in the cloud" in the drone's storage device. If the judgment result is yes, then calculate the cloud top height; Using a tolerance method, cloud locations with approximately the same gray level are marked, and the geographical location corresponding to each cloud location is calculated to obtain cloud coverage. Based on the cloud top height and the cloud coverage, a cloud-shaped product is obtained, and the cloud top height, the cloud coverage, and the cloud-shaped product are stored in the drone storage device respectively; Weather information is obtained from the drone images and stored in the drone's storage device.

2. The unmanned detection method for clouds and weather phenomena according to claim 1, characterized in that, When the drone is flying at the same altitude in two frames, the clouds are stationary, and the principal optical axis of the optoelectronic payload is perpendicular to the ground, the formula for calculating the cloud top height is: Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, and L is the UAV flight distance from time t1 to time t2.

3. The unmanned detection method for clouds and weather phenomena according to claim 1, characterized in that, When the drone is flying at different altitudes in two frames, the clouds are stationary, and the principal optical axis of the optoelectronic payload is perpendicular to the ground, the formula for calculating the cloud top height is: Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, and Δh is the difference in UAV flight altitude between time t1 and time t2.

4. The unmanned detection method for clouds and weather phenomena according to claim 1, characterized in that, When the drone flies at different altitudes in two frames, the principal optical axis of the optoelectronic payload is perpendicular to the ground, and the cloud has shifted, the formula for calculating the cloud top height is: Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, and S is the horizontal displacement of the cloud from time t1 to time t2.

5. The unmanned detection method for clouds and weather phenomena according to claim 1, characterized in that, When the drone is at the same altitude in two frames, the clouds are stationary, and the principal optical axis of the optoelectronic payload has a certain tilt angle, the formula for calculating the cloud top height is: r1 = F·tg(α1+β1); r2=F·tg(α2+β2); Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, g1 is the distance from the actual optical payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t1, α1 is the angle between the principal optical axis of the actual optical payload and the cloud edge at time t1, β1 is the angle between the principal optical axis of the actual optical payload and the principal optical axis of the rotated optical payload at time t1, g2 is the distance from the actual optical payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t2, α2 is the angle between the principal optical axis of the actual optical payload and the cloud edge at time t2, and β2 is the angle between the principal optical axis of the actual optical payload and the principal optical axis of the rotated optical payload at time t2.

6. The unmanned detection method for clouds and weather phenomena according to claim 1, characterized in that, When the drone is flying at different altitudes in two frames, the clouds are stationary, and the principal optical axis of the optoelectronic payload has a certain tilt angle, the formula for calculating the cloud top height is: r1 = F·tg(α1+β1); r2=F·tg(α2+β2); Where H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, g1 is the distance from the actual optical payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t1, α1 is the angle between the principal optical axis of the actual optical payload and the cloud edge at time t1, β1 is the angle between the principal optical axis of the actual optical payload and the principal optical axis of the rotated optical payload at time t1, g2 is the distance from the actual optical payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t2, α2 is the angle between the principal optical axis of the actual optical payload and the cloud edge at time t2, and β2 is the angle between the principal optical axis of the actual optical payload and the principal optical axis of the rotated optical payload at time t2.

7. The unmanned detection method for clouds and weather phenomena according to claim 1, characterized in that, When the drone flies at different altitudes in two frames, the clouds have shifted, and the principal optical axis of the optoelectronic payload has a certain tilt angle, the formula for calculating the cloud top height is: r1 = F·tg(α1+β1); r2=F·tg(α2+β2); Wherein, H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, S is the horizontal displacement of the cloud from time t1 to time t2, g1 is the distance from the actual photoelectric payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t1, α1 is the angle between the principal optical axis of the actual photoelectric payload and the edge of the cloud at time t1, β1 is the angle between the principal optical axis of the actual photoelectric payload and the principal optical axis of the rotated photoelectric payload at time t1, g2 is the distance from the actual photoelectric payload focus along the actual imaging focal plane to the extension line of the cloud-payload straight line at time t2, α2 is the angle between the principal optical axis of the actual photoelectric payload and the edge of the cloud at time t2, and β2 is the angle between the principal optical axis of the actual photoelectric payload and the principal optical axis of the rotated photoelectric payload at time t2.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the unmanned detection method for cloud and weather phenomena according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the unmanned detection method for cloud and weather phenomena as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the unmanned detection method for cloud and weather phenomena as described in any one of claims 1-7.

Images