Cloud and weather phenomenon unmanned detection method, equipment, medium and product

Abstract

The invention discloses a cloud and weather phenomenon unmanned detection method and device, a medium and a product, and relates to the technical field of aircraft meteorological detection, and the method comprises the steps: obtaining an unmanned plane image; determining the environment state of the unmanned aerial vehicle according to the unmanned aerial vehicle image; when the environment states of the unmanned aerial vehicle are in the cloud, clear sky and on the cloud, the current environment states of the unmanned aerial vehicle are recorded respectively and stored in the unmanned aerial vehicle storage equipment respectively; when the cloud top height calculation condition is met, the cloud top height is obtained through calculation; adopting a tolerance method to mark clouded position points with approximately same gray levels, and calculating geographic positions corresponding to the clouded position points one by one to obtain cloud coverage; obtaining a cloud-shaped product based on the cloud top height and the cloud coverage; the cloud top height, the cloud coverage and the cloud-shaped product are stored in unmanned aerial vehicle storage equipment; and obtaining weather phenomenon information according to the unmanned aerial vehicle image, and storing the weather phenomenon information into the unmanned aerial vehicle storage device. The meteorological detection cost can be reduced, and the maneuverability is high.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft meteorological detection, and in particular to an unmanned detection method, equipment, medium and product for cloud and weather phenomena. Background Art

[0002] Clouds play an important role in indicating and regulating weather and climate change. At the same time, clouds are also an important weather phenomenon that affects daily work and life, especially one of the important limiting factors for aircraft takeoff and flight. The current cloud measurement methods mainly include ground-based conventional manual observation, ground-based remote sensing cloud measurement, balloon platform-based cloud measurement, airborne microwave radiometer cloud measurement, airborne passive remote sensing cloud measurement, and airborne active remote sensing cloud measurement. The ground-based conventional manual observation method has the characteristics of high temporal and spatial resolution, but it has a large subjective component. Generally, it can only obtain cloud base information, and multi-layer clouds cannot be identified. It is difficult to observe accurately at night and when vertical visibility is poor. The ground-based remote sensing cloud measurement method uses radar, microwave radiometer, all-sky imager and other equipment, which has the characteristics of high data quality and high local temporal and spatial resolution, but it is expensive. The balloon platform-based cloud measurement has the advantage of long observation time, but it is not easy to operate, the cost is high, the temporal and spatial resolution is low, and because the balloon drifts with the wind, it is impossible to carry out long-term continuous observation of the target cloud. The airborne microwave radiometer cloud measurement method is direct and reliable, and can be observed at any time and place. In particular, commercial aircraft can cover the entire world, but the temporal and spatial resolution is low, and the data is limited due to the influence of airspace and severe convective weather, and the cost is also high. The air-based passive remote sensing cloud measurement method uses visible light, infrared, microwave passive remote sensing and other technologies to obtain cloud top information more accurately. It has wide spatial coverage, high accuracy, and can achieve global observation, but it is powerless to detect low clouds, and is affected by various atmospheric substances (such as aerosols, ice crystals, etc.) and the characteristics of the earth's surface, and the inversion ability is limited. The air-based active remote sensing method uses satellite-borne radar laser detection and other technologies to detect the three-dimensional distribution of global clouds, but the cost is high and it is impossible to carry out long-term continuous observation of local target areas.

[0003] In summary, the above methods are suitable for continuous observation of target clouds in local areas. There are conventional ground-based manual observations, ground-based remote sensing cloud measurement, airborne microwave radiometer cloud measurement and other methods. However, they are costly and are greatly affected by subjective factors, observation site selection, observation range, airport location, personnel support and other factors. They are not suitable for portable and mobile application scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, medium and product for unmanned detection of clouds and weather phenomena, which can reduce the cost of meteorological detection and have strong mobility.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] In a first aspect, the present invention provides an unmanned cloud and weather phenomenon detection method, the unmanned cloud and weather phenomenon detection method comprising:

[0007] Obtain UAV images and UAV image parameters; the UAV image is a video data file image after two consecutive frames of encoding; the UAV image parameters include: shooting time, location, UAV attitude and optoelectronic mission payload attitude.

[0008] The environmental state of the drone is determined according to the drone image; the environmental state includes: in the clouds, clear sky and above the clouds.

[0009] When the environment state of the drone is in the cloud, the current environment state of the drone is recorded as in the cloud, and the current environment state of the drone is stored in the drone storage device as in the cloud.

[0010] When the environment state of the drone is clear sky, if the position of the cloud relative to the drone in the previous frame image is still clear sky, the current environment state of the drone is recorded as clear sky, and the current environment state of the drone is stored as clear sky in the drone storage device; if the drone is under the clouds in the previous frame image, the cloud bottom height is calculated and stored in the drone storage device.

[0011] When the drone is in the cloud, it is determined whether the cloud top height calculation condition is met and a determination result is obtained.

[0012] If the judgment result is no, the current environmental state of the drone is recorded as being on the cloud, and the current environmental state of the drone is stored as being on the cloud in the drone storage device.

[0013] If the judgment result is yes, the cloud top height is calculated.

[0014] The tolerance method is used to mark the cloud locations with approximately the same grayscale, and the geographical locations corresponding to the cloud locations are calculated one by one to obtain the cloud coverage.

[0015] Based on the cloud top height and the cloud coverage, a cloud shape product is obtained, and the cloud top height, the cloud coverage and the cloud shape product are respectively stored in a drone storage device.

[0016] Weather phenomenon information is obtained based on the drone image, and the weather phenomenon information is stored in a drone storage device.

[0017] Optionally, when the flight altitude of the UAV in the two frames of images is the same, the cloud is stationary, and the main optical axis of the optoelectronic mission payload is perpendicular to the ground, the calculation formula for the cloud top height is:

[0018]

[0019] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the UAV at time t1, and L is the flight distance of the UAV from time t1 to time t2.

[0020] Optionally, when the flying altitude of the UAV in the two frames of images is different, the cloud is stationary, and the main optical axis of the optoelectronic mission payload is perpendicular to the ground, the calculation formula for the cloud top height is:

[0021]

[0022] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the UAV at time t1, L is the flight distance of the UAV from time t1 to time t2, and Δh is the difference in flight height between the UAV at time t1 and time t2.

[0023] Optionally, when the flying altitude of the UAV in the two frames of images is different, the main optical axis of the optoelectronic mission payload is perpendicular to the ground, and the cloud is displaced, the calculation formula for the cloud top height is:

[0024]

[0025] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the UAV at time t1, L is the flight distance of the UAV from time t1 to time t2, Δh is the difference in flight height of the UAV between time t1 and time t2, and S is the horizontal displacement of the cloud from time t1 to time t2.

[0026] Optionally, when the flight altitude of the UAV in the two frames of images is the same, the cloud is stationary, and the main optical axis of the optoelectronic mission payload has a certain inclination angle, the calculation formula for the cloud top height is:

[0027]

[0028] r 1 =F·tg(α 1 +β 1 );

[0029]

[0030] r 2=F·tg(α 2 +β 2 );

[0031] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the drone at time t1, L is the flight distance of the drone from time t1 to time t2, g 1 is the distance from the actual photoelectric mission payload focus at time t1 to the extension line of the cloud-payload extension line along the actual imaging focal plane direction, α 1 is the angle between the principal optical axis of the actual optoelectronic mission payload and the cloud edge at time t1, β 1 is the angle between the principal optical axis of the actual optoelectronic mission payload at time t1 and the principal optical axis of the optoelectronic mission payload after rotation, g 2 is the distance from the actual photoelectric mission payload focus at time t2 to the extension line of the cloud-payload extension line along the actual imaging focal plane direction, α 2 is the angle between the principal optical axis of the actual optoelectronic mission payload and the cloud edge at time t2, β 2 It is the angle between the principal optical axis of the actual optoelectronic mission load at time t2 and the principal optical axis of the optoelectronic mission load after rotation.

[0032] Optionally, when the flying altitude of the UAV in the two frames of images is different, the cloud is stationary, and the main optical axis of the optoelectronic mission payload has a certain inclination angle, the calculation formula of the cloud top height is:

[0033]

[0034] r 1 =F·tg(α 1 +β 1 );

[0035]

[0036] r 2 =F·tg(α 2 +β 2 );

[0037] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the UAV at time t1, L is the flight distance of the UAV from time t1 to time t2, Δh is the difference in flight height between the UAV at time t1 and time t2, g 1 is the distance from the actual photoelectric mission payload focus at time t1 to the extension line of the cloud-payload extension line along the actual imaging focal plane direction, α1 is the angle between the principal optical axis of the actual optoelectronic mission payload and the cloud edge at time t1, β 1 is the angle between the principal optical axis of the actual optoelectronic mission payload at time t1 and the principal optical axis of the optoelectronic mission payload after rotation, g 2 is the distance from the actual photoelectric mission payload focus at time t2 to the extension line of the cloud-payload extension line along the actual imaging focal plane direction, α 2 is the angle between the principal optical axis of the actual optoelectronic mission payload and the cloud edge at time t2, β 2 It is the angle between the principal optical axis of the actual optoelectronic mission load at time t2 and the principal optical axis of the optoelectronic mission load after rotation.

[0038] Optionally, when the flying altitude of the UAV in the two frames of images is different, the cloud is displaced, and the main optical axis of the optoelectronic mission payload has a certain inclination angle, the calculation formula of the cloud top height is:

[0039]

[0040] r 1 =F·tg(α 1 +β 1 );

[0041]

[0042] r 2 =F·tg(α 2 +β 2 );

[0043] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the drone at time t1, L is the flight distance of the drone from time t1 to time t2, Δh is the difference in flight height of the drone from time t1 to time t2, S is the horizontal displacement of the cloud from time t1 to time t2, g 1 is the distance from the actual photoelectric mission payload focus at time t1 to the extension line of the cloud-payload extension line along the actual imaging focal plane direction, α 1 is the angle between the principal optical axis of the actual optoelectronic mission payload and the cloud edge at time t1, β 1 is the angle between the principal optical axis of the actual optoelectronic mission payload at time t1 and the principal optical axis of the optoelectronic mission payload after rotation, g 2 is the distance from the actual photoelectric mission payload focus at time t2 to the extension line of the cloud-payload extension line along the actual imaging focal plane direction, α 2 is the angle between the principal optical axis of the actual optoelectronic mission payload and the cloud edge at time t2, β 2 It is the angle between the principal optical axis of the actual optoelectronic mission load at time t2 and the principal optical axis of the optoelectronic mission load after rotation.

[0044] In a third aspect, the present invention provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described unmanned detection methods for clouds and weather phenomena.

[0045] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described unmanned detection methods for clouds and weather phenomena.

[0046] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned unmanned detection methods for clouds and weather phenomena.

[0047] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] The present invention provides a method, device, medium and product for unmanned detection of clouds and weather phenomena. The environmental state of the drone is determined based on the acquired drone images. When the drone is in the cloud, the cloud top height can be calculated. The cloud position points with approximately the same grayscale can be marked using a tolerance method. The geographical locations corresponding to the cloud position points are calculated one by one to obtain cloud coverage. Cloud products can be obtained based on the cloud top height and the cloud coverage. In addition, weather phenomenon information can be obtained based on the drone images. Ordinary drone platforms and optoelectronic imaging payloads are currently inexpensive on the market. Compared with traditional cloud measurement methods, the cost of the present invention is significantly reduced. At the same time, a set of drone platforms and optoelectronic imaging payloads are very light, and observations can be carried out by a single person, and are almost not restricted by the site. Finally, by operating the drone platform, continuous, mobile tracking observations of target clouds can be carried out for a long time, and the drone platform battery can be replaced to maintain continuous cloud observations. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 This is a diagram of the application environment of an unmanned cloud and weather phenomenon detection method in one embodiment of the present invention.

[0051] Figure 2A schematic flow chart of an unmanned cloud and weather phenomenon detection method provided by one embodiment of the present invention.

[0052] Figure 3 A schematic diagram of the imaging principle of an optoelectronic mission payload provided in one embodiment of the present invention.

[0053] Figure 4 A schematic diagram of cloud top height calculation under ideal conditions provided by one embodiment of the present invention.

[0054] Figure 5 Schematic diagram of cloud top height calculation principle for different flight altitudes provided by an embodiment of the present invention

[0055] Figure 6 A schematic diagram of a cloud top height calculation principle when a cloud is displaced according to an embodiment of the present invention.

[0056] Figure 7 A schematic diagram of cloud top height calculation when the main optical axis has a certain inclination angle provided by an embodiment of the present invention.

[0057] Figure 8 A processing flow chart of a cloud and weather phenomenon analysis subsystem provided in one embodiment of the present invention.

[0058] Fig. 9 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] The cloud and weather phenomenon unmanned detection method provided by the embodiment of the present invention can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the acquired drone image and drone image parameters to the server 104, and the drone image is a video data file image after 2 consecutive frames of encoding; the drone image parameters include: shooting time, location, drone attitude and optoelectronic mission load attitude; after the server 104 receives the drone image and the drone image parameters, for the drone image and the drone image parameters, the server 104 determines the environmental state of the drone according to the drone image; the environmental state includes: in the cloud, clear sky and on the cloud; when the environmental state of the drone is in the cloud, the current environmental state of the drone is recorded as in the cloud, and the current environmental state of the drone is stored in the drone storage device as in the cloud; when the environmental state of the drone is clear sky, if the position of the previous frame image cloud relative to the drone is still clear sky, the current environmental state of the drone is recorded as clear sky, and the current environmental state of the drone is stored in the drone storage device as in the cloud; If the state is clear sky, the drone storage device is stored; if the drone is under the cloud in the previous frame image, the cloud bottom height is calculated and stored in the drone storage device; when the drone is in the cloud, it is determined whether the cloud top height calculation condition is met to obtain a judgment result; if the judgment result is no, the current state of the drone is recorded as being in the cloud, and the current state of the drone is in the cloud is stored in the drone storage device; if the judgment result is yes, the cloud top height is calculated; the tolerance method is used to mark the cloud position points with approximately the same grayscale, and the geographical locations corresponding to the cloud position points are calculated one by one to obtain cloud coverage; based on the cloud top height and the cloud coverage, a cloud product is obtained, and the cloud top height, the cloud coverage and the cloud product are respectively stored in the drone storage device; according to the drone image, weather phenomenon information is obtained, and the weather phenomenon information is stored in the drone storage device. The server 104 can feed back the obtained cloud product and weather phenomenon information to the terminal 102. In addition, in some embodiments, the unmanned detection method of clouds and weather phenomena can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly perform unmanned detection processing on the acquired drone images and drone image parameters, or the server 104 can obtain drone images and drone image parameters from the data storage system and perform unmanned detection processing on the drone images and drone image parameters.

[0062] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, and tablet computers. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0063] In an exemplary embodiment, Figure 2 As shown, a method for unmanned detection of cloud and weather phenomena is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In an embodiment of the present invention, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S1 to S10. Among them:

[0064] S1: Acquire UAV images and UAV image parameters; the UAV image is a video data file image after two consecutive frames of encoding; the UAV image parameters include: shooting time, location, UAV attitude and optoelectronic mission payload attitude.

[0065] S2: Determine the environmental state of the drone according to the drone image; the environmental state includes: in the clouds, clear sky and above the clouds.

[0066] S3: When the environment state of the drone is in the cloud, the current environment state of the drone is recorded as in the cloud, and the current environment state of the drone is stored in the drone storage device.

[0067] S4: When the environment state of the drone is clear sky, if the position of the cloud relative to the drone in the previous frame image is still clear sky, then the current environment state of the drone is recorded as clear sky, and the current environment state of the drone is stored as clear sky in the drone storage device; if the drone is under the cloud in the previous frame image, the cloud bottom height is calculated and stored in the drone storage device.

[0068] S5: When the drone is in the cloud, determine whether the cloud top height calculation condition is met and obtain a determination result.

[0069] S6: If the judgment result is no, the current environment state of the drone is recorded as being on the cloud, and the current environment state of the drone is stored as being on the cloud in the drone storage device.

[0070] S7: If the judgment result is yes, calculate the cloud top height.

[0071] S8: Using the tolerance method, mark the cloud position points with approximately the same grayscale, and calculate the geographical locations corresponding to the cloud position points one by one to obtain the cloud coverage.

[0072] S9: Based on the cloud top height and the cloud coverage, a cloud shape product is obtained, and the cloud top height, the cloud coverage and the cloud shape product are respectively stored in a drone storage device.

[0073] S10: Obtain weather phenomenon information according to the drone image, and store the weather phenomenon information in a drone storage device.

[0074] By implementing the above steps S1 to S10, the present invention has the following beneficial effects:

[0075] First, the cost is low. The current market price of ordinary UAV platforms and optoelectronic imaging payloads is low, which significantly reduces the cost compared with traditional cloud measurement methods.

[0076] Second, it is highly maneuverable. A set of drone platforms and optoelectronic imaging payloads are very light, so a single person can conduct observations, and there are almost no restrictions on the site.

[0077] The third is continuous tracking and observation. By operating the drone platform, continuous and mobile tracking and observation of the target cloud can be carried out for a long time. The battery of the drone platform can be replaced to maintain continuous cloud observation.

[0078] After the UAV enters the cruise flight phase, the flight altitude is 5000m, and the cloud top height and cloud coverage of the middle and low clouds can be detected, and then the cloud top height and cloud coverage products in the track area can be generated. Figure 3 .

[0079] Clouds of different heights may be located at the same position in the image, so it is difficult to determine the height and position of the cloud from a single frame of image. There are two ways to obtain the cloud top height and cloud boundary position.

[0080] One is to use the operator's experience judgment method. When the UAV flies directly above the cloud boundary, the position of the cloud boundary is determined by the UAV's position information. The operator then makes an experience judgment on the video taken by the visible light payload, that is, estimates the cloud top height based on the shape of the clouds in the video.

[0081] The second is to use the triangulation method in the binocular imaging system to calculate the cloud top height and the position of the cloud boundary.

[0082] 1) Cloud top height measurement under ideal conditions.

[0083] Assume that at time t1 and time t2 (the drone is closer to the cloud at time t2), the drone is flying at the same altitude, the cloud is stationary, and the main optical axis of the optoelectronic mission payload is perpendicular to the ground. Figure 4 .

[0084] According to the basic principle of similar triangles:

[0085]

[0086] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the UAV at time t1, L is the flight distance of the UAV from time t1 to time t2, and X is the horizontal distance between the UAV and the edge of the cloud at time t2.

[0087] After sorting:

[0088]

[0089] From formula (2), we can see that r 1 -r 2 It should not be too small (such as close to zero).

[0090] 2) The flight altitude at time t1 and time t2 is different.

[0091] Assume that at time t1 and time t2, the drone is flying at different altitudes, the cloud is stationary, and the main optical axis of the optoelectronic mission payload is perpendicular to the ground. Figure 5 .

[0092] According to the basic principle of similar triangles:

[0093]

[0094] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the UAV at time t1, L is the flight distance of the UAV from time t1 to time t2, Δh is the difference in flight height between the UAV at time t1 and time t2, and X is the horizontal distance between the UAV and the edge of the cloud at time t2.

[0095] After sorting:

[0096]

[0097] 3) The cloud shifted at time t1-t2.

[0098] Usually clouds will move and change under the influence of wind. Consider the situation where the cloud position changes at time t1 and time t2 (at time t2, the drone is closer to the cloud), and the rest is the same as situation 2). Figure 6 .

[0099] According to the basic principle of similar triangles:

[0100]

[0101] Where H is the cloud top height, r 1 is the imaging offset distance at time t1, r 2 is the imaging offset distance at time t2, F is the imaging focal plane, h 1 is the flight height of the UAV at time t1, L is the flight distance of the UAV from time t1 to time t2, Δh is the difference in flight height of the UAV from time t1 to time t2, S is the horizontal displacement of the cloud from time t1 to time t2, and X is the horizontal distance between the UAV and the edge of the cloud at time t2.

[0102] After sorting:

[0103]

[0104] 4) The main optical axis of the optoelectronic mission payload has a certain inclination angle.

[0105] Considering the case where the main optical axis of the optoelectronic mission payload has a certain inclination angle, the imaging system can be seen Figure 7 .in:

[0106]

[0107] r 1 =F·tg(α 1 +β 1 ) (8);

[0108]

[0109] r 2 =F·tg(α 2 +β 2 ) (10);

[0110] In the formula, g 1 is the distance from the actual photoelectric mission payload focus at time t1 to the extension line of the cloud-payload extension line along the actual imaging focal plane direction, α 1 is the angle between the principal optical axis of the actual optoelectronic mission payload and the cloud edge at time t1, β 1 is the angle between the principal optical axis of the actual optoelectronic mission payload at time t1 and the principal optical axis of the optoelectronic mission payload after rotation, g 2 is the distance from the actual photoelectric mission payload focus at time t2 to the extension line of the cloud-payload extension line along the actual imaging focal plane direction, α 2 is the angle between the principal optical axis of the actual optoelectronic mission payload and the cloud edge at time t2, β 2 It is the angle between the principal optical axis of the actual optoelectronic mission load at time t2 and the principal optical axis of the optoelectronic mission load after rotation.

[0111] The r in formula (8)1 and r in formula (10) 2 Substituting into formula (2), formula (4) or formula (6) respectively, the cloud top height under different assumptions can be calculated.

[0112] If β 1 and β 2 When it is larger, 1 and r 2 The calculation results will be greatly affected, so β ​​should be kept as close as possible during flight. 1 =0 and β 2 =0.

[0113] When using binocular ranging to measure the distance of an object with a rigid boundary, it is necessary to align the feature points of the two images (generally the boundary is selected). Since the edge of the cloud is fuzzy, it is difficult to find the feature points for alignment. The r in the above formula is 1 and r 2 Cannot be given individually.

[0114] For formula (4), if Δh = 0 or r 2 = 0 (i.e. the drone is flying level or the edge of the cloud happens to be at the center of the image at time t2), then formula (4) can be simplified to formula (2), where r 1 -r 2 is the displacement of the cloud edge. Therefore, in actual calculation, we only need to find a way to get r 1 -r 2 It is not necessary to find the feature points of the two images.

[0115] For formula (5), if S is negligible (i.e., the ground speed of the cloud is very small relative to the UAV), it can also be simplified to formula (2).

[0116] 5) Edge mobility detection in the cloud.

[0117] The edge of the cloud moves in the opposite direction to the drone, and in the video it appears to always move vertically downward. In the field of computer vision, one of the effective methods for detecting moving objects is the "optical flow" method. In the application of the present invention, the "optical flow" method is used to calculate the vertical motion pixel speed of each point on the edge of the cloud.

[0118] Assume that the gray value of the observed object point at the time t, position (x, y) is f(x, y, t). When the object point moves, at the time (t+Δt), the object moves from (x, y) to (x+Δx, y+Δy). Since Δt is very small, it can be approximated as:

[0119] f(x,y,t)=f(x+Δx,y+Δy,t+Δt) (11);

[0120] Expand the right side of the above equation by tailor and set Δt→0, then we have:

[0121]

[0122] Right now:

[0123] fxu+fyv+ft=0 (13);

[0124] In the formula,

[0125] In the above formula, f x 、f y 、f t The estimate is:

[0126]

[0127]

[0128] So the object's velocity (u, v) satisfies:

[0129]

[0130] Here, ε is the error estimate.

[0131] By imposing smooth constraints, the optical flow field is obtained. The variational model is:

[0132]

[0133] In the formula, E S is the variational optical flow energy function, λ weighting factor.

[0134] After discretization, the minimization model is:

[0135]

[0136] Where e is the discretized variational optical flow energy function, s ij is the data item of the optical flow variational energy function, c ij is the smoothing term of the optical flow variational energy function.

[0137]

[0138] In the formula, u i,j 、v i,j are the u and v values ​​at position (i, j) respectively.

[0139] So we can get the iterative process:

[0140]

[0141] in, for u kl 、v kl The average of the four neighborhoods of u. kl 、v kl are the u and v values ​​at position (k, l) respectively.

[0142] The method for selecting moving points in an image is to count the grayscale distribution of two adjacent images and find the first maximum point in the histogram, which corresponds to the grayscale of the image background and is set to g. 1 , g 2 ,like

[0143] |I n -I n+1 |>|g n -g n+1 |+ε (24);

[0144] The pixel is regarded as a point on the moving target, I i is the grayscale of the i-th image, g i is the grayscale of the image background corresponding to the first maximum point in the histogram of the i-th image, and ε is the error estimate, which is obtained according to the experimental results in the specific implementation.

[0145] The average velocity of all moving target points in the image is calculated to get the estimated velocity of the cloud, which is multiplied by the interval between image captures to get the displacement r of the cloud edge. 1 -r 2 .

[0146] In summary, in order to automatically calculate the cloud top height within the track range, the following points should be achieved:

[0147] a) During the UAV mission planning stage, the flight track and flight altitude should be planned based on the cloud coverage and cloud top height obtained by inverting the satellite cloud image at that time.

[0148] b) Before reaching the edge of the cloud, the UAV should adjust the main optical axis of the optoelectronic mission payload to make it perpendicular to the ground.

[0149] c) The drone should try to maintain level flight when it reaches the edge of the cloud.

[0150] d) The detected cloud should have a smaller ground speed and the UAV should have a larger ground speed.

[0151] e) To reduce errors, it should not be too small and should be able to be calculated continuously to obtain the average value.

[0152] After the cloud top height is measured, the location of the cloud edge can be further calculated. Using the "tolerance" method in image processing, the "cloudy" location points with roughly the same grayscale are marked, and their corresponding geographical locations are calculated one by one to generate cloud cover products.

[0153] By continuously calculating the cloud top height and cloud cover of continuous video or replayed images, cloud top height and cloud cover products within the track area can be generated. The operator combines the video and replayed images to interpret and annotate the clouds within the determined geographical range, and then generates cloud shape products within the track area.

[0154] In another exemplary embodiment of the present invention, Figure 8 As shown, the real-time monitoring subsystem can store video files by unit when encoding and saving the video. After the video files are stored, they are processed according to the following flow chart. It mainly includes:

[0155] A1: Separate the encoded video data file into two consecutive images.

[0156] A2: According to the shooting time of the two images, by querying the drone storage device, obtain the time including the shooting time, drone position, attitude, optoelectronic mission payload attitude and other parameters, and obtain the drone position, attitude, optoelectronic mission payload attitude and other parameters that completely correspond to the shooting time through linear interpolation, calculate the geographical range of the optoelectronic mission payload field of view, and store it in the drone storage device.

[0157] A3: Based on two consecutive images, the environmental status of the drone can be determined, including clear sky, in the clouds, and above the clouds.

[0158] A31: If the drone is in the cloud, record the current state of the drone and store it in the drone storage device.

[0159] A32: If the drone is in the cloud, first determine whether the cloud top height calculation conditions are met. If so, calculate the cloud top height and cloud coverage, and obtain cloud shape information through manual interpretation and store it in the drone storage device. If not, record the current state of the drone and store it in the drone storage device.

[0160] A33: If the current environment state of the drone is clear sky, the position of the cloud relative to the drone is determined based on the previous frame image. If it is still clear sky, it is recorded as clear sky and stored in the drone storage device; if the drone is under the cloud in the previous frame image, the cloud bottom height is obtained and stored in the drone storage device.

[0161] A4: Based on two consecutive images, weather phenomenon information is obtained through manual interpretation and stored in the drone storage device.

[0162] The present invention has the following beneficial effects:

[0163] First, the cost is low. The current market price of ordinary UAV platforms and optoelectronic imaging payloads is low, which significantly reduces the cost compared with traditional cloud measurement methods.

[0164] Second, it is highly maneuverable. A set of drone platforms and optoelectronic imaging payloads are very light, so a single person can conduct observations, and there are almost no restrictions on the site.

[0165] The third is continuous tracking and observation. By operating the drone platform, continuous and mobile tracking and observation of the target cloud can be carried out for a long time. The battery of the drone platform can be replaced to maintain continuous cloud observation.

[0166] The present invention also provides an application scenario, which applies the above-mentioned unmanned detection method of clouds and weather phenomena. Specifically: The unmanned detection method of clouds and weather phenomena provided in this embodiment can be applied in aircraft meteorological detection scenarios. The aircraft meteorological detection scenario includes: a drone image acquisition link, a drone environmental state determination link, an environmental state recording link, a cloud product calculation link and a weather phenomenon information determination link; according to the acquired drone image, the drone environmental state is determined; the drone image is a video data file image after 2 consecutive frames of encoding; the environmental state includes: in the cloud, clear sky and above the cloud; when the drone's environmental state is in the cloud, the drone's current environmental state is recorded as in the cloud, and the drone's current environmental state is stored in the cloud in the drone storage device; when the drone's environmental state is clear sky, if the previous frame of the image cloud relative to the drone is still clear sky, then the drone's current environmental state is recorded as clear sky, and the drone's current environmental state is stored in the drone storage device as clear sky; if the previous frame of the image cloud is still clear sky, then the drone's current environmental state is recorded as clear sky, and the drone's current environmental state is stored in the drone storage device as clear sky; if the previous frame of the image cloud is still clear sky, then the drone's current environmental state is recorded as clear sky, and the drone's current environmental state is stored in the drone storage device as clear sky; if the previous frame of the image cloud is still clear sky, then the drone's current environmental state is recorded as clear sky, and the drone's current environmental state is stored in the drone storage device as clear sky; if the previous frame of the image cloud is still clear sky, then the drone's current environmental state is recorded as clear sky, and the drone's current environmental state is stored in the drone storage device as clear sky. If the drone is under the clouds in a frame image, the cloud base height is calculated and stored in the drone storage device; when the drone is in the cloud state, it is determined whether the cloud top height calculation condition is met to obtain a determination result; if the determination result is no, it is recorded that the drone is currently in the cloud state, and the drone is currently in the cloud state and stored in the drone storage device; if the determination result is yes, the cloud top height is calculated; a tolerance method is used to mark cloud position points with approximately the same grayscale, and the geographical locations corresponding to the cloud position points are calculated one by one to obtain cloud coverage; based on the cloud top height and the cloud coverage, a cloud shape product is obtained, and the cloud shape product is stored in the drone storage device; weather phenomenon information is obtained based on the drone image, and the weather phenomenon information is stored in the drone storage device.

[0167] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store drone images and drone image parameters. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for unmanned detection of clouds and weather phenomena is implemented.

[0168] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0169] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above-mentioned method embodiments are implemented when the processor executes the computer program.

[0170] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0171] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0173] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0174] The database involved in each embodiment provided by the present invention may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided by the present invention may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for unmanned detection of clouds and weather phenomena, characterized in that: The cloud and weather phenomenon unmanned detection method comprises: Obtaining drone images and drone image parameters; the drone image is a video data file image after two consecutive frames of encoding; the drone image parameters include: shooting time, location, drone attitude and optoelectronic mission payload attitude; Determine the environmental state of the drone according to the drone image; the environmental state includes: in the clouds, clear sky and above the clouds; When the environment state of the drone is in the cloud, the current environment state of the drone is recorded as in the cloud, and the current environment state of the drone is stored in the drone storage device; When the environment state of the drone is clear sky, if the position of the cloud relative to the drone in the previous frame image is still clear sky, then the current environment state of the drone is recorded as clear sky, and the current environment state of the drone is clear sky is stored in the drone storage device; if the drone is under the cloud in the previous frame image, the cloud bottom height is calculated and stored in the drone storage device; When the drone is in the cloud, determine whether the cloud top height calculation condition is met and obtain the judgment result; If the judgment result is no, the current environment state of the drone is recorded as being on the cloud, and the current environment state of the drone is stored in the drone storage device as being on the cloud; If the judgment result is yes, then calculating the cloud top height; The tolerance method is used to mark cloud locations with approximately the same grayscale, and the geographical locations corresponding to the cloud locations are calculated one by one to obtain cloud coverage. Based on the cloud top height and the cloud cover, a cloud shape product is obtained, and the cloud top height, the cloud cover and the cloud shape product are respectively stored in a drone storage device; Weather phenomenon information is obtained based on the drone image, and the weather phenomenon information is stored in a drone storage device.

2. The unmanned cloud and weather phenomenon detection method according to claim 1, characterized in that: When the flying altitude of the UAV in the two frames is the same, the cloud is stationary, and the main optical axis of the optoelectronic mission payload is perpendicular to the ground, the calculation formula for the cloud top height is: Among them, H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, and L is the UAV flight distance from time t1 to time t2.

3. The unmanned cloud and weather phenomenon detection method according to claim 1, characterized in that: When the UAV is flying at different altitudes in the two frames, the cloud is stationary, and the main optical axis of the optoelectronic mission payload is perpendicular to the ground, the calculation formula for the cloud top height is: Among them, H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, and Δh is the difference in UAV flight altitude between time t1 and time t2.

4. The unmanned cloud and weather phenomenon detection method according to claim 1, characterized in that: When the UAV's flight altitudes in the two frames are different, the main optical axis of the optoelectronic mission payload is perpendicular to the ground, and the cloud is displaced, the calculation formula for the cloud top height is: Among them, H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the UAV flight altitude difference between time t1 and time t2, and S is the horizontal displacement of the cloud from time t1 to time t2.

5. The unmanned cloud and weather phenomenon detection method according to claim 1, characterized in that: When the flying altitude of the UAV in the two frames of images is the same, the cloud is stationary, and the main optical axis of the optoelectronic mission payload has a certain inclination angle, the calculation formula for the cloud top height is: r1=F·tg(α1+β1); r2=F·tg(α2+β2); Wherein, H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, g1 is the distance from the focus of the actual optoelectronic mission payload at time t1 to the extension line of the cloud-payload extension line along the direction of the actual imaging focal plane, α1 is the angle between the main optical axis of the actual optoelectronic mission payload at time t1 and the cloud edge, β1 is the angle between the main optical axis of the actual optoelectronic mission payload at time t1 and the main optical axis of the optoelectronic mission payload after rotation, g2 is the distance from the focus of the actual optoelectronic mission payload at time t2 to the extension line of the cloud-payload extension line along the direction of the actual imaging focal plane, α2 is the angle between the main optical axis of the actual optoelectronic mission payload at time t2 and the cloud edge, β2 is the angle between the main optical axis of the actual optoelectronic mission payload at time t2 and the main optical axis of the rotated optoelectronic mission payload.

6. The unmanned cloud and weather phenomenon detection method according to claim 1, characterized in that: When the UAV is flying at different altitudes in the two frames of images, the cloud is stationary, and the main optical axis of the optoelectronic mission payload has a certain inclination angle, the calculation formula for the cloud top height is: r1=F·tg(α1+β1); r2=F·tg(α2+β2); Wherein, H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude between time t1 and time t2, g1 is the distance from the focus of the actual optoelectronic mission payload at time t1 to the extension line of the cloud-payload extension line along the direction of the actual imaging focal plane, α1 is the angle between the main optical axis of the actual optoelectronic mission payload at time t1 and the cloud edge, β1 is the angle between the main optical axis of the actual optoelectronic mission payload at time t1 and the main optical axis of the rotated optoelectronic mission payload, g2 is the distance from the focus of the actual optoelectronic mission payload at time t2 to the extension line of the cloud-payload extension line along the direction of the actual imaging focal plane, α2 is the angle between the main optical axis of the actual optoelectronic mission payload at time t2 and the cloud edge, β2 is the angle between the main optical axis of the actual optoelectronic mission payload at time t2 and the main optical axis of the rotated optoelectronic mission payload.

7. The unmanned cloud and weather phenomenon detection method according to claim 1, characterized in that: When the UAV's flight altitudes in the two frames are different, the clouds are displaced, and the main optical axis of the optoelectronic mission payload has a certain inclination angle, the calculation formula for the cloud top height is: r1=F·tg(α1+β1); r2=F·tg(α2+β2); Wherein, H is the cloud top height, r1 is the imaging offset distance at time t1, r2 is the imaging offset distance at time t2, F is the imaging focal plane, h1 is the UAV flight altitude at time t1, L is the UAV flight distance from time t1 to time t2, Δh is the difference in UAV flight altitude from time t1 to time t2, S is the horizontal displacement of the cloud from time t1 to time t2, g1 is the distance from the focus of the actual optoelectronic mission payload at time t1 to the extension line of the cloud-payload extension line along the direction of the actual imaging focal plane, α1 is the angle between the main optical axis of the actual optoelectronic mission payload at time t1 and the cloud edge, β1 is the angle between the main optical axis of the actual optoelectronic mission payload at time t1 and the main optical axis of the rotated optoelectronic mission payload, g2 is the distance from the focus of the actual optoelectronic mission payload at time t2 to the extension line of the cloud-payload extension line, α2 is the angle between the main optical axis of the actual optoelectronic mission payload at time t2 and the cloud edge, β2 is the angle between the main optical axis of the actual optoelectronic mission payload at time t2 and the main optical axis of the rotated optoelectronic mission payload.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the unmanned detection method for clouds and weather phenomena according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the unmanned cloud and weather phenomenon detection method described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the unmanned cloud and weather phenomenon detection method described in any one of claims 1 to 7 is implemented.

Images