A 3D cloud scene modeling and visualization method based on ground-based cloud images

Abstract

The present invention provides a three-dimensional cloud scene modeling and visualization method based on ground-based cloud maps, wherein a CPU extracts a cumulus image area from a two-dimensional ground-based cloud map, and calculates the cloud base height and cloud thickness as parameters for three-dimensional cumulus modeling. The position information of the center points of all voxels constituting the cumulus model is determined based on the above two parameters, and is transmitted to the GPU as a vertex array. In the geometric shader stage of the GPU, all cumulus voxels are drawn according to the vertex array, and finally the fragment shader stage of the GPU is entered. The cloud density value is determined according to the distance between each voxel and the center point of the cloud cluster, and the color of the voxel is calculated according to the density value, thereby realizing the integration of three-dimensional cloud scene modeling and visualization. The present invention uses voxels as the smallest geometric unit, constructs a three-dimensional cumulus model based on a GPU, and realizes visualization in a three-dimensional scene, thereby improving the efficiency of three-dimensional cumulus modeling and meeting the requirements of smooth roaming in a three-dimensional geographic scene.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling and three-dimensional geographic information visualization, and in particular to a method for constructing a three-dimensional cumulus cloud model for ground-based cloud map data and realizing cumulus cloud visualization based on a three-dimensional geographic scene. Background Art

[0002] Solar photovoltaic power generation is a clean energy production activity that converts sunlight into electricity, and clouds are the main factor causing large fluctuations in solar radiation (Wang Chengliang, Zhu Lingzhi, Dang Dongsheng, Zhao Liang, Ding Huang. Impact of cloud movement on photovoltaic power station output characteristics and system frequency regulation [J]. Renewable Energy, 2017, 35(11): 1626-1631). Especially in a clear sky environment, the appearance of clouds causes an instantaneous reduction in solar radiation reaching the surface, which causes fluctuations in photovoltaic output power and affects the stable operation of the power grid system. Therefore, building a three-dimensional cloud model based on ground-based cloud map data in a three-dimensional geographic information platform can reflect the real cloud scene in three-dimensional geographic space, which is of great significance for the calculation of instantaneous and continuous photovoltaic power generation.

[0003] Solar radiation intensity is the most important factor that directly affects photovoltaic power generation. Clouds that appear in sunny environments make the solar radiation reaching the ground intermittent and unstable, resulting in uncontrollable output power of photovoltaic power stations. In particular, cumulus clouds are characterized by low altitude and small area, which places high demands on observation instruments. Traditional satellite sensor observation methods cannot meet the requirements in terms of time and spatial resolution (Kleiss J, Riley E, Long C, et al. Cloud Area Distributions of Shallow Cumuli: A New Method for Ground-Based Images [J]. Atmosphere, 2018, 9: 258). In recent years, the emergence of ground-based remote sensing cloud instruments with high temporal and spatial resolution has realized real-time monitoring and collection of clouds above photovoltaic power stations. For example, the all-sky imager uses a 160° fisheye lens to shoot ground-based cloud images from bottom to top with a time resolution of 30s, thereby capturing the actual situation of clouds in small scenes (Zhu Tingting. Research on ultra-short-term prediction of direct solar irradiance based on ground-based cloud images [D]. Southeast University, 2019). Existing studies based on ground-based cloud images mainly focus on using image processing technology to determine the distribution of clouds in the sky and predict the movement trend of clouds in the short term (Marquez R, Coimbra CF M. Intra-hour DNI forecasting based on cloud tracking image analysis [J]. Solar Energy, 2013, 91: 327-336). However, these studies use two-dimensional images as the research object, and analyze the shielding of the sun by the distribution of clouds in the image. They lack consideration of the different degrees of absorption of solar radiation caused by the thickness and density of clouds, which leads to differences in the intensity of solar radiation reaching the ground. Therefore, building a three-dimensional cloud model in a three-dimensional scene can further improve the accuracy of the analysis.

[0004] For 3D cloud modeling and visualization, the commonly used methods proposed in the relevant literature are mainly based on simulating the appearance of clouds, such as the cloud modeling method based on cellular automata (Xu Jiangbin, Zhao Jian, Yang Chao, Wu Lingda. Rapid modeling of realistic clouds [J]. Small and Micro Computer Systems, 2010, 31(08): 1590-1594.), and the volumetric cloud rendering method based on noise texture (Tang Yong, Liu Zhixuan, Guo Huiling, Zhao Jing, Zhang Xiaobi. Research on real-time simulation of multi-noise volumetric clouds [J]. Small and Micro Computer Systems, 2019, 40(11): 2461-2465.). Although the above algorithms significantly improve the realism of 3D scenes by rendering clouds, they can only realize clouds with random positions and sizes in 3D space. They lack real cloud map data and 3D geographic scenes, cannot realize user-specified clouds, and cannot reproduce real cloud scenes. Three-dimensional cloud visualization in geographic space can be achieved based on remote sensing images (Zhang Z, Liang X, Yuan C, et al. Modeling Cumulus Cloud Scenes from High-resolution Satellite Images[J]. Computer Graphics Forum, 2017, 36(7): 229-238.), but the temporal and spatial resolution of remote sensing images is low and cannot meet the actual needs of quasi-real-time monitoring of clouds in photovoltaic power generation application scenarios. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to propose a GPU-based three-dimensional cloud scene modeling and visualization method for ground-based cloud images. By applying the method of the present invention to ground-based cloud images, three-dimensional cumulus modeling parameters can be more accurately calculated based on the cloud-like areas in the ground-based cloud images, and combined with the geographical location range of the ground-based cloud image, three-dimensional cumulus modeling with voxels as the smallest geometric unit can be quickly achieved in the three-dimensional geographic scene, so that the impact of clouds on solar radiation can be intuitively studied.

[0006] The technical solution of the present invention includes the following process:

[0007] First, obtain the shooting time of the ground-based cloud image, denoted as time; obtain the geographical location of the ground-based cloud image imaging device, denoted as O; correct the ground-based cloud image to a square image of 400×400 pixels; define the image coordinate system and the cloud cluster coordinate system; initialize the vertex array pointArray.

[0008] First, the following cloud base height and cloud thickness calculations are performed based on the CPU.

[0009] Step a1, match the Landsat8 image at the same time point according to the shooting time time of the ground-based cloud image, and determine the actual longitude and latitude range covered by the corrected ground-based cloud image according to the longitude and latitude information of the Landsat8 image. The range is a square area with O as the center point, and the area boundary is determined by the maximum longitude maxLon, the minimum longitude minLon, the maximum latitude maxLat, and the minimum latitude minLat, thereby establishing an image coordinate system; then establish a cloud cluster coordinate system with (minLon, minLat) as the origin, and establish a functional relationship between the pixel coordinates in the image coordinate system and the longitude and latitude coordinates in the geographic coordinate system.

[0010] Step a2, using the adaptive binarization method of OpenCV image detection to separate the cloud and sky parts from the ground-based cloud image, and extract the cloud part. Then use the contour detection algorithm of OpenCV image detection to identify the outer contour of the cloud, and determine the number of cloud clusters according to the number of outer contours, which is recorded as N;

[0011] Step a3: For the N cloud clusters extracted from the ground-based cloud image, the CPU calculates the cloud base height CBH and the cloud cluster thickness information T represented by each cloud pixel for each cloud cluster.

[0012] Step a4: The CPU calculates the side length of the cube voxel, denoted as L voxel The side length is determined by the area side length of the ground-based cloud map in WGS84 coordinates and the area side length in the image coordinate system.

[0013] Step a5: CPU calculates the length of the voxel according to its side length L. voxel , cloud thickness information T, cloud base height CBH calculate the spatial position of each voxel center point in the cloud coordinate system, and store it in the vertex array pointArray. After completion, the vertex array pointArray is passed to the GPU.

[0014] The vertex array pointArray is transferred from the CPU to the GPU for drawing cumulus voxels. The GPU drawing process includes the following steps:

[0015] Step b1, use OpenGL to create a FBO and a 2D texture densityTexture2D, and bind the 2D texture to the FBO. Pass the vertex array into the vertex shader of the GPU programmable rendering pipeline, and transform it from the cloud coordinate system to the standard texture coordinate system through matrix transformation. The vertex array in the texture coordinate system is recorded as NPCPointArray;

[0016] Step b2, pass NPCPointArray to the geometry shader of the GPU, define the vertex index array in the geometry shader, record it as vIndexArray; define the voxel vertex array with the voxel center point as the origin, record it as pointArray. Use the vertex index drawing method to draw the cube voxel.

[0017] Step b3, the two-dimensional texture densityTexture2D is passed to the fragment shader of the GPU, the distance between each vertex and the center point of the cloud in the cloud coordinate system is calculated, the density value of each voxel is calculated based on the distance, and the result is stored in densityTexture2D.

[0018] Step b4, in the fragment shader of the GPU, set the transparency of each voxel color according to the density value ρ, transparency alpha = ρ, each voxel color color.rgba = (0.8, 0.8, 0.8, ρ). After completion, it is output from the GPU to realize voxel-based three-dimensional cumulus cloud modeling and visualization rendering.

[0019] Moreover, in step a1, the definitions of the image coordinate system and the cloud coordinate system are as follows:

[0020] The image coordinate system is a two-dimensional coordinate system with the upper left corner of the foundation cloud image as the origin, the horizontal rightward direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis; the image range is determined by (0,0), (0,v), (u,v), (u,0), u and v are the maximum image ranges of the foundation cloud image on the x-axis and y-axis, respectively. In the present invention, u and v are both 400 pixels.

[0021] The cloud cluster coordinate system is defined as a three-dimensional coordinate system with (minX, minY) as the origin, the x-axis is parallel to the ground and horizontally to the right, the y-axis is perpendicular to the x-axis on the ground, and the z-axis is vertically upward. Here, maxX and maxY are the maximum longitude maxLon and maximum latitude maxLat of the ground-based cloud map range, respectively, obtained through coordinate conversion to obtain the WGS84 world coordinates.

[0022] Furthermore, in step a1, the functional relationship between the pixel coordinates in the image coordinate system and the latitude and longitude coordinates in the geographic coordinate system is defined as follows:

[0023] Assume that the longitude and latitude position of any pixel point (x, y) in the ground-based cloud map in the geographic coordinate system is (lat, lon),

[0024]

[0025] Among them, maxLon, minLon, maxLat, minLat are respectively the maximum longitude, minimum longitude, maximum latitude, and minimum latitude of the ground-based cloud map area. u and v are respectively the maximum length and maximum width of the ground-based cloud map in the image coordinate system. In the present invention, u and v are both 400 pixels.

[0026] Furthermore, in step a3, the cloud base height CBH is calculated as follows:

[0027] ① Get the coordinates of the center of the cloud cluster in the ground-based cloud image, recorded as C(x,y); convert the image coordinates into longitude and latitude coordinates through the functional relationship, recorded as worldC(lat,lon); Get the coordinates of the center of the ground-based cloud image, recorded as O′(x0,y0); convert the image coordinates into longitude and latitude coordinates through the functional relationship, recorded as worldO′(lat0,lon0). Calculate the distance between worldC and worldO' according to the Haversine formula, recorded as d0.

[0028] ② Calculate the azimuth angle az of the cloud according to the formula: Calculate the cloud's altitude angle alt according to the formula: Where D is the distance between O′ and C in the ground-based cloud map.

[0029] ③ Calculate the base height CBH of the cloud according to the altitude angle alt and d0: CBH = d0 × tan (alt)

[0030] Furthermore, in step a3, the thickness value represented by each pixel in a single cloud object is calculated as follows:

[0031] ① Construct a bounding box for the cloud in the image, and obtain the width of the bounding box in the image coordinate system, denoted as w, which is converted to the length in the WGS84 coordinate system through coordinate transformation, denoted as w world .

[0032] ② Use image detection to process OpenCV's RGB image to HSL image algorithm to convert the RGB format ground cloud map into an HSL format image, and extract the value of the L channel, recorded as l.

[0033] ③ Calculate the thickness of the cloud represented by each pixel, and record the thickness of the cloud as T.

[0034] Furthermore, in step a5, the position of the voxel center point in the cloud coordinate system is calculated as follows:

[0035] Read the cloud pixel value of the current cloud object, and set the image coordinate of the i-th pixel to P i (u i ,v i)i∈[1,M], M represents the total number of pixels in the cloud. i Convert from image coordinates to cloud coordinates, denoted as worldP i (x i ,y i ), according to the cloud base height CBH and voxel side length L voxel , calculate the actual number of voxels num represented by the i-th pixel i and the elevation h of each voxel center point in the cloud coordinate system k :

[0036]

[0037] h k =CBH+k×L voxel (0≤k≤num i )

[0038] The coordinates of the kth sampling point represented by the i-th pixel in the ground-based cloud image in the cloud coordinate system are marked as P world (x i ,y i ,h k ).

[0039] Moreover, in step b2, the vertex index array and vertex coordinates are defined as follows,

[0040] The vertex index array is defined as vIndexArray={0,1,2,3,2,3,4,5,4,5,6,7,6,0,4,2,1,7,3,5,0,6,1,7}; there are 8 vertex coordinates of the cube voxel centered on the coordinates in the vertex array pointArray. The present invention uses the vertex index drawing method of OpenGL to draw voxels, and the vertex array is defined as vec3Array=(-L,L,-L,L,L,-L,-L,L,L,L,L,-L,-L,-L,L,-L,-L,-L,-L,-L,-L,-L,-L).

[0041] Furthermore, in step b3, the voxel density value is calculated and stored as follows:

[0042] In the cloud coordinate system, obtain the coordinates of the center of the cloud, denoted as centerP(x P ,y P ,z P ), obtain the coordinates of the voxel center point voxel by voxel, calculate the straight-line distance from the voxel center point to centerP, recorded as d, according to the distance d and the maximum thickness of the cloud w world Calculate the cloud density of this voxel, denoted as ρ: Then, the texture coordinates (u, v) of the voxel center are stored in the texture space of the two-dimensional texture densityTexture2D, denoted as (u, v, ρ).

[0043] The present invention uses the cloud base height and cloud thickness extracted from the ground-based cloud map as modeling parameters, and uses voxels as the smallest geometric unit of cumulus clouds, to achieve rapid modeling of three-dimensional cumulus clouds and storage of cumulus cloud density parameters based on GPU. This method not only effectively solves the problems of complex modeling and long modeling time caused by the irregular and complex shapes of clouds, and achieves quasi-real-time modeling of three-dimensional cloud scenes in a specified area, but also stores cloud density information in voxels, providing a model basis for further research on the weakening effect of clouds on solar radiation intensity. In addition, based on the GPU visualization of the three-dimensional cumulus cloud model, it is possible to achieve smooth roaming and other human-computer interactions in the three-dimensional geographic scene, thereby meeting the needs of building a stable and smooth three-dimensional geographic system. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of an image coordinate system according to an embodiment of the present invention.

[0045] Figure 2 Schematic diagram of a cloud coordinate system according to an embodiment of the present invention.

[0046] Figure 3 Schematic diagram of a voxel coordinate system according to an embodiment of the present invention.

[0047] Figure 4 is a flow chart of an embodiment of the present invention.

[0048] Figure 5 This is a visualization effect diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clear, the real-time method of the present invention will be further described in detail below with reference to the accompanying drawings.

[0050] Related terms:

[0051] GPU Graphics Processing Unit

[0052] CPU Central Processing Unit

[0053] OpenCV is an open source, cross-platform computer vision and machine learning software library

[0054] OpenGL is the most widely accepted 2D / 3D graphics API in the industry.

[0055] WGS84 World Geodetic System 1984, a coordinate system

[0056] FBO Framebuffer Object

[0057] The embodiment adopts the method of the present invention, adopts OpenGL and GLSL shading languages, and implements three-dimensional cumulus cloud modeling based on ground-based cloud maps and visualization based on three-dimensional geographic scenes based on OpenCV and osgEarth three-dimensional rendering engines.

[0058] A three-dimensional cumulus cloud modeling and visualization method based on ground-based cloud images provided by an embodiment of the present invention includes the following processes:

[0059] First, obtain the shooting time of the ground-based cloud image, denoted as time; obtain the geographical location of the ground-based cloud image imaging device, denoted as O; correct the ground-based cloud image to a square image of 400×400 pixels; define the image coordinate system and the cloud cluster coordinate system; initialize the vertex array pointArray.

[0060] Assume that the current time is t, obtain the ground-based cloud image at that time, and obtain the geographical location of the device according to the ground-based imaging device parameters of the ground-based cloud image, denoted as O; use the spherical perspective projection image correction method of OpenCV to correct the ground-based cloud image to an image of 400×400 pixels; define the image coordinate system and the cloud coordinate system; initialize the vertex array pointArray used to store the voxel center points in the cloud coordinate system.

[0061] First, the following cloud base height and cloud thickness parameters of three-dimensional cumulus cloud modeling are calculated and processed based on the CPU.

[0062] Step a1, match the Landsat8 image at the same time point according to the shooting time time of the ground-based cloud map, and determine the actual longitude and latitude range covered by the corrected ground-based cloud map according to the longitude and latitude information of the Landsat8 image. The range is a square area with O as the center point, and the area boundary is determined by the maximum longitude maxLon, the minimum longitude minLon, the maximum latitude maxLat, and the minimum latitude minLat; establish a cloud cluster coordinate system with (minLon, minLat) as the origin, and establish a functional relationship between the pixel coordinates in the image coordinate system and the longitude and latitude coordinates in the geographic coordinate system.

[0063] Step a2, separating the cloud and sky parts from the ground-based cloud image, extracting the cloud part, and then using the contour detection algorithm to identify the outer contour of the cloud, and determining the number of cloud clusters according to the number of outer contours, denoted as N;

[0064] Step a3: for the N cloud clusters extracted from the ground-based cloud image, the CPU calculates the cloud base height and the cloud thickness information represented by each cloud pixel for each cloud cluster;

[0065] Step a4, calculate the side length of the cube voxel, denoted as L voxelThe side length is determined by the area side length of the ground-based cloud map in the WGS84 coordinate system and the area side length in the image coordinate system;

[0066] Step a5, calculating the spatial position of each voxel center point in the cloud coordinate system according to the voxel side length, cloud thickness, and cloud base height, and storing it in the vertex array pointArray;

[0067] Then the vertex array pointArray is transferred from the CPU to the GPU for drawing the cumulus voxels;

[0068] Step b1, create a frame buffer object FBO and a two-dimensional texture densityTexture2D, bind the two-dimensional texture to the FBO, pass the vertex array pointArray into the vertex shader of the GPU programmable rendering pipeline, and transform it from the cloud coordinate system to the standard texture coordinate system through matrix transformation. The vertex array in the texture coordinate system is recorded as NPCPointArray;

[0069] Step b2, pass NPCPointArray to the geometry shader of the GPU, define the vertex index array in the geometry shader, record it as vIndexArray; define the voxel vertex array with the voxel center point as the origin, record it as pointArray. Use the vertex index drawing method to draw the cube voxel;

[0070] Step b3, the two-dimensional texture densityTexture2D is passed to the fragment shader of the GPU, the distance between each vertex and the center point of the cloud in the cloud coordinate system is calculated, the density value of each voxel is calculated based on the distance, and the result is stored in densityTexture2D;

[0071] Step b4, in the fragment shader of the GPU, set the transparency of each voxel color according to the density value ρ, transparency alpha = ρ, each voxel color color.rgba = (0.8, 0.8, 0.8, ρ). After completion, it is output from the GPU to realize voxel-based three-dimensional cumulus cloud modeling and visualization rendering.

[0072] In specific implementation, computer software can be used to realize automatic process operation, providing the above three-dimensional cumulus cloud modeling method based on ground-based cloud images.

[0073] For the sake of reference implementation, the specific process design of the embodiment is provided as follows, see Figure 4 :

[0074] Step 1, let the current time be t, and use OpenCV to read the ground-based cloud map at that time. According to the ground-based imaging device parameters of the ground-based cloud map, obtain the longitude and latitude information of the device, denoted as O; use the spherical perspective projection image correction method of OpenCV to correct the ground-based cloud map to a 400×400 pixel image. Initialize the OpenGL drawing environment, define the image coordinate system and the cloud coordinate system; initialize the vertex array pointArray. According to the shooting time time of the ground-based cloud map, match the Landsat8 image at the same time point, determine the actual longitude and latitude range covered by the corrected ground-based cloud map according to the longitude and latitude information of the Landsat8 image, and define the cloud coordinate system and the image coordinate system;

[0075] The initialization step may be performed by the CPU.

[0076] The present invention defines an image coordinate system (such as Figure 1 ) and the cloud coordinate system (as shown Figure 2 shown):

[0077] The image coordinate system is a two-dimensional coordinate system with the upper left corner of the ground-based cloud image as the origin, the horizontal rightward direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis;

[0078] The cloud cluster coordinate system is defined as a three-dimensional coordinate system with (minX, minY) as the origin, the x-axis is parallel to the ground and horizontally to the right, the y-axis is perpendicular to the x-axis on the ground, and the z-axis is vertically upward. Here, maxX and maxY are the maximum longitude maxLon and maximum latitude maxLat of the ground-based cloud map range, respectively, obtained through coordinate conversion to obtain the WGS84 world coordinates.

[0079] In the embodiment, the ground cloud image at time t is read, and the ground cloud image is corrected by the spherical perspective projection image method of OpenCV. An image coordinate system is constructed for the corrected image, and the image range is determined by (0,0), (0,v), (u,v), (u,0), where u and v are the maximum image ranges of the ground cloud image on the x-axis and y-axis, respectively. In this embodiment, u and v are both 400 pixels.

[0080] The position of the ground-based imaging device is O (lon, lat). The actual range of the ground-based cloud map is a square area with O as the center point. The area boundary is determined by the maximum longitude maxLon, the minimum longitude minLon, the maximum latitude maxLat, and the minimum latitude minLat. The cloud cluster coordinate system is established with (minLon, minLat) as the origin.

[0081] In a specific implementation, the location information of the ground-based cloud map can be determined according to actual conditions.

[0082] Step 2: Calculate the cloud base height and cloud thickness parameters of each cloud cluster based on the calibrated ground-based cloud image at time t.

[0083] The present invention proposes:

[0084] First, the functional relationship between the pixel coordinates in the image coordinate system and the longitude and latitude coordinates in the geographic coordinate system is established. The longitude and latitude position of any pixel point (x, y) in the ground-based cloud map in the geographic coordinate system is assumed to be (lat, lon).

[0085]

[0086] Among them, maxLon, minLon, maxLat, minLat are respectively the maximum longitude, minimum longitude, maximum latitude, and minimum latitude of the ground-based cloud map area. u and v are respectively the maximum length and maximum width of the ground-based cloud map in the image coordinate system. In the present invention, u and v are both 400 pixels.

[0087] For the cloud base height CBH, the coordinates of the center point of the cloud cluster and the center point of the ground-based cloud image in the image coordinate system are obtained respectively, and converted into the corresponding longitude and latitude coordinates through the functional relationship, and the distance between the two points is calculated according to the Haversine formula. Then the azimuth of the cloud cluster is calculated, and then the altitude angle of the cloud cluster is calculated, and finally the base height of the cloud cluster is calculated based on the altitude angle and distance.

[0088] For the thickness of the cloud represented by each pixel in a single cloud object, a bounding box is first constructed for the cloud to obtain the width of the bounding box in the image coordinate system, denoted as w, which is then converted to the length in the WGS84 coordinate system through coordinate transformation, denoted as w world Then, we use the image detection and processing OpenCV RGB image to HSL image algorithm to convert the RGB ground cloud image into an HSL image, extract the value of the L channel, and record it as l. Calculate the cloud thickness represented by each pixel, and record the cloud thickness as T.

[0089] The specific implementation in the embodiment is as follows:

[0090] ① The coordinates of the center point of the ground-based cloud image are recorded as O′(x0,y0). The image coordinates are converted into longitude and latitude coordinates through the functional relationship, recorded as worldO′(lat0,lon0). All cumulus cloud areas are extracted based on the ground-based cloud image, and the number is recorded as N; let i=0.

[0091] ②The coordinates of the center point of the ith cloud in the image coordinate system are C i (x i ,y i ); get the longitude and latitude coordinates worldC through coordinate conversion i (lati ,lon i ). Calculate worldC according to the Haversine formula i The distance between and worldO′ is denoted as d i Calculate the azimuth of the cloud using the formula Calculate the altitude angle of the cloud according to the formula Where D i O′ and C i The distance in the cloud coordinate system. Finally, according to the altitude angle alt i and d i Calculate the base height CBH of the cloud i =d i ×tan(alt i ).

[0092] ③ Construct a bounding box for the ith cloud cluster, and let the maximum value of the bounding box on the x-axis of the image coordinate system be x max , the minimum value on the x-axis is x min , calculate the bounding box width w = x max -x min , through coordinate transformation, we get the length w of w in the world coordinate system world The RGB image to HSL image algorithm of OpenCV converts the RGB ground cloud image into an HSL image.

[0093] ④ Use OpenCV to read all pixel coordinates and pixel values ​​of the ith cloud cluster. Assume that the ith cloud cluster in the ground-based cloud map contains M pixels. Let j = 1. When j ≤ M, extract the value of the L channel of the pixel in the HSL format image, denoted as l j ,according to Calculate the thickness of the cloud represented by the pixel, recorded as T j .

[0094] ⑤ Let j = j + 1; if j ≤ M, repeat ④; if j> M, the cloud thickness values ​​represented by all pixels of the cloud are calculated.

[0095] ⑥ Let i = i + 1; if i ≤ N, repeat ②-⑤; if i>N, the cloud base height and cloud thickness of N cloud clusters are calculated.

[0096] During specific implementation, the number of clouds, cloud height and cloud thickness information can be determined based on actual conditions.

[0097] Step 3: The CPU calculates the side length of the cube voxel. The side length is determined by the area side length of the ground-based cloud map in the WGS84 coordinate system and the area side length in the image coordinate system, which is recorded as L voxel : According to the voxel side length Lvoxel , cloud thickness information T, cloud base height CBH calculate the coordinates of each voxel center point in the cloud coordinate system, and store all vertex coordinates in the vertex array.

[0098] The present invention proposes that the method for calculating the position of the voxel center point in the cloud coordinate system is as follows:

[0099] Read the cloud pixel value of the current cloud object, and set the image coordinate of the i-th pixel to P i (u i ,v i )i∈[1,M], M represents the total number of pixels in the cloud. i Convert from image coordinates to cloud coordinates, denoted as worldP i (x i ,y i ), according to the cloud base height CBH and voxel side length L voxel , calculate the actual number of voxels num represented by the i-th pixel i and the elevation h of each voxel center point in the cloud coordinate system k :

[0100]

[0101] h k =CBH+k×L voxel (0≤k≤num i )

[0102] The coordinates of the kth sampling point represented by the i-th pixel in the ground-based cloud image in the cloud coordinate system are marked as P world (x i ,y i ,h k ).

[0103] The specific implementation in the embodiment is as follows:

[0104] According to the maximum longitude maxLon and minimum longitude minLon of the area, calculate maxX and minX in the WGS84 coordinate system, u = 400, The vertex array pointArray is initialized by the GPU.

[0105] For the N cumulus cloud areas in the ground-based cloud image, let i = 1. When i ≤ N, the following calculation is performed for the i-th cloud cluster:

[0106] ① Assume that the i-th cloud cluster in the ground-based cloud image contains M pixels, and let j = 1.

[0107] ②According to the cloud thickness T j and voxel side length L voxelCalculate the number of voxels contained in the jth pixel in the cloud coordinate system, denoted as num j :

[0108] Let k = 0;

[0109] ③ Calculate the coordinate of the kth voxel on the z-axis in the cloud coordinate system, denoted as h k :h k =CBH j +k×L voxel That is, the coordinate of the kth sampling point represented by the jth pixel in the cloud coordinate system is marked as P world (x j ,y j ,h k ) and stored in the vertex array pointArray.

[0110] ④k=k+1, if k≤num j , repeat ③, otherwise go to ⑤.

[0111] j=j+1, if j≤M, repeat ②-④, otherwise the coordinates of all voxel center points of the i-th cloud are calculated.

[0112] After completing the above operations for all N cloud clusters, the vertex array pointArray of the center point position information of all voxels ends storage.

[0113] During specific implementation, the position of the voxel center point in the cloud coordinate system and the vertex array can be determined according to the actual cloud thickness.

[0114] Step 4: The vertex array is passed from the CPU to the vertex shader of the GPU, which converts the vertex array into texture coordinates in the texture coordinate system through matrix transformation. The texture coordinates are passed to the geometry shader of the GPU, and the vertex index is used in the geometry shader to draw voxels.

[0115] The present invention proposes to use OpenGL to create an FBO and a two-dimensional texture densityTexture2D, and the two-dimensional texture is bound to the FBO. The vertex array is passed into the vertex shader of the GPU programmable rendering pipeline, and converted from the cloud coordinate system to the standard texture coordinate system through matrix transformation. The vertex array in the texture coordinate system is recorded as NPCPointArray. The NPCPointArray is passed into the geometry shader of the GPU, and the vertex index array is defined in the geometry shader, recorded as vIndexArray; a voxel vertex array with the voxel center point as the origin is defined, recorded as vec3Array, and a vertex index drawing method is used to draw cube voxels.

[0116] For the vertex index array, it is defined as vIndexArray = {0,1,2,3,2,3,4,5,4,5,6,7,6,0,4,2,1,7,3,5,0,6,1,7};

[0117] For vertex arrays, such as Figure 3 As shown, a voxel coordinate system is constructed with the coordinates in the vertex array pointArray as the center to determine the coordinate positions of the eight vertices of the cube voxel. The vertex array is defined as vec3Array = (-L, L, -L, L, L, -L, -L, L, L, L, L, -L, -L, -L, -L, -L, -L, -L, -L, -L).

[0118] The specific implementation in the embodiment is as follows:

[0119] ① Pass the vertex array pointArray into the vertex shader of the GPU, and OpenGL obtains the built-in matrix variable gl_ModelViewProjectionMatrix of the shader. The vertex array is multiplied by the matrix to convert it to the standard texture coordinate array under the texture coordinate system, recorded as gl_Position.

[0120] ② Pass gl_Position to the geometry shader of the GPU, define the vertex index array vIndexArray = {0,1,2,3,2,3,4,5,4,5,6,7,6,0,4,2,1,7,3,5,0,6,1,7}; define the vertex array vec3Array = (-L,L,-L,L,L,-L,-L,L,L,L,L,-L,-L,-L,-L,-L,-L,-L,-L,-L,-L), where Calculate the inverse matrix of gl_ModelViewProjectionMatrix, recorded as inverseMat, and multiply gl_Position by inverseMat to get the coordinates in the cloud coordinate system, recorded as vec3Array. According to the vertex index array vIndexArray, index the corresponding vertices in the vertex array vec3Array in sequence until the index ends, and the voxel drawing is completed.

[0121] During specific implementation, the voxels of the cloud can be drawn according to actual conditions.

[0122] Step 5, calculate the density value of each voxel in the fragment shader and store it in a 2D texture. Set the voxel color transparency based on the density value.

[0123] The present invention proposes that in the fragment shader of the GPU, the coordinates of the center point of the cloud in the cloud coordinate system are calculated according to the texture coordinate array gl_Position of the cloud, and are recorded as centerP(x P ,y P ,z P ), and then calculate the straight-line distance from the center point of each voxel in the cloud coordinate system to centerP, recorded as d, according to the distance d and the maximum thickness of the cloud w world Calculate the cloud density of this voxel, denoted as ρ: Then, the texture coordinates (u, v) at the center of the voxel are stored in the two-dimensional texture densityTexture2D, recorded as (u, v, ρ). The color transparency of each voxel is set according to ρ, with transparency alpha = ρ, and the color of each voxel color.rgba = (0.8, 0.8, 0.8, ρ). After completion, it is output from the GPU to achieve voxel-based three-dimensional cumulus cloud modeling and visualization rendering.

[0124] The specific implementation in the embodiment is as follows:

[0125] In the fragment shader of the GPU, the coordinates of the center point of the cloud in the cloud coordinate system are calculated according to the texture coordinate array gl_Position of the cloud, which is denoted as centerP(x P ,y P ,z P ). Let the number of voxels be N, and let i = 1,

[0126] The coordinates of the center point of the i-th voxel in the cloud coordinate system are P i (x i ,y i ,z i ), calculate P according to the straight line formula between two points i The straight-line distance d from centerP i . Then according to the distance d i and the maximum thickness of the cloud w world Calculate the cloud density ρ of the voxel i : According to the texture coordinates of the voxel center (u i ,v i ), storing the cloud density in the 2D texture densityTexture2D (u i ,v i ) at which the value is ρ i According to ρ i Set the color of the voxel texture, color.rgb = (0.8, 0.8, 0.8, ρ i ).

[0127] Let i = i + 1; if i ≤ N, repeat the above operation. If i > N, end the calculation of cloud density and cloud color. Output from GPU to realize voxel-based 3D cloud scene visualization. When the visualization simulation ends, release FBO. The visualization effect in 3D geographic scene is as follows: Figure 5 shown.

[0128] Through the above specific implementation, it can be seen that based on the cloud base height and cloud thickness extracted from the ground-based cloud map as modeling parameters, voxels are used as the smallest geometric unit of cumulus clouds, and the fast modeling of 3D cumulus clouds and the storage of cumulus cloud density parameters are realized in the GPU. Quasi-real-time 3D cloud scene modeling in the specified area is achieved, and good system fluency is also shown in the roaming browsing of the 3D scene.

[0129] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A three-dimensional cloud scene modeling and visualization method based on ground-based cloud images, characterized in that: The process includes: First, obtain the shooting time of the ground-based cloud image, recorded as time; obtain the geographical location of the ground-based cloud image imaging device, recorded as O; correct the ground-based cloud image to a square image, define the image coordinate system and the cloud coordinate system, and initialize the vertex array pointArray used to store the voxel center points in the cloud coordinate system; First, the following cloud base height and cloud thickness parameters of three-dimensional cumulus cloud modeling are calculated and processed based on the CPU; Step a1, match the Landsat8 image at the same time point according to the shooting time time of the ground-based cloud image, and determine the actual longitude and latitude range covered by the corrected ground-based cloud image according to the longitude and latitude information of the Landsat8 image. The range is a square area with O as the center point, and the area boundary is determined by the maximum longitude maxLon, the minimum longitude minLon, the maximum latitude maxLat, and the minimum latitude minLat, and then establish an image coordinate system; then establish a cloud cluster coordinate system with (minLon, minLat) as the origin, and establish a functional relationship between the pixel coordinates in the image coordinate system and the longitude and latitude coordinates in the geographic coordinate system; Step a2, separating the cloud and sky parts from the ground-based cloud image, extracting the cloud part, and then using the contour detection algorithm to identify the outer contour of the cloud, and determining the number of cloud clusters according to the number of outer contours, denoted as N; Step a3: for the N cloud clusters extracted from the ground-based cloud image, the CPU calculates the cloud base height and the cloud thickness information represented by each cloud pixel for each cloud cluster; Step a4, calculate the side length of the cube voxel, denoted as L voxel The side length is determined by the area side length of the ground-based cloud map in the WGS84 coordinate system and the area side length in the image coordinate system; Step a5, calculating the spatial position of each voxel center point in the cloud coordinate system according to the voxel side length, cloud thickness, and cloud base height, and storing it in the vertex array pointArray; Then the vertex array pointArray is transferred from the CPU to the GPU for drawing the cumulus voxels; Step b1, create a frame buffer object FBO and a two-dimensional texture densityTexture2D, bind the two-dimensional texture to the FBO, pass the vertex array pointArray into the vertex shader of the GPU programmable rendering pipeline, and transform it from the cloud coordinate system to the standard texture coordinate system through matrix transformation. The vertex array in the texture coordinate system is recorded as NPCPointArray; Step b2, passing NPCPointArray into the geometry shader of the GPU, defining a vertex index array in the geometry shader, recorded as vIndexArray; defining a voxel vertex array with the voxel center point as the origin, recorded as pointArray, and drawing the cube voxel using the vertex index drawing method; Step b3, the two-dimensional texture densityTexture2D is passed to the fragment shader of the GPU, the distance between each vertex and the center point of the cloud in the cloud coordinate system is calculated, the density value of each voxel is calculated based on the distance, and the result is stored in densityTexture2D; Step b4, in the fragment shader of the GPU, the transparency of each voxel color is set according to the density value ρ, and the color value of each voxel is set. After completion, it is output from the GPU to realize voxel-based three-dimensional cumulus cloud modeling and visualization rendering.

2. A three-dimensional cloud scene modeling and visualization method based on ground-based cloud images as claimed in claim 1, characterized in that: In step a1, the definitions of the image coordinate system and the cloud coordinate system are as follows: The image coordinate system is a two-dimensional coordinate system with the upper left corner of the ground-based cloud image as the origin, the horizontal rightward direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis; the image range is determined by (0,0), (0,v), (u,v), (u,0), where u and v are the maximum image ranges of the ground-based cloud image on the x-axis and y-axis respectively; The cloud cluster coordinate system is defined as a three-dimensional coordinate system with (minX, minY) as the origin, the positive x-axis direction parallel to the ground and horizontally to the right, the y-axis direction perpendicular to the x-axis on the ground, and the positive z-axis direction vertically upward; here maxX and maxY are the maximum longitude maxLon and maximum latitude maxLat of the ground-based cloud map range, respectively, obtained by coordinate conversion to the WGS84 world coordinates.

3. The three-dimensional cloud scene modeling and visualization method based on ground-based cloud images as claimed in claim 1, characterized in that: In step a1, the functional relationship between the pixel coordinates in the image coordinate system and the longitude and latitude coordinates in the geographic coordinate system is defined as follows: Assume that the longitude and latitude position of any pixel point (x, y) in the ground-based cloud map in the geographic coordinate system is (lat, lon). Among them, maxLon, minLon, maxLat, minLat are the maximum longitude, minimum longitude, maximum latitude, and minimum latitude of the ground-based cloud map area, respectively; u and v are the maximum length and maximum width of the ground-based cloud map in the image coordinate system, respectively.

4. The method for modeling and visualizing a three-dimensional cloud scene based on a ground-based cloud map according to claim 1, characterized in that: In step a3, the calculation method for the cloud base height CBH is as follows: ① Get the coordinates of the center of the cloud cluster in the ground-based cloud image, recorded as C(x,y); convert the image coordinates into longitude and latitude coordinates through the functional relationship, recorded as worldC(lat,lon); Get the coordinates of the center of the ground-based cloud image, recorded as O′(x0,y0); convert the image coordinates into longitude and latitude coordinates through the functional relationship, recorded as worldO′(lat0,lon0), and calculate the distance between worldC and worldO′ according to the Haversine formula, recorded as d0; ② Calculate the azimuth angle az of the cloud according to the formula: Calculate the cloud's altitude angle alt according to the formula: Where D is the distance between O′ and C in the ground-based cloud map; ③ Calculate the base height CBH of the cloud cluster based on the altitude angle alt and d0: CBH = d0 × tan (alt).

5. The three-dimensional cloud scene modeling and visualization method based on ground-based cloud images as claimed in claim 1, characterized in that: In step a3, the thickness value represented by each pixel in a single cloud object is calculated as follows: ① Construct a bounding box for the cloud in the image, and obtain the width of the bounding box in the image coordinate system, denoted as w, which is converted to the length in the WGS84 coordinate system through coordinate transformation, denoted as w world ; ②Use image detection to process OpenCV's RGB image to HSL image algorithm to convert the RGB ground cloud image into an HSL image, and extract the value of the L channel, recorded as l; ③ Calculate the thickness of the cloud represented by each pixel, and record the thickness of the cloud as T.

6. The method for modeling and visualizing a three-dimensional cloud scene based on a ground-based cloud map according to claim 1, characterized in that: In step a5, the position of the voxel center point in the cloud coordinate system is calculated as follows: Read the cloud pixel value of the current cloud object, and set the image coordinate of the i-th pixel to P i (u i ,v i )i∈[1,M], M represents the total number of pixels in the cloud; i Convert from image coordinates to cloud coordinates, denoted as worldP i (x i ,y i ), according to the cloud base height CBH and voxel side length L voxel , calculate the actual number of voxels num represented by the i-th pixel i and the elevation h of each voxel center point in the cloud coordinate system k : h k =CBH+k×L voxel (0≤k≤num i ) The coordinates of the kth sampling point represented by the i-th pixel in the ground-based cloud image in the cloud coordinate system are marked as P world (x i ,y i ,h k ).

7. The method for modeling and visualizing a three-dimensional cloud scene based on a ground-based cloud map according to claim 1, characterized in that: In step b2, the vertex index array and vertex coordinates are defined as follows: The vertex index array is defined as vIndexArray = {0,1,2,3,2,3,4,5,4,5,6,7,6,0,4,2,1,7,3,5,0,6,1,7}; the cube voxel with the coordinates in the vertex array pointArray as the center has a total of 8 vertex coordinates, and the voxel is drawn using OpenGL's vertex index drawing method. The vertex array is defined as vec3Array = (-L,L,-L,L,L,-L,-L,L,L,L,L,-L,-L,-L,-L,-L,-L,-L,-L,-L), where 8. The method for modeling and visualizing a three-dimensional cloud scene based on a ground-based cloud map according to claim 1, characterized in that: In step b3, the voxel density value is calculated and stored as follows: In the cloud coordinate system, obtain the coordinates of the center of the cloud, denoted as centerP(x P ,y P ,z P ), obtain the coordinates of the voxel center point voxel by voxel, calculate the straight-line distance from the voxel center point to centerP, recorded as d, according to the distance d and the maximum thickness of the cloud w world Calculate the cloud density of this voxel, denoted as ρ: Then, the texture coordinates (u, v) of the voxel center are stored in the texture space of the two-dimensional texture densityTexture2D, denoted as (u, v, ρ).

9. The method for modeling and visualizing a three-dimensional cloud scene based on a ground-based cloud map according to claim 1, characterized in that: In step b4, the color transparency alpha of each voxel is ρ, and the color of each voxel is color.rgba=(0.8, 0.8, 0.8, ρ).

10. The three-dimensional cloud scene modeling and visualization method based on ground-based cloud images as claimed in claim 1, characterized in that: In step a2, the adaptive binarization method of OpenCV is used to separate the cloud and sky parts from the ground-based cloud image, and the cloud part is extracted; then the contour detection algorithm of OpenCV is used to identify the outer contour of the cloud.

Images