Three-dimensional terrain modeling method, device and equipment and storage medium

Abstract

The present application is suitable for the technical field of image processing, and provides a three-dimensional terrain modeling method, device and equipment and storage medium, the method comprising: acquiring a prompt word, inputting the prompt word into a trained terrain map generation model to obtain a target three-dimensional terrain digital elevation map, wherein the terrain map generation model comprises a diffusion model and a quantization decoder connected in sequence, the quantization decoder is used for decoding the RGB image output by the diffusion model into the target three-dimensional terrain digital elevation map, and a target three-dimensional terrain map is generated according to the target three-dimensional terrain digital elevation map, so that three-dimensional terrain modeling can be adaptively performed by acquiring the prompt word, and the present application is simple to use and only needs to input ordinary natural language.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a three-dimensional terrain modeling method, device, equipment and storage medium. BACKGROUND

[0002] In recent years, BIM (Building Information Modeling), virtual reality, digital twin and many other fields often need to involve large-scale terrain scene display, and tend to be fine, realistic, large in volume and diversified. Traditional terrain generation algorithms, such as quadtree terrain generation algorithm, are only suitable for simple terrains. Although GeoMipmap and moving cube algorithms can generate high-precision terrains, they cannot adaptively fine model various terrains such as deserts, grasslands, mountains and hills. Other complex terrain generation algorithms require users to have rich modeling experience and preset some complex parameters, which raises the use threshold and limits application and promotion. In addition, all popular terrain generation algorithms do not support prompt words, cannot use natural language programming to set terrain modeling details, and do not support adaptive three-dimensional terrain local modification. SUMMARY

[0003] The present application aims to provide a three-dimensional terrain modeling method, device, equipment and storage medium, and aims to solve the problem that the prior art cannot adaptively fine model.

[0004] In one aspect, the present application provides a three-dimensional terrain modeling method, which comprises the following steps:

[0005] obtaining a prompt word;

[0006] inputting the prompt word into a trained terrain map generation model to obtain a target three-dimensional terrain digital elevation map, wherein the terrain map generation model comprises a diffusion model and a quantization decoder connected in sequence, and the quantization decoder is used to decode an RGB image output by the diffusion model into the target three-dimensional terrain digital elevation map;

[0007] generating a target three-dimensional terrain map according to the target three-dimensional terrain digital elevation map.

[0008] Optionally, the training step of the terrain map generation model comprises:

[0009] obtaining an initial training data set, each initial sample pair in the initial training data set comprising a three-dimensional terrain digital elevation map and a corresponding prompt word;

[0010] encoding the three-dimensional terrain digital elevation map in each group of initial sample pairs into an RGB image to obtain a target training data set;

[0011] training the terrain map generation model using the target training data set to obtain the trained terrain map generation model.

[0012] Optionally, the diffusion model is a stable diffusion-based diffusion model.

[0013] Optionally, the loss function used in the training of the terrain map generation model includes a regularization loss term and a gradient loss term, the regularization loss term is used to measure the similarity between the three-dimensional terrain digital elevation map generated by the terrain map generation model and the real three-dimensional terrain digital elevation map, and the gradient loss term is used to measure the height change degree of the terrain structure.

[0014] Optionally, the regularization loss term is calculated by the following formula:

[0015] ;

[0016] wherein, denotes the regularization loss value, denotes the three-dimensional terrain digital elevation map generated by the terrain map generation model, denotes the real three-dimensional terrain digital elevation map.

[0017] Optionally, the gradient loss term is calculated by the following formula:

[0018] ;

[0019] ;

[0020] ;

[0021] wherein, denotes the gradient loss value, denotes the gradient value corresponding to pixel [i, j] in the generated RGB image, denotes the gradient value corresponding to pixel [i, j] in the real RGB image, 、 、 respectively denote the elevation values corresponding to pixels [i+1, j], [i, j], [i, j+1] in the generated RGB image, 、 、 respectively denote the elevation values corresponding to pixels [i+1, j], [i, j], [i, j+1] in the real RGB image.

[0022] Optionally, the step of encoding the three-dimensional terrain digital elevation map in each group of initial sample pairs into an RGB image comprises:

[0023] performing first quantization coding on the three-dimensional terrain digital elevation map in the initial sample pair to obtain a first channel data map;

[0024] performing second quantization coding on the three-dimensional terrain digital elevation map in the initial sample pair according to the first channel data map to obtain a second channel data map;

[0025] performing third quantization coding on the three-dimensional terrain digital elevation map in the initial sample pair according to the first channel data map and the second channel data map to obtain a third channel data map;

[0026] obtaining an encoded RGB image according to the first channel data map, the second channel data map and the third channel data map.

[0027] Optionally, a formula used in the first quantization coding is:

[0028] ;

[0029] ;

[0030] ;

[0031] wherein H represents any elevation value of the three-dimensional terrain digital elevation map in the initial sample pair, and R represents an R component in the first channel data map corresponding to the elevation value;

[0032] a formula used in the second quantization coding is:

[0033] ;

[0034] wherein G represents a G component in the second channel data map corresponding to the elevation value;

[0035] a formula used in the third quantization coding is:

[0036] ;

[0037] wherein B represents a B component in the third channel data map corresponding to the elevation value;

[0038] a formula used in decoding by the quantization decoder is:

[0039] ;

[0040] ;

[0041] ;

[0042] wherein R, G, and B respectively represent the R component, the G component, and the B component of any pixel point in the RGB image output by the diffusion model, and H represents the elevation value corresponding to the pixel point in the target three-dimensional terrain digital elevation map.

[0043] Optionally, the step of inputting the prompt into the trained terrain map generation model to obtain the target three-dimensional terrain digital elevation map further comprises:

[0044] inputting the prompt into the trained terrain map generation model to obtain an initial three-dimensional terrain digital elevation map;

[0045] obtaining a region on the initial three-dimensional terrain digital elevation map that needs to be locally modified;

[0046] re-generating the three-dimensional terrain digital elevation map in the region by the trained terrain map generation model to obtain the target three-dimensional terrain digital elevation map.

[0047] Optionally, the method further comprises:

[0048] obtaining a local modification parameter;

[0049] The step of re-generating the three-dimensional terrain digital elevation map in the region by the trained terrain map generation model further comprises:

[0050] re-generating the three-dimensional terrain digital elevation map in the region by the trained terrain map generation model according to the local modification parameter.

[0051] Optionally, before the step of generating a target three-dimensional terrain map according to the target three-dimensional terrain digital elevation map, the method further comprises:

[0052] obtaining a conditional guide map;

[0053] inputting the prompt and the conditional guide map into the trained terrain map generation model to obtain the target three-dimensional terrain digital elevation map, wherein the conditional guide map is used to guide the diffusion model to generate the target three-dimensional terrain digital elevation map.

[0054] In another aspect, the present application provides a three-dimensional terrain modeling device, which comprises:

[0055] a prompt obtaining unit configured to obtain a prompt;

[0056] an elevation map generation unit configured to input the prompt into a trained terrain map generation model to obtain a target three-dimensional terrain digital elevation map, wherein the terrain map generation model comprises a diffusion model and a quantization decoder connected in sequence, and the quantization decoder is configured to decode an RGB image output by the diffusion model into the target three-dimensional terrain digital elevation map.

[0057] a topographic map generation unit configured to generate a target three-dimensional topographic map according to the target three-dimensional topographic digital elevation map.

[0058] In another aspect, the present application also provides a three-dimensional topographic modeling device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method as described above when executing the computer program.

[0059] In another aspect, the present application also provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable by a processor to implement the steps of the method as described above.

[0060] The present application acquires a prompt word, inputs the prompt word into a trained topographic map generation model, and obtains a target three-dimensional topographic digital elevation map, wherein the topographic map generation model comprises a diffusion model and a quantization decoder connected in sequence, the quantization decoder is used for decoding an RGB image output by the diffusion model into the target three-dimensional topographic digital elevation map, and a target three-dimensional topographic map is generated according to the target three-dimensional topographic digital elevation map, so that three-dimensional topographic modeling can be adaptively performed by acquiring the prompt word, and the present application is simple to use, does not require the user to have rich modeling experience, programming knowledge, and complex parameter presetting, and only needs to input ordinary natural language. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 is an implementation flowchart of the three-dimensional topographic modeling method provided by the first embodiment of the present application;

[0062] Figure 2A is an initial three-dimensional topographic digital elevation map generated by the topographic map generation model provided by the first embodiment of the present application;

[0063] Figure 2B is a three-dimensional topographic digital elevation map after a local modification region is labeled on the topographic map provided by the first embodiment of the present application; Figure 2A

[0064] Figure 2C is a three-dimensional topographic digital elevation map after the local modification region shown in the topographic map provided by the first embodiment of the present application is modified; Figure 2B

[0065] Figure 3A is a condition guide map provided by the first embodiment of the present application;

[0066] Figure 3B is a target three-dimensional topographic digital elevation map generated by using the condition guide map provided by the first embodiment of the present application as a condition guide map; Figure 3A

[0067] ​​​Figure 3C is a target three-dimensional terrain map generated based on the three-dimensional terrain modeling method provided by the embodiment one of the present application; Figure 3B

[0068] Figure 4 is a flowchart of the terrain map generation model training method provided by the embodiment two of the present application;

[0069] Figure 5 is a structural schematic diagram of the terrain map generation device provided by the embodiment three of the present application; and

[0070] Figure 6 is a structural schematic diagram of the terrain map generation device provided by the embodiment four of the present application. DETAILED DESCRIPTION

[0071] In order to make the objects, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0072] The specific implementation of the present application is described in detail below in combination with specific embodiments: Embodiment one

[0073] Figure 1 The implementation flow of the three-dimensional terrain modeling method provided by the embodiment one of the present application is shown, only the part related to the embodiment of the present application is shown for the convenience of description, and the details are described as follows.

[0074] In step S101, a prompt word is acquired.

[0075] The embodiment of the present application is applicable to three-dimensional terrain modeling, that is, the surface form can be digitally expressed by the way of three-dimensional modeling, and the three-dimensional terrain can be desert, grassland, mountain, hilly land, etc., which is not limited in the present embodiment. The prompt word can include the type of terrain, for example, desert, grassland, mountain, hilly land, etc., and optionally, the prompt word can also include an adjective related to the type of terrain, for example, “dry desert”, “endless grassland” or “steep mountain” etc., so as to facilitate subsequent setting of the details of the three-dimensional terrain map according to the prompt word.

[0076] In step S102, the prompt word is input to the trained terrain map generation model to obtain a target three-dimensional terrain digital elevation map.

[0077] In the embodiment of the present application, the terrain map generation model includes a diffusion model and a quantization decoder connected in sequence, and the quantization decoder is used to decode the RGB image output by the diffusion model into a target three-dimensional terrain digital elevation map. The implementation flow of the terrain map generation model training method can refer to the description of the embodiment two, and the present embodiment is not described in detail. ​

[0078] In a specific implementation, a diffusion model can be used to output an image corresponding to the prompt word, and the diffusion model can be a DALL-E3 model or the like. Optionally, the diffusion model is a Stable Diffusion model. Taking the stable diffusion diffusion model as an example, a text encoder in the stable diffusion diffusion model can be used to generate an embedding representation of the prompt word with a size of [B, k, E]. Then, noise with a size of [B, C, H / 8, W / 8] is randomly generated. Based on a U-Net model in the stable diffusion diffusion model, a DDPM (Denoising Diffusion Probabilistic Models) algorithm or a DDIM (Denoising Diffusion Implicit Models) algorithm is used for iteration and gradual reverse denoising, and finally the latent variable implicit expression of the image is recovered. Finally, an image decoder is used to decode and generate a final RGB image based on the input implicit expression.

[0079] Optionally, the formula used by the quantization decoder when decoding is as follows:

[0080] ;

[0081] ;

[0082] ;

[0083] wherein R, G, and B respectively represent the R component, the G component, and the B component of any pixel point in the RGB image output by the diffusion model, and H represents an elevation value in the target three-dimensional terrain digital elevation map corresponding to the pixel point. The data range that can be represented by the quantization decoder is very large, which can be from -18446744073709551616 to 18446744073709551616. In addition, the quantization decoder can represent positive numbers and negative numbers. Whether it is a low plain or a high mountain, it can be represented without losing detailed information.

[0084] In view of the fact that the result of the first generation of the three-dimensional terrain digital elevation map by the terrain map generation model may not be satisfactory, and subsequent multiple modifications and edits are required, optionally, the prompt words are input into the trained terrain map generation model to obtain an initial three-dimensional terrain digital elevation map, the area on the initial three-dimensional terrain digital elevation map that needs to be locally modified is obtained, the three-dimensional terrain digital elevation map in the area is regenerated by the trained terrain map generation model, and a target three-dimensional terrain digital elevation map is obtained, thereby realizing local modification of the three-dimensional terrain digital elevation map. In a specific implementation, a user-drawn mask area representing a local area that needs to be modified can be obtained, and the three-dimensional terrain digital elevation map in the area is regenerated based on the terrain map generation model. This process can be repeated until an ideal three-dimensional terrain digital elevation map, i.e., a target three-dimensional terrain digital elevation map, is generated. When the three-dimensional terrain digital elevation map in the area is regenerated based on the terrain map generation model, the terrain part that needs to be modified can be reset to a random number, and then input into the terrain map generation model again, so that a new result can be output on the basis of the original result.

[0085] In the local modification of the three-dimensional terrain digital elevation map, optionally, local modification parameters are obtained, and the three-dimensional terrain digital elevation map in the area is regenerated by the trained terrain map generation model according to the local modification parameters, thereby improving the efficiency of local modification. In a specific implementation, parameters such as a redraw coefficient for local modification can be set, wherein a higher redraw coefficient indicates a larger modification range.

[0086] As an example, as shown in Figures 2A-2C , an initial three-dimensional terrain digital elevation map generated by the terrain map generation model; Figure 2A , a local modification area marked by a black brush tool on the three-dimensional terrain digital elevation map shown in Figure 2B ; and Figure 2A , a three-dimensional terrain digital elevation map obtained after regeneration of the local modification area shown in Figure 2C . Figure 2B , a three-dimensional terrain digital elevation map obtained after regeneration of the local modification area shown in

[0087] In view of the fact that the three-dimensional terrain digital elevation map regenerated or modified by the above method still has the problem of not being able to achieve an ideal accuracy due to its large randomness, optionally, a conditional guide map is obtained, the prompt words and the conditional guide map are input into the trained terrain map generation model, and a target three-dimensional terrain digital elevation map is obtained, thereby guiding the generation process of the target three-dimensional terrain digital elevation map by setting the conditional guide map, effectively controlling the general terrain structure, and realizing intelligent editing and optimization of the three-dimensional terrain. The conditional guide map is used to guide the diffusion model to generate the target three-dimensional terrain digital elevation map.

[0088] Optionally, when the prompt word contains a terrain type and an adjective related to the terrain type, the terrain map generation model generates a target three-dimensional terrain digital elevation map matching the terrain type and the adjective related to the terrain type.

[0089] In step S103, a target three-dimensional terrain map is generated according to the target three-dimensional terrain digital elevation map.

[0090] In the embodiment of the present application, the target three-dimensional terrain digital elevation map can be modeled or rendered, etc., to obtain the target three-dimensional terrain map. In specific implementation, computer graphics software such as Unity, Unreal Engine, etc. can be used to draw the target three-dimensional terrain digital elevation map on the computer screen. In addition, some special rendering techniques such as global illumination, ray tracing, radiosity lighting, etc. can be used to enhance the realism and visual effect of the terrain.

[0091] As an example, Figure 3A A conditional guide map used in the generation process of the target three-dimensional terrain digital elevation map, Figure 3B A target three-dimensional terrain digital elevation map generated by using Figure 3A As a conditional guide map, Figure 3C A target three-dimensional terrain digital elevation map generated by using Figure 3B The three-dimensional terrain digital elevation map shown in the figure is drawn by a rendering program to obtain a target three-dimensional terrain map close to reality.

[0092] In the embodiment of the present application, the prompt word is obtained, the prompt word is input into the trained terrain map generation model, and a target three-dimensional terrain digital elevation map is obtained, wherein the terrain map generation model comprises a diffusion model and a quantization decoder connected in sequence, the quantization decoder is used to decode the RGB image output by the diffusion model into the target three-dimensional terrain digital elevation map, and a target three-dimensional terrain map is generated according to the target three-dimensional terrain digital elevation map, so that three-dimensional terrain modeling can be adaptively performed by obtaining the prompt word, and the reality of three-dimensional terrain modeling is greatly improved. In addition, the present scheme can support adaptive local modification and intelligent editing optimization of three-dimensional terrain, in addition, the present scheme is simple to use, and the user does not need to have any rich modeling experience, programming knowledge and complex parameter presetting, only needs to input ordinary natural language. Embodiment two

[0093] Figure 4 The implementation process of the training method of the terrain map generation model provided in the embodiment two of the present application is shown, only the part related to the embodiment of the present application is shown for the convenience of description, and the details are as follows.

[0094] In step S201, an initial training data set is obtained, and each initial sample pair in the initial training data set includes a three-dimensional terrain digital elevation map and a corresponding prompt word.

[0095] In the embodiment of the present application, the initial training data set can include a three-dimensional terrain digital elevation map data set and a corresponding prompt word data set. When obtaining the three-dimensional terrain digital elevation map data set, various methods such as remote sensing measurement, laser scanning, radar detection, etc. can be used to collect the real three-dimensional terrain structure. After the real three-dimensional terrain structure is collected, the collected terrain data can be processed, such as data cleaning, format conversion, noise reduction, etc. Then, the processed data can be arranged into an initial three-dimensional terrain digital elevation map data set. After the initial three-dimensional terrain digital elevation map data set is arranged, the data set can be further expanded, such as flipping, rotating, cropping, etc. of the three-dimensional terrain digital elevation map, to obtain the final three-dimensional terrain digital elevation map data set.

[0096] After obtaining the three-dimensional terrain digital elevation map data set, the three-dimensional terrain digital elevation map data set can be labeled. The labeling method can include manual labeling and machine labeling. When manually labeling, a prompt word can be manually labeled for a three-dimensional terrain digital elevation map. The prompt word can include specific terrain types, such as desert, grassland, mountain, and hilly land, etc. Further, the prompt word can also include adjectives related to terrain details, such as "dry desert" or "steep mountain" etc. When machine labeling, to further enrich the labeled data, the "picture to text" function of the stable diffusion diffusion model can be used to generate corresponding prompt words for the terrain height map.

[0097] In step S202, the three-dimensional terrain digital elevation map in each initial sample pair is encoded into an RGB image to obtain a target training data set.

[0098] In the embodiment of the present application, to realize the quantization encoding of the three-dimensional terrain digital elevation map to the RGB image, the three-dimensional terrain digital elevation map in the initial sample pair is optionally first quantization encoded to obtain a first channel data graph. The three-dimensional terrain digital elevation map in the initial sample pair is second quantization encoded according to the first channel data graph to obtain a second channel data graph. The three-dimensional terrain digital elevation map in the initial sample pair is third quantization encoded according to the first channel data graph and the second channel data graph to obtain a third channel data graph. The first channel data graph, the second channel data graph, and the third channel data graph are used to obtain the encoded RGB image, i.e. the RGB image after the three-dimensional terrain digital elevation map in the initial sample pair is encoded.

[0099] Optionally, the formula used for the first quantization encoding is:

[0100] ;

[0101] ;

[0102] ;

[0103] wherein H represents any elevation value of the digital elevation map of the three-dimensional terrain in the initial sample pair, R represents the R component in the first channel data map corresponding to the elevation value, represents the floor function;

[0104] The formula used by the second quantization encoding is:

[0105] ;

[0106] wherein G represents the G component in the second channel data map corresponding to the elevation value;

[0107] The formula used by the third quantization encoding is:

[0108] ;

[0109] wherein B represents the B component in the third channel data map corresponding to the elevation value. The data range that can be represented by the above formula for quantization encoding is very large, which can be from -18446744073709551616 to 18446744073709551616, and can also represent positive and negative numbers, and can represent both low plains and high mountains without losing detailed information.

[0110] In step S203, the terrain map generation model is trained using the target training data set to obtain a trained terrain map generation model.

[0111] In the embodiment of the present application, the stable diffusion model is taken as an example, and the stable diffusion model includes a text encoder, an image encoder and a U-Net model. First, in the first encoding stage, i.e. the encode_first_stage stage, the input RGB image can be mapped to the latent variable space based on the trained image encoder, the input image size is [B, C, H, W], and the output implicit expression is [B, C, H / 8, W / 8].

[0112] Secondly, in the learning condition stage, i.e. the get_learned_conditioning stage, the prompt word is encoded into an embedding representation with a size of [B, K, E] based on the trained text encoder, wherein B represents the batch size used in calculation, K represents the maximum length of text encoding, and E represents the dimension size of the embedding.

[0113] Then, in the forward diffusion stage, noise is added to the implicit representation of the image in the latent variable space, and the U-Net model is used to estimate and train the process of adding noise. The U-Net model also inputs the text encoding embedding as a condition, and uses an attention mechanism to learn the matching relationship between the text and the image. The U-Net model will output the predicted value of the noise, and the error between the predicted value and the real noise is calculated as the loss, and the U-Net model parameters are updated in the back propagation process, while the image encoder and the text encoder are fixed.

[0114] The terrain map generation model can be trained using the loss function of the diffusion model. In order to make the generated terrain map more realistic, the calculation of the model loss in the training process can be improved, that is, other loss terms are added to the existing loss function. Optionally, the loss function used when training the terrain map generation model also includes a regularization loss term, which is used to measure the similarity between the three-dimensional terrain digital elevation map generated by the terrain map generation model and the real three-dimensional terrain digital elevation map.

[0115] Further optionally, the regularization loss term is calculated using the following formula:

[0116] ;

[0117] Wherein, denotes the regularization loss value, denotes the three-dimensional terrain digital elevation map generated by the terrain map generation model, denotes the real three-dimensional terrain digital elevation map.

[0118] Optionally, the loss function used when training the terrain map generation model also includes a gradient loss term, which is used to measure the degree of height change of the terrain structure. In specific implementation, the gradient value can be calculated according to the elevation data, and the gradient calculation process can be realized by using a template convolution operation.

[0119] Further optionally, the gradient loss term is calculated using the following formula:

[0120] ;

[0121] ;

[0122] ;

[0123] Wherein, denotes the gradient loss value, denotes the gradient value corresponding to pixel [i,j] in the generated RGB image, denotes the gradient value corresponding to pixel [i,j] in the real RGB image, , , respectively represent the elevation values corresponding to the pixels [i+1, j], [i, j], [i, j+1] in the generated RGB image, , , respectively represent the elevation values corresponding to the pixels [i+1, j], [i, j], [i, j+1] in the real RGB image.

[0124] In the implementation, the regularization loss term and the gradient loss term can be added to the existing loss function to measure the spatial structure similarity of the terrain map (RGB image) output by the diffusion model, so that the gradient information of the terrain is more accurate, and the generated terrain map (RGB image) can be more realistic.

[0125] In the embodiment of the application, an initial training data set is obtained, each initial sample pair in the initial training data set includes a three-dimensional terrain digital elevation map and a corresponding prompt word, the three-dimensional terrain digital elevation map in each initial sample pair is encoded into an RGB image to obtain a target training data set, the terrain map generation model is trained using the target training data set to obtain a trained terrain map generation model. Since the diffusion model based on deep learning is used in the scheme, real three-dimensional terrain data can be used as training data, so the trained terrain map generation model has been greatly improved in terms of the realism of three-dimensional terrain modeling. At the same time, for various terrain structures, only the related data set needs to be input, and the deep learning model can adaptively generate various terrains without manual parameter adjustment. Finally, the scheme can also support input of the prompt word to train the output of the model, thereby improving the performance of the terrain map generation model in details. Embodiment three

[0126] Figure 5 The structure of the three-dimensional terrain modeling device provided by the third embodiment of the application is shown. For ease of illustration, only parts related to the embodiments of the application are shown, including.

[0127] The prompt word acquisition unit 31 is configured to acquire a prompt word.

[0128] The elevation map generation unit 32 is configured to input the prompt word into the trained terrain map generation model to obtain a target three-dimensional terrain digital elevation map, wherein the terrain map generation model includes a diffusion model and a quantization decoder connected in sequence, and the quantization decoder is configured to decode the RGB image output by the diffusion model into the target three-dimensional terrain digital elevation map.

[0129] The terrain map generation unit 33 is configured to generate a target three-dimensional terrain map according to the target three-dimensional terrain digital elevation map.

[0130] Optionally, the device further includes:

[0131] a data set acquisition unit, configured to acquire an initial training data set, each initial sample pair in the initial training data set comprising a three-dimensional terrain digital elevation map and a corresponding prompt word;

[0132] a quantization encoding unit, configured to encode the three-dimensional terrain digital elevation map in each group of initial sample pairs into an RGB image to obtain a target training data set; and

[0133] a model training unit, configured to train the terrain map generation model using the target training data set to obtain a trained terrain map generation model.

[0134] Optionally, the diffusion model is a stable diffusion-based diffusion model, and the loss function used when training the terrain map generation model comprises a regularization loss term and a gradient loss term. The regularization loss term is used to measure the similarity between the three-dimensional terrain digital elevation map generated by the terrain map generation model and the real three-dimensional terrain digital elevation map, and the gradient loss term is used to measure the degree of height change of the terrain structure.

[0135] Optionally, the regularization loss term is calculated using the following formula:

[0136] ;

[0137] wherein, denotes the regularization loss value, denotes the three-dimensional terrain digital elevation map generated by the terrain map generation model, denotes the real three-dimensional terrain digital elevation map.

[0138] Optionally, the gradient loss term is calculated using the following formula:

[0139] ;

[0140] ;

[0141] ;

[0142] wherein, denotes the gradient loss value, denotes the gradient value corresponding to pixel [i, j] in the generated RGB image, denotes the gradient value corresponding to pixel [i, j] in the real RGB image, 、 、 respectively denote the elevation values corresponding to pixels [i+1, j], [i, j], [i, j+1] in the generated RGB image, 、 、 respectively represent the elevation values corresponding to pixels [i+1, j], [i, j], [i, j+1] in the real RGB image.

[0143] Optionally, the quantization encoding unit is further configured to:

[0144] perform first quantization encoding on the three-dimensional terrain digital elevation map in the initial sample pair to obtain a first channel data map;

[0145] perform second quantization encoding on the three-dimensional terrain digital elevation map in the initial sample pair according to the first channel data map to obtain a second channel data map;

[0146] perform third quantization encoding on the three-dimensional terrain digital elevation map in the initial sample pair according to the first channel data map and the second channel data map to obtain a third channel data map;

[0147] obtain an encoded RGB image according to the first channel data map, the second channel data map and the third channel data map.

[0148] Optionally, the first quantization encoding adopts a formula:

[0149] ;

[0150] ;

[0151] ;

[0152] wherein H represents any elevation value of the three-dimensional terrain digital elevation map in the initial sample pair, R represents an R component in the first channel data map corresponding to the elevation value, denotes rounding down;

[0153] The second quantization encoding adopts a formula:

[0154] ;

[0155] wherein G represents a G component in the second channel data map corresponding to the elevation value;

[0156] The third quantization encoding adopts a formula:

[0157] ;

[0158] wherein B represents a B component in the third channel data map corresponding to the elevation value.

[0159] The formula adopted by the quantization decoder when decoding is:

[0160] ;

[0161] ;

[0162] ;

[0163] wherein R, G, B respectively represent R component, G component and B component of any pixel point in the RGB image output by the diffusion model, and H represents an elevation value corresponding to the pixel point in the target three-dimensional terrain digital elevation map.

[0164] Optionally, the terrain map generation unit 33 is further configured to:

[0165] input the prompt word into the trained terrain map generation model to obtain an initial three-dimensional terrain digital elevation map;

[0166] obtain a region in the initial three-dimensional terrain digital elevation map that needs to be locally modified;

[0167] re-generate the three-dimensional terrain digital elevation map in the region by using the trained terrain map generation model to obtain a target three-dimensional terrain digital elevation map.

[0168] Optionally, the device further comprises:

[0169] a local modification parameter acquisition unit configured to acquire a local modification parameter;

[0170] the elevation map generation unit is further configured to re-generate the three-dimensional terrain digital elevation map in the region by using the trained terrain map generation model according to the local modification parameter.

[0171] Optionally, the device further comprises:

[0172] a guide map acquisition unit configured to acquire a conditional guide map;

[0173] the elevation map generation unit is further configured to input the prompt word and the conditional guide map into the trained terrain map generation model to obtain the target three-dimensional terrain digital elevation map, wherein the conditional guide map is used to guide the diffusion model to generate the target three-dimensional terrain digital elevation map.

[0174] In the embodiments of the present application, each unit of the three-dimensional terrain modeling device can be realized by a corresponding hardware or software unit, and each unit can be an independent software or hardware unit, or can be integrated into a software or hardware unit, which does not limit the present application. The specific implementation of each unit of the three-dimensional terrain modeling device can refer to the description of the foregoing method embodiments, which will not be repeated here. Embodiment four

[0175] Figure 6 The structure of the three-dimensional terrain modeling device provided by the fourth embodiment of the present application is shown, and only the parts related to the embodiments of the present application are shown for ease of illustration.

[0176] The three-dimensional terrain modeling device 4 of the embodiment of the present application comprises a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. The processor 40 implements the steps in the above method embodiments when executing the computer program 42, for example Figure 1 the steps S101-S103 shown. Alternatively, the processor 40 implements the functions of the units in the above device embodiments when executing the computer program 42, for example Figure 5 the functions of the units 31-33 shown. Embodiment Five

[0177] In the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments, for example, Figure 1 the steps S101-S103 shown, and for another example, Figure 4 the steps S201-S203 shown. Alternatively, the computer program is executed by a processor to implement the functions of the units in the above device embodiments, for example Figure 5 the functions of the units 31-33 shown.

[0178] The computer readable storage medium of the embodiments of the present application can include any entity or device capable of carrying the computer program code, recording medium, such as ROM / RAM, magnetic disk, optical disk, flash memory, etc.

[0179] The above only describes the preferred embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of modeling a three-dimensional terrain, characterized by, The method comprises: obtaining a prompt word; inputting the prompt word into a trained terrain map generation model to obtain a target three-dimensional terrain digital elevation map, wherein the terrain map generation model comprises a diffusion model and a quantization decoder connected in sequence, and the quantization decoder is used for decoding an RGB image output by the diffusion model into the target three-dimensional terrain digital elevation map; generating a target three-dimensional terrain map according to the target three-dimensional terrain digital elevation map; the loss function used in the training of the terrain map generation model comprises a regularization loss term and a gradient loss term, the regularization loss term is used for measuring the similarity between the three-dimensional terrain digital elevation map generated by the terrain map generation model and the real three-dimensional terrain digital elevation map, and the gradient loss term is used for measuring the height change degree of the terrain structure; the regularization loss term is calculated by the following formula: ； wherein, denotes a regularization loss value, denotes a three-dimensional terrain digital elevation map generated by the terrain map generation model, denotes a real three-dimensional terrain digital elevation map; the gradient loss term is calculated by the following formula: ； ； ； wherein, denotes the gradient loss value, denotes the gradient value corresponding to pixel [i,j] in the generated RGB image, denotes the gradient value corresponding to pixel [i,j] in the real RGB image, , , denote the elevation value corresponding to pixel [i+1,j], [i,j], [i,j+1] in the generated RGB image, respectively, , , denote the elevation value corresponding to pixel [i+1,j], [i,j], [i,j+1] in the real RGB image, respectively.

2. The method of claim 1, wherein, the training steps of the terrain map generation model comprise: obtaining an initial training data set, each initial sample pair in the initial training data set comprising a three-dimensional terrain digital elevation map and a corresponding prompt word; encoding the three-dimensional terrain digital elevation map in each group of initial sample pairs into an RGB image to obtain a target training data set; training the terrain map generation model using the target training data set to obtain the trained terrain map generation model.

3. The method of claim 1 or 2, wherein, The diffusion model is a diffusion model based on stable diffusion.

4. The method of claim 2, wherein, The step of encoding the three-dimensional terrain digital elevation map in each group of initial sample pairs into an RGB image comprises: first quantization encoding the three-dimensional terrain digital elevation map in the initial sample pair to obtain a first channel data graph; second quantization encoding the three-dimensional terrain digital elevation map in the initial sample pair according to the first channel data graph to obtain a second channel data graph; third quantization encoding the three-dimensional terrain digital elevation map in the initial sample pair according to the first channel data graph and the second channel data graph to obtain a third channel data graph; obtaining an encoded RGB image according to the first channel data, the second channel data graph and the third channel data graph.

5. The method of claim 4, wherein, The formula used in the first quantization encoding is: ； ； ； wherein H represents any elevation value of the three-dimensional terrain digital elevation map in the initial sample pair, and R represents the R component in the first channel data graph corresponding to the elevation value; The formula used in the second quantization encoding is: ； wherein G represents the G component in the second channel data graph corresponding to the elevation value; The formula used in the third quantization encoding is: ； wherein B represents the B component in the third channel data graph corresponding to the elevation value; The formula used in the decoding of the quantization decoder is: ； ； ； wherein R, G and B respectively represent the R component, the G component and the B component of any pixel point in the RGB image output by the diffusion model, and H represents the elevation value in the target three-dimensional terrain digital elevation map corresponding to the pixel point.

6. The method of claim 1, wherein, The step of inputting the prompt word into the trained terrain map generation model to obtain the target three-dimensional terrain digital elevation map further comprises: inputting the prompt word into the trained terrain map generation model to obtain an initial three-dimensional terrain digital elevation map; obtaining a region on the initial three-dimensional terrain digital elevation map that needs to be locally modified; regenerating a three-dimensional terrain digital elevation map in the region through the trained terrain map generation model to obtain the target three-dimensional terrain digital elevation map.

7. The method of claim 1, wherein, Before the step of generating a target three-dimensional terrain map according to the target three-dimensional terrain digital elevation map, the method further comprises: obtaining a conditional guide map; inputting the prompt word and the conditional guide map into the trained terrain map generation model to obtain the target three-dimensional terrain digital elevation map, wherein the conditional guide map is used to guide the diffusion model to generate the target three-dimensional terrain digital elevation map.

8. A three-dimensional terrain modeling apparatus, characterized by comprising: The device comprises: a prompt word obtaining unit configured to obtain a prompt word; an elevation map generation unit configured to input the prompt word into a trained terrain map generation model to obtain a target three-dimensional terrain digital elevation map, wherein the terrain map generation model comprises a diffusion model and a quantization decoder connected in sequence, and the quantization decoder is configured to decode an RGB image output by the diffusion model into the target three-dimensional terrain digital elevation map; and a terrain map generation unit configured to generate a target three-dimensional terrain map according to the target three-dimensional terrain digital elevation map; a loss function used when training the terrain map generation model comprises a regularization loss term and a gradient loss term, the regularization loss term is used to measure the similarity between the three-dimensional terrain digital elevation map generated by the terrain map generation model and the real three-dimensional terrain digital elevation map, and the gradient loss term is used to measure the height variation degree of the terrain structure; the regularization loss term is calculated using the following formula: ； wherein, denotes a regularization loss value, denotes a three-dimensional terrain digital elevation map generated by the terrain map generation model, denotes a real three-dimensional terrain digital elevation map; the gradient loss term is calculated using the following formula: ； ； ； wherein, denotes the gradient loss value, denotes the gradient value corresponding to pixel [i, j] in the generated RGB image, denotes the gradient value corresponding to pixel [i, j] in the real RGB image, , , denote the elevation value corresponding to pixel [i+1, j], [i, j], [i, j+1] in the generated RGB image, respectively, , , denote the elevation value corresponding to pixel [i+1, j], [i, j], [i, j+1] in the real RGB image, respectively.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1-9. The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 7.

Images