Image color-based three-dimensional grassland scene construction method, system, terminal and medium

Abstract

The application discloses a three-dimensional grassland scene construction method and system based on image colors, a terminal and a medium, relates to the technical field of image processing, and comprises the following steps: acquiring a two-dimensional color image of a grassland; determining a color category corresponding to a pixel in the two-dimensional color image of the grassland according to a color feature of the two-dimensional color image of the grassland; generating at least one mask image corresponding to the two-dimensional color image of the grassland according to the color category, wherein one mask image corresponds to one color category; determining a model instance attribute parameter set corresponding to the color category according to the mask image; and performing instance rendering according to the model instance attribute parameter set to generate a three-dimensional grassland scene. In this way, the authenticity and generation effect of the three-dimensional grassland scene are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, system, terminal, and medium for constructing a three-dimensional grassland scene based on image color. Background Technology

[0002] With the development of science and technology, the application of 3D modeling is becoming increasingly widespread. For example, 3D modeling technology can be used to construct 3D urban scenes. In the process of constructing 3D scenes, natural scene elements such as vegetation and grassland are crucial for enhancing the overall realism.

[0003] In related technologies, grassland scenes are typically constructed by placing models. Specifically, the same grassland model is repeatedly placed in a regular grid to build the grassland scene. The problem with existing technologies is that they completely ignore the distribution patterns of plants in real-world grassland scenes, and can only achieve mechanical repetition, which is not conducive to improving the realism of the grassland scene and affects the generation effect.

[0004] Therefore, the relevant technologies still need to be improved and developed. Summary of the Invention

[0005] The main purpose of this application is to provide a method, system, terminal and medium for constructing a three-dimensional grassland scene based on image color. It aims to solve the technical problem that the scheme of repeatedly placing the same grassland model in a regular grid to construct a grassland scene in related technologies completely ignores the distribution pattern of plants in the grassland in the real scene. It can only achieve mechanical repetition, which is not conducive to improving the realism of the grassland scene and affects the generation effect of the grassland scene.

[0006] To achieve the above objectives, the first aspect of this application provides a method for constructing a 3D grassland scene based on image color, wherein the method includes:

[0007] Obtain a two-dimensional color image of the grassland;

[0008] Based on the color features of the above two-dimensional color image of the grassland, determine the color category corresponding to the pixel in the above two-dimensional color image of the grassland;

[0009] Based on the aforementioned color categories, at least one mask image corresponding to the aforementioned two-dimensional color image of the grassland is generated, wherein one of the aforementioned mask images corresponds to one of the aforementioned color categories;

[0010] Based on the above mask image, determine the set of model instance attribute parameters corresponding to the above color categories;

[0011] Based on the set of attribute parameters of the above model instance, instantiation rendering is performed to generate a 3D grassland scene.

[0012] Optionally, the above two-dimensional color image of the grassland is an RGB image;

[0013] The above-mentioned determination of the color category corresponding to the pixel in the above-mentioned two-dimensional color image of grassland based on the color features of the above-mentioned two-dimensional color image of grassland includes:

[0014] The above two-dimensional color image of the grassland is converted to an HSV image by color space conversion;

[0015] Based on the hue characteristics of the HSV image, the color category corresponding to the pixel in the two-dimensional color image of the grassland is determined.

[0016] Optionally, determining the color category corresponding to a pixel in the two-dimensional color image of the grassland based on the hue features of the HSV image includes:

[0017] Based on the hue values ​​corresponding to each pixel in the HSV image and the multiple hue intervals obtained by pre-dividing the hue domain, the pixels in the HSV image are divided into the corresponding hue intervals, and the number of pixels corresponding to each of the above hue intervals is determined.

[0018] Based on the number of pixels corresponding to each of the above hue intervals, the total number of pixels corresponding to the above HSV image, and the preset pixel ratio threshold, the target hue interval is determined from the above hue intervals;

[0019] The grass element category associated with the above target hue range is taken as the color category corresponding to each pixel in the above target hue range;

[0020] Based on the color category corresponding to each pixel in the target hue range, the color category of the corresponding pixel in the above two-dimensional color image of the grassland is determined.

[0021] Optionally, the above determination of the set of model instance attribute parameters corresponding to the above color category based on the above mask image includes:

[0022] Obtain the sliding window size, divide the mask image according to the sliding window size, and obtain multiple processing blocks corresponding to the mask image;

[0023] The target processing block is determined based on the number of pixels in each of the above processing blocks and the preset pixel threshold.

[0024] Based on the pixels in the aforementioned target processing blocks, determine the model instance attribute parameters corresponding to each of the aforementioned target processing blocks;

[0025] Based on all target processing blocks corresponding to the above mask image, and the model instance attribute parameters corresponding to the above target processing blocks, determine the set of model instance attribute parameters corresponding to the above mask image.

[0026] Optionally, the above model instance attribute parameters include position parameters, rotation parameters, scaling parameters, and color parameters;

[0027] The above-mentioned determination of model instance attribute parameters corresponding to each of the above-mentioned target processing blocks based on the pixels in the above-mentioned target processing blocks includes: performing aggregation processing on the pixels in the above-mentioned target processing blocks to obtain aggregated pixel regions, and determining the minimum outer circle of the above-mentioned aggregated pixel regions;

[0028] The position parameters are determined based on the center point of the minimum circumcircle.

[0029] Based on the preset starting direction and the intersection of the minimum circumcircle and the aggregated pixel region, the main direction corresponding to the aggregated pixel region is determined, and the rotation parameters are determined based on the main direction.

[0030] The scaling parameters are determined based on the radius of the minimum circumcircle.

[0031] The color parameters are determined based on the average color value of all pixels within the aggregated pixel region.

[0032] Optionally, the above method further includes:

[0033] In response to the scene expansion command, a supplementary two-dimensional color image is generated based on the aforementioned mask image and the trained image generation model;

[0034] The above-mentioned supplementary two-dimensional color image is used as the newly added two-dimensional color image of the grassland. The process returns to the above steps of determining the color category of the pixels in the above-mentioned two-dimensional color image of the grassland based on the color characteristics of the above-mentioned two-dimensional color image of the grassland, and the subsequent steps.

[0035] Optionally, generating a supplementary two-dimensional color image based on the mask image and the trained image generation model includes:

[0036] Based on preset random offset parameters, the position offset processing is performed on the aggregated pixel region in the above mask image to obtain at least one mask supplement image corresponding to the above mask image;

[0037] The mask image and the mask supplement image are respectively input into the image generation model to obtain the two-dimensional color images output by the image generation model.

[0038] A second aspect of this application provides a three-dimensional grassland scene construction system based on image color, wherein the system includes:

[0039] The data acquisition module is used to acquire two-dimensional color images of the grassland.

[0040] The color category determination module is used to determine the color category corresponding to the pixels in the two-dimensional color image of the grassland based on the color characteristics of the two-dimensional color image of the grassland.

[0041] The mask generation module is used to generate at least one mask image corresponding to the two-dimensional color image of the grassland according to the above color categories, wherein one of the above mask images corresponds to one of the above color categories;

[0042] The parameter determination module is used to determine the set of model instance attribute parameters corresponding to the above color categories based on the above mask image;

[0043] The rendering module is used to instantiate and render the model instance based on the set of attribute parameters mentioned above, generating a 3D grassland scene.

[0044] A third aspect of this application provides a terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements any of the steps of the above-described method for constructing a three-dimensional grassland scene based on image color.

[0045] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the steps of the above-described method for constructing a three-dimensional grassland scene based on image color.

[0046] As can be seen from the above, the present application provides a method for constructing a three-dimensional grassland scene based on image color. Specifically, it involves acquiring a two-dimensional color image of the grassland; determining the color category corresponding to the pixels in the two-dimensional color image of the grassland based on the color features of the two-dimensional color image of the grassland; generating at least one mask image corresponding to the two-dimensional color image of the grassland based on the color category, wherein one mask image corresponds to one of the color categories; determining the set of model instance attribute parameters corresponding to the color category based on the mask image; and performing instantiation rendering based on the set of model instance attribute parameters to generate a three-dimensional grassland scene.

[0047] Therefore, when constructing a 3D grassland scene, instead of directly using a regular mesh to repeatedly place the same grassland model, the 3D grassland scene is generated based on a 2D color image of grassland in a real-world scene. Specifically, based on the color features of the 2D color image of the grassland, the color category corresponding to each pixel in the image is determined, thereby generating mask images corresponding to different color categories. Then, based on the mask images, the set of attribute parameters for model instances is determined for instantiation and rendering, generating the 3D grassland scene. The different colors in the 2D color image of the grassland can realistically reflect the distribution of different vegetation types in the image. Therefore, the obtained mask images can be used to characterize the distribution of different vegetation types, thereby allowing for better control over the distribution of different model instances, which is beneficial for improving the realism and generation effect of the 3D grassland scene. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating a method for constructing a 3D grassland scene based on image color, as provided in an embodiment of this application.

[0050] Figure 2 This is a schematic diagram illustrating the process of determining the attribute parameters of a model instance provided in an embodiment of this application;

[0051] Figure 3 This is a schematic diagram of the placement rules provided in the embodiments of this application;

[0052] Figure 4 This is a schematic diagram of the constituent modules of a three-dimensional grassland scene construction system based on image color provided in an embodiment of this application;

[0053] Figure 5 This is a block diagram illustrating the internal structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0054] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0055] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0056] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0057] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0058] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to classification." Similarly, the phrases "if determined" or "if classified to [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once classified to [the described condition or event]," or "in response to classification to [the described condition or event]."

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0060] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0061] Currently, 3D modeling technology is being used more and more widely, and 3D virtual geographic environment modeling has become an important research direction in the fields of Geographic Information Science (GIS) and Computer Graphics (CG). When constructing 3D urban scenes, natural scene elements such as vegetation and grassland are crucial to improving the overall realism. However, grassland has characteristics such as wide coverage and complex microstructure, making it difficult to model with high realism.

[0062] In some applications, grassland scenes are constructed using model placement methods. While repeatedly placing the same model using a regular grid or distributing the model using a completely random algorithm can create a 3D structure, it ignores the specific color and spatial distribution patterns of plants in real grasslands caused by factors such as growth competition and the alternation of life and death. The resulting scenes lack ecological basis and have a stiff visual effect. Furthermore, it typically only achieves mechanical repetition, which is detrimental to improving the realism of the grassland scene and affects the generated effect. In addition, oblique photogrammetry reconstruction is prone to texture stretching and geometric distortion; if modeling is done plant by plant, the data volume is enormous and the frame rate drops drastically.

[0063] In other application scenarios, texture tiling is used to construct grassland scenes. The captured grassland image is converted into a texture and then repeatedly mapped onto the terrain surface. This method is efficient but results in severe texture duplication and fails to represent the three-dimensional structure and spatial distribution of the grassland, thus affecting the quality of the generated grassland scene.

[0064] To address at least one of the aforementioned technical problems, this application proposes a method for constructing a 3D grassland scene based on image color. The method involves: acquiring a 2D color image of the grassland; determining the color category corresponding to each pixel in the 2D color image based on its color features; generating at least one mask image corresponding to the 2D color image based on the color category, wherein each mask image corresponds to one of the aforementioned color categories; determining a set of model instance attribute parameters corresponding to the aforementioned color category based on the mask image; and performing instantiation rendering based on the set of model instance attribute parameters to generate a 3D grassland scene.

[0065] Therefore, when constructing a 3D grassland scene, instead of directly using a regular mesh to repeatedly place the same grassland model, the 3D grassland scene is generated based on a 2D color image of grassland in a real-world scene. Specifically, based on the color features of the 2D color image of the grassland, the color category corresponding to each pixel in the image is determined, thereby generating mask images corresponding to different color categories. Then, based on the mask images, the set of attribute parameters for model instances is determined for instantiation and rendering, generating the 3D grassland scene. The different colors in the 2D color image of the grassland can realistically reflect the distribution of different vegetation types in the image. Therefore, the obtained mask images can be used to characterize the distribution of different vegetation types, thereby allowing for better control over the distribution of different model instances, which is beneficial for improving the realism and generation effect of the 3D grassland scene.

[0066] like Figure 1 As shown in the figure, this application provides a method for constructing a three-dimensional grassland scene based on image color. Specifically, the method includes the following steps:

[0067] Step S100: Obtain a two-dimensional color image of the grassland;

[0068] Step S200: Determine the color category corresponding to the pixel in the two-dimensional color image of the grassland based on the color features of the two-dimensional color image of the grassland.

[0069] Step S300: Based on the above color categories, generate at least one mask image corresponding to the above two-dimensional color image of the grassland, wherein one of the above mask images corresponds to one of the above color categories;

[0070] Step S400: Based on the above mask image, determine the set of model instance attribute parameters corresponding to the above color category;

[0071] Step S500: Based on the above set of model instance attribute parameters, perform instantiation rendering to generate a 3D grassland scene.

[0072] In this embodiment, a multi-stage processing flow is used to convert an input 2D color image of grassland into a highly realistic 3D grassland scene, and scene expansion is supported. Specifically, in actual use, the above processing flow includes two stages: conversion from sample image to placement point and expansion from sample image to generated image. It should be noted that the expansion stage from sample image to generated image can be selected for execution according to actual needs.

[0073] Specifically, the above two-dimensional color image of the grassland is an RGB image;

[0074] The above-mentioned determination of the color category corresponding to the pixel in the above-mentioned two-dimensional color image of grassland based on the color features of the above-mentioned two-dimensional color image of grassland includes:

[0075] The above two-dimensional color image of the grassland is converted to an HSV image by color space conversion;

[0076] Based on the hue characteristics of the HSV image, the color category corresponding to the pixel in the two-dimensional color image of the grassland is determined.

[0077] Specifically, based on the hue characteristics of the HSV image, the determination of the color category corresponding to the pixels in the two-dimensional color image of the grassland includes:

[0078] Based on the hue values ​​corresponding to each pixel in the HSV image and the multiple hue intervals obtained by pre-dividing the hue domain, the pixels in the HSV image are divided into the corresponding hue intervals, and the number of pixels corresponding to each of the above hue intervals is determined.

[0079] Based on the number of pixels corresponding to each of the above hue intervals, the total number of pixels corresponding to the above HSV image, and the preset pixel ratio threshold, the target hue interval is determined from the above hue intervals;

[0080] The grass element category associated with the above target hue range is taken as the color category corresponding to each pixel in the above target hue range;

[0081] Based on the color category corresponding to each pixel in the target hue range, the color category of the corresponding pixel in the above two-dimensional color image of the grassland is determined.

[0082] In this embodiment, the acquired RGB format two-dimensional color image of grassland is first converted to HSV color space to obtain an HSV image. HSV color space is more consistent with human visual perception characteristics, can better maintain color consistency, and facilitates subsequent color feature analysis.

[0083] Furthermore, the color category corresponding to each pixel is determined based on the hue characteristics of the HSV image. Specifically, according to multiple hue intervals pre-divided into the hue gamut (e.g., divided into intervals of 35° each), all pixels in the HSV image are traversed. Based on the hue value of each pixel, the pixels are assigned to the corresponding hue intervals, and the number of pixels corresponding to each hue interval is counted. The proportion of pixels in each hue interval to the total number of pixels in the HSV image is calculated, and this proportion is compared with a preset pixel proportion threshold (the specific value can be set according to actual needs, for example, it can be set to 4%). Target hue intervals that meet the threshold requirements are selected; for example, hue intervals whose pixel proportion to the total number of pixels in the HSV image is greater than 4% are selected as target hue intervals.

[0084] Associate each target hue range with a corresponding grass element category (such as soil, green grass, withered grass, flowers, etc.), and use this grass element category as the color category of each pixel within the target hue range, thereby determining the color category of pixels in the entire 2D color image of the grassland.

[0085] Furthermore, for the two-dimensional color image of grassland labeled with color categories, multiple mask images are obtained according to the color category corresponding to each pixel. It should be noted that the size of the mask images is the same as that of the two-dimensional color image of grassland. Each mask image includes pixels of one category, indicating the location of pixels corresponding to that category. The mask images are binary images, which can clearly represent the distribution of pixels of the corresponding color category in the original image, providing a foundation for subsequent calculation of model instance attributes.

[0086] In this embodiment of the application, determining the set of model instance attribute parameters corresponding to the color category based on the mask image includes:

[0087] Obtain the sliding window size, divide the mask image according to the sliding window size, and obtain multiple processing blocks corresponding to the mask image;

[0088] The target processing block is determined based on the number of pixels in each of the above processing blocks and the preset pixel threshold.

[0089] Based on the pixels in the aforementioned target processing blocks, determine the model instance attribute parameters corresponding to each of the aforementioned target processing blocks;

[0090] Based on all target processing blocks corresponding to the above mask image, and the model instance attribute parameters corresponding to the above target processing blocks, determine the set of model instance attribute parameters corresponding to the above mask image.

[0091] Specifically, the attribute parameters of the above model instance include position parameters, rotation parameters, scaling parameters, and color parameters;

[0092] The above-mentioned determination of model instance attribute parameters corresponding to each of the above-mentioned target processing blocks based on the pixels in the above-mentioned target processing blocks includes: performing aggregation processing on the pixels in the above-mentioned target processing blocks to obtain aggregated pixel regions, and determining the minimum outer circle of the above-mentioned aggregated pixel regions;

[0093] The position parameters are determined based on the center point of the minimum circumcircle.

[0094] Based on the preset starting direction and the intersection of the minimum circumcircle and the aggregated pixel region, the main direction corresponding to the aggregated pixel region is determined, and the rotation parameters are determined based on the main direction.

[0095] The scaling parameters are determined based on the radius of the minimum circumcircle.

[0096] The color parameters are determined based on the average color value of all pixels within the aggregated pixel region.

[0097] In this embodiment, based on the generated mask image, a pixel aggregation algorithm is used to calculate and determine the spatial and color attributes of each model instance, that is, to convert color-based pixel data into model placement parameters.

[0098] Figure 2 This is a schematic diagram illustrating the process of determining model instance attribute parameters according to an embodiment of this application, such as... Figure 2 As shown, for each mask image, firstly, the size of the sliding window (Slider) related to the physical size of the target model is obtained (the specific size can be set and adjusted according to actual needs). The mask image is then divided according to this size to obtain multiple processing blocks corresponding to the mask image. The number of effective pixels in each processing block is counted, and the number of effective pixels is compared with a preset pixel threshold. Target processing blocks with an effective pixel ratio exceeding the pixel threshold t (the specific value can be set and adjusted according to actual needs) are selected.

[0099] For each target processing block, pixels are aggregated to obtain aggregated pixel regions, and the minimum bounding circle of these aggregated pixel regions is determined, for example... Figure 2 In the circle O, based on the relevant features of the minimum circumcircle and the aggregated pixel region, the various attribute parameters of the model instance are determined respectively.

[0100] Specifically, using the center point of the smallest circumcircle (e.g.) Figure 2 Point O in the model is used as the position parameter, determining the placement location of the model instance. The radius of the smallest circumcircle is used as the scaling parameter to determine the final scaling parameter; for example, the final scaling parameter can be a value determined based on a preset scaling factor of the radius. The main direction of the aggregated pixel region is determined by combining the preset starting direction and the intersection of the smallest circumcircle and the aggregated pixel region, and the rotation parameter is determined accordingly; for example, Figure 2 The preset starting direction is the three o'clock direction, i.e., the direction from point O to point A. The main direction is: starting from the preset starting direction (such as the three o'clock direction), rotating clockwise, the direction of the line connecting the first encountered boundary point of the aggregated pixel region and the center O (i.e., the direction from point O to point B). The angle between the main direction and the horizontal line (or the starting direction) is used as the rotation parameter. The average color value of all pixels within the aggregated pixel region is calculated and used as the color parameter (i.e., the shading parameter). The model instance attribute parameters of all target processing blocks corresponding to the mask image are integrated to form the model instance attribute parameter set corresponding to the mask image.

[0101] The set of model instance attribute parameters obtained from the above steps is input into the graphics rendering engine. Based on a pre-built 3D model library containing various basic components such as grass, flowers, and rocks, the rendering engine creates a GPU (Graphics Processing Unit) instantiation rendering process for each basic model. By applying the attribute parameters to the basic 3D model, different transformation matrices and material parameters are applied to each model instance. GPU instantiation technology is used to achieve batch rendering, ultimately generating a 3D grass scene. Each model instance is generated from the same basic 3D model by applying different transformation matrices and material parameters, thus ensuring high rendering efficiency while achieving diversity in model appearance and effectively avoiding visual repetition.

[0102] Considering the need to construct large-scale scenes, this application embodiment also provides a method for further scene expansion. Specifically, the above method further includes:

[0103] In response to the scene expansion command, a supplementary two-dimensional color image is generated based on the aforementioned mask image and the trained image generation model;

[0104] The above-mentioned supplementary two-dimensional color image is used as the newly added two-dimensional color image of the grassland. The process returns to the above steps of determining the color category of the pixels in the above two-dimensional color image of the grassland based on the color features of the above two-dimensional color image of the grassland, and the subsequent steps, until an extended three-dimensional grassland scene is obtained based on the original two-dimensional color image of the grassland and the newly added two-dimensional color image of the grassland.

[0105] The process of generating a supplementary two-dimensional color image based on the mask image and the trained image generation model includes:

[0106] Based on preset random offset parameters, the position offset processing is performed on the aggregated pixel region in the above mask image to obtain at least one mask supplement image corresponding to the above mask image;

[0107] The mask image and the mask supplement image are respectively input into the image generation model to obtain the supplementary two-dimensional color image output by the image generation model.

[0108] It should be noted that the aforementioned scene expansion command can be input by the user, control device, or other applications when scene expansion is needed, to construct a larger grassland scene based on the current 2D color image of the grassland. When scene expansion is not required, the scene expansion command does not need to be input.

[0109] To overcome the limitations of single-sample images and achieve large-scale scene generation, this application employs a generative artificial intelligence method for extension. It should be noted that the image generation model in this application uses a conditional generative adversarial network, but this is not intended as a specific limitation. Specifically, this network uses a U-Net-structured generator and a PatchGAN discriminator, and learns the mapping relationship from the mask to the real image in advance through adversarial training.

[0110] Specifically, if a scene expansion command is received, the scene expansion process can be initiated. According to preset random offset parameters, the position of the aggregated pixel region in the mask image is offset (the offset distance d and a random number r between 0 and 1 are set to control the degree of offset), generating at least one mask supplement image.

[0111] The original mask image and the mask supplement image are input into the trained image generation model, which outputs a supplementary 2D color image with the same color style as the original grassland image but a different spatial layout. The supplementary 2D color image is used as the newly added 2D color image of grassland, and the operations of steps S200 to S500 are returned to convert the expanded image content into a 3D grassland scene, realizing the construction of a large-scale grassland scene.

[0112] It should be noted that the above-mentioned conditional generative adversarial network is pre-trained. During the training process, in order to expand the training data, new masks can also be created by randomly shifting the positions of the identified pixel aggregates in the mask. For example, by applying random shifts (d is the shift distance and r is a random number between 0 and 1) to the pixel aggregates in the main feature mask, diverse new masks can be generated.

[0113] To expand the training dataset, a large number of low-resolution images (e.g., 512×512 pixels) and their corresponding masks can be generated by slicing the high-resolution original images (e.g., 4000×3000 pixels) (e.g., with a stride of 128 pixels). During training, the quality of the generated images gradually increases. Initially, the network could only generate blurry, meaningless color blocks. After 200 training iterations, it learned to generate high-quality images with clear semantic information, thus obtaining the trained network.

[0114] In this embodiment, a 3D grassland scene is generated by instantiating and rendering based on the obtained set of model instance attribute parameters. This process relies on a pre-built 3D model library containing various basic components (such as grass, flowers, and stones). In the graphics rendering engine, a GPU Instancing rendering process is created for each basic model.

[0115] Before finally placing the model (component), a series of predefined placement rules are applied to perform final verification and adjustment of the position of the model components in order to resolve conflicts between models and ensure that the scene conforms to physical laws (such as a rigid body covering a non-rigid body).

[0116] Figure 3 This is a schematic diagram of the placement rules provided in the embodiments of this application. It should be noted that... Figure 3 The components in the model library represent the components of the grass model that need to be placed, that is, the components corresponding to the basic 3D model.

[0117] like Figure 3 As shown in the embodiments of this application, four placement rules are predefined. Specifically, Rule 1: Only one model can be placed at a spatial point. Because the final rendering is done in three-dimensional space, two models located at the same spatial point will clip each other. Conflicts can be resolved by adjusting the placement order of the models, placing models of relatively lower importance at other levels. Rule 2: The placement hierarchy should follow a bottom-up order. Referring to the natural growth process of grassland, the placement hierarchy order is set as "soil-grass-flower-leaf-branch-stone". Among these components, "soil" is modeled only using texture, while other components need to be modeled using both geometry and texture. Rule 3: The height of each model component should be equal to the terrain height at its location. Image-based point generation algorithms cannot obtain the height of components. Therefore, the z-value (i.e., height value) of each model component is sampled from the surrounding terrain, allowing the model component to conform to the terrain slope. In addition, the tilt direction is always perpendicular to the tangent direction of the sampling point. Rule 4: Rigid components need to cover non-rigid components. Since gravity is an important physical factor affecting the placement of model components, but simulating a complete physical system is very complex, this application embodiment only considers the covering relationship between rigid and non-rigid components. For example, in natural scenes, stones usually cover grass.

[0118] Thus, based on the above placement rules, the realism and generation effect of the 3D grassland scene can be further improved.

[0119] This application provides a method for constructing a 3D grassland scene based on image color. The method involves: acquiring a 2D color image of the grassland; determining the color category corresponding to each pixel in the 2D color image based on its color features; generating at least one mask image corresponding to the 2D color image based on the color category, wherein each mask image corresponds to one of the color categories; determining a set of model instance attribute parameters corresponding to the color category based on the mask image; and performing instantiation rendering based on the set of model instance attribute parameters to generate a 3D grassland scene.

[0120] Therefore, when constructing a 3D grassland scene, instead of directly using a regular mesh to repeatedly place the same grassland model, the 3D grassland scene is generated based on a 2D color image of grassland in a real-world scene. Specifically, based on the color features of the 2D color image of the grassland, the color category corresponding to each pixel in the image is determined, thereby generating mask images corresponding to different color categories. Then, based on the mask images, the set of attribute parameters for model instances is determined for instantiation and rendering, generating the 3D grassland scene. The different colors in the 2D color image of the grassland can realistically reflect the distribution of different vegetation types in the image. Therefore, the obtained mask images can be used to characterize the distribution of different vegetation types, thereby allowing for better control over the distribution of different model instances, which is beneficial for improving the realism and generation effect of the 3D grassland scene.

[0121] Specifically, addressing the problems of visual repetition, unreasonable distribution, lack of realism, and difficulty in scaling to large areas in existing grassland modeling techniques, this application proposes a novel grassland scene construction method combining procedural generation and generative artificial intelligence. First, color features are extracted from real grassland sample images. Based on feature integration theory, the color distribution of different hues in the images is mapped to the placement attributes (including position, rotation, scaling, and color) of 3D models of different types of components (such as grass, flowers, and stones), achieving pixel-to-placement point conversion based on color quantization. Second, to solve the problem of large-scale scaling, a conditional generative adversarial network (cGAN) is trained. Using a color mask segmented from the sample images as a condition, it learns the color and spatial distribution patterns, thereby generating a batch of new images with consistent color styles but diverse spatial layouts. Finally, these generated images are again converted into placement points through the aforementioned color mapping process, and GPU instantiation technology is used to efficiently render large-scale, highly realistic grassland scenes. This solution uses color as a core visual feature to seamlessly connect image analysis with 3D modeling, achieving efficient and realistic construction of grassland scenes from micro to macro levels. At the same time, it ensures the naturalness and diversity of the generated scene in terms of color and spatial distribution, effectively avoiding visual repetition.

[0122] This application converts RGB images to the HSV color space for color classification and combines this with pixel percentage thresholds to filter target hue ranges, enabling precise identification of different element categories in grassland images and providing accurate color distribution data for subsequent scene construction. Furthermore, it establishes an automated calculation method from image color distribution to 3D model spatial attributes (position, rotation, scaling, color), which is the core of achieving highly realistic procedural generation.

[0123] The model instance attributes are determined by processing blocks based on mask images and sliding windows, achieving a precise mapping from image pixels to 3D model parameters. This ensures that the position, rotation, scaling, and color parameters of each model instance conform to the distribution patterns of real grass elements, enhancing the realism and rationality of the scene.

[0124] Batch rendering using GPU instantiation technology significantly improves rendering efficiency while ensuring the diversity of model appearance, and solves the problem of sharp frame rate drops caused by traditional per-model modeling.

[0125] An optional scene expansion process, based on conditional generative adversarial networks (GANs), enables batch generation of grassland images while maintaining a consistent color style with the original images. This effectively solves the challenge of expanding large-scale grassland scene construction and avoids visual repetition. Furthermore, a method is proposed that uses a color segmentation mask as a condition, and a GAN learns and generates new images, achieving spatial layout changes while maintaining color style consistency, thus solving the problem of large-scale scene expansion.

[0126] It should be noted that, in the extension phase, this application uses a conditional generative adversarial network (cGAN) as an example for illustration. In practical applications, the image generation model can be replaced with other types of conditional generation models, such as diffusion models or variational autoencoders (VAEs). When performing color quantization, clustering algorithms such as K-means can be used to automatically determine the dominant color tone; in the pixel aggregation stage, other spatial clustering algorithms (such as DBSCAN) can also be used instead.

[0127] It should be further noted that the solutions in this application can be used not only for grassland modeling, but also for modeling other natural elements, such as flower beds, shrubs, fallen leaves, and gravel beaches. Dynamic textures and skeletal animations can also be introduced to simulate the effect of grass blades swaying in the wind, further enhancing the realism of real-time rendering.

[0128] like Figure 4 As shown, corresponding to the above-described method for constructing a 3D grassland scene based on image color, this application embodiment also provides a system for constructing a 3D grassland scene based on image color, the system comprising:

[0129] Data acquisition module 410 is used to acquire a two-dimensional color image of the grassland;

[0130] The color category determination module 420 is used to determine the color category corresponding to the pixels in the two-dimensional color image of the grassland based on the color characteristics of the two-dimensional color image of the grassland.

[0131] The mask generation module 430 is used to generate at least one mask image corresponding to the two-dimensional color image of the grassland according to the above color category, wherein one of the above mask images corresponds to one of the above color categories;

[0132] The parameter determination module 440 is used to determine the set of model instance attribute parameters corresponding to the above color category based on the above mask image;

[0133] The rendering module 450 is used to instantiate and render the model instance based on the set of attribute parameters mentioned above, generating a 3D grassland scene.

[0134] Therefore, when constructing a 3D grassland scene, instead of directly using a regular mesh to repeatedly place the same grassland model, the 3D grassland scene is generated based on a 2D color image of grassland in a real-world scene. Specifically, based on the color features of the 2D color image of the grassland, the color category corresponding to each pixel in the image is determined, thereby generating mask images corresponding to different color categories. Then, based on the mask images, the set of attribute parameters for model instances is determined for instantiation and rendering, generating the 3D grassland scene. The different colors in the 2D color image of the grassland can realistically reflect the distribution of different vegetation types in the image. Therefore, the obtained mask images can be used to characterize the distribution of different vegetation types, thereby allowing for better control over the distribution of different model instances, which is beneficial for improving the realism and generation effect of the 3D grassland scene.

[0135] It should be noted that the specific structure and implementation of the above-mentioned image color-based 3D grassland scene construction system and its various modules or units can be referred to the corresponding descriptions in the above method embodiments, and will not be repeated here.

[0136] It should be noted that the division of the modules in the above-mentioned image color-based 3D grassland scene construction system is not unique and is not intended as a specific limitation.

[0137] Based on the above embodiments, this application also provides a terminal, the principle block diagram of which can be as follows: Figure 5 As shown. The terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps of any of the above-described image color-based 3D grassland scene construction methods. The display screen can be a liquid crystal display (LCD) or an e-ink display.

[0138] Those skilled in the art will understand that Figure 5 The block diagram shown is only a partial structural diagram related to the solution of this application and does not constitute a limitation on the terminal on which the solution of this application is applied. The specific terminal may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0139] In one embodiment, a terminal is provided, the terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of any of the image color-based three-dimensional grassland scene construction methods provided in the embodiments of this application.

[0140] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of any of the image color-based three-dimensional grassland scene construction methods provided in this application.

[0141] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0143] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0144] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0145] In the embodiments provided in this application, it should be understood that the disclosed systems / terminal devices and methods can be implemented in other ways. For example, the system / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units described above is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0146] If the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, and software distribution media, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0147] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions are not in essence a departure from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for constructing a 3D grassland scene based on image color, characterized in that, The method includes: Obtain a two-dimensional color image of the grassland; Based on the color features of the two-dimensional color image of the grassland, determine the color category corresponding to the pixel in the two-dimensional color image of the grassland; Based on the color category, at least one mask image corresponding to the two-dimensional color image of the grassland is generated, wherein one mask image corresponds to one color category; Based on the mask image, determine the set of model instance attribute parameters corresponding to the color category; Based on the set of attribute parameters of the model instance, instantiation rendering is performed to generate a three-dimensional grassland scene; The step of determining the set of model instance attribute parameters corresponding to the color category based on the mask image includes: obtaining the sliding window size; dividing the mask image according to the sliding window size to obtain multiple processing blocks corresponding to the mask image; determining a target processing block based on the number of pixels in each processing block and a preset pixel threshold; determining the model instance attribute parameters corresponding to each target processing block based on the pixels in the target processing block; and determining the set of model instance attribute parameters corresponding to the mask image based on all target processing blocks corresponding to the mask image and the model instance attribute parameters corresponding to the target processing blocks. The model instance attribute parameters include position parameters, rotation parameters, scaling parameters, and color parameters; The step of determining the model instance attribute parameters corresponding to each target processing block based on the pixels in the target processing block includes: performing aggregation processing on the pixels in the target processing block to obtain aggregated pixel regions, and determining the minimum circumcircle of the aggregated pixel regions; determining the position parameters based on the center point of the minimum circumcircle; determining the main direction corresponding to the aggregated pixel regions based on a preset starting direction and the intersection of the minimum circumcircle and the aggregated pixel regions, and determining the rotation parameters based on the main direction; determining the scaling parameters based on the radius of the minimum circumcircle; and determining the color parameters based on the average color value of all pixels in the aggregated pixel regions.

2. The method for constructing a 3D grassland scene based on image color according to claim 1, characterized in that, The two-dimensional color image of the grassland is an RGB image; The step of determining the color category corresponding to the pixels in the two-dimensional color image of the grassland based on the color features of the grassland image includes: The two-dimensional color image of the grassland is converted to an HSV image by color space conversion. Based on the hue characteristics of the HSV image, the color category corresponding to the pixel in the two-dimensional color image of the grassland is determined.

3. The method for constructing a 3D grassland scene based on image color according to claim 2, characterized in that, The step of determining the color category corresponding to the pixel in the two-dimensional color image of the grassland based on the hue features of the HSV image includes: Based on the hue value corresponding to each pixel in the HSV image and multiple hue intervals obtained by pre-dividing the hue domain, the pixels in the HSV image are divided into corresponding hue intervals, and the number of pixels corresponding to each hue interval is determined. The target hue interval is determined from the hue intervals based on the number of pixels corresponding to each hue interval, the total number of pixels corresponding to the HSV image, and a preset pixel ratio threshold. The grass element category associated with the target hue range is used as the color category corresponding to each pixel in the target hue range; The color category of the corresponding pixel in the two-dimensional color image of the grassland is determined based on the color category of each pixel in the target hue range.

4. The method for constructing a 3D grassland scene based on image color according to claim 1, characterized in that, The method further includes: In response to the scene expansion command, a supplementary two-dimensional color image is generated based on the mask image and the trained image generation model; The supplementary two-dimensional color image is used as the newly added two-dimensional color image of the grassland. The process then returns to the step of determining the color category of the pixels in the two-dimensional color image of the grassland based on its color features, and the subsequent steps.

5. The method for constructing a 3D grassland scene based on image color according to claim 4, characterized in that, The step of generating a supplementary two-dimensional color image based on the mask image and the trained image generation model includes: Based on preset random offset parameters, the position offset processing of the aggregated pixel region in the mask image is performed to obtain at least one mask supplement image corresponding to the mask image; The mask image and the mask supplement image are respectively input into the image generation model to obtain the two-dimensional color images output by the image generation model.

6. A three-dimensional grassland scene construction system based on image color, characterized in that, The system includes: The data acquisition module is used to acquire two-dimensional color images of the grassland. The color category determination module is used to determine the color category corresponding to the pixels in the two-dimensional color image of the grassland based on the color features of the grassland image. A mask generation module is used to generate at least one mask image corresponding to the two-dimensional color image of the grassland according to the color category, wherein one mask image corresponds to one color category; The parameter determination module is used to determine the set of model instance attribute parameters corresponding to the color category based on the mask image; The rendering module is used to instantiate and render the model instance based on the set of attribute parameters, thereby generating a 3D grassland scene. The parameter determination module is specifically used for: obtaining the sliding window size; dividing the mask image according to the sliding window size to obtain multiple processing blocks corresponding to the mask image; determining target processing blocks according to the number of pixels in each processing block and a preset pixel threshold; determining model instance attribute parameters corresponding to each target processing block according to the pixels in the target processing block; and determining the set of model instance attribute parameters corresponding to the mask image according to all target processing blocks corresponding to the mask image and the model instance attribute parameters corresponding to the target processing blocks. The model instance attribute parameters include position parameters, rotation parameters, scaling parameters, and color parameters; The parameter determination module is further specifically used for: performing aggregation processing on the pixels in the target processing block to obtain an aggregated pixel region, and determining the minimum circumcircle of the aggregated pixel region; determining the position parameter based on the center point of the minimum circumcircle; determining the main direction corresponding to the aggregated pixel region based on a preset starting direction and the intersection of the minimum circumcircle and the aggregated pixel region, and determining the rotation parameter based on the main direction; determining the scaling parameter based on the radius of the minimum circumcircle; and determining the color parameter based on the average color value of all pixels in the aggregated pixel region.

7. A terminal, characterized in that, The terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the steps of the image color-based three-dimensional grassland scene construction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image color-based three-dimensional grassland scene construction method as described in any one of claims 1 to 5.

Images