Three-dimensional modeling method, device and computer program product

Abstract

The embodiment of the invention discloses a three-dimensional modeling method and device and a computer program product. According to the main technical scheme, the method comprises the steps that a target geographic entity is labeled on the basis of an image containing the target geographic entity, labeling information of the target geographic entity is obtained, and the labeling information at least comprises geometric parameters and surface parameters; constructing a shape model of the target geographic entity based on the geometric parameters; based on the surface parameters, obtaining component models corresponding to components contained in the target geographic entity from a component model library; assembling the obtained part model and the shape model to obtain an initial model; and obtaining a three-dimensional model of the target geographic entity based on the initial model. According to the method, a parameterized modeling mode is adopted, generation of the three-dimensional model is restrained based on explicit geometric parameters and surface parameters, it is guaranteed that the structure and details of the building better conform to the real physical law, the whole modeling process is automatically achieved, and the modeling efficiency is greatly improved.

Description

Technical Field

[0001] This application relates to the field of geographic information technology, and in particular to a three-dimensional modeling method, apparatus and computer program product. Background Technology

[0002] With the acceleration of urbanization, people's demand for more realistic map representations is becoming increasingly strong. Traditional maps mainly rely on 2D (two-dimensional) representation and relatively simple 3D (three-dimensional) effects. While this method can present geographical locations, it often fails to provide detailed architectural features, spatial perception, and a stereoscopic visual experience, resulting in poor intuitiveness and realism. Therefore, achieving a realistic three-dimensional representation of the real world on maps has become a problem that various map service providers are striving to solve. Summary of the Invention

[0003] In view of this, this application provides a three-dimensional modeling method, apparatus and computer program product, providing a foundation for realizing a realistic three-dimensional reconstruction of the real world on a map.

[0004] This application provides the following solution: According to the first aspect, a three-dimensional modeling method is provided, the method comprising: The target geographic entity is labeled based on an image containing the target geographic entity to obtain the labeling information of the target geographic entity, and the labeling information includes at least geometric parameters and surface parameters; Based on the geometric parameters, construct a shape model of the target geographic entity; Based on the surface parameters, obtain the component models corresponding to the components contained in the target geographic entity from the component model library; The acquired component models are assembled with the shape models to obtain the initial model; Based on the initial model, a three-dimensional model of the target geographic entity is obtained.

[0005] According to a second aspect, a three-dimensional modeling apparatus is provided, the apparatus comprising: The annotation unit is configured to annotate the target geographic entity based on an image containing the target geographic entity, thereby obtaining annotation information of the target geographic entity, wherein the annotation information includes at least geometric parameters and surface parameters; The building unit is configured to build a shape model of the target geographic entity based on the geometric parameters; The acquisition unit is configured to acquire, based on the surface parameters, the component models corresponding to the components contained in the target geographic entity from the component model library; An assembly unit is configured to assemble the acquired component model with the shape model to obtain an initial model; The post-processing unit is configured to obtain a three-dimensional model of the target geographic entity based on the initial model.

[0006] According to a third aspect, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0007] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application employs a parametric modeling approach. Based on images containing the target geographic entities, the target geographic entities are annotated to obtain geometric and surface parameters. These explicit geometric and surface parameters then constrain the generation of the 3D model. Specifically, geometric parameters constrain the model's shape, while surface parameters assemble the component models corresponding to the parts contained within the target geographic entity. This ensures that the generated 3D model conforms to the basic structure of buildings in the real world. Compared to traditional methods of generating 3D models based on large generative models, this approach guarantees that the architecture conforms more closely to real physical laws in terms of structure and detail. Furthermore, the entire 3D modeling process can be automated, significantly improving modeling efficiency compared to traditional manual modeling using 3D modeling software. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A flowchart of a three-dimensional modeling method provided in an embodiment of this application.

[0010] Figure 2 This is a schematic diagram of the coloring process provided in an embodiment of this application.

[0011] Figure 3 The diagram shows a comparison of the effects of this application and traditional modeling methods in the embodiments of this application.

[0012] Figure 4 The image shows the effect of applying the 3D modeling provided in the embodiments of this application to the display of a 3D map.

[0013] Figure 5 A schematic block diagram of a three-dimensional modeling apparatus provided in an embodiment of this application.

[0014] Figure 6 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0016] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0017] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0018] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0019] Currently, there are several technologies available for creating 3D models of real-world buildings, primarily including the following two methods: One approach is to manually create 3D building models using 3D modeling software. This method is inefficient, especially for repetitive modeling of similar elements.

[0020] Another approach is to use a generative large model, which involves inputting a 2D image containing the building into the generative large model, which then generates a 3D building model. However, currently, the 3D building models generated by generative large models differ significantly from real buildings, resulting in uncontrollable generation quality and poor editability.

[0021] In view of this, this application provides a new approach. Figure 1 A flowchart illustrating the three-dimensional modeling method provided in this application embodiment. Figure 1 As shown, the method may include the following steps: Step 101: Annotate the target geographic entity based on the image containing the target geographic entity to obtain the annotation information of the target geographic entity. The annotation information includes at least geometric parameters and surface parameters.

[0022] Step 102: Construct a shape model of the target geographic entity based on geometric parameters.

[0023] Step 103: Based on surface parameters, obtain the component models corresponding to the components contained in the target geographic entity from the component model library.

[0024] Step 104: Assemble the acquired component model and shape model to obtain the initial model.

[0025] Step 105: Based on the initial model, obtain the three-dimensional model of the target geographic entity.

[0026] As can be seen from the above process, this application adopts a parametric modeling approach. Based on images containing the target geographic entities, the target geographic entities are annotated to obtain geometric and surface parameters. This allows for the explicit constraint of 3D model generation based on these explicit geometric and surface parameters. Specifically, geometric parameters constrain the model's shape, while surface parameters assemble the component models corresponding to the parts contained within the target geographic entity. This ensures that the generated 3D models conform to the basic structures of buildings in the real world. Compared to traditional methods of generating 3D models based on large generative models, this approach guarantees that the architecture conforms more closely to real physical laws in terms of structure and detail. Furthermore, the entire 3D modeling process can be automated, significantly improving modeling efficiency compared to traditional manual modeling using 3D modeling software.

[0027] It should be noted that the 3D modeling method provided in this application embodiment can be executed by a 3D modeling device installed on a server. This server can be a single server or a server cluster consisting of multiple servers. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system. This addresses the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak service scalability. Besides being installed on a server, the 3D modeling device can also be installed on a computer terminal device with strong computing capabilities.

[0028] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments. It should be noted that the terms "first" and "second" used in this disclosure do not have limitations in terms of size, order, or quantity, but are only used to distinguish them by name. For example, "first surface parameter" and "second surface parameter" are only used to distinguish two types of surfaces by name.

[0029] First, the above step 101, namely "annotating the target geographic entity based on the image containing the target geographic entity to obtain the annotation information of the target geographic entity, wherein the annotation information includes at least geometric parameters and surface parameters", will be described in detail with reference to the embodiments.

[0030] The images containing target geographic entities involved in the embodiments of this application may include, but are not limited to, satellite images, street view images, aerial images, etc.

[0031] Satellite imagery can clearly display the Earth's surface topography, such as mountains, rivers, lakes, oceans, urban layouts, and road networks. In this embodiment, satellite imagery can be used for 3D modeling of geographic entities. Satellite imagery can be acquired through various means, such as by different satellite platforms or image acquisition devices like sensors.

[0032] The geographical entities involved in the embodiments of this application mainly include buildings. In addition to buildings, they may also include transportation facilities such as bridges and platforms, as well as public facilities, landscape and cultural facilities, and so on.

[0033] In the embodiments of this application, one or more geographic entities in the image can be designated as target geographic entities for modeling, or one type of geographic entity can be designated as target geographic entities for modeling, or all geographic entities in the image can be designated as target geographic entities for modeling, and so on.

[0034] In this step, the target geographic entities can be labeled using, but are not limited to, the following methods: The first method involves using an entity segmentation model to identify the base shape of the target geographic entity contained in the image; inputting the image and the base shape of the target geographic entity into a VLLM (Visual-Language Large Model) to obtain the geometric parameters and first surface parameters of the target geographic entity. The geometric parameters include the shape and height of the target geographic entity, and the first surface parameters include the parameters of the top component of the target geographic entity.

[0035] This approach is suitable for images with a certain degree of overhead view, such as satellite images and aerial photographs. First, an entity segmentation model can be used to identify the base shape of the target entities contained in the image. This entity segmentation model can be a Transformer-based model. The base shape refers to the planar contour of the bottom portion of the target geographic entity (e.g., a building) that contacts the ground; typically, it refers to the vertical projection of the target geographic entity onto the ground plane in an overhead view. The base shape can be represented using a mask. Then, the image containing the target geographic entity and the identified base shape are input into a Virtual LLM (Virtual LLM Model). The Virtual LLM uses an autoregressive approach to predict the geometric parameters and first surface parameters of the target geographic entity.

[0036] Taking a building as an example, the aforementioned geometric parameters and first surface parameters can be predicted on a building-by-building basis, or a single building can be divided into multiple independent sub-units according to specific rules. Each sub-unit has clear spatial boundaries and functional attributes. These sub-units are usually called building blocks, and the aforementioned geometric parameters and first surface parameters can be predicted on a building block basis. For example, if the lower floors of a mixed-use building are a shopping mall and the upper floors are office buildings, then the office buildings and the shopping mall can be divided into different building blocks.

[0037] The aforementioned geometric parameters are primarily used to describe the geometric state of the target geographic entity, mainly including shape and height. The shape can be expressed using polygons. The first surface parameters mainly include parameters of the top components, such as the shape and color of the roof. The shape of the roof can include flat roofs, flat slopes, pointed roofs, spherical surfaces, sloping surfaces, etc.

[0038] The aforementioned entity segmentation model and VLLM can be fine-tuned using training samples constructed with manual annotations. For example, target geographic entities can be annotated on satellite images, aerial images, etc., and the geometric parameters such as the base shape, the shape and height of the target geographic entity, and the top component parameters such as the shape and color of the top can be annotated to obtain training samples. Then, the entity segmentation model and VLLM can be trained using the training samples.

[0039] The second method involves using VLLM to identify the facades of target geographic entities contained in the image, thereby obtaining second surface parameters. These second surface parameters include the layout of facade components and the parameters of each facade component.

[0040] This approach is suitable for images with a certain side-view angle, such as satellite images, aerial images, and street view images. In this approach, image segmentation models can be used to first identify the facade of the target geographic entity. Then, the facade area is input into a Virtual LLM (Very Large Scale Model), which outputs the second surface parameters of the target geographic entity.

[0041] Taking buildings as an example, the facade of a single building or building block may contain multiple facade components, such as ground-floor shops, windows, doors, wall protrusions and recesses, columns, curtain walls, and cornices. These facade components can be used to refine the model of the target geographic entity during the 3D modeling process. The layout of facade components mainly includes information such as the spacing, order, and position of the components. Facade component parameters can include information such as the type, shape, size, and color of the facade component. The type of facade component can be selected with coarser granularity, such as windows, curtain walls, and cornices, or with finer granularity; for example, curtain walls can be further divided into horizontal curtain walls, vertical curtain walls, and uniform curtain walls.

[0042] The VLLM used in this approach can be fine-tuned using training samples constructed with manual annotations. For example, after identifying the facade of the target geographic entity on images such as satellite images and aerial images, the facade is annotated to mark the local parts of the facade components and the parameters of each facade component, thus obtaining training samples. Then, the VLLM is trained using the training samples.

[0043] The above methods enable the annotation of target geographic entities using entity segmentation models and VLLM with powerful understanding capabilities, yielding geometric and surface parameters. This automated annotation method improves annotation efficiency, making large-scale geographic entity annotation possible and providing a foundation for large-scale geographic entity modeling in map applications. Furthermore, this annotation method allows for the acquisition of the geometric structure and component details of target geographic entities in the real world, facilitating the subsequent creation of more accurate and detailed 3D models that conform to reality.

[0044] After the above annotation process, a large amount of annotation data can be obtained, namely the geometric and surface parameters of a large number of geographic entities. This annotation data can be stored in a database as a dataset for querying and retrieval, in order to execute subsequent processes to build a 3D model of the target geographic entity.

[0045] In addition, these annotation information can be preprocessed, such as through cross-validation, detail enhancement, integration, and standardization. Since a target geographic entity may exist in multiple images, multiple annotations may be obtained. These multiple annotations corresponding to a single target geographic entity can be cross-validated, and the correctly validated annotations are then stored. Furthermore, multiple annotations corresponding to the same target geographic entity can be used for detail enhancement or integration, resulting in more refined annotation information.

[0046] The following describes step 102, namely "constructing a shape model of the target geographic entity based on geometric parameters", in detail with reference to the embodiments.

[0047] This step primarily utilizes the shape and height of the target geographic entity to generate a shape model of that entity. This shape model represents the main structure of the building, giving it an outline consistent with its real-world counterpart. The shape model can be automatically generated using 3D modeling software. In addition, modeling users can adjust certain geometric parameters to control key dimensions and forms of the model, such as the building's height (down to the floor height and number of floors), top shape, and color. This approach makes the model generation process more flexible, allowing for rapid adjustments based on different parameters.

[0048] The following describes in detail, with reference to the embodiments, step 103, namely "based on surface parameters, obtain the component model corresponding to the component contained in the target geographic entity from the component model library" and step 104, namely "assemble the obtained component model with the shape model to obtain the initial model".

[0049] These two steps are used to further add detailed components to the shape model that represents the main structure, enriching and refining the model's appearance. The components contained in a target geographic entity can usually be determined based on surface parameters. Taking a common building as an example, its components mainly include a roof and facade. For other types of target geographic entities, there may be no roof component, only a facade component. There may also be some target geographic entities that only include a roof component and not a facade component.

[0050] In this embodiment, a component model library can be pre-built, that is, component models can be pre-suggested based on common components, and these component models have the same geometric structure as the corresponding components. The component models in the component model library can correspond to component types, which can be either coarse-grained or fine-grained.

[0051] One feasible approach is to retrieve the component models corresponding to the top components of the target geographic entity from a component model library based on the parameters of the top components of the target geographic entity. For example, based on the type of the top component, the component model corresponding to that type can be selected from the component model library. For example, based on the type of roof, such as flat roof, flat slope, pointed roof, spherical roof, sloping roof, etc., the corresponding roof model can be retrieved.

[0052] As another feasible approach, component models corresponding to the facade components of the target geographic entity can be obtained from a component model library based on the facade component parameters of the target geographic entity. For example, based on the type of facade component, the component model corresponding to that type can be selected from the component model library. For instance, based on the type of window, door, eaves, column, curtain wall, etc., the corresponding window model, door model, eaves model, column model, curtain wall type, etc., can be obtained.

[0053] By acquiring the component models corresponding to the top and facade components, the 3D modeling of the target geographic entity can be improved in terms of shape detail, thus making it closer to the effect in the real world. Furthermore, by integrating these components and the main structure into a whole model, a component-level management granularity is constructed, which allows for the reuse of components while ensuring the consistency and scalability of the model.

[0054] In addition, multiple modeling standards can be used for geographic entities during the 3D modeling process. In this step, in addition to obtaining the component model corresponding to the target component based on the surface parameters mentioned above, we can further combine the modeling standard of the target geographic entity, that is, based on the surface parameters and the modeling standard of the target geographic entity, determine the target component that is compatible with the modeling standard from the components contained in the target geographic entity; and obtain the component model that is compatible with the modeling standard from the component model corresponding to the target component in the component model library.

[0055] To facilitate understanding, we will first provide a simple description of the modeling standards for geographic entities. Modeling standards can be divided into several levels; generally, the higher the level, the more detail is in the corresponding 3D model. Below is one example of how modeling standards are divided, into the following 5 levels: Level L1: Only the main structure is presented, with no components, and geographical entities are presented in monochrome.

[0056] Level L2: Based on the main structure, simple styles are added (e.g., rounded corners), simple top components are distinguished, such as sloping roofs, and geographic entities are presented in a monochrome effect. The 3D model has a fidelity of 0.2 to real-world geographic entities.

[0057] Level L3: Based on the main structure, additional component models describing the top details are added. The details of the top components are more refined, and the geographic entities are presented in a monochrome effect.

[0058] Level L3+: Based on the main structure, additional elements such as windows and steps are added. Geographic entities are colored. Light source information is added.

[0059] Level 4: Based on the main structure, add texture maps, apply gradient colors based on the structure, and ensure smooth transitions. Add light source information.

[0060] It can be seen that different modeling standards vary in the number and granularity of components added to geographic entities. Therefore, when obtaining component models for a target geographic entity, one can identify the target component that matches the modeling standard from the components contained in the target geographic entity, and then retrieve the corresponding component model from the component model library. For example, if the target geographic entity adopts an L2-level modeling standard, only the top component is considered as the target component, and the corresponding L2-level standard component model is retrieved from the component model library. This approach forms a standardized specification, which facilitates the reuse of standardized component models and allows for flexible control of the modeling granularity of the target geographic entity according to actual scenario requirements.

[0061] After obtaining the component models, the component models can be assembled with the shape models based on the layout of the facade components obtained in step 101 (e.g., the spacing, order, and position of the components). For example, window models can be assembled on the shape models according to the positions of the windows. Alternatively, window models can be assembled on the shape models according to the spacing and order of the windows. The assembly involved in this step is essentially a geometric merging of the component models and the shape models, thereby forming a complete model in terms of geometric structure.

[0062] The following describes step 105, namely "obtaining a three-dimensional model of the target geographic entity based on the initial model," in detail with reference to an embodiment.

[0063] After obtaining the initial model, it can be directly used as the 3D model of the target geographic entity. Alternatively, the initial model can be further refined to obtain the 3D model of the target geographic entity, where refinement may include at least one of style transfer, texture mapping, and shading.

[0064] One possible approach is to perform detailed processing on the initial model according to a preset method, such as applying texture mapping to all geographic entities or specific geographic entities, or performing style transfer and coloring on all geographic entities or specific geographic entities, and so on.

[0065] As another feasible approach, the initial model can be processed with details adapted to the modeling standards of the target geographic entity to obtain a 3D model of the target geographic entity. For example, if the target geographic entity is at level L3+, the initial model can be shaded. As another example, if the target geographic entity is at level L4, texture mapping and shading can be applied. This method allows the modeling standards of the target geographic entity to be determined according to the actual scene requirements, thereby flexibly controlling the modeling detail and effect richness of the target geographic entity.

[0066] The modeling standard for the target geographic entity can be set by the modeling user or use the default settings. Since a key application of geographic entity modeling is map display—that is, rendering and displaying various geographic entities using 3D models within a 3D map context—a preferred approach is to determine the modeling standard based on at least one of the following: the popularity of the geographic region where the target entity is located, the type of the target geographic entity, and the distance between the target geographic entity and the observation point on the map. The closer the target geographic entity is to the observation point on the map, the higher the modeling standard. For example, a higher modeling standard is used for geographic entities in hot cities. Another example is using a higher modeling standard for landmark buildings and a lower modeling standard for residential buildings. Furthermore, when displaying a map, geographic entities closer to the observation point use a higher modeling standard, while those farther away use a lower modeling standard. The observation point refers to the point set when presenting spatial map information. This point determines the "starting point" of the observation perspective, directly affecting the visible range and relative positional relationships of the displayed map. Examples of this display effect will be provided in subsequent embodiments.

[0067] The following sections will describe in detail the processing of the above-mentioned details.

[0068] Style transition: Style transfer refers to adjusting the visual style of an initial model while keeping its basic geometric structure unchanged, so that it better suits the aesthetic requirements of a specific scene.

[0069] When performing style transfer, the three-dimensional surface of the initial model can be unfolded and projected onto a two-dimensional plane to obtain the first surface image. Then, based on the transfer strategy corresponding to the target style or using a style transfer model, the geometry and / or style of some elements of the first surface image can be adjusted to obtain the second surface image. The second surface image can then be mapped back to the three-dimensional space where the initial model is located.

[0070] When unfolding the 3D surface of the initial model and projecting it onto a 2D plane, UV coordinates can be generated for the initial model. UV coordinates are essentially a mapping representation of the model surface on a 2D plane, where "U" and "V" represent the horizontal and vertical axes of the 2D plane, respectively, and are usually used to describe the corresponding positions of the model surface vertices in 3D space on the 2D image.

[0071] When using transformation strategies to adjust the geometry and / or style of some elements in the first surface image, some relatively simple changes or replacements can be made. For example, the scale of the initial model can be fine-tuned to better match the architectural characteristics of a specific region or era. Another example is changing the corners of the initial model from right angles to rounded corners, adding carvings to the walls, etc.

[0072] When using style transfer models to adjust the geometry and / or style of some elements of a first surface image, more complex changes or replacements can be made. For example, a style transfer model based on a Transformer network can change the first surface image from a realistic style to a cartoon style, retro style, etc.

[0073] Besides the methods mentioned above, for simple adjustments during style transfer, adjustments can also be made directly on the 3D surface of the initial model. Due to the parametric modeling approach described above, style transfer strategies or models can be used to switch styles without rebuilding the model, resulting in higher efficiency and lower cost.

[0074] Texture mapping: Texture mapping refers to the technique of adding details, materials, colors, and visual effects to the surface of a 3D model using 2D images. It allows the model to present richer visual details and enhance realism without increasing geometric complexity.

[0075] When creating a texture map, you can first unfold the 3D surface of the initial model and project it onto a 2D plane to obtain the first surface image; then obtain texture materials based on the surface parameters; finally, map the texture materials to the corresponding areas in the first surface image to obtain the third surface image; and finally, map the third surface image back to the 3D space where the initial model is located.

[0076] When obtaining texture materials based on surface parameters, the following methods can be used, but are not limited to: The first method: Extract texture materials from the above images containing target geographic entities based on surface parameters.

[0077] For example, based on the parameters of the facade components in the surface parameters, the image area corresponding to the facade component is determined; after removing the background and adjusting the distortion of the image area corresponding to the facade component, it is converted into texture material.

[0078] The second method: Based on surface parameters, use a program to generate texture materials.

[0079] For example, based on the parameters of the facade components in the surface parameters, such as the type of facade components, some modeling programs are used to generate texture materials.

[0080] The third method is to obtain the corresponding texture material from the material library based on the surface parameters. The texture material in the material library is pre-drawn by hand.

[0081] For example, texture materials can be pre-drawn using modeling programs for some common types of parts, and these texture materials can be stored in a material library along with the part types. During the 3D modeling process, the corresponding texture materials are retrieved from the material library based on the part type.

[0082] When mapping texture materials to the corresponding areas in the first surface image, the mapping of texture materials is mainly based on the UV coordinates of the vertices of the model surface. The specific method of using UV coordinates for mapping is an existing technology and will not be described in detail here.

[0083] As can be seen, obtaining texture materials based on surface parameters and then applying texture mapping to the model is a parametric texture implementation method. On the one hand, it breaks the limitations of traditional manual mapping and is more efficient. On the other hand, it enables large-scale reuse of texture materials based on surface parameters. Furthermore, applying texture mapping based on the surface parameters of the target geographic entity essentially achieves accurate mapping based on real physical attributes, thereby greatly improving visual realism.

[0084] Coloring process: Since there are a vast number of colors in the real world, using the actual colors of the target geographic entities for coloring would require pre-storing a large number of color values ​​(tens of thousands), and the final visual effect would be poor, such as inconsistent style or a chaotic appearance due to the large number of colors. Therefore, this application provides a preferred implementation method: obtaining the color region and color type of the target geographic entity from surface parameters; determining the target color value for color type mapping based on a preset mapping strategy; and assigning color values ​​to the surface corresponding to the color region in the initial model using the target color values.

[0085] In other words, the specific colors of target geographic entities are categorized into color types. For example, they are first categorized into color families, such as blue, black, and red, with each family further subdivided into dark blue, medium blue, light blue, etc. The preset mapping strategy maps color types to a single color value, for example, mapping a specific gradient color range to a single color value. Based on this mapping strategy, the target color value for each color type can be determined. In this way, the model, to a certain extent, reproduces colors from the real world while maintaining color consistency and improving aesthetics.

[0086] For example, such as Figure 2 The real-world images shown in (a) contain a wide variety of colors; for example, buildings contain more than just a few colors. Figure 2These are shown in (b). In the embodiments of this application, colors are pre-categorized into color systems, such as dark blue, medium blue, light blue, dark gray, and medium gray. Taking gray as an example, ground-floor shops, building facades, and rooftops are mapped to the same color value, i.e., the color value corresponding to light gray; ground-floor shop windows and building windows are mapped to the same color value, i.e., the color value corresponding to dark gray, such as... Figure 2 As shown in (c). Then, the mapped color values ​​are used to match the corresponding surfaces on the model.

[0087] It should be noted that different modes can be set for the model, such as day mode and night mode. Different color values ​​can be mapped to day mode and night mode to achieve different effects, such as... Figure 2 As shown in (d).

[0088] It should be noted that the above coloring process can be performed directly in three-dimensional space, or on the first surface image obtained by unfolding the initial model and mapping it to two-dimensional space.

[0089] It should be noted that the above-mentioned detailed processing methods can be used individually or in any combination. When used in combination, such as combining style transfer, texture mapping, and shading, the 3D surface of the initial model can be unfolded and projected onto a 2D plane to obtain the first surface image. Then, based on the surface parameters, style transfer, texture mapping, and shading are performed on the first surface image. Finally, the processed surface image is mapped back into 3D space to obtain the final 3D model.

[0090] The methods provided in this application embodiment can be applied to various application scenarios, including but not limited to: Virtual scene display: For example, a user can send a scene display request for a specific building to the server. The server returns the 3D model of the building to the user, who then uses the 3D model to render and display the building, allowing the user to see a virtual 3D image of the building.

[0091] 3D map display: For example, when a user requests to view a 3D map or navigate to the map, the visible range of the map can be determined based on the current observation point, and the 3D models of the geographical entities within that visible range can be rendered to present the 3D visual effect of the geographical entities on the map.

[0092] The following describes the 3D modeling effect achieved using the method provided in the embodiments of this application, utilizing satellite imagery and a street view of a building. If traditional modeling methods are used, the resulting 3D model only has a basic shape, such as... Figure 3 As shown in (a). However, the three-dimensional model obtained by modeling using the method provided in the embodiments of this application has more structural details, such as... Figure 3As shown in (b), details such as ground floor shops, rooftops, eaves, windows, and doors are shown, and the proportions and shapes of these details are more in line with the physical laws of the real world.

[0093] For example, the modeling method provided in the embodiments of this application, when applied to 3D map display, can achieve the following effect: Figure 4 As shown. Figure 4 In the images, (a) and (b) show the effects of daytime and nighttime modes, respectively, with differences in coloring between the two modes. Furthermore, when applied to 3D map display, the closer to the observation point, the higher the corresponding modeling standard. For example... Figure 4 As shown in (a), building B, which is farther from the observation point, uses a lower modeling standard, for example, only showing the basic main structure (i.e., shape model); building A, which is closer to the observation point, uses a higher modeling standard, for example, having components such as ground floor shops, rooftops, eaves, windows, and doors, and having certain textures and shading. Figure 4 As shown in (b), a higher modeling standard is used for the landmark building C, even though it is far from the observation point, so as to show more details, such as component shapes, textures and shading.

[0094] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0095] Figure 5 A schematic block diagram of the three-dimensional modeling apparatus provided in the embodiments of this application, such as Figure 5 As shown, the device 500 includes: a labeling unit 501, a construction unit 502, an acquisition unit 503, an assembly unit 504, and a post-processing unit 505. The main functions of each component are as follows: The annotation unit 501 is configured to annotate the target geographic entity based on an image containing the target geographic entity, thereby obtaining annotation information of the target geographic entity, wherein the annotation information includes at least geometric parameters and surface parameters.

[0096] Construction unit 502 is configured to construct a shape model of the target geographic entity based on the geometric parameters.

[0097] The acquisition unit 503 is configured to acquire, based on the surface parameters, the component model corresponding to the component contained in the target geographic entity from the component model library.

[0098] Assembly unit 504 is configured to assemble the acquired component model with the shape model to obtain an initial model.

[0099] The post-processing unit 505 is configured to obtain a three-dimensional model of the target geographic entity based on the initial model.

[0100] As one possible implementation, the annotation unit 501 can be specifically configured to: identify the base shape of the target geographic entity contained in the image using an entity segmentation model; input the image and the base shape of the target geographic entity into a large visual model to obtain the geometric parameters and first surface parameters of the target geographic entity, wherein the geometric parameters include the shape and height of the target geographic entity, and the first surface parameters include the top component parameters of the target geographic entity.

[0101] As another possible approach, the annotation unit 501 can be specifically configured to: use a large visual model to identify the facade of the target geographic entity contained in the image to obtain a second surface parameter, which includes the layout of the facade components and the parameters of each facade component.

[0102] You can choose to execute one of the two methods mentioned above, or you can execute both.

[0103] As one possible implementation method, the acquisition unit 503 can be specifically configured to: acquire the component model corresponding to the top component contained in the target geographic entity from the component model library based on the top component parameters of the target geographic entity; and / or, acquire the component model corresponding to the facade component of the target geographic entity from the component model library based on the facade component parameters of the target geographic entity.

[0104] As one possible implementation method, the acquisition unit 503 can be specifically configured to: determine the target component that is compatible with the modeling standard from the components contained in the target geographic entity based on surface parameters and the modeling standard of the target geographic entity; and acquire the component model that is compatible with the modeling standard from the component model corresponding to the target component in the component model library.

[0105] As one possible implementation, the post-processing unit 505 can be specifically configured to: perform detailed processing on the initial model in accordance with the modeling standard of the target geographic entity to obtain a three-dimensional model of the target geographic entity, wherein the detailed processing includes at least one of style transfer, texture mapping and shading.

[0106] The modeling standard for the target geographic entity is determined based on at least one of the following: the heat of the geographic region where the target geographic entity is located, the type of the target geographic entity, and the distance between the target geographic entity and the observation point on the map. The closer the target geographic entity is to the observation point on the map, the higher the modeling standard for the target geographic entity.

[0107] As one possible implementation, the post-processing unit 505 can be specifically configured to perform style transfer as follows: The three-dimensional surface of the initial model is unfolded and projected onto a two-dimensional plane to obtain the first surface image; Based on the conversion strategy corresponding to the target style or using a style conversion model, the geometry and / or style of some elements of the first surface image are adjusted to obtain the second surface image; The second surface image is mapped back to the three-dimensional space where the initial model is located.

[0108] As one possible implementation, the post-processing unit 505 can be specifically configured during texture mapping as follows: The three-dimensional surface of the initial model is unfolded and projected onto a two-dimensional plane to obtain the first surface image; Texture materials are obtained based on surface parameters; The texture material is mapped to the corresponding area in the first surface image to obtain the third surface image; Map the third surface image back to the three-dimensional space where the initial model is located.

[0109] As one possible implementation, the post-processing unit 505 can be specifically configured to perform coloring as follows: Obtain the color region of the target geographic entity and the color type of the color region from surface parameters; Based on the preset mapping strategy, determine the target color value for color type mapping; Using the target color value, the surface corresponding to the color region in the initial model is assigned a color value.

[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. The system and device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0111] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0112] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0113] And an electronic device, comprising: One or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0114] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0115] in, Figure 6 An exemplary architecture of an electronic device is shown, which may include a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, and a memory 620. The processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, and memory 620 can communicate with each other via a communication bus 630.

[0116] The processor 610 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0117] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store the operating system 621 for controlling the operation of the electronic device 600, and the basic input / output system (BIOS) 622 for controlling the low-level operations of the electronic device 600. Additionally, it can store a web browser 623, a data storage management system 624, and a 3D modeling device 500, etc. The aforementioned 3D modeling device 500 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 620 and executed by the processor 610.

[0118] Input / output interface 613 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0119] Network interface 614 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0120] Bus 630 includes a pathway for transmitting information between various components of the device, such as processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, and memory 620.

[0121] It should be noted that although the above-described device only shows the processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, memory 620, bus 630, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0122] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0123] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A three-dimensional modeling method, characterized in that, The method includes: The target geographic entity is labeled based on an image containing the target geographic entity to obtain the labeling information of the target geographic entity, and the labeling information includes at least geometric parameters and surface parameters; Based on the geometric parameters, construct a shape model of the target geographic entity; Based on the surface parameters, obtain the component models corresponding to the components contained in the target geographic entity from the component model library; The acquired component models are assembled with the shape models to obtain the initial model; Based on the initial model, a three-dimensional model of the target geographic entity is obtained.

2. The method according to claim 1, characterized in that, The step of annotating the target geographic entity based on an image containing the target geographic entity to obtain the annotation information of the target geographic entity includes: The base shape of the target geographic entity contained in the image is identified using an entity segmentation model; The image and the base shape of the target geographic entity are input into a large visual model to obtain the geometric parameters and first surface parameters of the target geographic entity. The geometric parameters include the shape and height of the target geographic entity, and the first surface parameters include the top component parameters of the target geographic entity.

3. The method according to claim 1, characterized in that, The step of annotating the target geographic entity based on an image containing the target geographic entity to obtain the annotation information of the target geographic entity includes: The facade of the target geographic entity contained in the image is identified using a large visual model to obtain a second surface parameter, which includes the layout of facade components and parameters of each facade component.

4. The method according to claim 1, characterized in that, The surface parameters include a first surface parameter and / or a second surface parameter. The first surface parameter includes the top component parameter of the target geographic entity, and the second surface parameter includes the layout of the exterior components of the target geographic entity and the parameters of each exterior component. Based on the surface parameters, obtaining the component models corresponding to the components contained in the target geographic entity from the component model library includes: Based on the top component parameters of the target geographic entity, obtain the component model corresponding to the top component contained in the target geographic entity from the component model library; And / or, Based on the exterior component parameters of the target geographic entity, obtain the component model corresponding to the exterior component of the target geographic entity from the component model library.

5. The method according to claim 1, characterized in that, Based on the surface parameters, the component models corresponding to the facade components of the target geographic entity are obtained from the component model library, including: Based on the surface parameters and the modeling criteria of the target geographic entity, target components that are compatible with the modeling criteria are determined from the components contained in the target geographic entity; Obtain a component model that is compatible with the modeling standard from the component models corresponding to the target component in the component model library.

6. The method according to claim 1, wherein obtaining a three-dimensional model of the target geographic entity based on the initial model comprises: Based on the modeling standard of the target geographic entity, the initial model is subjected to detail processing adapted to the modeling standard to obtain a three-dimensional model of the target geographic entity. The detail processing includes at least one of style transfer, texture mapping, and shading.

7. The method according to claim 6, characterized in that, The style transfer includes: The three-dimensional surface of the initial model is unfolded and projected onto a two-dimensional plane to obtain a first surface image; Based on the conversion strategy corresponding to the target style or using a style conversion model, the geometry and / or style of some elements of the first surface image are adjusted to obtain the second surface image. The second surface image is mapped back to the three-dimensional space where the initial model is located.

8. The method according to claim 6, characterized in that, The texture map includes: The three-dimensional surface of the initial model is unfolded and projected onto a two-dimensional plane to obtain a first surface image; Texture materials are obtained based on the surface parameters; The texture material is mapped to the corresponding area in the first surface image to obtain the third surface image; The third surface image is mapped back to the three-dimensional space where the initial model is located.

9. The method according to claim 6, characterized in that, The coloring includes: The color region and color type of the target geographic entity are obtained from the surface parameters. Based on a preset mapping strategy, determine the target color value mapped to the color type; Using the target color value, the surface corresponding to the color region in the initial model is assigned a color value.

10. A three-dimensional modeling device, characterized in that, The device includes: The annotation unit is configured to annotate the target geographic entity based on an image containing the target geographic entity, thereby obtaining annotation information of the target geographic entity, wherein the annotation information includes at least geometric parameters and surface parameters; The building unit is configured to build a shape model of the target geographic entity based on the geometric parameters; The acquisition unit is configured to acquire, based on the surface parameters, the component models corresponding to the components contained in the target geographic entity from the component model library; An assembly unit is configured to assemble the acquired component model with the shape model to obtain an initial model; The post-processing unit is configured to obtain a three-dimensional model of the target geographic entity based on the initial model.

11. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 9.

Images