Information processing apparatus, method, and program

Abstract

To provide a technique capable of generating a three dimensional model with high visual quality and customizability.SOLUTION: A program according to one aspect of the present disclosure causes a computer to function as means for acquiring image information including a photograph obtained by photographing a geographic element existing on a ground surface, means for generating attribute information of the geographic element corresponding to model input data by applying an attribute prediction model capable of predicting an attribute of the geographic element to the model input data based on the image information, means for generating a geographic space database including the attribute information, and means for generating a three dimensional model based on the geographic space database.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a method, and a program. [Background technology]

[0002] Conventionally, techniques for generating city models or 3D maps based on satellite images have been proposed. Patent Document 1 discloses a method for generating a 3D point cloud based on satellite images, converting the 3D point cloud into a mesh, and pasting the latest satellite images onto the mesh. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-008802 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology disclosed in Patent Document 1, a three-dimensional model of a three-dimensional structure is generated by pasting a satellite image as a texture onto a three-dimensional mesh corresponding to the three-dimensional structure. The three-dimensional model generated in this way has several problems when used in content that requires high visual quality, such as movies, video games, VR (Virtual Reality) and the Metaverse.

[0005] First, the visual quality of a 3D model is significantly affected by the optical conditions and resolution of the satellite imagery. If satellite images captured under different optical conditions were applied as textures to the same 3D model, viewers might find it unnatural. Furthermore, when such a 3D model is placed in a virtual space, the model's appearance may become even more unnatural depending on the optical conditions set in the virtual space. For example, if noise such as shadows is present in the satellite imagery used as texture, the lighting effects of the time of day (morning / evening) or weather in the virtual space may not be reproduced as intended by the content creator. Furthermore, if low-resolution satellite images are used as textures, the roughness of the texture becomes noticeable when the texture is enlarged. Therefore, such 3D models may not meet the required visual quality, especially in content such as video games where players can freely observe the 3D model.

[0006] Second, because such 3D models are generated by applying textures to 3D meshes, even minor customizations, such as increasing or decreasing the number of floors in a building, are difficult to implement and are therefore inflexible. Furthermore, such 3D models are difficult to reproduce the textures specific to the building materials that make up a three-dimensional structure, especially under optical conditions that differ from those used when satellite images are taken.

[0007] Third, satellite images may contain people's faces, car license plates, signs, etc. Therefore, 3D models that use satellite images directly as textures may infringe on the rights of third parties, which may discourage content creators from using such 3D models.

[0008] An object of the present disclosure is to provide a technology that can generate three-dimensional models with high visual quality and customizability. [Means for solving the problem]

[0009] A program according to one aspect of the present disclosure causes a computer to function as a means for acquiring image information including photographs of geographic elements existing on the Earth's surface, a means for generating attribute information of geographic elements corresponding to model input data by applying an attribute prediction model capable of predicting attributes of the geographic elements to model input data based on the image information, a means for generating a geospatial database including the attribute information, and a means for generating a three-dimensional model based on the geospatial database. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of an information processing apparatus according to an embodiment of the present invention; [Figure 2] FIG. 1 is an explanatory diagram of one aspect of the present embodiment. [Figure 3] FIG. 2 is a diagram showing the data structure of a geospatial database of buildings according to the present embodiment. [Figure 4] FIG. 2 is a diagram showing the data structure of a geospatial database of land according to the present embodiment. [Figure 5] FIG. 2 is a diagram showing the data structure of a geospatial database of roads and railways according to the present embodiment. [Figure 6] 10 is a flowchart of a three-dimensional model generation process according to the present embodiment. [Figure 7] 10A to 10C are diagrams illustrating examples of input information acquired in the three-dimensional model generation process of the present embodiment. [Figure 8] 4A to 4C are diagrams illustrating examples of first attribute information generated by the three-dimensional model generation processing of the present embodiment. [Figure 9] 10 is a diagram illustrating an example of second attribute information generated by the three-dimensional model generation processing of the present embodiment. FIG. [Figure 10] 10 is a flowchart of a three-dimensional model editing process according to the present embodiment. [Figure 11] 10A to 10C are diagrams showing examples of screens displayed in the three-dimensional model editing process of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the drawings for explaining the embodiment, the same components are generally designated by the same reference numerals, and repeated description thereof will be omitted.

[0012] (1) Configuration of the information processing device The configuration of the information processing device will be described below with reference to Fig. 1, which is a block diagram showing the configuration of the information processing device of this embodiment.

[0013] The information processing device 10 is a computer (for example, a personal computer, a server computer, a workstation, etc.).

[0014] 1, the information processing device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14. The information processing device 10 is connected to a display 21.

[0015] The storage device 11 is configured to store programs and data, and is, for example, a combination of a read-only memory (ROM), a random access memory (RAM), and a storage (for example, a flash memory or a hard disk).

[0016] The programs include, for example, the following programs: OS (Operating System) programs Applications that process information (e.g., web browsers)

[0017] The data includes, for example, the following data: Databases referenced in information processing Data obtained by performing information processing (i.e., the results of performing information processing)

[0018] The processor 12 is a computer that implements the functions of the information processing device 10 by running a program stored in the storage device 11. The processor 12 is, for example, at least one of the following: ·CPU(Central Processing Unit) ·GPU(Graphic Processing Unit) ·ASIC(Application Specific Integrated Circuit) ·FPGA(Field Programmable Array)

[0019] The input / output interface 13 is configured to acquire information (for example, a user's instruction) from an input device connected to the information processing device 10, and to output information (for example, an image) to an output device connected to the information processing device 10. The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display 21, a speaker, or a combination thereof.

[0020] The communication interface 14 is configured to control communication between the information processing device 10 and an external device (for example, a server or client device, not shown).

[0021] The display 21 is configured to display an image (a still image or a moving image). The display 21 is, for example, a liquid crystal display or an organic EL display.

[0022] (2) One aspect of the embodiment An example of this embodiment will now be described with reference to Fig. 2, which is an explanatory diagram of this example.

[0023] 2, an artificial satellite ST11 photographs geographical elements present on the Earth's surface from above. The information processing device 10 acquires image information including photographs taken by the artificial satellite ST11. The information processing device 10 may acquire the image information directly from the artificial satellite ST11, or may acquire image information stored in an external server (not shown).

[0024] The information processing device 10 recognizes geographic elements from the acquired image information and extracts portions corresponding to the geographic elements. The information processing device 10 applies the attribute prediction model LM12 to model input data based on the extraction results. The attribute prediction model LM12 generates attribute information by predicting attributes of the geographic elements corresponding to the model input data.

[0025] The information processing device 10 generates a geospatial database GD13 including attribute information output by the attribute prediction model LM12.

[0026] The geospatial database GD13 corresponds to information structuring the corresponding geographic elements using attribute information. Therefore, the three-dimensional model TM14 that can be generated (modeled) from the geospatial database GD13 does not deteriorate with enlargement, unlike three-dimensional models generated by applying textures to three-dimensional meshes. Furthermore, even if noise such as shadows appears in the photograph taken by the artificial satellite ST11, the noise merely affects the predicted base color of the geographic elements represented by the three-dimensional model TM14 and does not become noise in the three-dimensional model TM14. Therefore, the three-dimensional model TM14 can express the lighting effects intended by the content creator, regardless of the optical conditions in the virtual space in which it is placed.

[0027] Human faces and vehicle license plates are not recognized as geographic elements and are therefore not reflected in the 3D model TM14. Furthermore, attachments to geographic elements, such as signs, can be predicted as attributes of the geographic element by the attribute prediction model LM12, making it possible to generate a 3D model TM14 that represents the geographic element without the attachments. This makes it possible to generate a 3D model TM14 with a low risk of infringing the rights of third parties. Furthermore, if the attribute prediction model LM12 predicts the building materials of each part that makes up the geographic element as attribute information, the unique texture of the building materials can be reproduced, enhancing the realism of the 3D CG (computer graphics) rendered from the 3D model TM14.

[0028] (3) Data Structure The data structure of the geospatial database of this embodiment will be described.

[0029] (3-1) Geospatial database of buildings The geospatial database of buildings of this embodiment will now be described with reference to Fig. 3, which is a diagram showing the data structure of the geospatial database of buildings of this embodiment.

[0030] As shown in FIG. 3, the geospatial database of buildings includes an “ID” field, a “type” field, a “height” field, a “shape” field, a “side structure” field, a “top structure” field, a “sign” field, a “color” field, and a “building material” field.

[0031] The "ID" field stores a geographic element ID, which is information for identifying a geographic element represented by a geospatial database.

[0032] The "type" field stores type information. The type information is information about the type of geographic element represented by the geospatial database. The type of geographic element represented by the geospatial database may be, for example, a building, land, or road / railway, but is not limited to these examples. The data structure of the geospatial database differs depending on the value of the type information. When the type information of a geospatial database has a value corresponding to a building, the geospatial database has the data structure illustrated in FIG. 3.

[0033] The "height" field stores height information, which is information about the height of a building represented by the geospatial database.

[0034] The "shape" field stores building shape information. The building shape information is information about the shape of a building represented by the geospatial database.

[0035] The "side structure" field stores side structure information, which is information about the side structure of a building represented by the geospatial database.

[0036] The "top structure" field stores top structure information. The top structure information is information about the top (i.e., roof) structure of a building represented by the geospatial database.

[0037] The "Signboard" field stores signboard information. The signboard information is information about a signboard (an example of a "specific type of attachment") attached to a building represented by the geospatial database.

[0038] The "Color" field stores color information, which is information about the color of the building represented by the geospatial database.

[0039] The "Building Material" field stores building material information. The building material information is information about the building materials that make up the building represented by the geospatial database. The building material information may include information about the physical properties (e.g., optical properties, mechanical properties, or thermodynamic properties), density (mass per unit volume), or chemical properties of the building material. The optical properties may include, for example, absorption, reflection, or transmission properties for visible light.

[0040] (3-2) Land geospatial database The geospatial land database of this embodiment will be described below with reference to Fig. 4, which is a diagram showing the data structure of the geospatial land database of this embodiment.

[0041] As shown in FIG. 4, the geospatial database of land includes an "ID" field, a "type" field, a "terrain" field, a "ground" field, and a "use" field.

[0042] The "ID" field stores the geographic element ID.

[0043] The "type" field stores type information. When the type information of a geospatial database has a value corresponding to land, the geospatial database has the data structure exemplified in FIG.

[0044] The "Topography" field stores topography information, which is information about the topography of land represented by a geospatial database.

[0045] The "Ground" field stores ground information, which is information about the ground of the land represented by the geospatial database.

[0046] The "Use" field stores use information, which is information about the use of land (i.e., land use) represented by the geospatial database.

[0047] (3-3) Geospatial database of roads and railways The geospatial database of roads and railways of this embodiment will be described below. Fig. 5 is a diagram showing the data structure of the geospatial database of roads and railways of this embodiment.

[0048] As shown in Figure 5, the road and railway geospatial database includes an "ID" field, a "type" field, a "shape" field, and a "number of lanes" field. Each field is associated with the others.

[0049] The "ID" field stores the geographic element ID.

[0050] The "Type" field stores type information. When the type information of a geospatial database has a value corresponding to road or railway, the geospatial database has the data structure shown in FIG. 5.

[0051] The "Shape" field stores road and railway shape information, which is information about the shape of roads and railways expressed by a geospatial database.

[0052] The "Number of lanes" field stores information about the number of lanes on roads and railways or the number of tracks represented by the geospatial database.

[0053] (4) Information processing The information processing of this embodiment will be described.

[0054] (4-1) 3D model generation processing The three-dimensional model generation processing of this embodiment will be described. Fig. 6 is a flowchart of the three-dimensional model generation processing of this embodiment. Fig. 7 is a diagram illustrating input information acquired in the three-dimensional model generation processing of this embodiment. Fig. 8 is a diagram illustrating first attribute information generated by the three-dimensional model generation processing of this embodiment. Fig. 9 is a diagram illustrating second attribute information generated by the three-dimensional model generation processing of this embodiment.

[0055] The three-dimensional model generation process in FIG. 6 starts in response to, for example, a request to generate a three-dimensional model. First, the information processing device 10 acquires various pieces of information (S110). Specifically, the information processing device 10 acquires, for example, input information IN10 shown in FIG. 7. The input information IN10 includes map information IN11, three-dimensional building information IN12, satellite photo information IN13 (an example of a "photograph of geographic elements present on the ground surface taken from above"), ground photo information IN14, and people flow information IN15. The information processing device 10 may acquire different types of input information from different types of information sources. Note that the information processing device 10 may acquire input information not shown in FIG. 7, or may not acquire any of the input information shown in FIG. 7.

[0056] The map information IN11 is information relating to a two-dimensional or three-dimensional map. The three-dimensional building information IN12 is three-dimensional information including an image of a specific building. The three-dimensional building information is, for example, three-dimensional CAD (Computer-Aided Design) data or BIM (Building Information Modeling) data of the specific building.

[0057] The satellite photo information IN13 is image information relating to photographs of geographical elements present on the earth's surface taken from above by an artificial satellite.

[0058] Ground photo information IN14 is image information relating to photographs of geographic elements present on the earth's surface taken from the ground (for example, photographs of mainly the sides of buildings taken with a camera mounted on a smartphone, a camera mounted on a moving object such as a car, etc.). The people flow information IN15 is information about people flow (that is, when, where and how many people are there).

[0059] After step S110, the information processing device 10 executes recognition of various geographical elements (S111). Specifically, the information processing device 10 recognizes a portion of the input information (image information) acquired in step S110 that corresponds to any type of geographic element, and extracts the portion. The information processing device 10 assigns a geographic element ID that identifies the recognized geographic element to the recognized geographic element.

[0060] As an example, the information processing device 10 recognizes portions (pixel regions) corresponding to buildings in the satellite photo information IN13 by applying a building recognition model to input data based on satellite photo information IN13. Here, the building recognition model can be constructed by supervised learning using a large amount (e.g., several thousand) of training data. Each training data includes training input data based on a satellite photo and correct answer data representing a building (correct answer) that should be recognized in the training input data. The building recognition model may be constructed on the information processing device 10 or on an external device (e.g., a cloud server).

[0061] Similarly, the information processing device 10 may prepare a recognition model for each type of geographic element and recognize the corresponding type of geographic element by applying the recognition model to image information (satellite photo information IN13 or terrestrial photo information IN14). Alternatively, the information processing device 10 may recognize the geographic element by analyzing map information IN11.

[0062] After step S111, the information processing device 10 analyzes the first attribute information (S112). Specifically, the information processing device 10 analyzes first attribute information from the input information acquired in step S110. The first attribute information is attribute information that is included in the geospatial database for structurally expressing geographic elements and is suitable for analysis using some predetermined algorithm. The first attribute information to be analyzed depends on the type of geographic element recognized in step S111 and the input information that was acquired for the geographic element in step S110. Note that if the attribute information in the geospatial database corresponding to the type of geographic element recognized in step S111 does not include any first attribute information, the information processing device 10 can omit step S112.

[0063] When the type of geographic element is a building, the information processing device 10 analyzes the first attribute information AT20 shown in Fig. 8 for the input information. The first attribute information AT20 includes height information AT21 and shape information AT22. The information processing device 10 may analyze first attribute information not shown in Fig. 8, or may not analyze any of the first attribute information shown in Fig. 8.

[0064] The information processing device 10 generates the height information AT21 and the shape information AT22 by analyzing, for example, the satellite photo information IN13 (particularly, the portion recognized as corresponding to the geographic element in step S111). The information processing device 10 may generate the height information AT21 and the shape information AT22 by analyzing the three-dimensional building information IN12 or the terrestrial photo information IN14 in addition to or instead of the satellite photo information IN13.

[0065] When the type of the geographic element is land, the information processing device 10 may analyze at least one of topographic information, ground information, and usage information as first attribute information for the map information IN11. The information processing device 10 may analyze at least one of satellite photo information IN13 and terrestrial photo information IN14 as first attribute information in addition to the map information IN11.

[0066] When the type of geographic element is a road or railway, the information processing device 10 may analyze at least one of shape information and number of lanes information as first attribute information for the map information IN11. The information processing device 10 may analyze at least one of satellite photo information IN13 and terrestrial photo information IN14 as first attribute information in addition to the map information IN11.

[0067] After step S112, the information processing device 10 predicts the second attribute information (S113). Specifically, the information processing device 10 generates second attribute information by applying an attribute prediction model to model input data based on the input information acquired in step S110 (particularly, the portion recognized in step S111 as corresponding to the geographic element). The second attribute information is attribute information that is suitable for prediction (inference) by the attribute prediction model, among attribute information included in the geospatial database for structurally expressing the geographic element. The second attribute information to be predicted depends on the type of geographic element recognized in step S111 and the input information acquired for the geographic element in step S110. Note that if the attribute information in the geospatial database corresponding to the type of geographic element recognized in step S111 does not include any second attribute information, the information processing device 10 can omit step S113.

[0068] Here, the attribute prediction model can be constructed, for example, for each attribute defining a geospatial database, by supervised learning using a large amount (e.g., several thousand) of training data. Each training data includes training input data based on at least one of satellite images or terrestrial images, and correct answer data representing attribute information (correct answer) to be predicted from the training input data. The attribute prediction model may be constructed on the information processing device 10 or on an external device (e.g., a cloud server).

[0069] When the type of geographic element is a building, the information processing device 10 applies an attribute prediction model to the input information to predict second attribute information AT30 shown in Fig. 9. The second attribute information AT30 includes side structure information AT31, color information AT32, building material information AT33, top structure information AT34, and signboard information AT35. The information processing device 10 may analyze second attribute information not shown in Fig. 9, or may not analyze any of the second attribute information shown in Fig. 9.

[0070] The information processing device 10 generates side structure information AT31 of a geographic element (building) corresponding to model input data, for example, by applying a side structure prediction model to model input data based on at least one of satellite photo information IN13 or terrestrial photo information IN14.

[0071] The information processing device 10 applies a color prediction model to model input data based on at least one of, for example, satellite photo information IN13 or terrestrial photo information IN14, thereby generating color information AT32 of a geographic element (building) corresponding to the model input data.

[0072] The information processing device 10 applies a building material prediction model to model input data based on, for example, at least one of satellite photo information IN13 or terrestrial photo information IN14, thereby generating building material information AT33 of a geographic element (building) corresponding to the model input data.

[0073] The information processing device 10 generates top surface structure information AT34 of a geographic element (building) corresponding to model input data, for example, by applying an upper surface structure prediction model to model input data based on at least one of satellite photo information IN13 or terrestrial photo information IN14.

[0074] The information processing device 10 applies a signboard prediction model to model input data based on at least one of, for example, satellite photo information IN13 or terrestrial photo information IN14, thereby generating signboard information AT35 of a geographic element (building) corresponding to the model input data.

[0075] When the type of geographic element is land, the information processing device 10 can generate usage information for the geographic element (land) corresponding to the model input data by applying a usage prediction model (an example of an "attribute prediction model") to model input data based on at least one of satellite photo information IN13 or terrestrial photo information IN14.

[0076] After step S113, the information processing device 10 generates a geospatial database (S114). Specifically, the information processing device 10 generates a geospatial database by associating the attribute information generated in at least one of step S112 or step S113 with the geographic element ID assigned in step S111. The geospatial database includes the attribute information generated in at least one of step S112 or step S113, and structurally represents the geographic elements by the attribute information.

[0077] After step S114, the information processing device 10 generates a three-dimensional model (S115). Specifically, the information processing device 10 generates a three-dimensional model by performing a modeling process on the geospatial database generated in step S114.

[0078] With step S114, the information processing device 10 ends the three-dimensional model generation process shown in Fig. 6. However, if the information processing device 10 recognizes multiple geographic elements in step S111, the information processing device 10 may repeat the processes from step S112 to step S114 for each geographic element.

[0079] (4-2) 3D model editing process The three-dimensional model editing process of this embodiment will now be described. Fig. 10 is a flowchart of the three-dimensional model editing process of this embodiment. Fig. 11 is a diagram showing an example of a screen displayed in the three-dimensional model editing process of this embodiment.

[0080] The three-dimensional model editing process shown in Figure 10 starts when a user of the three-dimensional model (for example, a person who receives, for a fee or free of charge, a three-dimensional model generated by the three-dimensional model generation process shown in Figure 6) operates an input device on his or her user terminal to select the three-dimensional model to be edited. This process can be performed in various forms. As a first example, the information processing device 10 as a server may perform this process in response to a request from a user terminal as a client. As a second example, the three-dimensional model may be stored in a user environment (including a cloud environment used by the user), and the user terminal or a server in the user environment (including a cloud server) may perform this process. The following explanation will be based on the first example, but in the second example, the "information processing device 10" can be appropriately interpreted as a "user terminal" or a "server in the user environment."

[0081] As shown in FIG. 10, the information processing apparatus 10 executes modeling and rendering (S210). Specifically, the information processing device 10 reads a geospatial database corresponding to a three-dimensional model to be edited, and generates (models) a three-dimensional model of the geographic elements represented by the geospatial database from attribute information included in the geospatial database. Then, the information processing device 10 performs a rendering process on the three-dimensional model to generate three-dimensional CG. Here, the information processing device 10 generates a three-dimensional CG that expresses the exterior of the building with the signboard removed, for example, based on the attribute information shown in Fig. 3. For example, the information processing device 10 recreates the three-dimensional CG of the building with the signboard removed, based on the height information, shape information, and signboard information of the building. Furthermore, the information processing device 10 generates a three-dimensional CG representing the exterior of the building with the characters, drawings, pictures, or photographs expressed on the surface of the building removed, for example, based on the attribute information shown in Fig. 3. For example, the information processing device 10 reproduces the texture of the building with the characters, drawings, pictures, or photographs expressed on the surface of the building removed, based on the side structure information, top structure information, color information, and building material information.

[0082] After step S210, the information processing device 10 executes display of three-dimensional CG (S211). Specifically, the information processing device 10 displays the 3D CG generated in step S210 and a UI (User Interface) for receiving editing instructions for the 3D model to be edited from the user on the display of the user terminal. As an example, the information processing device 10 displays the screen shown in Fig. 11 on the display of the user terminal.

[0083] The screen shown in FIG. 11 includes display objects A21 and A22 and operation objects B21 and B22. The display object A21 displays the three-dimensional CG generated in step S210. In response to a user instruction for the display object A21, the information processing device 10 rotates the three-dimensional CG around the roll axis, pitch axis, or yaw axis, or enlarges or reduces the three-dimensional CG and displays it on the display object A21.

[0084] The display object A22 displays values ​​of attribute information stored in a geospatial database corresponding to the three-dimensional model to be edited, and accepts instructions to edit the values ​​of the attribute information. The information processing device 10 can arrange various input forms (e.g., text fields, menus, check boxes, radio buttons, etc.) on the display object A22 to accept editing instructions. The user provides desired editing instructions to the information processing device 10 by operating an input device of the user terminal. In response to the editing instructions accepted by the display object A22, the information processing device 10 (temporarily) updates the geospatial database corresponding to the three-dimensional model to be edited, and displays three-dimensional CG based on the updated geospatial database on the display object A22. This allows the user to freely customize the three-dimensional model while checking changes in the appearance of the three-dimensional CG resulting from editing the values ​​of the attribute information.

[0085] The operation object B21 accepts an instruction to cancel the previous editing instruction for the three-dimensional model. The user operates an input device of the user terminal to select the operation object B21, thereby providing the cancel instruction to the information processing device 10. Upon receiving the cancel instruction, the information processing device 10 cancels the previous editing instruction. As a result, the attribute information of the three-dimensional model returns to the state before the previous editing instruction was issued.

[0086] The operation object B22 receives an instruction to confirm the editing of the three-dimensional model. The user operates an input device of the user terminal to select the operation object B22, thereby providing the confirmation instruction to the information processing device 10. Upon receiving the confirmation instruction, the information processing device 10 overwrites the geospatial database corresponding to the three-dimensional model to be edited with the latest three-dimensional model at the time of the confirmation instruction (i.e., a three-dimensional model reflecting a series of valid editing instructions). This confirms the editing of the three-dimensional model. Note that overwriting is not essential, and the information processing device 10 may save the edited three-dimensional model separately from the three-dimensional model to be edited. This allows the pre-editing (original) three-dimensional model and the edited three-dimensional model to coexist.

[0087] If the user issues an editing instruction (for example, an operation on display object A22) before the editing of the three-dimensional model is finalized in step S211 (for example, before the selection of operation object B22 is detected), the information processing device 10 acquires the editing instruction (S212). Specifically, the information processing device 10 acquires an editing instruction (for example, attribute information to be edited and an edited value of the attribute information) from the user terminal.

[0088] After step S212, the information processing device 10 updates the geospatial database (S213). Specifically, the information processing device 10 updates the geospatial database corresponding to the three-dimensional model to be edited so as to reflect the editing instruction acquired in step S212. After step S213, the information processing device 10 re-executes modeling and rendering (S210) and display of the three-dimensional CG (S211). That is, the information processing device 10 generates a new three-dimensional model and three-dimensional CG from the updated geospatial database and displays the new three-dimensional CG. This allows the user to view three-dimensional CG that reflects the editing instructions acquired in step S212.

[0089] On the other hand, when the editing of the three-dimensional model is confirmed in step S211, the information processing device 10 executes saving of the geospatial database (S214). Specifically, the information processing device 10 stores in the storage device 11 a geospatial database corresponding to the latest three-dimensional model. With step S214, the information processing device 10 ends the three-dimensional model editing process shown in FIG.

[0090] (5) Summary As described above, the information processing device 10 applies an attribute prediction model capable of predicting the attributes of geographic elements to model input data based on satellite photo information IN13, thereby generating attribute information for the geographic elements corresponding to the model input data, and then generates a 3D model from a geospatial database containing the attribute information. This prevents degradation of the 3D model and the 3D CG generated from the 3D model due to enlargement. Furthermore, even if noise such as shadows is present in the satellite photo information IN13, the noise only affects the prediction of the base color of the geographic elements represented by the 3D model and does not become noise in the 3D CG generated from the 3D model. Therefore, the 3D CG can express the lighting effects intended by the content creator, regardless of the optical conditions set in the virtual space in which it is placed.

[0091] The geospatial database may include attribute information about the building materials of the geographic elements represented by the geospatial database. This allows the characteristics of the building materials that make up the geographic elements to be reflected in the 3D CG generated from the 3D model, enhancing realism. The attribute information about the building materials may include information about the optical properties of the building materials. This allows the textures unique to the building materials that make up the geographic elements to be reproduced in the 3D CG generated from the 3D model, enhancing realism.

[0092] The information processing device 10 may generate a 3D CG representation of the geographic element represented by the 3D model from the 3D model. The 3D CG may represent the appearance of the geographic element with a specific type of accessory (e.g., a sign) removed. This prevents signs from appearing in the 3D CG, thereby providing a 3D model with a low risk of infringing third-party rights. The information processing device 10 may also generate attribute information about a specific type of accessory of the geographic element corresponding to the model input data by applying an attribute prediction model capable of predicting attributes about a specific type of accessory to the model input data based on satellite photo information IN13. This makes it possible to easily reproduce the shape of the geographic element with a specific type of accessory removed.

[0093] The information processing device 10 may generate a 3D CG representation of the geographic element represented by the 3D model from the 3D model. The 3D CG may represent the appearance of the geographic element with any text, drawings, pictures, or photographs on the surface of the geographic element removed. This prevents copyrighted material, such as murals, from appearing in the 3D CG, thereby providing a 3D model with a low risk of infringing third-party rights. The information processing device 10 may also generate attribute information about the color of the geographic element corresponding to the model input data based on satellite photo information IN13 by applying an attribute prediction model capable of predicting color attributes of the geographic element to the model input data. This allows for easy reproduction of the color (base color) of the geographic element with any murals or other content removed.

[0094] The geospatial database includes attribute information about the type of geographic element represented by the geospatial database, and if the type of the geographic element is a building, the database may include attribute information about at least one of the height, shape, side structure, construction material, color, top structure, and sign of the geographic element. This makes it possible to provide a 3D model with attribute information that appropriately represents the structural characteristics of the building.

[0095] The information processing device 10 may generate, from the 3D model, 3D CG of the geographic elements represented by the 3D model, and display the 3D CG and information for accepting user instructions for editing the 3D model. This allows the user to simultaneously check the 3D CG and customize the 3D model. The information processing device 10 may also update a geospatial database corresponding to the 3D model in response to a user instruction, and generate new 3D CG from the updated geospatial database. This allows the user to easily check the effect of updating the geospatial database on the 3D CG and efficiently perform customization work.

[0096] (6) Other variations The storage device 11 may be connected to the information processing device 10 via a network NW. The display 21 may be integrated with the information processing device 10 or may be external.

[0097] Each step of the above information processing may be performed by the information processing device 10, or may be performed in cooperation with an external device, or may be performed by the external device. In addition, although the above description shows an example in which each step in each process is executed in a specific order, the execution order of each step is not limited to the example described above as long as there is no dependency between the steps.

[0098] In the embodiment, an example has been described in which image information including photographs of geographic elements on the Earth's surface taken from the sky by an artificial satellite is used. However, image information including photographs of geographic elements on the Earth's surface taken from the sky by an aircraft (plane or drone) may also be used. Furthermore, in addition to or instead of photographs taken from the sky, image information including photographs of geographic elements on the Earth's surface taken from the ground (e.g., the above-mentioned ground photo information IN14) may also be used. In other words, attribute information of geographic elements corresponding to the model input data may be generated by applying an attribute prediction model capable of predicting attributes of geographic elements to model input data based on such image information.

[0099] In the embodiment, an example in which a recognition model is prepared for each type of geographic element has been shown. However, it is also possible to subdivide the geographic elements at any granularity and use a recognition model for each classification of the subdivided geographic elements. For example, it is possible to use a recognition model for residential areas and a recognition model for office buildings in combination, or a recognition model for Asia and a recognition model for Europe in combination.

[0100] In the embodiment, the attribute information of the geographic elements has been described as being divided into first attribute information and second attribute information. However, each piece of attribute information may be generated by either analysis or prediction alone, or part of the attribute information may be generated by analysis and the rest by prediction. Furthermore, even for the same attribute information, depending on the input information that can be acquired for the attribute information, it may be generated by analysis alone, prediction alone, or a combination of both.

[0101] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the above-described embodiments. Furthermore, the above-described embodiments can be improved or modified in various ways without departing from the spirit of the present invention. Furthermore, the above-described embodiments and modifications can be combined. [Explanation of symbols]

[0102] 10: Information processing device 11:Storage device 12: Processor 13: Input / output interface 14: Communication interface 21: Display

Claims

1. Computer, means for acquiring image information including photographs of geographical elements present on the earth's surface; a means for generating attribute information of the geographic elements corresponding to the model input data by applying an attribute prediction model capable of predicting attributes of the geographic elements to the model input data based on the image information; means for generating a geospatial database including the attribute information; means for generating a three-dimensional model based on said geospatial database; A program that functions as a

2. the geospatial database includes attribute information relating to building materials of the geographic elements represented by the geospatial database; The program according to claim 1.

3. The attribute information on the building material of the geographic element includes information on the optical properties of the building material. The program according to claim 2.

4. causing the computer to function as a means for generating, from the three-dimensional model, three-dimensional CG (Computer Graphics) of the geographical elements represented by the three-dimensional model; The three-dimensional CG represents the appearance of the geographic element with certain types of attachments removed. The program according to claim 1.

5. The specific type of attachment includes a signboard. The program according to claim 4.

6. the means for generating attribute information applies an attribute prediction model capable of predicting attributes related to the specific type of attachment to model input data based on the image information, thereby generating attribute information related to the specific type of attachment of the geographic element corresponding to the model input data; The program according to claim 4.

7. causing the computer to function as a means for generating, from the three-dimensional model, three-dimensional CG (Computer Graphics) of the geographical elements represented by the three-dimensional model; The three-dimensional CG represents the appearance of the geographic element with any characters, drawings, pictures, or photographs displayed on the surface of the geographic element removed. The program according to claim 1.

8. the means for generating attribute information applies an attribute prediction model capable of predicting attributes related to colors of geographic elements to model input data based on the image information, thereby generating attribute information related to colors of geographic elements corresponding to the model input data. The program according to claim 7.

9. the geospatial database includes attribute information related to the type of the geographic element, and when the type of the geographic element is a building, includes attribute information related to at least one of the height, shape, side structure, building material, color, top structure, and sign of the geographic element; The program according to claim 1.

10. The computer a means for generating, from the three-dimensional model, three-dimensional CG (Computer Graphics) of the geographical elements represented by the three-dimensional model; and further functioning as a means for displaying the three-dimensional CG generated by the means for generating the three-dimensional CG and information for receiving user instructions for editing the three-dimensional model. The program according to claim 1.

11. causing the computer to further function as a means for updating a geospatial database corresponding to the three-dimensional model in response to the user instruction; the means for generating the 3D CG newly generates the 3D CG from the updated geospatial database; The program according to claim 10.

12. A means for acquiring image information including photographs of geographical elements present on the earth's surface; a means for applying an attribute prediction model capable of predicting attributes of geographic elements to model input data based on the image information, to generate attribute information of the geographic elements corresponding to the model input data; means for generating a geospatial database including the attribute information; means for generating a three-dimensional model based on the geospatial database; An information processing device comprising:

13. The computer acquiring image information including photographs of geographical features present on the Earth's surface; generating attribute information of a geographic element corresponding to the model input data by applying an attribute prediction model capable of predicting attributes of the geographic element to the model input data based on the image information; generating a geospatial database including the attribute information; generating a three-dimensional model based on the geospatial database; How to perform.

Images