Information processing device, method and program

Abstract

To provide a technique capable of generating a three-dimensional model with a high visual quality and customization property.SOLUTION: A program causes a computer to function as means of: acquiring image information including a photograph obtained by photographing a geographic element present on a ground surface; applying an attribute prediction model capable of predicting an attribute of the geographic element to model input data based on the image information; generating attribute information of the geographical element corresponding to the model input data; generating a geographical space database including the attribute information; and generating a three-dimensional model based on the geographical space database.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, method, and program.

Background Art

[0002] Conventionally, technologies for generating a city model or a three-dimensional map based on satellite images have been proposed. Patent Document 1 discloses generating a three-dimensional point cloud based on a satellite image, meshing the three-dimensional point cloud, and attaching the latest satellite image to the mesh.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Secondly, since such three-dimensional models are generated by applying textures to a three-dimensional mesh, even minor customizations, such as increasing or decreasing the number of floors in a building, cannot be easily implemented due to the risk of damaging the three-dimensional mesh or textures, making them inflexible. Furthermore, such three-dimensional models have difficulty reproducing the unique textures of building materials that make up the three-dimensional structure, especially under optical conditions different from those used when satellite images are taken.

[0007] Thirdly, satellite imagery may contain human faces, license plates, signs, and other intrusive details. Therefore, 3D models that use satellite imagery directly as textures risk infringing on the rights of third parties, which may cause content creators to hesitate in adopting such models.

[0008] The purpose of this disclosure is to provide a technology capable of generating three-dimensional models with high visual quality and customizability. [Means for solving the problem]

[0009] A program in one aspect of this disclosure causes a computer to function as: means for acquiring image information including photographs of geographic elements present on the Earth's surface; means for generating attribute information of geographic elements corresponding to model input data by applying an attribute prediction model capable of predicting the attributes of geographic elements to model input data based on the image information; means for generating a geospatial database including attribute information; and means for generating a three-dimensional model based on the geospatial database. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of the information processing device of this embodiment. [Figure 2] This is an explanatory diagram of one aspect of this embodiment. [Figure 3] This figure shows the data structure of the geospatial database for the building in this embodiment. [Figure 4] This figure shows the data structure of the geospatial database of land in this embodiment. [Figure 5] This diagram shows the data structure of the road and railway geospatial database in this embodiment. [Figure 6] This is a flowchart of the three-dimensional model generation process in this embodiment. [Figure 7] This figure illustrates the input information obtained in the three-dimensional model generation process of this embodiment. [Figure 8] This figure illustrates the first attribute information generated by the three-dimensional model generation process of this embodiment. [Figure 9] This figure illustrates the second attribute information generated by the three-dimensional model generation process of this embodiment. [Figure 10] This is a flowchart of the three-dimensional model editing process in this embodiment. [Figure 11] This figure shows an example of a screen displayed during the three-dimensional model editing process of this embodiment. [Modes for carrying out the invention]

[0011] Hereinafter, an embodiment of the present invention will be described in detail based on the drawings. In the drawings for explaining the embodiment, the same components are generally denoted by the same reference numerals, and repeated explanations thereof are omitted.

[0012] (1) Configuration of the information processing apparatus The configuration of the information processing apparatus will be described. FIG. 1 is a block diagram showing the configuration of the information processing apparatus of the present embodiment.

[0013] The information processing apparatus 10 is a computer (for example, a personal computer, a server computer, a workstation, etc.).

[0014] As shown in FIG. 1, the information processing apparatus 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14. The information processing apparatus 10 is connected to a display 21.

[0015] The storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of a ROM (Read Only Memory), a RAM (Random Access Memory), and a storage (for example, a flash memory or a hard disk).

[0016] The program includes, for example, the following programs. · Program of an OS (Operating System) · Program of an application (for example, a web browser) that executes information processing

[0017] The data includes, for example, the following data. · Database referred to in information processing · Data obtained by executing information processing (that is, the execution result of information processing)

[0018] The processor 12 is a computer that realizes the functions of the information processing device 10 by starting a program stored in the memory device 11. The processor 12 is, for example, at least one of the following: ·CPU(Central Processing Unit) ·GPU(Graphic Processing Unit) ·ASIC(Application Specific Integrated Circuit) ·FPGA(Field Programmable Array)

[0019] The input / output interface 13 is configured to acquire information (e.g., user instructions) from an input device connected to the information processing device 10, and to output information (e.g., an image) to an output device connected to the information processing device 10. Input devices include, for example, keyboards, pointing devices, touch panels, or combinations thereof. The output device is, for example, a display 21, a speaker, or a combination thereof.

[0020] The communication interface 14 is configured to control communication between the information processing device 10 and an external device (for example, a server or client device not shown).

[0021] The display 21 is configured to display images (still images or moving images). The display 21 is, for example, a liquid crystal display or an organic EL display.

[0022] (2) One embodiment One aspect of this embodiment will be described. Figure 2 is an explanatory diagram of one aspect of this embodiment.

[0023] As shown in Figure 2, the ST11 satellite takes images of geographic elements on the Earth's surface from above. The information processing device 10 acquires image information, including the photographs taken by the ST11 satellite. The information processing device 10 may acquire the image information directly from the ST11 satellite, or it may acquire the image information stored on an external server (not shown).

[0024] The information processing device 10 recognizes geographic elements from the acquired image information and extracts the portion corresponding to that geographic element. The information processing device 10 applies the attribute prediction model LM12 to the model input data based on this extraction result. The attribute prediction model LM12 generates attribute information by predicting the attributes of the geographic elements corresponding to the model input data.

[0025] The information processing device 10 generates a geospatial database GD13 that includes attribute information output by the attribute prediction model LM12.

[0026] The geospatial database GD13 corresponds to information structured by attribute information for corresponding geographic elements. Therefore, the three-dimensional model TM14 that can be generated (modeled) from the geospatial database GD13 does not degrade when enlarged, unlike three-dimensional models generated by applying textures to a three-dimensional mesh. Furthermore, even if noise such as shadows is captured in a photograph taken by the ST11 satellite, this noise only affects the prediction of the base color of the geographic elements represented by the three-dimensional model TM14, and does not become noise in the three-dimensional model TM14 itself. Therefore, such a three-dimensional model TM14 can express the lighting effects intended by the content creator, regardless of the optical conditions in the virtual space in which it is placed.

[0027] Human faces or car license plates are not recognized as geographic elements and are therefore not reflected in the 3D model TM14. Furthermore, attachments to geographic elements, such as signs, can be predicted as one of the attributes of the geographic element by the attribute prediction model LM12, allowing for the generation of a 3D model TM14 that represents the geographic element without these attachments. Therefore, a 3D model TM14 with a low risk of infringing third-party rights can be generated. Moreover, if the attribute prediction model LM12 predicts the building materials of each component constituting the geographic element as attribute information, it is possible to reproduce the unique texture of those building materials and enhance the realism of the 3D CG (Computer Graphics) rendered from the 3D model TM14.

[0028] (3) Data structure The data structure of the geospatial database in this embodiment will be described.

[0029] (3-1) Geospatial database of buildings The geospatial database of the building in this embodiment will now be described. Figure 3 shows the data structure of the geospatial database of the building in this embodiment.

[0030] As shown in Figure 3, the building's geospatial database includes the following fields: "ID", "Type", "Height", "Shape", "Side Structure", "Top Structure", "Signage", "Color", and "Building Materials".

[0031] The "ID" field stores the geographic element ID. The geographic element ID is information that identifies a geographic element represented by the geospatial database.

[0032] The "Type" field stores type information. Type information is information about the type of geographic element represented by the geospatial database. The types of geographic elements represented by the geospatial database may be, for example, buildings, land, or roads / railways, but are not limited to these examples. The data structure of the geospatial database differs depending on the value of the type information. When the type information of the geospatial database has a value corresponding to a building, the geospatial database has the data structure illustrated in Figure 3.

[0033] The "Height" field stores height information. This height information is data about the building's height as represented by the geospatial database.

[0034] The "Shape" field stores building shape information. Building shape information is information about the building's shape as represented by the geospatial database.

[0035] The "Side Structure" field stores side structure information. Side structure information is information about the building's side structure as represented by the geospatial database.

[0036] The "Upper Structure" field stores upper structure information. Upper structure information is information about the upper structure (i.e., rooftop or roof) of a building as represented by the geospatial database.

[0037] The "Sign" field stores sign information. Sign information is information about signs attached to buildings (an example of a "specific type of attachment") as represented by the geospatial database.

[0038] The "Color" field stores color information. This color information is about the building's color as represented by the geospatial database.

[0039] The "Building Materials" field stores building materials information. Building materials information is information about the building materials that make up a building as represented by a geospatial database. Building materials information can include information about the physical properties (e.g., optical properties, mechanical properties, or thermodynamic properties), density (mass per unit volume), or chemical properties of the building materials. Optical properties can include, for example, absorption, reflection, or transmission properties of visible light.

[0040] (3-2) Geospatial database of land The geospatial database of land in this embodiment will now be described. Figure 4 shows the data structure of the geospatial database of land in this embodiment.

[0041] As shown in Figure 4, the geospatial database of land includes the "ID" field, the "Type" field, the "Topography" field, the "Ground" field, and the "Usage" field.

[0042] The "ID" field stores the geographic element ID.

[0043] The "Type" field stores type information. When the type information in the geospatial database has a value corresponding to land, the geospatial database has the data structure illustrated in Figure 4.

[0044] The "Terrain" field stores terrain information. Terrain information is information about the land's topography as represented by a geospatial database.

[0045] The "Ground" field stores ground information. Ground information is information about the land surface as represented by a geospatial database.

[0046] The "Usage" field stores usage information. This usage information is information about land use (i.e., land purpose) as represented by the geospatial database.

[0047] (3-3) Geospatial database of roads and railways This section describes the geospatial database of roads and railways according to this embodiment. Figure 5 shows the data structure of the geospatial database of roads and railways according to this embodiment.

[0048] As shown in Figure 5, the geospatial database of roads and railways includes an "ID" field, a "Type" field, a "Shape" field, and a "Number of Lanes" field. Each field is related to the others.

[0049] The "ID" field stores the geographic element ID.

[0050] The "Type" field stores type information. When the type information of a geospatial database has a value corresponding to roads and railways, the geospatial database has the data structure illustrated in Figure 5.

[0051] The "Shape" field stores road and railway shape information. This road and railway shape information is information about the shape of roads and railways as represented by the geospatial database.

[0052] The "Number of Lanes" field stores lane count information. This information pertains to the number of lanes or tracks on roads and railways as represented by the geospatial database.

[0053] (4) Information processing The information processing of this embodiment will now be described.

[0054] (4-1) Three-dimensional model generation process The three-dimensional model generation process of this embodiment will now be described. Figure 6 is a flowchart of the three-dimensional model generation process of this embodiment. Figure 7 is a diagram illustrating the input information acquired in the three-dimensional model generation process of this embodiment. Figure 8 is a diagram illustrating the first attribute information generated by the three-dimensional model generation process of this embodiment. Figure 9 is a diagram illustrating the second attribute information generated by the three-dimensional model generation process of this embodiment.

[0055] The three-dimensional model generation process shown in Figure 6 is initiated, for example, in response to a request to generate a three-dimensional model. First, the information processing device 10 performs the acquisition of various types of information (S110). Specifically, the information processing device 10 acquires input information IN10, for example, as shown in Figure 7. Input information IN10 includes map information IN11, three-dimensional building information IN12, satellite image information IN13 (an example of "a photograph taken from above of geographic elements present on the Earth's surface"), ground image information IN14, and pedestrian flow information IN15. The information processing device 10 may acquire different types of input information from different types of information sources. In addition, the information processing device 10 may acquire input information not shown in Figure 7, or may not acquire any of the input information shown in Figure 7.

[0056] Map information IN11 is information about two-dimensional or three-dimensional maps. 3D building information IN12 is 3D information that includes images of a specific building. 3D building information can be, for example, 3D CAD (Computer-Aided Design) data or BIM (Building Information Modeling) data for a specific building.

[0057] Satellite image information IN13 is image information related to photographs taken from above by artificial satellites of geographical elements present on the Earth's surface.

[0058] Ground image information IN14 is image information relating to photographs taken from the ground of geographical elements present on the Earth's surface (for example, photographs of the sides of buildings taken by cameras mounted on smartphones, cameras mounted on mobile vehicles such as automobiles, etc.). The IN15 human flow information is information about human flow (i.e., when, where, and how many people are).

[0059] After step S110, the information processing device 10 performs recognition of various geographic elements (S111). Specifically, the information processing device 10 recognizes the portion of the input information (image information) acquired in step S110 that corresponds to one of the types of geographic elements, and extracts that portion. The information processing device 10 assigns a geographic element ID to the recognized geographic element to identify that geographic element.

[0060] As an example, the information processing device 10 recognizes the parts (pixel regions) corresponding to buildings in the satellite image information IN13 by applying a building recognition model to the input data based on the satellite image information IN13. Here, the building recognition model can be constructed by supervised learning using a large amount (e.g., several thousand) of training data. Each training data includes training input data based on satellite imagery and ground truth data representing the buildings (correct answers) that should be recognized in the training input data. The building recognition model may be constructed on the information processing device 10 or on an external device (e.g., a cloud server).

[0061] Similarly, the information processing device 10 may prepare recognition models for various geographic elements and recognize various geographic elements by applying these recognition models to image information (satellite image information IN13 or ground image information IN14). Alternatively, the information processing device 10 may recognize geographic elements by analyzing map information IN11.

[0062] After step S111, the information processing device 10 performs analysis of the first attribute information (S112). Specifically, the information processing device 10 performs analysis of first attribute information on the input information acquired in step S110. First attribute information refers to attribute information included in the geospatial database for structurally representing geographic elements, which is suitable for analysis by some predetermined algorithm. The first attribute information to be analyzed depends on the type of geographic element recognized in step S111 and the input information that was acquired in step S110 for that geographic element. Note that if the attribute information in the geospatial database corresponding to the type of geographic element recognized in step S111 does not include any first attribute information, the information processing device 10 can omit step S112.

[0063] When the type of geographic element is a building, the information processing device 10 performs an analysis of the input information using the first attribute information AT20 shown in Figure 8. The first attribute information AT20 includes height information AT21 and shape information AT22. The information processing device 10 may also perform an analysis of first attribute information not shown in Figure 8, or it may not perform an analysis of any of the first attribute information shown in Figure 8.

[0064] The information processing device 10 generates height information AT21 and shape information AT22 by analyzing, for example, satellite image information IN13 (particularly the portion recognized as corresponding to a geographic element in step S111). In addition to, or instead of, the information processing device 10 may generate height information AT21 and shape information AT22 by analyzing three-dimensional building information IN12 or ground image information IN14.

[0065] When the type of geographic element is land, the information processing device 10 may perform analysis on the map information IN11 as first attribute information, including at least one of topographic information, ground information, or usage information. In addition to the map information IN11, the information processing device 10 may also perform analysis on at least one of the satellite image information IN13 or ground image information IN14 as first attribute information.

[0066] When the type of geographic element is roads and railways, the information processing device 10 may perform analysis on the map information IN11 to obtain at least one of shape information or lane count information as first attribute information. In addition to the map information IN11, the information processing device 10 may also perform analysis on at least one of satellite image information IN13 or ground image information IN14 to obtain first attribute information.

[0067] After step S112, the information processing device 10 performs prediction of second attribute information (S113). Specifically, the information processing device 10 generates second attribute information by applying an attribute prediction model to the model input data based on the input information acquired in step S110 (in particular, the portion recognized as corresponding to a geographic element in step S111). Second attribute information is attribute information included in the geospatial database for structurally representing geographic elements that is suitable for prediction (inference) by the attribute prediction model. The second attribute information to be predicted depends on the type of geographic element recognized in step S111 and the input information that was acquired in step S110 for that geographic element. Note that if the attribute information in the geospatial database corresponding to the type of geographic element recognized in step S111 does not include any second attribute information, the information processing device 10 can omit this step S113.

[0068] Here, the attribute prediction model can be constructed for each attribute that defines a geospatial database, for example, by supervised learning using a large amount of training data (e.g., several thousand). Each training data set includes training input data based on at least one satellite image or ground image, and ground truth data representing the attribute information (correct answer) to be predicted from the training input data. The attribute prediction model may be constructed on the information processing device 10, or on an external device (e.g., a cloud server).

[0069] When the type of geographic element is a building, the information processing device 10 predicts the second attribute information AT30 shown in Figure 9 by applying an attribute prediction model to the input information. The second attribute information AT30 includes side structure information AT31, color information AT32, building material information AT33, top structure information AT34, and sign information AT35. The information processing device 10 may perform analysis of second attribute information not shown in Figure 9, or it may not perform analysis of any of the second attribute information shown in Figure 9.

[0070] The information processing device 10 generates side structure information AT31 of a geographic element (building) corresponding to the model input data by applying a side structure prediction model to model input data based on, for example, at least one of satellite image information IN13 or ground image information IN14.

[0071] The information processing device 10 generates color information AT32 of a geographic element (building) corresponding to the model input data by applying a color prediction model to model input data based on, for example, at least one of satellite image information IN13 or ground image information IN14.

[0072] The information processing device 10 generates building material information AT33 for geographic elements (buildings) corresponding to the model input data by applying a building material prediction model to model input data based on, for example, at least one of satellite image information IN13 or ground image information IN14.

[0073] The information processing device 10 generates upper structure information AT34 of a geographic element (building) corresponding to the model input data by applying an upper structure prediction model to model input data based on, for example, at least one of satellite image information IN13 or ground image information IN14.

[0074] The information processing device 10 generates signboard information AT35 for geographic elements (buildings) corresponding to the model input data by applying a signboard prediction model to model input data based on, for example, at least one of satellite image information IN13 or ground image information IN14.

[0075] When the type of geographic element is land, the information processing device 10 can generate utilization information for the geographic element (land) corresponding to the model input data by applying a utilization prediction model (an example of an "attribute prediction model") to the model input data, which is based on at least one of satellite image information IN13 or ground image information IN14.

[0076] After step S113, the information processing device 10 performs the generation of a geospatial database (S114). Specifically, the information processing device 10 generates a geospatial database by associating the attribute information generated in at least one of step S112 or step S113 with the geographic element ID assigned in step S111. The geospatial database includes the attribute information generated in at least one of step S112 or step S113, and structurally represents the geographic elements using this attribute information.

[0077] After step S114, the information processing device 10 performs the generation of a three-dimensional model (S115). Specifically, the information processing device 10 generates a three-dimensional model by performing modeling processing on the geospatial database generated in step S114.

[0078] In step S114, the information processing device 10 terminates the three-dimensional model generation process shown in Figure 6. However, if the information processing device 10 recognizes multiple geographic elements in step S111, it may repeat the process from step S112 to step S114 for each geographic element.

[0079] (4-2) Three-dimensional model editing process The three-dimensional model editing process of this embodiment will now be described. Figure 10 is a flowchart of the three-dimensional model editing process of this embodiment. Figure 11 is a diagram showing an example of a screen displayed during the three-dimensional model editing process of this embodiment.

[0080] The 3D model editing process shown in Figure 10 is initiated when a user of the 3D model (for example, a person who receives a 3D model generated by the 3D model generation process shown in Figure 6, either for a fee or free of charge) selects the 3D model to be edited by operating the input device on their user terminal. This process can be executed in various forms. As a first example, the information processing device 10, acting as a server, may execute this process in response to a request from a user terminal acting as a client. As a second example, the three-dimensional model may be stored in the user environment (including the cloud environment used by the user), and the user terminal or a server within the user environment (including a cloud server) may execute this process. The following explanation will be based on the first example, but in the second example, "information processing device 10" can be appropriately replaced with "user terminal" or "server within the user environment."

[0081] As shown in Figure 10, the information processing device 10 performs modeling and rendering (S210). Specifically, the information processing device 10 reads a geospatial database corresponding to the three-dimensional model to be edited, and generates (models) a three-dimensional model of the geographic elements represented by the geospatial database from the attribute information contained in the geospatial database. Then, the information processing device 10 generates three-dimensional computer graphics by performing rendering processing on the three-dimensional model. Here, the information processing device 10 generates a three-dimensional computer graphics (CG) representing the exterior of the building with the sign removed, based on the attribute information shown in Figure 3, for example. For example, the information processing device 10 reproduces a three-dimensional CG of the building with the sign removed, based on the building's height information, shape information, and sign information. Furthermore, the information processing device 10 generates a three-dimensional computer graphics (CG) representing the exterior of the building with the text, drawings, pictures, or photographs displayed on its surface removed, based on the attribute information shown in Figure 3, for example. For example, the information processing device 10 reproduces the texture of the building with the text, drawings, pictures, or photographs displayed on its surface removed, based on the side structure information, top structure information, color information, and building material information.

[0082] After step S210, the information processing device 10 performs the display of three-dimensional computer graphics (S211). Specifically, the information processing device 10 displays the three-dimensional computer graphics generated in step S210, along with a user interface (UI) for receiving editing instructions from the user for the three-dimensional model to be edited, on the user terminal's display. As an example, the information processing device 10 displays the screen shown in Figure 11 on the user terminal's display.

[0083] The screen shown in Figure 11 includes display objects A21 and A22, and operation objects B21 and B22. Display object A21 displays the three-dimensional CG generated in step S210. The information processing device 10 rotates the three-dimensional CG around the roll axis, pitch axis, or yaw axis, or enlarges or reduces it, and displays it on display object A21 in response to user instructions for display object A21.

[0084] Display object A22 displays the attribute information values ​​in the geospatial database corresponding to the three-dimensional model to be edited, and accepts instructions to edit these attribute information values. Information processing device 10 can place various input forms (e.g., text fields, menus, checkboxes, radio buttons, etc.) on display object A22 to accept editing instructions. The user provides the information processing device 10 with the desired editing instructions by operating the input device on the user terminal. In response to the editing instructions received in display object A22, the information processing device 10 (temporarily) updates the geospatial database corresponding to the three-dimensional model to be edited, and displays the three-dimensional CG based on the updated geospatial database on display object A22. This allows the user to freely customize the three-dimensional model while checking the changes in the appearance of the three-dimensional CG that occur as a result of editing the attribute information values.

[0085] Operation object B21 accepts an instruction to cancel the previous editing instruction on the three-dimensional model. The user gives the cancel instruction to the information processing device 10 by operating the input device on the user terminal to select operation object B21. When the information processing device 10 receives the cancel instruction, it cancels the previous editing instruction. As a result, the attribute information of the three-dimensional model returns to the state it was in before the previous editing instruction was made.

[0086] Operation object B22 receives an instruction to confirm the editing of the three-dimensional model. The user gives the confirmation instruction to the information processing device 10 by operating the input device of the user terminal to select operation object B22. Upon receiving the confirmation instruction, the information processing device 10 overwrites the geospatial database corresponding to the three-dimensional model to be edited with the latest three-dimensional model at the time of the confirmation instruction (i.e., the three-dimensional model reflecting the series of valid editing instructions). This confirms the editing of the three-dimensional model. Note that overwriting is not mandatory, and the information processing device 10 may save the edited three-dimensional model separately from the three-dimensional model to be edited. This allows the original three-dimensional model and the edited three-dimensional model to coexist.

[0087] If the user makes an editing instruction (for example, an operation on display object A22) before the editing of the three-dimensional model is confirmed in step S211 (for example, before the selection of operation object B22 is detected), the information processing device 10 executes the acquisition of the editing instruction (S212). Specifically, the information processing device 10 obtains editing instructions from the user terminal (for example, attribute information to be edited, and the edit value of said attribute information).

[0088] After step S212, the information processing device 10 performs an update of the geospatial database (S213). Specifically, the information processing device 10 updates the geospatial database corresponding to the three-dimensional model to be edited so as to reflect the editing instructions acquired in step S212. After step S213, the information processing device 10 re-executes modeling and rendering (S210) and display of three-dimensional CG (S211). In other words, the information processing device 10 generates a new three-dimensional model and three-dimensional CG from the updated geospatial database and displays the three-dimensional CG. This allows the user to be presented with three-dimensional CG that reflects the editing instructions obtained in step S212.

[0089] On the other hand, if the editing of the three-dimensional model is confirmed in step S211, the information processing device 10 performs saving of the geospatial database (S214). Specifically, the information processing device 10 stores a geospatial database corresponding to the latest three-dimensional model in the storage device 11. In step S214, the information processing device 10 terminates the three-dimensional model editing process shown in Figure 10.

[0090] (5) Summary As explained above, the information processing device 10 applies an attribute prediction model capable of predicting the attributes of geographic elements to the model input data based on satellite image information IN13, thereby generating attribute information of geographic elements corresponding to the model input data, and generates a three-dimensional model from a geospatial database containing said attribute information. As a result, the three-dimensional model and the three-dimensional CG generated from the three-dimensional model do not degrade when enlarged. Furthermore, even if noise such as shadows is captured in the satellite image information IN13, such noise only affects the prediction of the base color of the geographic elements represented by the three-dimensional model, and does not become noise in the three-dimensional CG generated from the three-dimensional model. Therefore, such three-dimensional CG can express the lighting effects intended by the content creator, regardless of the optical conditions set in the virtual space in which it is placed.

[0091] The geospatial database may include attribute information regarding the building materials of the geographic elements represented by the geospatial database. This allows the characteristics of the building materials constituting the geographic elements to be reflected in the 3D computer graphics generated from the 3D model, thereby enhancing realism. The attribute information regarding the building materials may also include information regarding the optical properties of those building materials. This allows the textures unique to the building materials constituting the geographic elements to be reproduced in the 3D computer graphics generated from the 3D model, thereby enhancing realism.

[0092] The information processing device 10 may generate a three-dimensional computer graphics (CG) of the geographic element represented by the three-dimensional model from the three-dimensional model, and the CG may represent the appearance of the geographic element with certain types of appendages (e.g., signs) removed. This prevents signs from appearing in the CG, thus providing a three-dimensional model with a low risk of infringing on the rights of third parties. The information processing device 10 may also generate attribute information regarding certain types of appendages of the geographic element corresponding to the model input data by applying an attribute prediction model capable of predicting attributes related to certain types of appendages to the model input data based on satellite image information IN13. This makes it easy to reproduce the shape of the geographic element with certain types of appendages removed.

[0093] The information processing device 10 may generate a three-dimensional computer graphics (CG) of the geographic elements represented by the three-dimensional model from the three-dimensional model, and the three-dimensional CG may represent the appearance of the geographic elements with any text, drawings, pictures, or photographs displayed on their surface removed. This prevents copyrighted works such as murals from appearing in the three-dimensional CG, thus providing a three-dimensional model with a low risk of infringing on the rights of third parties. Furthermore, the information processing device 10 may generate attribute information regarding the color of geographic elements corresponding to the model input data by applying an attribute prediction model capable of predicting attributes regarding the color of geographic elements to the model input data based on satellite image information IN13. This makes it easy to reproduce the color (base color) of geographic elements with murals and the like removed.

[0094] The geospatial database includes attribute information relating to the type of geographic element represented by the geospatial database, and if the type of geographic element is a building, it may also include attribute information relating to at least one of the height, shape, side structure, building materials, color, top structure, and signage of the geographic element. This makes it possible to provide a three-dimensional model that has attribute information that appropriately represents the structural characteristics of a building.

[0095] The information processing device 10 may generate a three-dimensional computer graphics (CG) of the geographic elements represented by the three-dimensional model from the three-dimensional model, and display information that accepts user instructions for editing the three-dimensional CG and the three-dimensional model. This allows the user to simultaneously check the three-dimensional CG and customize the three-dimensional model. The information processing device 10 may also update the geospatial database corresponding to the three-dimensional model in response to user instructions, and generate a new three-dimensional CG from the updated geospatial database. This allows the user to easily check the impact of updating the geospatial database on the three-dimensional CG and perform customization work efficiently.

[0096] (6) Other modifications The storage device 11 may be connected to the information processing device 10 via a network NW. The display 21 may be integrated with the information processing device 10 or it may be external.

[0097] Each step of the above information processing may be performed by the information processing device 10, in cooperation with an external device, or by the external device alone. Furthermore, although the above description shows an example in which each step in each process is executed in a specific order, the execution order of each step is not limited to the example described, unless there is a dependency.

[0098] In the embodiment, an example was described in which image information including photographs of geographic elements present on the Earth's surface taken from above by an artificial satellite was used. However, image information including photographs of geographic elements present on the Earth's surface taken from above by an aircraft (airplane or drone) may also be used. Furthermore, in addition to, or instead of, photographs taken from above, image information including photographs of geographic elements present on the Earth's surface taken from the ground (for example, the aforementioned ground photograph information IN14) may also be used. In other words, attribute information of geographic elements corresponding to the model input data may be generated by applying an attribute prediction model capable of predicting the attributes of geographic elements to model input data based on such image information.

[0099] In this embodiment, an example was shown in which a recognition model is prepared for each type of geographic element. However, geographic elements can be subdivided to any level of granularity, and a recognition model can be used for each subdivided geographic element. For example, a recognition model for private houses and a recognition model for office buildings can be used in combination, or a recognition model for Asia and a recognition model for Europe can be used in combination.

[0100] In this embodiment, the attribute information of geographic elements was organized and described as first attribute information and second attribute information. However, each attribute information may be generated by either analysis or prediction alone, or it may be generated partially by analysis and the remainder by prediction. Furthermore, even for the same attribute information, depending on the input information that could be obtained for that attribute information, it may be generated by analysis alone, prediction alone, or a combination of both.

[0101] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the embodiments described above. Furthermore, the embodiments described above can be improved or modified in various ways without departing from the spirit of the present invention. In addition, the embodiments and modifications described above can be combined. [Explanation of symbols]

[0102] 10: Information Processing Device 11:Storage device 12: Processor 13: Input / Output Interface 14: Communication Interface 21: Display

Claims

1. A computer, A means for acquiring image information including photographs of geographical elements present on the earth's surface; a means for applying an attribute prediction model capable of predicting attributes of the geographic elements to model input data based on the image information, thereby generating attribute information of the geographic elements corresponding to the model input data; means for generating a geospatial database including the attribute information; means for generating a three-dimensional model based on said geospatial database; a means for generating, from the three-dimensional model, three-dimensional CG (Computer Graphics) of the geographical element represented by the three-dimensional model; a display unit for displaying the three-dimensional CG generated by the three-dimensional CG generating unit and information for receiving a user instruction for editing the three-dimensional model; Functioning as a Displaying information for receiving a user instruction for editing the three-dimensional model, Displaying values ​​of attribute information of the three-dimensional model to be edited; and accepting the user instruction to edit the value of the attribute information. program.

2. The geospatial database includes attribute information regarding the type of the geographic element, and when the type of the geographic element is land, includes attribute information regarding at least one of the topography, ground, and land use of the geographic element. The program according to claim 1.

3. The geospatial database includes attribute information regarding the type of the geographic element, and when the type of the geographic element is a road or a railway, includes attribute information regarding at least one of the shape of the geographic element and the number of lanes. The program according to claim 1.

4. An information processing device, A means for acquiring image information including photographs of geographical elements present on the earth's surface; a means for applying an attribute prediction model capable of predicting attributes of the geographic elements to model input data based on the image information, thereby generating attribute information of the geographic elements corresponding to the model input data; means for generating a geospatial database including the attribute information; means for generating a three-dimensional model based on the geospatial database; a means for generating, from the three-dimensional model, three-dimensional CG (Computer Graphics) of the geographical element represented by the three-dimensional model; a display unit for displaying the three-dimensional CG generated by the three-dimensional CG generating unit and information for receiving a user instruction for editing the three-dimensional model; and a means for displaying information for receiving a user instruction for editing the three-dimensional model, Displaying values ​​of attribute information of the three-dimensional model to be edited; and accepting the user instruction to edit the value of the attribute information. Information processing device.

5. A computer comprising: acquiring image information including photographs of geographical features present on the earth's surface; generating attribute information of the geographic element corresponding to the model input data by applying an attribute prediction model capable of predicting attributes of the geographic element to the model input data based on the image information; generating a geospatial database including the attribute information; generating a three-dimensional model based on the geospatial database; a step of generating, from the three-dimensional model, three-dimensional CG (Computer Graphics) of the geographical element represented by the three-dimensional model; A step of displaying the three-dimensional CG generated by the three-dimensional CG generating means and information for receiving a user's instruction for editing the three-dimensional model. and execute Displaying information for receiving a user instruction for editing the three-dimensional model, Displaying values ​​of attribute information of the three-dimensional model to be edited; and accepting the user instruction to edit the value of the attribute information. method.

Images