Ortho-image generation device, ortho-image generation method, and program

Abstract

[PROBLEM] Provided are an ortho-image generation device, an ortho-image generation method, and a program capable of efficiently generating an ortho-image. [SOLUTION] An ortho-image generation device according to the present disclosure comprises: a storage unit that stores three-dimensional model data generated on the basis of a plurality of pieces of captured image data obtained by capturing an image of a subject in the real world (S1); and a control unit that executes reception processing for displaying the three-dimensional model data on a display unit used by a user (S2) and, at the same time, receiving designation information, including a designated range and a designated direction in which orthographic conversion is desired in the three-dimensional model data, from the user (S3), ortho-image generation processing for generating an ortho-image in the designated range and the designated direction included in the designation information by orthographically projecting and transforming the three-dimensional model data in a region corresponding to the designated range and the designated direction (S4), and output processing for displaying the generated ortho-image on the display unit (S5).

Description

Orthographic Image Generation Device, Orthographic Image Generation Method, and Program

[0001] The present disclosure relates to an orthographic image generation device, an orthographic image generation method, and a program.

[0002] Conventionally, there is known a technique for generating an orthographic image of an orthographic projection by orthogonally correcting captured image data of a perspective projection (central projection) (see Patent Document 1). Thereby, for example, an orthographic projection image of an object viewed from directly above can be generated.

[0003] Japanese Patent Application Laid-Open No. 2024-073601

[0004] However, when trying to generate orthographic images of a three-dimensional object viewed from a plurality of directions, it is necessary to select a plurality of image data including the target surface and match the scales of the image data, and there is a problem that the generation process of the orthographic image takes time.

[0005] Therefore, the present disclosure has been made in view of the above problems, and an object thereof is to provide an orthographic image generation device, an orthographic image generation method, and a program capable of efficiently generating orthographic images.

[0006] According to the present disclosure, there is provided a storage unit that stores three-dimensional model data generated based on a plurality of captured image data of a subject in the real world, the three-dimensional model data is displayed on a display unit used by a user, and reception processing for receiving designation information including a designated range and a designated direction in which orthogonality is desired in the three-dimensional model data from the user, an orthographic image generation process for generating an orthographic image of the designated range and the designated direction included in the designation information by orthogonally projecting and converting the three-dimensional model data of the region corresponding to the designated range and the designated direction, and an output process for displaying the generated orthographic image on the display unit, and an orthographic image generation device including a control unit that executes the processes is provided.

[0007] Furthermore, according to this disclosure, an orthophoto generation method is provided, in which a control unit is executed to store 3D model data generated based on multiple captured image data of subjects in the real world in a storage unit, display the 3D model data on a display unit used by the user, and receive specified information from the user, including a specified range and direction for which orthorectification is desired in the 3D model data, an orthophoto generation process is performed by orthorectifying the 3D model data of the region corresponding to the specified range and direction to generate an orthophoto of the specified range and direction included in the specified information, and an output process is performed to display the generated orthophoto on the display unit.

[0008] Furthermore, according to this disclosure, a program is provided which causes a control unit to execute the following: a process for storing three-dimensional model data generated based on multiple captured image data of a subject in the real world; a reception process for displaying the three-dimensional model data on a display unit used by the user and receiving specified information from the user, including a specified range and direction for which orthorectification is desired in the three-dimensional model data; an orthoimage generation process for generating an orthoimage of the specified range and direction included in the specified information by performing an ortho-projection transformation on the three-dimensional model data of the region corresponding to the specified range and direction; and an output process for displaying the generated orthoimage on the display unit.

[0009] This disclosure provides an orthomosaic image generation device, an orthomosaic image generation method, and a program capable of efficiently generating orthomosaic images.

[0010] This figure shows an example configuration of an orthomosaic image generation system according to one embodiment of the present disclosure. This figure shows an example configuration of a computer that implements the information processing device according to the same embodiment. This figure shows an example configuration of a user terminal according to the same embodiment. This is a flowchart diagram relating to a series of controls according to the same embodiment. This figure shows an example of the display of a user terminal screen according to the same embodiment. This figure shows another example of a user terminal screen according to the same embodiment.

[0011] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0012] Figure 1 shows an overview of the system of this embodiment. The orthomosaic image generation system 1 of this embodiment comprises an information processing device 10 and a user terminal 20. The information processing device 10 and the user terminal 20 are connected via a network NW and can communicate with each other. The information processing device corresponds to an example of an "orthomosaic image generation device".

[0013] The network NW consists of the Internet, intranet, wireless LAN (Local Area Network), WAN (Wide Area Network), fixed telephone line, mobile phone line, etc., or a combination thereof. Note that the user terminal 20 constituting this system 1 may be just one, or two or more. Also, in this example, the information processing device 10 can communicate with the imaging device 30, but the imaging device 30 is not a mandatory component.

[0014] Furthermore, in this example, the system is configured so that users communicate with the information processing device 10 via their respective user terminals 20. However, the system is not limited to this configuration, and each user may use the system via the input and output units of the information processing device 10. In other words, the system 1 can function with the information processing device 10 alone, without including user terminals and a network. Alternatively, all the functions of the information processing device 10 can be implemented in the user terminal, allowing the user terminal to function independently.

[0015] Figure 2 is a block diagram showing the functional configuration of the information processing device 10. The information processing device 10 is a device used by system administrators and others when operating and managing various services, and may be a general-purpose computer such as a workstation or personal computer, or it may be logically implemented using cloud computing technology.

[0016] The information processing device 10 in this example includes a control unit 11, a storage unit 12, an input unit 13, an output unit 14, and a communication unit 15. The information processing device 10 receives various input information from the input unit 13 or from an external device (such as a user terminal 20) via the communication unit 15. The control unit 11 executes processing according to a program corresponding to the input information and can output the processing result of the program (e.g., images or sound) from the output unit 14 or transmit it to an external device such as a user terminal 20 via the communication unit 15 for output. Note that a part of the above program may be transmitted to another information processing device (e.g., a user terminal, another server, etc.) and executed on that other information processing device. In this case, the other information processing device can be, for example, a smartphone, a mobile phone terminal, a tablet terminal, a personal computer, etc., and is connected to the information processing device 10 wirelessly or by wire via a network such as the Internet.

[0017] The control unit 11 handles the exchange of information between each part and controls the entire information processing device 10. This is achieved, for example, by having the CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit) execute a program stored in a predetermined memory.

[0018] The control unit 11 includes, for example, an information storage processing unit 111 that performs processing such as storing and updating (including changing and deleting) various types of information, a three-dimensional model display processing unit 112 that displays a three-dimensional model on a display unit of a user terminal or the like, an input information receiving processing unit 113 that receives input information from a user terminal, an orthomosaic image generation processing unit 114 that generates an orthomosaic image, and an orthomosaic image display processing unit 115 that displays an orthomosaic image on a display unit of a user terminal or the like. The control unit 11 may also include a model generation unit 116 that generates three-dimensional model data based on captured image data.

[0019] The model generation unit 116 generates a three-dimensional model, for example, by performing SfM (Structure from Motion) processing. SfM processing is a process that generates a three-dimensional model using the principle of triangulation based on multiple two-dimensional images generated by capturing an object having multiple feature points from multiple imaging positions. Preferably, SfM processing includes processes such as bundle adjustment to minimize reprojection errors. The three-dimensional model may be data generated without SfM processing, and may include three-dimensional data created by various analyses of the on-site conditions, such as using CG data or BIM models.

[0020] The model generation unit 116 acquires multiple two-dimensional images, including the subject image, from a frame or still image dataset that constitutes the video data. Next, the model generation unit 116 extracts feature points of the subject from each two-dimensional image and associates the feature points between the two-dimensional images (feature point matching). Next, based on the associated feature points, the model generation unit 116 estimates the imaging position 30 (camera position) and camera orientation (direction) at the time each two-dimensional image was generated, and calculates the three-dimensional coordinates of the three-dimensional points corresponding to the feature points based on the estimation results. The model generation unit 116 calculates the three-dimensional coordinates of multiple three-dimensional points corresponding to multiple feature points. In this case, the three-dimensional coordinates represent coordinates in a three-dimensional coordinate system set in virtual space. The three-dimensional coordinate system is defined by mutually orthogonal X, Y, and Z axes. The three-dimensional coordinate system CS may be a coordinate system with a predetermined position in the virtual space VS as its origin, or it may be a coordinate system to which geospatial coordinates (ground coordinates) or actual size information is assigned. In this way, the model generation unit 116 generates point cloud data consisting of multiple three-dimensional points. The model generation unit 116 may add one or more pieces of information from among color information, brightness information, normal vector information, and reflectance information to each point that makes up the point cloud.

[0021] Furthermore, the model generation unit 116 may perform MVS (Multi View Stereo) processing in addition to SfM processing. By performing MVS processing, the model generation unit 116 can generate point cloud data that represents a dense point cloud. In this case, a three-dimensional model is constructed from the dense point cloud. It is preferable for the model generation unit 116 to assign faces to the three-dimensional model based on the point cloud data. One example of the process of assigning faces is the process of converting the point cloud data that constitutes the three-dimensional model into mesh data. In this case, for example, the model generation unit 116 generates multiple polygonal faces (for example, triangles) that are generated by connecting points in the point cloud data that constitutes the three-dimensional model, and represents the three-dimensional model with multiple polygonal faces. In this case, for example, the model generation unit 116 generates TIN (Triangulated Irregular Network) data based on the point cloud data, and represents the three-dimensional model with the TIN data. As a result, the three-dimensional model is represented by a set of triangular faces. The process of assigning faces to a three-dimensional model is not particularly limited and may include the following methods. For example, the model generation unit 116 assigns a face to each point that makes up the point cloud data. In this case, the face is, for example, a circular face centered on each point that makes up the point cloud data. Alternatively, for example, the model generation unit 116 assigns faces to the three-dimensional model made up of point cloud data by executing an octree algorithm. In this case, specifically, the model generation unit 116 represents the three-dimensional model by multiple cubes by recursively dividing the three-dimensional model into eight cubes (octants). Alternatively, for example, the model generation unit 116 assigns faces to the three-dimensional model by converting the point cloud data into surface data.

[0022] Furthermore, the model generation unit 116 may add material information to the surfaces assigned to the three-dimensional model based on a two-dimensional image. The material information includes information about the color and pattern of the subject. For example, the model generation unit 116 may perform a process of mapping a texture to each surface (e.g., each polygon) that constitutes the mesh data of the three-dimensional model. In this case, the mesh data is, for example, TIN data.

[0023] The memory unit 12 stores various types of information. The memory unit 12 stores programs for executing various control processes and functions within the control unit 11, input information, etc., and is composed of one or any combination thereof of RAM (Random Access Memory), ROM (Read Only Memory), flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), and other storage devices.

[0024] The storage unit 12 includes, for example, a 3D model storage unit 121, a selection storage unit 122, and an ortho-image storage unit 123. The 3D model storage unit 121 stores various information related to the 3D model data. The 3D model data is generated based on multiple captured image data of a real-world subject. For example, the control unit of the information processing device generates a 3D model of a subject (target area, object) based on multiple 2D images generated by capturing the subject from multiple different imaging positions. The 3D model is placed in a virtual space (virtual 3D space). The captured image data may be, for example, images captured by an imaging device 30 and transmitted to the information processing device.

[0025] The input unit 13 is for the user to input various types of information as described above, and can be implemented using, for example, a keyboard, mouse, touch panel, microphone, etc.

[0026] The output unit 14 can output information generated by the control unit 11 or information received via the communication unit. The output unit 14 may include, for example, an image output unit (display unit) such as a liquid crystal display (LCD) or a touch panel, and an audio output unit such as a speaker. It can output various types of information based on commands from the control unit 11. Alternatively, the output unit 14 may output (transmit) various types of information to other information processing devices (user terminals, etc.) via the communication unit 15.

[0027] The communication unit 15 is for communicating with other information processing devices and has the function of a receiving unit that receives various data and signals transmitted from other information processing devices, etc., and the function of a transmitting unit that transmits various data and signals to other information processing devices, etc., in accordance with commands from the control unit 11.

[0028] The user terminal 20 is used by users who browse websites and perform other actions using the services provided by this system 1. The user terminal 20 can be, for example, an information processing device such as a personal computer, tablet, smartphone, or mobile phone.

[0029] Figure 3 is a functional block diagram showing the user terminal 20 of Figure 1. The user terminal 20 includes a control unit 21, a storage unit 22, an input unit 23, an output unit 24, and a communication unit 25.

[0030] The control unit 21, like the control unit 11 described above, handles the exchange of information between each unit and controls the entire user terminal 20. This is achieved, for example, by having the CPU, MPU, or GPU execute a program stored in a predetermined memory.

[0031] The memory unit 22, like the memory unit 12 described above, stores programs for executing various control processes and functions within the control unit 21, input information, etc., and is composed of one or any combination thereof of RAM, ROM, flash memory, HDD, SSD, or other storage devices.

[0032] The input unit 23, like the input unit 13 described above, is for the user to input various types of information as described above, and can be implemented using, for example, a touch panel, keyboard or mouse, microphone, etc.

[0033] The output unit 24, like the output unit 14 described above, can output information generated by the control unit 21 or information received via the communication unit. For example, it may include an image output unit (display unit) such as a liquid crystal display or touch panel, and an audio output unit such as a speaker.

[0034] The communication unit 25, like the communication unit 15 described above, is for communicating with other information processing devices and has the function of a receiving unit that receives various data and signals transmitted from other information processing devices, etc., and a transmitting unit that transmits various data and signals to other information processing devices, etc., in accordance with commands from the control unit 21. Users input various information via the input unit 23 of the user terminal 20 used by each user and transmit the input information to the information processing device 10, which acts as a server device, via the communication unit 25. They also receive the processing results output from the information processing device 10 and confirm them via the output unit 24.

[0035] The imaging device 30 can be any device capable of generating image data such as still images or videos through photography and transmitting it to the information processing device. The imaging device 30 may be, for example, a portable terminal such as a digital camera, smartphone, or tablet terminal, or it may be a mobile device such as an unmanned mobile device or a manned mobile device. Unmanned mobile devices include, for example, unmanned aircraft (unmanned flying objects) such as drones, unmanned ground vehicles, unmanned submersibles, or unmanned surface vessels. Unmanned ground vehicles include, for example, unmanned ground vehicles or unmanned ground vehicles that mimic living organisms (for example, snake-shaped unmanned ground vehicles). Manned mobile devices include, for example, aircraft, automobiles, ships, or submarines.

[0036] Figure 4 is a flowchart showing an example of an information processing method performed in this system. The information processing method includes an information storage process (S1) performed by the control unit 11, a three-dimensional model display process (S2), an input reception process (S3), an orthomosaic image generation process (S4), and an orthomosaic image display process (S5).

[0037] The information storage processing unit 111 of the control unit 11 stores three-dimensional model data generated based on multiple captured image data of a real-world subject (S1). The three-dimensional model data may be data generated by the model generation unit 116, or data acquired from an external device, etc.

[0038] The information storage processing unit 111 stores information on the selection of a specified range and a specified direction. The selection of the specified range and direction is stored in association with predetermined faces of the 3D model, such as the top, bottom, side, outer, and inner surfaces. The selection of the specified range and direction may be set by the system administrator, or the control unit may detect the faces that make up the 3D model and automatically assign a selection corresponding to each face. The specified direction is indicated as a direction perpendicular to the faces that make up the specified range. The specified direction may include two directions, such as the front and back, and either one or only one direction, such as the front or back.

[0039] The 3D model display processing unit 112 of the control unit 11 causes the 3D model to be displayed on a display unit such as a user terminal (S2). For example, in the examples of Figures 5 and 6, the 3D model display processing unit 112 displays a 3D model of the overpass 50. On this screen, the 3D model (or viewpoint) can be rotated in the virtual space in response to user operations (such as mouse movements with clicks or drag operations), and the 3D model viewed from any direction, images of options, etc., can be displayed. Furthermore, similar to known 3D model display devices, zooming in and out may also be possible in response to user operations.

[0040] In the examples shown in Figures 5 and 6, the 3D model display processing unit 112 displays a 3D model of the overpass 50, including the bridge girders 51 and bridge piers 52, and a cursor 60 that moves in response to the user's touch panel operation or mouse operation on the display unit's screen S. When the cursor 60 moves near the bridge piers 52 in response to the user's operation, a frame 71 indicating the selection options within the specified range is displayed. In other words, the 3D model display processing unit 112 detects the position of the cursor 60 and displays the selection options associated with that position. If there are no selection options associated with that position in the storage unit, no selection options are displayed.

[0041] Furthermore, when the cursor 60 moves in response to user operation, options not associated with the cursor 60's position are hidden, and other options associated with the cursor 60's position (frame 73, arrow 74) are displayed, as shown in Figure 6. The frame 71 in Figure 5 is associated with the side (inner surface) 52a of the bridge pier 52, and when the cursor 60 is moved near the side 52a, the frame 71 is displayed. The frame 71 is displayed at a predetermined distance from the associated surface 52a, but is not limited to this, and may be displayed at the same position as the surface. In the example in Figure 6, when the cursor 60 is located near the lower surface 51a of the bridge girder 51, the frame 73 and arrow 74 corresponding to the lower surface 51a are displayed. Note that the frames 71 and 73 are displayed as rectangular frames, but are not limited to this, and can be set to any shape. Preferably, the shape of the frames 71 and 73 corresponds to the surface in question (the side 52a in the example of Figure 5). Also, preferably, the size of the frames 71 and 73 corresponds to the surface in question. The corresponding shapes include, for example, identical or similar shapes. The corresponding sizes include, for example, identical sizes or similar sizes (such as a range that is larger or smaller by a predetermined value or predetermined percentage relative to the area of ​​the symmetry plane). In the example in Figure 5, an arrow 72 indicating the option for the specified direction is displayed, and in the example in Figure 6, an arrow 74 is displayed. In the illustrated examples, only one frame 71, 73 and one arrow 72, 74 are shown in each, but depending on the cursor position, multiple options for specified ranges and specified directions may be displayed simultaneously, and one of them may be selected.

[0042] The input information reception processing unit 113 of the control unit 11 receives input information from a user terminal or the like (S3). For example, it receives the selection of an option or the specified input of a specified surface by touch, long touch, click of a mouse, etc. The selection of an option may be to simultaneously select the displayed specified range and specified direction, or the specified direction may be selected after the specified range is selected. The input information reception processing unit 113 can receive, for example, the specification of the frame 71 in FIG. 5 or the frame 73 in FIG. 6. Alternatively, it may be possible to select a plurality of specified ranges simultaneously. For example, the frame 71 in FIG. 5 and the frame 73 in FIG. 6 may be specified in order or simultaneously. Also, the control unit may not display options. In that case, for example, the user may be able to directly select the side surface 52a of the pier 52 or the lower surface 51a of the bridge girder 51 by a selection operation such as touch or click.

[0043] The ortho-image generation processing unit 114 of the control unit 11 executes an ortho-image generation process for generating an ortho-image of a surface corresponding to the option specified by the specified input (for example, at least one of the side surface 52a of the pier 52 and the lower surface 51a of the bridge girder 51) (S4). The ortho-image generation processing unit 114 can generate an ortho-image of the specified range and specified direction included in the specified information by orthographically projecting and converting the specified surface of the three-dimensional model data. The method of ortho-correction may be other known methods.

[0044] The ortho-image display processing unit 115 of the control unit 11 displays the ortho-image on the display unit of the user terminal or the like (S5). When the ortho-image display processing unit 115 receives input information via the input unit of the information processing device, it can display the ortho-image on the display unit of the information processing device. That is, it is preferable to display the ortho-image on the display unit of the terminal device that input the input information. In this way, according to the present embodiment, while the user views the three-dimensional model displayed on the screen, the user can select a desired surface and confirm the ortho-image. The user can inspect the cracks, peeling, distortion, etc. of the target surface from the ortho-image.

[0045] As described above, the information processing device of this embodiment includes a storage unit that stores three-dimensional model data generated based on multiple captured image data of subjects in the real world, a display unit used by the user that displays the three-dimensional model data, and a control unit that performs a reception process that receives specified information from the user, including a specified range and direction for which orthorectification is desired in the three-dimensional model data, an orthoimage generation process that generates an orthoimage of the specified range and direction included in the specified information by performing an orthographic projection transformation on the three-dimensional model data of the region corresponding to the specified range and direction, and an output process that displays the generated orthoimage on the display unit. With this configuration, orthoimages can be generated more efficiently than in conventional orthoimage generation methods (where multiple two-dimensional images including a subject are selected, scaled, and then orthoimaged).

[0046] In this embodiment, the storage unit stores the selection of specified ranges and directions associated with multiple locations in the three-dimensional model data, and the control unit displays the selection of specified ranges and directions associated with the location specified in response to the user's input operation while the three-dimensional model data is displayed on the display unit, and generates orthophotos corresponding to the specified ranges and directions of one or more selections selected by the user. With this configuration, the user only needs to specify pre-prepared ranges and directions, so orthophotos can be generated more efficiently.

[0047] In this embodiment, the 3D model data may be generated by pasting 2D image data onto 3D point cloud data obtained by SFM processing of the plurality of captured image data.

[0048] In this embodiment, the options for the specified range may be displayed as a rectangular frame or surface. According to this, the specified range can be easily understood visually. Note that the shape of the options for the specified range is not limited to rectangular and may be circular, elliptical, or a shape along the outer shape of the 3D model. Also, the surface serving as the option for the specified range may be a flat surface or a curved surface or may be any surface (such as the upper surface, lower surface, side surface, etc.) of the 3D model data itself or a surface displayed at a predetermined interval from the surface.

[0049] In this embodiment, the options for the specified direction may be displayed as arrows. According to this, the specified direction can be easily understood visually.

[0050] In this embodiment, the options for the specified range and the specified direction may include the inner surface viewed from the inside of the 3D model data. The inner surface is the surface of the structure as the subject viewed from the inside and includes the inner surfaces of tunnels and bridge piers, the ceiling surface, etc. According to this, not only the outer surface of the 3D model but also an orthographic image of the inner surface can be generated.

[0051] In this embodiment, the options for the specified range and the specified direction may include the lower surface viewed from the lower side of the 3D model data. According to this, not only the upper surface of the 3D model but also an orthographic image of the lower surface can be generated. Note that the options for the specified range and the specified direction may include the upper surface or the outer surface of the 3D model.

[0052] As described above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field of the present disclosure can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present disclosure.

[0053] The apparatus described herein may be implemented as a single device, or as a group of devices (e.g., cloud servers) that are partially or entirely connected by a network. For example, the control unit and storage unit of an information processing apparatus may be implemented as different servers connected to each other by a network.

[0054] The series of processes performed by the apparatus described herein may be implemented using software, hardware, or a combination of software and hardware. Computer programs for implementing each function of the information processing apparatus according to this embodiment can be created and implemented on a PC or the like. Furthermore, a computer-readable recording medium on which such a computer program is stored can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, and flash memory. Alternatively, the computer program may be distributed without using a recording medium, for example, via a network.

[0055] Furthermore, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0056] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0057] Furthermore, the following configurations also fall within the technical scope of this disclosure. (Item 1) An orthophoto generation device comprising: a storage unit that stores three-dimensional model data generated based on multiple captured image data of subjects in the real world; a display unit used by the user that displays the three-dimensional model data and a control unit that performs a reception process that receives from the user a specified range and a specified direction for which the user wishes to orthorectify the three-dimensional model data; an orthophoto generation process that generates an orthophoto of the specified range and the specified direction included in the specified information by performing an orthophoto projection transformation on the three-dimensional model data of the region corresponding to the specified range and the specified direction; and an output process that displays the generated orthophoto on the display unit. (Item 2) The storage unit stores selections of a specified range and a specified direction associated with a plurality of locations in the three-dimensional model data, the control unit displays the selections of a specified range and a specified direction associated with a location specified in response to the user's input operation while the three-dimensional model data is displayed on the display unit, and generates an orthoimage corresponding to the specified range and specified direction of one or more selections selected by the user. This is the orthoimage generation apparatus according to claim 1. (Item 3) The three-dimensional model data is generated by pasting two-dimensional image data onto three-dimensional point cloud data obtained by SFM processing of a plurality of captured image data. This is the orthoimage generation apparatus according to claim 1 or 2. (Item 4) The selections of the specified range are displayed as a rectangular frame or surface. This is the orthoimage generation apparatus according to claim 1 or 2. (Item 5) The selections of the specified direction are displayed as arrows. This is the orthoimage generation apparatus according to claim 1 or 2. (Item 6) The selections of the specified range and specified direction include the inner surface as viewed from the inside of the three-dimensional model data. This is the orthoimage generation apparatus according to claim 1 or 2. (Item 7) The orthoimage generating apparatus according to claim 1 or 2, wherein the selection of the specified range and specified direction includes the lower surface viewed from below the three-dimensional model data.(Item 8) An orthophoto generation method, wherein the control unit stores three-dimensional model data generated based on multiple captured image data of subjects in the real world in a storage unit, displays the three-dimensional model data on a display unit used by the user, and receives specified information from the user, including a specified range and a specified direction for which orthorectification is desired in the three-dimensional model data; generates an orthophoto of the specified range and specified direction included in the specified information by performing an orthographic projection transformation on the three-dimensional model data of the region corresponding to the specified range and specified direction; and outputs the generated orthophoto on the display unit. (Item 9) A program that causes a control unit to execute the following: a process for storing three-dimensional model data generated based on multiple captured image data of subjects in the real world; a reception process for displaying the three-dimensional model data on a display unit used by the user and receiving specified information from the user, including a specified range and a specified direction for which the user wishes to orthorectify the three-dimensional model data; an orthoimage generation process for generating an orthoimage of the specified range and specified direction included in the specified information by performing an ortho-projection transformation on the three-dimensional model data of the region corresponding to the specified range and specified direction; and an output process for displaying the generated orthoimage on the display unit.

[0058] 1 Ortho-image generation system 10 Information processing device 20 User terminal 111 Information storage processing unit 112 3D model display processing unit 113 Input information receiving processing unit 114 Ortho-image generation processing unit 115 Ortho-image display processing unit 116 Model generation unit

Claims

1. An orthophoto generation device comprising: a storage unit that stores three-dimensional model data generated based on multiple captured image data of subjects in the real world; a display unit used by the user that displays the three-dimensional model data; a reception process that receives specified information from the user, including a specified range and direction for which orthorectification is desired in the three-dimensional model data; an orthophoto generation process that generates an orthophoto of the specified range and direction included in the specified information by performing orthophoto projection transformation on the three-dimensional model data of the region corresponding to the specified range and direction; and an output process that displays the generated orthophoto on the display unit.

2. The orthoimage generating apparatus according to claim 1, wherein the storage unit stores selections of a specified range and a specified direction associated with a plurality of locations in the three-dimensional model data, the control unit displays the selections of a specified range and a specified direction associated with a location specified in response to the user's input operation while the three-dimensional model data is displayed on the display unit, and generates orthoimages corresponding to the specified range and specified direction of one or more selections selected by the user.

3. The orthoimage generation apparatus according to claim 1 or 2, wherein the three-dimensional model data is generated by pasting two-dimensional image data onto three-dimensional point cloud data obtained by SFM processing of the plurality of captured image data.

4. The ortho-image generating apparatus according to claim 1 or 2, wherein the selection of the specified range is displayed in a rectangular frame or surface.

5. The ortho-image generating apparatus according to claim 1 or 2, wherein the selection of the specified direction is indicated by an arrow.

6. The orthoimage generating apparatus according to claim 1 or 2, wherein the selection of the specified range and specified direction includes the inner surface as viewed from the inside of the three-dimensional model data.

7. The orthoimage generating apparatus according to claim 1 or 2, wherein the selection of the specified range and specified direction includes the lower surface viewed from below the three-dimensional model data.

8. An orthophoto generation method, wherein the control unit stores three-dimensional model data generated based on multiple captured image data of subjects in the real world in a storage unit, displays the three-dimensional model data on a display unit used by the user, and receives specified information from the user, including a specified range and direction for which orthorectification is desired in the three-dimensional model data; generates an orthophoto image of the specified range and direction included in the specified information by performing an orthographic projection transformation on the three-dimensional model data of the region corresponding to the specified range and direction; and outputs the generated orthophoto image on the display unit.

9. A program that causes a control unit to execute the following: a process for storing three-dimensional model data generated based on multiple captured image data of a subject in the real world; a reception process for displaying the three-dimensional model data on a display unit used by the user and receiving specified information from the user, including a specified range and direction for which orthorectification is desired in the three-dimensional model data; an orthoimage generation process for generating an orthoimage of the specified range and direction included in the specified information by performing an ortho-projection transformation on the three-dimensional model data of the region corresponding to the specified range and direction; and an output process for displaying the generated orthoimage on the display unit.

Images