Information processing method, information processing device, and information processing program

Abstract

An information processing method according to the present invention comprises: displaying, on a display of an information terminal, a captured image which was obtained by capturing an image of a prescribed space and a bird's-eye view which corresponds to the prescribed space; receiving selection of arbitrary two points in the bird's-eye view or the captured image; and displaying, in the captured image, the distance between two points in the captured image which correspond to the two points that are selected in the bird's-eye view, displaying, in the bird's-eye view, the distance between two points in the bird's-eye view which correspond to the two points that are selected in the captured image, or displaying, in the captured image, the distance between the two points that are selected in the captured image.

Description

Information processing method, information processing device, and information processing program

[0001] The present disclosure relates to techniques for displaying images.

[0002] For example, the technology in Patent Document 1 discloses a technique in which, when a drag operation is performed to draw a straight line from a desired point to another point within the drawing information, distance information indicating the straight-line distance between these points is displayed overlapping the drawing information.

[0003] However, in the above-mentioned conventional technology, although the distance between two desired points in a drawing is displayed, the distance between the desired two points in the captured image is not presented to the user, and the accuracy of the distance in the depth direction is low, so further improvement is needed. Furthermore, the number of processing steps of the computer required to confirm the distance between two points increases, which may increase the burden on the computer.

[0004] Japanese Patent Application Laid-Open No. 2022-051601

[0005] The present disclosure has been made to solve the above problems, and aims to provide a technology that can present the distance between two desired points in a captured image to a user with high accuracy, reduce the computer processing steps required to confirm the distance between the two points, and reduce the burden on the computer.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, and includes displaying, on a display of an information terminal, a photographed image of a specified space and a bird's-eye view corresponding to the specified space, accepting the selection of any two points in the bird's-eye view or the photographed image, and displaying, in the photographed image, the distance between two points in the photographed image that correspond to the two points selected in the bird's-eye view, displaying, in the bird's-eye view, the distance between two points in the bird's-eye view that correspond to the two points selected in the photographed image, or displaying, in the photographed image, the distance between the two points selected in the photographed image.

[0007] According to the present disclosure, it is possible to accurately present to a user the distance between two desired points in a captured image, and it is also possible to reduce the number of computer processing steps required to confirm the distance between the two points, thereby reducing the burden on the computer.

[0008] 1 is a diagram showing the overall configuration of an information processing system in this embodiment. FIG. 1 is a diagram showing an example of a blueprint screen on which a plurality of photography location icons are mapped in this embodiment. FIG. 2 is a diagram showing an example of a blueprint screen on which one photography location icon is mapped in this embodiment. FIG. 3 is a diagram showing an example of a photographed image screen that is displayed on a display when a photography location icon is selected in this embodiment. FIG. 4 is a diagram showing a comparison display screen that displays a photographed image and a blueprint in this embodiment. FIG. 5 is a diagram showing a comparison display screen that displays the distance between two points in a photographed image in this embodiment. A flowchart showing an example of distance display processing of a server in this embodiment. A diagram showing a comparison display screen that displays the distance between two points in a photographed image in a first variation of this embodiment. A flowchart showing an example of distance display processing of a server in the first variation of this embodiment. A diagram showing a comparison display screen that displays the distance between two points in a photographed image in a second variation of this embodiment. A flowchart showing an example of distance display processing of a server in the second variation of this embodiment.

[0009] (Findings that form the basis of the present disclosure) A user interface is being developed that displays multiple photography location icons indicating locations where actual photographs of a construction site were taken on a two-dimensional blueprint of the construction site, etc., and when one photography location icon is selected, displays an image taken at the photography location indicated by that icon. This allows a construction site manager to understand the situation at the construction site without visiting the construction site.

[0010] Here, there is a desire from users to check the distance between two desired points in a photographed image while viewing the photographed image taken at the photographing location during or after photographing.

[0011] However, with the above-described conventional technology, the distance between two desired points is displayed on a drawing, but the distance between the two desired points in the captured image is not presented to the user. Furthermore, it is difficult to accurately read distance information in the depth direction from the image. Therefore, with the conventional technology, it is difficult for a user to accurately confirm the distance between two desired points in the captured image while viewing the captured image taken at the shooting location. Furthermore, the number of computer processing steps required to confirm the distance between the two points increases, which may increase the burden on the computer.

[0012] In order to solve the above problems, the following techniques are disclosed.

[0013] (1) An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, and includes displaying, on a display of an information terminal, a photographed image of a specified space and a bird's-eye view corresponding to the specified space; accepting the selection of any two points in the bird's-eye view or the photographed image; and displaying, in the photographed image, the distance between two points in the photographed image that correspond to the two points selected in the bird's-eye view; displaying, in the bird's-eye view, the distance between two points in the bird's-eye view that correspond to the two points selected in the photographed image; or displaying, in the photographed image, the distance between the two points selected in the photographed image.

[0014] According to this configuration, the distance between two points in the captured image corresponding to two points selected in the bird's-eye view is displayed in the captured image. Alternatively, the distance between two points in the bird's-eye view corresponding to two points selected in the captured image is displayed in the bird's-eye view. Alternatively, the distance between two points selected in the captured image is displayed in the captured image. Therefore, the distance between two desired points in the captured image can be presented to the user with high accuracy. Furthermore, the distance can be presented with high accuracy in both the X and Y directions, including the depth (Y direction). Furthermore, because the distance between two desired points in the captured image is presented to the user, the computer processing steps required to confirm the distance between the two points can be reduced, thereby reducing the burden on the computer.

[0015] (2) In the information processing method described in (1) above, the acceptance may include accepting the selection of any two points on the bird's-eye view, and the display of the distance may include calculating the distance between the two points selected on the bird's-eye view and displaying the calculated distance in the captured image as the distance between the two points in the captured image.

[0016] With this configuration, the distance between two points on the bird's-eye view is calculated, so that the distance can be calculated accurately. Furthermore, the calculated distance is displayed in the captured image as the distance between two points, so that the user can be presented with the accurate distance between two desired points in the captured image.

[0017] (3) In the information processing method described in (1) above, the acceptance may include accepting the selection of any two points in the captured image, and the display of the distance may include calculating the distance between two points in the bird's-eye view that correspond to the two points selected in the captured image, and displaying the calculated distance in the bird's-eye view.

[0018] According to this configuration, the distance between two points on the bird's-eye view that correspond to two selected points on the captured image is calculated, so that an accurate distance can be calculated. Furthermore, the calculated distance is displayed on the bird's-eye view, so that the accurate distance between two desired points on the captured image can be presented to the user.

[0019] (4) In the information processing method described in (1) above, the acceptance may include accepting the selection of any two points in the captured image, and the display of the distance may include calculating the distance between two points in the bird's-eye view corresponding to the two points selected in the captured image, and displaying the calculated distance in the captured image.

[0020] According to this configuration, the distance between two points on the bird's-eye view that correspond to two selected points on the photographed image is calculated, so that the accurate distance can be calculated. Furthermore, the calculated distance is displayed in the photographed image, so that the accurate distance between two desired points in the photographed image can be presented to the user.

[0021] (5) In the information processing method described in any one of (1) to (4) above, the captured image may be configured as an omnidirectional image.

[0022] According to this configuration, by changing the line of sight, a photographed image of the surroundings of the photographing point can be obtained, so that the situation in the specified space can be confirmed in more detail.

[0023] (6) In the information processing method described in (5) above, the display of the photographed image and the bird's-eye view may include, when a line of sight of the photographed image is changed, rotating the bird's-eye view so that the line of sight in the bird's-eye view coincides with the top of the bird's-eye view. The information processing method described in (5) above may also include displaying a figure such as an arrow or a triangle indicating the line of sight on the bird's-eye view, thereby clearly indicating the line of sight on the bird's-eye view.

[0024] According to this configuration, the user can check the line of sight of the captured image on the bird's-eye view.

[0025] (7) In the information processing method described in any one of (1) to (6) above, displaying the captured image and the bird's-eye view may include displaying one of a plurality of captured images taken consecutively, and further including displaying a switch icon for switching from the displayed one captured image to an image taken before the one captured image or an image taken after the one captured image, and displaying the distance may include, when the switch icon is selected, switching to an image taken before the one captured image or an image taken after the one captured image, and displaying the distance within the switched image.

[0026] With this configuration, even if the image is switched to an image taken before or after the first image, the distance between two desired points in the switched image can be presented to the user.

[0027] (8) In the information processing method described in any one of (1) to (7) above, the bird's-eye view may be configured as a two-dimensional plan view of the specified space, and the receiving may include receiving a selection of any two points in the horizontal direction on the two-dimensional plan view.

[0028] According to this configuration, it is possible to present to the user the desired distance between two points in the horizontal direction within the captured image with high accuracy.

[0029] (9) In the information processing method described in any one of (1) to (7) above, the bird's-eye view may be composed of a three-dimensional simulation image of the specified space or a two-dimensional side view of the specified space, and the acceptance may include accepting the selection of any two points in a vertical direction in the three-dimensional simulation image or the two-dimensional side view.

[0030] According to this configuration, it is possible to present to the user the desired distance between two points in the vertical direction within the captured image with high accuracy.

[0031] Furthermore, the present disclosure can be realized not only as an information processing method that executes the characteristic processes described above, but also as an information processing device having a characteristic configuration corresponding to the characteristic processes executed by the information processing method. Furthermore, the present disclosure can also be realized as a computer program that causes a computer to execute the characteristic processes included in such an information processing method. Therefore, the same effects as those of the above information processing method can also be achieved in the following other aspects.

[0032] (10) An information processing device according to another aspect of the present disclosure is an information processing device including a processor, wherein the processor displays, on a display of an information terminal, a photographed image of a specified space and a bird's-eye view corresponding to the specified space, accepts the selection of any two points in the bird's-eye view or the photographed image, and displays, in the photographed image, the distance between two points in the photographed image that correspond to the two points selected in the bird's-eye view, or displays, in the bird's-eye view, the distance between two points in the bird's-eye view that correspond to the two points selected in the photographed image, or displays, in the photographed image, the distance between the two points selected in the photographed image.

[0033] (11) An information processing program according to another aspect of the present disclosure causes a computer to display, on a display of an information terminal, a photographed image of a specified space and a bird's-eye view corresponding to the specified space, accept the selection of any two points in the bird's-eye view or the photographed image, and display, in the photographed image, the distance between two points in the photographed image that correspond to the two points selected in the bird's-eye view, or display, in the bird's-eye view, the distance between two points in the bird's-eye view that correspond to the two points selected in the photographed image, or display, in the photographed image, the distance between the two points selected in the photographed image.

[0034] (12) A non-transitory computer-readable recording medium according to another aspect of the present disclosure records the information processing program described in (11) above.

[0035] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components. Furthermore, in all embodiments, the respective contents can be combined.

[0036] FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to the present embodiment.

[0037] The information processing system 1 displays, on a display 23 of an information terminal 20, a captured image of a predetermined space and a bird's-eye view corresponding to the predetermined space, and also accepts the selection of any two points on the bird's-eye view and displays, within the captured image, the distance between the two points in the captured image that correspond to the two points selected on the bird's-eye view. The information processing system 1 includes a server 10, an information terminal 20, an imaging device 30, and a communication device 40.

[0038] The server 10 is an example of an information processing device and a computer. The server 10, the information terminal 20, and the communication device 40 are connected to each other so as to be able to communicate with each other via a network NT. An example of the network NT is the Internet. The server 10 is, for example, a cloud server configured with one or more computers. However, this is just one example, and the server 10 may be configured as an edge server or may be implemented in the information terminal 20. The aspect in which the server 10 is implemented in the information terminal 20 is an example of the aspect in which the information terminal 20 is configured as an information processing device.

[0039] The information terminal 20 is carried by a user. The user is, for example, a manager of a predetermined space in which the image capturing device 30 captures images. The predetermined space is, for example, a construction site. However, this is just one example, and the predetermined space may be a construction site, a factory, a store, an office, or the like. The information terminal 20 may be configured as a portable computer such as a smartphone or a tablet computer, or as a stationary computer. The information terminal 20 displays images on a display 23 under control of the server 10. Although one information terminal 20 is illustrated in the example of FIG. 1, multiple information terminals may be connected to the server 10 via a network NT. The information terminal 20 includes a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0040] The communication unit 21 is a communication interface that connects the information terminal 20 to the network NT. The communication unit 21 transmits instruction signals indicating various instructions received from the user by the operation unit 24 to the server 10. The communication unit 21 receives display data for displaying various display screens from the server 10.

[0041] The processor 22 is configured by, for example, a central processing unit (CPU), and displays on the display 23 a display screen indicated by the display data received by the communication unit 21 .

[0042] The display 23 is configured with various display devices such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays various display screens under the control of the processor 22 .

[0043] The operation unit 24 is composed of, for example, a keyboard, a touch panel, a mouse, and the like, and accepts various instructions input by the user.

[0044] The communication device 40 is configured by a mobile information terminal such as a smartphone or a tablet computer, and connects the photographing device 30 to the network NT. The communication device 40 and the photographing device 30 are connected to each other so that they can communicate with each other via a short-range wireless communication path such as Bluetooth (registered trademark).

[0045] The image capturing device 30 is, for example, an omnidirectional camera, and captures images at a predetermined frame rate. An omnidirectional camera is also called a 360-degree camera, and is a camera that can capture images in all directions of 360 degrees. The image capturing device 30 is, for example, a portable image capturing device carried by a photographer. The photographer may be, for example, a worker or a site supervisor at a construction site. The image capturing device 30 may be held in the photographer's hand, or may be worn on the photographer's body (for example, on the head). The image capturing device 30 may also be a regular camera.

[0046] The photographer moves through the construction site while taking pictures of the site with the camera device 30. At the start point of the shooting, the photographer points the camera device 30 in a northerly direction, presses the camera button on the camera device 30, and starts shooting. When the photographer reaches the end point of the shooting, he presses the camera button again to stop shooting. When the shooting is finished, the camera device 30 transmits the series of captured images to the communication device 40.

[0047] The image includes the date and time of the image capture. The date and time of the image capture is obtained, for example, by a clock provided in the image capture device 30. In this example, the image capture device 30 captures images at a predetermined frame rate (e.g., 10 frames per second), so the image capture location is defined in frame cycle units. However, this would result in an enormous amount of data, so the image capture location may be defined every predetermined time period (e.g., 1 second, 5 seconds, or 10 seconds).

[0048] The communication device 40 receives the multiple images transmitted by the image capturing device 30. The communication device 40 also displays a blueprint of the construction site and accepts designation by the photographer of the position of the image capturing start point and the position of the image capturing end point on the blueprint.

[0049] The start and end points of photography are specified by the photographer inputting instructions specifying the positions on the blueprint screen of the site displayed on the display of the communication device 40. Two-dimensional coordinate axes are defined on the blueprint screen. Therefore, the start and end points of photography are each defined by two-dimensional coordinate values. The start and end points of photography are used by the server 10 to specify the position of each photography point and the photography direction of each image.

[0050] The communication device 40 transmits photography information including the multiple captured images to the server 10 via the network NT. Photography information is generated each time a single photography operation is performed. One photography operation refers to a series of operations from when a worker holding the photography device 30 starts taking pictures at a construction site until when he finishes taking pictures. Multiple images are captured during one photography operation. The photography information includes the multiple images taken and meta information for each of the multiple images. The meta information includes a photography ID, a photography date and time, and a blueprint ID. The photography ID is an identifier for identifying the photography operation. The photography date and time is the date and time when the image was taken. The blueprint ID is an identifier for identifying a blueprint for a specified space corresponding to the image.

[0051] The server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 is configured, for example, by a central processing unit (CPU). The processor 11 includes an acquisition unit 111, a selection acceptance unit 112, and a display control unit 113. The acquisition unit 111, the selection acceptance unit 112, and the display control unit 113 may be realized by the processor 11 executing an information processing program, or may be configured by a dedicated hardware circuit such as an ASIC. The information processing program may be recorded on a non-transitory computer-readable recording medium.

[0052] The communication unit 13 is a communication interface that connects the server 10 to the network NT. The communication unit 13 receives shooting information transmitted by the communication device 40. The communication unit 13 receives instruction signals indicating various instructions received from the user from the information terminal 20. The communication unit 13 transmits display data for displaying various display screens to the information terminal 20.

[0053] The memory 12 is configured as a non-volatile rewritable storage device such as a hard disk drive or a solid state drive, etc. The memory 12 includes a design drawing information storage unit 121 and an image information storage unit 122.

[0054] The blueprint information storage unit 121 stores blueprint information. Blueprint information is image information showing a blueprint of a specified space. A blueprint ID that identifies the blueprint is associated with the blueprint information. The blueprint is an example of a bird's-eye view. The blueprint is a drawing showing the design of a construction site, and may be a two-dimensional floor plan, blueprint, map, or perspective view of the construction site. The bird's-eye view can also be called a bird's-eye view, and may be a view from above or a view from a high place.

[0055] The blueprint may also be a three-dimensional simulation image of a specified space or a two-dimensional side view of the specified space. The three-dimensional simulation image is a three-dimensional CAD image rendered from three-dimensional Computer Aided Design (CAD) information or a BIM image rendered from BIM (Building Information Modeling) information. For example, if the specified space is a construction site, an image rendered from the BIM information of a completed building based on a specified shooting direction is used as the three-dimensional simulation image. The two-dimensional plan view is a view of the construction site from above, and the two-dimensional side view is a view of the construction site from the side.

[0056] The acquisition unit 111 uses Visual SLAM (Simultaneous Localization and Mapping) technology to identify the location of each of the multiple images based on the shooting start point, shooting end point, and multiple images included in the shooting information received by the communication unit 13. The location of the shooting point is expressed as two-dimensional coordinate values ​​on the design drawing. The acquisition unit 111 also identifies the shooting direction of each of the multiple images using Visual SLAM technology. The shooting direction of each image is expressed, for example, as a three-dimensional polar coordinate vector.

[0057] The image information storage unit 122 stores image information. The image information is information that associates multiple captured images taken at multiple capture locations with meta information for each captured image. The image information is generated each time the above-mentioned capture information is received. The meta information includes a capture ID, capture date and time, capture direction, capture location, and blueprint ID. The capture direction is the capture direction of the capture device 30 that captured the image. The capture location is location information (two-dimensional coordinate values) that indicates the capture location of the image.

[0058] Furthermore, the communication unit 13 receives an instruction to specify the shooting date and time from the information terminal 20. The operation unit 24 of the information terminal 20 accepts the selection of the shooting date and time by the user, and the communication unit 21 transmits an instruction to specify the shooting date and time selected by the user to the server 10. The instruction to specify the shooting date and time includes the shooting ID.

[0059] The acquisition unit 111 acquires, from the design drawing information storage unit 121, a design drawing corresponding to a design drawing ID included in the instruction for displaying the design drawing received by the communication unit 13. The acquisition unit 111 also acquires, from the image information storage unit 122, a plurality of shooting locations corresponding to a shooting ID included in the instruction for specifying the shooting date and time received by the communication unit 13.

[0060] The display control unit 113 displays a blueprint corresponding to the predetermined space on the display 23 of the information terminal 20. The display control unit 113 displays the blueprint acquired by the acquisition unit 111 on the display 23 of the information terminal 20. The display control unit 113 transmits display data of the blueprint to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data transmitted by the server 10. The display 23 displays the blueprint received by the communication unit 21.

[0061] The display control unit 113 displays, on the design drawing, a plurality of photography location icons indicating a plurality of photography locations acquired by the acquisition unit 111. The plurality of photography location icons are associated with a plurality of photographed images taken at a plurality of photography locations.

[0062] 2A is a diagram showing an example of a design drawing screen 201 on which a plurality of photography location icons 221 are mapped in this embodiment. The design drawing screen 201 displays a design drawing 211 of a predetermined space. The plurality of photography location icons 221 are superimposed and displayed on the design drawing screen 201.

[0063] The photography location icon 221 is an icon indicating a photography location. The photography location icon 221 is associated with an image photographed at the photography location. In this example, the photography location icon 221 is configured as a circular image of a predetermined size. The multiple photography location icons 221 displayed here correspond to multiple photography locations belonging to one photography operation. The route tracing from the leading photography location icon 221 (the photography location icon at the bottom right of FIG. 2A ) to the terminal photography location icon 221 (the photography location icon at the top left of FIG. 2A ) indicates the trajectory followed by the photographer in this photography operation.

[0064] The image capturing device 30 captures images at a predetermined frame rate. In this case, the number of images is expressed as the frame rate (fps) * the capture time (seconds). In Fig. 2A, multiple image capturing location icons 221 are displayed overlapping each other along the path of the photographer's movement.

[0065] The operation unit 24 accepts the user's selection of one of the plurality of photography location icons 221. The user places the mouse pointer on one of the plurality of photography location icons 221 and clicks the left mouse button. Alternatively, the user touches one of the plurality of photography location icons 221. In the example of FIG. 2A , one photography location icon 222 is selected from the plurality of photography location icons 221.

[0066] When one of the multiple shooting location icons 221 is selected, the display control unit 113 reads out the photographed image corresponding to the selected shooting location icon from the image information storage unit 122 and displays the read photographed image on the display 23.

[0067] In FIG. 2A, multiple photography location icons 221 are displayed in an overlapping manner, but a single photography location icon may be displayed that displays only the points required for acquiring preselected data.

[0068] 2B is a diagram showing an example of a blueprint screen 201B on which one photography location icon 221B is mapped in this embodiment. The blueprint screen 201B displays a blueprint 211 of a predetermined space. One photography location icon 221B is superimposed and displayed on the blueprint screen 201B. One photography location icon 221B displayed here corresponds to one photography location.

[0069] The operation unit 24 accepts the user's selection of one of the photography location icons 221B. The user places the mouse pointer on one of the photography location icons 221B and clicks the left mouse button. Alternatively, the user touches one of the photography location icons 221B.

[0070] When one shooting location icon 221B is selected, the display control unit 113 reads out the photographed image corresponding to the selected shooting location icon 221B from the image information storage unit 122 and displays the read photographed image on the display 23.

[0071] FIG. 3 is a diagram showing an example of the photographed image screen 202 displayed on the display 23 when the photographing location icon 222 is selected in this embodiment.

[0072] The captured image screen 202 displays a captured image 212 captured at a capture location corresponding to a capture location icon 222. The display control unit 113 displays one of a plurality of captured images captured consecutively. Here, the display control unit 113 displays a rectangular captured image corresponding to the line of sight cut out from the omnidirectional image on the display 23. The captured image screen 202 includes a date field 231, a comparison display button 232, a design drawing display field 233, and a walk-through instruction button 234.

[0073] The date field 231 displays the date and time of the photographed image 212. The comparison display button 232 is a button for comparatively displaying the photographed image 212 and the design drawing 211. The design drawing display field 233 displays a reduced version of the design drawing screen 201 shown in Fig. 2A.

[0074] The display control unit 113 displays a walk-through instruction button 234 for switching from the displayed first captured image to a captured image captured before the first captured image or a captured image captured after the first captured image. The walk-through instruction button 234 is an example of a switching icon.

[0075] The walk-through instruction button 234 is a button for switching the captured image 212 to the captured image with the next later shooting date and time. Note that the user can also switch the captured image 212 to the captured image with the previous shooting date and time. In this case, the display control unit 113 displays the walk-through instruction button 234 and a walk-through instruction button 234 facing in the opposite direction on the display 23 in response to a user instruction, and when an instruction to select the backward walk-through instruction button 234 is input, the display control unit 113 displays the previous captured image.

[0076] To display the photographed image 212 and the design drawing 211 side by side on the display 23, the user presses the comparison display button 232. When the comparison display button 232 is pressed, the display 23 displays the comparison display screen 203 shown in FIG.

[0077] 4 is a diagram showing a comparison display screen 203 that displays the photographed image and the design drawing in this embodiment. The comparison display screen 203 is displayed on the display 23 when a selection instruction of the comparison display button 232 is accepted.

[0078] The display control unit 113 displays a captured image 212 of a predetermined space and a design drawing (bird's-eye view) 211 corresponding to the predetermined space on the display 23 of the information terminal 20. That is, the operation unit 24 of the information terminal 20 accepts a comparison display instruction for displaying the captured image and the design drawing side by side, and the communication unit 21 transmits the comparison display instruction to the server 10. The communication unit 13 receives the comparison display instruction for displaying the captured image and the design drawing side by side from the information terminal 20. When the communication unit 13 receives the comparison display instruction for displaying the captured image and the design drawing side by side, the display control unit 113 displays the captured image 212 and the design drawing 211 side by side.

[0079] The comparison display screen 203 includes a first area 241 and a second area 242 that are divided into two areas, left and right. The first area 241 displays the captured image 212, and the second area 242 displays the design drawing 211.

[0080] The comparison display screen 203 includes a date field 231 , a walk-through instruction button 234 , and an end comparison button 235 .

[0081] To end the display of the photographed image 212 and the design drawing 211, the user presses the comparison end button 235. When the comparison end button 235 is pressed, the display 23 ends the display of the comparison display screen 203 shown in FIG.

[0082] When the comparison end button 235 is pressed, the display control unit 113 returns the screen display on the display 23 from the comparison display screen 203 to the captured image screen 202. That is, the operation unit 24 of the information terminal 20 accepts a comparison end instruction to end the display of the captured image and the design drawing, and the communication unit 21 transmits the comparison end instruction to the server 10. The communication unit 13 receives the comparison end instruction to end the display of the captured image and the design drawing from the information terminal 20. When the communication unit 13 receives the comparison end instruction to end the display of the captured image and the design drawing, the display control unit 113 displays only the captured image 212.

[0083] The captured image is an omnidirectional image. Therefore, the display control unit 113 may accept an operation to change the line of sight of the captured image 212. When an operation to change the line of sight of the captured image 212 is input by the user using the operation unit 24, the display control unit 113 changes the line of sight of the captured image 212 according to the amount and direction of the operation. For example, the user places the mouse pointer on the captured image 212 and moves the mouse while clicking the left button of the mouse (drag operation). Alternatively, the user touches the captured image 212 with their finger and moves their finger (swipe operation). This changes the line of sight of the captured image 212.

[0084] Furthermore, when the line of sight of the photographed image 212 is changed, the display control unit 113 rotates the design drawing 211 so that the line of sight on the design drawing 211 coincides with the upper part of the design drawing 211. This causes the line of sight of the photographed image 212 to coincide with the upper part of the design drawing 211, allowing the user to easily recognize the line of sight on the design drawing 211.

[0085] In this embodiment, the display control unit 113 may display an arrow icon on the blueprint 211 indicating the viewing direction of the captured image 212. When the viewing direction of the captured image 212 is changed, the display control unit 113 may rotate the arrow icon so that the viewing direction on the blueprint 211 matches the direction indicated by the arrow icon. In this case, the display control unit 113 rotates the arrow icon without rotating the blueprint 211. This causes the viewing direction of the captured image 212 to match the direction indicated by the arrow icon displayed on the blueprint 211, allowing the user to easily recognize the viewing direction on the blueprint 211.

[0086] Furthermore, the display control unit 113 may accept an operation to enlarge or reduce the captured image 212. When an operation to enlarge or reduce the captured image 212 is input by the user using the operation unit 24, the display control unit 113 enlarges or reduces the captured image 212 according to the amount of operation. For example, the user presses a zoom button or a zoom button displayed on the captured image 212. Alternatively, the user touches the captured image 212 with two fingers and then spreads the fingers apart (pinch-out operation) or brings them closer together (pinch-in operation). This causes the captured image 212 to be enlarged or reduced.

[0087] The selection receiving unit 112 receives a user's selection of any two points on the design drawing or the photographed image.

[0088] FIG. 5 is a diagram showing a comparison display screen 203 that displays the distance between two points within a photographed image in this embodiment.

[0089] In this embodiment, the selection receiving unit 112 receives a user's selection of any two points on the design drawing 211. The user uses the operation unit 24 to specify two points on the design drawing 211 that define an interval between which the user wants to know the distance. The user moves the mouse pointer to the start point and end point of the interval between which the distance is to be measured on the design drawing 211, and clicks the left mouse button. Alternatively, the user may move the mouse pointer to the start point of the interval between which the distance is to be measured on the design drawing 211 and click the left mouse button, and while still clicking the left mouse button, move the mouse pointer to the end point and release the left mouse button.

[0090] The display control unit 113 displays an arrow icon 301 connecting the two points accepted by the selection accepting unit 112 on the design drawing 211 .

[0091] The display control unit 113 displays, in the photographed image 212, the distance between two points in the photographed image 212 that correspond to two points selected on the design drawing 211. The display control unit 113 calculates the distance between the two points selected on the design drawing 211 and displays the calculated distance in the photographed image 212 as the distance between the two points in the design drawing 211. The design drawing 211 includes grid lines 251 that indicate the actual dimensions of a specified space. Therefore, the display control unit 113 can accurately calculate the distance between the two points selected on the design drawing 211.

[0092] The design drawing 211 is composed of a two-dimensional plan view of a specified space. Therefore, the selection receiving unit 112 receives the selection of any two points in the horizontal direction on the two-dimensional plan view. The display control unit 113 calculates the distance between the two horizontal points selected on the design drawing 211 and displays the calculated distance in the captured image 212 as the distance between the two horizontal points in the captured image 212. Here, the horizontal direction includes both the X and Y directions. Furthermore, when selecting two points, the selection can be canceled by reselecting, etc.

[0093] Furthermore, the display control unit 113 extracts feature points of the photographed image 212 and recognizes structures such as pillars, walls, and doors in the photographed image 212. Then, based on the position of the structure in the photographed image 212 and the position of the structure on the design drawing 211, the display control unit 113 determines the positions of the start point and the end point on the photographed image 212 that correspond to the positions of the start point and the end point on the design drawing 211.

[0094] The display control unit 113 displays the distance 311 between the two points near the arrow icon 301. The selection receiving unit 112 may also receive a selection of two points for each of a plurality of intervals, and the display control unit 113 may calculate the distance between the two points for each of the plurality of intervals.

[0095] 5 , multiple arrow icons 301, 302, and 303 corresponding to multiple intervals are displayed in the design drawing 211, and multiple distances 311, 312, and 313 are displayed near the multiple arrow icons 301, 302, and 303. The multiple arrow icons 301, 302, and 303 may be displayed in different ways or in different colors. Furthermore, the multiple distances 311, 312, and 313 may be displayed in different ways or in different colors. For example, the arrow icon 301 and the distance 311 may be displayed in red, the arrow icon 302 and the distance 312 may be displayed in blue, and the arrow icon 303 and the distance 313 may be displayed in green.

[0096] Although the display control unit 113 displays the distance 311 between the two points on the design drawing 211, the present disclosure is not particularly limited to this. The display control unit 113 may display only the arrow icon 301 connecting the two points on the design drawing 211 without displaying the distance 311 between the two points on the design drawing 211.

[0097] Furthermore, the display control unit 113 displays, on the photographed image 212, an arrow icon 321 connecting two points on the photographed image 212 that correspond to the two points on the design drawing 211 accepted by the selection accepting unit 112, and also displays a distance 331 between the two points near the arrow icon 321. Furthermore, when the selection accepting unit 112 accepts the selection of two points for each of a plurality of intervals, the display control unit 113 may display, in the photographed image 212, the distance for each of the plurality of intervals in the photographed image 212 that corresponds to the plurality of intervals selected on the design drawing 211.

[0098] 5 , multiple arrow icons 321, 322, and 323 corresponding to multiple intervals are displayed in the captured image 212, and multiple distances 331, 332, and 333 are displayed near the multiple arrow icons 321, 322, and 323. The multiple arrow icons 321, 322, and 323 may be displayed in different ways or in different colors. Furthermore, the multiple distances 331, 332, and 333 may be displayed in different ways or in different colors. For example, the arrow icon 321 and the distance 331 may be displayed in red, the arrow icon 322 and the distance 332 may be displayed in blue, and the arrow icon 323 and the distance 333 may be displayed in green.

[0099] In addition, when the walk-through instruction button 234 (switch icon) is selected, the display control unit 113 may switch to an image taken before the first image or an image taken after the first image, and display the distance within the switched image.

[0100] Furthermore, although the arrow icons 301, 302, 303, 321, 322, and 323 have a shape with arrows on both ends, the present disclosure is not particularly limited to this, and the arrow icons 301, 302, 303, 321, 322, and 323 may have a linear shape without arrows on both ends.

[0101] Next, the distance display process of the server 10 in this embodiment will be described.

[0102] FIG. 6 is a flowchart showing an example of the distance display process of the server 10 in this embodiment.

[0103] First, in step S1 , the display control unit 113 displays a design drawing corresponding to a predetermined space on the display 23 of the information terminal 20 .

[0104] The communication unit 13 receives an instruction to display a blueprint from the information terminal 20. In this case, a menu screen for selecting a blueprint is displayed on the display 23 of the information terminal 20, and the operation unit 24 accepts a user instruction to select one blueprint from the menu screen. The input instruction is transmitted to the server 10 via the network NT and received by the communication unit 13. The instruction to display the blueprint includes a blueprint ID. The acquisition unit 111 acquires, from the blueprint information stored in the blueprint information storage unit 121, a blueprint corresponding to the blueprint ID included in the instruction to display the blueprint received by the communication unit 13. The display control unit 113 displays the blueprint on the display 23 of the information terminal 20 by transmitting display data for the blueprint to the information terminal 20 via the communication unit 13.

[0105] The design drawing also includes a plurality of photography location icons. The operation unit 24 may receive a user instruction to select a desired photography date and time. The acquisition unit 111 may acquire a plurality of photography locations corresponding to the selected photography date and time from the image information stored in the image information storage unit 122. The display control unit 113 may display a plurality of photography location icons indicating the acquired plurality of photography locations on the design drawing.

[0106] Next, in step S2, the display control unit 113 determines a photography location from among multiple photography locations based on the instruction received by the communication unit 13. As shown in FIG. 2A , when the information terminal 20 receives an instruction to select a photography location icon, it transmits the selection instruction to the server 10. This selection instruction includes the photography ID and the photography date and time. When the communication unit 13 receives this selection instruction, the display control unit 113 detects that the photography location icon has been selected. The display control unit 113 determines the photography location using the photography ID and the photography date and time included in the selection instruction received by the communication unit 13 as keys.

[0107] Next, in step S3, the display control unit 113 displays the photographed image photographed at the determined photographing location. Specifically, the display control unit 113 transmits display data of the photographed image to the information terminal 20 using the communication unit 13. In the information terminal 20, the processor 22 displays the display data received by the communication unit 21 on the display 23. As a result, the photographed image screen 202 shown in FIG. 3 is displayed on the display 23.

[0108] Next, in step S4, the communication unit 13 receives a comparison display instruction for displaying the photographed image and the design drawing side by side from the information terminal 20. The comparison display instruction is input by pressing the comparison display button 232 displayed on the photographed image screen 202. The processor 22 transmits the comparison display instruction to the server 10 using the communication unit 21.

[0109] Next, in step S5, the display control unit 113 displays the photographed image and the design drawing side by side. Specifically, the display control unit 113 transmits display data of the photographed image and the design drawing to the information terminal 20 using the communication unit 13. In the information terminal 20, the processor 22 displays the display data received by the communication unit 21 on the display 23. As a result, the comparison display screen 203 shown in FIG. 4 is displayed on the display 23.

[0110] Next, in step S6, the display control unit 113 accepts an operation on the captured image. Here, the display control unit 113 accepts an operation to change the line of sight of the captured image or an operation to enlarge or reduce the captured image. When an operation to change the line of sight of the captured image is performed, the display control unit 113 displays a rectangular image of the omnidirectional image corresponding to the changed line of sight as the captured image on the display 23. Furthermore, when an operation to enlarge or reduce the captured image is performed, the display control unit 113 enlarges or reduces the currently displayed captured image.

[0111] Next, in step S7, the display control unit 113 rotates the blueprint in response to a change in the line of sight of the captured image. When the line of sight of the captured image is changed, the display control unit 113 rotates the blueprint so that the line of sight on the blueprint coincides with the top of the blueprint.

[0112] Next, in step S8, the selection receiving unit 112 receives the user's selection of a start point and an end point on the design drawing. The operation unit 24 of the information terminal 20 receives the user's input of the start point and end point of the interval for which the user wants to measure the distance on the design drawing. The processor 22 transmits position information indicating the coordinates of the input start point and end point on the design drawing to the server 10 using the communication unit 21. The position information transmitted from the information terminal 20 is received by the communication unit 13 and input to the display control unit 113. As a result, the display control unit 113 receives the selection of the start point and end point on the design drawing.

[0113] Next, in step S9, the display control unit 113 calculates the distance between the start point and the end point on the design drawing.

[0114] Next, in step S10, the display control unit 113 displays an arrow icon connecting the start point and the end point on the design drawing, and also displays the distance between the start point and the end point near the arrow icon.

[0115] Next, in step S11, the display control unit 113 determines the positions of the start point and the end point in the captured image that correspond to the positions of the start point and the end point on the blueprint. For example, the display control unit 113 extracts feature points in the captured image and recognizes pillars and doors in the captured image. The display control unit 113 also acquires the shooting location and line of sight of the captured image in the blueprint and recognizes pillars and doors that exist in the line of sight from the shooting location in the blueprint. Then, the display control unit 113 determines the positions of the pillars and doors in the captured image that correspond to the positions of the start point and the end point in the blueprint using the recognized positions of the pillars and doors in the captured image, the recognized positions of the pillars and doors in the blueprint, and the positions of the start point and the end point in the blueprint.

[0116] Next, in step S12, the display control unit 113 displays an arrow icon connecting the start point and the end point on the captured image, and also displays the distance between the start point and the end point near the arrow icon, thereby causing the comparison display screen 203 shown in FIG.

[0117] Next, in step S13, the communication unit 13 determines whether or not a comparison end instruction to end the display of the photographed image and the design drawing has been received from the information terminal 20. The comparison end instruction is input by pressing the comparison end button 235 displayed on the comparison display screen 203. The processor 22 transmits the comparison end instruction to the server 10 using the communication unit 21.

[0118] If it is determined that the comparison end instruction has been received (YES in step S13), the process returns to step S3, and the display control unit 113 displays only the captured image. Note that if it is determined that the comparison end instruction has been received, the process returns to step S1, and the display control unit 113 may display only the design drawing.

[0119] On the other hand, if it is determined that the comparison end instruction has not been received (NO in step S13), the process returns to step S6, and the display control unit 113 accepts an operation on the captured image.

[0120] In this way, the distance between two points in the captured image corresponding to two points selected on the blueprint is displayed in the captured image. Therefore, the distance between two desired points in the captured image can be presented to the user with high accuracy. Furthermore, the distance can be presented with high accuracy in both the X and Y directions, including the depth (Y direction). Furthermore, because the distance between two desired points in the captured image is presented to the user, the computer processing steps required to confirm the distance between the two points can be reduced, thereby reducing the burden on the computer. Furthermore, because the distance between two points on the blueprint is calculated, an accurate distance can be calculated. Furthermore, because the calculated distance is displayed in the captured image as the distance between two points in the captured image, the accurate distance between the desired two points in the captured image can be presented to the user. The distance may be displayed on both the blueprint and the captured image as described above, or on either the blueprint or the captured image, or another display method may be used.

[0121] In the present embodiment, the display control unit 113 displays, on the blueprint, multiple photography location icons indicating multiple photography locations where images were taken consecutively, but the present disclosure is not particularly limited to this. The display control unit 113 may also display, on the blueprint, one photography location icon indicating one photography location where images were taken individually. Furthermore, the display control unit 113 may also display, on the blueprint, multiple photography location icons indicating multiple photography locations where images were taken individually.

[0122] In the present embodiment, the design drawing may be configured as a three-dimensional simulation image of a predetermined space or a two-dimensional side view of the predetermined space. In this case, the selection receiving unit 112 may receive the selection of any two vertical points in the three-dimensional simulation image or the two-dimensional side view. The display control unit 113 may calculate the distance between the two vertical points selected on the design drawing and display the calculated distance in the captured image as the distance between the two vertical points in the captured image.

[0123] (Variation 1) In the above-described embodiment, the selection of any two points on the design drawing (bird's-eye view) is accepted, and the distance between the two points in the captured image that correspond to the two points selected on the design drawing (bird's-eye view) is displayed in the captured image. In contrast, in Variation 1 of the present embodiment, the selection of any two points on the captured image is accepted, and the distance between the two points in the design drawing (bird's-eye view) that correspond to the two points selected in the captured image is displayed in the design drawing (bird's-eye view).

[0124] In the first modification of this embodiment, the same components as those in the above embodiment are denoted by the same reference numerals and their description will be omitted. Also, in the first modification of this embodiment, the block diagram of FIG. 1 is used.

[0125] In a first modification of the present embodiment, the selection receiving unit 112 receives a user's selection of any two points on the captured image 212. The user uses the operation unit 24 to specify two points on the captured image 212 that define an interval between which the user wants to know the distance. The user moves the mouse pointer to the start position and end position of the interval between which the distance is to be measured on the captured image 212, and clicks the left mouse button. Alternatively, the user may move the mouse pointer to the start position of the interval between which the distance is to be measured on the captured image 212 and click the left mouse button, and then, while still clicking the left mouse button, move the mouse pointer to the end position and release the left mouse button.

[0126] FIG. 7 is a diagram showing a comparison display screen 203A that displays the distance between two points within a photographed image in the first modification of this embodiment.

[0127] The display control unit 113 displays, in the design drawing 211 (bird's-eye view), the distance between two points in the design drawing 211 (bird's-eye view) that correspond to two points selected in the photographed image 212. The display control unit 113 calculates the distance between two points in the design drawing 211 (bird's-eye view) that correspond to the two points selected in the photographed image 212, and displays the calculated distance in the design drawing 211 (bird's-eye view).

[0128] 7, multiple arrow icons 321, 322, and 323 corresponding to multiple intervals are displayed within the captured image 212. The multiple arrow icons 321, 322, and 323 may be displayed in different ways or in different colors. For example, the arrow icon 321 may be displayed in red, the arrow icon 322 may be displayed in blue, and the arrow icon 323 may be displayed in green.

[0129] 7 , multiple arrow icons 301, 302, and 303 corresponding to multiple intervals are displayed in the design drawing 211, and multiple distances 311, 312, and 313 are displayed near the multiple arrow icons 301, 302, and 303. The multiple arrow icons 301, 302, and 303 may be displayed in different ways or in different colors. Furthermore, the multiple distances 311, 312, and 313 may be displayed in different ways or in different colors. For example, the arrow icon 301 and the distance 311 may be displayed in red, the arrow icon 302 and the distance 312 may be displayed in blue, and the arrow icon 303 and the distance 313 may be displayed in green.

[0130] In the first variant of this embodiment, the display control unit 113 displays arrow icons 321, 322, and 323 connecting the start point and end point on the captured image 212, but does not display the distance near the arrow icons 321, 322, and 323 on the captured image 212.

[0131] Next, the distance display process of the server 10 in the first modification of the present embodiment will be described.

[0132] FIG. 8 is a flowchart showing an example of the distance display process of the server 10 in the first modification of the present embodiment.

[0133] The processes in steps S21 to S27 are the same as those in steps S1 to S7 shown in FIG. 6, and therefore will not be described here.

[0134] Next, in step S28, the selection receiving unit 112 receives the user's selection of a start point and an end point in the captured image. The operation unit 24 of the information terminal 20 receives the user's input of the start point and end point of the interval for which the user wants to measure the distance in the captured image. The processor 22 transmits position information indicating the coordinates of the input start point and end point on the captured image to the server 10 using the communication unit 21. The position information transmitted from the information terminal 20 is received by the communication unit 13 and input to the display control unit 113. As a result, the display control unit 113 receives the selection of the start point and end point in the captured image.

[0135] Next, in step S29, the display control unit 113 displays an arrow icon connecting the start point and the end point on the captured image.

[0136] Next, in step S30, the display control unit 113 determines the positions of the start point and the end point on the blueprint that correspond to the positions of the start point and the end point on the captured image. For example, the display control unit 113 extracts feature points in the captured image and recognizes pillars and doors in the captured image. The display control unit 113 also acquires the shooting point and line of sight direction of the captured image in the blueprint and recognizes pillars and doors that exist in the line of sight direction from the shooting point in the blueprint. Then, the display control unit 113 determines the positions of the start point and the end point on the blueprint that correspond to the positions of the start point and the end point in the captured image using the recognized positions of the pillars and doors in the captured image, the recognized positions of the pillars and doors in the blueprint, and the positions of the start point and the end point in the captured image.

[0137] Next, in step S31, the display control unit 113 calculates the distance between the start point and the end point on the design drawing.

[0138] Next, in step S32, the display control unit 113 displays an arrow icon connecting the start point and the end point on the design drawing, and also displays the distance between the start point and the end point near the arrow icon, thereby displaying the comparison display screen 203A shown in FIG.

[0139] The process of step S33 is the same as the process of step S13 shown in FIG. 6, and therefore a description thereof will be omitted.

[0140] In this way, the distance between two points in the blueprint corresponding to two points selected in the captured image is displayed in the bird's-eye view. Therefore, the distance between two desired points in the captured image can be presented to the user with high accuracy. Furthermore, the distance can be presented with high accuracy in both the X and Y directions, including the depth (Y direction). Furthermore, since the distance between two points on the blueprint corresponding to two selected points in the captured image is calculated, an accurate distance can be calculated. Furthermore, since the calculated distance is displayed in the blueprint, the accurate distance between two desired points in the captured image can be presented to the user. The distance may be displayed on both the blueprint and the captured image, or on either the blueprint or the captured image, or another display method may be used.

[0141] (Variation 2) In the above-described embodiment, the selection of any two points on the design drawing (bird's-eye view) is accepted, and the distance between the two points in the captured image that correspond to the two points selected on the design drawing (bird's-eye view) is displayed in the captured image. In contrast, in Variation 2 of the present embodiment, the selection of any two points on the captured image is accepted, and the distance between the two selected points in the captured image is displayed in the captured image.

[0142] In the second modification of this embodiment, the same components as those in the above embodiment are denoted by the same reference numerals and their description will be omitted. Also, in the second modification of this embodiment, the block diagram of FIG. 1 is used.

[0143] In a second modification of the present embodiment, the selection receiving unit 112 receives a user's selection of any two points on the captured image 212. The user uses the operation unit 24 to specify two points on the captured image 212 that define an interval between which the user wants to know the distance. The user moves the mouse pointer to the start and end positions of the interval between which the distance is to be measured on the captured image 212, and clicks the left mouse button. Alternatively, the user may move the mouse pointer to the start position of the interval between which the distance is to be measured on the captured image 212 and click the left mouse button, and then, while still clicking the left mouse button, move the mouse pointer to the end position and release the left mouse button.

[0144] FIG. 9 is a diagram showing a comparison display screen 203B that displays the distance between two points within a photographed image in the second modification of this embodiment.

[0145] The display control unit 113 displays the distance between two points selected in the photographed image 212 within the photographed image 212. The display control unit 113 calculates the distance between two points in the design drawing 211 (bird's-eye view) that correspond to the two points selected in the photographed image 212, and displays the calculated distance within the photographed image 212.

[0146] 9 , multiple arrow icons 321, 322, and 323 corresponding to multiple intervals are displayed in the captured image 212, and multiple distances 331, 332, and 333 are displayed near the multiple arrow icons 321, 322, and 323. The multiple arrow icons 321, 322, and 323 may be displayed in different ways or in different colors. Furthermore, the multiple distances 331, 332, and 333 may be displayed in different ways or in different colors. For example, the arrow icon 321 and the distance 331 may be displayed in red, the arrow icon 322 and the distance 332 may be displayed in blue, and the arrow icon 323 and the distance 333 may be displayed in green.

[0147] 9, multiple arrow icons 301, 302, and 303 corresponding to multiple intervals are displayed in the design drawing 211. The multiple arrow icons 301, 302, and 303 may be displayed in different ways or in different colors. For example, the arrow icon 301 may be displayed in red, the arrow icon 302 may be displayed in blue, and the arrow icon 303 may be displayed in green.

[0148] In the second variant of this embodiment, the display control unit 113 displays arrow icons 301, 302, and 303 connecting the start point and end point on the design drawing 211, but does not display the distance near the arrow icons 301, 302, and 303 on the design drawing 211.

[0149] Next, the distance display process of the server 10 in the second modification of the present embodiment will be described.

[0150] FIG. 10 is a flowchart showing an example of the distance display process of the server 10 in the second modification of the present embodiment.

[0151] The processes in steps S41 to S47 are the same as those in steps S1 to S7 shown in Fig. 6, and therefore will not be described. The processes in steps S48 to S51 are the same as those in steps S28 to S31 shown in Fig. 8, and therefore will not be described.

[0152] After the process of step S50, the display control unit 113 may display an arrow icon connecting the start point and the end point on the design drawing.

[0153] Next, in step S52, the display control unit 113 displays the distance between the start point and the end point near the arrow icon displayed on the captured image, thereby displaying the comparison display screen 203B shown in FIG.

[0154] The process of step S53 is the same as the process of step S13 shown in FIG. 6, and therefore a description thereof will be omitted.

[0155] In this way, the distance between two points selected in the captured image is displayed in the captured image. Therefore, the distance between two desired points in the captured image can be presented to the user with high accuracy. Furthermore, the distance can be presented with high accuracy in both the X and Y directions, including the depth (Y direction). Furthermore, the distance between two points on the design drawing corresponding to the two selected points on the captured image is calculated, so that an accurate distance can be calculated. Furthermore, the calculated distance is displayed in the captured image, so that the accurate distance between two desired points in the captured image can be presented to the user. The distance may be displayed on both the design drawing and the captured image, or on either the design drawing or the captured image, or another display method may be used.

[0156] Note that some or all of the functions of the device according to the embodiment of the present disclosure may be realized by a processor such as a CPU executing a program.

[0157] Furthermore, all the numbers used above are merely examples to specifically explain the present disclosure, and the present disclosure is not limited to the numbers used as examples.

[0158] The order in which the steps are executed in the above flowchart is merely an example for specifically explaining the present disclosure, and other orders may be used as long as similar effects are obtained. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0159] The technology disclosed herein can present the user with a high degree of accuracy the distance between two desired points in a captured image, and is therefore useful as a technology for recognizing distance or size on-site during or after capture.

Claims

1. An information processing method executed by a computer, comprising: displaying, on an information terminal display, a captured image of a specified space and a bird's-eye view corresponding to the specified space; accepting the selection of any two points on the bird's-eye view or the captured image; and displaying, in the captured image, the distance between two points in the captured image that correspond to the two points selected on the bird's-eye view, displaying, in the bird's-eye view, the distance between two points in the bird's-eye view that correspond to the two points selected in the captured image, or displaying, in the captured image, the distance between the two points selected in the captured image.

2. An information processing method as described in claim 1, wherein the accepting includes accepting a selection of any two points on the bird's-eye view, and the displaying of the distance includes calculating the distance between the two points selected on the bird's-eye view and displaying the calculated distance in the captured image as the distance between the two points in the captured image.

3. An information processing method as described in claim 1, wherein the acceptance includes accepting a selection of any two points in the captured image, and the display of the distance includes calculating a distance between two points in the bird's-eye view that correspond to the two points selected in the captured image, and displaying the calculated distance in the bird's-eye view.

4. An information processing method as described in claim 1, wherein the acceptance includes accepting a selection of any two points in the captured image, and the display of the distance includes calculating a distance between two points in the bird's-eye view that correspond to the two points selected in the captured image, and displaying the calculated distance in the captured image.

5. The information processing method according to any one of claims 1 to 4, wherein the captured image is composed of an omnidirectional image.

6. An information processing method as described in claim 5, wherein displaying the captured image and the bird's-eye view includes, when the viewing direction of the captured image is changed, rotating the bird's-eye view so that the viewing direction in the bird's-eye view coincides with the top of the bird's-eye view.

7. An information processing method according to any one of claims 1 to 4, wherein displaying the captured image and the bird's-eye view includes displaying one of a plurality of captured images taken in succession, and further includes displaying a switch icon for switching from the one captured image displayed to an image taken before the one captured image or an image taken after the one captured image, and displaying the distance includes, when the switch icon is selected, switching to an image taken before the one captured image or an image taken after the one captured image, and displaying the distance within the switched image.

8. An information processing method according to any one of claims 1 to 4, wherein the bird's-eye view is configured as a two-dimensional plan view of the specified space, and the receiving step includes receiving a selection of any two points in a horizontal direction on the two-dimensional plan view.

9. An information processing method according to any one of claims 1 to 4, wherein the bird's-eye view is composed of a three-dimensional simulation image of the specified space or a two-dimensional side view of the specified space, and the receiving includes receiving a selection of any two points in a perpendicular direction on the three-dimensional simulation image or the two-dimensional side view.

10. An information processing device having a processor, wherein the processor displays an image of a specified space and a bird's-eye view corresponding to the specified space on a display of an information terminal, accepts the selection of any two points on the bird's-eye view or the captured image, and displays in the captured image the distance between two points in the captured image that correspond to the two points selected on the bird's-eye view, displays in the captured image the distance between two points in the bird's-eye view that correspond to the two points selected in the captured image, or displays in the bird's-eye view the distance between the two points selected in the captured image.

11. An information processing program that causes a computer to function as follows: displaying, on an information terminal display, an image of a specified space and a bird's-eye view corresponding to the specified space; accepting the selection of any two points on the bird's-eye view or the photographed image; and displaying, in the photographed image, the distance between two points in the photographed image that correspond to the two points selected on the bird's-eye view; displaying, in the bird's-eye view, the distance between two points in the bird's-eye view that correspond to the two points selected in the photographed image; or displaying, in the photographed image, the distance between the two points selected in the photographed image.

Images