Information processing method, information processing device, and information processing program

Abstract

This information processing method for a computer includes: displaying an overhead view on a display; displaying, on the overhead view, a plurality of imaging location icons indicating imaging locations; detecting that a specific icon among the plurality of imaging location icons has been selected; displaying a first image associated with the specific icon on the display; detecting that a prescribed object included in the first image has been selected; and, in response to the selection of the prescribed object, displaying, on the display, a second image that was captured from an imaging location different from the imaging location of the first image and that includes the prescribed object, and that was captured at a time different from the imaging time of the first image.

Description

Information processing method, information processing device, and information processing program

[0001] The present disclosure relates to techniques for displaying images.

[0002] Patent Literature 1 discloses a technique for creating image information on which guidance information indicating street names is superimposed on the route the user is walking when the user feels stable, and for displaying this image information on a liquid crystal display. Patent Literature 1 also discloses a technique for creating image information on which guidance information indicating the names of buildings around the route the user is walking when the user feels relaxed, and for outputting this image information on a liquid crystal display.

[0003] Patent Document 1 merely discloses a technology for presenting a user with image information on which guidance information indicating the names of streets and buildings is superimposed. Therefore, the technology described in Patent Document 1 makes it difficult for a user to check an image of a specific object included in an image taken from a different shooting location. In this case, the user may need to input an operation into a computer to search for an image of the specific object included in the image taken from a different shooting location. This results in an increase in the number of processing steps and a burden on the computer.

[0004] Japanese Patent Application Laid-Open No. 2017-167501

[0005] The present disclosure has been made in light of the above-mentioned problems, and aims to provide a technology that makes it easy to check an image of a specified object included in an image taken from a different shooting point.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method on a computer, comprising: displaying a bird's-eye view on a display; displaying a plurality of shooting location icons indicating shooting locations on the bird's-eye view; detecting that a specific icon from the plurality of shooting location icons has been selected; displaying a first image associated with the specific icon on the display; detecting that a predetermined object included in the first image has been selected; and, in response to the selection of the predetermined object, displaying on the display a second image including the predetermined object that has been photographed from a shooting location other than the shooting location of the first image and that has been photographed at a time different from the shooting time of the first image.

[0007] According to the present disclosure, a user can easily check a second image of a specified object included in a first image taken from a different shooting location, thereby reducing the burden on the computer.

[0008] FIG. 1 is a diagram showing the overall configuration of an information processing system in this embodiment. FIG. 2 is a diagram showing an example of a design drawing displayed on a display. FIG. 3 is a flowchart showing an example of a processing flow in the information processing system. FIG. 4 is a flowchart showing an example of a processing flow in the information processing system. FIG. 5 is a diagram showing an example of a second display area in a first image. FIG. 6 is a diagram showing a slum analysis diagram generated by a display control unit. FIG. 7 is a diagram showing a slum analysis diagram generated by a display control unit. FIG. 8 is a diagram showing an example of a second image. FIG. 9 is a diagram showing another example of the second image. FIG. 10 is a diagram showing a schematic view of a display screen displayed on a display in modified example (2). FIG. 11 is a diagram showing a first image related to modified example (3). FIG. 12 is a diagram showing a design drawing in which selectable areas are set.

[0009] (Findings that form the basis of the present disclosure) A user interface is being developed that displays multiple photography location icons indicating locations where actual photographs of a construction site were taken on a two-dimensional blueprint of the construction site, etc., and when one photography location icon is selected, displays an image taken at the photography location indicated by that icon. This allows a construction site manager to understand the situation at the construction site without visiting the construction site.

[0010] In the above user interface, a user may select a specific photography location icon, display an image (referred to as a first image) associated with the specific photography location icon on a display or the like, and check a specific object captured in the first image. Here, the user may wish to check a second image, which captures the specific object and was captured from a different photography location than the first image, in order to more closely understand the structure, shape, etc., of the specific object captured in the first image. In this case, the user may input an operation into the computer to search for a photography location icon associated with an appropriate image as the second image. This results in an increase in the number of processing steps performed by the computer, thereby increasing the load on the computer.

[0011] Such problems cannot be solved by the technology described in Patent Document 1. Patent Document 1 merely discloses a technology for superimposing the name of an object on an object appearing in an image, and does not consider the above problems at all.

[0012] In order to solve the above problems, the following techniques are disclosed.

[0013] (1) An information processing method according to one aspect of the present disclosure is an information processing method on a computer, comprising: displaying a bird's-eye view on a display; displaying a plurality of shooting location icons indicating shooting locations on the bird's-eye view; detecting that a specific icon from the plurality of shooting location icons has been selected; displaying a first image associated with the specific icon on the display; detecting that a predetermined object included in the first image has been selected; and, in response to the selection of the predetermined object, displaying on the display a second image including the predetermined object that has been photographed from a shooting location different from the shooting location of the first image and that has been photographed at a time different from the shooting time of the first image.

[0014] According to this configuration, in response to the selection of a predetermined object in the first image, a second image including the predetermined object photographed from a different photographing location from the photographing location of the first image is displayed on the display. In this way, the user can easily check the second image, and there is no need to input an operation into the computer to search for the second image. As a result, an increase in computer processing load is suppressed, and an increase in the burden on the computer is suppressed.

[0015] (2) The information processing method described in (1) above may further include selecting an image associated with a shooting location icon located within a predetermined distance from the specific icon as a second image candidate, and selecting the second image from the second image candidate.

[0016] According to this configuration, an image captured at a shooting location located within a predetermined distance from the shooting location of the first image is selected as the second image candidate. In other words, an image captured at a shooting location different from that of the first image and which is likely to include a predetermined object is selected as the second image candidate. Then, the second image is selected from these second image candidates. In this way, an image appropriate as the second image can be displayed on the display.

[0017] (3) In the information processing method described in (2) above, the method may further include detecting that a viewpoint switching operation has been input for the first image and acquiring the amount of operation of the viewpoint switching operation, wherein the second image candidate includes at least two or more images, and displaying the second image on the display may include selecting the second image from the second image candidate based on the amount of operation.

[0018] According to this configuration, the user can switch the second image displayed on the display by adjusting the amount of the viewpoint switching operation, thereby enabling the user to check various second images.

[0019] (4) In the information processing method described in (3) above, selecting the second image from the second image candidates may include determining a positional relationship between the shooting location of the second image candidate and the position of the specified object, calculating a rotation angle between a reference direction set in the second image candidate and a direction extending from the shooting location of the second image candidate as a base point toward the specified object based on the positional relationship, and selecting the second image candidate having the rotation angle corresponding to the amount of operation as the second image.

[0020] According to this configuration, the second image candidate having a rotation angle corresponding to the amount of viewpoint switching operation can be displayed on the display as the second image, allowing the user to intuitively switch the image displayed as the second image.

[0021] (5) In the information processing method described in any one of (1) to (4) above, the method may further include obtaining a first display size of the specified object in the first image, and displaying the second image on the display may include enlarging or reducing the specified object in the second image to set the second display size of the specified object in the second image to the same size as the first display size, and displaying the second image including the specified object set to the same size on the display.

[0022] According to this configuration, the second display size is set to the same size as the first display size, so that a predetermined object can be displayed on the display at a constant display size.

[0023] (6) In the information processing method described in any one of (1) to (5) above, the method may further include displaying on the display a bird's-eye view in which a point-of-interest icon indicating the shooting location of the second image among the plurality of shooting location icons is highlighted.

[0024] According to this configuration, the point-of-interest icon indicating the shooting location of the second image is highlighted on the bird's-eye view, allowing the user to intuitively understand the shooting location of the second image.

[0025] (7) The information processing method described in (6) above may further include setting a color of a frame surrounding the second image and a color of a frame surrounding the point of interest icon to the same color.

[0026] According to this configuration, the color of the frame surrounding the second image and the color of the frame surrounding the point of interest icon are set to the same color, allowing the user to more intuitively understand the location where the second image was taken.

[0027] (8) In the information processing method described in any one of (1) to (7) above, detecting that the specified object has been selected may include detecting an object included in the first image, highlighting and displaying on the display an object area in which the detected object is depicted, and accepting selection of the object area.

[0028] According to this configuration, the area in the first image in which the object is drawn is highlighted, and the user can select the object by selecting the highlighted area, which allows the user to easily select the object.

[0029] (9) In the information processing method described in any one of (1) to (8) above, detecting that the specified object has been selected may include displaying a bird's-eye view in which selectable areas are set on the display, detecting that the user has selected a specified area from the selectable areas, identifying a shooting location icon associated with an image of the specified area, identifying a corresponding area corresponding to the specified area from the image of the specified area, and estimating that the object included in the corresponding area is the specified object.

[0030] According to this configuration, an object present in an area selected by the user from among the selectable areas of the bird's-eye view is estimated as the predetermined object selected by the user. In this way, the user can select the predetermined object from the bird's-eye view. As a result, the user can more easily select the predetermined object.

[0031] Furthermore, the present disclosure can be realized not only as an information processing method that executes the characteristic processes described above, but also as an information processing device having a characteristic configuration corresponding to the characteristic processes executed by the information processing method. Furthermore, the present disclosure can also be realized as a computer program that causes a computer to execute the characteristic processes included in such an information processing method. Therefore, the same effects as those of the above information processing method can also be achieved in the following other aspects.

[0032] (10) An information processing device according to another aspect of the present disclosure is an information processing device including a processor, wherein the processor performs the following operations: displaying a bird's-eye view on a display; displaying a shooting location icon indicating a shooting location on the bird's-eye view; detecting that a specific icon from the plurality of shooting location icons has been selected; displaying a first image associated with the specific icon on the display; detecting that a predetermined object included in the first image has been selected; and, in response to the selection of the predetermined object, displaying on the display a second image including the predetermined object that has been photographed from a shooting location different from the shooting location of the first image and that has been photographed at a time different from the shooting time of the first image.

[0033] (11) An information processing program according to yet another aspect of the present disclosure causes a computer to perform the following operations: display a bird's-eye view on a display; display a shooting location icon indicating the shooting location on the bird's-eye view; detect that a specific icon from the plurality of shooting location icons has been selected; display a first image associated with the specific icon on the display; detect that a predetermined object included in the first image has been selected; and, in response to the selection of the predetermined object, display on the display a second image including the predetermined object that has been photographed from a shooting location other than the shooting location of the first image and that has been photographed at a time different from the shooting time of the first image.

[0034] The present disclosure can also be realized as an information processing system operated by such an information processing program. Needless to say, such a program can be distributed on a non-transitory recording medium such as a computer-readable CD-ROM, or via a communication network such as the Internet.

[0035] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in an independent claim that represents a top-level concept will be described as optional components. Furthermore, in all embodiments, the respective contents can be combined.

[0036] First Embodiment FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to this embodiment.

[0037] The information processing system 1 is a system that, when a viewpoint switching operation is received while a first image IM1 in which a predetermined object is captured is displayed, displays a second image IM2 in which the predetermined object is captured and captured from a different viewpoint. The information processing system 1 includes a server 10, an information terminal 20, and an image capturing device 30.

[0038] The server 10 is an example of an information processing device and a computer. The server 10, the information terminal 20, and the imaging device 30 are connected to each other so as to be able to communicate with each other via a network NT. An example of the network NT is the Internet. The server 10 is, for example, a cloud server configured with one or more computers. However, this is just one example, and the server 10 may be configured as an edge server or may be implemented in the information terminal 20. The aspect in which the server 10 is implemented in the information terminal 20 is an example of the aspect in which the information terminal 20 is configured as an information processing device.

[0039] The information terminal 20 is a terminal operated by a user. The user is, for example, a manager of a predetermined space that is photographed by the imaging device 30. The predetermined space is, for example, a construction site. However, this is just one example, and the predetermined space may be a construction site, a factory, a store, an office, etc. The information terminal 20 may be configured as a portable computer such as a smartphone or a tablet computer, or as a stationary computer. Although one information terminal 20 is illustrated in the example of FIG. 1, multiple information terminals may be connected to the server 10 via the network NT. The information terminal 20 includes a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0040] The communication unit 21 is a communication interface that connects the information terminal 20 to the network NT. The communication unit 21 transmits to the server 10 signals indicating various instructions that the operation unit 24 has received from the user.

[0041] The processor 22 is configured, for example, by a central processing unit (CPU), and displays the design drawing of the predetermined space transmitted from the server 10 on the display 23 .

[0042] The display 23 is configured with various display devices such as a liquid crystal display and an organic EL display, and displays various display screens under the control of the processor 22. In this embodiment, the display 23 displays a display screen of a design drawing of a predetermined space.

[0043] The operation unit 24 is configured with, for example, a keyboard, a touch panel, a mouse, etc. A user operates the operation unit 24 to input various instructions.

[0044] The image capturing device 30 is, for example, an omnidirectional camera that captures images at a predetermined frame rate. An omnidirectional camera is also called a 360-degree camera and is a camera that can capture images in all directions of 360 degrees. The image capturing device 30 is, for example, a portable image capturing device carried by a photographer. The photographer may be, for example, a construction site worker or site supervisor. The image capturing device 30 may also be configured as a normal camera. The image capturing device 30 has a communication unit 40. The communication unit 40 connects the information terminal 20 and the image capturing device 30 so that they can communicate with each other via a short-range wireless communication path such as Bluetooth (registered trademark) or a wireless LAN.

[0045] The photographer moves within a predetermined space while photographing the construction site with the photographing device 30. When the photographing is finished, photographing information including the series of images photographed by the photographing device 30 is transferred to the information terminal 20 via the communication unit 40 and temporarily stored in the information terminal 20. The photographing information stored in the information terminal 20 is uploaded to the server 10 by the photographer operating the information terminal 20.

[0046] The photographing information includes an image photographed by the photographing device 30, a photographing start point, a photographing end point, and a blueprint ID corresponding to the specified space. The blueprint ID is an identifier for identifying the blueprint of the specified space. The photographing start point and the photographing end point are added by the information terminal 20.

[0047] At the start point of shooting, the photographer points the camera 30 in a predetermined direction, presses the camera button on the camera 30, and starts the shooting operation. When the photographer reaches the end point of shooting, he or she presses the camera button again to end the shooting operation. The predetermined direction is a direction that aligns with the blueprint displayed on the display 23. The predetermined direction is, for example, north, but is not limited to this.

[0048] The start and end points of photography are identified by the photographer inputting instructions specifying the positions on the display screen of a blueprint of a specific space displayed on the display 23 of the information terminal 20 or the like. Two-dimensional coordinate axes are defined on the blueprint. Therefore, the start and end points of photography are each defined by two-dimensional coordinate values. The start and end points of photography are used by the server 10 to identify the position of each photography point and the photography direction of each image.

[0049] The server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 is configured by, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 11 includes an acquisition unit 111 and a display control unit 113. The acquisition unit 111 and the display control unit 113 may be realized by the processor 11 executing an information processing program, or may be configured by a dedicated hardware circuit such as an ASIC.

[0050] The acquisition unit 111 acquires the photography information transmitted from the photography device 30 using the communication unit 13. The acquisition unit 111 acquires a display instruction from the information terminal 20 by communicating with the information terminal 20. The display instruction is an instruction input by the user to the information terminal 20, and is an instruction to command the blueprint to be displayed on the display 23. The acquisition unit 111 also acquires a selection instruction for a photography location icon by communicating with the information terminal 20. The selection instruction is an instruction indicating that the user has selected one of the photography location icons displayed on the blueprint.

[0051] The photographing location calculation unit 112 acquires the photographing location of each of the multiple images captured by the photographing device 30 using the photographing information acquired by the acquisition unit 111. The photographing location calculation unit 112 calculates the photographing location of each image using a self-location estimation process. Visual simultaneous localization and mapping (VSLAM) can be used as the self-location estimation process. For example, the photographing location calculation unit 112 detects feature points from each of the multiple images included in the photographing information through image processing. Feature points are characteristic points that indicate the outline of an object included in the image. The photographing location calculation unit 112 performs a self-location estimation process on each of the multiple images using at least two of the multiple images in which feature points are set, and calculates the relative coordinate values ​​of the photographing location, which is the self-location corresponding to each image. Using these relative coordinate values, the photographing location calculation unit 112 creates a slum analysis map. The slum analysis map is a virtual map generated by mapping the photographing locations of each image onto a two-dimensional coordinate system.

[0052] The photography location calculation unit 112 estimates a series of photography locations from the photography start point to the photography end point on the blueprint based on the blueprint, the coordinate values ​​of the photography start point and the photography end point on the blueprint, and multiple images, and then creates multiple photography location icons indicating the photography positions on the blueprint of the series of photography locations, and superimposes these photography location icons on the blueprint.

[0053] The display control unit 113 displays a display screen of the blueprint on the display 23 of the information terminal 20 in response to the acquisition of the display instruction by the acquisition unit 111. The display control unit 113 displays, in response to an instruction to select a photography location icon, an image taken at the photography location indicated by the photography location icon on the display 23 of the information terminal 20.

[0054] The blueprint is an example of a bird's-eye view. The blueprint display instruction includes a blueprint ID. The display control unit 113 reads the blueprint indicated by the blueprint ID from the blueprint information storage unit 121 and displays the display screen of the read blueprint on the display 23.

[0055] A blueprint is a bird's-eye view of a building, including columns, walls, or materials. A blueprint is a drawing showing the design of a construction site, and may be a floor plan, blueprint, map, or perspective drawing of the construction site. A bird's-eye view can also be called a bird's-eye view, and may be a view from above or a view from a high place.

[0056] Fig. 2 is a diagram showing a design drawing 201, which is an example of a design drawing displayed on the display 23. As shown in Fig. 2, a plurality of photography location icons 200 are displayed on the design drawing 201. Specifically, the plurality of photography location icons 200 include photography location icons 211 to 213. Each of the photography location icons 211 to 213 is associated with an image taken at that photography location.

[0057] When the display control unit 113 detects that a specific icon from among the plurality of shooting location icons 200 has been selected, it displays on the display 23 the first image IM1 associated with the specific icon.

[0058] The display control unit 113 also detects that a predetermined object included in the first image IM1 has been selected by the user. In response to this detection, the display control unit 113 displays on the display 23 a second image IM2 that includes the predetermined object and was photographed from a different photographing location than the photographing location where the first image IM1 was photographed, and that was photographed at a different time than the photographing location where the first image IM1 was photographed. Details will be described later.

[0059] The memory 12 is configured as a non-volatile rewritable storage device such as a hard disk drive or a solid state drive. The memory 12 includes a blueprint information storage unit 121 and an image information storage unit 122. The blueprint information storage unit 121 stores blueprint information. The blueprint information is an image showing a blueprint of a specified space. The blueprint information is associated with a blueprint ID that identifies the blueprint.

[0060] The image information storage unit 122 stores image information including the photography information transmitted from the photography device 30. The image information includes multiple images, the photography start point, the photography end point, and the blueprint ID included in the photography information transmitted from the photography device 30, as well as meta information for each image. The image information is generated each time the photography information described above is transmitted. The meta information includes the photography ID, photography period ID, photography time, photography direction, and photography location.

[0061] The shooting ID is an identifier of the shooting location. The shooting period ID is an identifier of the shooting period. The shooting period refers to the period from the start to the end of the shooting operation. The shooting time is the date and time when the image was shot. The shooting direction is the shooting direction of each image by the shooting device 30. The shooting location is composed of a position (two-dimensional coordinate value) that indicates the shooting location of the image on the slum analysis diagram.

[0062] The communication unit 13 is a communication interface that connects the server 10 to a network.

[0063] Next, the processing of the information processing system 1 will be described. FIG. 3 is a flowchart showing an example of the flow of processing performed in the information processing system 1. For example, the processing of step S1 shown in FIG. 3 is started when a user selects an arbitrary photography location icon from the design drawing 201 (FIG. 2) displayed on the display 23. Note that, as preprocessing performed before the processing of step S1 in FIG. 3 is started, the display control unit 113 executes the above-mentioned slum analysis processing. Below, the processing of the information processing system 1 will be described using an example in which the user selects the photography location icon 211 from the multiple photography location icons 200 as a specific icon.

[0064] In step S1, the display control unit 113 displays the first image IM1 on the display 23. Specifically, the display control unit 113 displays the first image IM1 associated with the specific icon 211 selected by the user on the display 23.

[0065] The first image IM1 is an image captured using a 360-degree camera. Due to the characteristics of an image captured by a 360-degree camera, it is difficult to display the entire image on the display 23 at once. For this reason, when displaying the first image IM1 on the display 23, the display control unit 113 extracts a partial area from the entire first image IM1 and displays this partial area on the display 23. Hereinafter, the partial area of ​​the entire first image IM1 that is first displayed on the display 23 when the first image IM1 is displayed on the display 23 will be referred to as the first display area.

[0066] In step S2, the display control unit 113 detects that a predetermined object included in the first image IM1 has been selected. First, in step S2, the user searches for the predetermined object in the first display area. If the predetermined object is not included in the first display area, the user inputs a display area change operation. The display area change operation is an operation for displaying an area of ​​the first image IM1 that is different from the first display area on the display 23. Hereinafter, a portion of the entire first image IM1 that is displayed on the display 23 by inputting the display area change operation is referred to as the second display area. The display area change operation is realized, for example, by a swipe operation. That is, the user presses their finger on the display 23 and moves the finger in a predetermined direction. This rotates the viewpoint of the first image IM1, and the second display area is displayed on the display 23. Note that the swipe operation is merely an example, and the display area change operation may also be realized, for example, by a drag operation using a mouse.

[0067] 5 is a diagram showing an example of the second display area of ​​the first image IM1. As shown in FIG. 5, the second display area includes a pillar 301. The following describes an example in which the user selects this pillar 301 as a predetermined object. The user selects the object by, for example, inputting a drag operation such that the mouse cursor moves between two points including the vertex of the pillar 301, or by clicking on a part of the pillar 301.

[0068] In step S3, the display control unit 113 determines whether or not there is an image captured in the vicinity of the capture location of the first image IM1. For example, the display control unit 113 determines whether or not there is at least one capture location icon 200 within a predetermined distance from a specific icon 211 in the design plan 201 shown in FIG. 2. If there is at least one capture location icon 200 within the predetermined distance (YES in step S3), the process proceeds to step S4. On the other hand, if there is no capture location icon 200 within the predetermined distance (NO in step S3), the process ends.

[0069] The image capture device 30 according to this embodiment captures images at a predetermined frame rate (e.g., 30 frames per second). If a capture location icon 200 were generated for each image and all of the capture location icons 200 were superimposed on the blueprint 201, the blueprint 201 would be lined with many capture location icons 200, reducing the visibility of the blueprint 201. Therefore, even if the capture locations of some of the images captured by the image capture device 30 are estimated using the SLAM process, no capture location icon 200 indicating the capture location would be generated. Hereinafter, such images are referred to as thinned images. The thinned images and information indicating the capture locations of the thinned images are stored in a predetermined storage area of ​​the image information storage unit 122, for example. In step S3, in addition to the above-described determination process, the display control unit 113 may determine whether the capture location of the thinned image is within a predetermined distance from the specific icon 211. If the capture location of the thinned image is within the predetermined distance, the display control unit 113 may determine YES in step S3.

[0070] In step S4, the display control unit 113 determines whether or not the slum analysis process has been performed. Specifically, the display control unit 113 determines whether or not the photographing location calculation unit 112 has performed the slum analysis process using the first image IM1 and images photographed in the vicinity of the first image IM1. If the slum analysis process has been performed (YES in step S4), the process proceeds to step S5. On the other hand, if the slum analysis process has not been performed (NO in step S4), the process ends.

[0071] In step S5, the display control unit 113 determines whether the position of a predetermined object can be clarified on the slum analysis map. FIG. 6 is a diagram for explaining the processing in step S5, and is a diagram schematically illustrating a slum analysis map 500 generated by the display control unit 113. As shown in FIG. 6, a first photographing location 501, a second photographing location 502, a third photographing location 503, and a fourth photographing location 504 are mapped on the slum analysis map 500. The first photographing location 501 indicates the photographing location on the slum analysis map 500 of the first image IM1. The second photographing locations 502 to 504 indicate the positions on the slum analysis map 500 of images photographed around the first photographing location 501. Hereinafter, the image photographed at the second photographing location 502 will be referred to as Image A, the image photographed at the third photographing location 503 as Image B, and the image photographed at the fourth photographing location 504 as Image C. The first photographing location 501 corresponds to the specific icon 211 on the blueprint 201 in Figure 2. The second photographing location 502 corresponds to the photographing location icon 212 on the blueprint 201. The fourth photographing location 504 corresponds to the photographing location icon 213 on the blueprint 201. Note that the B image photographed at the third photographing location 503 is a thinned image, and no photographing location icon indicating the photographing location of the B image is displayed on the blueprint 201.

[0072] In step S5, the display control unit 113 estimates the position of a predetermined object (pillar 301) on the slum analysis diagram 500 based on the amount of the display area change operation performed by the user in step S2. For example, if the user performs a swipe operation (referred to as a first swipe operation) to change the display area in step S2, the display control unit 113 estimates the position of the pillar 301 on the slum analysis diagram 500 based on the swipe amount of this first swipe operation.

[0073] First, the display control unit 113 identifies the first field of view 401 ( FIG. 6 ), which is the field of view of the first display area, on the slum analysis diagram 500 using the position of the first photographing point 501 on the slum analysis diagram, a reference direction D that is preset for the first image IM1, and the horizontal field of view of the first display area of ​​the first image IM1. The reference direction D is, for example, a direction corresponding to the front direction of the photographing device 30. Alternatively, the reference direction D may be arbitrarily set by the worker. The first field of view 401 is a fan-shaped area extending from the first photographing point 501 along the reference direction D, and the field of view is determined by the horizontal field of view of the first display area.

[0074] Next, the display control unit 113 identifies a second field of view 402 ( FIG. 6 ) on the slum analysis diagram 500 based on the swipe amount of the first swipe operation input by the user in step S2. For example, assume that in step S2, the user inputs a first swipe operation that rotates the viewpoint by 90 degrees from the reference direction D. As a result, assume that the second display area of ​​the first image IM1 shown in FIG. 5 is displayed. In this case, the display control unit 113 identifies a fan-shaped area extending in a direction rotated 90 degrees from the reference direction D as the second field of view 402 on the slum analysis diagram 500. The second field of view 402 is an area whose field of view is determined by the horizontal angle of view of the second display area.

[0075] Next, the display control unit 113 performs predetermined image processing on the second display region of the first image IM1 to estimate the position of the pillar 301 within the second field of view range 402. Then, the position of the pillar 301 is mapped on the slum analysis diagram 500.

[0076] If the display control unit 113 can map the position of the pillar 301 on the slum analysis diagram, it determines YES in step S5. Then, the process proceeds to step S6. On the other hand, if the position of the pillar 301 cannot be mapped on the slum analysis diagram, it determines NO in step S5. In this case, the process ends.

[0077] In step S6, the display control unit 113 identifies the positional relationship based on the position of a predetermined object (pillar 301) on the slum analysis diagram 500. Figure 7 is a diagram for explaining the processing in step S6, and is a diagram schematically showing the slum analysis diagram 500 generated by the display control unit 113.

[0078] First, the display control unit 113 identifies the positional relationship between the position of the first photographing point 501 on the slum analysis map 500 and the position of the pillar 301. Specifically, the display control unit 113 sets a first vector V1 ( FIG. 7 ) on the slum analysis map 500, the first vector V1 extending from the first photographing point 501 toward the pillar 301. Then, the display control unit 113 acquires the magnitude and direction of this first vector V1.

[0079] Next, the display control unit 113 identifies the positional relationship between the position of the pillar 301 on the slum analysis map 500 and the positions of each of the second to fourth photographing points 502 to 504. Specifically, the display control unit 113 sets a second vector V2 on the slum analysis map 500, extending from the second photographing point 502 as a base point toward the pillar 301. Then, the display control unit 113 acquires the magnitude and direction of this second vector V2. Similarly, the display control unit 113 acquires the magnitude and direction of a third vector V3, extending from the third photographing point 503 as a base point toward the pillar 301. Furthermore, the display control unit 113 acquires the magnitude and direction of a fourth vector V4, extending from the fourth photographing point 504 as a base point toward the pillar 301.

[0080] Next, the display control unit 113 calculates a first rotation angle A1 ( FIG. 7 ) formed between the reference direction D set in image A captured at the second imaging point 502 and the second vector V2. For example, the display control unit 113 calculates that the first rotation angle A1 is 120 degrees. The display control unit 113 also calculates a second rotation angle A2 formed between the reference direction D set in image B captured at the third imaging point 503 and the third vector V3. For example, the display control unit 113 calculates that the second rotation angle A2 is 135 degrees. The display control unit 113 also calculates a third rotation angle A3 formed between the reference direction D set in image C captured at the fourth imaging point 504 and the fourth vector V4. For example, the display control unit 113 calculates that the third rotation angle A3 is 160 degrees.

[0081] In step S7, the display control unit 113 normalizes the magnitude of the first vector V1 and acquires a first display size of the pillar 301 reflected in the first image IM1. The display control unit 113 stores the normalized magnitude of the first vector V1 and the first display size in a predetermined storage area of ​​the memory 12. The first display size is defined by the proportion of the pillar 301 reflected in the second display area of ​​the first image IM1 to the entire screen of the display 23.

[0082] In step S8, the display control unit 113 displays on the display 23 a user interface (hereinafter referred to as a viewpoint switching UI) that allows the user to switch the viewpoint based on a predetermined object. For example, the display control unit 113 displays on the display 23 a button containing text such as "viewpoint switching mode." Then, upon detecting that this button has been pressed (selected) by the user, the display control unit 113 displays on the display 23 an initial screen of the viewpoint switching UI. For example, the display control unit 113 displays on the display 23 the second display area of ​​the first image IM1 as the initial screen. When the initial screen of the viewpoint switching UI is displayed, the user inputs a viewpoint switching operation for switching the viewpoint on the first image IM1. Switching the viewpoint refers to displaying on the display 23 a second image IM2 that includes a predetermined object photographed from a different photographing location from the photographing location where the first image IM1 was photographed. In other words, it refers to displaying on the display 23 a pillar 301 photographed from a different angle. The viewpoint switching operation is realized, for example, by a swipe operation. However, this is merely an example, and the viewpoint switching operation may be realized by, for example, a drag operation. Hereinafter, the swipe operation performed by the user to switch the viewpoint will be referred to as a second swipe operation.

[0083] In step S9, the display control unit 113 determines whether the user has performed an operation intended to switch the viewpoint toward a predetermined object (pillar 301). That is, it determines whether the user has performed a second swipe operation on the first image IM1. If the input of the second swipe operation is detected (YES in step S9), the process proceeds to step S10. If the input of the second swipe operation is not detected (NO in step S9), the process returns to step S8.

[0084] In step S10, the display control unit 113 acquires a second operation amount indicating the operation amount of the viewpoint switching operation (second swipe operation). That is, the display control unit 113 determines how far the user moved their finger in the second swipe operation.

[0085] In step S11, the display control unit 113 selects the second image IM2 based on the amount of viewpoint switching operation performed on the first image IM1. Specifically, the display control unit 113 selects the second image IM2 from the second image candidates based on the second amount of operation.

[0086] The second image candidate is composed of images taken from a shooting location located within a predetermined distance from the first shooting location 501. The display control unit 113 selects, as the second image candidate, an image that is likely to include the pillar 301 and was taken from a shooting location different from the first image. Specifically, the display control unit 113 selects the second image candidate based on the positional relationship between the shooting location of each image and the position where the pillar 301 is located, and the shooting direction of each image. That is, the display control unit 113 selects, as the second image candidate, an image that was taken in the vicinity of the first image IM1 and in the direction in which the pillar 301 is located. In this embodiment, since photography is performed using a 360-degree camera, an image taken in the vicinity of the shooting location of the first image IM is likely to include the pillar 301. Therefore, the display control unit 113 according to this embodiment selects, as the second image candidate, an image that was taken within a predetermined distance from the first shooting location 501. For example, image A taken at the second shooting location 502, image B taken at the third shooting location 503, and image C taken at the fourth shooting location 504 are selected as second image candidates. However, this is merely an example, and more or fewer second image candidates may be selected. For example, there may be only one second image candidate.

[0087] In step S11, the display control unit 113 selects, as the second image IM2, a second image candidate having a rotation angle corresponding to the second operation amount. As an example, assume that the second operation amount is small, i.e., the user moves their finger slightly during the second swipe operation. In this case, the display control unit 113 selects, as the second image IM2, an image captured from a shooting point with the smallest rotation angle. As described above, among the first rotation angle A1, the second rotation angle A2, and the third rotation angle A3 shown in FIG. 7 , the first rotation angle A1 (120 degrees in this example) is the smallest. Therefore, the display control unit 113 selects, as the second image IM2, image A, which is an image captured from the second shooting point 502.

[0088] As another example, assume that the second operation amount is medium, i.e., the user moves his / her finger moderately during the second swipe operation. In this case, the display control unit 113 selects the image captured from the shooting point with the second smallest rotation angle as the second image IM2. Specifically, the display control unit 113 selects image B captured from the third shooting point 503 ( FIG. 7 ) as the second image IM2.

[0089] As yet another example, when the second operation amount is large, that is, when the user moves his / her finger a large amount during the second swipe operation, the display control unit 113 selects the image captured from the shooting point with the largest rotation angle as the second image IM2. Specifically, the display control unit 113 selects image C captured from the fourth shooting point 504 ( FIG. 7 ) as the second image IM2.

[0090] The display control unit 113 may determine the magnitude of the second operation amount based on whether the movement amount of the user's finger exceeds a predetermined threshold. For example, if the movement amount of the user's finger on the display 23 is less than 100 pixels, the display control unit 113 determines that the second operation amount is small. Furthermore, if the movement amount of the user's finger on the display 23 is, for example, 100 pixels or more but less than 200 pixels, the display control unit 113 determines that the second operation amount is medium. Furthermore, if the movement amount of the user's finger on the display 23 is, for example, 200 pixels or more, the display control unit 113 determines that the second operation amount is large. However, these numerical values ​​are merely examples, and the thresholds can be changed as appropriate depending on the size of the display 23, etc. Furthermore, if there are more second image candidates, the second operation amount may be divided into more detailed categories by setting more thresholds.

[0091] In step S12, the display control unit 113 sets the angle of view of the second image IM2. As described above, because the second image IM2 is an image captured by a 360-degree camera, it is difficult to display the entire second image IM2 on the display 23 at once. For this reason, when displaying the second image IM2 on the display 23, the display control unit 113 extracts a partial area from the entire second image IM2 and displays this partial area on the display 23. Hereinafter, the partial area from the entire second image IM2 that is initially displayed on the display 23 will be referred to as the initial area. It is desirable that the pillar 301 be captured in this initial area. If the pillar 301 is not captured in the initial area, the user will be forced to perform the tedious task of rotating the viewpoint of the second image IM2 to find the area in the second image IM2 where the pillar 301 is located. In step S12, the angle of view of the second image IM2 is set so that the pillar 301 is captured in the initial region. For example, suppose the display control unit 113 selects image A captured at the second imaging point 502 as the second image IM2. In this case, the display control unit 113 sets the center of the angle of view of the second image IM2 in the direction in which the second vector V2 extends. In other words, the angle of view of the second image IM2 is set so that the field of view expands in the direction in which the pillar 301 is located. Similarly, if image B captured from the second imaging point 502 is selected as the second image IM2, the angle of view of the second image IM2 is set based on the direction in which the third vector V3 extends. If image C captured from the third imaging point 503 is selected as the second image IM2, the angle of view of the second image IM2 is set based on the direction in which the fourth vector V4 extends. As a result, when the image displayed on the display 23 is switched from the first image IM1 to the second image IM2, the second image IM2 in which the pillar 301 is reflected can be presented to the user. In other words, the second image IM2 in which the viewpoint is directed in the direction in which the pillar 301 exists can be presented to the user.

[0092] In step S13, the display control unit 113 determines whether the magnitudes of the vectors are different. Specifically, it determines whether the magnitudes of the first vector V1 extending from the first imaging point 501 toward the pillar 301 are different from the magnitude of the vector extending from the imaging point of the second image IM2 toward the pillar 301. For example, assume that image A captured at the second imaging point 502 is selected as the second image IM2. In this case, the display control unit 113 compares the magnitude of the first vector V1 with the magnitude of the second vector V2 to determine whether the magnitudes of these vectors are different. If the magnitudes of the vectors are different (YES in step S13), the process proceeds to step S14. If the magnitudes of the vectors are the same (NO in step S13), the process ends.

[0093] In step S14, the display control unit 113 adjusts the display size (referred to as the second display size) of the pillar 301 appearing in the second image IM2 to the first display size. The second display size is defined by the proportion of the pillar 301 included in the second image IM2 to the entire screen of the display 23.

[0094] If the vector of the second image IM2 is larger than the vector of the first image IM1, that is, if the second image IM2 is an image of the pillar 301 captured from a location farther away than the location of the first image IM1, the pillar 301 appears smaller in the second image IM2 than in the first image IM1. In this case, the display control unit 113 adjusts the second display size to the first display size by enlarging the portion of the second image IM2 in which the pillar 301 is drawn, using the first display size as a reference.

[0095] Furthermore, if the vector of the second image IM2 is smaller than the vector of the first image IM1, that is, if the second image IM2 is an image of the pillar 301 captured from a location closer than the location of the first image IM1, the pillar 301 will appear larger in the second image IM2 than in the first image IM1. In this case, the display control unit 113 reduces the portion of the second image in which the pillar 301 is depicted, using the first display size as a reference, thereby aligning the second display size with the first display size.

[0096] That is, in step S14, the display control unit 113 sets the second display size to the same size as the first display size by enlarging or reducing the pillar 301 in the second image IM2. In this way, the second image IM2 including the pillar 301 set to the same size as the first display size can be displayed on the display 23. That is, the pillar 301 can always be displayed at a constant display size on the display 23. Note that if it is difficult to completely match the first display size and the second display size, the display control unit 113 may approximately match the first display size and the second display size.

[0097] In step S15, the display control unit 113 displays the second image IM2 on the display 23. Fig. 8 is a diagram showing an example of the second image IM2. Fig. 8 is an image captured at the second imaging point 502. In the present embodiment, when the second operation amount of the second swipe operation is small, the second image IM2 shown in Fig. 8 is displayed on the display 23. As shown in Fig. 8, the second image IM2 includes a pillar 301 captured from a different imaging point (angle) from that of the first image IM1.

[0098] Fig. 9 is a diagram showing another example of the second image IM2. Fig. 9 is an image captured at the third imaging location 503. In the present embodiment, when the second operation amount of the second swipe operation is medium, the second image IM2 shown in Fig. 9 is displayed on the display 23. The second image IM2 shown in Fig. 9 includes a side surface 350 of the pillar 301. This side surface 350 is a portion that was not captured in the first image IM1 shown in Fig. 3.

[0099] In this way, by checking the second image IM2 in which the pillar 301 is captured from a different location than the first image IM1, the user can understand the shape and structure of the pillar 301.

[0100] The display control unit 113 may further receive a viewpoint switching operation after displaying the second image IM2. For example, if a viewpoint switching operation is further received after displaying the second image IM2 shown in Fig. 8, the second image IM2 shown in Fig. 9 may be displayed on the display 23.

[0101] According to the information processing system 1 described above, in response to the pillar 301 being selected in the first image IM1, a second image IM2 including the pillar 301 photographed from a different photographing location from the photographing location of the first image IM1 is displayed on the display 23. In this way, the user can easily check the second image IM2, and therefore there is no need to input an operation to search for the second image IM2 into the server 10, etc. As a result, an increase in the processing load of the server 10, etc. is suppressed, and an increase in the burden on the server 10, etc. can be suppressed.

[0102] Furthermore, with the above configuration, the user can understand the structure and configuration of a predetermined object in more detail.

[0103] Furthermore, in the information processing system 1, the angle of view of the second image IM2 is adjusted in advance so that the pillar 301 appears in the initial region of the second image IM2. In this way, the user does not need to search for the pillar 301 in the second image IM2. As a result, the user can efficiently check the shape and structure of the pillar 301.

[0104] Furthermore, in the information processing system 1, in order to ensure ease of viewing and operation of the blueprint 201, a shooting location icon for the thinned image is not generated. On the other hand, if the thinned image is an image taken in the vicinity of the first image IM1, the thinned image is also used as a candidate for the second image. This makes it possible to make the blueprint 201 easy to view and operate, while also allowing the pillar 301 to be viewed from different perspectives.

[0105] The present disclosure can employ the following modifications.

[0106] (1) In step S3 of the first embodiment, if the display control unit 113 determines that the shooting location of the thinned image exists within a predetermined distance from a specific icon 211, it may generate a new shooting location icon (hereinafter referred to as a thinned image icon) indicating the shooting location of the thinned image. This thinned image icon may then be superimposed on the blueprint 201. Furthermore, the display mode of the thinned image icon may be made different from the display mode of the shooting location icons other than the thinned image icon. For example, the display color of the thinned image icon on the blueprint 201 may be made different from the display color of the shooting location icons other than the thinned image icon.

[0107] (2) In conjunction with displaying the second image IM2 on the display 23, the display control unit 113 may change the display mode of the interest point icon indicating the shooting location of the second image IM2 in the design drawing 201.

[0108] 10 is a diagram schematically illustrating a display screen 601 displayed on the display 23 in this modified example. The display screen 601 includes a blueprint display area AR1 and a second image display area AR2. The blueprint display area AR1 displays a blueprint 201 on which a plurality of shooting location icons 200 are superimposed. The second image display area AR2 displays a second image IM2.

[0109] For example, assume that a second image IM2 (referred to as second image IM21) captured at the photography location indicated by the photography location icon 211 in FIG. 10 is displayed in the second image display area AR2. In this case, the display control unit 113 displays a red frame surrounding the contour of the photography location icon 211 in the blueprint display area AR1. Furthermore, the display control unit 113 displays a red frame surrounding the second image IM21 in the second image display area AR2. In other words, the color of the frame surrounding the point of interest icon and the color of the frame surrounding the second image IM2 are set to the same color. As a result, the display control unit 113 highlights the photography location icon 211 (an example of a point of interest icon). In other words, the display mode of the photography location icon 211 is made different from the display mode of the photography location icons other than the photography location icon 211. Note that the red frame is indicated by a thick solid line in FIG. 10 .

[0110] As another example, assume that a second image IM2 (referred to as second image IM22) captured at the photography location indicated by the photography location icon 212 in Fig. 10 is displayed in the second image display area AR2. In this case, the display control unit 113 displays a blue frame surrounding the photography location icon 212 and a blue frame surrounding the second image IM22 on the display screen 601. As a result, the display control unit 113 highlights the photography location icon 212. In Fig. 10, the blue frame is indicated by a dashed line.

[0111] As yet another example, consider a case in which a second image IM2 (referred to as second image IM23) captured at the capture location indicated by the capture location icon 220 in Fig. 10 is displayed in the second image display area AR2. In this case, the display control unit 113 displays a green frame surrounding the capture location icon 220 and a green frame surrounding the second image IM23 on the display screen 601. This causes the display control unit 113 to highlight the capture location icon 220. In Fig. 10, the green frame is indicated by a dashed line. Note that the capture location icon 220 is a thinned-out image icon.

[0112] As described above, in this modification, the second image IM2 and the design drawing 201 on which the point-of-interest icon indicating the location where the second image IM2 was taken is highlighted are displayed on the same screen of the display 23. In this way, the user can intuitively understand the position on the design drawing 201 of the second image IM2 currently displayed on the display 23.

[0113] In particular, in this modification, the color of the frame surrounding the point of interest icon and the color of the frame surrounding the second image IM2 are set to the same color, thereby emphasizing the point of interest icon, which allows the user to more intuitively understand the shooting point on the design drawing 201 of the second image IM2.

[0114] The display control unit 113 may emphasize the point of interest icon in various ways, such as by flashing the point of interest icon or by making the radius of the point of interest icon larger than the radius of the normal shooting location icon 200 .

[0115] (3) In the first embodiment, an example was described in which a user selects a specific object by specifying the outline of the specific object. However, the selection of an object may be realized in the following manner.

[0116] The display control unit 113 according to this modification performs object recognition processing and boundary recognition processing on the first image IM1 to detect objects included in the first image IM1. The display control unit 113 then highlights the area in which the object is depicted and displays it on the display 23. FIG. 11 is a diagram illustrating an example of the first image IM1 according to this modification. For example, suppose that a pillar 301 and a window 302 shown in FIG. 11 are detected as objects included in the first image IM1. In this case, the display control unit 113 displays bounding boxes BB1 and BB2 around the area in which the pillar 301 and the window 302 are depicted (an example of an object area). Alternatively, the highlighting may be achieved in various ways, such as coloring the pillar 301 and the window 302 or increasing the brightness of the pillar 301 and the window 302.

[0117] The user then selects any object region from among the highlighted object regions in the first image IM1. For example, the user moves the mouse pointer to the region in the first image IM1 where the pillar 301 is depicted, i.e., the region surrounded by the bounding box BB1, and selects the region by clicking the mouse, for example. The display control unit 113, having received the above operation, detects that a predetermined object has been selected. This modification allows the user to more easily select a predetermined object.

[0118] (4) The display control unit 113 may display a design drawing in which the selectable area 203 is set on the display 23. The display control unit 113 may also detect that a user has selected a predetermined area from the selectable area 203, and identify a shooting location icon associated with an image of the predetermined area. Then, the display control unit 113 may identify a corresponding area from the image of the predetermined area that corresponds to the predetermined area, and estimate an object included in the corresponding area as the predetermined object.

[0119] The display control unit 113 according to this modification first creates a bird's-eye view in which a selectable area 203 is set. FIG. 12 is a diagram showing an example of a blueprint 202 (an example of a bird's-eye view) according to this modification. The display control unit 113 recognizes the blueprint 202 and the positions of multiple shooting location icons 200 displayed on the blueprint 202. The display control unit 113 then sets the space in which the shooting location icons 200 are arranged (referred to as an icon arrangement space) as the selectable area 203. In other words, the display control unit 113 sets areas that are likely to have been photographed by the photographer's shooting action as the selectable area 203. For example, the shooting location icons 200 are arranged in the hallway C, room RM1, and room RM2 shown in FIG. 12. These spaces are likely to have been photographed by the photographer's shooting action. In other words, objects present in these areas are likely to have been photographed. Therefore, the display control unit 113 sets these spaces as the selectable area 203. The display control unit 113 also sets, in the selectable area 203, spaces that can be treated as substantially the same space as the icon arrangement space, for example, because they are connected to the icon arrangement space. For example, no shooting location icon 200 is placed in room RM3 shown in FIG. 12. Here, assume that information indicating that rooms RM2 and RM3 are connected is stored in a predetermined storage area of ​​the memory 12. In this case, the display control unit 113 sets room RM3 to the selectable area 203. In this way, the display control unit 113 creates a design drawing 202 on which the selectable area 203 is superimposed.

[0120] Next, the display control unit 113 presents the design drawing 202 to the user. At this time, the display control unit 113 highlights the selectable area 203 and displays it on the display 23. For example, the display 23 displays the design drawing 202 with the selectable area 203 colored green.

[0121] Next, the user selects one of the areas from the selectable area 203. The information terminal 20, which has accepted this selection, transmits the user's selection result to the display control unit 113. Here, it is assumed that the user has selected the area indicated by point P in FIG. 12, for example.

[0122] Next, the display control unit 113, which has acquired the user's selection result, identifies an object that exists at point P. First, the display control unit 113 identifies a shooting location icon (referred to as a corresponding icon) that is associated with an image in which the object that exists at point P is captured. Here, it is highly likely that the object that exists at point P is captured in an image that is associated with a shooting location icon that is located closest to point P. For this reason, the display control unit 113 identifies the shooting location icon that is located closest to point P as the corresponding icon. For example, the display control unit 113 identifies the shooting location icon 250 shown in FIG. 12 as the corresponding icon. At this time, the display control unit 113 sets a vector that extends from the shooting location icon 250 toward point P.

[0123] Next, the display control unit 113 identifies an object (referred to as a corresponding object) located at point P based on an image (referred to as a corresponding image) associated with the shooting location icon 250. First, the display control unit 113 determines the relative positional relationship between the shooting location icon 250 and point P based on the above vector. For example, the display control unit 113 determines that point P is located 100 pixels away from the shooting location icon 250 in the -X direction. Note that the -X direction is the negative direction of the horizontal coordinate axis of the blueprint 202. Next, the display control unit 113 analyzes the corresponding image to identify an area within the corresponding image that corresponds to a position 100 pixels away from the shooting location icon in the -X direction. Then, the display control unit 113 identifies an object (e.g., pillar 301) depicted in this area. The display control unit 113 then estimates the object as the object selected by the user.

[0124] According to the configuration of this modified example, an object selected by the user from the blueprint 202 is estimated as the predetermined object. Therefore, the user can more easily select the predetermined object compared to the case where the user opens the first image, searches for the predetermined object in the first image by performing a swipe operation or the like, and selects the predetermined object.

[0125] (5) In the first embodiment, an example of executing the process of adjusting the second display size to the first display size has been described, but this process is not essential. That is, the processes of step S8 in Fig. 3 and steps S14 and S15 in Fig. 4 can be omitted. Whether or not to omit these processes may be set in advance by the user.

[0126] (6) In the first embodiment, an example has been described in which the second image IM2 is selected in response to detection of a viewpoint switching operation, but detection of a viewpoint switching operation is not essential. The display control unit 113 may select the second image IM2 in response to detection of a user selecting a predetermined object.

[0127] (7) The display control unit 113 may determine whether an object corresponding to the pillar 301 is included in the second image candidate. For example, the display control unit 113 acquires a first feature amount indicating the shape, size, color, etc. of the pillar 301 in the first image. Then, using this first feature amount, it may determine whether an object (object corresponding to the pillar 301) whose similarity to the pillar 301 is equal to or greater than a predetermined reference similarity is included in the second image candidate. Images that do not include an object corresponding to the pillar 301 may be excluded from the second image candidate. In this way, it is possible to prevent inappropriate images from being included in the second image candidate. Specifically, it is possible to exclude from the second image candidate images images that do not include the pillar 301 or images in which the focus is not on the pillar 301.

[0128] The technology according to the present disclosure is useful in the technical field of displaying images on a display or the like.

Claims

1. An information processing method on a computer, comprising: displaying a bird's-eye view on a display; displaying a plurality of shooting location icons indicating shooting locations on the bird's-eye view; detecting that a specific icon from the plurality of shooting location icons has been selected; displaying a first image corresponding to the specific icon on the display; detecting that a specified object included in the first image has been selected; and in response to the selection of the specified object, displaying on the display a second image including the specified object photographed from a shooting location other than the shooting location of the first image, the second image being photographed at a time different from the shooting time of the first image.

2. The information processing method of claim 1, further comprising: selecting an image associated with a shooting location icon located within a predetermined distance from the specific icon as a second image candidate; and selecting the second image from the second image candidates.

3. The information processing method of claim 2, further comprising: detecting that a viewpoint switching operation has been input for the first image; and acquiring an amount of operation of the viewpoint switching operation; the second image candidates include at least two or more images; and displaying the second image on the display comprises selecting the second image from the second image candidates based on the amount of operation.

4. The information processing method of claim 3, wherein selecting the second image from the second image candidates includes: determining a positional relationship between a shooting point of the second image candidate and a position of the specified object; calculating a rotation angle between a reference direction set in the second image candidate and a direction extending from the shooting point of the second image candidate as a base point toward the specified object based on the positional relationship; and selecting the second image candidate having the rotation angle corresponding to the amount of operation as the second image.

5. An information processing method according to any one of claims 1 to 4, further comprising obtaining a first display size of the specified object in the first image, and displaying the second image on the display comprises: setting a second display size of the specified object in the second image to the same size as the first display size by enlarging or reducing the specified object in the second image; and displaying the second image including the specified object set to the same size on the display.

6. An information processing method according to any one of claims 1 to 4, further comprising displaying on the display a bird's-eye view in which a focus point icon indicating the shooting location of the second image among the plurality of shooting location icons is highlighted.

7. The information processing method according to claim 6, further comprising setting the color of a frame surrounding the second image and the color of a frame surrounding the point of interest icon to the same color.

8. An information processing method according to any one of claims 1 to 4, wherein detecting that the specified object has been selected includes: detecting an object contained in the first image; highlighting and displaying on the display an object area in which the detected object is depicted; and accepting selection of the object area.

9. An information processing method according to any one of claims 1 to 4, wherein detecting that the specified object has been selected includes: displaying on the display a bird's-eye view in which selectable areas are set; detecting that a user has selected a specified area from the selectable area; identifying a shooting location icon associated with an image of the specified area; identifying a corresponding area corresponding to the specified area from among the images of the specified area; and estimating an object included in the corresponding area as the specified object.

10. An information processing device including a processor, wherein the processor performs the following operations: displaying a bird's-eye view on a display; displaying a shooting location icon indicating the shooting location on the bird's-eye view; detecting that a specific icon from the multiple shooting location icons has been selected; displaying a first image corresponding to the specific icon on the display; detecting that a specified object included in the first image has been selected; and in response to the selection of the specified object, displaying on the display a second image including the specified object photographed from a shooting location other than the shooting location of the first image, the second image being photographed at a time different from the shooting time of the first image.

11. An information processing program that causes a computer to execute the following operations: display a bird's-eye view on a display; display a shooting location icon indicating the shooting location on the bird's-eye view; detect that a specific icon from the multiple shooting location icons has been selected; display a first image corresponding to the specific icon on the display; detect that a specified object included in the first image has been selected; and in response to the selection of the specified object, display on the display a second image including the specified object that was photographed from a shooting location other than the shooting location of the first image and that was photographed at a time different from the shooting time of the first image.