Information processing method, information processing device, and information processing program

Abstract

The information processing method is an information processing method in a computer, and comprises the following steps: displaying a bird's-eye view in a display; displaying a plurality of shooting site icons representing shooting sites in the aerial view; detecting that a specific icon among the plurality of shooting site icons has been selected; a display unit that displays a first image corresponding to the specific icon on the display; detecting a given object included in the selected first image; and displaying, on the display, on the basis of the selected predetermined object, a second image including the predetermined object captured from an imaging site different from the imaging site of the first image, and the second image captured at a time different from the imaging time of the first image.

Description

Technical Field

[0001] This disclosure relates to techniques for displaying images. Background Technology

[0002] Patent Document 1 discloses a technique that, when the user feels a sense of stability, creates image information that overlays guidance information indicating street names on the path the user is walking on, and displays this image information on a liquid crystal display. Additionally, Patent Document 1 discloses a technique that, when the user feels a sense of openness, creates image information that overlays guidance information indicating the names of surrounding buildings on the path the user is walking on, and outputs this image information to a liquid crystal display.

[0003] Patent Document 1 only discloses a technique that presents users with image information overlaid with guidance information indicating the names of streets and buildings. Therefore, in the technique described in Patent Document 1, it is not easy for a user to identify an image containing a given object captured from another shooting location. In this case, the user sometimes inputs an operation into the computer to find an image containing a given object captured from another shooting location. As a result, the computer's processing steps increase, and the computer's workload increases.

[0004] Prior art literature

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2017-167501 Summary of the Invention

[0007] This disclosure is made based on the aforementioned issues, and its purpose is to provide a technique that facilitates the identification of an image containing a given object captured from another shooting location.

[0008] One aspect of this disclosure relates to an information processing method in a computer, comprising: displaying a bird's-eye view on a display; displaying a plurality of shooting location icons representing shooting locations in the bird's-eye view; detecting the selection of a specific icon among the plurality of shooting location icons; displaying a first image corresponding to the specific icon on the display; detecting the selection of a given object contained in the first image; and, based on the selection of the given object, displaying a second image on the display containing the given object taken from a shooting location different from the shooting location of the first image, and the second image being taken at a different time than the shooting time of the first image.

[0009] According to this disclosure, since the user can easily confirm the second image of the given object contained in the first image taken from another shooting location, the increased computer load can be suppressed. Attached Figure Description

[0010] Figure 1 This is a diagram showing the overall structure of the information processing system in this embodiment.

[0011] Figure 2 This is an example of a design diagram displayed on a monitor.

[0012] Figure 3 This is a flowchart illustrating an example of the processing flow in an information processing system.

[0013] Figure 4 This is a flowchart illustrating an example of the processing flow in an information processing system.

[0014] Figure 5 This is an example diagram showing the second display area in the first image.

[0015] Figure 6 This is a schematic diagram showing the SLAM analysis diagram generated by the display control unit.

[0016] Figure 7 This is a schematic diagram showing the SLAM analysis diagram generated by the display control unit.

[0017] Figure 8 This is an example of the second image.

[0018] Figure 9 This is another example of the second image.

[0019] Figure 10 This is a schematic diagram showing the display screen on the monitor in variant example (2).

[0020] Figure 11 This is a diagram showing the first image involved in the variation (3).

[0021] Figure 12 This is a diagram showing a design with selectable blocks. Detailed Implementation

[0022] (The understanding that forms the basis of this disclosure)

[0023] The following user interface is being developed: on a two-dimensional design drawing of a construction site, multiple location icons representing actual photographic locations are displayed. When a location icon is selected, an image taken at that location is displayed. This allows construction site managers to monitor the construction site's condition without physically visiting the site.

[0024] In the user interface described above, users sometimes select a specific shooting location icon, causing a monitor or other display to show an image associated with that specific shooting location icon (referred to as the first image), and then confirm the given object captured in the first image. Here, to gain a more detailed understanding of the structure, shape, etc., of the given object captured in the first image, users sometimes wish to confirm a second image of the given object, taken from a different shooting location than the first image. In this case, users sometimes input an operation into the computer to find a shooting location icon associated with an appropriate image for the second image. As a result, the computer's processing steps increase, thus increasing the computer's load.

[0025] Such a problem cannot be solved using the technology described in Patent Document 1. Patent Document 1 only discloses a technique for overlaying the name of an object onto an object captured in an image, without addressing the aforementioned problem.

[0026] To address the above problems, the following technology has been disclosed.

[0027] (1) An information processing method in one aspect of the present disclosure is an information processing method in a computer, comprising: displaying a bird's-eye view on a display; displaying a plurality of shooting location icons representing shooting locations in the bird's-eye view; detecting the selection of a specific icon among the plurality of shooting location icons; displaying a first image corresponding to the specific icon on the display; detecting the selection of a given object contained in the first image; and, based on the selection of the given object, displaying a second image containing the given object taken from a shooting location different from the shooting location of the first image, and the second image taken at a time different from the shooting time of the first image on the display.

[0028] According to this structure, based on the selection of a given object in the first image, a second image containing the given object taken from a different shooting location than the first image is displayed on the monitor. Thus, since the user can easily confirm the second image, there is no need to input the operation for finding the second image into the computer. As a result, the increase in computer processing power is suppressed, thereby reducing the burden on the computer.

[0029] (2) In the information processing method described in (1) above, the information processing method may also include: selecting an image corresponding to a shooting location icon that exists within a given distance from the specific icon as a second image candidate; and selecting the second image from the second image candidate.

[0030] According to this structure, an image taken at a location within a given distance of the location where the first image was taken is selected as a candidate for the second image. In other words, an image with a higher probability of capturing a given object from a different location than the first image is selected as a candidate for the second image. Then, a second image is selected from these candidates. Thus, an appropriate image can be displayed on the monitor as the second image.

[0031] (3) In the information processing method described in (2) above, the information processing method may also include: detecting that a viewpoint switching operation on the first image has been input; and obtaining the operation amount of the viewpoint switching operation, wherein the second image candidate includes at least two or more images, and displaying the second image on the display includes: selecting the second image from the second image candidate based on the operation amount.

[0032] According to this structure, users can switch the second image displayed on the monitor by adjusting the amount of the viewpoint switching operation. This allows users to confirm various second images.

[0033] (4) In the information processing method described in (3) above, selecting the second image from the second image candidates may also include: determining the positional relationship between the shooting location of the second image candidate and the position of the given object; calculating the rotation angle between the reference direction set in the second image candidate and the direction extending towards the given object with the shooting location of the second image candidate as the reference point based on the positional relationship; and selecting the second image candidate having the rotation angle corresponding to the operation amount as the second image.

[0034] According to this structure, a second image candidate with a rotation angle corresponding to the operation amount of the viewpoint switching operation can be displayed as a second image on the display. In this way, the user can intuitively switch the image displayed as the second image.

[0035] (5) In any of the information processing methods described in (1) to (4) above, the information processing method may further include: obtaining a first display size of the given object in the first image, and displaying the second image on the display comprising: setting the second display size of the given object in the second image to the same size as the first display size by enlarging or shrinking the given object in the second image; and displaying the second image containing the given object set to the same size on the display.

[0036] According to this structure, the second display size and the first display size are set to the same size. In this way, a given object can be displayed on the monitor at a fixed display size.

[0037] (6) In any of the information processing methods described in (1) to (5) above, the information processing method may further include: displaying a bird's-eye view of the focus location icon representing the shooting location of the second image among the plurality of shooting location icons on the display.

[0038] According to this structure, since the location of interest icon, which indicates the location where the second image was taken, is highlighted on the bird's-eye view, users can intuitively understand the location where the second image was taken.

[0039] (7) In the information processing method described in (6) above, the information processing method may also include setting the color of the frame surrounding the second image and the color of the frame surrounding the attention location icon to the same color.

[0040] According to this structure, since the color of the frame surrounding the second image and the color of the frame surrounding the attention location icon are set to the same color, users can more intuitively understand the shooting location of the second image.

[0041] (8) In any of the information processing methods described in (1) to (7) above, the detection of the selected object may include: detecting objects contained in the first image; highlighting the object region of the detected object on the display; and accepting the selection of the object region.

[0042] According to this structure, the area in the first image where the object is drawn is highlighted. Furthermore, the user can select the object by selecting this highlighted area. In this way, the user can easily select objects.

[0043] (9) In any of the information processing methods described in (1) to (8) above, detecting the selection of the given object may also include: displaying a bird's-eye view with selectable blocks set on the display; detecting that the user has selected a given area in the selectable blocks; determining a shooting location icon corresponding to an image of the given area; determining a corresponding area from the images of the given area; and presuming the object contained in the corresponding area as the given object.

[0044] According to this structure, objects existing within the selectable blocks of the bird's-eye view are presumed to be the given objects selected by the user. Thus, the user can select a given object from the bird's-eye view. As a result, the user can select a given object more easily.

[0045] Furthermore, this disclosure can be implemented not only as an information processing method that performs the characteristic processes described above, but also as an information processing apparatus or the like that having a characteristic structure corresponding to the characteristic processes performed by the information processing method. It can also be implemented as a computer program that causes a computer to execute the characteristic processes included in such an information processing method. Therefore, the following other methods can also achieve the same effect as the information processing method described above.

[0046] (10) Another aspect of the information processing apparatus disclosed herein includes a processor that performs the following processing: displaying a bird's-eye view on a display; displaying shooting location icons representing shooting locations in the bird's-eye view; detecting the selection of a specific icon among the plurality of shooting location icons; displaying a first image corresponding to the specific icon on the display; detecting the selection of a given object contained in the first image; and, based on the selection of the given object, displaying a second image on the display containing a second image of the given object taken from a shooting location different from the shooting location of the first image, and the second image being taken at a time different from the shooting time of the first image.

[0047] (11) The information processing program involved in another aspect of this disclosure causes a computer to perform the following processing: displaying a bird's-eye view on a display; displaying shooting location icons representing shooting locations in the bird's-eye view; detecting the selection of a specific icon among the plurality of shooting location icons; displaying a first image corresponding to the specific icon on the display; detecting the selection of a given object contained in the first image; and, based on the selection of the given object, displaying a second image containing the given object taken from a shooting location different from the shooting location of the first image, and the second image taken at a time different from the shooting time of the first image on the display.

[0048] This disclosure can also be implemented as an information processing system operating according to such an information processing program. Furthermore, it is of course possible to distribute such a program via a non-transitory recording medium such as a computer-readable CD-ROM or a communication network such as the Internet.

[0049] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Furthermore, the embodiments described below represent specific examples of the present disclosure. The numerical values, shapes, constituent elements, steps, and order of steps shown in the following embodiments are examples and are not intended to limit the present disclosure. Additionally, any constituent elements in the following embodiments that are not described in the independent claim representing the highest-level concept will be described as arbitrary constituent elements. Furthermore, various elements can be combined in all embodiments.

[0050] (Implementation Method 1)

[0051] Figure 1 This is a diagram showing the overall structure of the information processing system 1 in this embodiment.

[0052] Information processing system 1 is a system that, while displaying a first image IM1 of a given object, displays a second image IM2 of the given object captured from another viewpoint when a viewpoint switching operation is received. Information processing system 1 includes a server 10, an information terminal 20, and a capturing device 30.

[0053] Server 10 is an example of an information processing device and a computer. Server 10, information terminal 20, and imaging device 30 are communicatively connected to each other via a network NT. An example of network NT is the Internet. Server 10 is, for example, a cloud server composed of one or more computers. However, this is just one example; server 10 can also be composed of an edge server or installed on information terminal 20. The way server 10 is installed on information terminal 20 is an example of the way information terminal 20 is composed of an information processing device.

[0054] Information terminal 20 is a user-operated terminal. The user, for example, is a manager of a given space captured by camera device 30. The given space is, for example, a construction site. However, this is just one example; the given space could also be a construction site, factory, shop, or office space, etc. Information terminal 20 can be, for example, a portable computer such as a smartphone or tablet, or a fixed-position computer. Figure 1 The example diagram shows one information terminal 20, but multiple information terminals can be connected to the server 10 via a network NT. The information terminal 20 includes: a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0055] The communication unit 21 is the communication interface that connects the information terminal 20 to the network NT. The communication unit 21 sends signals representing various instructions received by the operation unit 24 from the user to the server 10.

[0056] The processor 22, for example, is a central processing unit (CPU) that enables the display 23 to show the design drawing of a given space sent from the server 10.

[0057] The display 23 is composed of various display devices such as liquid crystal displays and organic EL displays, and displays various display screens under the control of the processor 22. In this embodiment, the display 23 displays a display screen of a design drawing of a given space.

[0058] The operation unit 24 may consist of, for example, a keyboard, a touch panel, and a mouse. The user inputs various instructions by operating the operation unit 24.

[0059] The shooting device 30 is, for example, a wide-angle camera that captures images at a given frame rate. A wide-angle camera, also known as a 360-degree camera, is a camera capable of capturing images from all 360 degrees. The shooting device 30 is, for example, a portable shooting device held by a person. The person being shot is, for example, a construction worker or site supervisor. Alternatively, the shooting device 30 can also be a conventional camera. The shooting device 30 has a communication unit 40. The communication unit 40 communicatively connects the information terminal 20 and the shooting device 30 via a near-field wireless communication path such as Bluetooth (registered trademark) or wireless LAN.

[0060] The photographer uses the camera device 30 to photograph the construction site and moves within a given space. When the photography ends, the photography information, including a series of images captured by the camera device 30, is transmitted via the communication unit 40 to the information terminal 20 and temporarily stored there. The photography information stored in the information terminal 20 is then uploaded to the server 10 by the photographer operating the information terminal 20.

[0061] The shooting information includes: the image captured by the shooting device 30, the shooting start location, the shooting end location, and the design drawing ID corresponding to the given space. The design drawing ID is an identifier used to identify the design drawing of the given space. The shooting start location and shooting end location are attached by the information terminal 20.

[0062] At the starting point of the shooting, the photographer aligns the shooting direction of the shooting device 30 with the given direction, presses the shooting button on the shooting device 30 to begin the shooting action, and presses the shooting button again when the shooting ends to end the shooting action. The given direction is used to align the direction with the design shown on the display 23. Figure 1 The direction to which it is directed. The given direction is, for example, north, but it is not limited to this.

[0063] The start and end points of the shooting are determined by the photographer inputting an instruction specifying their location on the display screen of the design drawing of a given space shown on the monitor 23 of the information terminal 20. The design drawing specifies a two-dimensional coordinate axis. Therefore, the start and end points of the shooting are each defined by two-dimensional coordinate values. The start and end points of the shooting are used by the server 10 to determine the location of each shooting location and the shooting direction of each image.

[0064] Server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 may be composed of, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 11 includes an acquisition unit 111 and a display control unit 113. The acquisition unit 111 and the display control unit 113 may be implemented by the processor 11 executing information processing programs, or they may be constructed by dedicated hardware circuits such as ASICs.

[0065] The acquisition unit 111 uses the communication unit 13 to acquire shooting information sent from the shooting device 30. The acquisition unit 111 acquires display instructions from the information terminal 20 by communicating with it. The display instructions are instructions input by the user into the information terminal 20, commanding the design drawing to be displayed on the display 23. Additionally, the acquisition unit 111 acquires selection instructions for shooting location icons by communicating with the information terminal 20. The selection instructions indicate that the user has selected any one of the shooting location icons displayed in the design drawing.

[0066] The shooting location calculation unit 112 uses the shooting information acquired by the acquisition unit 111 to acquire the shooting locations of each of the multiple images captured by the shooting device 30. The shooting location calculation unit 112 calculates the shooting location of each image using self-position estimation processing. As self-position estimation processing, VSLAM (Visual Simultaneous Localization and Mapping) can be employed. For example, the shooting location calculation unit 112 detects feature points from the multiple images included in the shooting information through image processing. Feature points are characteristic points representing the outlines of objects contained in the image. The shooting location calculation unit 112 performs self-position estimation processing on each of the multiple images using at least two images from which feature points are set, calculating the relative coordinate values ​​of the shooting location corresponding to each image. Using these relative coordinate values, the shooting location calculation unit 112 creates a SLAM analytical map. The SLAM analytical map is a virtual map generated by mapping the shooting locations of each image onto a two-dimensional coordinate system.

[0067] Furthermore, the shooting location calculation unit 112 estimates a series of shooting locations on the design drawing, based on the coordinates of the shooting start and ending points on the design drawing, and multiple images. It then creates multiple shooting location icons representing the shooting positions on the design drawing for the series of shooting locations, and overlaps these icons with the design drawing.

[0068] The display control unit 113 receives the display instruction from the acquisition unit 111 and displays the design drawing on the display screen 23 of the information terminal 20. The display control unit 113 also displays images captured at the locations indicated by the shooting location icons on the display screen 23 of the information terminal 20, based on the selection instructions from the shooting location icons.

[0069] The design drawing is an example of a bird's-eye view. The display instructions of the design drawing include a design drawing ID. The display control unit 113 only needs to read the design drawing represented by the design drawing ID from the design drawing information storage unit 121 and display the read design drawing on the display screen 23.

[0070] A design drawing is a bird's-eye view of a building, including its columns, walls, or materials. It represents the design of a construction site and can be a top view, blueprint, map, or perspective view of the site. A bird's-eye view can also be called an overview view; it can be a view seen from above or from a high vantage point.

[0071] Figure 2 This is a diagram of design drawing 201, which is an example of a design drawing displayed on display 23. For example... Figure 2 As shown in the design drawing 201, multiple shooting location icons 200 are displayed. Specifically, the multiple shooting location icons 200 include shooting location icons 211 to 213. Each of the shooting location icons 211 to 213 is associated with an image taken at that shooting location.

[0072] When the display control unit 113 detects a specific icon among the multiple shooting location icons 200 selected, it displays a first image IM1 corresponding to that specific icon on the display 23.

[0073] Furthermore, the display control unit 113 detects that the user has selected a given object contained in the first image IM1. Based on this detection, the display control unit 113 displays a second image IM2 containing the given object captured from a different location than the location where the first image IM1 was captured, and captured at a different time than the time when the first image IM1 was captured, on the display 23. Details will be described later.

[0074] The memory 12 is composed of a non-volatile, rewritable storage device such as a hard disk drive or a solid-state drive. The memory 12 includes a design drawing information storage unit 121 and an image information storage unit 122. The design drawing information storage unit 121 stores design drawing information. The design drawing information is an image representing a design drawing of a given space. A design drawing ID is established to correspond to the design drawing information, allowing for identification of the design drawing.

[0075] The image information storage unit 122 stores image information containing the shooting information sent from the shooting device 30. In addition to the multiple images, shooting start location, shooting end location, and design drawing ID contained in the shooting information sent from the shooting device 30, the image information also includes metadata for each image. Image information is generated each time the aforementioned shooting information is sent. The metadata includes: shooting ID, shooting period ID, shooting time, shooting direction, and shooting location.

[0076] The shooting ID is an identifier for the shooting location. The shooting period ID is an identifier for the shooting period. The shooting period refers to the time from the start to the end of the shooting action. The shooting time is the date and time the image was captured. The shooting direction is the shooting direction of each image in the shooting device 30. The shooting location is composed of the position (two-dimensional coordinate values) representing the shooting location of the image on the SLAM analysis map.

[0077] The communication unit 13 is the communication interface that connects the server 10 to the network.

[0078] Next, the processing of information processing system 1 will be explained. Figure 3 This is a flowchart illustrating an example of the processing flow performed in information processing system 1. For example, design drawing 201 displayed by the user on display 23 ( Figure 2 In the process, any shooting location icon was selected as the trigger to begin. Figure 3 The process shown in step S1. Furthermore, as in... Figure 3 Before the start of step S1, the preprocessing is performed, and the display control unit 113 performs the SLAM parsing process described above. Hereinafter, the processing of the information processing system 1 will be explained using the case where the user selects the shooting location icon 211 from multiple shooting location icons 200 as a specific icon.

[0079] In step S1, the display control unit 113 displays the first image IM1 on the display 23. Specifically, the display control unit 113 displays the first image IM1 corresponding to the specific icon 211 selected by the user on the display 23.

[0080] Furthermore, the first image IM1 is an image captured using a 360-degree camera. Due to the nature of images captured by a 360-degree camera, it is difficult to display the entire image on the display 23 at once. Therefore, when displaying the first image IM1 on the display 23, the display control unit 113 extracts a portion of the entire first image IM1 and displays that portion on the display 23. Hereinafter, the portion of the entire first image IM1 that is initially displayed on the display 23 when the first image IM1 is displayed is referred to as the first display area.

[0081] In step S2, the display control unit 113 detects and selects a given object contained in the first image IM1. First, in step S2, the user searches for the given object within the first display area. If the given object is not present in the first display area, the user inputs a display area change operation. A display area change operation is an operation used to make the display 23 display an area in the first image IM1 that is different from the first display area. Hereinafter, the area that is a part of the entire first image IM1 and is displayed on the display 23 by inputting a display area change operation is referred to as the second display area. The display area change operation is implemented, for example, by a sliding operation. That is, the user moves their finger in a given direction while pressing their finger against the display 23. As a result, the viewpoint of the first image IM1 rotates, and the second display area is displayed on the display 23. Furthermore, the sliding operation is only one example; the display area change operation can also be implemented, for example, by dragging the mouse.

[0082] Figure 5 This is an example diagram showing the second display area of ​​the first image IM1. (See diagram for example.) Figure 5 As shown, column 301 is included in the second display area. The following example illustrates how a user can select column 301 as a given object. The user selects the object by dragging the mouse cursor between, for example, two points containing the vertex of column 301, or by clicking on a portion of column 301.

[0083] In step S3, the display control unit 113 determines whether an image was captured in the vicinity of the capture location of the first image IM1. For example, the display control unit 113 determines that... Figure 2 In the design diagram 201 shown, at least one shooting location icon 200 exists within a given distance from a specific icon 211. If at least one shooting location icon 200 exists within the given distance (Yes in step S3), the process proceeds to step S4. On the other hand, if no shooting location icon 200 exists within the given distance (No in step S3), the process ends.

[0084] Furthermore, the imaging device 30 according to this embodiment captures images at a given frame rate (e.g., 30 frames per second). A shooting location icon 200 is generated for each of these images. When all the shooting location icons 200 are overlaid on the design drawing 201, a large number of shooting location icons 200 are arranged in the design drawing 201, degrading the visual recognizability of the design drawing 201. Therefore, even if the shooting location is estimated for a portion of the multiple images captured by the imaging device 30 through the above-described SLAM processing, a shooting location icon 200 indicating the shooting location is not generated. Hereinafter, such images are referred to as interval extraction images. The interval extraction images and information indicating the shooting locations of the interval extraction images are stored, for example, in a given storage area of ​​the image information storage unit 122. In step S3, in addition to the determination process described above, the display control unit 113 can also determine whether there is a shooting location for the interval extraction image within a given distance from a specific icon 211. Furthermore, if there is a shooting location for the interval extraction image within the given distance, the display control unit 113 can also determine that it is in step S3.

[0085] In step S4, the display control unit 113 determines whether the SLAM parsing process has been completed. Specifically, the display control unit 113 determines whether the shooting location calculation unit 112 has completed the SLAM parsing process using the first image IM1 and the images captured around the first image IM1. If SLAM parsing process has been performed (yes in step S4), the process proceeds to step S5. On the other hand, if SLAM parsing process has not been performed (no in step S4), the process ends.

[0086] In step S5, the display control unit 113 determines whether the position of a given object on the SLAM analytical map can be clarified. Figure 6 This is a diagram used to explain the processing in step S5, and it schematically shows the SLAM analysis diagram 500 generated by the display control unit 113. For example... Figure 6 As shown, the SLAM resolution map 500 maps to a first shooting location 501, a second shooting location 502, a third shooting location 503, and a fourth shooting location 504. The first shooting location 501 represents the shooting location of the first image IM1 on the SLAM resolution map 500. The second to fourth shooting locations 502 represent the positions of images captured around the first shooting location 501 on the SLAM resolution map 500. Hereinafter, the image captured at the second shooting location 502 will be referred to as image A, the image captured at the third shooting location 503 as image B, and the image captured at the fourth shooting location 504 as image C. The first shooting location 501 and... Figure 2The specific icon 211 on the design drawing 201 corresponds to the second shooting location 502, which corresponds to the shooting location icon 212 on the design drawing 201. The fourth shooting location 504 corresponds to the shooting location icon 213 on the design drawing 201. Furthermore, the B image captured at the third shooting location 503 is an interval-extracted image, and the shooting location icon representing the shooting location of the B image is not displayed in the design drawing 201.

[0087] In step S5, the display control unit 113 estimates the position of the given object (pillar 301) on the SLAM resolution map 500 based on the amount of the display area change operation performed by the user in step S2. For example, if the user performs a sliding operation to change the display area in step S2 (referred to as the first sliding operation), the position of the pillar 301 on the SLAM resolution map 500 is estimated based on the amount of sliding in the first sliding operation.

[0088] First, the display control unit 113 uses the position of the first shooting location 501 on the SLAM analysis map, the reference direction D preset in the first image IM1, and the left and right viewing angles of the first display area of ​​the first image IM1 to determine the field of view of the first display area, i.e., the first field of view 401, on the SLAM analysis map 500. Figure 6 Furthermore, the reference direction D is, for example, the direction corresponding to the frontal direction of the shooting device 30. Alternatively, the reference direction D can be arbitrarily set by the operator. The first field of view 401 is a fan-shaped area extending from the first shooting location 501 along the reference direction D, and the field of view angle is defined by the left-right viewing angle of the first display area.

[0089] Next, the display control unit 113 determines the second field of view 402 on the SLAM analysis map 500 based on the sliding amount of the first sliding operation input by the user in step S2. Figure 6 For example, suppose in step S2, the user inputs a first sliding operation that rotates the viewpoint 90 degrees from the reference direction D. The result, suppose, shows... Figure 5 The second display area of ​​the first image IM1 shown. In this case, the display control unit 113 determines a fan-shaped area extending in a direction rotated 90 degrees relative to the reference direction D on the SLAM analysis map 500 as the second field of view 402. The second field of view 402 is the area whose field of view angle is defined by the left and right viewing angle of the second display area.

[0090] Next, the display control unit 113 performs given image processing on the second display area of ​​the first image IM1 to estimate the position of the column 301 within the second field of view 402. Then, the position of the column 301 is mapped onto the SLAM analysis map 500.

[0091] If the display control unit 113 successfully maps the position of column 301 onto the SLAM resolution map, it determines "yes" in step S5. Then, the process proceeds to step S6. Conversely, if the position of column 301 is not successfully mapped onto the SLAM resolution map, it determines "no" in step S5. In this case, the process ends.

[0092] In step S6, the display control unit 113 determines the positional relationship based on the position of the given object (pillar 301) on the SLAM analysis map 500. Figure 7 This is a diagram used to explain the processing in step S6, and it schematically shows the SLAM analysis diagram 500 generated by the display control unit 113.

[0093] First, the display control unit 113 determines the positional relationship between the first shooting location 501 and the pillar 301 on the SLAM analysis map 500. Specifically, the display control unit 113 sets the position of the first vector V1 (V1) extending from the first shooting location 501 towards the pillar 301 as the base point. Figure 7 The first vector V1 is set on the SLAM analysis diagram 500. The display control unit 113 then acquires the magnitude and orientation of this first vector V1.

[0094] Next, the display control unit 113 determines the position of the column 301 on the SLAM analysis map 500 and its positional relationship with the respective positions of the second shooting location 502 to the fourth shooting location 504. Specifically, the display control unit 113 sets a second vector V2 extending from the second shooting location 502 towards the column 301 on the SLAM analysis map 500. Furthermore, it acquires the magnitude and orientation of this second vector V2. Similarly, the display control unit 113 acquires the magnitude and orientation of a third vector V3 extending from the third shooting location 503 towards the column 301. Additionally, the display control unit 113 acquires the magnitude and orientation of a fourth vector V4 extending from the fourth shooting location 504 towards the column 301.

[0095] Next, the display control unit 113 calculates the first rotation angle A1 between the reference direction D set in the image A captured at the second shooting location 502 and the second vector V2. Figure 7 For example, the display control unit 113 calculates the first rotation angle A1 as 120 degrees. Additionally, the display control unit 113 calculates the second rotation angle A2 formed by the reference direction D set in the B image captured at the third shooting location 503 and the third vector V3. For example, the display control unit 113 calculates the second rotation angle A2 as 135 degrees. Furthermore, the display control unit 113 calculates the third rotation angle A3 formed by the reference direction D set in the C image captured at the fourth shooting location 504 and the fourth vector V4. For example, the display control unit 113 calculates the third rotation angle A3 as 160 degrees.

[0096] In step S7, the display control unit 113 normalizes the size of the first vector V1 and obtains the first display size of the column 301 captured in the first image IM1. The display control unit 113 stores the normalized size of the first vector V1 and the first display size in a given storage area of ​​the memory 12. The first display size is defined by the proportion of the column 301 captured in the second display area of ​​the first image IM1 relative to the overall screen of the display 23.

[0097] In step S8, the display control unit 113 displays a user interface (hereinafter referred to as the viewpoint switching UI) on the display 23, which allows switching of viewpoints based on a given object. For example, the display control unit 113 displays a button on the display 23 containing text such as "viewpoint switching mode". When the button is detected to have been pressed (selected) by the user, the display control unit 113 displays the initial screen of the viewpoint switching UI on the display 23. For example, the display control unit 113 displays the second display area of ​​the first image IM1 as the initial screen on the display 23. When the initial screen of the viewpoint switching UI is displayed, the user inputs a viewpoint switching operation on the first image IM1 to switch viewpoints. Viewpoint switching refers to displaying a second image IM2 containing a given object taken from a different shooting location than the shooting location where the first image IM1 was taken on the display 23. In other words, it means displaying the column 301 taken from another angle on the display 23. The viewpoint switching operation is implemented, for example, by a sliding operation. However, this is just one example; the viewpoint switching operation can also be implemented, for example, by a dragging operation. Hereinafter, the swipe operation performed by the user to switch viewpoints will be referred to as the second swipe operation.

[0098] In step S9, the display control unit 113 determines whether the user has performed an operation intended to switch the viewpoint of a given object (pillar 301). That is, it determines whether the user has performed a second swipe operation on the first image IM1. If the input of the second swipe operation is detected (yes in step S9), the process proceeds to step S10. If the input of the second swipe operation is not detected (no in step S9), the process returns to step S8.

[0099] In step S10, the display control unit 113 acquires a second operation quantity representing the operation quantity of the viewpoint switching operation (second swipe operation). In other words, the display control unit 113 grasps the extent to which the user moved their finger during the second swipe operation.

[0100] In step S11, the display control unit 113 selects the second image IM2 based on the operation amount of the viewpoint switching operation on the first image IM1. Specifically, the display control unit 113 selects the second image IM2 from the second image candidates based on the second operation amount.

[0101] The second image candidate consists of images captured from a shooting location within a given distance of the first shooting location 501. The display control unit 113 selects images with a higher probability of capturing the column 301 from a shooting location different from the first image as the second image candidate. Specifically, the display control unit 113 selects the second image candidate based on the shooting location of each image, its positional relationship with the location where the column 301 exists, and the shooting direction of each image. That is, images captured around the first image IM1 and in the direction where the column 301 exists are selected as the second image candidate. In this embodiment, since a 360-degree camera is used for shooting, images captured around the shooting location of the first image IM1 have a higher probability of capturing the column 301. Therefore, the display control unit 113 of this embodiment selects images captured within a given distance of the first shooting location 501 as the second image candidate. For example, image A taken at the second shooting location 502, image B taken at the third shooting location 503, and image C taken at the fourth shooting location 504 can be selected as candidates for the second image. However, this is just one example; more or fewer candidates for the second image can also be selected. For example, there can be only one candidate for the second image.

[0102] In step S11, the display control unit 113 selects a second image candidate with a rotation angle corresponding to the second operation amount as the second image IM2. As an example, consider a case where the second operation amount is small, i.e., when the user moves their finger slightly during the second swipe operation. In this case, the display control unit 113 selects the image captured from the shooting location with the smallest rotation angle as the second image IM2. As described above, in Figure 7 Of the first rotation angle A1, the second rotation angle A2, and the third rotation angle A3 shown, the first rotation angle A1 (120 degrees in this example) is the smallest. Therefore, the display control unit 113 selects the image A captured from the second shooting location 502 as the second image IM2.

[0103] As another example, consider a case where the second operation amount is moderate, meaning the user moves their finger moderately during the second swipe operation. In this case, the display control unit 113 selects the image captured from the shooting location with the second smallest rotation angle as the second image IM2. Specifically, it selects the image captured from the third shooting location 503 (… Figure 7 The image B captured is used as the second image IM2.

[0104] As another example, in cases where the second operation involves a large amount of movement, i.e., when the user moves their finger significantly during the second swipe operation, the display control unit 113 selects the image captured from the shooting location with the largest rotation angle as the second image IM2. Specifically, it selects the image captured from the fourth shooting location 504 ( Figure 7 The image C captured is used as the second image IM2.

[0105] The display control unit 113 determines the magnitude of the second operation based solely on whether the user's finger movement exceeds a given threshold. For example, if the user's finger movement on the display 23 is less than 100 pixels, the display control unit 113 determines the second operation to be small. Conversely, if the user's finger movement on the display 23 is, for example, 100 pixels or more but less than 200 pixels, the display control unit 113 determines the second operation to be moderate. Furthermore, if the user's finger movement on the display 23 is, for example, 200 pixels or more, the display control unit 113 determines the second operation to be large. However, these values ​​are just examples, and the threshold can be appropriately changed depending on the size of the display 23, etc. Additionally, when there are more second image candidates, the second operation can be differentiated in more detail by setting more thresholds.

[0106] In step S12, the display control unit 113 sets the viewing angle of the second image IM2. As described above, since the second image IM2 is an image captured by a 360-degree camera, it is difficult to display the entire second image IM2 on the display 23 at once. Therefore, when displaying the second image IM2 on the display 23, the display control unit 113 extracts a portion of the entire second image IM2 and displays that portion of the image. Hereinafter, the portion of the entire second image IM2 that is initially displayed on the display 23 is referred to as the initial area. It is desirable to capture the column 301 in this initial area. If the column 301 is not captured in the initial area, the user is forced to perform a cumbersome task such as rotating the viewpoint of the second image IM2 to find the area where the column 301 exists in the second image IM2. In step S12, the viewing angle of the second image IM2 is set so that the column 301 is captured in the initial area. For example, suppose the display control unit 113 selects image A captured at the second shooting location 502 as the second image IM2. In this case, the display control unit 113 sets the center of the viewing angle of the second image IM2 in the direction extending from the second vector V2. That is, the viewing angle of the second image IM2 is set to widen the field of view in the direction where the pillar 301 is located. Similarly, when image B, captured from the second shooting location 502, is selected as the second image IM2, the viewing angle of the second image IM2 is set based on the direction extending from the third vector V3; when image C, captured from the third shooting location 503, is selected as the second image IM2, the viewing angle of the second image IM2 is set based on the direction extending from the fourth vector V4. Therefore, when the image displayed on the display 23 switches from the first image IM1 to the second image IM2, the second image IM2 that captures the pillar 301 can be displayed to the user. In other words, the second image IM2, whose viewpoint is directed towards the pillar 301, can be displayed to the user.

[0107] In step S13, the display control unit 113 determines whether the vectors are different in size. Specifically, it determines whether the first vector V1 extending from the first shooting location 501 toward the pillar 301 and the vector extending from the shooting location of the second image IM2 toward the direction where the pillar 301 exists are different in size. For example, suppose that image A captured at the second shooting location 502 is selected as the second image IM2. In this case, the display control unit 113 compares the size of the first vector V1 with the size of the second vector V2 and determines whether these vectors are different in size. If the vectors are different in size (yes in step S13), the process proceeds to step S14. If the vectors are the same in size (no in step S13), the process ends.

[0108] In step S14, the display control unit 113 aligns the display size (referred to as the second display size) of the column 301 captured in the second image IM2 with the first display size. The second display size is defined by the proportion of the column 301 contained in the second image IM2 relative to the overall screen of the display 23.

[0109] When the vector of the second image IM2 is larger than the vector of the first image IM1, that is, when the second image IM2 is an image of the column 301 taken from a shooting location farther than the shooting location of the first image IM1, the column 301 in the second image IM2 appears smaller compared to the first image IM1. In this case, the display control unit 113 enlarges the portion of the second image IM2 in which the column 301 is drawn, based on the first display size, thereby aligning the second display size with the first display size.

[0110] Furthermore, if the vector of the second image IM2 is smaller than the vector of the first image IM1, that is, if the second image IM2 is an image of the column 301 captured from a location closer to the capture location of the first image IM1, then the column 301 in the second image IM2 appears larger compared to the first image IM1. In this case, the display control unit 113 reduces the portion of the second image in which the column 301 is drawn, based on the first display size, thereby aligning the second display size with the first display size.

[0111] In other words, in step S14, the display control unit 113 sets the second display size to be the same as the first display size by enlarging or shrinking the pillars 301 in the second image IM2. Thus, the second image IM2, which includes pillars 301 set to the same size as the first display size, can be displayed on the monitor 23. That is, the pillars 301 can always be displayed on the monitor 23 at a fixed display size. Furthermore, if it is difficult to make the first display size and the second display size completely identical, the display control unit 113 can also make the first display size and the second display size approximately identical.

[0112] In step S15, the display control unit 113 displays the second image IM2 on the display 23. Figure 8 This is a diagram showing an example of the second image IM2. Figure 8 The image was captured at the second shooting location 502. In this embodiment, when the second operation amount of the second sliding operation is small, Figure 8 The second image IM2 shown is displayed on monitor 23. (See image IM2) Figure 8 As shown, in the second image IM2, a column 301 is captured from a different shooting location (angle) than in the first image IM1.

[0113] Figure 9This is another example of the second image IM2. Figure 9 The image was captured at the third shooting location 503. In this embodiment, when the second operation amount of the second sliding operation is of a moderate level, Figure 9 The second image IM2 shown is displayed on monitor 23. Figure 9 In the second image IM2 shown, the side 350 of the column 301 is captured. This side 350 is... Figure 3 The part not captured in the first image IM1 shown.

[0114] In this way, by confirming the second image IM2 of column 301, which was taken from a different shooting location than the first image IM1, the user can understand the shape and structure of column 301.

[0115] Furthermore, after displaying the second image IM2, the display control unit 113 can also accept viewpoint switching operations. For example, when displaying... Figure 8 If a viewpoint switching operation is received after the second image IM2 shown, it can also be... Figure 9 The second image IM2 shown is displayed on monitor 23.

[0116] According to the information processing system 1 described above, based on the selection of column 301 in the first image IM1, a second image IM2 containing column 301 captured from a different shooting location than the first image IM1 is displayed on the display 23. In this way, the user can easily confirm the second image IM2, thus eliminating the need to input the operation for finding the second image IM2 into the server 10, etc. As a result, the increased processing load on the server 10, etc., is suppressed, thereby reducing the burden on the server 10, etc.

[0117] In addition, based on the above structure, users can gain a more detailed understanding of the construction and structure of a given object.

[0118] Furthermore, in the information processing system 1, the viewing angle of the second image IM2 is pre-adjusted so that the column 301 is captured in the initial area of ​​the second image IM2. Thus, the user does not need to locate the column 301 from the second image IM2. As a result, the user can efficiently confirm the shape and structure of the column 301.

[0119] Furthermore, in information processing system 1, to ensure the ease of viewing and operation of design drawing 201, icons for the shooting locations of the interval-extracted images are not generated. On the other hand, if the interval-extracted image is an image taken around the first image IM1, the interval-extracted image is also used as a candidate for the second image. Thus, the design drawing 201 is made easy to view and operate, and the column 301 can be confirmed from different viewpoints.

[0120] The following variations are possible with respect to this disclosure.

[0121] (1) In step S3 of embodiment 1, if the display control unit 113 determines that there is a shooting location for interval image extraction within a distance given by the specific icon 211, it may also generate a new shooting location icon (hereinafter referred to as the interval image extraction icon) representing the shooting location for interval image extraction. Furthermore, the interval image extraction icon may be overlaid on the design drawing 201. Further, the display mode of the interval image extraction icon may be different from the display mode of the shooting location icons other than the interval image extraction icon. For example, the display color of the interval image extraction icon on the design drawing 201 may be different from the display color of the shooting location icons other than the interval image extraction icon.

[0122] (2) The display control unit 113 can also be linked with the display of the second image IM2 on the display 23 to change the display mode of the attention point icon on the design drawing 201 that indicates the shooting location of the second image IM2.

[0123] Figure 10 This diagram schematically illustrates the display screen 601 shown on the display 23 in this modified example. The display screen 601 includes a design drawing display area AR1 and a second image display area AR2. In the design drawing display area AR1, a design drawing 201 with multiple shooting location icons 200 superimposed is displayed. In the second image display area AR2, a second image IM2 is displayed.

[0124] For example, imagine that in Figure 10 When the second image IM2 (referred to as the second image IM21) captured at the location indicated by the shooting location icon 211 is displayed in the second image display area AR2, the display control unit 113 causes the design display area AR1 to display a red frame surrounding the outline of the shooting location icon 211. Furthermore, the display control unit 113 causes the second image display area AR2 to display a red frame surrounding the second image IM21. That is, the color of the frame surrounding the location icon and the color of the frame surrounding the second image IM2 are set to the same color. Therefore, the display control unit 113 emphasizes the shooting location icon 211 (an example of a location icon). In other words, the display method of the shooting location icon 211 is different from the display method of other shooting location icons. Furthermore, in Figure 10 In the middle, the red box is represented by a thick solid line.

[0125] As another example, imagine that... Figure 10When the second image IM2 (referred to as second image IM22) captured at the location indicated by the shooting location icon 212 is displayed in the second image display area AR2, the display control unit 113 displays a blue frame surrounding the shooting location icon 212 and a blue frame surrounding the second image IM22 on the display screen 601. Thus, the display control unit 113 emphasizes the shooting location icon 212. Figure 10 In the middle, the blue box is represented by a dashed line.

[0126] As yet another example, the idea is that... Figure 10 When the second image IM2 (referred to as the second image IM23) taken at the location indicated by the shooting location icon 220 is displayed in the second image display area AR2, the display control unit 113 causes the display screen 601 to display a green frame surrounding the shooting location icon 220 and a green frame surrounding the second image IM23. Thus, the display control unit 113 emphasizes the shooting location icon 220. Figure 10 In the image, the green box is represented by a single-dotted line. Additionally, the shooting location icon 220 is the image extraction icon for intervals.

[0127] As described above, in this modified example, the design drawing 201, which emphasizes the second image IM2 and the location of interest icon indicating the shooting location of the second image IM2, is displayed on the same screen of the display 23. This allows the user to intuitively understand the position of the second image IM2 on the design drawing 201 currently displayed on the display 23.

[0128] In particular, in this variation, the location icon is highlighted by setting the color of the box surrounding it to the same color as the box surrounding the second image IM2. This allows users to more intuitively understand the shooting location on the design drawing 201 of the second image IM2.

[0129] In addition, the display control unit 113 can also emphasize the location icon in various ways, such as making the location icon flash or making the radius of the location icon larger than the radius of the usual shooting location icon 200.

[0130] (3) In Implementation 1, an example of selecting a given object by specifying the outline of the given object by the user was described, but the selection of the object can also be achieved in the following ways.

[0131] In this modified example, the display control unit 113 performs object recognition processing and boundary recognition processing on the first image IM1 to detect objects contained in the first image IM1. Furthermore, the display control unit 113 emphasizes the areas where the objects are drawn and displays them on the display 23. Figure 11This is a diagram illustrating an example of the first image IM1 involved in this variation. For example, assuming that an object contained in the first image IM1 is detected... Figure 11 The column 301 and window 302 are shown. In this case, the display control unit 113 causes the display to show bounding boxes BB1 and BB2 around the area (an example of an object area) where the column 301 and window 302 are drawn. Alternatively, emphasis can be achieved by coloring the column 301 and window 302, increasing the brightness of the column 301 and window 302, etc.

[0132] Furthermore, the user selects any object region within the highlighted object region in the first image IM1. For example, the user moves the mouse pointer to the area in the first image IM1 where the column 301 is drawn, i.e., the area surrounded by the bounding box BB1, and clicks the mouse to select that area. The display control unit 113, upon receiving the above operation, detects that the given object has been selected. According to this modified example, the user can more easily select a given object.

[0133] (4) The display control unit 113 can also display the design drawing with the selectable block 203 set on the display 23. In addition, it can detect that the user has selected a given area in the selectable block 203 and determine the shooting location icon corresponding to the image of the given area. Furthermore, it can also determine the corresponding area from the image of the given area and presume the object contained in the corresponding area as the given object.

[0134] In this variation, the display control unit 113 first creates a bird's-eye view with selectable blocks 203 set. Figure 12 This diagram illustrates an example of the design drawing 202 (a bird's-eye view) involved in this variation. The display control unit 113 identifies the design drawing 202 and the positions of the multiple shooting location icons 200 displayed on the design drawing 202. Furthermore, the display control unit 113 sets the space where the shooting location icons 200 are arranged (referred to as the icon arrangement space) as a selectable block 203. That is, the display control unit 113 sets the block that is more likely to be captured by the photographer's shooting action as the selectable block 203. For example, in Figure 12 In the corridor C, room RM1, and room RM2 shown, shooting location icons 200 are arranged. These spaces are more likely to be captured by the photographer's shooting action. That is, objects in these blocks are more likely to be photographed. Therefore, the display control unit 113 sets these spaces as selectable blocks 203. In addition, the display control unit 113 also sets spaces that are connected to the icon arrangement spaces and can be treated as essentially the same spaces as the icon arrangement spaces as selectable blocks 203. For example, in Figure 12In the room RM3 shown, no shooting location icon 200 is configured. Here, it is assumed that information indicating the connection between room RM2 and room RM3 is stored in a given storage area of ​​memory 12. In this case, display control unit 113 sets room RM3 as selectable block 203. As described above, display control unit 113 creates design drawing 202 with selectable block 203 overlapping.

[0135] Next, the display control unit 113 displays the design drawing 202 to the user. At this time, the display control unit 113 emphasizes the selectable area 203 on the display 23. For example, the design drawing 202 with the selectable area 203 colored green is displayed on the display 23.

[0136] Next, the user selects any area from the selectable blocks 203. The information terminal 20, receiving this selection, sends the user's selection to the display control unit 113. Here, let's assume, for example, that the user selected... Figure 12 The region represented by point P.

[0137] Next, the display control unit 113, having obtained the user's selection, determines the object present at point P. First, the display control unit 113 determines a shooting location icon (called a corresponding icon) that corresponds to the image capturing the object present at point P. Here, the probability of capturing the object present at point P is higher in the image corresponding to the shooting location icon located closest to point P. Therefore, the display control unit 113 determines the shooting location icon located closest to point P as the corresponding icon. For example, the display control unit 113... Figure 12 The shooting location icon 250 is confirmed as the corresponding icon. At this time, the display control unit 113 sets the vector extending from the shooting location icon 250 toward point P.

[0138] Next, the display control unit 113 determines the object (called the corresponding object) existing at point P based on the image corresponding to the shooting location icon 250 (called the corresponding image). First, the display control unit 113 determines the relative positional relationship between the shooting location icon 250 and point P based on the aforementioned vector. For example, the display control unit 113 determines that point P exists at a position 100 pixels away from the shooting location icon 250 in the -X direction. Furthermore, the -X direction is the negative direction of the horizontal coordinate axis of design drawing 202. Next, the display control unit 113 analyzes the corresponding image to determine the region corresponding to the position 100 pixels away from the shooting location icon in the -X direction. Then, it determines the object drawn in that region (e.g., column 301). Finally, the display control unit 113 presumes the aforementioned object as the object selected by the user.

[0139] According to the structure of this variant, the object selected by the user from design drawing 202 is presumed to be the given object. Therefore, compared with the operation of opening the first image and performing a sliding operation to find the given object from the first image to select the given object, the user can select the given object more easily.

[0140] (5) In Embodiment 1, an example of performing the process of aligning the second display size with the first display size was described; however, this process is not mandatory. That is, Figure 3 Step S8 Figure 4 Steps S14 and S15 can be omitted. Whether to omit these steps can be preset by the user.

[0141] (6) In Embodiment 1, an example of selecting the second image based on the detection of a viewpoint switching operation was described, but the detection of a viewpoint switching operation is not necessary. The display control unit 113 may also select the second image IM2 based on the detection that the user has selected a given object.

[0142] (7) The display control unit 113 can also determine whether an object corresponding to column 301 has been captured in the second image candidate. For example, the display control unit 113 acquires a first feature quantity representing the shape, size, color, etc. of column 301 in the first image. Furthermore, it can use this first feature quantity to determine whether an object (the object corresponding to column 301) with a similarity of more than a given benchmark similarity to column 301 is included in the second image candidate. Furthermore, images that do not contain an object corresponding to column 301 can also be excluded from the second image candidate. In this way, it is possible to suppress inappropriate images from being included in the second image candidate. Specifically, images that do not capture column 301, images whose focus is not on column 301, etc., can be excluded from the second image candidate.

[0143] Industrial availability

[0144] The technology disclosed herein is useful in the field of displaying images in displays and the like.

Claims

1. An information processing method, which is an information processing method in a computer, comprising: Display a bird's-eye view on the monitor; The bird's-eye view displays multiple location icons indicating the shooting locations; The detector selected a specific icon from among the multiple shooting location icons; The first image corresponding to the specific icon is displayed on the monitor; The detector selected a given object contained in the first image; and Based on the selection of the given object, a second image of the given object, which is taken from a shooting location different from the shooting location of the first image, and taken at a time different from the shooting time of the first image, is displayed on the display.

2. The information processing method according to claim 1, wherein, The information processing method further includes: Select an image that corresponds to a shooting location icon that exists within a given distance from the specific icon as the second image candidate; and Select the second image from the second image candidate.

3. The information processing method according to claim 2, wherein, The information processing method further includes: The detection input indicated a viewpoint switching operation for the first image; and Obtain the operation amount of the viewpoint switching operation. The second image candidate includes at least two images. Displaying the second image on the display includes: selecting the second image from the second image candidates based on the operation amount.

4. The information processing method according to claim 3, wherein, Selecting the second image from the second image candidates includes: Determine the positional relationship between the shooting location of the second image candidate and the position of the given object; Based on the positional relationship, calculate the rotation angle between the reference direction set in the second image candidate and the direction extending towards the given object with the shooting location of the second image candidate as the base point; and The second image candidate with the rotation angle corresponding to the operation amount is selected as the second image.

5. The information processing method according to any one of claims 1 to 4, wherein, The information processing method further includes: obtaining a first display size of the given object in the first image. Displaying the second image on the monitor includes: By enlarging or shrinking the given object in the second image, the second display size of the given object in the second image is set to the same size as the first display size; and The second image, which contains the given object set to the same size, is displayed on the monitor.

6. The information processing method according to any one of claims 1 to 4, wherein, The information processing method further includes: displaying a bird's-eye view on the display of a focus location icon, which represents the shooting location of the second image, among the plurality of shooting location icons.

7. The information processing method according to claim 6, wherein, The information processing method further includes setting the color of the frame surrounding the second image and the color of the frame surrounding the attention location icon to the same color.

8. The information processing method according to any one of claims 1 to 4, wherein, The detector selected the given object, which includes: Detect objects contained in the first image; The object region of the detected object is highlighted and displayed on the display; and Accept the selection of the object region.

9. The information processing method according to any one of claims 1 to 4, wherein, The detector selected the given object, which includes: A bird's-eye view with selectable sections is displayed on the monitor; The system detects that the user has selected a given area within the selectable blocks; Establish corresponding shooting location icons for images of the given area; From the images captured of the given region, determine the corresponding region that corresponds to the given region; and The objects contained in the corresponding region are presumed to be the given objects.

10. An information processing apparatus, comprising a processor, The processor performs the following processing: Display a bird's-eye view on the monitor; The bird's-eye view shows location icons indicating the shooting locations; The detector selected a specific icon from among the multiple shooting location icons; The first image corresponding to the specific icon is displayed on the monitor; The detector selected a given object contained in the first image; and Based on the selection of the given object, a second image of the given object, which is taken from a shooting location different from the shooting location of the first image, and taken at a time different from the shooting time of the first image, is displayed on the display.

11. An information processing program that causes a computer to perform the following processing: Display a bird's-eye view on the monitor; The bird's-eye view shows location icons indicating the shooting locations; The detector selected a specific icon from among the multiple shooting location icons; The first image corresponding to the specific icon is displayed on the monitor; The detector selected a given object contained in the first image; and Based on the selection of the given object, a second image of the given object, which is taken from a shooting location different from the shooting location of the first image, and taken at a time different from the shooting time of the first image, is displayed on the display.

Images