Information processing method, information processing device, and information processing program

Abstract

An information processing method includes: displaying a first image taken at a first date and time at a given space on a display of an information terminal; displaying a second image taken at a second date and time at the given space on the display, the second date and time being different from the first date and time; in a state where the first image and the second image are displayed, in a case where selection of a photographing location is detected, switching the display of the first image to a third image taken at the selected photographing location, and switching the display of the second image to a fourth image taken at a corresponding photographing location, the selected photographing location being different from a photographing location at the first date and time or the second date and time, and the corresponding photographing location being a photographing location corresponding to the selected photographing location.

Description

Technical Field

[0001] This disclosure relates to techniques for displaying selected images. Background Technology

[0002] Patent Document 1 discloses an image information generation apparatus that, when input with information requesting a comparison of a first omnidirectional image and a second omnidirectional image, selects from all second omnidirectional images captured at a second shooting time the second omnidirectional image whose shooting position coordinates are closest to the shooting position coordinates of the first omnidirectional image, generates and outputs a composite image configured to be able to compare the second omnidirectional image and the first omnidirectional image.

[0003] However, Patent Document 1 does not disclose a process that directly transitions from a comparison display state comparing the first and second omnidirectional images to a comparison display state comparing the first and second omnidirectional images taken at different locations. Therefore, in order to achieve this in Patent Document 1, it is necessary to first return to the selection of the corresponding image in the first omnidirectional image and then display the comparison image, which increases the number of processing steps and the computer load.

[0004] Prior art literature

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2022-012444 Summary of the Invention

[0007] This disclosure was made to solve such a problem, and its purpose is to provide a technique that enables the following processing without increasing the number of processing steps: displaying multiple images taken at another shooting location while maintaining a comparative display of multiple images.

[0008] One aspect of this disclosure is an information processing method in a computer, comprising: displaying a first image taken at a given space on a display of an information terminal at a first date and time; displaying a second image taken at the given space on a second date and time, the second date and time being different from the first date and time; and, while displaying the first image and the second image, upon detecting the selection of a shooting location, switching the display of the first image to a third image taken at the selected shooting location, and switching the display of the second image to a fourth image taken at a corresponding shooting location, the selected shooting location being a shooting location different from the shooting location under the first date and time or the second date and time, the corresponding shooting location being a shooting location corresponding to the selected shooting location. Attached Figure Description

[0009] Figure 1A This is an overall structural diagram of the information processing system in Implementation Method 1.

[0010] Figure 1B This is an overall structural diagram of the information processing system in a variation of Implementation 1.

[0011] Figure 2 This is an example of a design drawing that maps to icons representing shooting locations.

[0012] Figure 3A This is an example of a display screen shown on a monitor when a shooting location icon has been selected.

[0013] Figure 3B This is a diagram showing an example of a display screen shown on the monitor when a shooting location icon is selected in a variation of Embodiment 1.

[0014] Figure 4 This is a diagram showing a comparison.

[0015] Figure 5 It shows the continuation Figure 4 The comparison screen that follows.

[0016] Figure 6 This is a diagram showing a comparison screen in the case where a BIM image is displayed as the nth comparison image.

[0017] Figure 7 This is a flowchart illustrating the processing of the information processing system in Implementation 1.

[0018] Figure 8 This is a diagram showing a comparison screen with annotation information.

[0019] Figure 9 It shows the continuation Figure 8 The comparison diagram that follows is shown next.

[0020] Figure 10 This is a flowchart illustrating the processing of the information processing system in Embodiment 2.

[0021] Figure 11 This is a flowchart illustrating the processing of the information processing system in Embodiment 3.

[0022] Figure 12 This is a flowchart illustrating the processing of the information processing system in Implementation 4.

[0023] Figure 13 This is a flowchart illustrating the processing of the information processing system in Implementation 5. Detailed Implementation

[0024] (The knowledge that forms the basis of this disclosure)

[0025] The following user interface is under development: It displays multiple camera location icons representing actual shooting locations on 2D design drawings of construction sites. If one camera location icon is selected, the image taken at that location is displayed. This allows construction site managers to monitor the construction site's status without physically visiting the site.

[0026] Furthermore, in such a user interface, the following function has been studied: to compare and display a first image taken at a shooting location indicated by a selected shooting location icon, and a second image taken at a shooting location close to the shooting location of the first image, which is a second image taken on a different date and time than the first image (e.g., Patent Document 1). This allows the supervisor to easily confirm the progress of the construction.

[0027] However, previously, the following processing was not performed: when comparing and displaying the first and second images, and when the first image was displayed in a walk-through mode, the second image was also displayed in a walk-through mode in conjunction with it. A walk-through display refers to a display method that sequentially switches between multiple images taken at different shooting locations based on user input. This allows the user to experience the feeling of actually walking on the shooting location, i.e., to have a first-person viewpoint.

[0028] In Patent Document 1, when the first and second images to be compared are switched to first and second images taken at a different location, the process involves handling the instruction to remove the comparison display and displaying a selection screen for the shooting location, handling the selection of the shooting location from the selection screen and displaying the first image, and handling the request for comparison and displaying the second image. This increases the number of processing steps and the computer's workload.

[0029] Therefore, the inventors of this invention came to this disclosure after gaining the following knowledge: if, in a state of comparing and displaying the first image and the second image, the second image is also displayed in a traversal manner in conjunction with the first image, then such a problem can be solved.

[0030] (1) An information processing method in one aspect of the present disclosure is an information processing method in a computer, comprising: displaying a first image obtained by taking a picture of a given space at a first date and time on a display of an information terminal; displaying a second image obtained by taking a picture of the given space at a second date and time on the display, the second date and time being a date and time different from the first date and time; and, while displaying the first image and the second image, when detecting the selection of a shooting location, switching the display of the first image to a third image taken at the selected shooting location, and switching the display of the second image to a fourth image taken at a corresponding shooting location, the selected shooting location being a shooting location different from the shooting location under the first date and time or the second date and time, the corresponding shooting location being a shooting location corresponding to the selected shooting location.

[0031] According to this configuration, while displaying the first and second images, upon detecting the selection of a shooting location, the first image is switched to a third image taken at the selected shooting location, and the second image is switched to a fourth image taken at the corresponding shooting location. This allows for a sequential display of two images taken at different dates and times. Consequently, it enables the display of multiple images taken at different shooting locations without increasing the processing steps.

[0032] (2) In the information processing method described in (1) above, the first image, the second image, the third image and the fourth image may each be composed of an all-around image.

[0033] In this case, the condition of a given space can be confirmed in more detail.

[0034] (3) In the information processing method described in (1) above, it is also possible that, in the image group including the first image and the third image and the image group including the second image and the fourth image, any one image group is composed of an all-round image, and any other image group is composed of a 3D simulation image of the given space or a 2D bird's-eye view image of the given space.

[0035] In this scenario, a walkthrough display is possible, comparing a 3D simulation image of a given space with a 2D bird's-eye view. Therefore, users can easily monitor the progress of construction within a given space. Furthermore, the 3D simulation image includes both 3D CAD images and BIM images.

[0036] (4) In any of the information processing methods described in (1) to (3) above, the information processing method may further include: displaying the first image and the second image on the display so that the shooting directions are consistent; and displaying the third image and the fourth image on the display so that the shooting directions are consistent.

[0037] In this case, it is possible to display two images taken at different dates and times simultaneously while maintaining a consistent viewpoint, thus making it easy to compare the two images.

[0038] (5) In any of the information processing methods described in (1) to (4) above, the information processing method may further include: displaying the first image and the second image on the display so that the viewing angles are consistent; and displaying the third image and the fourth image on the display so that the viewing angles are consistent.

[0039] In this case, it is possible to display two images taken at different dates and times simultaneously while maintaining a consistent perspective, thus making it easy to compare the two images.

[0040] (6) In any of the information processing methods described in (1) to (5) above, the information processing method may further include: if annotation information is included in either the first image or the second image, determining whether an annotation object corresponding to the annotation information is included in either the first image or the second image and at least one of the third image or the fourth image, and displaying the annotation information in the image determined to include the annotation object among the other image, the third image, and the fourth image.

[0041] In this case, if annotation information is included in either the first or second image, then as long as there is an annotation object corresponding to the annotation information in either the first or second image and at least one of the third or fourth images, the annotation information can be displayed in the image with the annotation object. Therefore, the identification of the annotation information becomes easy.

[0042] (7) In any of the information processing methods described in (1) to (6) above, the corresponding shooting location may be the shooting location with the shortest distance among shooting locations with a distance of less than a specified distance relative to the selected shooting location.

[0043] In this case, comparing two images taken at similar locations makes it easy to confirm changes in the conditions of a given space.

[0044] (8) In any of the information processing methods described in (1) to (7) above, the corresponding shooting location may be a shooting location whose shooting date and time are within a specified time relative to the selected shooting location.

[0045] In this case, comparing two images taken within a specified time period makes it easy to confirm changes in the condition of a given space within that specified time period.

[0046] (9) In any of the information processing methods described in (1) to (8) above, the given space may include multiple subspaces, and the information processing method may further include: determining whether the shooting location of the first image and the shooting location of the second image are located in the same subspace, and if they are determined to be located in the same subspace, displaying the second image; and determining whether the shooting location of the third image and the shooting location of the fourth image are located in the same subspace, and if they are determined to be located in the same subspace, displaying the fourth image. Here, a subspace means a space surrounded by columns or walls. For example, a subspace is a space in the same room or corridor as the given space, and in construction, it refers to a space where the view is not obstructed by columns or the like.

[0047] In this situation, comparing two images taken within the same subspace becomes easy. Without defined subspaces, it's possible to compare an image of a given space with images of subspaces different from that space, rather than comparing an image of that given space with images of spaces in the same room or corridor. This can lead to obstructed views by pillars or walls, making a direct comparison impossible, and potentially resulting in incorrect comparisons. Defining subspaces helps to prevent such errors.

[0048] (10) In any of the information processing methods described in (1) to (9) above, the third image may be an image taken one date or one time before or after the first date and time, the fourth image may be an image taken one date or one time before or after the second date and time, or an image taken at a shooting location close to the image taken one date or one time before or after the first date and time, and an image taken at a shooting time close to the second date and time.

[0049] In this case, it is possible to display the third and fourth images in a traversal manner, allowing movement along the actual shooting path taken by the photographer in a given space.

[0050] (11) In any of the information processing methods described in (1) to (10) above, the second image may be an image taken at a shooting location corresponding to the shooting location of the first image.

[0051] Based on this configuration, it is possible to prevent the display of a first image and a second image that do not correspond to the shooting location.

[0052] (12) In another aspect of this disclosure, the information processing apparatus includes a processor that performs the following processing: displaying a first image taken at a given space on a display of an information terminal at a first date and time; displaying a second image taken at the given space on a second date and time, the second date and time being different from the first date and time; and, while displaying the first image and the second image, upon detecting the selection of a shooting location, switching the display of the first image to a third image taken at the selected shooting location, and switching the display of the second image to a fourth image taken at a corresponding shooting location, the selected shooting location being a shooting location different from the shooting location under the first date and time or the second date and time, the corresponding shooting location being a shooting location corresponding to the selected shooting location.

[0053] In this case, an information processing apparatus is provided that can perform the following processing without increasing the number of processing steps: displaying multiple images captured at another shooting location while maintaining a comparative display of multiple images.

[0054] (13) In another embodiment of this disclosure, the information processing program causes a computer to perform the following processing: displaying a first image of a given space taken at a first date and time on a display of an information terminal; displaying a second image of the given space taken at a second date and time on the display, the second image being at a date and time different from the first date and time; and, while displaying the first image and the second image, upon detecting the selection of a shooting location, switching the display of the first image to a third image taken at the selected shooting location, and switching the display of the second image to a fourth image taken at a corresponding shooting location, the selected shooting location being a shooting location different from the shooting location under the first date and time or the second date and time, the corresponding shooting location being a shooting location corresponding to the selected shooting location.

[0055] In this case, an information processing program can be provided that performs the following processing without increasing the number of processing steps: displaying multiple images taken at another shooting location while maintaining a comparative display of multiple images.

[0056] This disclosure also enables the implementation of an information processing system that operates through such an information processing program. Furthermore, it goes without saying that such a computer program can be distributed via computer-readable non-transitory recording media such as CD-ROMs or communication networks such as the Internet.

[0057] Furthermore, the embodiments described below are all specific examples of this disclosure. The numerical values, shapes, constituent elements, steps, and order of steps shown in the following embodiments are examples and are not intended to limit this disclosure. In addition, constituent elements in the following embodiments that are not described in the independent claims representing the highest-level concept are described as arbitrary constituent elements. Moreover, the contents of all embodiments can be combined.

[0058] (Implementation Method 1)

[0059] Figure 1A This is an overall structural diagram of the information processing system 1 in Embodiment 1. The information processing system 1 is a system capable of traversing between two images captured at different times while maintaining a comparative display. The information processing system 1 includes a server 10, an information terminal 20, a capturing device 30, and a communication device 40. The server 10 (an example of an information processing device and a computer), the information terminal 20, and the communication device 40 are communicatively connected to each other via a network NT. An example of a network is the Internet. The server 10 is, for example, a cloud server composed of one or more computers. However, this is just one example; the server 10 can also be composed of an edge server or installed on the information terminal 20. The server 10 being installed on the information terminal 20 is an example of the information terminal 20 being composed of an information processing device.

[0060] Information terminal 20 is held by a user. The user is, for example, the manager of a given space captured by the camera device 30. The given space is, for example, a construction site. However, this is just one example; the given space could also be a construction site, factory, shop, office, etc. Information terminal 20 could also be a portable computer such as a smartphone or tablet, or it could be a fixed-type computer. Information terminal 20 displays images on a monitor according to control given by server 10. Figure 1A In the example, one information terminal 20 is shown, but multiple information terminals can also be connected to the server 10 via a network. The information terminal 20 includes a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0061] The communication unit 21 is the communication interface that connects the information terminal 20 to the network NT. The communication unit 21 sends instruction signals, representing various instructions received by the operation unit 24 from the user, to the server 10. The communication unit 21 receives display data from the server 10 for displaying various display screens.

[0062] The processor 22, for example, is a central processing unit, which displays the display screen represented by the display data received by the communication unit 21 on the display 23.

[0063] The display 23 is composed of various display devices such as liquid crystal displays and organic EL displays, and displays various display screens under the control of the processor 22.

[0064] The operation unit 24 consists of a keyboard, a touch panel, and a mouse, and is subject to various instructions input by the user.

[0065] The communication device 40, for example, is a portable information terminal such as a smartphone or tablet computer, which connects the imaging device 30 to the network NT. The communication device 40 and the imaging device 30 are connected via a short-range wireless communication line such as Bluetooth (registered trademark).

[0066] The shooting device 30 is, for example, a wide-angle camera that captures images at a given frame rate. A wide-angle camera, also known as a 360-degree camera, is a camera capable of acquiring images from all 360 degrees. The shooting device 30 can be, for example, a portable shooting device held by a person. The person being shot could be, for example, a construction worker or site supervisor. Alternatively, the shooting device 30 can also be a conventional camera.

[0067] The photographer moves within the construction site while using the camera device 30 to film it. Once filming is complete, the camera device 30 sends the filming information, including a series of captured images, to the server 10 via the communication device 40. The period from the start of filming to its end is called the filming period.

[0068] The shooting information includes the captured images and metadata. The metadata includes the start and end locations of the shooting.

[0069] The shooting date and time are obtained, for example, through a clock provided with the shooting device 30. Here, the shooting device 30 shoots images at a given frame rate, so the shooting location is defined in frame period units. However, this results in a large amount of data, so the shooting location can also be defined every given time interval (e.g., 1 second, 5 seconds, 10 seconds, 1 minute, etc.).

[0070] The photographer starts shooting by aligning the shooting device 30 with the shooting direction, for example, north, and presses the shooting button on the shooting device 30. When the shooting ends, the photographer presses the shooting button again to end the shooting.

[0071] The start and end points of the shooting are determined by the photographer's input on the site design drawing displayed on the monitor of the communication device 40, specifying their positions. A 2D coordinate axis is defined on the design drawing. Therefore, the start and end points of the shooting are each defined by 2D coordinate values. The start and end points are used by the server 10 to determine the location of each shooting point and the shooting direction of each image. The location of each shooting point can be determined based on the start and end points of the shooting and the image, using VSLAM (Visual Simultaneous Localization and Mapping). The location of each shooting point is represented by 2D coordinate values ​​on the design drawing. Furthermore, the shooting direction of each image can also be determined using an initial orientation setting and SLAM technology. The shooting direction of each image is represented, for example, by a 3D or 2D polar coordinate vector. Alternatively, the communication device 40 can calculate the location of each shooting point and the shooting direction of each image.

[0072] Server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 may be a central processing unit (CPU). The processor 11 includes an instruction receiving unit 111 and a display control unit 112. The instruction receiving unit 111 and the display control unit 112 may also be implemented by the processor 11 executing information processing programs, or they may be constructed by dedicated hardware circuits such as ASICs.

[0073] The instruction receiving unit 111 acquires instructions input to the information terminal 20 by the user. Specifically, the instruction receiving unit 111 detects the input of an instruction by the user when the communication unit 13 receives an instruction signal representing an instruction sent from the information terminal 20. Instructions include, for example, comparison instructions and walk-through instructions. A comparison instruction is an instruction to compare and display two images captured on-site at different dates and times. A walk-through instruction is an instruction to display the two images being compared in a walk-through manner.

[0074] The display control unit 112 displays a first image, captured on a given space at a first shooting date and time (an example of the first shooting date and time), on the display 23 of the information terminal 20. The display control unit 112 also displays a second image, captured on a given space at a second shooting date and time (an example of the second shooting date and time), which is different from the first shooting date and time, on the display 23. The first shooting date and time is a single shooting date and time included within the first shooting period. The first shooting period is the time from the start time of shooting to the end time of shooting within a single shooting period. The second shooting date and time is a single shooting date and time included within the second shooting period. The second shooting period is a shooting period different from the first shooting period.

[0075] The second image was taken at a location corresponding to the location where the first image was taken. A corresponding location, for example, is the location among the locations taken during the second shooting period that is closest to the location where the first image was taken.

[0076] While displaying a comparison between the first image and the second image, the display control unit 112 detects the selection of a shooting location, choosing a location different from the shooting location on the first or second shooting date and time. This selection is made, for example, through user input via a button press. Figure 4 The operation is performed using the walk-through indicator button 310 shown. Comparison display refers to... Figure 4 As shown, two images taken at different locations are displayed side by side.

[0077] When the selection is detected, the display control unit 112 switches the display of the first image to the third image captured at the selected shooting location, and switches the display of the second image to the fourth image captured at the corresponding shooting location, which is the shooting location corresponding to the selected shooting location.

[0078] The third image was taken at a date and time one day before or after the first shooting date and time. A shooting date and time preceding the first shooting date and time is one that falls within the first shooting period and is earlier than the first shooting date and time. A shooting date and time following the first shooting date and time is one that falls within the first shooting period and is more recent than the first shooting date and time. Shooting dates and times are expressed in year, month, day, hour, minute, and second.

[0079] The fourth image is an image taken on a date and time one day before or after the second shooting date and time. A shooting date and time one day before the second shooting date and time is one shooting date and time that falls within the second shooting period and is earlier than the second shooting date and time. A shooting date and time one day after the second shooting date and time is one shooting date and time that falls within the second shooting period and is later than the second shooting date and time. Alternatively, the fourth image may also be an image taken at a location close to an image taken on a date and time one day before or after the first date and time, and at a shooting time close to a date and time one day before or after the second date and time. "Close location" means the shooting location is within a given distance. "Close time" means the shooting date and time are within a given time period.

[0080] The display control unit 112 displays the first image and the second image on the display 23, aligning the shooting directions. The display control unit 112 also displays the third image and the fourth image on the display 23, aligning the shooting directions.

[0081] Here, the display control unit 112 determines the display area of ​​the first image mapped onto the spherical surface as rectangular, centered on a viewpoint. The viewpoint is the intersection of the omnidirectional image mapped onto the spherical surface and the shooting direction of the imaging device 30. Therefore, the display control unit 112 sets the same viewpoint for the second image mapped onto the spherical surface as the first image, and determines the rectangular display area of ​​the second image centered on this set viewpoint. Thus, the display control unit 112 aligns the shooting direction of the second image with the shooting direction of the third image. Similarly, the display control unit 112 aligns the shooting direction of the fourth image mapped onto the spherical surface with the shooting direction of the third image by setting the same viewpoint for the fourth image mapped onto the spherical surface as the third image, and by setting a rectangular display area centered on this viewpoint.

[0082] The display control unit 112 displays the first and second images on the monitor 23, ensuring consistent viewing angles. The display control unit 112 also displays the third and fourth images on the monitor 23, ensuring consistent viewing angles. Viewing angle refers to the display area of ​​each of the first to fourth images. Ensuring consistent viewing angles means making the width and height of the display area of ​​each of the first to fourth images consistent. This also applies to the second to fourth images.

[0083] The user can arbitrarily change the magnification of each of the first to fourth images displayed on the monitor 23. For example, if the magnification of the first image is changed, the display control unit 112 changes the display range of the first and second images corresponding to the changed magnification. In this way, the display control unit 112, in conjunction with the change in the magnification of the first image, also changes the magnification of the second image. The same applies to the third and fourth images.

[0084] Images 1, 2, 3, and 4 are each composed of omnidirectional images. However, this is just one example; it's also possible that in either the image group including images 1 and 3, or the image group including images 2 and 4, any one image group is composed of omnidirectional images, and any other image group is composed of a 3D simulation image of a given space or a 2D bird's-eye view image of a given space. An example of a 2D bird's-eye view image is a design drawing. A 3D simulation image is a 3D CAD image or BIM (Building Information Modeling) image rendered from 3D CAD information or BIM (Building Information Modeling) information. For example, in the case of a construction site, an image rendered from the BIM information of the completed building based on the shooting direction of the images being compared and displayed is used as a 3D simulation image.

[0085] The memory 12 is composed of a non-volatile, rewritable storage device such as a hard disk drive or a solid-state drive. The memory 12 includes a design information storage unit 121 and an image information storage unit 124. The design information storage unit 121 stores design information. Design information is image information representing a design drawing of a given space. A design ID is used to identify the design drawing and establish a correspondence between it and the design information. A design drawing is a drawing representing the design of a construction site, and can also be a top view, blueprint, map, or perspective view of the construction site. A bird's-eye view can also be called an overhead view; it can be a view viewed from above or from a high vantage point.

[0086] Image information storage unit 124 stores image information. Image information is information obtained by establishing a correspondence between multiple images captured at multiple shooting locations and the metadata of each image. Image information is generated whenever the aforementioned shooting information is sent. Meta-information includes shooting ID, shooting period ID, shooting date and time, shooting direction, shooting location, and design drawing ID. Shooting date and time refers to the date and time the image was captured. Shooting direction refers to the shooting direction of the shooting device 30 that captured the image. Shooting location is composed of the position (2D coordinate values) representing the location where the image was captured.

[0087] The Shooting ID is an identifier for the shooting location. The Shooting Period ID is an identifier for the shooting period. The Design Drawing ID is an identifier for the design drawing of a given space corresponding to the image.

[0088] The communication unit 13 is the communication interface that connects the server 10 to the network.

[0089] Figure 2 This is an example of a design drawing screen 200 that maps a shooting location icon 210. The design drawing screen 200 displays a design drawing of a given space. The shooting location icon 210 is overlaid on the design drawing screen 200.

[0090] The shooting location icon 210 is an icon representing the shooting location and corresponds to the image. In this example, the shooting location icon 210 consists of a circular image. The multiple shooting location icons 210 shown here correspond to multiple shooting locations belonging to a certain shooting period. The path from the front shooting location icon 210 to the back shooting location icon 210 represents the trajectory of the photographer during that shooting period. The movement direction D1 refers to the direction the photographer moved.

[0091] Trajectories 220 and 230 represent the photographer's trajectory during a shooting period different from the shooting period to which the shooting location icon 210 belongs. For example, if the shooting period to which the shooting location icon 210 belongs is the most recent shooting period, then trajectories 220 and 230 represent the trajectory during a past shooting period. Trajectories 220 and 230 are shown here for clarity, but they may not be displayed. Furthermore, the movement direction D1 may also be omitted.

[0092] When the instruction receiving unit 111 receives an instruction to select any one of the multiple shooting location icons 210 (e.g., shooting location icon 210n), the display control unit 112 displays the image Gn corresponding to the shooting location icon 210n on the display 23. Shooting location icon 210n+1 represents the shooting location Pn+1 whose shooting date and time is after image Gn, and shooting location icon 210n-1 represents the shooting location Pn-1 whose shooting date and time is before image Gn.

[0093] Figure 3A This diagram illustrates an example of a display screen 300 shown on the monitor 23 when the shooting location icon 210n is selected. The display screen 300 displays an image Gn captured at the shooting location Pn corresponding to the shooting location icon 210n. The display screen 300 includes a design display bar 301, an edit button 304, a date bar 306, a comparison indicator button 307, and a walk-through indicator button 310.

[0094] Design drawing display bar 301 will Figure 2The design drawing shown is displayed in a scaled-down view (200). The date field 306 displays the date and time the image Gn was captured. The comparison indicator button 307 is used to compare image Gn with an image captured at the location corresponding to the capture location Pn.

[0095] The walk-through indicator button 310 is used to switch image Gn to image Gn+1, which was captured one day after the previous image. Figure 2 In the example, the image captured at shooting location Pn+1 is equivalent to the next image Gn+1. Alternatively, the user can switch image Gn to image Gn-1 captured at shooting location Pn-1. In this case, the display control unit 112 displays the reverse-direction indicator button 310 on the display 23 according to the user's instruction; when an instruction to select the reverse-direction indicator button 310 is input, image Gn-1 is displayed.

[0096] In this way, by inputting an instruction on the selectable scrolling indicator button 310, the user can sequentially switch and display the images Gn captured during shooting on the display 23. Thus, the user can achieve scrolling display of images Gn.

[0097] Figure 4 This is a diagram showing the comparison screen 400. The comparison screen 400 is displayed on the display 23 when the instruction receiving unit 111 receives the selection instruction of the comparison instruction button 307.

[0098] The comparison screen 400 includes a first region 410 and a second region 420, which are divided into two parts, left and right. The first region 410 displays the nth reference image, and the second region 420 displays the nth comparison screen.

[0099] The so-called nth reference image is image Gn displayed on display screen 300 when the instruction of the selection walk-through indicator button 310 is input. The nth reference image is an image taken at the nth shooting location Pn among multiple shooting locations belonging to the first shooting period. The aforementioned first image is equivalent to the nth reference image.

[0100] The nth comparison image is the image taken at the shooting location Qn, which corresponds to the shooting location Pn. The second image mentioned above is equivalent to the nth comparison image.

[0101] The first area 410 displays the design display bar 301 in the lower left corner. The first area 410 includes a date bar 306 indicating the date and time of the capture of the nth reference image and a segmentation end button 308 in the upper left corner. When the segmentation end button 308 is pressed, the display control unit 112 returns the display screen 23 from the comparison screen 400 to the display screen 300. The second area 420 displays the date bar 306 indicating the date and time of the capture of the nth comparison image in the left corner. The second area 420 displays a viewpoint synchronization label 401 in the upper right corner indicating that the nth reference image and the nth comparison image are displayed synchronously. By setting this control to synchronization, the display control unit 112 can make the nth reference image and the nth comparison image move synchronously and simultaneously; by setting it to asynchronous, the nth reference image and the nth comparison image can move independently. Furthermore, by returning the setting from asynchronous to synchronous, the display control unit 112 can also be set from a state where the nth reference image and the nth comparison image move independently to a state where the nth reference image and the nth comparison image move synchronously and simultaneously.

[0102] The first area 410 displays walk-through indicator buttons 310 in the center.

[0103] If the user inputs a selection instruction on the walk-through indicator button 310 in the first area 410, the display of the nth reference image in the first area 410 will be switched to the (n+1)th reference image.

[0104] At this point, the display of the nth comparison image in the second region 420 is switched to the (n+1)th comparison image.

[0105] Figure 5 It shows the continuation Figure 4 The comparison screen that follows is a 400-pixel image. (Example) Figure 5 As shown, region 410 displays the (n+1)th reference image, and region 420 displays the (n+1)th comparison image. The (n+1)th reference image is an image taken at shooting location Pn+1. The (n+1)th reference image is equivalent to the third image mentioned above. The (n+1)th comparison image is an image taken at shooting location Qn+1, which corresponds to shooting location Pn+1. The (n+1)th comparison image is equivalent to the fourth image mentioned above.

[0106] Subsequently, whenever the walk-through indicator button 310 is pressed, the reference image and comparison image at the next shooting location are displayed on the monitor 23. Thus, walk-through display can be achieved while maintaining the comparison of two images.

[0107] In this example, the direction of the walk-through indicator button 310 is towards the movement direction D1. Therefore, when the walk-through indicator button 310 is pressed, the (n+1)th reference image and the (n+1)th comparison image are displayed on the display 23. On the other hand, when the walk-through indicator button 310, which is opposite to the movement direction D1, is pressed, the (n-1)th reference image and the (n-1)th comparison image are displayed on the display 23.

[0108] The nth reference image and the nth comparison image are omnidirectional images. Therefore, when the user inputs a change of shooting direction operation via the operation unit 24 for either the first region 410 or the second region 420, the display control unit 112 changes the shooting direction of the image displayed in one of the regions based on the operation amount and direction. At this time, the display control unit 112 also changes the shooting direction of the image displayed in the other region of the first region 410 or the second region 420 in conjunction with the change in the shooting direction of the first image. As a result, comparing the two images becomes easier.

[0109] Figure 7 This is a flowchart illustrating the processing of the information processing system 1 in Embodiment 1. The flowchart is designed to... Figure 2 The design diagram shown in screen 200 indicates that the shooting location icon 210n was selected as the trigger to start.

[0110] In step S1, the display control unit 112 determines the shooting location Pn based on the instruction received by the instruction receiving unit 111. For example... Figure 2 As shown, if the information terminal 20 receives a selection instruction for the shooting location icon 210n, it sends the selection instruction to the server 10. This selection instruction includes a shooting ID. The communication unit 13 receives this selection instruction, thereby instructing the receiving unit 111 to detect that the shooting location icon 210n has been selected. The display control unit 112 uses the shooting ID included in the selection instruction received by the receiving unit 111 as a key to determine the shooting location Pn.

[0111] Next, in step S2, the display control unit 112 displays the nth reference image captured at the shooting location Pn determined in step S1. Specifically, the display control unit 112 uses the communication unit 13 to send the display data of the nth reference image to the information terminal 20. In the information terminal 20, the processor 22 displays the display data received by the communication unit 21 on the display 23. Thus, the image is displayed on the display 23. Figure 3A The displayed screen is 300.

[0112] Next, in step S3, the instruction receiving unit 111 receives the comparison instruction. The comparison instruction is input by pressing the comparison instruction button 307 displayed on the display screen 300. The processor 22 uses the communication unit 21 to send the comparison instruction to the server 10. The comparison instruction sent from the information terminal 20 is received by the communication unit 13 and input to the instruction receiving unit 111. Thus, the instruction receiving unit 111 receives the comparison instruction.

[0113] Next, in step S4, the display control unit 112 selects the image captured at the shooting location Qn, which has the shortest distance to the shooting location Pn, as the nth comparison image. Specifically, the display control unit 112 determines from the image information storage unit 124 multiple shooting locations (multiple candidate shooting locations) belonging to a shooting period (the second shooting period) that is more recent than the shooting period to which the shooting location Pn belongs. The display control unit 112 then determines the candidate shooting location with the shortest distance to the shooting location Pn from among the determined multiple candidate shooting locations as the shooting location Qn. For each of the multiple candidate shooting locations stored in the image information storage unit 124, the display control unit 112 calculates the Euclidean distance to the shooting location Pn, and determines the candidate shooting location with the shortest Euclidean distance as the shooting location Qn.

[0114] Next, in step S5, the display control unit 112 determines whether the shooting directions of the nth reference image and the nth comparison image are different. If it is determined that the shooting directions are different (yes in step S5), the process proceeds to step S6; if it is determined that the shooting directions are not different (no in step S5), the process proceeds to step S7. The display control unit 112 may also determine "yes" in step S5 if the shooting direction of the nth comparison image is not within a given range relative to the shooting direction of the nth reference image.

[0115] Next, in step S6, the display control unit 112 makes the shooting direction of the nth comparison image consistent with the shooting direction of the nth comparison image.

[0116] Next, in step S7, the display control unit 112 determines whether the viewing angles of the nth reference image and the nth comparison image are different. If it is determined that the viewing angles are different (yes in step S7), the process proceeds to step S8; if it is determined that the viewing angles are not different (no in step S7), the process proceeds to step S9.

[0117] Next, in step S8, the display control unit 112 makes the viewing angle of the nth comparison image consistent with the viewing angle of the nth reference image.

[0118] Next, in step S9, the display control unit 112 compares and displays the nth reference image and the nth comparison image. Specifically, the display control unit 112 uses the communication unit 13 to... Figure 4The display data of the comparison screen 400 is sent to the information terminal 20. As a result, the comparison screen 400 is displayed on the monitor 23.

[0119] Next, in step S10, the receiving unit 111 is instructed to determine whether a walk-through instruction has been received. If the information terminal 20 is in... Figure 4 If the system receives a selection instruction from the walk-through instruction button 310, it will send the walk-through instruction to the server 10. If the communication unit 13 receives the walk-through instruction, the instruction receiving unit 111 will determine that it has received the walk-through instruction.

[0120] If it is determined that a walk-through instruction has been received (yes in step S10), the process proceeds to step S11; if it is determined that no walk-through instruction has been received (no in step S10), the process remains in standby mode in step S10.

[0121] Next, in step S11, the display control unit 112 selects the image captured at the shooting location Qn+1, which has the shortest distance to the shooting location Pn+1, as the (n+1)th comparison image. Specifically, the display control unit 112 determines from the image information storage unit 124 multiple shooting locations (multiple candidate shooting locations) belonging to a shooting period (the second shooting period) that is more recent than the shooting period to which the shooting location Pn+1 belongs. The display control unit 112 then determines the shooting location Qn+1 as the candidate shooting location among the multiple candidate shooting locations whose distance to the shooting location Pn is the shortest.

[0122] Next, in step S12, the display control unit 112 determines whether the shooting directions of the (n+1)th reference image and the (n+1)th comparison image are different. If it is determined that the shooting directions are different (yes in step S12), the process proceeds to step S13; if it is determined that the shooting directions are not different (no in step S12), the process proceeds to step S14. The display control unit 112 may also determine "yes" in step S12 if the shooting direction of the (n+1)th comparison image is not within a given range relative to the shooting direction of the (n+1)th reference image.

[0123] Next, in step S13, the display control unit 112 makes the shooting direction of the (n+1)th comparison image consistent with the shooting direction of the (n+1)th comparison image.

[0124] Next, in step S14, the display control unit 112 determines whether the viewing angles of the (n+1)th reference image and the (n+1)th comparison image are different. If it is determined that the viewing angles are different (yes in step S14), the process proceeds to step S15; if it is determined that the viewing angles are not different (no in step S14), the process proceeds to step S16.

[0125] Next, in step S15, the display control unit 112 makes the viewing angle of the (n+1)th comparison image consistent with the viewing angle of the (n+1)th reference image.

[0126] Next, in step S16, the display control unit 112 compares and displays the (n+1)th reference image and the (n+1)th comparison image. Specifically, the display control unit 112... Figure 4 The comparison screen 400 shown displays the (n+1)th reference image in its first area 410 and the (n+1)th comparison image in its second area 420. Thus, the display 23 shows... Figure 5 The comparison screen shown is 400. If the processing in step S16 is completed, the processing returns to step S10.

[0127] In this way, according to Embodiment 1, when a walkthrough indication is detected while the nth reference image and the nth comparison image are being displayed in comparison, the nth reference image is switched to the (n+1)th reference image, and the nth comparison image is switched to the (n+1)th comparison image. This allows for the walkthrough display of two images taken at different dates and times. As a result, it is possible to display multiple images taken at different locations while maintaining the comparison display of multiple images without increasing the number of processing steps.

[0128] (A variation of Implementation Method 1)

[0129] Figure 1B This is an overall structural diagram of the information processing system 1 in a variation of implementation method 1. Figure 1B In, with Figure 1A The differences are as follows: the memory 12 also has a BIM information storage unit 122 and an annotation information storage unit 125.

[0130] BIM information storage unit 122 stores BIM information for each of multiple given spaces. A correspondence is established between the design drawing ID and each piece of BIM information.

[0131] The annotation information storage unit 125 stores annotation information. The annotation information includes comments attached to the image and information about the annotation area. Comments are messages written by a user viewing the image displayed on the monitor 23, noting points of attention, etc. Comments are input using the operation unit 24. This allows the user to prompt attention from other users.

[0132] The annotation area is a region defined by the user on the image, and it is the area that displays the annotation object associated with the comment. The annotation area is defined using the operation unit 24. For example, if the given space is a construction site, the construction components are equivalent to annotation objects. For example, if the user adds a certain comment to a door, the annotation area is defined by surrounding the door with a frame, and the comment is entered in correspondence with that annotation area. A correspondence is established between the image ID and the annotation information. Thus, it can be determined which image the annotation information is set on.

[0133] Figure 3B This is a diagram showing an example of a display screen 300 on a monitor when a shooting location icon is selected in a variation of Embodiment 1. Figure 3B The display screen 300 relative to Figure 3A The display screen 300 differs from the previous one in that it also has a photo button 302 and a BIM button 303.

[0134] BIM button 303 switches the display of screen 300 from image Gn to a BIM image. A BIM image is a rendered image obtained by referencing BIM information in the same direction as the shooting direction of image Gn. Photo button 302 switches the display of the BIM image to image Gn.

[0135] Figure 6 This diagram shows a comparison screen 400 when a BIM image is displayed as the nth comparison image. For example, when the display control unit 112 instructs the receiving unit 111 to receive an instruction from the information terminal 20 to switch the display of the second area 420 to a BIM image, it displays a BIM image 405 observed from the shooting location Pn as the nth comparison image, replacing the image taken at the shooting location Qn. The display control unit 112 generates the image by setting a virtual camera at the shooting location Pn in the imaginary 3D space represented by the BIM information, setting the virtual camera's line of sight to the same direction as the shooting direction of the nth comparison image, and rendering the BIM information. In this case, the display control unit 112 sets the viewing angle of the BIM image 405 to the same viewing angle as the nth comparison image displayed in the second area 420. Thus, the user can compare the BIM image 405 with the nth reference image.

[0136] BIM image 405 represents the completed site condition, so users can confirm the site progress by comparing BIM image 405 with the nth reference image.

[0137] Furthermore, when the walk-through indicator button 310 is pressed, the display control unit 112 displays the image captured at shooting location Pn+1 in the first area 410 as the (n+1)th reference image. Simultaneously, the display control unit 112 displays the BIM image 405 viewed from shooting location Pn+1 in the second area 420 as the (n+1)th comparison image. At this time, the display control unit 112 sets the line of sight of the imaginary camera for BIM image 405 to the shooting direction of the (n+1)th reference image, and sets the viewing angle of BIM image 405 to be the same as the viewing angle of the (n+1)th reference image. Thus, walk-through display can be achieved simultaneously by comparing and displaying the image captured at the shooting location and the BIM image.

[0138] exist Figure 6 In the example, BIM image 405 is displayed in area 2 420. However, this is just one example; a 3D CAD image or a 2D bird's-eye view (design drawing) of a given space can also be displayed instead of BIM image 405. Area 2 420 can simply display the design drawing with the shooting location icon 210 superimposed. Area 2 420 can also display the design drawing such that the shooting location icon 210n is displayed near the center of area 2 420. Alternatively, area 2 420 can also display the entire design drawing such that the shooting location icon 210n is displayed in a different way than the other shooting location icons 210. For example, this display method could involve making the shooting location icon 210n blink or displaying it in a different color than the other shooting location icons.

[0139] (Implementation Method 2)

[0140] In Embodiment 2, when comparing and displaying the first and second images, if the annotation information corresponds to the first image, then the annotation information is also displayed in the second, third, and fourth images. In this embodiment, the same reference numerals are used for the same components as in Embodiment 1, and descriptions are omitted. Furthermore, in this embodiment, the block diagrams are retained. Figure 1B A block diagram.

[0141] Reference Figure 1B If the first image includes annotation information, the display control unit 112 determines whether the second, third, and fourth images include annotation objects corresponding to the annotation information. The display control unit 112 then displays the annotation information in the images determined to include annotation objects among the second, third, and fourth images. An annotation object is an object annotated using the annotation information.

[0142] Figure 8 This is a diagram showing a comparison screen 800 displaying annotation information 830. The annotation information 830 is, for example, pre-defined by the user. Figure 3B Input is shown in screen 300.

[0143] If a user presses the edit button 304 using the operation unit 24 in the display screen 300, an annotation input field (illustration omitted) is displayed on the left or right side of the display screen 300. The user uses the operation unit 24 to input an annotation object in the image Gn using a frame (illustration omitted). The area specified by the frame becomes the annotation area 820. In addition, the user uses the operation unit 24 to input annotations associated with the annotation area 820 in the annotation input field. The information terminal 20 sends the information including the annotation area 820 input using the operation unit 24 and the annotation information 810 representing the annotation as annotation information 830 to the server 10 using the communication unit 21. A correspondence is established between the shooting ID and the annotation information 830. The instruction receiving unit 111 of the server 10 stores the annotation information 830 received by the communication unit 13 in the annotation information storage unit 125 in association with the shooting ID of the image Gn. Thus, the annotation information 830 and the image Gn are established in correspondence using the shooting ID as the key.

[0144] exist Figure 8 In this example, annotation information 830 is established in the nth reference image. In this example, the area surrounding the window glass is set as annotation area 820. Furthermore, commentary information 810 corresponding to annotation area 820 is displayed in correspondence with annotation area 820.

[0145] exist Figure 8 In the example, the nth comparison image does not correspond to the annotation information 830. However, the nth comparison image includes the annotation object (window glass) shown in the annotation area 820 of the nth reference image. Therefore, the annotation information 830 set for the nth reference image is also displayed in the nth comparison image.

[0146] If the user inputs the operation of pressing the walk-through indicator button 310 using the operation unit 24 in area 410, then as Figure 9 As shown, while the display in the first region 410 is switched to the (n+1)th reference image, the display in the second region 420 is switched to the (n+1)th comparison image.

[0147] The (n+1)th reference image and the (n+1)th comparison image do not correspond to the annotation information 830. However, the (n+1)th reference image and the (n+1)th comparison image each include the annotation object (window glass) shown in the annotation area 820 defined for the nth reference image. Therefore, the annotation information 830 is also displayed in the (n+1)th reference image and the (n+1)th comparison image.

[0148] Figure 10This is a flowchart illustrating the processing of the information processing system 1 in Embodiment 2. The processing steps S21 to S24 are... Figure 7 The processing of steps S1 to S4 is the same.

[0149] In step S25, the display control unit 112 determines whether there is annotation information 830 in the nth reference image. If there is annotation information in the annotation information storage unit 125 that corresponds to the shooting ID of the nth reference image, the display control unit 112 determines that there is annotation information in the nth reference image.

[0150] If there is annotation information 830 in the nth reference image (yes in step S25), the process proceeds to step S26; if there is no annotation information 830 in the nth reference image (no in step S25), the process proceeds to step S28.

[0151] Next, in step S26, the display control unit 112 determines whether there is an annotation object in the nth comparison image that corresponds to the annotation information 830 of the nth reference image. The display control unit 112 can determine whether there is an annotation object in the nth comparison image by applying template matching with the annotation region 820 included in the nth reference image as a template.

[0152] If it is determined that there is an annotated object (yes in step S26), the process proceeds to step S27; if it is determined that there is no annotated object (no in step S26), the process proceeds to step S28.

[0153] Next, in step S27, the display control unit 112 adds annotation information 830 to the nth comparison image. Specifically, the display control unit 112 adds a frame to the outline of the annotation object included in the nth comparison image, and adds corresponding annotation information 810 to the annotation object.

[0154] Next, in step S28, the display control unit 112 compares the nth reference image and the (n+1)th comparison image by displaying the nth comparison image with (or without) added annotation information 830 in the second area 420. Thus, the image is displayed on the monitor 23. Figure 8 The comparison screen shown is 800.

[0155] The processing in steps S29 and S30 is respectively related to Figure 7 Steps S10 and S11 are the same.

[0156] Next, in step S31, the display control unit 112 determines whether there is annotation information 830 in the nth reference image. This process is the same as in step S25. If there is annotation information 830 in the nth reference image (yes in step S31), the process proceeds to step S33; if there is no annotation information 830 in the nth reference image (no in step S31), the process proceeds to step S34.

[0157] Next, in step S32, the display control unit 112 determines whether there is an annotation object in the (n+1)th reference image that corresponds to the annotation information 830 of the nth reference image. The details of this process are the same as in step S26.

[0158] If it is determined that there is an annotated object (yes in step S32), the process proceeds to step S33; if it is determined that there is no annotated object (no in step S32), the process proceeds to step S34.

[0159] Next, in step S33, the display control unit 112 adds annotation information 830 to the (n+1)th reference image. The details of this process are the same as in step S27.

[0160] Next, in step S34, the display control unit 112 determines whether there is annotation information 830 in the nth reference image. This process is the same as in step S25.

[0161] If there is annotation information 830 in the nth reference image (yes in step S34), the process proceeds to step S35; if there is no annotation information 830 in the nth reference image (no in step S34), the process proceeds to step S37.

[0162] Next, in step S35, the display control unit 112 determines whether the annotation object corresponding to the annotation information 830 of the nth reference image exists in the (n+1)th comparison image. The details of this process are the same as in step S26.

[0163] If it is determined that there is an annotated object (yes in step S35), the process proceeds to step S36; if it is determined that there is no annotated object (no in step S35), the process proceeds to step S37.

[0164] Next, in step S36, the display control unit 112 adds annotation information 830 to the (n+1)th comparison image. The details of this process are the same as in step S27.

[0165] Next, in step S37, the display control unit 112 displays the (n+1)th image with (or without) added annotation information 830 in the first area 410, and displays the (n+1)th comparison image with (or without) added annotation information 830 in the second area 420, thereby comparing and displaying the (n+1)th reference image and the (n+1)th comparison image (step S37). As a result, the image is displayed on the monitor 23. Figure 9 The comparison screen shown is 800. If the processing in step S37 is completed, the processing returns to step S29.

[0166] In this way, according to Embodiment 2, if annotation information 830 is included in the nth reference image, it is determined whether there is an annotation object corresponding to the annotation information 830 in the nth comparison image, the (n+1)th reference image, and the (n+1)th comparison image. Furthermore, if it is determined that there is an annotation object, the annotation information 830 is displayed in the (n+1)th reference image, the nth comparison image, and the (n+1)th comparison image where the annotation object is determined to be present. Therefore, confirming the annotation information after inputting the walkthrough instruction becomes easier.

[0167] In addition, Figure 7 The flowchart shows a case where annotation information 830 exists in the nth reference image, but this is just one example; annotation information 830 could also exist in the nth comparison image. In this case, if the annotation object corresponding to the annotation information 830 present in the nth comparison image exists in any of the nth reference image, the (n+1)th reference image, and the (n+1)th comparison image, the display control unit 112 can display the annotation information 830 in that image.

[0168] Furthermore, if there is no annotation information 830 in the nth reference image but there is annotation information 830 in the nth comparison image, and if the annotation object corresponding to the annotation information 830 present in the nth comparison image exists in the (n+1)th comparison image, then the annotation information 830 can also be displayed only in the (n+1)th comparison image.

[0169] Furthermore, if there is no annotation information 830 in the nth comparison image but there is annotation information 830 in the nth reference image, if the annotation object corresponding to the annotation information 830 present in the nth reference image exists in the (n+1)th reference image, then the annotation information 830 can also be displayed only in the (n+1)th reference image.

[0170] Furthermore, if the display control unit 112 displays different annotation information 830a and 830b in each of the nth reference image and the nth comparison image, and if any of the nth comparison image, the (n+1)th reference image, and the (n+1)th comparison image contains an annotation object corresponding to annotation information 830a, then annotation information 830a can also be displayed in the image containing the annotation object. In this case, if any of the nth reference image, the (n+1)th reference image, and the (n+1)th comparison image contains an annotation object corresponding to annotation information 830b, then the display control unit 112 can simply display annotation information 830b in the image containing the annotation object.

[0171] (Implementation Method 3)

[0172] In Embodiment 3, the shooting location with the shortest distance relative to the selected shooting location (shooting location Pn+1) that is less than or equal to a specified distance is determined as the corresponding shooting location (shooting location Qn+1). In this embodiment, the same reference numerals are used for the same constituent elements as in Embodiments 1 and 2, and descriptions are omitted. Furthermore, in this embodiment, the block diagram is retained. Figure 1A or Figure 1B A block diagram. Specify a distance, for example, using a distance preset by the user.

[0173] Figure 11 This is a flowchart illustrating the processing of the information processing system 1 in Embodiment 3. The processing steps S41 to S44 are... Figure 7 Steps S1 to S4 are the same.

[0174] In step S45, the display control unit 112 determines whether the distance between the shooting location Qn and the shooting location Pn is within a specified distance. If the distance is within the specified distance (yes in step S45), the display control unit 112 compares and displays the nth reference image and the nth comparison image (step S46). In this case, as... Figure 4 As shown, a comparison screen 400, in which the nth reference image is displayed in the first area 410 and the nth comparison image is displayed in the second area 420, is displayed on the monitor 23.

[0175] On the other hand, if the distance between the shooting location Qn and the shooting location Pn is not within the specified distance (no in step S45), the display control unit 112 does not compare and display the nth reference image and the nth comparison image (step S47). In this case, a comparison screen 400, in which the nth reference image is displayed in the first area 410 and nothing is displayed in the second area 420, is displayed on the monitor 23. As a way of displaying nothing, for example, the second area 420 can be displayed in a given color (e.g., black or gray). Furthermore, since there is no image captured at the shooting location within the specified distance relative to the shooting location Pn, a message indicating that the image cannot be displayed can also be displayed in the second area 420, which is displayed in the given color.

[0176] Processing of steps S48 and S49 Figure 7 Steps S10 and S11 are the same.

[0177] Next, in step S50, the display control unit 112 determines whether the distance between the shooting location Qn+1 and the shooting location Pn+1 is within a specified distance.

[0178] If the distance between shooting location Qn+1 and shooting location Pn+1 is within a specified distance (Yes in step S50), the display control unit 112 compares and displays the (n+1)th reference image and the (n+1)th comparison image (step S51). In this case, as... Figure 5 As shown, a comparison screen 400, in which the (n+1)th reference image is displayed in the first area 410 and the (n+1)th comparison image is displayed in the second area 420, is displayed on the monitor 23.

[0179] On the other hand, if the distance between shooting location Qn+1 and shooting location Pn+1 is not within the specified distance (no in step S50), the display control unit 112 does not compare and display the (n+1)th reference image and the (n+1)th comparison image (step S52). In this case, a comparison screen 400, in which the (n+1)th reference image is displayed in the first area 410 and nothing is displayed in the second area 420, is displayed on the monitor 23.

[0180] In this way, according to embodiment 3, two images taken at similar locations are compared and displayed, thus making it easy to confirm changes in the conditions of a given space.

[0181] (Implementation Method 4)

[0182] In embodiment 4, the shooting location whose shooting date and time are within a specified time relative to the selected shooting location (shooting location Pn+1) is determined as the corresponding shooting location (shooting location Qn+1). In this embodiment, the same reference numerals are used for the same constituent elements as in embodiments 1 to 3, and the descriptions are omitted. Furthermore, in this embodiment, the block diagrams are retained. Figure 1A or Figure 1B The diagram shows the block diagram. A specified time can be set, for example, using a distance preset by the user. As a specified time, appropriate values ​​such as 1 month, 1 week, or 1 day can be used.

[0183] Figure 12 This is a flowchart illustrating the processing of the information processing system 1 in Embodiment 4. The processing steps S61 to S68 are... Figure 11 Steps S41 to S48 are the same.

[0184] In step S69, the display control unit 112 selects from the image information storage unit 124 an image whose shooting date and time are within a specified time relative to the nth comparison image. Specifically, the display control unit 112 selects from the image information storage unit 124 an image taken within a specified time relative to the nth comparison image from among multiple shooting locations belonging to a shooting period (second shooting period) that is more recent than the shooting period to which the shooting location Pn belongs.

[0185] Next, in step S70, the display control unit 112 selects the image taken at the shooting location Qn+1, which is the closest to the shooting location Pn+1, from the selected images, and uses it as the (n+1)th comparison image.

[0186] Next, in step S71, the display control unit 112 determines whether the distance between shooting location Qn+1 and shooting location Pn+1 is within a specified distance. If the distance is within the specified distance (yes in step S71), the process proceeds to step S72; if the distance is not within the specified distance (no in step S71), the process proceeds to step S73.

[0187] The processing of steps S72 and S73 and Figure 11 Steps S51 and S52 are the same. If the processing of steps S72 and S73 is completed, the processing returns to step S68.

[0188] In this way, according to embodiment 4, two images taken within a specified time period are compared and displayed, thus making it easy to confirm the changes in the condition of a given space within the specified time period.

[0189] (Implementation Method 5)

[0190] In implementation method 5, when two images are taken within the same space located in a given space, they are compared and displayed. A subspace refers to a space divided by columns or walls; rooms, partitions, corridors, elevator lobbies, and entryways are equivalent to subspaces. Furthermore, during construction, a subspace refers to a space where the view is not obstructed by columns or similar structures.

[0191] Figure 13 This is a flowchart illustrating the processing of the information processing system 1 in Embodiment 5. The processing steps S81 to S83 are... Figure 7 Steps S1 to S3 are the same.

[0192] In step S84, the display control unit 112 selects an image captured in the same subspace as the nth reference image from the image information storage unit 124. For example, the display control unit 112 determines the area corresponding to the subspace to which the shooting location Pn belongs in the design drawing of the given space, extracts the image of the shooting location within the area corresponding to the determined subspace from the image information storage unit 124, and selects the extracted image as the image captured in the same subspace. Furthermore, if the given space is a construction site, the area of ​​the subspace changes according to the progress of construction. Therefore, the design drawing of the given space sometimes updates the outline representing the subspace according to the progress of construction. In this case, the display control unit 112 uses the latest design drawing to determine whether a shot was taken in the same subspace. Alternatively, sometimes multiple design drawings are prepared in advance, corresponding to the progress of construction from the start to the end. In this case, the display control unit 112 selects the design drawing closest to the current time point from the multiple prepared design drawings and uses the selected design drawing to determine whether a shot was taken in the same subspace.

[0193] Next, in step S85, the display control unit 112 selects the image captured at the shooting location Qn with the shortest distance relative to the shooting location Pn from the images selected in step S84, and uses it as the nth comparison image.

[0194] Processing in steps S86 to S89 Figure 12 Steps S65 to S68 are the same.

[0195] Next, in step S90, the display control unit 112 selects an image captured in the same subspace as the (n+1)th reference image from the image information storage unit 124. The details of this process are the same as in step S84.

[0196] Next, the processing in steps S91 to S94 is... Figure 12 Steps S70 to S73 are the same.

[0197] In this way, the display control unit 112 determines whether the shooting location of the nth reference image (first image) and the shooting location of the nth comparison image (second image) are located in the same subspace. If it is determined that they are located in the same subspace, the nth comparison image (second image) is displayed. Furthermore, the display control unit 112 determines whether the shooting location of the (n+1)th reference image (third image) and the shooting location of the (n+1)th comparison image (fourth image) are located in the same subspace. If it is determined that they are located in the same subspace, the (n+1)th comparison image (fourth image) is displayed. Therefore, comparing two images taken within the same subspace becomes easier. Without defining subspaces, there is a possibility that points in subspaces that are not in the same room or corridor as the given space, but are different from the given space, may be associated. As a result, the view may be obstructed by pillars or walls, making it impossible to perform the desired on-site comparison, and consequently, errors may occur in the comparison. By defining subspaces, the occurrence of such errors can be suppressed.

[0198] The following variations are possible with respect to this disclosure.

[0199] (1) In the above embodiment, the walk-through instruction is input via the walk-through instruction button 310, but this disclosure is not limited to this. The walk-through instruction can also be input by selecting the shooting location icon displayed in the design drawing display bar 301 shown on the comparison screen 400. In this case, the display control unit 112 can also scroll the design drawing displayed in the design drawing display bar 301 so that the selected shooting location icon 210 is located near the center of the screen.

[0200] (2) When an operation to change the viewing angle is input in either the first region 410 or the second region 420, the display control unit 112 can also change the viewing angle of the image displayed in the other region of the first region 410 or the second region 420 in conjunction with the change of the viewing angle in one region.

[0201] Furthermore, in this case, the display control unit 112 can also be linked with the change of viewing angle to enlarge or reduce the display of the design drawing displayed in the design drawing display bar 301.

[0202] (3) The annotation information 830 consists of the comment information 810 and the annotation area 820, but it can also be composed of either one.

[0203] (4) In 4 and Figure 5In this configuration, the walk-through indicator button 310 is located in the first area 410, but it can also be located in the second area 420. In this case, when the walk-through indicator button 310 in the second area 420 is pressed, the image displayed in the second area 420 is used as the reference image, and the image displayed in the first area 410 is used as the comparison image.

[0204] Industrial availability

[0205] This disclosure is useful in the technical field of managing the progress of construction at a construction site from a remote location.

Claims

1. An information processing method, which is an information processing method in a computer, comprising: The first image, taken at the first date and time, is displayed on the screen of the information terminal. A second image, taken at a second date and time, of the given space, will be displayed on the monitor. The second date and time is different from the first date and time. and While the first image and the second image are displayed, Upon detecting the selection of a shooting location, the display of the first image is switched to the third image taken at the selected shooting location, and the display of the second image is switched to the fourth image taken at the corresponding shooting location. The selected shooting location is a shooting location different from the shooting location under the first date and time or the second date and time, and the corresponding shooting location is the shooting location corresponding to the selected shooting location.

2. The information processing method according to claim 1, wherein, The first image, the second image, the third image, and the fourth image are each composed of omnidirectional images.

3. The information processing method according to claim 1, wherein, In the image group including the first image and the third image and the image group including the second image and the fourth image, any one image group consists of a 360-degree image, and any other image group consists of a 3D simulation image of the given space or a 2D bird's-eye view image of the given space.

4. The information processing method according to claim 1 or 2, wherein, The information processing method further includes: The first image and the second image are displayed on the monitor, with the shooting directions aligned; and The third and fourth images are displayed on the monitor so that the shooting directions are consistent.

5. The information processing method according to claim 1 or 2, wherein, The information processing method further includes: The first image and the second image are displayed on the monitor so that the viewing angles are consistent; The third and fourth images are displayed on the monitor so that the viewing angles are consistent.

6. The information processing method according to claim 1 or 2, wherein, The information processing method further includes: if annotation information is included in either the first image or the second image, determining whether an annotation object corresponding to the annotation information is included in any other image among the first image and the second image and at least one image among the third image and the fourth image; and displaying the annotation information in the image among the other image, the third image, and the fourth image that is determined to include the annotation object.

7. The information processing method according to claim 1 or 2, wherein, The corresponding shooting location is the shooting location with the shortest distance among shooting locations that are within a specified distance of the selected shooting location.

8. The information processing method according to claim 1 or 2, wherein, The corresponding shooting location is a shooting location within a specified time period relative to the selected shooting location.

9. The information processing method according to claim 1 or 2, wherein, The given space includes multiple subspaces. The information processing method further includes: Determine whether the shooting locations of the first image and the second image are located in the same subspace. If they are determined to be in the same subspace, display the second image; and Determine whether the shooting location of the third image and the shooting location of the fourth image are located in the same subspace. If they are determined to be located in the same subspace, display the fourth image.

10. The information processing method according to claim 1 or 2, wherein, The third image is an image taken at a date or time one day before or after the first date or time. The fourth image is an image taken at a date or time one day before or after the second date or time, or an image taken at a location close to an image taken at a date or time one day before or after the first date or time, and at a time close to a date or time one day before or after the second date or time.

11. The information processing method according to claim 1 or 2, wherein, The second image is an image taken at a location corresponding to the location where the first image was taken.

12. An information processing apparatus, comprising a processor, The processor performs the following processing: The first image, taken at the first date and time, is displayed on the screen of the information terminal. A second image, captured at a second date and time different from the first date and time, will be displayed on the monitor. While the first image and the second image are displayed, Upon detecting the selection of a shooting location, the display of the first image is switched to the third image taken at the selected shooting location, and the display of the second image is switched to the fourth image taken at the corresponding shooting location. The selected shooting location is a shooting location different from the shooting location under the first date and time or the second date and time, and the corresponding shooting location is the shooting location corresponding to the selected shooting location.

13. An information processing program that causes a computer to perform the following processing: The first image, taken at the first date and time, is displayed on the screen of the information terminal. A second image, captured at a second date and time different from the first date and time, will be displayed on the monitor. While the first image and the second image are displayed, Upon detecting the selection of a shooting location, the display of the first image is switched to the third image taken at the selected shooting location, and the display of the second image is switched to the fourth image taken at the corresponding shooting location. The selected shooting location is a shooting location different from the shooting location under the first date and time or the second date and time, and the corresponding shooting location is the shooting location corresponding to the selected shooting location.

Images