Information processing method, information processing device, and information processing program

Abstract

The information processing apparatus displays a first image obtained by capturing an image of a predetermined space at a first date time on a display of an information terminal, and when a selection instruction of a second date time, which is a date time different from the first date time, is detected, displays the first image on the display of the information terminal. And a display unit that displays, on the display, a second image obtained by imaging the predetermined space at the second date time, and that changes the default viewpoint so as to match the viewpoint of the first image and that is displayed on the display.

Description

Technical Field

[0001] The present disclosure relates to a technology for displaying a selected image. Background Art

[0002] Patent Document 1 discloses that an image (eg, outline) representing a designated area designated by a worker is displayed on a bird's-eye view image of a work site, and a chat associated with the designated area input by the worker is displayed in a display field provided next to the bird's-eye view image.

[0003] However, the prior art disclosed in Patent Document 1 does not disclose that an image captured at a scene represented by a bird's-eye view image is displayed on a display according to a user's selection instruction. Therefore, the prior art does not disclose that a second image associated with a first image displayed on a display according to a user's selection instruction is displayed on a display. Therefore, when the prior art displays a second image associated with a first image on a display, it is not possible to display the second image on a display as a subject displayed in the first image through a simple operation.

[0004] Prior Art Literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-86224 Summary of the invention

[0007] The present disclosure is made to solve such a problem, and its purpose is to provide a technology for displaying the second image on the display as a subject displayed in the first image through a simple operation when displaying the second image associated with the first image on the display.

[0008] An information processing method in one aspect of the present invention is an information processing method in a computer, wherein a first image obtained by photographing a given space at a first date and time is displayed on a display of an information terminal; when a selection indication of a second date and time that is a date and time different from the first date and time is detected, a second image obtained by photographing the given space at the second date and time is displayed on the display; and the second image changes its default viewpoint so as to be consistent with the viewpoint of the first image and is displayed on the display.

[0009] According to this configuration, when displaying the second image associated with the first image on the display, it is possible to display the second image on the display so as to display the subject displayed in the first image through a simple operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1This is a block diagram showing an example of the configuration of an information processing system in an embodiment of the present disclosure.

[0011] Figure 2 The figure shows an example of a display screen displayed on the display of the terminal device.

[0012] Figure 3 FIG. 1 is a diagram showing an example of a menu screen for selecting a shooting date and time.

[0013] Figure 4 The diagram shows an example of a first image and a second image displayed on a display.

[0014] Figure 5 This is a conceptual diagram of a change in viewpoint in this embodiment.

[0015] Figure 6 It is shown Figure 1 A flowchart of an example of processing of the information processing device is shown.

[0016] Figure 7 It is shown Figure 6 The flowchart shown shows the details of the processing.

[0017] Figure 8 It is an explanatory diagram of the first image and the second image in the modification example of the present disclosure. DETAILED DESCRIPTION

[0018] (Process of realizing one mode of the present disclosure)

[0019] Issues at the construction site include communication issues such as specific instructions not being conveyed to workers and the time it takes to explain the instructions, and confirmation issues such as the need for manpower to patrol the entire construction site and the time it takes to move to the construction site.

[0020] In order to solve such a problem, it is also considered that multiple cameras are installed at the construction site, and the on-site supervisor at a remote location refers to the images obtained from the multiple cameras to issue instructions to the workers. However, at the construction site, as the construction progresses, operations such as removing the installed sensors and installing the removed sensors in other places will occur. Such operations take time, so it is not practical to install sensors at the construction site. Therefore, the inventors of the present invention have studied a technology that can confirm the conditions of the construction site in detail from a remote location without installing sensors.

[0021] It was then realized that if an operation for selecting a shooting date is input in the design drawing of the construction site displayed on the display, shooting is performed on that shooting day and the shooting location of the image is displayed superimposed on the design drawing. If an indication for selecting a shooting location is input, the image captured at the selected shooting location is displayed. As long as there is a user interface such as the one described above, the conditions of the construction site can be confirmed in detail from a remote location.

[0022] However, in such displayed images, the subject that the user is interested in may be captured while being blocked by another subject. In addition, if the user can observe the object of interest through another line of sight, such a subject can be observed in more detail.

[0023] In this case, if the second image captured near the first image, which is a second image captured on a different shooting date at the same construction site as the first image, is displayed on the monitor, the user can observe the subject of interest in more detail.

[0024] However, sometimes the first image and the second image are displayed by displaying a portion of an area centered on a default viewpoint corresponding to a given direction (for example, a direction parallel to the horizontal plane and facing north) on the display. For example, wide-angle images such as omnidirectional images and panoramic images are sometimes used as the first image and the second image. When such a display method is adopted, it may not be possible to display the second image as including the subject of interest based on the default viewpoint of the second image. In this case, the user needs to scroll the second image to display the subject of interest on the display, which takes time. Furthermore, since the process of scrolling the image in accordance with the scrolling operation occurs, the processing burden of the computer increases.

[0025] Therefore, the inventors of the present invention have acquired the following knowledge and finally conceived the present disclosure: when displaying a second image associated with a first image on a display, such a problem can be solved by changing the default viewpoint of the second image so as to match the viewpoint of the first image for display.

[0026] (1) An information processing method in one aspect of the present disclosure is an information processing method in a computer, wherein a first image obtained by photographing a given space at a first date and time is displayed on a display of an information terminal; when a selection indication of a second date and time that is different from the first date and time is detected, a second image obtained by photographing the given space at the second date and time is displayed on the display; and the second image is displayed on the display by changing a default viewpoint so as to be consistent with the viewpoint of the first image.

[0027] According to this structure, when displaying the second image captured at a different date and time from the first image, the second image whose default viewpoint is changed so as to match the viewpoint of the first image is displayed. Therefore, the user can display the subject of interest in the second image without performing laborious operations such as scrolling the second image to display the subject displayed in the first image on the display. Furthermore, since such scrolling operations are not required, the processing burden of the computer can be reduced.

[0028] (2) In the information processing method described in (1) above, further, the overhead image as an overhead image of the given space, in which a plurality of shooting location icons indicating shooting locations at a first date and time are superimposed and displayed, may be displayed on the display, and further, selection of one shooting location icon is detected from among the plurality of shooting location icons, the first image being an image shot at the shooting location indicated by the one shooting location icon.

[0029] According to this configuration, the first image can be intuitively selected from the bird's-eye view image.

[0030] (3) In the information processing method described in (1) or (2) above, the second image may be an image of the given space photographed at a shooting location closest to a shooting location of the first image among a plurality of images photographed at the second date and time.

[0031] According to this configuration, the second image that is likely to display the subject included in the first image can be displayed without inputting an operation of selecting one image from a plurality of images captured at the second date and time.

[0032] (4) In the information processing method described in any one of (1) to (3) above, the change of the default viewpoint in the second image may also include: based on the first image, detecting corresponding points of the viewpoint of the first image from the second image; and setting the detected corresponding points as the viewpoint of the second image.

[0033] According to this configuration, since the corresponding point of the viewpoint of the first image is set as the viewpoint of the second image, the viewpoint of the second image can be accurately matched with the viewpoint of the first image.

[0034] (5) In the information processing method described in any one of the above (1) to (4), the viewpoint of the first image may be the center of the display area of ​​the first image displayed on the display, and the viewpoint of the second image may be the center of the display area of ​​the second image displayed on the display.

[0035] According to this configuration, the first image and the second image are displayed on the display as images within the display area centered on the viewpoint, respectively.

[0036] (6) In the information processing method described in any one of the above (1) to (5), when a scrolling indication of the first image is detected, the first image may be scrolled according to the scrolling indication, and the second image may be scrolled in conjunction with the scrolling of the first image.

[0037] According to this configuration, when the first image is scrolled, the second image is also scrolled in conjunction with the first image, so that the two images can be easily compared.

[0038] (7) In the information processing method described in any one of the above (1) to (6), when the corresponding point of the viewpoint of the first image cannot be detected from the second image, the corresponding point of the default viewpoint of the second image can be detected from the first image, and the viewpoint of the first image can be changed to the detected viewpoint for display on the display.

[0039] According to this configuration, even when a point corresponding to the viewpoint of the first image cannot be detected from the second image, the first image and the second image can be displayed so as to show the same subject.

[0040] (8) In the information processing method described in any one of (1) to (7) above, the given space may be a work site.

[0041] According to this configuration, the situation at the work site can be easily grasped.

[0042] (9) In the information processing method described in any one of (1) to (8) above, the first image and the second image may be displayed side by side on the display.

[0043] According to this configuration, comparison between the first and second images becomes easy.

[0044] (10) An information processing device in another embodiment of the present invention is an information processing device including a processor, wherein the processor displays a first image obtained by photographing a given space at a first date and time on a display of an information terminal; when a selection indication of a second date and time that is a date and time different from the first date and time is detected, a second image obtained by photographing the given space at the second date and time is displayed on the display; and the second image is displayed on the display by changing a default viewpoint so as to be consistent with the viewpoint of the first image.

[0045] According to this configuration, it is possible to provide an information processing device that can realize, when displaying a second image associated with a first image on a display, displaying the second image on the display as a subject displayed in the first image by a simple operation.

[0046] (11) An information processing program according to another aspect of the present disclosure causes a computer to execute the information processing method described in any one of (1) to (9) above.

[0047] According to this configuration, it is possible to provide an information processing program that realizes displaying the second image on the display as a subject displayed in the first image by a simple operation when displaying the second image associated with the first image on the display.

[0048] The present disclosure can also be realized as an information processing system operated by such an information processing program. In addition, it is needless to say that such an information processing program can be circulated via a computer-readable non-transitory recording medium such as a CD-ROM or a communication network such as the Internet.

[0049] In addition, the embodiments described below all illustrate a specific example of the present disclosure. The numerical values, shapes, constituent elements, steps, the order of steps, etc. shown in the following embodiments are examples, and their purpose is not to limit the present disclosure. In addition, for the constituent elements in the following embodiments that are not recorded in the independent claims representing the highest concept, they are described as arbitrary constituent elements. In addition, the various contents can be combined in all embodiments.

[0050] (Implementation Method)

[0051] Figure 1 1 is a block diagram showing an example of the structure of the information processing system 1 in the embodiment of the present disclosure. The information processing system 1 includes an information processing device 10, a shooting device 20, and a terminal device 30. The information processing device 10, the shooting device 20, and the terminal device 30 are communicatively connected via a network. An example of a network is the Internet. The information processing device 10 is, for example, a cloud server composed of one or more computers. However, this is an example, and the information processing device 10 may also be composed of an edge server or may be installed in the terminal device 30.

[0052] The photographing device 20 is, for example, composed of an omnidirectional camera, and photographs images at a given frame rate. The photographing device 20 is, for example, a portable photographing device held by a user. The user is, for example, a worker or a site supervisor at a construction site. The user moves within the construction site while photographing the construction site with the photographing device 20. The photographing device 20 sends image information representing the photographed image to the information processing device 10 via a network. The position of the photographing location representing the photographing location and the photographing date and time are associated with the image information. The position of the photographing location is obtained, for example, by a position sensor such as a magnetic sensor or a GPS sensor provided by the photographing device 20, and is represented by latitude and longitude. The photographing date and time is obtained, for example, by a clock provided by the photographing device 20. Thus, the information processing device 10 can obtain image information at multiple photographing locations at the construction site. Here, the photographing device 20 photographs images at a given frame rate, and thus the photographing location is specified in units of frame periods. However, this is an example, and the photographing location can also be specified at given intervals (for example, 1 second, 1 minute, etc.). The imaging device 20 includes an image sensor, an operating device, a communication circuit, a signal processing circuit, etc. The imaging device 20 may be constituted by a portable computer such as a smartphone or a tablet computer.

[0053] The terminal device 30 is held by the user. The terminal device 30 may be a portable computer such as a smartphone or a tablet computer, or may be a stationary computer. The terminal device 30 displays image information on a display according to the control of the information processing device 10. Figure 1 In the example of FIG. 1 , one terminal device 30 is shown, but a plurality of terminal devices may be connected to the information processing device 10 via a network. The terminal device 30 includes a central processing unit (CPU), a memory, an operating device such as a display, a touch screen, and a keyboard, and a communication circuit.

[0054] The information processing device 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 is composed of, for example, a central processing unit (CPU). The processor 11 includes a display control unit 111. The display control unit 111 may be implemented by the processor 11 executing an information processing program, or may be composed of a dedicated hardware circuit such as an ASIC.

[0055] The display control unit 111 obtains an instruction from the terminal device 30 for selecting a design drawing by the user, reads the design drawing information indicated by the instruction from the memory 12, and displays the read design drawing information on the display of the terminal device 30. The design drawing information is information of a design drawing representing a construction site (an example of a given space). The design drawing information is an example of an overhead image. The display control unit 111 displays the shooting location icon for selecting an arbitrary position of the design drawing in an overlapping manner on the design drawing. If an instruction is obtained from the terminal device 30 for selecting the first date and time by the user, the display control unit 111 displays multiple shooting location icons representing the shooting location under the first date and time in an overlapping manner on the design drawing. Their display is realized by the display control unit 111 sending the display instruction to the terminal device 30 using the communication unit 13. The selection icon is configured to be movable on the design drawing. In the design drawing, the latitude and longitude are pre-correlated with the positions that become key points. The positions that become key points are, for example, the positions of the four corners of the design drawing.

[0056] The display control unit 111 detects the selection of one shooting location icon by obtaining a selection instruction for one shooting location icon from a plurality of shooting location icons from the terminal device 30. If the selection of one shooting location icon is detected, the display control unit 111 displays the first image captured at the shooting location represented by the one shooting location icon on the display of the terminal device 30. The first image is an omnidirectional image. However, this is an example, and the first image may also be a panoramic image. Therefore, the display control unit 111 sets the display area with the default viewpoint of the first image as the center, and uses the communication unit 13 to send a display instruction to display the first image in the set display area on the display to the terminal device 30. As a result, the terminal device 30 displays the first image included in the display area centered on the default viewpoint on the display. The default viewpoint is a viewpoint set as an initial value, for example, a direction parallel to the horizontal plane and facing north.

[0057] When the display control unit 111 detects a scroll instruction of the first image input by the user from the terminal device 30, it sends a display instruction to scroll the first image according to the operation amount indicated by the detected scroll instruction to the terminal device 30. In this way, the terminal device 30 can change the display area of ​​the first image. In this case, the viewpoint of the first image becomes the center coordinate of the display area after scrolling. The display area of ​​the first image has a size and shape that is predetermined according to the size and shape of the display area of ​​the first image in the display of the terminal device 30, for example. The shape of the display area is, for example, a quadrilateral.

[0058] When the display control unit 111 detects the selection instruction of the second date and time by acquiring the selection instruction of the second date and time which is different from the first date and time from the terminal device 30, the display control unit 111 displays the second image obtained by photographing the construction site at the second date and time on the display of the terminal device 30. In detail, the display control unit 111 determines the image obtained by photographing the construction site at the shooting location closest to the shooting location of the first image among the multiple images photographed at the second date and time, and displays the determined image as the second image on the display of the terminal device 30. The structure of the second image is the same as that of the first image.

[0059] The display control unit 111 changes the default viewpoint of the second image so as to match the viewpoint of the first image, and displays the second image whose default viewpoint has been changed on the display of the terminal device 30. Specifically, the display control unit 111 detects the corresponding point of the viewpoint of the first image from the second image, and sets the detected corresponding point as the viewpoint of the second image. Furthermore, the display control unit 111 sets the display area of ​​the second image with the set viewpoint of the second image as the center, and transmits a display instruction to display the second image in the set display area on the display to the terminal device 30 using the communication unit 13. Thus, the terminal device 30 can display the second image whose display area is set to include the subject included in the display area of ​​the first image on the display. For example, the display control unit 111 can detect the corresponding point from the second image by applying pattern matching using the display area of ​​the first image as a template to the second image. The display area of ​​the second image has a size and shape predetermined according to the size and shape of the display area of ​​the second image in the display of the terminal device 30, for example. The shape of the display area is, for example, a quadrilateral. Here, the display area of ​​the first image and the display area of ​​the second image are set to be the same in size and shape. In addition, the display area of ​​the first image and the display area of ​​the second image are set to be arranged side by side. Thus, the first image and the second image are displayed side by side.

[0060] The memory 12 is composed of a non-transient rewritable storage device such as a hard disk drive and a solid state drive. The memory 12 stores the design information, the shooting information, the annotation information, the image information, and the annotation area information. The design information is the image information representing the design. A design ID for identifying the design is associated with the design information. In the design, the latitude and longitude of the actual construction site are set at key points as described above.

[0061] The shooting information indicates information related to one shooting action using the shooting device 20. The shooting information is generated each time one shooting action is performed. One shooting action refers to a series of actions from the start of shooting to the end of shooting by the worker holding the shooting device 20 at the construction site. Multiple images are captured by one shooting action. The shooting information includes a design drawing ID, a shooting ID, a shooting date and time, a representative value of the shooting date and time, the position of the shooting location, and the position of the shooting location icon. The shooting ID is an identifier for identifying each shooting included in one shooting action. The shooting date and time is the shooting date and time of the shooting indicated by the shooting ID. The representative value of the shooting date and time is the shooting date and time when the shooting starts. The first date and time and the second date and time mentioned above refer to the representative values ​​of the shooting date and time. The shooting location indicates the location (latitude and longitude) where the shooting indicated by the shooting ID is performed. The position of the shooting location icon indicates the display position (coordinates) of the shooting location icon corresponding to the shooting ID on the design drawing. The position of the shooting location icon is calculated by mapping the shooting location on the design drawing based on the shooting location (latitude and longitude) at the key point of the design drawing indicated by the design drawing information and the shooting location (latitude and longitude) corresponding to the shooting ID.

[0062] The annotation information is information indicating the annotation. One annotation information corresponds to one annotation. The annotation information is associated with the photographing ID and the annotation area ID. The annotation area ID is an identifier of the annotation area to which the annotation is given, which is set in the image information corresponding to the photographing ID.

[0063] The image information indicates one image captured by each shot included in one shooting action. That is, the image information indicates the first image or the second image mentioned above. A shooting ID and an annotation area ID are associated with the image information. When multiple annotation areas are set in the image information corresponding to the shooting ID, multiple annotation area IDs are included in the image information.

[0064] The annotation area information stores the positions (coordinates) of key points of the annotation area set in the image information corresponding to the shooting ID. The key points are vertices on the outline of the annotation area. The annotation area information is associated with the shooting ID and the annotation area ID.

[0065] In this way, since the shooting ID is associated with the shooting information and the image information, the image information corresponding to the shooting location icon can be determined using the shooting ID as a key. Since the annotation area ID is associated with the annotation information and the annotation area information, the annotation area ID can be used as a key to determine the annotation information corresponding to the annotation area information. Since the shooting ID is associated with the annotation area information and the image information, the shooting ID can be used as a key to determine the image information corresponding to the annotation area information.

[0066] The communication unit 13 is a communication circuit that connects the information processing device 10 to a network.

[0067] Figure 2 1 is a diagram showing an example of a display screen G1 displayed on the display of the terminal device 30. The display screen G1 is a basic screen of an application provided by the information processing device 10. The display screen G1 includes an image information display column R1, an annotation information display column R2, a design drawing display column R3, and an information display column R4.

[0068] The image information display field R1 displays image information associated with one shooting location icon determined by the display control unit 111 .

[0069] The annotation information display column R2 displays annotation information corresponding to the determined shooting location icon. Here, annotations C1 input by multiple users for the image information displayed in the image information display column R1 are displayed at a glance. In the annotation information display column R2, annotations C1 input by users other than the user are displayed on the left, and annotations C1 input by the user are displayed on the right.

[0070] In the default display screen G1 immediately after the application is activated, the shooting location icon has not been selected by the user. Therefore, in the default display screen G1, the image information display field R1 and the comment information display field R2 are empty.

[0071] The design drawing display area R3 displays the design drawing of the construction site. In the design drawing displayed in the design drawing display area R3, a selection icon 201, a shooting location icon 202, and a track 203 are superimposed and displayed.

[0072] The selection icon 201 is configured to be movable by a drag-and-drop operation. In this example, the selection icon 201 is configured by an image of a simulated person.

[0073] The shooting location icon 202 is an icon indicating the shooting location, and is associated with the image information. In this example, the shooting location icon 202 is composed of a circular image. The trajectory 203 indicates the trajectory of the user who has shot the image information. In this example, the trajectory 203 is composed of a line connecting adjacent shooting location icons 202. The shooting location icon 202 located at the front end of the trajectory 203 and the shooting location icon 202 located at the rear end of the trajectory 203 are displayed in a larger size than the other shooting location icons. The shooting location icon 202 located at the front end (e.g., the right end) of the trajectory 203 indicates the shooting start position, and the shooting location icon 202 located at the rear end (e.g., the left end) of the trajectory 203 indicates the shooting end position.

[0074] When an operation (for example, tap or click) for selecting the image displayed in the image information display field R1 is input, the design display field R3 displays the image displayed in the image information display field R1 in the design display field R3 .

[0075] The information display section R4 displays messages input by the user as various messages related to the construction site.

[0076] For example, when the selection icon 201 is released in a given area of ​​any shooting location icon 202, the shooting location icon 202 is determined as one shooting location icon, and the image corresponding to the one shooting location icon is detected as the first image. Then, the first image is displayed in the image information display column R1. In this case, as long as annotation information is associated with the one shooting location icon, the annotation information corresponding to the one shooting location icon is displayed in the annotation information display column R2.

[0077] For example, it is assumed that the selection icon 201 is not released in the given area at any of the shooting location icons 202. In this case, the shooting location icon with the shortest distance to the release position and the shooting location icon associated with the annotation information is determined as one shooting location icon, and the image corresponding to the one shooting location icon is detected as the first image. Then, the first image is displayed in the image information display column R1, and the annotation information corresponding to the first image is displayed in the annotation information display column R2.

[0078] Figure 3 1 is a diagram showing an example of a menu screen 300 for selecting a shooting date and time. When a given operation for displaying the menu screen 300 is performed on the display screen G1, the menu screen 300 is displayed. In the menu screen 300, a list of a plurality of shooting dates and times that were shot at a certain construction site is displayed. One shooting date and time included in the menu screen 300 represents a representative value of the shooting date and time of one shooting action.

[0079] When the user inputs an instruction to select a shooting date and time from the menu screen 300, the display control unit 111 obtains the instruction from the terminal device 30. Furthermore, the display control unit 111 sends a display instruction to the terminal device 30 to overlap and display the trajectory 500 corresponding to the shooting date and time indicated by the obtained instruction in the design drawing display column R3. Thus, the user can display the trajectory 500 corresponding to the selected shooting date and time in the design drawing display column R3. In addition, in the display screen G1 that is first displayed after the application is started, the trajectory 500 corresponding to the latest shooting date and time is displayed in the design drawing display column R3. Figure 3 In order to facilitate the description, the shooting location icon 202 is omitted, but in fact, Figure 2 The shooting location icon 202 is displayed as shown.

[0080] If an operation of selecting the first image displayed in the image information display column R1 is input, the design drawing display column R3 displays the first image. In this state, if an operation of selecting a second date and time that is different from the first date and time corresponding to the shooting date and time of the first image is input from the menu screen 300, the display control unit 111 displays the first image and the second image side by side in the design drawing display column R3. The second image is an image whose shooting location is closest to the first image among the multiple images shot at the second date and time. For example, the display control unit 111 calculates the distance between the coordinates of the shooting location icons of the multiple images shot at the second date and time and the coordinates of the shooting location icon of the first image, and determines the image corresponding to the shooting location icon with the shortest distance as the second image.

[0081] Figure 4 1 is a diagram showing an example of a first image G31 and a second image G32 displayed on a display. Figure 4 As shown, the first image G31 and the second image G32 are displayed in the design drawing display column R3. In this example, the design drawing originally displayed in the design drawing display column R3 is set to non-display. The design drawing display column R3 displays the first image G31 on the left side facing the observer, and displays the second image G32 on the right side. The circular mark at the center of the first image G31 is the viewpoint O1 of the first image G31. The circular mark at the center of the second image G32 is the viewpoint O2 of the second image G32. In this example, for the convenience of explanation, the circular marks representing the viewpoints O1 and O2 are illustrated, but in fact, these circular marks are not displayed. The viewpoint O1 is located at the center of the display area 301 of the first image G31, and the viewpoint O2 is located at the center of the display area 302 of the second image G32. The shooting dates of the first image G31 and the second image G32 are displayed below the first image G31 and the second image G32.

[0082] The display control unit 111 performs pattern matching using the first image G31 in the display area 301 as a template for the entire area of ​​the second image, thereby detecting corresponding points of the viewpoint O1 from the second image G32. The display control unit 111 sets the detected corresponding points as the viewpoint O2, sets a certain area centered on the viewpoint O2 as the display area 302, and displays the second image G32 included in the display area 302 on the display of the terminal device 30.

[0083] If the display area 302 is set with the default viewpoint as the center to display the second image G32, the subject that the user is interested in and included in the display area 301 of the first image G31 may not be displayed in the display area 302. In this case, in order to display the subject of interest in the display area 302, the user needs to scroll the second image G32, which takes time.

[0084] Therefore, in the present embodiment, the display area 302 is set with the viewpoint O2, which is the corresponding point of the viewpoint O1, as the center to display the second image G32. Thus, the second image G32 is displayed on the display so that the viewpoint O2 coincides with the viewpoint O1. As a result, even if the second image G32 is not scrolled, the subject included in the display area 301 can be observed through another line of sight. In addition, even if the subject of interest in the display area 301 is blocked by another subject, the user can observe the subject of interest in the display area 302 without scrolling the second image G32.

[0085] When a scroll instruction of the first image G31 is detected, the display control unit 111 may scroll the first image G31 according to the scroll instruction, and scroll the second image G32 in conjunction with the scroll of the first image G31. For example, when the operation amounts in the horizontal direction and the vertical direction indicated by the scroll instruction are Δx and Δy, respectively, the display control unit 111 may display the first image G31 and the second image G32 in conjunction by shifting the viewpoints of the first image G31 and the second image G32 by Δx and Δy, respectively. Furthermore, when a scroll instruction of the second image G32 is detected, the display control unit 111 may scroll the second image G32 according to the scroll instruction, and scroll the first image G31 and the second image G32 in conjunction with each other.

[0086] Figure 5: is a conceptual diagram of the change of the viewpoint in the present embodiment. The line of sight direction K1 is the direction from the shooting location P1 of the first image G31 toward the subject A corresponding to the viewpoint of the first image G31. The line of sight direction K2 is the direction from the shooting location P2 of the second image G32 toward the subject B corresponding to the default viewpoint of the second image G32. The trajectory 501 is the trajectory at the first date and time, and the trajectory 502 is the trajectory at the second date and time. The subject B is far away from the subject A, so if the display area 302 is set with the default viewpoint as the center to display the second image G32, the subject A may not be displayed in the display area 302 of the second image G32. Therefore, the display control unit 111 changes the default viewpoint of the second image G32 to the viewpoint of the first image G31, and sets the display area 302 with the changed viewpoint as the center to display the second image G32. Thus, the sight line direction K2 is changed to the sight line direction K2′, and the viewpoint of the second image G32 is changed to coincide with the viewpoint of the first image G31. As a result, the subject A can be displayed in the display area 302 of the second image G32.

[0087] Figure 6 It is shown Figure 1 Flowchart of an example of the processing of the information processing device 10 shown in FIG. 1 . The display control unit 111 obtains an instruction from the user to select a design drawing (step S1). In this case, a menu screen for selecting a design drawing is displayed on the display of the terminal device 30, and an instruction to select one design drawing from the menu screen is input. The input instruction is sent to the information processing device 10 via the network and received by the communication unit 13. The display control unit 111 obtains the instruction via the communication unit 13. The design drawing ID is included in the instruction, so the display control unit 111 can obtain the design drawing information representing the instructed design drawing from the design drawing information stored in the memory 12.

[0088] Next, the display control unit 111 displays the display screen G1 on the display of the terminal device 30 by sending the display instruction of the display screen G1 to the terminal device 30 via the communication unit 13 (step S2). The display instruction of the display screen G1 displayed by default includes the design drawing information indicating the design drawing selected in step S1 and the shooting information corresponding to the latest shooting date and time. Therefore, in the display screen G1 displayed by default, as shown in FIG. Figure 2As shown, the design drawing includes a selection icon 201, a shooting location icon 202 corresponding to the latest shooting date and time, and a track 203. At this point in time, no shooting location icon has been determined, so the image information display column R1 and the annotation information display column R2 are blank. In addition, when an instruction to select a shooting date and time is input in step S3 described later, the display control unit 111 displays a display screen G1 of the design drawing including the shooting location icon 202 and the track 203 corresponding to the selected shooting date and time in an overlapping manner on the display.

[0089] Next, the display control unit 111 determines whether an instruction from the user to select the shooting date and time is obtained (step S3). In this case, the menu screen 300 for selecting the shooting date and time is displayed on the display of the terminal device 30. The user inputs an instruction to select one shooting date and time from the menu screen 300. The shooting date and time displayed in the menu screen 300 is a representative value of the shooting date and time included in the shooting information stored in the memory 12. If the shooting date and time is selected, one shooting action corresponding to the shooting date and time is selected. The input instruction is sent to the information processing device 10 via the network and received by the communication unit 13. The display control unit 111 obtains the instruction via the communication unit 13. The representative value of the shooting date and time is included in the instruction, so the display control unit 111 can determine one shooting information stored in the memory 12. When the instruction to select the shooting date and time is input (yes in step S3), the process proceeds to step S4. When the instruction to select the shooting date and time is not input (no in step S3), the process returns to step S2.

[0090] Next, the display process is performed (step S4). The details of the display process will be shown in Figure 7 The display process is a process of displaying the first image and the second image side by side in the design drawing display area R3.

[0091] Next, the display control unit 111 determines whether an instruction to input an annotation has been acquired (step S5). The instruction to input an annotation is an instruction input when the user intends to input an annotation for the image displayed in the image information display field R1. The instruction is input, for example, by selecting an annotation input instruction button (not shown) displayed on the display screen G1. The input instruction is sent to the information processing device 10 via the network and received by the communication unit 13. The display control unit 111 acquires the instruction via the communication unit 13.

[0092] Next, when an instruction to input annotations is obtained (Yes in step S5), the display control unit 111 obtains annotation area information (step S6). The annotation area information is input by performing an operation such as moving and deforming a rectangular frame in the image information display bar R1. The input annotation area information is sent to the information processing device 10 via the network and received by the communication unit 13. The display control unit 111 obtains the annotation area information via the communication unit 13. The display control unit 111 assigns an annotation area ID to the obtained annotation area information and stores it in the memory 12 in correspondence with the shooting ID. As a result, Figure 3 As shown, the comment area D1 is set.

[0093] Next, when the instruction to input a comment is not acquired (NO in step S5 ), the process proceeds to step S8 .

[0094] Next, the display control unit 111 acquires annotation information (step S7). Figure 3 As shown, the annotation information is input by inputting the annotation C1 into the annotation information display field R2 and pressing the send button (not shown). The input annotation information is sent to the information processing device 10 via the network and received by the communication unit 13. The display control unit 111 obtains the annotation information via the communication unit 13. The display control unit 111 stores the acquired annotation information in the memory 12 in correspondence with the shooting ID and the annotation area ID.

[0095] Next, the display control unit 111 determines whether an end instruction has been obtained (step S8). The end instruction is an instruction to close the display screen G1 displayed in step S3. The instruction is input by pressing an end button (not shown) displayed on the display screen G1. When the end instruction is obtained (yes in step S8), the process ends. When the end instruction is not obtained (no in step S8), the process returns to step S3. In this case, the display of the display screen G1 is maintained. The end instruction is sent to the information processing device 10 via the network and received by the communication unit 13. The display control unit 111 obtains the end instruction via the communication unit 13.

[0096] Figure 7 It is shown Figure 6 The display control unit 111 obtains an instruction to select the first image (step S21). Figure 2 As shown, the instruction is performed by dragging and dropping the selection icon 201 on the design drawing. As described above, the display control unit 111 determines one shooting location icon selected by the user based on the release position of the selection icon 201 and the position of the shooting location icon 202, and detects the image corresponding to the one shooting location icon as the first image.

[0097] Next, the display control unit 111 displays the first image in the image information display field R1 (step S22). Next, the display control unit 111 determines whether an instruction to select the first image displayed in the image information display field R1 is obtained from the terminal device 30 (step S23). The instruction is given by tapping or clicking the first image displayed in the image information display field R1.

[0098] If an instruction to select the first image is obtained (Yes in step S23), the display control unit 111 displays the first image in the design display area R3 (step S24). On the other hand, if an instruction to select the first image is not obtained (No in step S23), the process returns to step S22.

[0099] Next, the display control unit 111 determines whether an instruction to select a second date and time has been acquired from the terminal device 30 (step S25). Figure 3 This is performed by selecting one of the shooting dates and times in the shooting date and time on the menu screen 300 shown.

[0100] When an instruction to select the second date and time is obtained (Yes in step S25), the display control unit 111 determines the image whose shooting location is closest to the first image among the multiple images shot at the second date and time as the second image (step S26). When an instruction to select the second date and time is not obtained (No in step S25), the process returns to step S22.

[0101] Next, the display control unit 111 changes the viewpoint of the second image so as to match the viewpoint of the first image (step S27). As described above, this process is performed by detecting corresponding points of the viewpoint of the first image from the second image through pattern matching. As a result, the default viewpoint of the second image is changed so as to match the viewpoint of the first image.

[0102] Next, the display control unit 111 sets a display area for the second image with the determined viewpoint as the center, and sends a display instruction to display the second image in the set display area to the terminal device 30, thereby displaying the second image on the display of the terminal device 30 (step S28). Figure 4 As shown in FIG. 1 , the second image G32 is displayed so that the subject included in the display area 301 of the first image G31 is included in the display area 302 of the second image G32. When the processing of step S28 is completed, the processing proceeds to step S34. Figure 6 Step S5.

[0103] Thus, according to the present embodiment, when displaying the second image captured at a different date and time from the first image, the second image is displayed with the default viewpoint changed so as to match the viewpoint of the first image. Therefore, the user can display the subject of interest in the second image without performing laborious operations such as scrolling the second image to display the subject included in the display area of ​​the first image on the display. Furthermore, since such scrolling operations are not required, the processing burden of the computer can be reduced.

[0104] In addition, the present disclosure can adopt the following modified examples.

[0105] (1) Figure 7 In step S27, sometimes the corresponding points of the viewpoint of the first image cannot be detected from the second image. For example, a subject such as a building under construction that should be included in the display area of ​​the first image is blocked by another subject such as a person or construction material. In this case, the display control unit 111 may also change the viewpoint of the first image so that the viewpoint of the first image is consistent with the default viewpoint of the second image. For example, the display control unit 111 only needs to detect the corresponding points of the default viewpoint of the second image from the first image, and set the display range of the first image with the detected corresponding points as the center to display the first image. In detail, the display control unit 111 only needs to detect the corresponding points corresponding to the default viewpoint of the second image from the first image by applying pattern matching using the display area of ​​the second image with the default viewpoint as the center as a template to the entire area of ​​the first image.

[0106] (2) The bird's-eye view may be a layout diagram showing the layout of a house. In this case, the present disclosure can be applied to the renovation of the interior decoration of a house. In addition, the bird's-eye view may be a layout diagram showing the layout of a house in a simple manner.

[0107] (3) In the above embodiment, a construction site is exemplified as the site, but the present disclosure is not limited thereto, and a manufacturing site, a logistics site, a distribution site, a farmland, a construction site, a small commodity market, an office, a hospital, a commercial facility, a nursing facility, etc. may be employed as the site.

[0108] (4) In the above embodiment, the second image G32 is an image obtained by photographing a given space at a second date and time, but the present disclosure is not limited to this. For example, the second image G32 may also be a virtual image generated by rendering a 3D model of the given space. The 3D model may also be a model generated based on 3D measurement data or a model generated based on BIM (Building Information Modeling) data. In this case, the date and time when the 3D model is photographed by the virtual camera becomes the second date and time.

[0109] Figure 8 3 is an explanatory diagram of the first image G31 and the second image G32 in the modified example of the present disclosure. Figure 8 In the example of , the viewpoint O2 indicates the viewpoint of the second image G32 before the viewpoint is changed. Figure 8 In the example of , the first image G31 is an image obtained by actually photographing the construction site, and the second image G32 is a virtual image obtained by rendering the 3D model of the same construction site. For example, when the display control unit 111 detects an instruction to select the second date and time after displaying the first image G31, the second image G32 is displayed so that the viewpoint O2 of the second image G32 coincides with the viewpoint O1 of the first image G31.

[0110] Furthermore, when a scroll operation is input in the first image G31 and the viewpoint O1 becomes the center of the display area 301, the display control unit 111 changes the viewpoint O2 of the second image G32 so as to coincide with the viewpoint O1. That is, the display control unit 111 scrolls the second image G32 in conjunction with the scrolling of the first image G31. Thus, the user can easily confirm whether the actual construction site situation is proceeding as in the virtual image.

[0111] The details of the processing in this modification are as follows. When the first image G31 is an omnidirectional image, a virtual camera composed of an omnidirectional camera captures a 3D model, thereby generating a second image G32 in advance and storing it in the memory 12. The display control unit 111 determines the viewpoint O2 of the second image G2 by applying pattern matching using the image of the display area 301 after the scroll operation as a template to the entire area of ​​the second image. In addition, the display control unit 111 only needs to set the display area 302 with the viewpoint O2 as the center to display the second image G2 on the display.

[0112] Industrial Applicability

[0113] The present disclosure can confirm the status of a construction site from a remote location, and is therefore useful in managing the construction site.

Claims

1. An information processing method is an information processing method in a computer. Displaying, on a display of an information terminal, a first image obtained by photographing a given space at a first date and time; When a selection instruction of a second date and time different from the first date and time is detected, displaying a second image obtained by photographing the given space at the second date and time on the display; The second image is displayed on the display by changing a default viewpoint so as to match a viewpoint of the first image.

2. The information processing method according to claim 1, in, Furthermore, the overhead image as the overhead image of the given space on which a plurality of shooting location icons indicating shooting locations at a first date and time are superimposed and displayed is displayed on the display. Further, detecting the selection of one shooting location icon from the plurality of shooting location icons, The first image is an image captured at the shooting location indicated by the one shooting location icon.

3. The information processing method according to claim 1 or 2, in, The second image is an image obtained by photographing the given space at a photographing location closest to a photographing location of the first image among a plurality of images photographed at the second date and time.

4. The information processing method according to claim 1 or 2, in, The change of the default viewpoint in the second image includes: Based on the first image, detecting corresponding points of the viewpoint of the first image from the second image; and The detected corresponding point is set as the viewpoint of the second image.

5. The information processing method according to claim 1 or 2, in, The viewpoint of the first image is the center of a display area of ​​the first image displayed on the display. The viewpoint of the second image is the center of a display area of ​​the second image displayed on the display.

6. The information processing method according to claim 1 or 2, in, Furthermore, when a scroll instruction of the first image is detected, the first image is scrolled according to the scroll instruction, and the second image is scrolled in conjunction with the scrolling of the first image.

7. The information processing method according to claim 1 or 2, in, Furthermore, when the corresponding point of the viewpoint of the first image cannot be detected from the second image, the corresponding point of the default viewpoint of the second image is detected from the first image, and the viewpoint of the first image is changed to the detected viewpoint and displayed on the display.

8. The information processing method according to claim 1 or 2, in, The given space is a work site.

9. The information processing method according to claim 1 or 2, in, The first image and the second image are displayed side by side on the display.

10. An information processing device, comprising a processor, The processor performs the following processing: Displaying, on a display of an information terminal, a first image obtained by photographing a given space at a first date and time; When a selection instruction of a second date and time different from the first date and time is detected, displaying a second image obtained by photographing the given space at the second date and time on the display; The second image is displayed on the display by changing a default viewpoint so as to match a viewpoint of the first image.

11. An information processing program for causing a computer to execute the information processing method according to claim 1 or 2.

Images