Information processing method, information processing device, and information processing program

Abstract

This information processing method includes: displaying a bird's-eye view on a display of an information terminal; and displaying, in the bird's-eye view, a plurality of imaging point icons indicating a series of consecutive imaging points from the imaging start point to the imaging end point. The plurality of imaging point icons are associated with a plurality of images captured at a plurality of imaging points. The display of the plurality of imaging point icons includes displaying, in the bird's-eye view in a case where there are a first plurality of imaging points and a second plurality of imaging points captured before the imaging date and time at the first plurality of imaging points, a third plurality of imaging point icons indicating a third plurality of imaging points within a predetermined range from each of the second plurality of imaging points from the imaging start point to the imaging end point among the first plurality of imaging points.

Description

Information processing method, information processing device, and information processing program

[0001] The present disclosure relates to techniques for displaying images.

[0002] For example, the technology of Patent Document 1 discloses a system in which a desired location on displayed construction drawing data is specified, a photography instruction is given to take an image of the construction site corresponding to the specified desired location, and placement data for the image of the construction site taken based on the photography instruction is linked to the desired location on the specified construction drawing data and displayed superimposed on the construction drawing data.

[0003] However, in the above-mentioned conventional technology, the location to be photographed is determined in advance, and images are photographed only at the determined location. Therefore, in the above-mentioned conventional technology, multiple images are not photographed consecutively, and the problem of an increase in the amount of image data is not taken into consideration.

[0004] International Publication No. 2022 / 145021

[0005] The present disclosure has been made to solve the above problems, and aims to provide a technique that can reduce the amount of image data.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, and includes displaying a bird's-eye view on a display of an information terminal, and displaying a plurality of shooting location icons on the bird's-eye view, each of which indicates a series of consecutive shooting locations from a shooting start location to a shooting end location, wherein the plurality of shooting location icons correspond to a plurality of images taken at the plurality of shooting locations, and the display of the plurality of shooting location icons includes, when the bird's-eye view includes a first plurality of shooting locations and a second plurality of shooting locations that were taken before the shooting dates and times at the first plurality of shooting locations, displaying a third plurality of shooting location icons that indicate a third plurality of shooting locations among the first plurality of shooting locations that are within a predetermined range from each of the second plurality of shooting locations from the shooting start location to the shooting end location.

[0007] According to the present disclosure, the amount of image data can be reduced.

[0008] 1 is a diagram showing the overall configuration of an information processing system in the present embodiment. FIG. 2 is a flowchart showing an example of processing by a communication device in the present embodiment. FIG. 3 is a first flowchart showing an example of image reduction processing by a server in the present embodiment. FIG. 4 is a second flowchart showing an example of image reduction processing by a server in the present embodiment. FIG. 5 is a diagram showing a second plurality of photography location icons drawn on a blueprint in the present embodiment. FIG. 6 is a diagram showing a first plurality of photography location icons and a second plurality of photography location icons drawn on a blueprint in the present embodiment. FIG. 7 is a diagram showing a third plurality of photography location icons extracted from the first plurality of photography location icons based on the second plurality of photography location icons in the present embodiment. FIG. 8 is a diagram showing an example of a display screen for accepting a change of a selected image corresponding to one selected photography location icon. FIG. 9 is a flowchart showing an example of image display processing by a server in the present embodiment.

[0009] (Findings that form the basis of the present disclosure) A user interface is being developed that displays multiple photography location icons indicating locations where actual photographs of a construction site were taken on a two-dimensional blueprint of the construction site, etc., and when one photography location icon is selected, displays an image taken at the photography location indicated by that icon. This allows a construction site manager to understand the situation at the construction site without visiting the construction site.

[0010] The photographer moves around the construction site while taking pictures of the site with a camera. The camera takes multiple images at a specified frame rate from the start to the end of the shoot. If the photographer walks slowly or stops, a large number of similar images may be saved, which may increase the amount of image data.

[0011] In the above-mentioned conventional technology, the location to be photographed is determined in advance, and images are photographed only at the determined location. Therefore, in the above-mentioned conventional technology, multiple images are not photographed consecutively, and the problem of an increase in the amount of image data is not taken into consideration.

[0012] In order to solve the above problems, the following techniques are disclosed.

[0013] (1) An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, comprising: displaying a bird's-eye view on a display of an information terminal; and displaying, on the bird's-eye view, a plurality of shooting location icons indicating a series of consecutive shooting locations from a shooting start location to a shooting end location, wherein the plurality of shooting location icons correspond to a plurality of images taken at the plurality of shooting locations, and the display of the plurality of shooting location icons includes, when the bird's-eye view includes a first plurality of shooting locations and a second plurality of shooting locations that were photographed before the shooting dates and times at the first plurality of shooting locations, displaying a third plurality of shooting location icons indicating a third plurality of shooting locations that are within a predetermined range from each of the second plurality of shooting locations from the shooting start location to the shooting end location, among the first plurality of shooting locations.

[0014] According to this configuration, when a bird's-eye view includes a first plurality of photographing locations and a second plurality of photographing locations that were photographed before the photographing date and time at the first plurality of photographing locations, a third plurality of images taken at a third plurality of photographing locations that are within a predetermined range from each of the second plurality of photographing locations from the photographing start location to the photographing end location among the first plurality of photographing locations are retained, and other images taken at other photographing locations are deleted, thereby reducing the amount of image data.

[0015] (2) In the information processing method described in (1) above, displaying the plurality of photographing location icons may include drawing a first plurality of photographing location icons indicating the first plurality of photographing locations and a second plurality of photographing location icons indicating the second plurality of photographing locations on the bird's-eye view, and displaying only the third plurality of photographing location icons among the first plurality of photographing location icons that overlap with each of the second plurality of photographing location icons from the photographing start location to the photographing end location.

[0016] According to this configuration, only the third plurality of images corresponding to the third plurality of shooting location icons that overlap with each of the second plurality of shooting location icons from the shooting start point to the shooting end point are left among the first plurality of shooting location icons, and other images corresponding to the other shooting location icons are deleted, thereby reducing the amount of image data.

[0017] (3) In the information processing method described in (2) above, displaying the plurality of shooting location icons may include displaying only the third plurality of shooting location icons among the first plurality of shooting location icons that have the largest overlapping area with each of the second plurality of shooting location icons from the shooting start point to the shooting end point.

[0018] According to this configuration, only the third plurality of images corresponding to the third plurality of shooting location icons among the first plurality of shooting location icons that have the largest overlapping area with each of the second plurality of shooting location icons from the shooting start point to the shooting end point are left, and other images corresponding to the other shooting location icons are deleted, thereby reducing the amount of image data.

[0019] (4) In the information processing method described in (1) above, displaying the plurality of photographing location icons may include displaying the third plurality of photographing location icons that indicate only the third plurality of photographing locations among the first plurality of photographing locations that are closest to each of the second plurality of photographing locations from the photographing start location to the photographing end location.

[0020] According to this configuration, only the third plurality of images taken at the third plurality of photographing points that are closest to each of the second plurality of photographing points from the photographing start point to the photographing end point among the first plurality of photographing points are kept, and other images taken at other photographing points are deleted, thereby reducing the amount of image data.

[0021] (5) In the information processing method described in any one of (1) to (4) above, the method may further include storing in memory only a third plurality of images, among the first plurality of images taken at the first plurality of shooting locations, that are taken at a third plurality of shooting locations within a predetermined range from each of the second plurality of shooting locations from the shooting start location to the shooting end location, and deleting the other images.

[0022] According to this configuration, of the first plurality of images taken at the first plurality of shooting locations, only the third plurality of images taken at the third plurality of shooting locations within a predetermined range from each of the second plurality of shooting locations from the shooting start location to the shooting end location are stored in memory, and the other images are deleted, thereby reducing the amount of image data.

[0023] (6) In the information processing method described in any one of (1) to (4) above, the method may further include creating copies of the first plurality of images taken at the first plurality of shooting locations, and storing in memory only the third plurality of images, among the copies of the first plurality of images, that are taken at the third plurality of shooting locations within a predetermined range from each of the second plurality of shooting locations from the shooting start location to the shooting end location, and deleting the other images.

[0024] With this configuration, other images are deleted from the copies of the first plurality of images, so that if the first plurality of images needs to be modified, the first plurality of images can be modified using the originals of the first plurality of images.

[0025] (7) In the information processing method described in any one of (1) to (6) above, when a change by a user to at least one of the shooting start point and the shooting end point is accepted, a series of the first plurality of consecutive shooting points from the shooting start point to the shooting end point on the bird's-eye view may be created based on the bird's-eye view, the plurality of images, the shooting start point after the change has been accepted, and the shooting end point after the change has been accepted.

[0026] If the user registers an incorrect location as the shooting start point and the shooting end point, the trajectory of the first plurality of shooting points may deviate from the actual trajectory. However, if a change by the user to at least one of the shooting start point and the shooting end point is accepted, the first plurality of shooting points are created based on the bird's-eye view, the plurality of images, the shooting start point after accepting the change, and the shooting end point after accepting the change, so that the deviation of the trajectory of the first plurality of shooting points can be corrected.

[0027] (8) In the information processing method described in any one of (1) to (7) above, when a user's selection of one of the displayed third plurality of shooting location icons is accepted, the method may further include displaying a selected image corresponding to the selected one shooting location icon, a first candidate image taken before the selected image, and a second candidate image taken after the selected image, when an instruction to change the selected image to the first candidate image or the second candidate image is accepted, changing the selected image corresponding to the one shooting location icon to the first candidate image or the second candidate image.

[0028] The images associated with the thinned third plurality of photographing locations may be misaligned with images photographed at the same photographing locations in the past. However, when an instruction to change the selected image corresponding to one photographing location icon to a first candidate image photographed before the selected image or a second candidate image photographed after the selected image is accepted, the selected image corresponding to one photographing location icon is changed to the first candidate image or the second candidate image, so that an optimal image can be associated with one photographing location icon.

[0029] Furthermore, the present disclosure can be realized not only as an information processing method that executes the characteristic processes described above, but also as an information processing device having a characteristic configuration corresponding to the characteristic processes executed by the information processing method. Furthermore, the present disclosure can also be realized as a computer program that causes a computer to execute the characteristic processes included in such an information processing method. Therefore, the same effects as those of the above information processing method can also be achieved in the following other aspects.

[0030] (9) An information processing device according to another aspect of the present disclosure is an information processing device including a processor, wherein the processor displays a bird's-eye view on a display of an information terminal, and displays on the bird's-eye view a plurality of shooting location icons indicating a series of consecutive shooting locations from a shooting start location to a shooting end location, the plurality of shooting location icons corresponding to a plurality of images taken at the plurality of shooting locations, and when the bird's-eye view includes a first plurality of shooting locations and a second plurality of shooting locations that were photographed before the shooting date and time at the first plurality of shooting locations, a third plurality of shooting location icons indicating a third plurality of shooting locations that are within a predetermined range from each of the second plurality of shooting locations from the shooting start location to the shooting end location among the first plurality of shooting locations are displayed.

[0031] (10) An information processing program according to another aspect of the present disclosure causes a computer to display a bird's-eye view on the display of an information terminal and to display, on the bird's-eye view, a plurality of shooting location icons indicating a series of consecutive shooting locations from a shooting start location to a shooting end location, the plurality of shooting location icons corresponding to a plurality of images taken at the plurality of shooting locations, and when the bird's-eye view includes a first plurality of shooting locations and a second plurality of shooting locations that were photographed before the shooting date and time at the first plurality of shooting locations, a third plurality of shooting location icons indicating a third plurality of shooting locations that are within a predetermined range from each of the second plurality of shooting locations from the shooting start location to the shooting end location among the first plurality of shooting locations are displayed.

[0032] (11) A non-transitory computer-readable recording medium according to another aspect of the present disclosure records the information processing program described in (10) above.

[0033] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components. Furthermore, in all embodiments, the respective contents can be combined.

[0034] FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to the present embodiment.

[0035] The information processing system 1 is a system that displays a bird's-eye view of a workspace on a display 23 of an information terminal 20, and also displays, on the bird's-eye view, a plurality of photography location icons that indicate a series of a plurality of consecutive photography locations from a photography start location to a photography end location. The information processing system 1 includes a server 10, an information terminal 20, a photography device 30, and a communication device 40.

[0036] The server 10 is an example of an information processing device and a computer. The server 10, the information terminal 20, and the communication device 40 are connected to each other so as to be able to communicate with each other via a network NT. An example of the network NT is the Internet. The server 10 is, for example, a cloud server configured with one or more computers. However, this is just one example, and the server 10 may be configured as an edge server or may be implemented in the information terminal 20. The aspect in which the server 10 is implemented in the information terminal 20 is an example of the aspect in which the information terminal 20 is configured as an information processing device.

[0037] The information terminal 20 is carried by a user. The user is, for example, a manager of a predetermined space in which the image capturing device 30 captures images. The predetermined space is, for example, a construction site. However, this is just one example, and the predetermined space may be a construction site, a factory, a store, an office, or the like. The information terminal 20 may be configured as a portable computer such as a smartphone or a tablet computer, or as a stationary computer. The information terminal 20 displays images on a display 23 under control of the server 10. Although one information terminal 20 is illustrated in the example of FIG. 1, multiple information terminals may be connected to the server 10 via a network NT. The information terminal 20 includes a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0038] The communication unit 21 is a communication interface that connects the information terminal 20 to the network NT. The communication unit 21 transmits instruction signals indicating various instructions received from the user by the operation unit 24 to the server 10. The communication unit 21 receives display data for displaying various display screens from the server 10.

[0039] The processor 22 is configured by, for example, a central processing unit (CPU), and displays a display screen indicated by the display data received by the communication unit 21 on the display 23 .

[0040] The display 23 is configured with various display devices such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays various display screens under the control of the processor 22.

[0041] The operation unit 24 is composed of, for example, a keyboard, a touch panel, a mouse, and the like, and accepts various instructions input by the user.

[0042] The communication device 40 is configured by a mobile information terminal such as a smartphone or a tablet computer, and connects the photographing device 30 to the network NT. The communication device 40 and the photographing device 30 are connected to each other so that they can communicate with each other via a short-range wireless communication path such as Bluetooth (registered trademark).

[0043] The image capturing device 30 is, for example, an omnidirectional camera, and captures images at a predetermined frame rate. An omnidirectional camera is also called a 360-degree camera, and is a camera that can capture images in all directions of 360 degrees. The image capturing device 30 is, for example, a portable image capturing device carried by a photographer. The photographer may be, for example, a worker or a site supervisor at a construction site. The image capturing device 30 may be held in the photographer's hand, or may be worn on the photographer's body (for example, on the head). The image capturing device 30 may also be a regular camera.

[0044] The photographer moves through the construction site while photographing the construction site with the camera device 30. At the start point of photography, the photographer points the camera device 30 in a northerly direction and presses the camera button on the camera device 30 to start photographing. When the photographer reaches the end point of photography, he presses the camera button again to stop photographing. When photography is finished, the camera device 30 transmits the series of captured images (first plurality of images) to the communication device 40.

[0045] The image includes the date and time of the image capture. The date and time of the image capture is obtained, for example, by a clock provided in the image capture device 30. In this example, the image capture device 30 captures images at a predetermined frame rate (e.g., 10 frames per second), so the image capture location is defined in frame cycle units. However, this would result in an enormous amount of data, so the image capture location may be defined every predetermined time period (e.g., 1 second, 5 seconds, or 10 seconds).

[0046] The communication device 40 receives the first plurality of images transmitted by the image capturing device 30. The communication device 40 also displays a blueprint of the construction site and accepts designation by the photographer of the position of the image capturing start point and the position of the image capturing end point on the blueprint.

[0047] The start and end points of photography are specified by the photographer inputting instructions specifying the positions on the blueprint screen of the site displayed on the display of the communication device 40. Two-dimensional coordinate axes are defined on the blueprint screen. Therefore, the start and end points of photography are each defined by two-dimensional coordinate values. The start and end points of photography are used by the server 10 to specify the position of each photography point and the photography direction of each image.

[0048] The communication device 40 transmits photography information including the first plurality of captured images to the server 10 via the network NT. The photography information is generated each time a photography operation is performed. One photography operation refers to a series of operations from when a worker holding the photography device 30 starts taking pictures at a construction site until when he finishes taking pictures. A single photography operation captures multiple images. The photography information includes the first plurality of captured images and meta information for each of the first plurality of images. The meta information includes a photography ID, a photography date and time, and a blueprint ID. The photography ID is an identifier for identifying the photography operation. The photography date and time is the date and time the image was taken. The blueprint ID is an identifier for identifying a blueprint of a specified space corresponding to the image.

[0049] The server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 is configured, for example, by a central processing unit (CPU). The processor 11 includes an acquisition unit 111, an image processing unit 112, and a display control unit 113. The acquisition unit 111, the image processing unit 112, and the display control unit 113 may be realized by the processor 11 executing an information processing program, or may be configured by a dedicated hardware circuit such as an ASIC. The information processing program may be recorded on a non-transitory computer-readable recording medium.

[0050] The communication unit 13 is a communication interface that connects the server 10 to the network NT. The communication unit 13 receives shooting information transmitted by the communication device 40. The communication unit 13 receives instruction signals indicating various instructions received from the user from the information terminal 20. The communication unit 13 transmits display data for displaying various display screens to the information terminal 20.

[0051] The memory 12 is configured as a non-volatile rewritable storage device such as a hard disk drive or a solid state drive, and includes a design drawing information storage unit 121, a first image information storage unit 122, and a second image information storage unit 123.

[0052] The blueprint information storage unit 121 stores blueprint information. Blueprint information is image information showing a blueprint of a specified space. A blueprint ID that identifies the blueprint is associated with the blueprint information. A blueprint is an example of a bird's-eye view. A blueprint is a drawing showing the design of a construction site, and may be a floor plan, blueprint, map, perspective view, or the like of the construction site. A bird's-eye view can also be called a bird's-eye view, and may be a view from above or a view from a high place.

[0053] The acquisition unit 111 creates a series of first multiple consecutive shooting locations from the shooting start location to the shooting end location on the blueprint based on the blueprint, the first multiple images, the shooting start location, and the shooting end location. The acquisition unit 111 identifies the location of the shooting location of each of the first multiple images by using Visual SLAM (Simultaneous Localization and Mapping) technology from the shooting start location, shooting end location, and first multiple images included in the shooting information received by the communication unit 13. The location of the shooting location is expressed as two-dimensional coordinate values ​​on the blueprint. The acquisition unit 111 also identifies the shooting direction of each of the first multiple images using Visual SLAM technology. The shooting direction of each image is expressed, for example, as a three-dimensional polar coordinate vector.

[0054] The acquisition unit 111 creates first image information based on the photography information and blueprints received by the communication unit 13. The first image information is information that associates the first plurality of images, before thinning, taken at the first plurality of photography locations with meta information for each image. The meta information includes a photography ID, photography date and time, photography direction, photography location, and blueprint ID. The photography direction is the photography direction of the photography device 30 that took the image. The photography location is location information (two-dimensional coordinate values) that indicates the photography location of the image.

[0055] The first image information storage unit 122 stores the first image information. The acquisition unit 111 stores the created first image information in the first image information storage unit 122.

[0056] The second image information storage unit 123 stores second image information. The second image information is information that associates a second plurality of images, which have been thinned out and taken at a second plurality of photographing locations that were taken before the photographing date and time at the first plurality of photographing locations, with meta information for each image. The number of the second plurality of photographing locations is less than the number of the first plurality of photographing locations. The meta information includes a photographing ID, a photographing date and time, a photographing direction, a photographing location, and a blueprint ID. The photographing direction is the photographing direction of the photographing device 30 that took the image. The photographing location is location information (two-dimensional coordinate values) that indicates the photographing location of the image. The second image information is created by the image processing unit 112, which will be described later.

[0057] Furthermore, the communication unit 13 receives an instruction to specify the shooting date and time from the information terminal 20. The operation unit 24 of the information terminal 20 accepts the selection of the shooting date and time by the user, and the communication unit 21 transmits an instruction to specify the shooting date and time selected by the user to the server 10. The instruction to specify the shooting date and time includes the shooting ID.

[0058] The acquisition unit 111 acquires, from the design drawing information storage unit 121, a design drawing corresponding to a design drawing ID included in the instruction to display the design drawing received by the communication unit 13. The acquisition unit 111 also acquires, from the second image information storage unit 123, a second plurality of shooting locations corresponding to a shooting ID included in the instruction to specify the shooting date and time received by the communication unit 13.

[0059] After creating the first image information, the acquisition unit 111 acquires second image information similar to the first image information from the second image information storage unit 123. Second image information similar to the first image information refers to second image information whose shooting start point and shooting end point are substantially the same as those of the first image information. For example, the acquisition unit 111 acquires second image information from the second image information storage unit 123 whose shooting start point and shooting end point are within a predetermined range from the shooting start point and shooting end point included in the first image information. The predetermined range is, for example, a circle with a radius of 2 meters.

[0060] The acquisition unit 111 determines whether second image information similar to the first image information exists. If second image information similar to the first image information does not exist, the image processing unit 112 accepts the user's selection of images to be retained from the first plurality of images. The image processing unit 112 displays the blueprint on the display 23 and superimposes a first plurality of photography location icons indicating the first plurality of photography locations on the blueprint. The user selects a plurality of photography location icons to be retained as the second plurality of photography locations from the first plurality of photography location icons displayed on the display 23. At this time, the user may select a plurality of photography location icons while checking the images corresponding to the photography location icons. The image processing unit 112 retains only a plurality of images corresponding to the plurality of photography location icons selected by the user from the first plurality of images corresponding to the first plurality of photography location icons as the second plurality of images, and deletes the other images. The image processing unit 112 then stores second image information in the second image information storage unit 123, which associates the second plurality of images with meta information for each image.

[0061] On the other hand, if there is second image information similar to the first image information, the image processing unit 112 performs a process of thinning out the first plurality of images included in the first image information based on the second plurality of shooting locations included in the second image information. That is, the image processing unit 112 stores, among the first plurality of images taken at the first plurality of shooting locations, only the third plurality of images taken at third plurality of shooting locations within a predetermined range from each of the second plurality of shooting locations from the shooting start location to the shooting end location in the second image information storage unit 123 of the memory 12, and deletes the other images.

[0062] First, the image processing unit 112 draws, on the blueprint, a first plurality of photography location icons indicating a first plurality of photography locations and a second plurality of photography location icons indicating a second plurality of photography locations. The image processing unit 112 stores, in the second image information storage unit 123 of the memory 12, only a third plurality of images corresponding to a third plurality of photography location icons that overlap with each of the second plurality of photography location icons from the photography start point to the photography end point, among the first plurality of photography location icons, and deletes the other images. More specifically, the image processing unit 112 stores, in the second image information storage unit 123 of the memory 12, only a third plurality of images corresponding to a third plurality of photography location icons that overlap with each of the second plurality of photography location icons from the photography start point to the photography end point, among the first plurality of photography location icons, the largest area, and deletes the other images corresponding to the other photography location icons.

[0063] The image processing unit 112 creates copies of the first plurality of images captured at the first plurality of photographing locations by creating copies of the first image information stored in the first image information storage unit 122. Then, the image processing unit 112 stores, among the copies of the first plurality of images, only the third plurality of images captured at third plurality of photographing locations within a predetermined range from each of the second plurality of photographing locations from the photographing start location to the photographing end location in the second image information storage unit 123 of the memory 12, and deletes the other images. At this time, the image processing unit 112 leaves, in the copies of the first plurality of images, only the third plurality of images corresponding to the third plurality of photographing location icons that overlap the largest area with each of the second plurality of photographing location icons from the photographing start location to the photographing end location, and deletes the other images corresponding to the other photographing location icons from the copies of the first plurality of images. Then, the image processing unit 112 stores the thinned copies of the first plurality of images as the second plurality of images in the second image information storage unit 123.

[0064] In the present embodiment, the acquisition unit 111 determines whether or not there is second image information similar to the first image information, but the present disclosure is not particularly limited to this. The acquisition unit 111 may accept a user's selection of second image information that was taken in the same location as the created first image information from among the plurality of second image information.

[0065] The display control unit 113 displays a blueprint of the workspace on the display 23 of the information terminal 20. The display control unit 113 displays the blueprint acquired by the acquisition unit 111 on the display 23 of the information terminal 20. The display control unit 113 transmits display data of the blueprint to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data transmitted by the server 10. The display 23 displays the blueprint received by the communication unit 21.

[0066] The display control unit 113 displays, on the blueprint, multiple photography location icons indicating a series of multiple consecutive photography locations from the photography start point to the photography end point. The multiple photography location icons correspond to multiple images photographed at the multiple photography locations. If second image information similar to the first image information exists, i.e., if the blueprint includes the first multiple photography locations and second multiple photography locations photographed before the photography dates and times at the first multiple photography locations, the display control unit 113 displays third multiple photography location icons indicating third multiple photography locations among the first multiple photography locations that are within a predetermined range from each of the second multiple photography locations from the photography start point to the photography end point. In this case, the display control unit 113 does not display multiple photography location icons indicating multiple photography locations among the first multiple photography locations that are not within a predetermined range from each of the second multiple photography locations from the photography start point to the photography end point.

[0067] Furthermore, even if the current shooting location is within a predetermined range from the previous shooting location, the display control unit 113 may not display the current shooting location icon if the current shooting location icon extends beyond the predetermined range or if the current shooting location icon overlaps with another shooting location icon.

[0068] The display control unit 113 draws, on the blueprint, a first plurality of photography location icons indicating the first plurality of photography locations and a second plurality of photography location icons indicating the second plurality of photography locations. The display control unit 113 displays only a third plurality of photography location icons, among the first plurality of photography location icons, that overlap with each of the second plurality of photography location icons from the photography start location to the photography end location.

[0069] The display control unit 113 displays only the third plurality of photographing location icons among the first plurality of photographing location icons that have the largest overlapping area with each of the second plurality of photographing location icons from the photographing start point to the photographing end point.

[0070] In this embodiment, the start and end shooting points are determined by the user registering positions on the blueprint. Therefore, if the user registers incorrect locations as the start and end shooting points, the trajectory of the first plurality of shooting points may deviate from the actual trajectory. Therefore, the display control unit 113 may accept a user change to at least one of the start and end shooting points. When a user change to at least one of the start and end shooting points is accepted, the image processing unit 112 may create a series of consecutive first plurality of shooting points from the start and end shooting points on the blueprint based on the blueprint, the first plurality of images, the start and end shooting points after accepting the change, and the end shooting point after accepting the change. Then, the image processing unit 112 changes the first plurality of shooting points associated with the first plurality of images of the first image information stored in the first image information storage unit 122 to the newly created first plurality of shooting points.

[0071] The display control unit 113 may also accept a user's selection of one of the third plurality of displayed photographing location icons. When a user's selection of one of the third plurality of displayed photographing location icons is accepted, the display control unit 113 may display a selected image corresponding to the selected photographing location icon, a first candidate image taken before the selected image, and a second candidate image taken after the selected image. In this embodiment, the images associated with the thinned third plurality of photographing locations may be misaligned with images previously taken at the same photographing location. Therefore, the user may change the selected image corresponding to one photographing location icon to the first candidate image or the second candidate image of another frame. The display control unit 113 may accept a user's instruction as to whether to change the selected image to the first candidate image or the second candidate image. When an instruction to change the selected image to the first candidate image or the second candidate image is accepted, the display control unit 113 may change the selected image corresponding to one photographing location icon to the first candidate image or the second candidate image. For example, the first candidate image is an image taken one frame before the selected image, and the second candidate image is an image taken one frame after the selected image.

[0072] In this embodiment, the display control unit 113 may display a third plurality of image capture location icons indicating only a third plurality of image capture locations that are closest to each of the second plurality of image capture locations from the image capture start location to the image capture end location, among the first plurality of image capture locations. In this case, the image processing unit 112 may store only the third plurality of images corresponding to the third plurality of image capture locations that are closest to each of the second plurality of image capture locations from the image capture start location to the image capture end location, among the first plurality of image capture locations, in the second image information storage unit 123 of the memory 12, and may delete the other images.

[0073] Furthermore, the image processing unit 112 may delete, at any timing, the first image information stored in the first image information storage unit 122. For example, the image processing unit 112 may delete the first image information stored in the first image information storage unit 122 after at least one of the shooting start point and the shooting end point is changed, or after the selected image corresponding to one shooting location icon is changed to the first candidate image or the second candidate image. Furthermore, the image processing unit 112 may delete the first image information stored in the first image information storage unit 122 when receiving an instruction from the user to delete the first image information.

[0074] First, the processing of the communication device 40 in this embodiment will be described.

[0075] FIG. 2 is a flowchart showing an example of processing performed by the communication device 40 in this embodiment.

[0076] First, in step S11, the communication device 40 displays the design drawing selected by the photographer. At this time, the communication device 40 accepts the selection of the design drawing by the photographer.

[0077] Next, in step S12, the communication device 40 determines whether the start point registration button has been pressed. At this time, the communication device 40 displays the start point registration button together with the design drawing. The communication device 40 accepts the photographer's input of the start point position on the design drawing, and also accepts the photographer's pressing of the start point registration button. For example, the photographer touches the start point position on the displayed design drawing. This inputs the start point position. The photographer then presses the start point registration button.

[0078] After pressing the start point registration button, the photographer presses the shooting button on the photographing device 30 to start photographing an image with the photographing device 30 .

[0079] If it is determined that the start point registration button has not been pressed (NO in step S12), the process of step S12 is repeated.

[0080] On the other hand, if it is determined that the start point registration button has been pressed (YES in step S12), in step S13, the communication device 40 stores the photographing start point on the design drawing. The communication device 40 stores the coordinates of the photographing start point on the design drawing.

[0081] Next, in step S14, the communication device 40 determines whether the first plurality of images have been received from the photographing device 30. When the photographer reaches the photographing end point, the photographer presses the photographing button on the photographing device 30 again to end photographing. When the photographing button is pressed, the photographing device 30 transmits the first plurality of photographed images to the communication device 40.

[0082] If it is determined that the first plurality of images have not been received from the image capturing device 30 (NO in step S14), the process of step S14 is repeated.

[0083] On the other hand, if it is determined that the first plurality of images have been received from the photographing device 30 (YES in step S14), the communication device 40 stores the received first plurality of images in step S15.

[0084] Next, in step S16, the communication device 40 determines whether the end point registration button has been pressed. At this time, the communication device 40 displays the end point registration button together with the design drawing. The communication device 40 accepts the photographer's input of the end point position on the design drawing, and also accepts the photographer's pressing of the end point registration button. For example, the photographer touches the end point position on the displayed design drawing. This inputs the end point position. The photographer then presses the end point registration button.

[0085] If it is determined that the end point registration button has not been pressed (NO in step S16), the process of step S16 is repeated.

[0086] On the other hand, if it is determined that the end point registration button has been pressed (YES in step S16), in step S17, the communication device 40 stores the photographing end point on the design drawing. The communication device 40 stores the coordinates of the photographing end point on the design drawing.

[0087] Next, in step S18 , the communication device 40 transmits the photographic information including the first plurality of images to the server 10 .

[0088] Next, the image reduction process of the server 10 in this embodiment will be described.

[0089] FIG. 3 is a first flowchart showing an example of the image reduction process of the server 10 in this embodiment, and FIG. 4 is a second flowchart showing an example of the image reduction process of the server 10 in this embodiment.

[0090] First, in step S31 , the communication unit 13 receives the photographing information transmitted by the communication device 40 .

[0091] Next, in step S32, the acquisition unit 111 acquires a design drawing from the design drawing information storage unit 121. The acquisition unit 111 reads out from the design drawing information storage unit 121 a design drawing corresponding to a design drawing ID included in the shooting information received by the communication unit 13.

[0092] Next, in step S33, the acquisition unit 111 creates first image information based on the photography information and the blueprint. The acquisition unit 111 calculates the photography location and photography direction of each of the first plurality of images on the blueprint using Visual SLAM technology. The first image information is information that associates the first plurality of images, before thinning, captured by the photography device 30 with meta information for each image. The meta information includes a photography ID, photography date and time, photography direction, photography location, and blueprint ID.

[0093] Next, in step S34 , the image processing unit 112 stores the first image information created by the acquisition unit 111 in the first image information storage unit 122 .

[0094] Next, in step S35, the image processing unit 112 acquires from the second image information storage unit 123 second image information whose shooting start point and shooting end point are within a predetermined range from the shooting start point and shooting end point of the first image information.

[0095] If no second image information exists in the second image information storage unit 123, the image processing unit 112 accepts a user's selection of images to be retained from the first plurality of images. At this time, the image processing unit 112 displays the blueprint on the display 23 and superimposes a first plurality of photography location icons indicating the first plurality of photography locations on the blueprint. The image processing unit 112 retains, as the second plurality of images, only the images corresponding to the first plurality of photography location icons selected by the user from the first plurality of images corresponding to the first plurality of photography location icons, and deletes the other images. The image processing unit 112 then stores, in the second image information storage unit 123, second image information that associates the second plurality of images with meta information for each image.

[0096] Furthermore, when multiple pieces of second image information corresponding to multiple shooting dates and times are stored and the shooting start and end points are close to each other, the image processing unit 112 may acquire the second image information corresponding to the oldest shooting date and time, or the second image information corresponding to the most recent shooting date and time.

[0097] Next, in step S 36 , the image processing unit 112 creates a copy of the first image information stored in the first image information storage unit 122 .

[0098] Next, in step S37, the image processing unit 112 draws a second plurality of shooting location icons indicating a second plurality of shooting locations on the design drawing based on the second image information acquired from the second image information storage unit 123.

[0099] FIG. 5 is a diagram showing a second plurality of photography location icons drawn on a design drawing in this embodiment.

[0100] A second plurality of photographing location icons 211 are superimposed on the blueprint 201 shown in FIG. 5 . The second plurality of photographing location icons 211 are icons indicating photographing locations where photographs were taken in the past. Each of the second plurality of photographing location icons 211 is associated with an image photographed at the photographing location. In this example, each of the second plurality of photographing location icons 211 is composed of a circular image and is equivalent in size to a circle with a diameter of 1 meter. The second plurality of photographing location icons 211 displayed here correspond to a plurality of photographing locations belonging to a single photographing operation. The route from the leading photographing location icon (the photographing location icon at the bottom of FIG. 5 ) to the trailing photographing location icon (the photographing location icon at the top right of FIG. 2 ) indicates the trajectory followed by the photographer during this photographing operation.

[0101] Returning to FIG. 3, next, in step S38, the image processing unit 112 draws a first plurality of photography location icons indicating a first plurality of photography locations on the design drawing based on the created copy of the first image information.

[0102] FIG. 6 is a diagram showing a first plurality of photography location icons and a second plurality of photography location icons drawn on a design drawing in this embodiment.

[0103] 6 has a second plurality of photography location icons 211 superimposed thereon, and a first plurality of photography location icons 221 superimposed thereon. The shape and size of each of the first plurality of photography location icons 221 are the same as those of the second plurality of photography location icons 211. Furthermore, the photography start points and photography end points of the first plurality of photography location icons 221 are substantially the same as those of the second plurality of photography location icons 211.

[0104] The image capturing device 30 captures images at a predetermined frame rate. In this case, the number of images is expressed as the frame rate (fps) * the capture time (seconds). In FIG. 6 , the first plurality of images is not thinned out, and the number of the first plurality of image capturing location icons 221 is not changed. Therefore, the first plurality of image capturing location icons 211 are displayed overlapping each other along the trajectory of the photographer's movement.

[0105] Returning to Figure 3, next, in step S39, the image processing unit 112 extracts a third plurality of shooting location icons from the first plurality of shooting location icons that have the largest overlapping area with each of the second plurality of shooting location icons from the shooting start point to the shooting end point.

[0106] FIG. 7 is a diagram showing a third plurality of photography location icons extracted from the first plurality of photography location icons based on the second plurality of photography location icons in this embodiment.

[0107] 7 has a second plurality of photography location icons 211 superimposed thereon, and also a third plurality of photography location icons 231 superimposed thereon that have been thinned out from the first plurality of photography location icons 221. Of the first plurality of photography location icons 221, the third plurality of photography location icons 231 that overlap with the second plurality of photography location icons 211 respectively are extracted. The number of the third plurality of photography location icons 231 is the same as the number of the second plurality of photography location icons 211.

[0108] Returning to Figure 3, next, in step S40, the image processing unit 112 leaves only the third plurality of images corresponding to the extracted third plurality of shooting location icons in the copy of the first image information, and deletes the other images from the copy of the first image information.

[0109] Next, in step S41, the image processing unit 112 stores a copy of the first image information including the thinned third plurality of images in the second image information storage unit 123 as second image information.

[0110] Next, in step S42, the display control unit 113 acquires a third plurality of photography locations corresponding to the third plurality of photography location icons extracted from the first plurality of photography location icons.

[0111] Next, in step S43, the display control unit 113 sends a display instruction to the information terminal 20 via the communication unit 13 to display third plurality of photography location icons indicating the third plurality of photography locations on the design drawing, thereby displaying the third plurality of photography location icons indicating the third plurality of photography locations on the design drawing displayed on the display 23 of the information terminal 20. At this time, the display control unit 113 displays only the third plurality of photography location icons, among the first plurality of photography location icons, that overlap the largest areas with each of the second plurality of photography location icons from the photography start point to the photography end point.

[0112] The display control unit 113 may display, for example, the design drawing 201 on the display screen with the third plurality of photography location icons 231 superimposed thereon as shown in Fig. 7. The display screen displays the third plurality of photography location icons 231 indicating the third plurality of photography locations on the design drawing 201.

[0113] Next, in step S44, the display control unit 113 sends a display instruction to the information terminal 20 via the communication unit 13 to display images of each of the start and end shooting points among the third plurality of shooting points, thereby displaying images of each of the start and end shooting points among the third plurality of shooting points on the display 23 of the information terminal 20.

[0114] The user then checks the images of the start and end points displayed and determines whether or not it is necessary to change either the start or end point of photography. The display control unit 113 accepts the change of either the start or end point of photography on the blueprint.

[0115] Next, in step S45, the display control unit 113 determines whether a change to at least one of the start and end shooting points has been accepted. If it is determined that a change to at least one of the start and end shooting points has been accepted (YES in step S45), in step S46, the image processing unit 112 changes at least one of the start and end shooting points. The image processing unit 112 changes at least one of the start and end shooting points of the first plurality of shooting points included in the first image information stored in the first image information storage unit 122. Then, the processing returns to step S33, and the acquisition unit 111 recreates the first image information based on the design drawing, the first plurality of images, the changed start and end shooting points, and the changed end shooting point. The acquisition unit 111 recreates a series of consecutive first plurality of shooting points from the start and end shooting points on the design drawing.

[0116] On the other hand, if it is determined that a change to at least one of the shooting start point and the shooting end point has not been accepted (NO in step S45), the display control unit 113 determines in step S47 whether one of the third plurality of displayed shooting location icons has been selected. If it is determined that one shooting location icon has not been selected (NO in step S47), the processing ends.

[0117] On the other hand, if it is determined that one shooting location icon has been selected (YES in step S47), in step S48, the display control unit 113 displays the selected image corresponding to the selected shooting location icon, a first candidate image taken before the selected image, and a second candidate image taken after the selected image.

[0118] Next, in step S49, the display control unit 113 accepts a user's selection of the displayed first or second candidate image. When changing the selected image to the first or second candidate image, the user selects the first or second candidate image.

[0119] Next, in step S50, the display control unit 113 determines whether the first or second candidate image has been selected. If the first or second candidate image has not been selected (NO in step S50), the process ends.

[0120] On the other hand, if it is determined that the first or second candidate image has been selected (YES in step S50), in step S51, the display control unit 113 changes the selected image corresponding to one shooting location icon to the selected first or second candidate image.

[0121] FIG. 8 is a diagram showing an example of a display screen for accepting a change of the selected image corresponding to one selected photography location icon.

[0122] 8 , a currently selected image 311 indicates a selected image corresponding to a selected shooting location icon, a candidate image 312 one frame before indicates a first candidate image captured before the selected image, and a candidate image 314 one frame after indicates a second candidate image captured after the selected image. In addition to the above images, a candidate image 313 two frames before and a candidate image 315 two frames after are also displayed on the display screen 301. The user checks the currently selected image 311 and the multiple candidate images 312, 313, 314, and 315 that are also displayed. To change the currently selected image 311, the user selects one candidate image from the multiple candidate images 312, 313, 314, and 315 and presses a confirm button 321.

[0123] When one candidate image is selected from the plurality of candidate images 312, 313, 314, and 315 and the confirm button 321 is pressed, the display control unit 113 changes the currently selected image 311 corresponding to the one shooting location icon to the one candidate image selected from the plurality of candidate images 312, 313, 314, and 315.

[0124] In this way, when a blueprint includes a first plurality of photographing locations and a second plurality of photographing locations where photographs were taken before the photographing date and time at the first plurality of photographing locations, only the third plurality of images taken at the third plurality of photographing locations that are within a predetermined range from each of the second plurality of photographing locations from the photographing start location to the photographing end location among the first plurality of photographing locations are retained, and other images taken at other photographing locations are deleted, thereby reducing the amount of image data.

[0125] Furthermore, only the third plurality of images taken at the third plurality of photographing points that are within a predetermined range from each of the second plurality of photographing points among the first plurality of photographing points are kept, making it possible to perform fixed-point observation at a construction site or the like.

[0126] Furthermore, by reducing the amount of image data in the server 10, the amount of data stored in the memory 12 of the server 10 can be reduced, and the time required to transmit the data to the information terminal 20 can be shortened.

[0127] Next, the image display process of the server 10 in this embodiment will be described.

[0128] FIG. 9 is a flowchart showing an example of image display processing by the server 10 in this embodiment.

[0129] First, in step S61, the communication unit 13 receives an instruction to display a blueprint from the information terminal 20. In this case, a menu screen for selecting a blueprint is displayed on the display 23 of the information terminal 20, and the operation unit 24 accepts a user instruction to select one blueprint from the menu screen. The input instruction is transmitted to the server 10 via the network NT and received by the communication unit 13. The instruction to display the blueprint includes a blueprint ID.

[0130] Next, in step S62, the acquisition unit 111 acquires, from the design drawing information stored in the design drawing information storage unit 121, a design drawing corresponding to the design drawing ID included in the instruction to display the design drawing received by the communication unit 13.

[0131] Next, in step S63, the display control unit 113 transmits display data of the blueprint to the information terminal 20 via the communication unit 13, thereby displaying the blueprint on the display 23 of the information terminal 20. The display screen displayed by default displays the blueprint selected in step S61 and a selection acceptance field for selecting the shooting date and time. The selection acceptance field displays selectable shooting dates and times that can be displayed, and the operation unit 24 accepts a user instruction to select the desired shooting date and time in the selection acceptance field. The shooting date and time displayed in the selection acceptance field is a representative value of the shooting dates and times included in the image information stored in the memory 12. The representative value is, for example, the shooting start date and time. When a shooting date and time is selected, one shooting operation corresponding to that shooting date and time is selected. An instruction to specify the shooting date and time is transmitted to the server 10 via the network NT and received by the communication unit 13. The instruction to specify the shooting date and time includes a shooting ID.

[0132] Next, in step S64, the acquisition unit 111 determines whether or not the communication unit 13 has received an instruction to specify the shooting date and time. If it is determined that the communication unit 13 has not received an instruction to specify the shooting date and time (NO in step S64), the process returns to step S63.

[0133] On the other hand, if it is determined that the communication unit 13 has received an instruction to specify the shooting date and time (YES in step S64), in step S65, the acquisition unit 111 acquires, from the second image information stored in the second image information storage unit 123, second multiple shooting locations corresponding to the shooting ID included in the instruction to specify the shooting date and time received by the communication unit 13.

[0134] Next, in step S66, the display control unit 113 transmits a display instruction to the information terminal 20 via the communication unit 13 to display second plurality of photography location icons indicating the second plurality of photography locations on the design drawing, thereby displaying the second plurality of photography location icons indicating the second plurality of photography locations on the design drawing displayed on the display 23 of the information terminal 20. For example, the display control unit 113 displays the design drawing 201 shown in FIG. 7 on the display 23.

[0135] In addition, when multiple pieces of second image information are stored in which the shooting start point and shooting end point are substantially the same, the display control unit 113 may superimpose and display multiple second shooting location icons for each of the multiple second image information, or may display multiple second shooting location icons for only the specified second image information.

[0136] Next, in step S67, the acquisition unit 111 determines whether the communication unit 13 has received an end instruction. The end instruction is an instruction to close the design drawing displayed in step S63. This instruction is input by pressing an end button (not shown) displayed on the display 23. If it is determined that the communication unit 13 has received the end instruction (YES in step S67), the processing ends. On the other hand, if it is determined that the communication unit 13 has not received the end instruction (NO in step S67), the processing returns to step S64. In this case, the display of the design drawing is maintained. The end instruction is transmitted to the server 10 via the network NT and received by the communication unit 13. The acquisition unit 111 acquires the end instruction via the communication unit 13.

[0137] In this embodiment, if the second plurality of images do not exist when the first plurality of images are acquired, the image processing unit 112 accepts the user's request to reduce the first plurality of images, but the present disclosure is not particularly limited to this, and the image processing unit 112 may automatically reduce the first plurality of images.

[0138] Specifically, first, the image processing unit 112 acquires a reference image from the first plurality of images. The reference image is an image captured at the start of image capture. Next, the image processing unit 112 acquires a comparison image from the first plurality of images that was captured at a later time than the reference image. The comparison image is an image of the frame next to the reference image. Next, the image processing unit 112 extracts first and second feature points from first and second regions obtained by horizontally dividing the reference image. Next, the image processing unit 112 extracts third and fourth feature points from first and second regions obtained by horizontally dividing the comparison image. Next, the image processing unit 112 calculates a first change amount between the position of the first feature point and the position of the third feature point. Next, the image processing unit 112 calculates a second change amount between the position of the second feature point and the position of the fourth feature point. If at least one of the first change amount and the second change amount is equal to or greater than a threshold, the image processing unit 112 stores the comparison image in the memory 12 as second image information. On the other hand, if at least one of the first change amount and the second change amount is not equal to or greater than the threshold value, the image processing unit 112 deletes the comparison image.

[0139] When a comparison image is deleted, the image processing unit 112 acquires an image of a frame next to the deleted comparison image as the comparison image, calculates a first change amount and a second change amount, and determines whether to store the comparison image in memory 12 based on the calculated first change amount and second change amount. Furthermore, when the comparison image is stored in memory 12 as second image information, the image processing unit 112 acquires the stored comparison image as a reference image. The image processing unit 112 then acquires an image of a frame next to the acquired reference image as the comparison image, calculates a first change amount and a second change amount, and determines whether to store the comparison image in memory 12 based on the calculated first change amount and second change amount. The image processing unit 112 repeatedly acquires the reference image and stores or deletes the comparison image until an image captured at the shooting end point is acquired as the comparison image and the determination of whether to store the comparison image in memory 12 is completed. This results in thinning out the first plurality of images captured by the shooting device 30.

[0140] The technology according to the present disclosure is useful as a technology for displaying images because it can reduce the amount of image data.

Claims

1. A method of information processing performed by a computer, Displaying a bird's-eye view on the information terminal's display, The aforementioned bird's-eye view displays multiple shooting location icons indicating a series of consecutive shooting locations from the shooting start point to the shooting end point, Includes, The aforementioned multiple shooting location icons are associated with multiple images taken at the aforementioned multiple shooting locations. The display of the multiple shooting location icons includes, if the bird's-eye view includes a first set of multiple shooting locations and a second set of multiple shooting locations photographed before the date and time of photography at the first set of multiple shooting locations, displaying a third set of multiple shooting location icons that indicate a third set of multiple shooting locations that are within a predetermined range from each of the second set of multiple shooting locations between the start point and the end point of photography. Information processing methods.

2. The display of the multiple shooting location icons includes drawing a first set of multiple shooting location icons indicating the first set of multiple shooting locations and a second set of multiple shooting location icons indicating the second set of multiple shooting locations on the bird's-eye view, and displaying only the third set of multiple shooting location icons that overlap each of the second set of multiple shooting location icons from the shooting start point to the shooting end point. The information processing method according to claim 1.

3. The display of the multiple shooting location icons includes displaying only the third of the multiple shooting location icons that has the largest overlapping area with each of the second of the multiple shooting location icons from the shooting start point to the shooting end point, among the first of the multiple shooting location icons. The information processing method according to claim 2.

4. The display of the multiple shooting location icons includes displaying the third multiple shooting location icons that indicate only the third multiple shooting location that is closest to each of the second multiple shooting locations from the shooting start point to the shooting end point, among the first multiple shooting locations. The information processing method according to claim 1.

5. The process further includes storing in memory only the third set of images taken at the third set of shooting locations, which are within a predetermined range from each of the second set of shooting locations from the shooting start point to the shooting end point, out of the first set of images taken at the first set of shooting locations, and deleting the other images. The information processing method according to any one of claims 1 to 4.

6. Creating copies of the first set of images taken at the first set of shooting locations, Of the copies of the first plurality of images, only the third plurality of images taken at the third plurality of shooting locations that are within a predetermined range from each of the second plurality of shooting locations from the shooting start point to the shooting end point are stored in memory, and the other images are deleted. Further including, The information processing method according to any one of claims 1 to 4.

7. When a user changes the shooting start point and the shooting end point, a series of consecutive first shooting points from the shooting start point to the shooting end point in the bird's-eye view are created based on the bird's-eye view, the multiple images, the shooting start point after the change is accepted, and the shooting end point after the change is accepted. When a user makes a change to at least one of the shooting start point and the shooting end point, a series of consecutive first shooting points from the shooting start point to the shooting end point in the bird's-eye view are created based on the bird's-eye view, the multiple images, the shooting start point after the change is accepted, and the shooting end point after the change is accepted. When a user changes the shooting start point, a series of consecutive first shooting points from the shooting start point to the shooting end point in the bird's-eye view are created based on the bird's-eye view, the multiple images, the shooting start point after the change is accepted, and the shooting end point. If a user changes the aforementioned shooting end point, a series of consecutive first multiple shooting points from the shooting start point to the shooting end point in the bird's-eye view are created based on the bird's-eye view, the multiple images, the shooting start point, and the shooting end point after the change has been accepted. Further including, The information processing method according to any one of claims 1 to 4.

8. When a user selects one of the displayed third shooting location icons, the system displays the selected image corresponding to the selected first shooting location icon, a first candidate image taken before the selected image, and a second candidate image taken after the selected image. When an instruction is received to change the selected image to the first candidate image or the second candidate image, the selected image corresponding to the shooting location icon in 1 is changed to the first candidate image or the second candidate image, Further including, The information processing method according to any one of claims 1 to 4.

9. An information processing device equipped with a processor, The aforementioned processor, A bird's-eye view is displayed on the information terminal's screen. The aforementioned bird's-eye view displays multiple shooting location icons indicating a series of consecutive shooting locations from the shooting start point to the shooting end point. The aforementioned multiple shooting location icons are associated with multiple images taken at the aforementioned multiple shooting locations. In the display of the multiple shooting location icons, if the bird's-eye view includes a first set of multiple shooting locations and a second set of multiple shooting locations that were photographed before the date and time of photography at the first set of multiple shooting locations, a third set of multiple shooting location icons is displayed, indicating a third set of multiple shooting locations that are within a predetermined range from each of the second set of multiple shooting locations between the shooting start point and the shooting end point. Information processing device.

10. A bird's-eye view is displayed on the information terminal's screen. The computer is configured to display multiple shooting location icons on the aforementioned bird's-eye view, indicating a series of consecutive shooting locations from the shooting start point to the shooting end point. The aforementioned multiple shooting location icons are associated with multiple images taken at the aforementioned multiple shooting locations. In the display of the multiple shooting location icons, if the bird's-eye view includes a first set of multiple shooting locations and a second set of multiple shooting locations that were photographed before the date and time of photography at the first set of multiple shooting locations, a third set of multiple shooting location icons is displayed, indicating a third set of multiple shooting locations that are within a predetermined range from each of the second set of multiple shooting locations between the shooting start point and the shooting end point. Information processing program.