Information processing method, information processing device, and information processing program

Abstract

This information processing method includes: displaying a bird's-eye view on a display of an information terminal; and displaying a plurality of imaging point icons indicating a plurality of imaging points in the bird's-eye view. The plurality of imaging point icons are associated with a plurality of images captured at the plurality of imaging points. The plurality of images include a first image and a second image captured at a time later than the first image. The displaying of the plurality of imaging point icons includes displaying, in the bird's-eye view, a first imaging point icon associated with the first image and a second imaging point icon associated with the second image in which the amount of change from the first image is equal to or greater than a prescribed amount.

Description

Information processing method, information processing device, and information processing program

[0001] The present disclosure relates to techniques for displaying images.

[0002] For example, the technology of Patent Document 1 discloses a system in which a desired location on displayed construction drawing data is specified, a photography instruction is given to take an image of the construction site corresponding to the specified desired location, and placement data for the image of the construction site taken based on the photography instruction is linked to the desired location on the specified construction drawing data and displayed superimposed on the construction drawing data.

[0003] However, in the above conventional technology, the location to be photographed is determined in advance, and images are photographed only at the determined location. Therefore, in the above conventional technology, multiple images are not photographed consecutively, and the problem of an increase in the amount of image data is not taken into consideration.

[0004] International Publication No. 2022 / 145021

[0005] The present disclosure has been made to solve the above problems, and aims to provide a technique that can reduce the amount of image data.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, and includes displaying a bird's-eye view on a display of an information terminal, and displaying, on the bird's-eye view, a plurality of shooting location icons indicating a plurality of shooting locations, wherein the plurality of shooting location icons correspond to a plurality of images taken at a plurality of shooting locations, the plurality of images including a first image and a second image taken at a later time than the first image, and displaying the plurality of shooting location icons includes displaying, on the bird's-eye view, a first shooting location icon associated with the first image and a second shooting location icon associated with the second image for which the amount of change between the second image and the first image is greater than or equal to a threshold.

[0007] According to the present disclosure, the amount of image data can be reduced.

[0008] 1 is a diagram showing the overall configuration of an information processing system in this embodiment. FIG. 1 is a diagram showing an example of a blueprint screen on which a plurality of shooting location icons are superimposed before a plurality of images are thinned out. FIG. 2 is a diagram showing an example of a blueprint screen on which a plurality of shooting location icons are superimposed after a plurality of images are thinned out. FIG. 3 is a flowchart showing an example of processing by a communication device in this embodiment. FIG. 4 is a first flowchart showing an example of image reduction processing by a server in this embodiment. FIG. 5 is a second flowchart showing an example of image reduction processing by a server in this embodiment. FIG. 6 is a diagram showing an example of a reference image and a comparison image obtained when the shooting direction of the shooting device turns right in this embodiment. FIG. 7 is a diagram showing an example of a reference image and a comparison image obtained when the shooting direction of the shooting device faces upward in this embodiment. FIG. 8 is a diagram showing an example of a reference image and a comparison image obtained when the shooting direction of the shooting device faces downward in this embodiment. FIG. 9 is a diagram showing an example of a reference image and a comparison image obtained when the shooting device moves forward in the shooting direction in this embodiment. FIG. 10 is a flowchart showing an example of image display processing by a server in this embodiment.

[0009] (Findings that form the basis of the present disclosure) A user interface is being developed that displays multiple photography location icons indicating locations where actual photographs of a construction site were taken on a two-dimensional blueprint of the construction site, etc., and when one photography location icon is selected, displays an image taken at the photography location indicated by that icon. This allows a construction site manager to understand the situation at the construction site without visiting the construction site.

[0010] The photographer moves around the construction site while taking pictures of the site with a camera. The camera takes multiple images at a specified frame rate from the start to the end of the shoot. If the photographer walks slowly or stops, a large number of similar images may be saved, which may increase the amount of image data.

[0011] In the above-mentioned conventional technology, the location to be photographed is determined in advance, and images are photographed only at the determined location. Therefore, in the above-mentioned conventional technology, multiple images are not photographed consecutively, and the problem of an increase in the amount of image data is not taken into consideration.

[0012] In order to solve the above problems, the following techniques are disclosed.

[0013] (1) An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, comprising: displaying a bird's-eye view on a display of an information terminal; and displaying, on the bird's-eye view, a plurality of shooting location icons indicating a plurality of shooting locations, wherein the plurality of shooting location icons correspond to a plurality of images taken at a plurality of shooting locations, the plurality of images including a first image and a second image taken at a later time than the first image; and displaying the plurality of shooting location icons includes displaying, on the bird's-eye view, a first shooting location icon associated with the first image and a second shooting location icon associated with the second image that has changed by a predetermined amount or more from the first image.

[0014] According to this configuration, by storing a first image and a second image that has changed from the first image by a predetermined amount or more in memory, it is possible to delete the image that was between the first image and the second image, thereby reducing the amount of image data.

[0015] (2) In the information processing method described in (1) above, displaying the plurality of shooting location icons may include displaying on the bird's-eye view a first shooting location icon associated with the first image and a second shooting location icon associated with the second image in which the amount of change between the second image and the first image is greater than or equal to a threshold value.

[0016] According to this configuration, by storing the first image and the second image whose change amount is equal to or greater than the threshold in memory, it is possible to delete the image between the first image and the second image, thereby reducing the amount of image data.

[0017] (3) The information processing method described in (2) above may further include acquiring a reference image from the plurality of images, acquiring a comparison image from the plurality of images that was taken at a later time than the reference image, extracting a first feature point and a second feature point from each of first and second regions obtained by horizontally dividing the reference image, extracting a third feature point and a fourth feature point from each of the first and second regions obtained by horizontally dividing the comparison image, calculating a first amount of change between a position of the first feature point and a position of the third feature point, calculating a second amount of change between a position of the second feature point and a position of the fourth feature point, and displaying the comparison image as the second image if at least one of the first amount of change and the second amount of change is equal to or greater than the threshold.

[0018] According to this configuration, images with large changes are stored and displayed, and the amount of image data can be reduced.

[0019] (4) The information processing method according to (3) above may further include not displaying the comparison image when at least one of the first change amount and the second change amount is not equal to or greater than the threshold value.

[0020] According to this configuration, images that appear to have been taken at the same location but with little change are deleted and not displayed, thereby reducing the amount of image data.

[0021] (5) In the information processing method described in (3) above, acquiring the reference image may include first acquiring an image taken at the start point of shooting as the reference image, and if the comparison image is left as the second image, acquiring the remaining comparison image as the reference image.

[0022] According to this configuration, among the plurality of images, comparison images in which at least one of the first change amount and the second change amount is equal to or greater than the threshold value can be sequentially retained and stored in the memory.

[0023] (6) In the information processing method described in any one of (3) to (5) above, the method may further include calculating a first direction of change between the position of the first feature point and the position of the third feature point, calculating a second direction of change between the position of the second feature point and the position of the fourth feature point, and, when both the first direction of change and the second direction of change are pointing leftward or when both the first direction of change and the second direction of change are pointing rightward, determining whether or not at least one of the first amount of change and the second amount of change is equal to or greater than the threshold value.

[0024] For example, if the shooting direction of the camera device turns to the right, both the first change direction and the second change direction will face the left. Also, for example, if the shooting direction of the camera device turns to the left, both the first change direction and the second change direction will face the right. Therefore, if both the first change direction and the second change direction face either the left or the right, it can be determined that the comparison image is an image acquired when the shooting direction of the camera device turned either the right or the left. In this case, it becomes possible to determine whether at least one of the first change amount and the second change amount is equal to or greater than a threshold.

[0025] (7) In the information processing method described in (6) above, the method may further include deleting the comparison image when both the first change direction and the second change direction are pointing upward, or when both the first change direction and the second change direction are pointing downward.

[0026] For example, if the shooting direction of the camera is facing downward, both the first change direction and the second change direction will be facing upward. Also, for example, if the shooting direction of the camera is facing upward, both the first change direction and the second change direction will be facing downward. Therefore, if both the first change direction and the second change direction are facing either upward or downward, it can be determined that the comparison image is an image acquired when the shooting direction of the camera is facing either downward or upward. In this case, it is possible to delete the comparison image.

[0027] (8) In the information processing method described in (6) or (7) above, when one of the first change direction and the second change direction is pointing leftward and the other of the first change direction and the second change direction is pointing rightward, it may further include determining whether at least one of the first change amount and the second change amount is greater than or equal to the threshold value.

[0028] For example, when the image capture device moves forward in the capture direction, the first change direction is to the left and the second change direction is to the right. Therefore, when the first change direction is to the left and the second change direction is to the right, it can be determined that the comparison image is an image acquired when the image capture device is moving forward in the capture direction. In this case, it can be determined whether at least one of the first change amount and the second change amount is equal to or greater than a threshold.

[0029] (9) In the information processing method described in any one of (6) to (8) above, extracting the first feature point and the second feature point may include recognizing a first feature from the first region included in the reference image and extracting the center of gravity of the recognized first feature as the first feature point, recognizing a second feature from the second region included in the reference image and extracting the center of gravity of the recognized second feature as the second feature point, and extracting the third feature point and the fourth feature point may include recognizing a third feature from the first region included in the comparison image and extracting the center of gravity of the recognized third feature as the third feature point, and recognizing a fourth feature from the second region included in the comparison image and extracting the center of gravity of the recognized fourth feature as the fourth feature point.

[0030] According to this configuration, the first feature and the second feature are recognized from the first region and the second region included in the reference image, and the third feature and the fourth feature are recognized from the first region and the second region included in the comparison image, so that the centers of gravity of the recognized first feature, second feature, third feature, and fourth feature can be extracted as the first feature point, second feature point, third feature point, and fourth feature point, respectively.

[0031] Furthermore, the present disclosure can be realized not only as an information processing method that executes the characteristic processes described above, but also as an information processing device having a characteristic configuration corresponding to the characteristic processes executed by the information processing method. Furthermore, the present disclosure can also be realized as a computer program that causes a computer to execute the characteristic processes included in such an information processing method. Therefore, the same effects as those of the above information processing method can also be achieved in the following other aspects.

[0032] (10) Another aspect of the present disclosure is an information processing device including a processor, wherein the processor displays a bird's-eye view on a display of an information terminal and displays, on the bird's-eye view, a plurality of shooting location icons indicating a plurality of shooting locations, the plurality of shooting location icons corresponding to a plurality of images taken at a plurality of shooting locations, the plurality of images including a first image and a second image taken at a later time than the first image, and in displaying the plurality of shooting location icons, a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to the second image that has changed by a predetermined amount or more from the first image are displayed on the bird's-eye view.

[0033] (11) Another aspect of the information processing program of the present disclosure is to cause a computer to display a bird's-eye view on the display of an information terminal and to display, on the bird's-eye view, a plurality of shooting location icons indicating a plurality of shooting locations, the plurality of shooting location icons corresponding to a plurality of images taken at a plurality of shooting locations, the plurality of images including a first image and a second image taken at a later time than the first image, and in displaying the plurality of shooting location icons, a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to the second image that has changed by a predetermined amount or more from the first image are displayed on the bird's-eye view.

[0034] (12) A non-transitory computer-readable recording medium according to another aspect of the present disclosure records the information processing program described in (9) above.

[0035] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components. Furthermore, in all embodiments, the respective contents can be combined.

[0036] FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to the present embodiment.

[0037] The information processing system 1 is a system that displays a bird's-eye view of a workspace on a display 23 of an information terminal 20, and also displays a plurality of photography location icons indicating a plurality of photography locations on the bird's-eye view. The information processing system 1 includes a server 10, an information terminal 20, a photography device 30, and a communication device 40.

[0038] The server 10 is an example of an information processing device and a computer. The server 10, the information terminal 20, and the communication device 40 are connected to each other so as to be able to communicate with each other via a network NT. An example of the network NT is the Internet. The server 10 is, for example, a cloud server configured with one or more computers. However, this is just one example, and the server 10 may be configured as an edge server or may be implemented in the information terminal 20. The aspect in which the server 10 is implemented in the information terminal 20 is an example of the aspect in which the information terminal 20 is configured as an information processing device.

[0039] The information terminal 20 is carried by a user. The user is, for example, a manager of a predetermined space in which the image capturing device 30 captures images. The predetermined space is, for example, a construction site. However, this is just one example, and the predetermined space may be a construction site, a factory, a store, an office, or the like. The information terminal 20 may be configured as a portable computer such as a smartphone or a tablet computer, or as a stationary computer. The information terminal 20 displays images on a display 23 under control of the server 10. Although one information terminal 20 is illustrated in the example of FIG. 1, multiple information terminals may be connected to the server 10 via a network NT. The information terminal 20 includes a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0040] The communication unit 21 is a communication interface that connects the information terminal 20 to the network NT. The communication unit 21 transmits instruction signals indicating various instructions received from the user by the operation unit 24 to the server 10. The communication unit 21 receives display data for displaying various display screens from the server 10.

[0041] The processor 22 is configured by, for example, a central processing unit (CPU), and displays a display screen indicated by the display data received by the communication unit 21 on the display 23 .

[0042] The display 23 is configured with various display devices such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays various display screens under the control of the processor 22.

[0043] The operation unit 24 is composed of, for example, a keyboard, a touch panel, a mouse, and the like, and accepts various instructions input by the user.

[0044] The communication device 40 is configured by a mobile information terminal such as a smartphone or a tablet computer, and connects the photographing device 30 to the network NT. The communication device 40 and the photographing device 30 are connected to each other so that they can communicate with each other via a short-range wireless communication path such as Bluetooth (registered trademark).

[0045] The image capturing device 30 is, for example, an omnidirectional camera, and captures images at a predetermined frame rate. An omnidirectional camera is also called a 360-degree camera, and is a camera that can capture images in all directions of 360 degrees. The image capturing device 30 is, for example, a portable image capturing device carried by a photographer. The photographer may be, for example, a worker or a site supervisor at a construction site. The image capturing device 30 may be held in the photographer's hand, or may be worn on the photographer's body (for example, on the head). The image capturing device 30 may also be a regular camera.

[0046] The photographer moves through the construction site while taking pictures of the site with the camera device 30. At the start point of the shooting, the photographer points the camera device 30 in a northerly direction, presses the camera button on the camera device 30, and starts shooting. When the photographer reaches the end point of the shooting, he presses the camera button again to stop shooting. When the shooting is finished, the camera device 30 transmits the series of captured images to the communication device 40.

[0047] The image includes the date and time of the image capture. The date and time of the image capture is obtained, for example, by a clock provided in the image capture device 30. In this example, the image capture device 30 captures images at a predetermined frame rate (e.g., 10 frames per second), so the image capture location is defined in frame cycle units. However, this would result in an enormous amount of data, so the image capture location may be defined every predetermined time period (e.g., 1 second, 5 seconds, or 10 seconds).

[0048] The communication device 40 receives the multiple images transmitted by the image capturing device 30. The communication device 40 also displays a blueprint of the construction site and accepts designation by the photographer of the position of the image capturing start point and the position of the image capturing end point on the blueprint.

[0049] The start and end points of photography are specified by the photographer inputting instructions specifying the positions on the blueprint screen of the site displayed on the display of the communication device 40. Two-dimensional coordinate axes are defined on the blueprint screen. Therefore, the start and end points of photography are each defined by two-dimensional coordinate values. The start and end points of photography are used by the server 10 to specify the position of each photography point and the photography direction of each image.

[0050] The communication device 40 transmits photography information including the multiple captured images to the server 10 via the network NT. Photography information is generated each time a single photography operation is performed. One photography operation refers to a series of operations from when a worker holding the photography device 30 starts taking pictures at a construction site until when he finishes taking pictures. Multiple images are captured during one photography operation. The photography information includes the multiple images taken and meta information for each of the multiple images. The meta information includes a photography ID, a photography date and time, and a blueprint ID. The photography ID is an identifier for identifying the photography operation. The photography date and time is the date and time when the image was taken. The blueprint ID is an identifier for identifying a blueprint for a specified space corresponding to the image.

[0051] The server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 is configured, for example, by a central processing unit (CPU). The processor 11 includes an acquisition unit 111, an image processing unit 112, and a display control unit 113. The acquisition unit 111, the image processing unit 112, and the display control unit 113 may be realized by the processor 11 executing an information processing program, or may be configured by a dedicated hardware circuit such as an ASIC. The information processing program may be recorded on a non-transitory computer-readable recording medium.

[0052] The communication unit 13 is a communication interface that connects the server 10 to the network NT. The communication unit 13 receives shooting information transmitted by the communication device 40. The communication unit 13 receives instruction signals indicating various instructions received from the user from the information terminal 20. The communication unit 13 transmits display data for displaying various display screens to the information terminal 20.

[0053] The memory 12 is configured as a non-volatile rewritable storage device such as a hard disk drive or a solid state drive, etc. The memory 12 includes a design drawing information storage unit 121 and an image information storage unit 122.

[0054] The blueprint information storage unit 121 stores blueprint information. Blueprint information is image information showing a blueprint of a specified space. A blueprint ID that identifies the blueprint is associated with the blueprint information. A blueprint is an example of a bird's-eye view. A blueprint is a drawing showing the design of a construction site, and may be a floor plan, blueprint, map, perspective view, or the like of the construction site. A bird's-eye view can also be called a bird's-eye view, and may be a view from above or a view from a high place.

[0055] The acquisition unit 111 uses Visual SLAM (Simultaneous Localization and Mapping) technology to identify the location of each of the multiple images based on the shooting start point, shooting end point, and multiple images included in the shooting information received by the communication unit 13. The location of the shooting point is expressed as two-dimensional coordinate values ​​on the design drawing. The acquisition unit 111 also identifies the shooting direction of each of the multiple images using Visual SLAM technology. The shooting direction of each image is expressed, for example, as a three-dimensional polar coordinate vector.

[0056] The acquisition unit 111 creates initial image information based on the photography information and blueprints received by the communication unit 13. The initial image information is information that associates multiple images taken at multiple photography locations before thinning with meta information for each image. The meta information includes a photography ID, photography date and time, photography direction, photography location, and blueprint ID. The photography direction is the photography direction of the photography device 30 that took the image. The photography location is location information (two-dimensional coordinate values) that indicates the photography location of the image.

[0057] The image information storage unit 122 stores image information. The image information is information that associates multiple images taken at multiple shooting locations after thinning with meta information for each image. The image information is generated each time the above-mentioned shooting information is received. The meta information includes the shooting ID, shooting date and time, shooting direction, shooting location, and blueprint ID. The shooting direction is the shooting direction of the shooting device 30 that took the image. The shooting location is location information (two-dimensional coordinate values) that indicates the shooting location of the image. The image information is created by the image processing unit 112, which will be described later.

[0058] Furthermore, the communication unit 13 receives an instruction to specify the shooting date and time from the information terminal 20. The operation unit 24 of the information terminal 20 accepts the selection of the shooting date and time by the user, and the communication unit 21 transmits an instruction to specify the shooting date and time selected by the user to the server 10. The instruction to specify the shooting date and time includes the shooting ID.

[0059] The acquisition unit 111 acquires, from the design drawing information storage unit 121, a design drawing corresponding to a design drawing ID included in the instruction for displaying the design drawing received by the communication unit 13. The acquisition unit 111 also acquires, from the image information storage unit 122, a plurality of shooting locations corresponding to a shooting ID included in the instruction for specifying the shooting date and time received by the communication unit 13.

[0060] The image processing unit 112 acquires a reference image from among the multiple images included in the initial image information created by the acquisition unit 111. The image processing unit 112 first acquires an image captured at the shooting start point as the reference image. Note that if the image is an omnidirectional image, the image processing unit 112 acquires an image within a predetermined range in the shooting direction from the omnidirectional image as the reference image.

[0061] The image processing unit 112 acquires a comparison image that was captured later than the reference image from among the multiple images included in the initial image information created by the acquisition unit 111. The image processing unit 112 acquires the image of the frame next to the reference image as the comparison image. Note that if the image is an omnidirectional image, the image processing unit 112 acquires an image of the omnidirectional image within a predetermined range in the shooting direction as the comparison image.

[0062] The image processing unit 112 extracts first and second feature points from a first region and a second region obtained by horizontally dividing the acquired reference image. Specifically, the image processing unit 112 extracts first and second feature points from a first region and a second region obtained by horizontally dividing the reference image into two equal parts. The image processing unit 112 recognizes a first feature from the first region included in the reference image and extracts the center of gravity of the recognized first feature as the first feature point. The image processing unit 112 also recognizes a second feature from a second region included in the reference image and extracts the center of gravity of the recognized second feature as the second feature point.

[0063] The image processing unit 112 inputs a reference image into an image recognition model created in advance by machine learning, and acquires a first feature in the first region and a second feature in the second region output as recognition results from the image recognition model. The first feature and the second feature are surrounded by a rectangular frame. The center of gravity of the frame surrounding the first feature is the first feature point, and the center of gravity of the frame surrounding the second feature is the second feature point.

[0064] The image processing unit 112 extracts a third feature point and a fourth feature point from each of a first region and a second region obtained by horizontally dividing the acquired comparison image. Specifically, the image processing unit 112 extracts a third feature point and a fourth feature point from each of the first region and the second region obtained by horizontally dividing the comparison image in half. The image processing unit 112 recognizes a third feature object from the first region included in the comparison image and extracts the center of gravity of the recognized third feature object as the third feature point. The image processing unit 112 also recognizes a fourth feature object from the second region included in the comparison image and extracts the center of gravity of the recognized fourth feature object as the fourth feature point.

[0065] The image processing unit 112 inputs the comparison image into an image recognition model created in advance by machine learning, and acquires a third feature in the first region and a fourth feature in the second region output as recognition results from the image recognition model. The third feature and the fourth feature are surrounded by a rectangular frame. The center of gravity of the frame surrounding the third feature is the third feature point, and the center of gravity of the frame surrounding the fourth feature is the fourth feature point.

[0066] The image processing unit 112 calculates a first direction of change between the positions of the first feature point and the third feature point, and a second direction of change between the positions of the second feature point and the fourth feature point.

[0067] The image processing unit 112 calculates a first amount of change between the position of the first feature point and the position of the third feature point. The image processing unit 112 calculates a second amount of change between the position of the second feature point and the position of the fourth feature point.

[0068] When both the first direction of change and the second direction of change are pointing leftward, or when both the first direction of change and the second direction of change are pointing rightward, the image processing unit 112 determines whether at least one of the first change amount and the second change amount is equal to or greater than a threshold. For example, when the shooting direction of the image capture device 30 turns rightward, both the first change amount and the second change amount will be pointing leftward. Also, when the shooting direction of the image capture device 30 turns leftward, both the first change amount and the second change amount will be pointing rightward. Therefore, when both the first direction of change and the second change amount are pointing leftward or rightward, it can be determined that the comparison image is an image acquired when the shooting direction of the image capture device 30 was turning rightward or leftward. In this case, the image processing unit 112 determines whether at least one of the first change amount and the second change amount is equal to or greater than a threshold.

[0069] Furthermore, the image processing unit 112 deletes the comparison image when both the first change direction and the second change direction are pointing upward, or when both the first change direction and the second change direction are pointing downward. For example, when the shooting direction of the image capture device 30 is pointing downward, both the first change direction and the second change direction are pointing upward. For example, when the shooting direction of the image capture device 30 is pointing upward, both the first change direction and the second change direction are pointing downward. Therefore, when both the first change direction and the second change direction are pointing upward or downward, it can be determined that the comparison image is an image acquired when the shooting direction of the image capture device 30 is pointing downward or upward. In this case, the image processing unit 112 deletes the comparison image.

[0070] In addition, when both the first change direction and the second change direction are pointing upward, or when both the first change direction and the second change direction are pointing downward, the image processing unit 112 may determine whether at least one of the first change amount and the second change amount is greater than or equal to a threshold value.

[0071] Furthermore, when one of the first direction of change and the second direction of change is facing leftward and the other of the first direction of change and the second direction of change is facing rightward, the image processing unit 112 determines whether at least one of the first change amount and the second change amount is equal to or greater than a threshold. For example, when the image capture device 30 moves forward in the capture direction, the first change amount is facing leftward and the second change amount is facing rightward. Therefore, when the first direction of change is facing leftward and the second change amount is facing rightward, it can be determined that the comparison image is an image acquired when the image capture device 30 is moving forward in the capture direction. In this case, the image processing unit 112 determines whether at least one of the first change amount and the second change amount is equal to or greater than a threshold.

[0072] If at least one of the first change amount and the second change amount is equal to or greater than the threshold, the image processing unit 112 stores the comparison image as the second image in the image information storage unit 122 of the memory 12. Furthermore, if at least one of the first change amount and the second change amount is not equal to or greater than the threshold, the image processing unit 112 deletes the comparison image. This results in the multiple images captured by the imaging device 30 being thinned out.

[0073] The image processing unit 112 may determine whether both the first change amount and the second change amount are equal to or greater than a threshold. If both the first change amount and the second change amount are equal to or greater than a threshold, the image processing unit 112 may store the comparison image as a second image in the image information storage unit 122 of the memory 12, and may delete the comparison image if both the first change amount and the second change amount are not equal to or greater than a threshold. The image processing unit 112 may also determine whether the first change amount is equal to or greater than a threshold. If the first change amount is equal to or greater than a threshold, the image processing unit 112 may store the comparison image as a second image in the image information storage unit 122 of the memory 12, and may delete the comparison image if the first change amount is not equal to or greater than the threshold. The image processing unit 112 may also determine whether the second change amount is equal to or greater than a threshold. If the second change amount is equal to or greater than a threshold, the image processing unit 112 may store the comparison image as a second image in the image information storage unit 122 of the memory 12, and may delete the comparison image if the second change amount is not equal to or greater than a threshold.

[0074] When a comparison image is deleted, the image processing unit 112 acquires an image of a frame next to the deleted comparison image as the comparison image, calculates a first change amount and a second change amount, and determines whether to store the comparison image in memory 12 based on the calculated first change amount and second change amount. Furthermore, when the comparison image is left as the second image, i.e., when the comparison image is stored in memory 12 as the second image, the image processing unit 112 acquires the stored comparison image as a reference image. Then, the image processing unit 112 acquires an image of a frame next to the acquired reference image as the comparison image, calculates the first change amount and the second change amount, and determines whether to store the comparison image in memory 12 based on the calculated first change amount and second change amount. The image processing unit 112 repeatedly performs the processes from acquiring the reference image to storing or deleting the comparison image until an image captured at the shooting end point is acquired as the comparison image and the determination of whether to store the comparison image in memory 12 is completed.

[0075] The display control unit 113 displays a blueprint of the workspace on the display 23 of the information terminal 20. The display control unit 113 displays the blueprint acquired by the acquisition unit 111 on the display 23 of the information terminal 20. The display control unit 113 transmits display data of the blueprint to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data transmitted by the server 10. The display 23 displays the blueprint received by the communication unit 21.

[0076] The display control unit 113 displays, on the blueprint, multiple photography location icons indicating multiple photography locations. The multiple photography location icons are associated with multiple images photographed at the multiple photography locations. The multiple images include a first image and a second image photographed at a later time than the first image. The display control unit 113 displays, on the blueprint, the first photography location icon associated with the first image and the second photography location icon associated with a second image in which the amount of change from the first image is equal to or greater than a predetermined amount. The display control unit 113 displays, on the blueprint, the first photography location icon associated with the first image and the second photography location icon associated with a second image in which the amount of change between the second image and the first image is equal to or greater than a threshold amount.

[0077] Furthermore, when at least one of the first change amount and the second change amount is equal to or greater than a threshold, the display control unit 113 displays the comparison image as the second image. Furthermore, when at least one of the first change amount and the second change amount is not equal to or greater than a threshold, the display control unit 113 does not display the comparison image.

[0078] 2 is a diagram showing an example of a blueprint screen 100 on which a plurality of shooting location icons 102 are superimposed before a plurality of images are thinned out. The blueprint screen 100 displays a blueprint 101 of a workspace. The blueprint screen 100 displays a plurality of shooting location icons 102 superimposed on it.

[0079] The photography location icon 102 is an icon that indicates a photography location. The photography location icon 102 is associated with an image that was taken at the photography location. In this example, the photography location icon 102 is configured as a circular image. The multiple photography location icons 102 displayed here correspond to multiple photography locations that belong to one photography operation. The route that runs from the leading photography location icon 102 (the photography location icon at the bottom right of Figure 2) to the trailing photography location icon 102 (the photography location icon at the top left of Figure 2) indicates the trajectory that the photographer followed in this photography operation.

[0080] The image capturing device 30 captures images at a predetermined frame rate. In this case, the number of images is expressed as the frame rate (fps) * capturing time (seconds). In FIG. 2, the images are not thinned out, and the number of image capturing location icons 102 is not changed. Therefore, the image capturing location icons 102 are displayed overlapping each other along the trajectory of the photographer's movement.

[0081] 3 is a diagram showing an example of a blueprint screen 100A on which multiple photography location icons 102 are superimposed after multiple images have been thinned. The blueprint screen 100A displays a blueprint 101 of a workspace. On the blueprint screen 100A, a first photography location icon 102A associated with a first image and a second photography location icon 102B associated with a second image in which the amount of change between the second image and the first image is equal to or greater than a threshold are superimposed on the blueprint 101. The spacing between the multiple photography location icons 102 shown in FIG. 3 is wider than the spacing between the multiple photography location icons 102 shown in FIG. 2. This is because the multiple images captured by the photography device 30 have been reduced.

[0082] The photography location icon 102 is configured as a circular image. Note that the shape of the photography location icon 102 is an example, and other shapes may be used. An image taken at a photography location is associated with the photography location icon 102. The operation unit 24 may accept a user's selection of one photography location icon 102 from among the plurality of photography location icons 102. When one photography location icon 102 from among the plurality of photography location icons 102 is selected, the display control unit 113 may read an image corresponding to the selected one photography location icon 102 from the image information storage unit 122, and display the read image on the display 23.

[0083] First, the processing of the communication device 40 in this embodiment will be described.

[0084] FIG. 4 is a flowchart showing an example of processing by the communication device 40 in this embodiment.

[0085] First, in step S11, the communication device 40 displays the design drawing selected by the photographer. At this time, the communication device 40 accepts the selection of the design drawing by the photographer.

[0086] Next, in step S12, the communication device 40 determines whether the start point registration button has been pressed. At this time, the communication device 40 displays the start point registration button together with the design drawing. The communication device 40 accepts the photographer's input of the start point position on the design drawing, and also accepts the photographer's pressing of the start point registration button. For example, the photographer touches the start point position on the displayed design drawing. This inputs the start point position. The photographer then presses the start point registration button.

[0087] After pressing the start point registration button, the photographer presses the shooting button on the photographing device 30 to start photographing an image with the photographing device 30 .

[0088] If it is determined that the start point registration button has not been pressed (NO in step S12), the process of step S12 is repeated.

[0089] On the other hand, if it is determined that the start point registration button has been pressed (YES in step S12), in step S13, the communication device 40 stores the photographing start point on the design drawing. The communication device 40 stores the coordinates of the photographing start point on the design drawing.

[0090] Next, in step S14, the communication device 40 determines whether or not multiple images have been received from the photographing device 30. When the photographer reaches the photographing end point, he or she ends photographing by pressing the photographing button on the photographing device 30 again. When the photographing button is pressed, the photographing device 30 transmits the multiple photographed images to the communication device 40.

[0091] If it is determined that multiple images have not been received from the image capturing device 30 (NO in step S14), the process of step S14 is repeated.

[0092] On the other hand, if it is determined that a plurality of images have been received from the photographing device 30 (YES in step S14), the communication device 40 stores the received plurality of images in step S15.

[0093] Next, in step S16, the communication device 40 determines whether the end point registration button has been pressed. At this time, the communication device 40 displays the end point registration button together with the design drawing. The communication device 40 accepts the photographer's input of the end point position on the design drawing, and also accepts the photographer's pressing of the end point registration button. For example, the photographer touches the end point position on the displayed design drawing. This inputs the end point position. The photographer then presses the end point registration button.

[0094] If it is determined that the end point registration button has not been pressed (NO in step S16), the process of step S16 is repeated.

[0095] On the other hand, if it is determined that the end point registration button has been pressed (YES in step S16), in step S17, the communication device 40 stores the photographing end point on the design drawing. The communication device 40 stores the coordinates of the photographing end point on the design drawing.

[0096] Next, in step S18 , the communication device 40 transmits the photographic information including the plurality of images to the server 10 .

[0097] Next, the image reduction process of the server 10 in this embodiment will be described.

[0098] FIG. 5 is a first flowchart showing an example of the image reduction process of the server 10 in this embodiment, and FIG. 6 is a second flowchart showing an example of the image reduction process of the server 10 in this embodiment.

[0099] First, in step S31 , the communication unit 13 receives the photographing information transmitted by the communication device 40 .

[0100] Next, in step S32, the acquisition unit 111 acquires a design drawing from the design drawing information storage unit 121. The acquisition unit 111 reads out from the design drawing information storage unit 121 a design drawing corresponding to a design drawing ID included in the shooting information received by the communication unit 13.

[0101] Next, in step S33, the acquisition unit 111 creates initial image information based on the photography information and the blueprint. The acquisition unit 111 calculates the photography location and photography direction for each of the multiple images on the blueprint using Visual SLAM technology. The initial image information is information that associates multiple images captured by the photography device 30 before thinning with meta information for each image. The meta information includes the photography ID, photography date and time, photography direction, photography location, and blueprint ID.

[0102] Next, in step S34, the image processing unit 112 acquires a reference image from among the multiple images included in the initial image information created by the acquisition unit 111. Here, the image processing unit 112 acquires the image captured at the shooting start point as the reference image.

[0103] Next, in step S35, the image processing unit 112 extracts first and second feature points from the first and second regions of the acquired reference image. The first region is the left region of the two regions obtained by dividing the reference image by a perpendicular bisector, and the second region is the right region of the two regions obtained by dividing the reference image by the perpendicular bisector. Note that the first and second feature points are represented by coordinates in a Cartesian coordinate system with the lower left pixel of the reference image as the origin, for example.

[0104] Next, in step S36, the image processing unit 112 acquires a comparison image that was captured later than the reference image from among the multiple images included in the initial image information created by the acquisition unit 111. The image processing unit 112 acquires the image of the frame next to the reference image as the comparison image.

[0105] Next, in step S37, the image processing unit 112 extracts a third feature point and a fourth feature point from each of the first and second regions of the acquired comparison image. The first region is the left region of the two regions obtained by dividing the comparison image by a perpendicular bisector, and the second region is the right region of the two regions obtained by dividing the comparison image by the perpendicular bisector. The third feature point and the fourth feature point are represented by coordinates in a Cartesian coordinate system with the lower left pixel of the comparison image as the origin, for example.

[0106] Next, in step S38, the image processing unit 112 calculates a first direction of change and a first amount of change between the positions of the first feature point and the third feature point. The first direction of change and the first amount of change are represented by a first vector that starts at the coordinates of the first feature point and ends at the coordinates of the third feature point. The direction of the first vector is the first direction of change, and the magnitude of the first vector is the first amount of change.

[0107] Next, in step S39, the image processing unit 112 calculates a second direction of change and a second amount of change between the positions of the second feature point and the fourth feature point. The second direction of change and the second amount of change are represented by a second vector that starts at the coordinates of the second feature point and ends at the coordinates of the fourth feature point. The direction of the second vector is the second direction of change, and the magnitude of the second vector is the second amount of change.

[0108] Next, in step S40, the image processing unit 112 determines whether or not both the first change direction and the second change direction are directed leftward.

[0109] FIG. 7 shows an example of a reference image and a comparative image obtained when the photographing direction of the photographing device 30 is turned right in this embodiment.

[0110] 7 is divided into a first region 221 and a second region 222 by a perpendicular bisector 211. The image processing unit 112 extracts the center of gravity of a first feature 231 in the first region 221 as a first feature point 2311, and extracts the center of gravity of a second feature 232 in the second region 222 as a second feature point 2321.

[0111] 7 is an image captured at a later time than the reference image 200. The comparison image 201 is divided into a first region 221 and a second region 222 by a perpendicular bisector 211. The image processing unit 112 extracts the center of gravity of the third feature 241 in the first region 221 as a third feature point 2411, and extracts the center of gravity of the fourth feature 242 in the second region 222 as a fourth feature point 2421.

[0112] In the first area 221, the boundary portions of the walls are recognized as the first feature 231 and the third feature 241, and in the second area 222, the displays provided on the walls are recognized as the second feature 232 and the fourth feature 242. Furthermore, the first feature 231 and the third feature 241 are the same object, and the second feature 232 and the fourth feature 242 are the same object. In the comparison image 201 of Figure 7, the frame line indicating the first feature 231 is indicated by a dashed line, and the frame line indicating the second feature 232 is indicated by a dashed line.

[0113] When the shooting direction of the shooting device 30 turns to the right, both the first change direction 251 from the first feature point 2311 to the third feature point 2411 and the second change direction 252 from the second feature point 2321 to the fourth feature point 2421 will face to the left.

[0114] Note that the first and second change directions are not necessarily parallel to the horizontal direction (X direction), and the first and second change directions may be tilted relative to the horizontal direction (X direction). Therefore, the image processing unit 112 may determine whether the X component of the first change direction is larger than the Y component, the X component of the second change direction is larger than the Y component, and both the X component of the first change direction and the X component of the second change direction are pointing to the left (-X direction).

[0115] Returning to FIG. 5, if it is determined that both the first change direction and the second change direction are pointing leftward (YES in step S40), the process proceeds to step S44.

[0116] On the other hand, if it is determined that neither the first change direction nor the second change direction is pointing to the left (NO in step S40), in step S41, the image processing unit 112 determines whether both the first change direction and the second change direction are pointing to the right.

[0117] When the shooting direction of the shooting device 30 is turned to the left, both the first change direction 251 from the first feature point 2311 to the third feature point 2411 and the second change direction 252 from the second feature point 2321 to the fourth feature point 2421 will face to the right.

[0118] Note that the first and second change directions are not necessarily parallel to the horizontal direction (X direction), and may be tilted relative to the horizontal direction (X direction). Therefore, the image processing unit 112 may determine whether the X component of the first change direction is larger than the Y component, the X component of the second change direction is larger than the Y component, and both the X component of the first change direction and the X component of the second change direction are pointing to the right (+X direction).

[0119] If it is determined that both the first change direction and the second change direction are pointing to the right (YES in step S41), the process proceeds to step S44.

[0120] On the other hand, if it is determined that both the first change direction and the second change direction are not pointing to the right (NO in step S41), in step S42, the image processing unit 112 determines whether both the first change direction and the second change direction are pointing downward.

[0121] FIG. 8 is a diagram showing an example of a reference image and a comparative image obtained when the photographing direction of the photographing device 30 is directed upward in this embodiment.

[0122] 8 is divided into a first region 321 and a second region 322 by a perpendicular bisector 311. The image processing unit 112 extracts the center of gravity of a first feature 331 in the first region 321 as a first feature point 3311, and extracts the center of gravity of a second feature 332 in the second region 322 as a second feature point 3321.

[0123] 8 is an image captured at a later time than the reference image 300. The comparison image 301 is divided into a first region 321 and a second region 322 by a perpendicular bisector 311. The image processing unit 112 extracts the center of gravity of the third feature 341 in the first region 321 as a third feature point 3411, and extracts the center of gravity of the fourth feature 342 in the second region 322 as a fourth feature point 3421.

[0124] In the first area 321, the door is recognized as the first feature 331 and the third feature 341, and in the second area 322, the display mounted on the wall is recognized as the second feature 332 and the fourth feature 342. Furthermore, the first feature 331 and the third feature 341 are the same object, and the second feature 332 and the fourth feature 342 are the same object. In the comparison image 301 of Figure 8, the frame line indicating the first feature 331 is indicated by a dashed line, and the frame line indicating the second feature 332 is indicated by a dashed line.

[0125] When the shooting direction of the shooting device 30 is facing upward, both the first change direction 351 from the first feature point 3311 to the third feature point 3411 and the second change direction 352 from the second feature point 3321 to the fourth feature point 3421 will face downward.

[0126] Note that the first and second change directions are not necessarily parallel to the vertical direction (Y direction), and the first and second change directions may be tilted relative to the vertical direction (Y direction). Therefore, the image processing unit 112 may determine whether the Y component of the first change direction is larger than the X component, the Y component of the second change direction is larger than the X component, and both the Y component of the first change direction and the Y component of the second change direction are pointing downward (-Y direction).

[0127] In addition, in the present embodiment, the image processing unit 112 determines whether both the first change direction and the second change direction are downward, but the present disclosure is not particularly limited to this. The image processing unit 112 may determine only whether the first change direction is downward, or may determine only whether the second change direction is downward.

[0128] Returning to FIG. 6, if it is determined that both the first change direction and the second change direction are downward (YES in step S42), the process proceeds to step S45.

[0129] On the other hand, if it is determined that both the first change direction and the second change direction are not pointing downward (NO in step S42), in step S43, the image processing unit 112 determines whether both the first change direction and the second change direction are pointing upward.

[0130] FIG. 9 shows an example of a reference image and a comparative image obtained when the photographing direction of the photographing device 30 faces downward in this embodiment.

[0131] 9 is divided into a first region 421 and a second region 422 by a perpendicular bisector 411. The image processing unit 112 extracts the center of gravity of a first feature 431 in the first region 421 as a first feature point 4311, and extracts the center of gravity of a second feature 432 in the second region 422 as a second feature point 4321.

[0132] 9 is an image captured at a later time than the reference image 400. The comparison image 401 is divided into a first region 421 and a second region 422 by a perpendicular bisector 411. The image processing unit 112 extracts the center of gravity of a third feature 441 in the first region 421 as a third feature point 4411, and extracts the center of gravity of a fourth feature 442 in the second region 422 as a fourth feature point 4421.

[0133] In the first area 421, the displays mounted on the wall are recognized as the first feature 431 and the third feature 441, and in the second area 422, the doors are recognized as the second feature 432 and the fourth feature 442. The first feature 431 and the third feature 441 are the same object, and the second feature 432 and the fourth feature 442 are the same object. In the comparison image 401 of Figure 9, the frame line indicating the first feature 431 is indicated by a dashed line, and the frame line indicating the second feature 432 is indicated by a dashed line.

[0134] When the shooting direction of the shooting device 30 faces downward, both the first change direction 451 from the first feature point 4311 to the third feature point 4411 and the second change direction 452 from the second feature point 4321 to the fourth feature point 4421 face upward.

[0135] Note that the first and second change directions are not necessarily parallel to the vertical direction (Y direction), and the first and second change directions may be tilted relative to the vertical direction (Y direction). Therefore, the image processing unit 112 may determine whether the Y component of the first change direction is larger than the X component, the Y component of the second change direction is larger than the X component, and both the Y component of the first change direction and the Y component of the second change direction are pointing upward (+Y direction).

[0136] In addition, in the present embodiment, the image processing unit 112 determines whether both the first change direction and the second change direction are upward, but the present disclosure is not particularly limited to this. The image processing unit 112 may determine only whether the first change direction is upward, or may determine only whether the second change direction is upward.

[0137] Returning to FIG. 6, if it is determined that both the first change direction and the second change direction are upward (YES in step S43), the process proceeds to step S45.

[0138] On the other hand, if it is determined that both the first change direction and the second change direction are not pointing upward, i.e., if the first change direction is pointing left and the second change direction is pointing right (NO in step S43), in step S44, the image processing unit 112 determines whether at least one of the first change amount and the second change amount is greater than or equal to a threshold value.

[0139] FIG. 10 is a diagram showing an example of a reference image and a comparative image obtained when the image capturing device 30 moves forward in the image capturing direction in this embodiment.

[0140] 10 is divided into a first region 521 and a second region 522 by a perpendicular bisector 511. The image processing unit 112 extracts the center of gravity of a first feature 531 in the first region 521 as a first feature point 5311, and extracts the center of gravity of a second feature 532 in the second region 522 as a second feature point 5321.

[0141] 10 is an image captured at a later time than the reference image 500. The comparison image 501 is divided into a first region 521 and a second region 522 by a perpendicular bisector 511. The image processing unit 112 extracts the center of gravity of a third feature 541 in the first region 521 as a third feature point 5411, and extracts the center of gravity of a fourth feature 542 in the second region 522 as a fourth feature point 5421.

[0142] In the first region 521, the inner corners of the table are recognized as a first feature 531 and a third feature 541, and in the second region 522, the pillars are recognized as a second feature 532 and a fourth feature 542. Furthermore, the first feature 531 and the third feature 541 are the same object, and the second feature 532 and the fourth feature 542 are the same object. In the comparison image 501 of Figure 10, the frame line indicating the first feature 531 is indicated by a dashed line, and the frame line indicating the second feature 532 is indicated by a dashed line.

[0143] When the image capturing device 30 moves forward in the image capturing direction, a first change direction 551 from the first feature point 5311 to the third feature point 5411 faces the left direction, and a second change direction 552 from the second feature point 5321 to the fourth feature point 5421 faces the right direction. More precisely, when the image capturing device 30 moves forward in the image capturing direction, the first change direction 551 from the first feature point 5311 to the third feature point 5411 faces the lower left direction, and the second change direction 552 from the second feature point 5321 to the fourth feature point 5421 faces the lower right direction.

[0144] If it is determined in step S43 that neither the first change direction nor the second change direction points upward, the image processing unit 112 may determine whether the X component of the first change direction points leftward (−X direction) and whether the X component of the second change direction points rightward (+X direction).

[0145] 6 , if it is determined that at least one of the first change amount and the second change amount is not equal to or greater than the threshold value, i.e., if it is determined that both the first change amount and the second change amount are smaller than the threshold value (NO in step S44), the image processing unit 112 deletes the comparison image in step S45. Then, the process returns to step S36, where the image processing unit 112 acquires the image of the frame next to the deleted comparison image as a new comparison image.

[0146] On the other hand, if it is determined that at least one of the first change amount and the second change amount is equal to or greater than the threshold value (YES in step S44), in step S46, the image processing unit 112 stores the comparison image in the image information storage unit 122 of the memory 12. At this time, the image processing unit 112 stores image information including the reference image and the comparison image in the image information storage unit 122. The image information is information that associates multiple images captured by the imaging device 30 after thinning with meta information for each image. The meta information includes the imaging ID, imaging date and time, imaging direction, imaging location, and blueprint ID.

[0147] Next, in step S47, the image processing unit 112 determines whether or not all images included in the initial image information created by the acquisition unit 111 have been acquired as comparison images.

[0148] Here, if it is determined that not all images have been acquired as comparison images (NO in step S47), the process returns to step S34, and the image processing unit 112 acquires the stored comparison image as a new reference image.

[0149] On the other hand, if it is determined that all images have been acquired as comparison images (YES in step S47), the process ends.

[0150] Next, the image display process of the server 10 in this embodiment will be described.

[0151] FIG. 11 is a flowchart showing an example of image display processing by the server 10 in this embodiment.

[0152] First, in step S61, the communication unit 13 receives an instruction to display a blueprint from the information terminal 20. In this case, a menu screen for selecting a blueprint is displayed on the display 23 of the information terminal 20, and the operation unit 24 accepts a user instruction to select one blueprint from the menu screen. The input instruction is transmitted to the server 10 via the network NT and received by the communication unit 13. The instruction to display the blueprint includes a blueprint ID.

[0153] Next, in step S62, the acquisition unit 111 acquires, from the design drawing information stored in the design drawing information storage unit 121, a design drawing corresponding to the design drawing ID included in the instruction to display the design drawing received by the communication unit 13.

[0154] Next, in step S63, the display control unit 113 transmits display data of the blueprint to the information terminal 20 via the communication unit 13, thereby displaying the blueprint on the display 23 of the information terminal 20. The display screen displayed by default displays the blueprint selected in step S61 and a selection acceptance field for selecting the shooting date and time. The selection acceptance field displays selectable shooting dates and times that can be displayed, and the operation unit 24 accepts a user instruction to select the desired shooting date and time in the selection acceptance field. The shooting date and time displayed in the selection acceptance field is a representative value of the shooting dates and times included in the image information stored in the memory 12. The representative value is, for example, the shooting start date and time. When a shooting date and time is selected, one shooting operation corresponding to that shooting date and time is selected. An instruction to specify the shooting date and time is transmitted to the server 10 via the network NT and received by the communication unit 13. The instruction to specify the shooting date and time includes a shooting ID.

[0155] Next, in step S64, the acquisition unit 111 determines whether or not the communication unit 13 has received an instruction to specify the shooting date and time. If it is determined that the communication unit 13 has not received an instruction to specify the shooting date and time (NO in step S64), the process returns to step S63.

[0156] On the other hand, if it is determined that the communication unit 13 has received an instruction to specify the shooting date and time (YES in step S64), in step S65, the acquisition unit 111 acquires, from the image information stored in the image information storage unit 122, multiple shooting locations corresponding to the shooting ID included in the instruction to specify the shooting date and time received by the communication unit 13.

[0157] Next, in step S66, the display control unit 113 sends a display instruction to the information terminal 20 via the communication unit 13 to display multiple photography location icons indicating multiple photography locations on the blueprint, thereby displaying multiple photography location icons indicating the multiple photography locations on the blueprint displayed on the display 23 of the information terminal 20. At this time, the display control unit 113 displays, on the blueprint, the first photography location icon associated with the first image and the second photography location icon associated with the second image in which the amount of change between the second image and the first image is equal to or greater than a threshold. For example, the display control unit 113 displays the blueprint screen 100A shown in FIG. 3 on the display 23.

[0158] Next, in step S67, the acquisition unit 111 determines whether the communication unit 13 has received an end instruction. The end instruction is an instruction to close the design drawing displayed in step S63. This instruction is input by pressing an end button (not shown) displayed on the display 23. If it is determined that the communication unit 13 has received the end instruction (YES in step S67), the processing ends. On the other hand, if it is determined that the communication unit 13 has not received the end instruction (NO in step S67), the processing returns to step S64. In this case, the display of the design drawing is maintained. The end instruction is transmitted to the server 10 via the network NT and received by the communication unit 13. The acquisition unit 111 acquires the end instruction via the communication unit 13.

[0159] In this way, by storing the first image and the second image whose change amount is greater than or equal to the threshold value in memory 12, the image between the first image and the second image can be deleted, thereby reducing the amount of image data.

[0160] By reducing the amount of image data in the server 10, the amount of data stored in the memory 12 of the server 10 can be reduced, and the time required to transmit the data to the information terminal 20 can be shortened.

[0161] Furthermore, the communication device 40 may include an image processing unit 112. In this case, the amount of image data is reduced in the communication device 40, thereby reducing the amount of data to be transmitted to the server 10 and shortening the time required to transmit the data to the server 10.

[0162] The technology according to the present disclosure is useful as a technology for displaying images because it can reduce the amount of image data.

Claims

1. An information processing method executed by a computer, comprising: displaying a bird's-eye view on a display of an information terminal; and displaying a plurality of photography location icons indicating a plurality of photography locations on the bird's-eye view, wherein the plurality of photography location icons correspond to a plurality of images taken at a plurality of photography locations, the plurality of images including a first image and a second image taken at a later time than the first image, and displaying the plurality of photography location icons includes displaying on the bird's-eye view a first photography location icon associated with the first image and a second photography location icon associated with the second image which has changed in amount from the first image by a predetermined amount or more.

2. An information processing method as described in claim 1, wherein displaying the multiple shooting location icons includes displaying on the bird's-eye view a first shooting location icon associated with the first image and a second shooting location icon associated with the second image in which the amount of change between the second image and the first image is equal to or greater than a threshold value.

3. The information processing method of claim 2, further comprising: acquiring a reference image from the plurality of images; acquiring a comparison image from the plurality of images that is taken at a later time than the reference image; extracting a first feature point and a second feature point from each of a first region and a second region obtained by horizontally dividing the reference image; extracting a third feature point and a fourth feature point from each of the first region and the second region obtained by horizontally dividing the comparison image; calculating a first amount of change between a position of the first feature point and a position of the third feature point; calculating a second amount of change between a position of the second feature point and a position of the fourth feature point; and displaying the comparison image as the second image if at least one of the first amount of change and the second amount of change is equal to or greater than the threshold value.

4. The information processing method according to claim 3, further comprising not displaying the comparison image when at least one of the first change amount and the second change amount is not equal to or greater than the threshold value.

5. The information processing method of claim 3, wherein obtaining the reference image includes first obtaining an image taken at a shooting start point as the reference image, and if the comparison image is left as the second image, obtaining the remaining comparison image as the reference image.

6. The information processing method of claim 3, further comprising: calculating a first direction of change between the position of the first feature point and the position of the third feature point; calculating a second direction of change between the position of the second feature point and the position of the fourth feature point; and determining whether or not at least one of the first amount of change and the second amount of change is equal to or greater than the threshold value when both the first direction of change and the second direction of change are facing leftward or when both the first direction of change and the second direction of change are facing rightward.

7. The information processing method according to claim 6, further comprising deleting the comparison image when the first change direction and the second change direction are both pointing upward, or when the first change direction and the second change direction are both pointing downward.

8. An information processing method according to claim 6 or 7, further comprising: determining whether or not at least one of the first change amount and the second change amount is equal to or greater than the threshold value when one of the first change direction and the second change direction is facing leftward and the other of the first change direction and the second change direction is facing rightward.

9. An information processing method as described in claim 6, wherein extracting the first feature point and the second feature point includes recognizing a first feature from the first region included in the reference image and extracting the center of gravity of the recognized first feature as the first feature point, recognizing a second feature from the second region included in the reference image and extracting the center of gravity of the recognized second feature as the second feature point, and extracting the third feature point and the fourth feature point includes recognizing a third feature from the first region included in the comparison image and extracting the center of gravity of the recognized third feature as the third feature point, and recognizing a fourth feature from the second region included in the comparison image and extracting the center of gravity of the recognized fourth feature as the fourth feature point.

10. An information processing device having a processor, wherein the processor displays a bird's-eye view on a display of an information terminal, and displays a plurality of shooting location icons indicating a plurality of shooting locations on the bird's-eye view, the plurality of shooting location icons corresponding to a plurality of images taken at a plurality of shooting locations, the plurality of images including a first image and a second image taken at a later time than the first image, and in displaying the plurality of shooting location icons, a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to the second image which has changed in amount from the first image by a predetermined amount or more are displayed on the bird's-eye view.

11. An information processing program that displays a bird's-eye view on the display of an information terminal, causes a computer to function so as to display, on the bird's-eye view, a plurality of shooting location icons indicating a plurality of shooting locations, the plurality of shooting location icons corresponding to a plurality of images taken at a plurality of shooting locations, the plurality of images including a first image and a second image taken at a later time than the first image, and in displaying the plurality of shooting location icons, displays, on the bird's-eye view, a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to the second image which has changed in amount from the first image by a predetermined amount or more.