Information processing method, information processing device, and information processing program

Abstract

The information processing method comprises the following steps: displaying a bird's-eye view on a display of the information terminal; and displaying a plurality of imaging site icons indicating a plurality of imaging sites in the bird's-eye view, the plurality of imaging site icons being associated with a plurality of images captured at the plurality of imaging sites, the plurality of images including a first image and a second image captured at a later time than the first image, the display of the plurality of imaging point icons includes displaying, in the bird's-eye view, a first imaging point icon associated with the first image, and a second imaging point icon associated with a second image in which the amount of change from the first image is equal to or greater than a predetermined amount.

Description

Technical Field

[0001] This disclosure relates to techniques for displaying images. Background Technology

[0002] For example, in the technology of Patent Document 1, a system is disclosed that specifies a desired part on the displayed construction drawing data, provides a shooting instruction for taking an image of the construction site corresponding to the specified desired part, associates configuration data related to the image of the construction site taken based on the shooting instruction with the desired part on the specified construction drawing data, and displays them overlaid on the construction drawing data.

[0003] However, in the aforementioned existing technologies, the area to be photographed is predetermined, and images are only captured at the predetermined area. Therefore, the existing technologies do not capture multiple images consecutively, and do not take into account the problem of increased image data volume.

[0004] Prior art literature

[0005] Patent documents

[0006] Patent Document 1: International Publication No. 2022 / 145021 Summary of the Invention

[0007] This disclosure was made to solve the above-mentioned problems, and its purpose is to provide a technique that can reduce the amount of data in an image.

[0008] The information processing method disclosed herein is executed by a computer, the information processing method comprising: displaying a bird's-eye view on a display of an information terminal; and displaying a plurality of shooting location icons representing a plurality of shooting locations in the bird's-eye view, the plurality of shooting location icons corresponding to a plurality of images taken at the plurality of shooting locations, the plurality of images including: a first image and a second image taken at a later time than the first image, the display of the plurality of shooting location icons comprising: displaying a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to the second image whose change from the second image to the first image is greater than or equal to a threshold in the bird's-eye view.

[0009] According to this disclosure, it is possible to reduce the amount of image data. Attached Figure Description

[0010] Figure 1 This is a diagram showing the overall structure of the information processing system in this embodiment.

[0011] Figure 2 This is an example of a design diagram showing multiple shooting location icons that overlap and are then periodically removed from the image.

[0012] Figure 3 This is an example of a design diagram showing multiple shooting location icons that have been overlapped and then periodically removed from multiple images.

[0013] Figure 4 This is a flowchart illustrating an example of the processing of the communication device in this embodiment.

[0014] Figure 5 This is a first flowchart illustrating an example of image reduction processing of the server in this embodiment.

[0015] Figure 6 This is a second flowchart illustrating an example of image reduction processing on the server in this embodiment.

[0016] Figure 7 This is a diagram illustrating an example of a reference image and a comparison image obtained when the shooting direction of the shooting device is rotated to the right in this embodiment.

[0017] Figure 8 This is a diagram showing an example of a reference image and a comparison image obtained when the shooting direction of the shooting device is upward in this embodiment.

[0018] Figure 9 This is a diagram showing an example of a reference image and a comparison image obtained when the shooting direction of the shooting device is downward in this embodiment.

[0019] Figure 10 This is a diagram illustrating an example of a reference image and a comparison image obtained when the imaging device moves in the shooting direction in this embodiment.

[0020] Figure 11 This is a flowchart illustrating an example of image display processing of the server in this embodiment. Detailed Implementation

[0021] (The understanding that forms the basis of this disclosure)

[0022] The following user interface is being developed: on a two-dimensional design drawing of a construction site, multiple location icons representing actual photographic locations are displayed. When a location icon is selected, an image taken at that location is displayed. This allows construction site managers to monitor the construction site's condition without physically visiting the site.

[0023] The photographer moves within the construction site while simultaneously filming it. The camera captures multiple images at a given frame rate throughout the filming process. There is a concern that if the photographer moves slowly or remains stationary, storing a large number of identical images could increase the data size.

[0024] In the aforementioned existing technology, the area to be photographed is predetermined, and images are only captured at the predetermined area. Therefore, the existing technology does not capture multiple images consecutively, and does not take into account the problem of increased image data volume.

[0025] To address the above issues, the following technology has been disclosed.

[0026] (1) The information processing method involved in one aspect of the present disclosure is executed by a computer, the information processing method comprising: displaying a bird's-eye view on a display of an information terminal; and displaying a plurality of shooting location icons representing a plurality of shooting locations in the bird's-eye view, the plurality of shooting location icons corresponding to a plurality of images taken at the plurality of shooting locations, the plurality of images including: a first image and a second image taken at a later time than the first image, the display of the plurality of shooting location icons comprising: displaying a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to the second image having a change of more than a given amount in the first image in the bird's-eye view.

[0027] According to this structure, by storing the first image and the second image, which is a given change from the first image, in memory, it is possible to delete the image located between the first image and the second image, thereby reducing the amount of image data.

[0028] (2) In the information processing method described in (1) above, the display of the multiple shooting location icons may also include: displaying the first shooting location icon corresponding to the first image and the second shooting location icon corresponding to the second image whose change amount is above a threshold in the bird's-eye view.

[0029] According to this structure, by storing the first image and the second image whose change is above a threshold in memory, it is possible to delete the image located between the first image and the second image, thereby reducing the amount of image data.

[0030] (3) In the information processing method described in (2) above, the information processing method may also further include: acquiring a reference image among the plurality of images; acquiring a comparison image among the plurality of images that was captured at a later time than the reference image; extracting a first feature point and a second feature point from a first region and a second region that are divided along the horizontal direction of the reference image, respectively; extracting a third feature point and a fourth feature point from the first region and the second region that are divided along the horizontal direction of the comparison image, respectively; calculating a first change in the position of the first feature point and the position of the third feature point; calculating a second change in the position of the second feature point and the position of the fourth feature point; and displaying the comparison image as the second image when at least one of the first change and the second change is above the threshold.

[0031] According to this structure, images with significant variations are stored and displayed, which reduces the amount of image data.

[0032] (4) In the information processing method described in (3) above, the information processing method may also include: not displaying the comparison image if at least one of the first change and the second change is not above the threshold.

[0033] According to this structure, images of the same location with minor variations are deleted and not displayed, thus reducing the amount of image data.

[0034] (5) In the information processing method described in (3) above, the acquisition of the reference image may include: initially acquiring an image captured at the shooting start location as the reference image, and if the comparison image is left as the second image, acquiring the left-behind comparison image as the reference image.

[0035] According to this structure, it is possible to sequentially retain comparison images in which at least one of the first change and the second change among multiple images is above a threshold and store them in memory.

[0036] (6) In any of the information processing methods described in (3) to (5) above, the information processing method may further include: calculating the first direction of change of the position of the first feature point and the position of the third feature point; calculating the second direction of change of the position of the second feature point and the position of the fourth feature point; and determining whether at least one of the first change amount and the second change amount is above the threshold when both the first change direction and the second change direction are to the left, or when both the first change direction and the second change direction are to the right.

[0037] For example, when the shooting direction of the imaging device rotates to the right, both the first and second change directions point to the left. Conversely, when the shooting direction of the imaging device rotates to the left, both the first and second change directions point to the right. Therefore, if either the first or second change direction points to the left or right, it can be determined that the comparison image was acquired when the shooting direction of the imaging device rotated to either the right or left. Furthermore, in this case, it is possible to determine whether at least one of the first and second change amounts is above a threshold.

[0038] (7) In the information processing method described in (6) above, the information processing method may also include: deleting the comparison image when both the first change direction and the second change direction are upward, or when both the first change direction and the second change direction are downward.

[0039] For example, when the shooting direction of the imaging device is downward, both the first and second changing directions are upward. Conversely, when the shooting direction of the imaging device is upward, both the first and second changing directions are downward. Therefore, if either the first or second changing direction is upward or downward, it can be determined that the comparison image was acquired when the shooting direction of the imaging device was either downward or upward. Furthermore, in this case, deletion of the comparison image is possible.

[0040] (8) In the information processing method described in (6) or (7) above, the information processing method may also include: when one of the first change direction and the second change direction is to the left and the other of the first change direction and the second change direction is to the right, determining whether at least one of the first change amount and the second change amount is above the threshold.

[0041] For example, when the imaging device moves towards the shooting direction, the first change direction is to the left and the second change direction is to the right. Therefore, if the first change direction is to the left and the second change direction is to the right, it can be determined that the comparison image was acquired when the imaging device moved towards the shooting direction. Furthermore, in this case, it is possible to determine whether at least one of the first change amount and the second change amount is above a threshold.

[0042] (9) In any of the information processing methods described in (6) to (8) above, the extraction of the first feature point and the second feature point may include: identifying a first feature object from the first region included in the reference image, and extracting the centroid of the identified first feature object as the first feature point; identifying a second feature object from the second region included in the reference image, and extracting the centroid of the identified second feature object as the second feature point; and the extraction of the third feature point and the fourth feature point may include: identifying a third feature object from the first region included in the comparison image, and extracting the centroid of the identified third feature object as the third feature point; identifying a fourth feature object from the second region included in the comparison image, and extracting the centroid of the identified fourth feature object as the fourth feature point.

[0043] According to this structure, a first feature and a second feature are identified from the first and second regions included in the reference image, and a third feature and a fourth feature are identified from the first and second regions included in the comparison image. Thus, the centroids of the identified first, second, third, and fourth features can be extracted as the first feature point, the second feature point, the third feature point, and the fourth feature point, respectively.

[0044] Furthermore, this disclosure can be implemented not only as an information processing method that performs the characteristic processing described above, but also as an information processing apparatus having a characteristic structure corresponding to the characteristic processing performed by the information processing method. Additionally, it can be implemented as a computer program that causes a computer to execute the characteristic processing included in such an information processing method. Therefore, the following other embodiments can also achieve the same effect as the information processing method described above.

[0045] (10) Another aspect of the present disclosure relates to an information processing apparatus having a processor, the processor performing the following processing: displaying a bird's-eye view on a display of an information terminal, wherein multiple shooting location icons representing multiple shooting locations are displayed in the bird's-eye view, the multiple shooting location icons being associated with multiple images taken at the multiple shooting locations, the multiple images including: a first image and a second image taken at a later time than the first image, and in the display of the multiple shooting location icons, displaying a first shooting location icon associated with the first image and a second shooting location icon associated with the second image whose change from the first image is given or greater in the bird's-eye view.

[0046] (11) The information processing program involved in another aspect of this disclosure enables a computer to perform the following functions: displaying a bird's-eye view on the display of an information terminal; and displaying multiple shooting location icons representing multiple shooting locations in the bird's-eye view, the multiple shooting location icons corresponding to multiple images taken at the multiple shooting locations, the multiple images including: a first image, and a second image taken at a later time than the first image, in the display of the multiple shooting location icons, displaying a first shooting location icon corresponding to the first image, and a second shooting location icon corresponding to the second image whose change from the first image is given or greater in the bird's-eye view.

[0047] (12) Another aspect of this disclosure involves a non-transitory computer-readable recording medium that records the information processing program described in (9) above.

[0048] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Furthermore, the embodiments described below represent specific examples of the present disclosure. The numerical values, shapes, constituent elements, steps, and order of steps shown in the following embodiments are examples and are not intended to limit the present disclosure. Additionally, constituent elements in the following embodiments that are not described as independent technical solutions representing the highest-level concept are described as arbitrary constituent elements. Furthermore, various elements can be combined in all embodiments.

[0049] (Implementation Method)

[0050] Figure 1 This is a diagram showing the overall structure of the information processing system 1 in this embodiment.

[0051] Information processing system 1 is a system that displays a bird's-eye view of the workspace on the display 23 of information terminal 20, and displays multiple shooting location icons representing multiple shooting locations on the bird's-eye view. Information processing system 1 includes: server 10, information terminal 20, shooting device 30, and communication device 40.

[0052] Server 10 is an example of an information processing device and a computer. Server 10, information terminal 20, and communication device 40 are communicatively connected to each other via a network NT. An example of a network NT is the Internet. Server 10 is, for example, a cloud server consisting of one or more computers. However, this is just one example; server 10 can also be composed of an edge server or installed on information terminal 20. The way server 10 is installed on information terminal 20 is an example of the way information terminal 20 is composed of an information processing device.

[0053] Information terminal 20 is held by a user. The user is, for example, the manager of a given space captured by the camera device 30. The given space is, for example, a construction site. However, this is just one example; the given space could also be a construction site, factory, shop, or office, etc. Information terminal 20 can be a portable computer, such as a smartphone or tablet, or a fixed computer. Information terminal 20 displays images on display 23 under the control of server 10. Figure 1 In the example, one information terminal 20 is shown, but multiple information terminals can be connected to the server 10 via the network NT. The information terminal 20 includes a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0054] The communication unit 21 is a communication interface that connects the information terminal 20 to the network NT. The communication unit 21 sends instruction signals, representing various instructions received by the operation unit 24 from the user, to the server 10. The communication unit 21 receives display data from the server 10 for displaying various display screens.

[0055] The processor 22, for example, is a central processing unit (CPU) that displays the display screen represented by the display data received by the communication unit 21 on the display screen 23.

[0056] The display 23 is composed of various display devices such as liquid crystal displays or organic EL (Electro-Luminescence) displays, and displays various display screens under the control of the processor 22.

[0057] The operation unit 24 may consist of, for example, a keyboard, a touch panel, and a mouse, and accepts various instructions input by the user.

[0058] The communication device 40, for example, is a portable information terminal such as a smartphone or tablet computer, which connects the imaging device 30 to the network NT. The communication device 40 and the imaging device 30 are communicatively connected to each other via a near-field wireless communication path such as Bluetooth (registered trademark).

[0059] The shooting device 30 is, for example, a wide-angle camera that captures images at a given frame rate. A wide-angle camera, also known as a 360-degree camera, is a camera capable of acquiring images from all 360 degrees. The shooting device 30 is, for example, a portable shooting device held by a person. The person being shot may be, for example, a construction worker or a site supervisor. The shooting device 30 can be used while being held by the subject or attached to the person's body (e.g., head). Alternatively, the shooting device 30 may also be a conventional camera.

[0060] The photographer moves within the construction site while simultaneously filming using the filming device 30. At the starting point of the filming, the photographer points the filming device 30 northward and presses the filming button to begin shooting. Upon reaching the ending point, the photographer presses the filming button again to end the filming. When filming concludes, the filming device 30 transmits the series of images captured to the communication device 40.

[0061] The image includes the date and time of capture. The date and time of capture can be obtained, for example, by a clock provided with the capturing device 30. Here, since the capturing device 30 captures images at a given frame rate (e.g., 10 frames per second), the capture location is defined in units of frame periods. However, this results in a large amount of data, so the capture location can also be defined by a given time (e.g., 1 second, 5 seconds, or 10 seconds, etc.).

[0062] The communication device 40 receives multiple images transmitted by the imaging device 30. Additionally, the communication device 40 displays a design drawing of the construction site and accepts the photographer's designation of the start and end points of the imaging session on the design drawing.

[0063] The start and end points of the shooting are determined by the photographer inputting instructions specifying their locations on a site design drawing displayed on the monitor of the communication device 40. The design drawing includes a two-dimensional coordinate axis. Therefore, the start and end points of the shooting are defined by two-dimensional coordinate values. These start and end points are used by the server 10 to determine the location of each shooting location and the shooting direction of each image.

[0064] The communication device 40 sends capture information, including multiple captured images, to the server 10 via the network NT. Capture information is generated each time a capture action is performed. A single capture action refers to a series of actions taken by the operator holding the capture device 30 at the construction site, from the start to the end of the capture process. Multiple images are captured in a single capture action. The capture information includes the multiple captured images and metadata for each image. The metadata includes the capture ID, capture date and time, and design drawing ID. The capture ID is an identifier used to identify the capture action. The capture date and time is the date and time the image was captured. The design drawing ID is an identifier used to identify the design drawing corresponding to a given space.

[0065] Server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 may be, for example, a central processing unit (CPU). The processor 11 includes an acquisition unit 111, an image processing unit 112, and a display control unit 113. The acquisition unit 111, image processing unit 112, and display control unit 113 can be implemented by the processor 11 executing information processing programs, or they may be constructed using dedicated hardware circuits such as ASICs. The information processing programs may also be recorded on a non-transitory computer-readable recording medium.

[0066] Communication unit 13 is the communication interface connecting server 10 to network NT. Communication unit 13 receives image capture information sent by communication device 40. Communication unit 13 receives instruction signals from information terminal 20 indicating various instructions received from the user. Communication unit 13 sends display data for displaying various display screens to information terminal 20.

[0067] The memory 12 is composed of a non-volatile, rewritable storage device such as a hard disk drive or a solid-state drive. The memory 12 includes a design information storage unit 121 and an image information storage unit 122.

[0068] Design drawing information storage unit 121 stores design drawing information. Design drawing information is image information representing a design drawing of a given space. A correspondence is established between the design drawing information and the design drawing ID used to identify the design drawing. A design drawing is an example of a bird's-eye view. A design drawing is a drawing representing the design of a building site; it can also be a floor plan, blueprint, map, or perspective view of the building site. A bird's-eye view can also be called an aerial view; it can be a view viewed from above or from a high vantage point.

[0069] The acquisition unit 111 uses Visual SLAM (Simultaneous Localization and Mapping) technology to determine the location of each of the multiple images based on the shooting start location, shooting end location, and multiple images included in the shooting information received by the communication unit 13. The location of the shooting location is represented by two-dimensional coordinate values ​​on the design drawing. In addition, the acquisition unit 111 also uses Visual SLAM technology to determine the shooting direction of each of the multiple images. The shooting direction of each image is represented, for example, by a three-dimensional polar coordinate vector.

[0070] The acquisition unit 111 creates initial image information based on the shooting information and design drawing received from the communication unit 13. The initial image information is information that establishes a correspondence between multiple images taken at multiple shooting locations before they were periodically discarded and the metadata of each image. The metadata includes: shooting ID, shooting date and time, shooting direction, shooting location, and design drawing ID. The shooting direction is the shooting direction of the shooting device 30 that captured the image. The shooting location is the location information (two-dimensional coordinate values) indicating the location where the image was captured.

[0071] Image information storage unit 122 stores image information. Image information is information that establishes a correspondence between multiple images captured at multiple shooting locations (after being periodically filtered out) and the metadata of each image. Image information is generated whenever the aforementioned shooting information is received. Meta-information includes: shooting ID, shooting date and time, shooting direction, shooting location, and design drawing ID. The shooting direction is the shooting direction of the shooting device 30 that captured the image. The shooting location is the location information (two-dimensional coordinate values) indicating the shooting location of the image. Image information is created by image processing unit 112, described later.

[0072] Additionally, the communication unit 13 receives an instruction from the information terminal 20 to specify the shooting date and time. The operation unit 24 of the information terminal 20 accepts the user's selection of the shooting date and time, and the communication unit 21 sends the instruction to specify the shooting date and time selected by the user to the server 10. The instruction to specify the shooting date and time includes a shooting ID.

[0073] The acquisition unit 111 acquires from the design drawing information storage unit 121 the design drawing corresponding to the design drawing ID included in the instruction for displaying the design drawing received by the communication unit 13. Additionally, the acquisition unit 111 acquires from the image information storage unit 122 multiple shooting locations corresponding to the shooting ID included in the instruction for specifying the shooting date and time received by the communication unit 13.

[0074] The image processing unit 112 acquires a reference image from among multiple images included in the initial image information created by the acquisition unit 111. The image processing unit 112 initially acquires an image taken at the shooting start location as the reference image. Furthermore, in the case of a 360-degree image, the image processing unit 112 acquires an image within a given range of shooting directions in the 360-degree image as the reference image.

[0075] The image processing unit 112 acquires a comparison image, taken at a later time than the reference image, from among multiple images included in the initial image information created by the acquisition unit 111. The image processing unit 112 also acquires an image from the next frame after the reference image as a comparison image. Furthermore, in the case of a 360-degree image, the image processing unit 112 acquires images within a given range of the shooting direction in the 360-degree image as comparison images.

[0076] The image processing unit 112 extracts a first feature point and a second feature point from a first region and a second region, which are horizontally segmented from the acquired reference image. Specifically, the image processing unit 112 extracts the first feature point and the second feature point from the first region and the second region, which are horizontally bisected from the reference image. The image processing unit 112 identifies a first feature object from the first region included in the reference image and extracts the centroid of the identified first feature object as a first feature point. Furthermore, the image processing unit 112 identifies a second feature object from the second region included in the reference image and extracts the centroid of the identified second feature object as a second feature point.

[0077] The image processing unit 112 inputs a reference image into an image recognition model pre-created by machine learning, and acquires a first feature in a first region and a second feature in a second region, which are output as recognition results from the image recognition model. The first feature and the second feature are surrounded by a quadrilateral frame. The centroid of the frame surrounding the first feature is the first feature point, and the centroid of the frame surrounding the second feature is the second feature point.

[0078] The image processing unit 112 extracts a third feature point and a fourth feature point from a first region and a second region, which are horizontally segmented from the acquired comparison image. Specifically, the image processing unit 112 extracts a third feature point and a fourth feature point from the first region and the second region, which are horizontally bisected from the comparison image. The image processing unit 112 identifies a third feature object from the first region included in the comparison image and extracts the centroid of the identified third feature object as a third feature point. Furthermore, the image processing unit 112 identifies a fourth feature object from the second region included in the comparison image and extracts the centroid of the identified fourth feature object as a fourth feature point.

[0079] The image processing unit 112 inputs a comparison image into an image recognition model pre-created by machine learning, and acquires a third feature in a first region and a fourth feature in a second region, which are output as recognition results from the image recognition model. The third and fourth features are surrounded by quadrilateral frames. The centroid of the frame surrounding the third feature is the third feature point, and the centroid of the frame surrounding the fourth feature is the fourth feature point.

[0080] The image processing unit 112 calculates the first direction of change of the position of the first feature point and the position of the third feature point. The image processing unit 112 calculates the second direction of change of the position of the second feature point and the position of the fourth feature point.

[0081] The image processing unit 112 calculates the first change in the position of the first feature point compared to the position of the third feature point. The image processing unit 112 also calculates the second change in the position of the second feature point compared to the position of the fourth feature point.

[0082] When both the first and second change directions are oriented to the left, or when both the first and second change directions are oriented to the right, the image processing unit 112 determines whether at least one of the first and second change amounts is above a threshold. For example, when the shooting direction of the imaging device 30 rotates to the right, both the first and second change directions are oriented to the left. Conversely, when the shooting direction of the imaging device 30 rotates to the left, both the first and second change directions are oriented to the right. Therefore, when either the first or second change direction is oriented to the left or right, it can be determined that the comparison image was acquired when the shooting direction of the imaging device 30 rotated to the right or left. In this case, the image processing unit 112 determines whether at least one of the first and second change amounts is above a threshold.

[0083] Furthermore, the image processing unit 112 deletes the comparison image when both the first and second changing directions are upward, or when both the first and second changing directions are downward. For example, when the shooting direction of the imaging device 30 is downward, both the first and second changing directions are upward. Conversely, when the shooting direction of the imaging device 30 is upward, both the first and second changing directions are downward. Therefore, if either the first or second changing direction is upward or downward, it can be determined that the comparison image was acquired when the shooting direction of the imaging device 30 was either downward or upward. In this case, the image processing unit 112 deletes the comparison image.

[0084] Furthermore, when both the first and second change directions are upward, or when both the first and second change directions are downward, the image processing unit 112 can also determine whether at least one of the first change amount and the second change amount is above a threshold.

[0085] Furthermore, when one of the first and second change directions is to the left and the other is to the right, the image processing unit 112 determines whether at least one of the first and second change amounts is above a threshold. For example, when the imaging device 30 moves towards the shooting direction, the first change direction is to the left and the second change direction is to the right. Therefore, when the first change direction is to the left and the second change direction is to the right, it can be determined that the comparison image was acquired when the imaging device 30 moved towards the shooting direction. In this case, the image processing unit 112 determines whether at least one of the first and second change amounts is above a threshold.

[0086] If at least one of the first change amount and the second change amount is above a threshold, the image processing unit 112 stores the comparison image as a second image in the image information storage unit 122 of the memory 12. Conversely, if at least one of the first change amount and the second change amount is not above a threshold, the image processing unit 112 deletes the comparison image. Thus, multiple images captured by the imaging device 30 are periodically discarded.

[0087] Furthermore, the image processing unit 112 can also determine whether both the first change amount and the second change amount are above a threshold. If both the first change amount and the second change amount are above the threshold, the image processing unit 112 can store the comparison image as a second image in the image information storage unit 122 of the memory 12; if neither the first change amount nor the second change amount is above the threshold, the comparison image is deleted. Alternatively, the image processing unit 112 can also determine whether the first change amount is above the threshold. If the first change amount is above the threshold, the image processing unit 112 can store the comparison image as a second image in the image information storage unit 122 of the memory 12; if the first change amount is not above the threshold, the comparison image is deleted. Similarly, the image processing unit 112 can also determine whether the second change amount is above the threshold. If the second change amount is above the threshold, the image processing unit 112 can store the comparison image as a second image in the image information storage unit 122 of the memory 12; if the second change amount is not above the threshold, the comparison image is deleted.

[0088] When a comparison image is deleted, the image processing unit 112 acquires the image of the next frame after the deleted comparison image as a comparison image, calculates a first change and a second change, and determines whether to store the comparison image in the memory 12 based on the calculated first and second change. Conversely, when the comparison image is retained as a second image (i.e., stored in the memory 12 as a second image), the image processing unit 112 acquires the stored comparison image as a reference image. Furthermore, the image processing unit 112 acquires the image of the next frame after the acquired reference image as a comparison image, calculates the first change and the second change, and determines whether to store the comparison image in the memory 12 based on the calculated first and second change. The image processing unit 112 repeatedly performs the storage or deletion of comparison images from the acquisition of the reference image until an image captured at the end of the shooting is acquired as a comparison image and the determination of whether to store the comparison image in the memory 12 ends.

[0089] The display control unit 113 displays the design drawing of the workspace on the display 23 of the information terminal 20. The display control unit 113 displays the design drawing acquired by the acquisition unit 111 on the display 23 of the information terminal 20. The display control unit 113 sends the display data of the design drawing to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data sent by the server 10. The display 23 displays the design drawing received by the communication unit 21.

[0090] The display control unit 113 displays multiple shooting location icons representing multiple shooting locations in the design drawing. Each shooting location icon corresponds to multiple images captured at the multiple shooting locations. The multiple images include a first image and a second image captured at a later time than the first image. The display control unit 113 displays a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to a second image whose change from the first image is a given or greater. The display control unit 113 also displays a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to a second image whose change from the first image is a threshold or greater.

[0091] Furthermore, if at least one of the first change amount and the second change amount is above a threshold, the display control unit 113 displays a comparison image as a second image. Conversely, if at least one of the first change amount and the second change amount is not above a threshold, the display control unit 113 does not display a comparison image.

[0092] Figure 2This is an example of a design screen 100 showing multiple shooting location icons 102 overlapping before they are periodically removed. Design screen 100 displays a design drawing 101 of the workspace. In design screen 100, multiple shooting location icons 102 are displayed overlapping.

[0093] The shooting location icon 102 is an icon representing the shooting location. Images taken at the shooting location are associated with shooting location icons 102. In this example, the shooting location icon 102 consists of a circular image. The multiple shooting location icons 102 shown here correspond to multiple shooting locations belonging to a particular shooting action. From the front-end shooting location icon 102 ( Figure 2 (The shooting location icon in the lower right corner) Follow the shooting location icon at the end 102 ( Figure 2 The path (the shooting location icon in the upper left corner) indicates the trajectory followed by the photographer during the shooting action.

[0094] The imaging device 30 captures images at a given frame rate. In this case, the number of images is represented by frame rate (fps) * shooting time (seconds). Figure 2 In this process, multiple images are not removed at intervals, and the number of multiple shooting location icons 102 is not changed. Therefore, along the trajectory of the photographer's movement, multiple shooting location icons 102 are displayed overlapping each other.

[0095] Figure 3 This is an example of a design drawing screen 100A showing multiple shooting location icons 102 after overlapping and intermittently removing multiple images. Design drawing screen 100A displays a design drawing 101 of the workspace. In design drawing screen 100A, a first shooting location icon 102A corresponding to the first image and a second shooting location icon 102B corresponding to the second image whose change from the first image is greater than or equal to a threshold are overlaid on design drawing 101. Figure 3 The intervals between the multiple shooting location icons 102 shown are... Figure 2 The spacing between the multiple shooting location icons 102 shown is wide. This is because multiple images captured by the shooting device 30 are cropped.

[0096] The shooting location icon 102 is a circular image. Furthermore, the shape of the shooting location icon 102 is one example, but it can also be other shapes. An image captured at the shooting location is associated with the shooting location icon 102. The operation unit 24 can also accept selection by the user from one of multiple shooting location icons 102. When one of the multiple shooting location icons 102 is selected, the display control unit 113 can read the image corresponding to the selected shooting location icon 102 from the image information storage unit 122 and display the read image on the display 23.

[0097] First, the processing of the communication device 40 in this embodiment will be explained.

[0098] Figure 4 This is a flowchart illustrating an example of the processing of the communication device 40 in this embodiment.

[0099] First, in step S11, the communication device 40 displays the design drawing selected by the photographer. At this time, the communication device 40 accepts the photographer's selection of the design drawing.

[0100] Next, in step S12, the communication device 40 determines whether the start-up registration button has been pressed. At this time, the communication device 40 communicates with the design... Figure 1 The starting point registration button is displayed. The communication device 40 accepts the photographer's input of the starting point position on the design drawing and accepts the photographer's pressing of the starting point registration button. For example, the photographer touches the starting point position on the displayed design drawing. Thus, the starting point position is input. And the photographer presses the starting point registration button.

[0101] In addition, after pressing the start-up registration button, the photographer presses the shooting button on the shooting device 30 to begin shooting based on the image from the shooting device 30.

[0102] Here, if it is determined that the start registration button has not been pressed (no in step S12), the process of step S12 is repeated.

[0103] On the other hand, if it is determined that the starting point registration button has been pressed (yes in step S12), in step S13, the communication device 40 stores the shooting start location on the design drawing. The communication device 40 stores the coordinates of the shooting start location on the design drawing.

[0104] Next, in step S14, the communication device 40 determines whether multiple images have been received from the shooting device 30. When the photographer reaches the end of the shooting, they press the shooting button on the shooting device 30 again to end the shooting. When the shooting button is pressed, the shooting device 30 sends the captured multiple images to the communication device 40.

[0105] Here, if it is determined that no multiple images have been received from the imaging device 30 (no in step S14), the process of step S14 is repeated.

[0106] On the other hand, if it is determined that multiple images have been received from the imaging device 30 (yes in step S14), in step S15, the communication device 40 stores the multiple received images.

[0107] Next, in step S16, the communication device 40 determines whether the endpoint registration button has been pressed. At this time, the communication device 40 communicates with the design... Figure 1 The endpoint registration button is displayed. The communication device 40 accepts the photographer's input of the endpoint location on the design drawing and accepts the photographer's pressing of the endpoint registration button. For example, the photographer touches the endpoint location on the displayed design drawing. Thus, the endpoint location is input. And, the photographer presses the endpoint registration button.

[0108] Here, if it is determined that the endpoint registration button has not been pressed (no in step S16), the process of step S16 is repeated.

[0109] On the other hand, if it is determined that the endpoint registration button has been pressed (yes in step S16), in step S17, the communication device 40 stores the shooting end point on the design drawing. The communication device 40 stores the coordinates of the shooting end point on the design drawing.

[0110] Next, in step S18, the communication device 40 sends the capture information, including multiple images, to the server 10.

[0111] Next, the image reduction processing of server 10 in this embodiment will be described.

[0112] Figure 5 This is a first flowchart illustrating an example of image reduction processing of server 10 in this embodiment. Figure 6 This is a second flowchart illustrating an example of image reduction processing of server 10 in this embodiment.

[0113] First, in step S31, the communication unit 13 receives the shooting information sent by the communication device 40.

[0114] Next, in step S32, the acquisition unit 111 acquires the design drawing from the design drawing information storage unit 121. The acquisition unit 111 reads the design drawing corresponding to the design drawing ID included in the shooting information received by the communication unit 13 from the design drawing information storage unit 121.

[0115] Next, in step S33, the acquisition unit 111 creates initial image information based on the shooting information and the design drawing. The acquisition unit 111 uses Visual SLAM technology to calculate the shooting location and shooting direction of each of the multiple images on the design drawing. The initial image information is information that establishes a correspondence between the multiple images captured by the shooting device 30 before they were periodically removed and the metadata of each image. The metadata includes: shooting ID, shooting date and time, shooting direction, shooting location, and design drawing ID.

[0116] Next, in step S34, the image processing unit 112 acquires a reference image from among the multiple images included in the initial image information created by the acquisition unit 111. Here, the image processing unit 112 acquires the image captured at the shooting start location as the reference image.

[0117] Next, in step S35, the image processing unit 112 extracts a first feature point and a second feature point from the first region and the second region of the acquired reference image, respectively. The first region is the left region of the two regions divided by a vertical bisecting line of the reference image, and the second region is the right region of the two regions divided by a vertical bisecting line of the reference image. Furthermore, the first feature point and the second feature point are represented, for example, by coordinates in an orthogonal coordinate system with the lower left pixel of the reference image as the origin.

[0118] Next, in step S36, the image processing unit 112 acquires a comparison image, which was captured at a later time than the reference image, from among the multiple images included in the initial image information created by the acquisition unit 111. The image processing unit 112 acquires an image of the next frame after the reference image as the comparison image.

[0119] Next, in step S37, the image processing unit 112 extracts the third feature point and the fourth feature point from the first region and the second region of the acquired comparison image, respectively. The first region is the left region of the two regions divided by a vertical bisecting line in the comparison image, and the second region is the right region of the two regions divided by a vertical bisecting line in the comparison image. Furthermore, the third and fourth feature points are represented, for example, by coordinates in an orthogonal coordinate system with the lower left pixel of the comparison image as the origin.

[0120] Next, in step S38, the image processing unit 112 calculates the first direction of change and the first amount of change of the positions of the first feature point and the third feature point. The first direction of change and the first amount of change are represented by a first vector with the coordinates of the first feature point as the starting point and the coordinates of the third feature point as the ending point. The direction of the first vector is the first direction of change, and the magnitude of the first vector is the first amount of change.

[0121] Next, in step S39, the image processing unit 112 calculates the second direction of change and the second amount of change between the positions of the second feature point and the fourth feature point. The second direction of change and the second amount of change are represented by a second vector with the coordinates of the second feature point as the starting point and the coordinates of the fourth feature point as the ending point. The direction of the second vector is the second direction of change, and the magnitude of the second vector is the second amount of change.

[0122] Next, in step S40, the image processing unit 112 determines whether the first change direction and the second change direction are both pointing to the left.

[0123] Figure 7 This is a diagram showing an example of a reference image and a comparison image obtained when the shooting direction of the shooting device 30 is rotated to the right in this embodiment.

[0124] Figure 7 The reference image 200 shown is divided into a first region 221 and a second region 222 by a vertical bisecting line 211. The image processing unit 112 extracts the centroid of the first feature 231 in the first region 221 as the first feature point 2311, and extracts the centroid of the second feature 232 in the second region 222 as the second feature point 2321.

[0125] Figure 7 The comparison image 201 shown was captured at a later time than the reference image 200. The comparison image 201 is divided into a first region 221 and a second region 222 by a vertical bisecting line 211. The image processing unit 112 extracts the centroid of the third feature 241 in the first region 221 as the third feature point 2411, and extracts the centroid of the fourth feature 242 in the second region 222 as the fourth feature point 2421.

[0126] Furthermore, in region 221, the boundary portion of the wall is identified as first feature 231 and third feature 241, and in region 222, the display mounted on the wall is identified as second feature 232 and fourth feature 242. Additionally, first feature 231 and third feature 241 are the same object, and second feature 232 and fourth feature 242 are the same object. Figure 7 In the comparison image 201, the frame line representing the first feature 231 is represented by a dashed line, and the frame line representing the second feature 232 is represented by a dashed line.

[0127] When the shooting direction of the shooting device 30 is rotated to the right, the first change direction 251 from the first feature point 2311 to the third feature point 2411 and the second change direction 252 from the second feature point 2321 to the fourth feature point 2421 both move to the left.

[0128] Furthermore, the first and second directions of change are not limited to being parallel to the horizontal direction (X direction); they may be tilted relative to the horizontal direction (X direction). Therefore, the image processing unit 112 can also determine whether the X component of the first direction of change is larger than the Y component, and whether the X component of the second direction of change is larger than the Y component, and whether both the X components of the first and second directions of change are oriented to the left (-X direction).

[0129] Return to Figure 5 Here, if it is determined that both the first and second change directions are pointing to the left (yes in step S40), the process proceeds to step S44.

[0130] On the other hand, if it is determined that neither the first change direction nor the second change direction is pointing to the left (no in step S40), in step S41, the image processing unit 112 determines whether the first change direction and the second change direction are pointing to the right.

[0131] When the shooting direction of the shooting device 30 is rotated to the left, the first change direction 251 from the first feature point 2311 to the third feature point 2411 and the second change direction 252 from the second feature point 2321 to the fourth feature point 2421 both move to the right.

[0132] Furthermore, the first and second directions of change are not limited to being parallel to the horizontal direction (X direction); they may be tilted relative to the horizontal direction (X direction). Therefore, the image processing unit 112 can also determine whether the X component of the first direction of change is larger than the Y component, and whether the X component of the second direction of change is larger than the Y component, and whether both the X components of the first and second directions of change are oriented to the right (+X direction).

[0133] Here, if it is determined that both the first and second change directions are pointing to the right (yes in step S41), the process proceeds to step S44.

[0134] On the other hand, if it is determined that neither the first change direction nor the second change direction is pointing to the right (no in step S41), in step S42, the image processing unit 112 determines whether the first change direction and the second change direction are pointing downwards.

[0135] Figure 8 This is a diagram showing an example of a reference image and a comparison image obtained when the shooting direction of the shooting device 30 is upward in this embodiment.

[0136] Figure 8 The reference image 300 shown is divided into a first region 321 and a second region 322 by a vertical bisecting line 311. The image processing unit 112 extracts the centroid of the first feature 331 in the first region 321 as the first feature point 3311, and extracts the centroid of the second feature 332 in the second region 322 as the second feature point 3321.

[0137] Figure 8 The comparison image 301 shown was captured at a later time than the reference image 300. The comparison image 301 is divided into a first region 321 and a second region 322 by a vertical bisecting line 311. The image processing unit 112 extracts the centroid of the third feature 341 in the first region 321 as the third feature point 3411, and extracts the centroid of the fourth feature 342 in the second region 322 as the fourth feature point 3421.

[0138] Furthermore, in area 1 321, the door is identified as first feature 331 and third feature 341, and in area 2 322, the display mounted on the wall is identified as second feature 332 and fourth feature 342. Additionally, first feature 331 and third feature 341 are the same object, and second feature 332 and fourth feature 342 are the same object. Furthermore, in... Figure 8 In the comparison image 301, the frame line representing the first feature 331 is represented by a dashed line, and the frame line representing the second feature 332 is represented by a dashed line.

[0139] When the shooting direction of the shooting device 30 is upward, the first change direction 351 from the first feature point 3311 to the third feature point 3411 and the second change direction 352 from the second feature point 3321 to the fourth feature point 3421 are both downward.

[0140] Furthermore, the first and second directions of change are not limited to being parallel to the vertical direction (Y direction); they may be tilted relative to the vertical direction (Y direction). Therefore, the image processing unit 112 can also determine whether the Y component of the first direction of change is larger than the X component, and whether the Y component of the second direction of change is larger than the X component, and whether both the Y components of the first and second directions of change are oriented downwards (-Y direction).

[0141] Furthermore, in this embodiment, the image processing unit 112 determines whether both the first change direction and the second change direction are downward, but this disclosure is not particularly limited to this. The image processing unit 112 may also determine only whether the first change direction is downward, or only whether the second change direction is downward.

[0142] Return to Figure 6 Here, if it is determined that both the first and second change directions are downward (yes in step S42), the process proceeds to step S45.

[0143] On the other hand, if it is determined that neither the first change direction nor the second change direction is downward (no in step S42), in step S43, the image processing unit 112 determines whether the first change direction and the second change direction are upward.

[0144] Figure 9 This is a diagram showing an example of a reference image and a comparison image obtained when the shooting direction of the shooting device 30 is downward in this embodiment.

[0145] Figure 9 The reference image 400 shown is divided into a first region 421 and a second region 422 by a vertical bisecting line 411. The image processing unit 112 extracts the centroid of the first feature 431 in the first region 421 as the first feature point 4311, and extracts the centroid of the second feature 432 in the second region 422 as the second feature point 4321.

[0146] Figure 9 The comparison image 401 shown was captured at a later time than the reference image 400. The comparison image 401 is divided into a first region 421 and a second region 422 by a vertical bisecting line 411. The image processing unit 112 extracts the centroid of the third feature 441 in the first region 421 as the third feature point 4411, and extracts the centroid of the fourth feature 442 in the second region 422 as the fourth feature point 4421.

[0147] Furthermore, in area 1 421, the display mounted on the wall is identified as first feature 431 and third feature 441, and in area 2 422, the door is identified as second feature 432 and fourth feature 442. Additionally, first feature 431 and third feature 441 are the same object, and second feature 432 and fourth feature 442 are the same object. Figure 9 In the comparison image 401, the frame line representing the first feature 431 is represented by a dashed line, and the frame line representing the second feature 432 is represented by a dashed line.

[0148] When the shooting direction of the shooting device 30 is downward, the first change direction 451 from the first feature point 4311 to the third feature point 4411 and the second change direction 452 from the second feature point 4321 to the fourth feature point 4421 are both upward.

[0149] Furthermore, the first and second directions of change are not limited to being parallel to the vertical direction (Y direction); they may be tilted relative to the vertical direction (Y direction). Therefore, the image processing unit 112 can also determine whether the Y component of the first direction of change is larger than the X component, and whether the Y component of the second direction of change is larger than the X component, and whether both the Y components of the first and second directions of change are oriented upwards (+Y direction).

[0150] Furthermore, in this embodiment, the image processing unit 112 determines whether both the first change direction and the second change direction are upward, but this disclosure is not particularly limited to this. The image processing unit 112 may also determine only whether the first change direction is upward, or it may determine only whether the second change direction is upward.

[0151] Return to Figure 6 Here, if it is determined that both the first and second change directions are upward (yes in step S43), the process proceeds to step S45.

[0152] On the other hand, if it is determined that neither the first change direction nor the second change direction is upward, that is, if the first change direction is to the left and the second change direction is to the right (no in step S43), in step S44, the image processing unit 112 determines whether at least one of the first change amount and the second change amount is above the threshold.

[0153] Figure 10 This is a diagram illustrating an example of a reference image and a comparison image obtained when the imaging device 30 moves forward in the imaging direction in this embodiment.

[0154] Figure 10 The reference image 500 shown is divided into a first region 521 and a second region 522 by a vertical bisecting line 511. The image processing unit 112 extracts the centroid of the first feature 531 in the first region 521 as the first feature point 5311, and extracts the centroid of the second feature 532 in the second region 522 as the second feature point 5321.

[0155] Figure 10 The comparison image 501 shown was captured at a later time than the reference image 500. The comparison image 501 is divided into a first region 521 and a second region 522 by a vertical bisecting line 511. The image processing unit 112 extracts the centroid of the third feature 541 in the first region 521 as the third feature point 5411, and extracts the centroid of the fourth feature 542 in the second region 522 as the fourth feature point 5421.

[0156] Furthermore, in region 521, the inner corner of the table is identified as feature 531 and feature 541, and in region 522, the pillar is identified as feature 532 and feature 542. Additionally, feature 531 and feature 541 are the same object, and feature 532 and feature 542 are the same object. Figure 10 In the comparison image 501, the frame line representing the first feature 531 is represented by a dashed line, and the frame line representing the second feature 532 is represented by a dashed line.

[0157] When the shooting device 30 moves in the shooting direction, the first direction of change 551 from the first feature point 5311 to the third feature point 5411 is to the left, and the second direction of change 552 from the second feature point 5321 to the fourth feature point 5421 is to the right. More precisely, when the shooting device 30 moves in the shooting direction, the first direction of change 551 from the first feature point 5311 to the third feature point 5411 is to the lower left, and the second direction of change 552 from the second feature point 5321 to the fourth feature point 5421 is to the lower right.

[0158] Furthermore, if it is determined in step S43 that neither the first change direction nor the second change direction is pointing upwards, the image processing unit 112 may also determine whether the X component of the first change direction is pointing to the left (-X direction) and the X component of the second change direction is pointing to the right (+X direction).

[0159] Return to Figure 6 Here, if it is determined that neither the first change nor the second change is above the threshold, that is, if it is determined that both the first change and the second change are below the threshold (not in step S44), in step S45, the image processing unit 112 deletes the comparison image. Then, the process returns to step S36, where the image processing unit 112 acquires the image of the next frame after the deleted comparison image as a new comparison image.

[0160] On the other hand, if at least one of the first change and the second change is determined to be above a threshold (yes in step S44), in step S46, the image processing unit 112 stores the comparison image in the image information storage unit 122 of the memory 12. At this time, the image processing unit 112 stores image information including the reference image and the comparison image in the image information storage unit 122. The image information is information that establishes a correspondence between multiple images captured by the imaging device 30 after being periodically discarded and the metadata of each image. The metadata includes: shooting ID, shooting date and time, shooting direction, shooting location, and design drawing ID.

[0161] Next, in step S47, the image processing unit 112 determines whether all the images included in the initial image information created by the acquisition unit 111 have been acquired as comparison images.

[0162] Here, if it is determined that not all images have been acquired as comparison images (no in step S47), the process returns to step S34, and the image processing unit 112 acquires the stored comparison images as new reference images.

[0163] On the other hand, if it is determined that all images have been acquired as comparison images (yes in step S47), the process ends.

[0164] Next, the image display processing of the server 10 in this embodiment will be described.

[0165] Figure 11 This is a flowchart illustrating an example of image display processing of server 10 in this embodiment.

[0166] First, in step S61, the communication unit 13 receives an instruction from the information terminal 20 to display a design drawing. In this case, a menu screen for selecting a design drawing is displayed on the screen 23 of the information terminal 20, and the operation unit 24 receives the user's instruction to select a design drawing from the menu screen. The input instruction is sent to the server 10 via the network NT and received by the communication unit 13. The instruction for displaying the design drawing includes the design drawing ID.

[0167] Next, in step S62, the acquisition unit 111 acquires the design drawing corresponding to the design drawing ID included in the instruction for displaying the design drawing received by the communication unit 13 from the design drawing information stored in the design drawing information storage unit 121.

[0168] Next, in step S63, the display control unit 113 sends the display data of the design drawing to the information terminal 20 via the communication unit 13, and displays the design drawing on the display 23 of the information terminal 20. In the default display screen, the design drawing selected in step S61 and a selection acceptance bar for selecting the shooting date and time are displayed. The selectable shooting date and time are selectively displayed in the selection acceptance bar, and the operation unit 24 accepts the user's instruction to select the desired shooting date and time in the selection acceptance bar. The shooting date and time displayed in the selection acceptance bar is a representative value of the shooting date and time included in the image information stored in the memory 12. The representative value is, for example, the shooting start date and time. When a shooting date and time is selected, a shooting action corresponding to that shooting date and time is selected. The instruction for specifying the shooting date and time is sent to the server 10 via the network NT and received by the communication unit 13. The instruction for specifying the shooting date and time includes the shooting ID.

[0169] Next, in step S64, the acquisition unit 111 determines whether the communication unit 13 has received an instruction to specify the shooting date and time. Here, if it is determined that the communication unit 13 has not received an instruction to specify the shooting date and time (no in step S64), the process returns to step S63.

[0170] On the other hand, if it is determined that the communication unit 13 has received an instruction to specify the shooting date and time (yes in step S64), in step S65, the acquisition unit 111 acquires multiple shooting locations from the image information stored in the image information storage unit 122 that correspond to the shooting ID included in the instruction to specify the shooting date and time received by the communication unit 13.

[0171] Next, in step S66, the display control unit 113 sends a display instruction via the communication unit 13 to the information terminal 20 to display multiple shooting location icons representing multiple shooting locations in the design drawing. The multiple shooting location icons are then displayed in the design drawing on the display 23 of the information terminal 20. At this time, the display control unit 113 displays the first shooting location icon corresponding to the first image, and the second shooting location icon corresponding to the second image where the change between the second image and the first image is greater than or equal to a threshold, in the design drawing. For example, the display control unit 113 displays... Figure 3 The design drawing shown in Figure 100A is displayed on monitor 23.

[0172] Next, in step S67, the acquisition unit 111 determines whether the communication unit 13 has received an end instruction. The end instruction is an instruction to close the design drawing displayed in step S63. This instruction is input by pressing the end button (not shown) displayed on the display 23. If it is determined that the communication unit 13 has received the end instruction (yes in step S67), the process ends. On the other hand, if it is determined that the communication unit 13 has not received the end instruction (no in step S67), the process returns to step S64. In this case, the display of the design drawing is maintained. The end instruction is sent to the server 10 via the network NT and received by the communication unit 13. The acquisition unit 111 acquires the end instruction via the communication unit 13.

[0173] In this way, by storing the first image and the second image whose change is above a threshold in the memory 12, it is possible to delete the image located between the first image and the second image, thereby reducing the amount of image data.

[0174] Furthermore, by reducing the amount of image data in server 10, the amount of data stored in the memory 12 of server 10 can be reduced, and the data transmission time to information terminal 20 can be shortened.

[0175] Alternatively, the communication device 40 may also include an image processing unit 112. In this case, by reducing the amount of image data in the communication device 40, the amount of data sent to the server 10 can be reduced, and the data transmission time to the server 10 can be shortened.

[0176] Industrial availability

[0177] The technology disclosed herein can reduce the amount of data in an image, and is therefore useful as a technology for displaying images.

Claims

1. An information processing method, executed by a computer, The information processing method includes: An aerial view is displayed on the monitor of the information terminal; and The bird's-eye view displays multiple location icons representing various shooting locations. The multiple shooting location icons are associated with multiple images taken at the multiple shooting locations. The plurality of images includes: a first image, and a second image taken at a later time than the first image. The display of the multiple shooting location icons includes: displaying a first shooting location icon corresponding to the first image and a second shooting location icon corresponding to the second image whose change from the first image is given or greater in the bird's-eye view.

2. The information processing method according to claim 1, wherein, The display of the multiple shooting location icons includes: displaying the first shooting location icon corresponding to the first image and the second shooting location icon corresponding to the second image where the change amount between the second image and the first image is above a threshold in the bird's-eye view.

3. The information processing method according to claim 2, wherein, The information processing method further includes: Obtain the reference image from among the multiple images; Acquire a comparison image from among the plurality of images that was captured at a later time than the reference image; First feature points and second feature points are extracted from the first region and the second region, which are segmented horizontally from the reference image. The third feature point and the fourth feature point are extracted from the first region and the second region, which are segmented along the horizontal direction of the comparison image, respectively. Calculate the first change in position between the first feature point and the third feature point; Calculate the second change in position between the second feature point and the fourth feature point; and If at least one of the first change and the second change is above the threshold, the comparison image is displayed as the second image.

4. The information processing method according to claim 3, wherein, The information processing method further includes: The comparison image is not displayed if at least one of the first change and the second change is not above the threshold.

5. The information processing method according to claim 3, wherein, The acquisition of the reference image includes: initially acquiring an image taken at the shooting start location as the reference image, and, if the comparison image is retained as the second image, acquiring the retained comparison image as the reference image.

6. The information processing method according to claim 3, wherein, The information processing method further includes: Calculate the first direction of change between the position of the first feature point and the position of the third feature point; Calculate the second direction of change between the position of the second feature point and the position of the fourth feature point; and When both the first and second directions of change are oriented to the left, or when both the first and second directions of change are oriented to the right, it is determined whether at least one of the first and second changes is above the threshold.

7. The information processing method according to claim 6, wherein, The information processing method further includes: The comparison image is deleted when both the first and second change directions are pointing upwards, or when both the first and second change directions are pointing downwards.

8. The information processing method according to claim 6 or 7, wherein, The information processing method further includes: If one of the first change direction and the second change direction is oriented to the left and the other of the first change direction and the second change direction is oriented to the right, it is determined whether at least one of the first change amount and the second change amount is above the threshold.

9. The information processing method according to claim 6, wherein, The extraction of the first feature point and the second feature point includes: identifying a first feature object from the first region included in the reference image, and extracting the centroid of the identified first feature object as the first feature point; identifying a second feature object from the second region included in the reference image, and extracting the centroid of the identified second feature object as the second feature point. The extraction of the third feature point and the fourth feature point includes: identifying a third feature object from the first region included in the comparison image, extracting the centroid of the identified third feature object as the third feature point, identifying a fourth feature object from the second region included in the comparison image, and extracting the centroid of the identified fourth feature object as the fourth feature point.

10. An information processing device, comprising a processor, The processor performs the following processing: An aerial view is displayed on the information terminal's monitor. The bird's-eye view displays multiple location icons representing various shooting locations. The multiple shooting location icons are associated with multiple images taken at the multiple shooting locations. The plurality of images includes: The first image, and the second image taken at a later time than the first image. In the display of the multiple shooting location icons, the first shooting location icon corresponding to the first image and the second shooting location icon corresponding to the second image whose change from the first image is given or greater are displayed in the bird's-eye view.

11. An information processing program that enables a computer to perform the following functions: The information terminal displays a bird's-eye view; and The bird's-eye view displays multiple location icons representing various shooting locations. The multiple shooting location icons are associated with multiple images taken at the multiple shooting locations. The plurality of images includes: The first image, and the second image taken at a later time than the first image. In the display of the multiple shooting location icons, the first shooting location icon corresponding to the first image and the second shooting location icon corresponding to the second image whose change from the first image is given or greater are displayed in the bird's-eye view.

Images