Information processing method, information processing device, and information processing program

Abstract

This information processing method includes: displaying an overhead view on a display of an information terminal; displaying, in the overhead view, an imaging location icon group indicating a series of a plurality of imaging locations from an imaging start point to an imaging end point; accepting selection of an arbitrarily defined region by a user on the overhead view; and, if the region contains two or more imaging location icon groups and the imaging positions of the two or more imaging location icon groups are similar, extracting two or more items of data corresponding to the two or more similar imaging location icon groups.

Description

Information processing method, information processing device, and information processing program

[0001] The present disclosure relates to techniques for displaying images.

[0002] For example, the technology in Patent Document 1 discloses a technology in which a range from a desired first position to a desired second position is identified from a group of main images taken of a site, the main images of the identified range are displayed, and a pointer indicating the corresponding point of the target site represented by the displayed main image is displayed on a drawing image representing a drawing of the target site.

[0003] However, the above-mentioned conventional technology does not disclose how to access two or more sets of data corresponding to two or more desired groups of image capture location icons, and further improvement is needed. Also, accessing two or more sets of data increases the number of processing steps of the computer, which may increase the burden on the computer.

[0004] JP 2022-24440 A

[0005] The present disclosure has been made to solve the above problems, and aims to provide a technology that enables easy access to two or more pieces of data corresponding to two or more desired groups of shooting location icons, thereby improving user convenience and reducing the computer processing steps required to access two or more pieces of data, thereby reducing the burden on the computer.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, and includes: displaying a bird's-eye view on a display of an information terminal; displaying, on the bird's-eye view, a group of shooting location icons indicating a series of multiple shooting locations from a shooting start location to a shooting end location; accepting a user's selection of an arbitrary area on the bird's-eye view; and, if two or more groups of shooting location icons are included within the area and the shooting locations of the two or more groups of shooting location icons are similar, extracting two or more pieces of data corresponding to the two or more similar groups of shooting location icons.

[0007] According to the present disclosure, it is possible to easily access two or more pieces of data corresponding to two or more desired groups of image capture location icons, thereby improving user convenience. Furthermore, it is possible to reduce the processing steps of a computer required to access two or more pieces of data, thereby reducing the burden on the computer.

[0008] FIG. 1 is a diagram showing the overall configuration of an information processing system in this embodiment. FIG. 2 is a diagram showing an example of a blueprint screen on which a group of multiple photography location icons are mapped in this embodiment. FIG. 3 is a diagram showing an example of a blueprint screen on which an area is drawn on a blueprint in this embodiment. FIG. 4 is a diagram showing an example of a photographed image screen displayed on a display when an area is selected in this embodiment. FIG. 5 is a first flowchart showing an example of image display processing of a server in this embodiment. FIG. 6 is a second flowchart showing an example of image display processing of a server in this embodiment.

[0009] (Findings underlying the present disclosure) In the above-described prior art, a series of main images taken of a site is only one, and multiple main images taken at different dates and times are not displayed. Furthermore, in the prior art, the shooting location of one of the main images is displayed on a site map, but the shooting locations of each of the main images are not displayed on the site map. Therefore, the prior art does not disclose how to access data corresponding to a desired main image from among multiple main image groups when multiple main image groups exist. Furthermore, accessing two or more data sets increases the number of computer processing steps, potentially increasing the burden on the computer.

[0010] In order to solve the above problems, the following techniques are disclosed.

[0011] (1) An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, and includes: displaying a bird's-eye view on a display of an information terminal; displaying, on the bird's-eye view, a group of photography location icons indicating a series of multiple photography locations from a photography start point to a photography end point; accepting a user's selection of an arbitrary area on the bird's-eye view; and, if two or more photography location icon groups are included within the area and the photography locations of the two or more photography location icon groups are similar, extracting two or more pieces of data corresponding to the two or more similar photography location icon groups.

[0012] According to this configuration, by selecting an area on the bird's-eye view map that includes two or more desired groups of photography location icons, two or more pieces of data corresponding to the two or more desired groups of photography location icons can be easily accessed, improving user convenience. Also, by selecting an area on the bird's-eye view map that includes two or more desired groups of photography location icons, two or more pieces of data corresponding to the two or more groups of photography location icons can be extracted, reducing the number of computer processing steps required to access two or more pieces of data and reducing the burden on the computer.

[0013] (2) In the information processing method described in (1) above, the extraction of the two or more pieces of data may include extracting the two or more pieces of data corresponding to the two or more groups of shooting location icons when at least one of the distance between the shooting start points of each of the two or more groups of shooting location icons and the distance between the shooting end points of each of the two or more groups of shooting location icons is equal to or less than a threshold value.

[0014] According to this configuration, by determining whether or not at least one of the distance between the shooting start points of each of two or more groups of shooting location icons and the distance between the shooting end points of each of two or more groups of shooting location icons is below a threshold value, it is possible to easily determine whether or not the shooting positions of two or more groups of shooting location icons are similar.

[0015] (3) In the information processing method described in (1) above, the extraction of the two or more data may include extracting the two or more data corresponding to the two or more groups of shooting location icons when the two or more groups of shooting location icons overlap within the area.

[0016] According to this configuration, by determining whether two or more groups of shooting location icons overlap within an area, it is possible to easily determine whether the shooting locations of the two or more groups of shooting location icons are similar.

[0017] (4) In the information processing method described in (1) above, the two or more groups of photographing location icons may include a first group of photographing location icons and a second group of photographing location icons, and the extraction of the two or more data may include extracting first data corresponding to the first group of photographing location icons and second data corresponding to the second group of photographing location icons when the shortest distance between each of the plurality of photographing locations corresponding to the first group of photographing location icons in the area and each of the plurality of photographing locations corresponding to the second group of photographing location icons in the area is equal to or less than a threshold value.

[0018] According to this configuration, by determining whether the shortest distance between each of the multiple shooting locations corresponding to the first group of shooting location icons in the area and each of the multiple shooting locations corresponding to the second group of shooting location icons in the area is less than or equal to a threshold value, it is possible to easily determine whether the shooting locations of two or more groups of shooting location icons are similar.

[0019] (5) The information processing method according to any one of (1) to (4) above may further include displaying, on the display, a name for identifying the group of shooting location icons together with the bird's-eye view.

[0020] According to this configuration, the names for identifying the group of shooting location icons are displayed on the display together with the bird's-eye view, so the user can confirm the names of the group of shooting location icons.

[0021] (6) In the information processing method described in (5) above, when two or more groups of photography location icons are included within the area and the photography locations of the two or more groups of photography location icons are similar, the method may further include changing the display mode of the two or more groups of photography location icons and the display mode of two or more names corresponding to the two or more groups of photography location icons.

[0022] According to this configuration, by selecting an area on the bird's-eye view that includes two or more desired groups of shooting location icons, the display mode of the two or more desired groups of shooting location icons and the display mode of two or more names corresponding to the two or more desired groups of shooting location icons are changed, so that the user can easily check the names of the two or more desired groups of shooting location icons.

[0023] (7) In the information processing method described in (6) above, the change in the display mode may include changing the color of the two or more shooting location icon groups and changing the color of the two or more names to the same color as the color of the two or more shooting location icon groups.

[0024] According to this configuration, the colors of two or more groups of photography location icons are changed, and the colors of two or more names are changed to the same colors as the colors of two or more groups of photography location icons, so that the names of two or more groups of photography location icons selected by the user can be easily confirmed.

[0025] (8) In the information processing method described in any one of (1) to (7) above, the data may be an image taken at a photography location.

[0026] According to this configuration, two or more images taken at two or more shooting locations corresponding to two or more groups of shooting location icons are extracted, so that two or more images can be displayed and two or more images taken at similar locations can be compared.

[0027] (9) In the information processing method described in any one of (1) to (8) above, displaying the group of shooting location icons may include displaying the group of shooting location icons in a color corresponding to the shooting date and time.

[0028] According to this configuration, the group of shooting location icons is displayed in a color corresponding to the shooting date and time, so the user can intuitively recognize the shooting date and time by checking the color of the group of shooting location icons.

[0029] Furthermore, the present disclosure can be realized not only as an information processing method that executes the characteristic processes described above, but also as an information processing device having a characteristic configuration corresponding to the characteristic processes executed by the information processing method. Furthermore, the present disclosure can also be realized as a computer program that causes a computer to execute the characteristic processes included in such an information processing method. Therefore, the same effects as those of the above information processing method can also be achieved in the following other aspects.

[0030] (10) Another aspect of the present disclosure is an information processing device including a processor, which displays a bird's-eye view on a display of an information terminal, displays on the bird's-eye view a group of shooting location icons indicating a series of multiple shooting locations from a shooting start location to a shooting end location, accepts a user's selection of an arbitrary area on the bird's-eye view, and, if the area includes two or more groups of shooting location icons and the shooting locations of the two or more groups of shooting location icons are similar, extracts two or more pieces of data corresponding to the two or more similar groups of shooting location icons.

[0031] (11) An information processing program according to another aspect of the present disclosure displays a bird's-eye view on the display of an information terminal, displays on the bird's-eye view a group of photography location icons indicating a series of multiple photography locations from a photography start point to a photography end point, accepts a user's selection of an arbitrary area on the bird's-eye view, and, if the area includes two or more photography location icon groups and the photography locations of the two or more photography location icon groups are similar, causes a computer to function to extract two or more pieces of data corresponding to the two or more similar photography location icon groups.

[0032] (12) A non-transitory computer-readable recording medium according to another aspect of the present disclosure records the information processing program described in (11) above.

[0033] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components. Furthermore, in all embodiments, the respective contents can be combined.

[0034] FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to the present embodiment.

[0035] The information processing system 1 displays a bird's-eye view on the display 23 of the information terminal 20, displays a group of photography location icons on the bird's-eye view that indicate a series of multiple photography locations from a photography start location to a photography end location, accepts a user's selection of an arbitrary area on the bird's-eye view, and, if the area includes two or more photography location icon groups and the photography locations of the two or more photography location icon groups are similar, extracts two or more pieces of data corresponding to the two or more similar photography location icon groups. The information processing system 1 includes a server 10, an information terminal 20, a photography device 30, and a communication device 40.

[0036] The server 10 is an example of an information processing device and a computer. The server 10, the information terminal 20, and the communication device 40 are connected to each other so as to be able to communicate with each other via a network NT. An example of the network NT is the Internet. The server 10 is, for example, a cloud server configured with one or more computers. However, this is just one example, and the server 10 may be configured as an edge server or may be implemented in the information terminal 20. The aspect in which the server 10 is implemented in the information terminal 20 is an example of the aspect in which the information terminal 20 is configured as an information processing device.

[0037] The information terminal 20 is carried by a user. The user is, for example, a manager of a predetermined space in which the image capturing device 30 captures images. The predetermined space is, for example, a construction site. However, this is just one example, and the predetermined space may be a construction site, a factory, a store, an office, or the like. The information terminal 20 may be configured as a portable computer such as a smartphone or a tablet computer, or as a stationary computer. The information terminal 20 displays images on a display 23 under control of the server 10. Although one information terminal 20 is illustrated in the example of FIG. 1, multiple information terminals may be connected to the server 10 via a network NT. The information terminal 20 includes a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0038] The communication unit 21 is a communication interface that connects the information terminal 20 to the network NT. The communication unit 21 transmits instruction signals indicating various instructions received from the user by the operation unit 24 to the server 10. The communication unit 21 receives display data for displaying various display screens from the server 10.

[0039] The processor 22 is configured by, for example, a central processing unit (CPU), and displays a display screen indicated by the display data received by the communication unit 21 on the display 23 .

[0040] The display 23 is configured with various display devices such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays various display screens under the control of the processor 22.

[0041] The operation unit 24 is composed of, for example, a keyboard, a touch panel, a mouse, and the like, and accepts various instructions input by the user.

[0042] The communication device 40 is configured by a portable information terminal such as a smartphone or a tablet computer, and connects the image capturing device 30 to the network NT. The communication device 40 and the image capturing device 30 are connected to each other so as to be able to communicate with each other via a short-range wireless communication path such as Bluetooth (registered trademark) or a wireless LAN (Local Area Network).

[0043] The image capturing device 30 is, for example, an omnidirectional camera, and captures images at a predetermined frame rate. An omnidirectional camera is also called a 360-degree camera, and is a camera that can capture images in all directions of 360 degrees. The image capturing device 30 is, for example, a portable image capturing device carried by a photographer. The photographer may be, for example, a worker or a site supervisor at a construction site. The image capturing device 30 may be held in the photographer's hand, or may be worn on the photographer's body (for example, on the head). The image capturing device 30 may also be a regular camera.

[0044] The photographer moves through the construction site while taking pictures of the construction site with the camera device 30. At the start point of the shooting, the photographer points the camera device 30 in a predetermined direction and presses the camera button on the camera device 30 to start shooting. When the photographer reaches the end point of the shooting, he or she presses the camera button again to stop shooting. The predetermined direction is a direction that aligns with the blueprint displayed on the display 23. The predetermined direction is, for example, north, but is not limited to this. When the shooting is finished, the camera device 30 transmits the series of captured images to the communication device 40.

[0045] The image includes the date and time of the image capture. The date and time of the image capture is obtained, for example, by a clock provided in the image capture device 30. In this example, the image capture device 30 captures images at a predetermined frame rate (e.g., 10 frames per second), so the image capture location is defined in frame cycle units. However, this would result in an enormous amount of data, so the image capture location may be defined every predetermined time period (e.g., 1 second, 5 seconds, or 10 seconds).

[0046] The communication device 40 receives the multiple images transmitted by the image capturing device 30. The communication device 40 also displays a blueprint of the construction site and accepts designation by the photographer of the position of the image capturing start point and the position of the image capturing end point on the blueprint.

[0047] The start and end points of photography are specified by the photographer inputting instructions specifying the positions on the blueprint screen of the site displayed on the display of the communication device 40. Two-dimensional coordinate axes are defined on the blueprint screen. Therefore, the start and end points of photography are each defined by two-dimensional coordinate values. The start and end points of photography are used by the server 10 to specify the position of each photography point and the photography direction of each image.

[0048] The communication device 40 transmits photography information including the multiple captured images to the server 10 via the network NT. Photography information is generated each time a single photography operation is performed. One photography operation refers to a series of operations from when a worker holding the photography device 30 starts taking pictures at a construction site until when he finishes taking pictures. Multiple images are captured during one photography operation. The photography information includes the multiple images taken and meta information for each of the multiple images. The meta information includes a photography ID, a photography date and time, and a blueprint ID. The photography ID is an identifier for identifying the photography operation. The photography date and time is the date and time when the image was taken. The blueprint ID is an identifier for identifying a blueprint for a specified space corresponding to the image.

[0049] The server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 is configured, for example, by a central processing unit (CPU). The processor 11 includes an acquisition unit 111, a selection acceptance unit 112, a display control unit 113, and an extraction unit 114. The acquisition unit 111, the selection acceptance unit 112, the display control unit 113, and the extraction unit 114 may be realized by the processor 11 executing an information processing program, or may be configured by a dedicated hardware circuit such as an ASIC. The information processing program may be recorded on a non-transitory computer-readable recording medium.

[0050] The communication unit 13 is a communication interface that connects the server 10 to the network NT. The communication unit 13 receives shooting information transmitted by the communication device 40. The communication unit 13 receives instruction signals indicating various instructions received from the user from the information terminal 20. The communication unit 13 transmits display data for displaying various display screens to the information terminal 20.

[0051] The memory 12 is configured as a non-volatile rewritable storage device such as a hard disk drive or a solid state drive, etc. The memory 12 includes a design drawing information storage unit 121 and a shooting trajectory data storage unit 122.

[0052] The blueprint information storage unit 121 stores blueprint information. Blueprint information is image information showing a blueprint of a specified space. A blueprint ID that identifies the blueprint is associated with the blueprint information. The blueprint is an example of a bird's-eye view. The blueprint is a drawing showing the design of a construction site, and may be a floor plan, blueprint, map, or perspective view of the construction site. The bird's-eye view can also be referred to as a bird's-eye view, and may be a view from above or a view from a high place. Furthermore, if the construction site has multiple floors, the blueprint information includes blueprints for each of the multiple floors.

[0053] The acquisition unit 111 uses Visual SLAM (Simultaneous Localization and Mapping) technology to identify the location of each of the multiple images based on the shooting start point, shooting end point, and multiple images included in the shooting information received by the communication unit 13. The location of the shooting point is expressed as two-dimensional coordinate values ​​on the design drawing. The acquisition unit 111 also identifies the shooting direction of each of the multiple images using Visual SLAM technology. The shooting direction of each image is expressed, for example, as a three-dimensional polar coordinate vector.

[0054] The shooting trajectory data storage unit 122 stores multiple pieces of shooting trajectory data. The shooting trajectory data is data that associates multiple shooting images taken at a series of multiple shooting locations from the shooting start location to the shooting end location with a shooting ID, shooting start date and time, blueprint ID, and registrant ID. The registrant ID is information that identifies the registrant who registered the shooting trajectory data in the shooting trajectory data storage unit 122. The registrant is, for example, the photographer. The shooting trajectory data is generated each time the above-mentioned shooting information is received. The captured image includes meta information. The meta information includes the shooting ID, shooting date and time, shooting direction, shooting location, and blueprint ID. The shooting direction is the shooting direction of the camera device 30 that captured the image. The shooting location is location information (two-dimensional coordinate values) that indicates the shooting location of the image.

[0055] The communication unit 13 also receives an instruction to specify a floor of the construction site from the information terminal 20. The operation unit 24 of the information terminal 20 accepts the user's selection of a floor of the construction site, and the communication unit 21 transmits an instruction to the server 10 to specify the floor of the construction site selected by the user.

[0056] The acquisition unit 111 acquires, from the design drawing information storage unit 121, a design drawing corresponding to a floor of the construction site included in the instruction for specifying a floor of the construction site received by the communication unit 13. The acquisition unit 111 also acquires, from the design drawing information storage unit 122, shooting trajectory data associated with the design drawing ID of the acquired design drawing.

[0057] The display control unit 113 displays a blueprint corresponding to the predetermined space on the display 23 of the information terminal 20. The display control unit 113 displays the blueprint acquired by the acquisition unit 111 on the display 23 of the information terminal 20. The display control unit 113 transmits display data of the blueprint to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data transmitted by the server 10. The display 23 displays the blueprint received by the communication unit 21.

[0058] The display control unit 113 displays, on the blueprint, a group of photography location icons indicating a series of multiple photography locations from the photography start location to the photography end location. The group of photography location icons is associated with multiple photographed images taken at multiple photography locations. The display control unit 113 displays the group of photography location icons on the blueprint based on the photography trajectory data acquired by the acquisition unit 111. When multiple photography trajectory data are associated with the blueprint, the display control unit 113 displays multiple photography location icon groups on the blueprint.

[0059] 2 is a diagram showing an example of a design drawing screen 201 on which a plurality of groups of photography location icons are mapped in this embodiment. The design drawing screen 201 displays a design drawing 211 of a predetermined floor of a predetermined construction site, a color bar 212 indicating the correspondence between photography date and time and color, a list display field 213, and a photographed image display button 214. A plurality of groups of photography location icons 221, 222, 223, 224, 225, and 226 are superimposed on the design drawing 211.

[0060] The photography location icon group 221 is a collection of multiple photography location icons indicating a series of multiple photography locations from the photography start point to the photography end point. One photography location icon is associated with one photographed image taken at one photography location. In this example, the photography location icon is composed of a circular image of a predetermined size. The photography location icon group 221 displayed here corresponds to multiple photography locations belonging to one photography operation. The route tracing from the photography location icon indicating the photography start point to the photography location icon indicating the photography end point indicates the trajectory followed by the photographer in this photography operation. Note that the other photography location icon groups 222, 223, 224, 225, and 226 are similar to the photography location icon group 221.

[0061] The image capture device 30 captures images at a predetermined frame rate. In this case, the number of multiple images is expressed as the frame rate (fps) * capture time (seconds). In FIG. 2, multiple image capture location icons are displayed overlapping each other along the path of the photographer's movement. From the capture start point to the capture end point, new image capture location icons are displayed overlapping each other in order.

[0062] The multiple photography location icon groups 221, 222, 223, 224, 225, and 226 each have a different photography date and time (photography start date and time). The display control unit 113 displays the photography location icon groups 221, 222, 223, 224, 225, and 226 in colors corresponding to the photography date and time. In FIG. 2, the photography location icon groups 221, 222, 223, 224, 225, and 226 are displayed in different colors for each photography date. For example, the photography location icon groups 221, 222, 223, and 224 indicate multiple photography locations photographed on June 23, 2023, and the photography location icon groups 225 and 226 indicate multiple photography locations photographed on July 3, 2023. The photography date of each photography location icon group can be identified by the color of the photography location icon group and the color bar 212.

[0063] Furthermore, when multiple image capture location icon groups overlap, image capture location icon groups with newer image capture dates and times are displayed above image capture location icon groups with older image capture dates and times. For example, the image capture date and time of the first image capture location icon group 223 is "2023 / 06 / 23 17:02:58," and the image capture date and time of the second image capture location icon group 225 is "2023 / 07 / 03 17:52:58," so the second image capture location icon group 225 is displayed above the first image capture location icon group 223.

[0064] The list display field 213 includes a name 2131 for identifying the group of photography location icons, a check box 2132 for selecting the group of photography location icons, and a photography location icon 2133. The name 2131 is represented by the photography date and time and the name of the registrant. The photography date and time is, for example, the date and time when photography started. The name 2131 in the list display field 213 represents the names of the photography location icons 221, 222, 223, 224, 225, and 226 in order from top to bottom. The user presses the check box 2132 to the left of the group of photography location icons for which the photographed image is to be displayed. When the check box 2132 is pressed, the display control unit 113 changes the display of the check box 2132 to a checked state.

[0065] The photographed image display button 214 is a button for displaying photographed images. When the photographed image display button 214 is pressed, the display control unit 113 displays the photographed images corresponding to the group of photographing location icons selected by the check boxes 2132.

[0066] The selection receiving unit 112 receives a selection of an arbitrary area on the design drawing by the user.

[0067] FIG. 3 is a diagram showing an example of the design drawing screen 201 in which an area 215 is drawn on the design drawing 211 in this embodiment.

[0068] The selection receiving unit 112 receives a user's selection of an arbitrary area 215 on the design drawing 211. The area 215 has, for example, a rectangular shape. The selection receiving unit 112 receives a mouse drag operation by the user. The user selects the rectangular area 215 by dragging the mouse. In this case, the display 23 draws the rectangular area 215 on the design drawing 211, with the straight line connecting the point where the mouse dragging started and the point where the mouse dragging ended as the diagonal line.

[0069] When selecting a group of photography location icons for displaying photographed images, the user selects the area 215 including the group of photography location icons. In Fig. 3, the area 215 including part of the groups of photography location icons 223 and 225 is selected.

[0070] The selection receiving unit 112 may receive a mouse click operation by the user. The user selects a rectangular area 215 by clicking the mouse. In this case, the display 23 may draw a circular or rectangular area 215 of a predetermined size on the design drawing 211, with the point where the mouse was clicked as the center.

[0071] The display control unit 113 determines whether two or more groups of photography location icons are included in the area. Furthermore, if two or more groups of photography location icons are included in the area, the display control unit 113 determines whether the photography locations of the two or more groups of photography location icons are similar. If at least one of the distance between the photography start points of each of the two or more groups of photography location icons and the distance between the photography end points of each of the two or more groups of photography location icons is equal to or less than a threshold, the display control unit 113 determines that the photography locations of the two or more groups of photography location icons are similar.

[0072] 3 , the two or more image capture location icon groups in the area 215 include a first image capture location icon group 223 and a second image capture location icon group 225. If at least one of the distance between the image capture start point of the first image capture location icon group 223 and the image capture start point of the second image capture location icon group 225 and the distance between the image capture end point of the first image capture location icon group 223 and the image capture end point of the second image capture location icon group 225 is equal to or less than a threshold, the display control unit 113 determines that the image capture position of the first image capture location icon group 223 and the image capture position of the second image capture location icon group 225 are similar.

[0073] When two or more groups of photography location icons are included in the area and the photography locations of the two or more groups of photography location icons are similar, the extraction unit 114 extracts two or more pieces of data corresponding to the two or more similar groups of photography location icons. Here, the extracted data are photographed images taken at the photography locations.

[0074] 3 , when at least one of the distance between the photography start point of the first photography location icon group 223 and the photography start point of the second photography location icon group 225 and the distance between the photography end point of the first photography location icon group 223 and the photography end point of the second photography location icon group 225 is equal to or less than a threshold, the extraction unit 114 extracts first data corresponding to the first photography location icon group 223 and second data corresponding to the second photography location icon group 225. Here, the extracted first data and second data are photographed images.

[0075] In the example of Figure 3, the distance between the start point of photography of the first photography location icon group 223 and the start point of photography of the second photography location icon group 225 is less than a threshold, and the distance between the end point of photography of the first photography location icon group 223 and the end point of photography of the second photography location icon group 225 is less than a threshold.

[0076] Furthermore, the display control unit 113 displays names 2131 for identifying the group of photography location icons on the display 23 along with the blueprint 211. When two or more groups of photography location icons are included in the area and the photography locations of the two or more groups of photography location icons are similar, the display control unit 113 changes the display mode of the two or more groups of photography location icons and the display mode of the two or more names corresponding to the two or more groups of photography location icons. For example, the display control unit 113 changes the color of the two or more groups of photography location icons and changes the color of the two or more names to the same color as the color of the two or more groups of photography location icons.

[0077] In the example of FIG. 3 , the display control unit 113 changes the color of the first image capture location icon group 223 and the second image capture location icon group 225 to red, and also changes the color of the names of the first image capture location icon group 223 and the second image capture location icon group 225 to the same red as the color of the first image capture location icon group 223 and the second image capture location icon group 225. In the list display field 213, the name of the first image capture location icon group 223 is "2023 / 06 / 23 17:02:58 Registered by: A," and the name of the second image capture location icon group 225 is "2023 / 07 / 03 17:52:58 Registered by: A." In FIG. 3 , the change in color of the names of the first image capture location icon group 223 and the second image capture location icon group 225 is indicated by changing the thickness of the characters. The color of the image capture location icons 2133 in the list display field 213 also changes along with the names.

[0078] Furthermore, the display control unit 113 displays the two or more captured images extracted by the extraction unit 114 on the display 23. The display control unit 113 transmits display data of the two or more captured images to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data transmitted by the server 10. The display 23 displays the two or more captured images received by the communication unit 21.

[0079] 3 , the operation unit 24 of the information terminal 20 accepts user input of check boxes 2132 corresponding to the first image capture location icon group 223 and the second image capture location icon group 225 for displaying photographed images. The user selects the first image capture location icon group 223 and the second image capture location icon group 225 for displaying photographed images by inputting check marks in the check boxes 2132 corresponding to the first image capture location icon group 223 and the second image capture location icon group 225 for displaying photographed images. Then, when the user presses the photographed image display button 214, the communication unit 21 transmits information indicating the first image capture location icon group 223 and the second image capture location icon group 225 selected by the user to the server 10. The communication unit 13 receives information indicating the first image capture location icon group 223 and the second image capture location icon group 225 selected by the user from the information terminal 20.

[0080] The extraction unit 114 extracts photographed images corresponding to the selected first photographing location icon group 223 and photographed images corresponding to the selected second photographing location icon group 225. The display control unit 113 displays the photographed images corresponding to the first photographing location icon group 223 and the photographed images corresponding to the selected second photographing location icon group 225, which have been extracted by the extraction unit 114, on the display 23.

[0081] FIG. 4 is a diagram showing an example of the captured image screen 202 displayed on the display 23 when the area 215 is selected in this embodiment.

[0082] The photographed image screen 202 displays a photographed image 231 photographed at a photographing location corresponding to one photographing location icon in the first photographing location icon group 223, and a photographed image 232 photographed at a photographing location corresponding to one photographing location icon in the second photographing location icon group 225. The extraction unit 114 extracts from the photographing trajectory data storage unit 122 the photographed image 231 photographed at a photographing location corresponding to the photographing icon with the oldest photographing date and time in the area 215 among the first photographing location icon group 223, and extracts from the photographing trajectory data storage unit 122 the photographed image 232 photographed at a photographing location corresponding to the photographing icon with the oldest photographing date and time in the area 215 among the second photographing location icon group 225. The display control unit 113 displays the photographed image 231 and the photographed image 232 extracted by the extraction unit 114 on the display 23.

[0083] Here, the display control unit 113 displays rectangular photographed images 231 and 232 corresponding to the line of sight cut out from the omnidirectional image on the display 23. The line of sight direction of the photographed image 231 is the same as the line of sight direction of the photographed image 232. The photographed image screen 202 includes a date field 233 indicating the photographing location icon and the date on which the photographed image 231 was taken, and a date field 234 indicating the photographing location icon and the date on which the photographed image 232 was taken.

[0084] In addition, the extraction unit 114 may extract from the shooting trajectory data storage unit 122 a photographed image 231 taken at a shooting location corresponding to the shooting icon with the most recent shooting date and time in area 215 among the first shooting location icon group 223, and extract from the shooting trajectory data storage unit 122 a photographed image 232 taken at a shooting location corresponding to the shooting icon with the most recent shooting date and time in area 215 among the second shooting location icon group 225.

[0085] In addition, in the present embodiment, the data extracted by the extraction unit 114 is a captured image, but the present disclosure is not particularly limited to this. The data extracted by the extraction unit 114 may be audio data collected at the capture location or text data input at the capture location.

[0086] Next, the image display process of the server 10 in this embodiment will be described.

[0087] FIG. 5 is a first flowchart showing an example of the image display process of the server 10 in this embodiment, and FIG. 6 is a second flowchart showing an example of the image display process of the server 10 in this embodiment.

[0088] First, in step S1, the selection receiving unit 112 receives a user's selection of a floor of the construction site. The display control unit 113 displays a floor selection screen on the display 23 to receive the selection of one of the multiple floors at the construction site. The floor selection screen displays the name of the construction site and multiple selectable floor names. The operation unit 24 of the information terminal 20 receives the user's selection of the name of the construction site and the floor name of the construction site. The user selects one of the multiple floor names. The communication unit 21 transmits an instruction to the server 10 to specify the floor of the construction site selected by the user. The communication unit 13 receives the instruction to specify the floor of the construction site from the information terminal 20.

[0089] Next, in step S2, the display control unit 113 displays a design drawing corresponding to the floor selected by the user on the display 23. The acquisition unit 111 acquires, from the design drawing information storage unit 121, a design drawing corresponding to the floor of the construction site included in the instruction for specifying the floor of the construction site received by the communication unit 13. The display control unit 113 displays the design drawing acquired by the acquisition unit 111 on the display 23.

[0090] Next, in step S3, the display control unit 113 determines whether there are multiple pieces of photography trajectory data associated with the selected design drawing. The acquisition unit 111 acquires from the photography trajectory data storage unit 122 at least one piece of photography trajectory data that is associated with the design drawing ID of the acquired design drawing and that was taken at a different date and time.

[0091] If it is determined here that there are not multiple pieces of shooting trajectory data (NO in step S3), in step S4, the display control unit 113 displays one group of shooting location icons on the blueprint based on the acquired one piece of shooting trajectory data. One group of shooting location icons is displayed so that it can be selected. The display control unit 113 transmits display data of one group of shooting location icons indicating a series of multiple shooting locations from the shooting start point to the shooting end point of the multiple shooting images included in one piece of shooting trajectory data to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data transmitted by the server 10. The display 23 displays one group of shooting location icons received by the communication unit 21.

[0092] Next, in step S5, the display control unit 113 determines whether one photography location icon has been selected from the group of one displayed photography location icons. The operation unit 24 of the information terminal 20 accepts the user's selection of one photography location icon from the group of one photography location icons. The user selects one photography location icon from the group of one photography location icons. The communication unit 21 transmits an instruction to the server 10 to designate the one photography location icon selected by the user. The communication unit 13 receives the instruction to designate the one photography location icon from the information terminal 20.

[0093] If it is determined that no photography location icon has been selected (NO in step S5), the determination process in step S5 is repeated until one photography location icon is selected.

[0094] On the other hand, if it is determined that one photography location icon has been selected (YES in step S5), in step S6, the display control unit 113 displays one photographed image corresponding to the one selected photography location icon on the display 23. The photography location icon represents a photography location on the blueprint. The display control unit 113 displays one photographed image taken at the one photography location corresponding to the one photography location icon. In detail, the display control unit 113 transmits display data of the one photographed image to the information terminal 20 using the communication unit 13. In the information terminal 20, the processor 22 displays the display data received by the communication unit 21 on the display 23. As a result, one photographed image is displayed on the display 23.

[0095] If there is no shooting trajectory data associated with the selected design drawing, the display control unit 113 may end the image display process.

[0096] On the other hand, if it is determined that there are multiple pieces of shooting trajectory data (YES in step S3), in step S7, the display control unit 113 displays a group of multiple shooting location icons on the design drawing based on the acquired multiple pieces of shooting trajectory data. The display control unit 113 transmits display data of the group of multiple shooting location icons indicating a series of multiple shooting locations from the shooting start point to the shooting end point of the multiple shooting images included in each of the multiple shooting trajectory data to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data transmitted by the server 10. The display 23 displays the group of multiple shooting location icons received by the communication unit 21.

[0097] Next, in step S8, the display control unit 113 displays the names of the group of multiple photography location icons together with the blueprint. The display control unit 113 displays the names of the group of multiple photography location icons displayed on the blueprint together with check boxes.

[0098] Next, in step S9, the selection receiving unit 112 receives the user's selection of an arbitrary area on the design drawing. The operation unit 24 of the information terminal 20 receives the user's selection of an arbitrary area on the design drawing. The user selects a rectangular area on the design drawing. The communication unit 21 transmits information for identifying the area selected by the user to the server 10. The information for identifying the area is the coordinates on the design drawing of the four vertices of the rectangular area. The communication unit 13 receives the information for identifying the area from the information terminal 20.

[0099] Next, in step S10 , the display control unit 113 determines whether or not there are two or more groups of shooting location icons within the area selected by the selection receiving unit 112 .

[0100] If it is determined that there are no two or more photography location icon groups within the area (NO in step S10), then in step S11 the display control unit 113 determines whether there is one photography location icon group within the area selected by the selection receiving unit 112. If it is determined that there is no one photography location icon group within the area (NO in step S11), the processing ends.

[0101] On the other hand, if it is determined that one photography location icon group is present in the area (YES in step S11), in step S12, the display control unit 113 changes the display mode of the one photography location icon group and the display mode of one name corresponding to the one photography location icon group. For example, the display control unit 113 changes the color of the one photography location icon group to a color different from that of the other photography location icon groups, and changes the color of the one name to the same color as the color of the one photography location icon group.

[0102] On the other hand, if it is determined that there are two or more groups of photography location icons within the area (YES in step S10), in step S13, the display control unit 113 determines whether the photography locations of the two or more groups of photography location icons are similar. In this case, if at least one of the distance between the photography start points of each of the two or more groups of photography location icons and the distance between the photography end points of each of the two or more groups of photography location icons is equal to or less than a threshold, the display control unit 113 determines that the photography locations of the two or more groups of photography location icons are similar. Furthermore, if at least one of the distance between the photography start points of each of the two or more groups of photography location icons and the distance between the photography end points of each of the two or more groups of photography location icons is not equal to or less than a threshold, the display control unit 113 determines that the photography locations of the two or more groups of photography location icons are not similar.

[0103] If it is determined that the shooting locations of two or more shooting location icon groups are not similar (NO in step S13), the process proceeds to step S15.

[0104] On the other hand, if it is determined that the shooting locations of two or more groups of shooting location icons are similar (YES in step S13), in step S14, the display control unit 113 changes the display mode of the two or more groups of shooting location icons and the display mode of the two or more names corresponding to the two or more groups of shooting location icons. For example, the display control unit 113 changes the color of the two or more groups of shooting location icons within the area to a color different from that of other groups of shooting location icons outside the area, and changes the color of the two or more names to the same color as the color of the two or more groups of shooting location icons.

[0105] Next, in step S15, the selection receiving unit 112 receives a selection of at least one group of photography location icons for displaying photographed images. At this time, the operation unit 24 of the information terminal 20 receives a user selection of at least one group of photography location icons for displaying photographed images. The user selects at least one group of photography location icons for displaying photographed images by inputting a check mark in a checkbox corresponding to the name of the at least one group of photography location icons for displaying photographed images. Then, when the user presses the photographed image display button, the communication unit 21 transmits information indicating the at least one group of photography location icons selected by the user to the server 10. The communication unit 13 receives information indicating the at least one group of photography location icons selected by the user from the information terminal 20.

[0106] When two or more groups of photography location icons are included in the area and the photography locations of the two or more groups of photography location icons are similar, the selection receiving unit 112 accepts the selection of two or more groups of photography location icons for displaying the photographed images. When one group of photography location icons is included in the area, the selection receiving unit 112 accepts the selection of one group of photography location icons for displaying the photographed images.

[0107] Next, in step S16, the extraction unit 114 extracts at least one photographed image corresponding to at least one group of photographing location icons selected by the user from the photographing trajectory data storage unit 122. At this time, the extraction unit 114 extracts from the photographing trajectory data storage unit 122 a photographed image that was photographed at a photographing location corresponding to the photographing icon with the oldest photographing date and time within the area, among the group of photographing location icons.

[0108] When two or more groups of photography location icons are included within the area and the photography positions of the two or more groups of photography location icons are similar, the extraction unit 114 extracts two or more photography images corresponding to the two or more groups of photography location icons selected by the user from the photography trajectory data storage unit 122. When one group of photography location icons is included within the area, the extraction unit 114 extracts one photography image corresponding to the one group of photography location icons selected by the user from the photography trajectory data storage unit 122.

[0109] Next, in step S17, the display control unit 113 displays at least one captured image extracted by the extraction unit 114 on the display 23. In detail, the display control unit 113 transmits display data of the at least one captured image to the information terminal 20 using the communication unit 13. In the information terminal 20, the processor 22 displays the display data received by the communication unit 21 on the display 23. As a result, at least one captured image is displayed on the display 23.

[0110] When two or more groups of photography location icons are included in the area and the photography locations of the two or more groups of photography location icons are similar, the display control unit 113 displays two or more photographed images extracted by the extraction unit 114 on the display 23. When one group of photography location icons is included in the area, the display control unit 113 displays one photographed image extracted by the extraction unit 114 on the display 23.

[0111] In this way, by selecting an area on the design drawing that includes two or more desired groups of photography location icons, it becomes possible to easily access two or more pieces of data corresponding to the two or more desired groups of photography location icons, thereby improving user convenience. Furthermore, by selecting an area on the design drawing that includes two or more desired groups of photography location icons, two or more pieces of data corresponding to the two or more groups of photography location icons are extracted, which reduces the computer processing steps required to access two or more pieces of data and reduces the burden on the computer.

[0112] In this embodiment, the display control unit 113 determines whether the shooting positions of two or more groups of shooting location icons are similar by determining whether at least one of the distance between the shooting start points of each of two or more groups of shooting location icons and the distance between the shooting end points of each of two or more groups of shooting location icons is below a threshold, but the present disclosure is not limited to this.

[0113] In a first modification of the present embodiment, the display control unit 113 may determine whether the shooting locations of two or more groups of shooting location icons are similar by determining whether the two or more groups of shooting location icons overlap each other within the area. In this case, in step S13 of FIG. 6 , the display control unit 113 may determine that the shooting locations of the two or more groups of shooting location icons are similar if the two or more groups of shooting location icons overlap each other within the area. Furthermore, the display control unit 113 may determine that the shooting locations of the two or more groups of shooting location icons are not similar if the two or more groups of shooting location icons do not overlap each other within the area.

[0114] That is, when two or more groups of photography location icons are included in a region and the two or more groups of photography location icons overlap each other within the region, the extraction unit 114 may extract two or more pieces of data corresponding to the two or more groups of photography location icons. Furthermore, when two or more groups of photography location icons are included in a region and the two or more groups of photography location icons overlap each other within the region, the display control unit 113 may change the display mode of the two or more groups of photography location icons and the display mode of two or more names corresponding to the two or more groups of photography location icons.

[0115] 3 , when the first image capturing location icon group 223 and the second image capturing location icon group 225 are included in the region 215 and the first image capturing location icon group 223 and the second image capturing location icon group 225 overlap in the region 215, the extraction unit 114 may extract captured images corresponding to the first image capturing location icon group 223 and the second image capturing location icon group 225. Furthermore, when the first image capturing location icon group 223 and the second image capturing location icon group 225 are included in the region 215 and the first image capturing location icon group 223 and the second image capturing location icon group 225 overlap in the region 215, the display control unit 113 may change the display mode of the first image capturing location icon group 223, the display mode of the second image capturing location icon group 225, the display mode of the names corresponding to the first image capturing location icon group 223, and the display mode of the names corresponding to the second image capturing location icon group 225.

[0116] In a second modification of the present embodiment, the display control unit 113 may determine whether the shooting locations of two or more groups of shooting location icons are similar by determining whether the shortest distance between each of the multiple shooting locations corresponding to a first group of shooting location icons in the region and each of the multiple shooting locations corresponding to a second group of shooting location icons in the region is equal to or less than a threshold. In this case, in step S13 of FIG. 6 , the display control unit 113 may determine that the shooting locations of two or more groups of shooting location icons are similar if the shortest distance between each of the multiple shooting locations corresponding to the first group of shooting location icons in the region and each of the multiple shooting locations corresponding to the second group of shooting location icons in the region is equal to or less than a threshold. Furthermore, the display control unit 113 may determine that the shooting locations of two or more groups of shooting location icons are not similar if the shortest distance between each of the multiple shooting locations corresponding to the first group of shooting location icons in the region and each of the multiple shooting locations corresponding to the second group of shooting location icons in the region is not equal to or less than a threshold.

[0117] That is, when a first image capture location icon group and a second image capture location icon group are included in the region, and the shortest distance among the distances between each of the plurality of image capture locations corresponding to the first image capture location icon group in the region and each of the plurality of image capture locations corresponding to the second image capture location icon group in the region is equal to or less than a threshold, the extraction unit 114 may extract first data corresponding to the first image capture location icon group and second data corresponding to the second image capture location icon group. Furthermore, when a first image capture location icon group and a second image capture location icon group are included in the region, and the shortest distance among the distances between each of the plurality of image capture locations corresponding to the first image capture location icon group in the region and each of the plurality of image capture locations corresponding to the second image capture location icon group in the region is equal to or less than a threshold, the display control unit 113 may change the display mode of the first image capture location icon group, the display mode of the second image capture location icon group, the display mode of the names corresponding to the first image capture location icon group, and the display mode of the names corresponding to the second image capture location icon group.

[0118] In the example of Figure 3, the extraction unit 114 may extract photographed images corresponding to the first group of photographing location icons 223 and the second group of photographing location icons 225 when the area 215 includes a first group of photographing location icons 223 and a second group of photographing location icons 225, and the shortest distance between each of the multiple photographing locations corresponding to the first group of photographing location icons 223 in the area 215 and each of the multiple photographing locations corresponding to the second group of photographing location icons 225 in the area 215 is less than or equal to a threshold value. In addition, when the area 215 includes a first group of photographing location icons 223 and a second group of photographing location icons 225, and the shortest distance between each of the multiple photographing locations corresponding to the first group of photographing location icons 223 in the area 215 and each of the multiple photographing locations corresponding to the second group of photographing location icons 225 in the area 215 is below a threshold, the display control unit 113 may change the display mode of the first group of photographing location icons 223, the display mode of the second group of photographing location icons 225, the display mode of the names corresponding to the first group of photographing location icons 223, and the display mode of the names corresponding to the second group of photographing location icons 225.

[0119] Note that some or all of the functions of the device according to the embodiment of the present disclosure may be realized by a processor such as a CPU executing a program.

[0120] Furthermore, all the numbers used above are merely examples to specifically explain the present disclosure, and the present disclosure is not limited to the numbers used as examples.

[0121] The order in which the steps are executed in the above flowchart is merely an example for specifically explaining the present disclosure, and other orders may be used as long as similar effects are obtained. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0122] The technology disclosed herein is useful as a technology for displaying images because it enables easy access to two or more pieces of data corresponding to two or more desired groups of shooting location icons, thereby improving user convenience.

Claims

1. An information processing method executed by a computer, comprising: displaying a bird's-eye view on an information terminal display; displaying, on the bird's-eye view, a group of photography location icons indicating a series of multiple photography locations from a photography start point to a photography end point; accepting a user's selection of an arbitrary area on the bird's-eye view; and, if two or more photography location icon groups are included within the area and the photography locations of the two or more photography location icon groups are similar, extracting two or more pieces of data corresponding to the two or more similar photography location icon groups.

2. The information processing method of claim 1, wherein the extraction of the two or more pieces of data includes extracting the two or more pieces of data corresponding to the two or more groups of shooting location icons when at least one of the distance between the shooting start points of each of the two or more groups of shooting location icons and the distance between the shooting end points of each of the two or more groups of shooting location icons is equal to or less than a threshold value.

3. The information processing method according to claim 1, wherein the extraction of the two or more pieces of data includes extracting the two or more pieces of data corresponding to the two or more groups of shooting location icons when the two or more groups of shooting location icons overlap within the area.

4. An information processing method as described in claim 1, wherein the two or more groups of photographing location icons include a first group of photographing location icons and a second group of photographing location icons, and the extraction of the two or more pieces of data includes extracting first data corresponding to the first group of photographing location icons and second data corresponding to the second group of photographing location icons when the shortest distance between each of the multiple photographing locations corresponding to the first group of photographing location icons in the area and each of the multiple photographing locations corresponding to the second group of photographing location icons in the area is equal to or less than a threshold value.

5. The information processing method according to any one of claims 1 to 4, further comprising displaying on said display a name for identifying said group of shooting location icons together with said bird's-eye view.

6. The information processing method according to claim 5, further comprising, when the area contains two or more groups of photography location icons and the photography locations of the two or more groups of photography location icons are similar, changing the display mode of the two or more groups of photography location icons and the display mode of two or more names corresponding to the two or more groups of photography location icons.

7. An information processing method according to claim 6, wherein the change in the display mode includes changing the color of the two or more shooting location icons and changing the color of the two or more names to the same color as the two or more shooting location icons.

8. The information processing method according to any one of claims 1 to 4, wherein the data is an image taken at a shooting location.

9. The information processing method according to any one of claims 1 to 4, wherein displaying the group of shooting location icons includes displaying the group of shooting location icons in a color corresponding to the shooting date and time.

10. An information processing device having a processor, wherein the processor displays a bird's-eye view on an information terminal display, displays on the bird's-eye view a group of shooting location icons indicating a series of multiple shooting locations from a shooting start point to a shooting end point, accepts a user's selection of an arbitrary area on the bird's-eye view, and, if two or more shooting location icon groups are included within the area and the shooting positions of the two or more shooting location icon groups are similar, extracts two or more pieces of data corresponding to the two or more similar shooting location icon groups.

11. An information processing program that causes a computer to function as follows: displaying a bird's-eye view on an information terminal display; displaying, on said bird's-eye view, a group of shooting location icons indicating a series of multiple shooting locations from a shooting start point to a shooting end point; accepting a user's selection of any area on said bird's-eye view; and, if said area includes two or more shooting location icon groups and the shooting positions of the two or more shooting location icon groups are similar, extracting two or more pieces of data corresponding to the two or more similar shooting location icon groups.