Information processing method, information processing device, and information processing program

Abstract

The information processing method includes: displaying an overview on a display of an information terminal; displaying a photographing location icon group representing a plurality of photographing locations in a series from a photographing start location to a photographing end location in the overview; accepting a user's selection of an arbitrary area on the overview; and extracting two or more pieces of data corresponding to two or more photographing location icon groups that are similar in photographing position in a case where two or more photographing location icon groups are included in the area.

Description

Technical Field

[0001] This disclosure relates to techniques for displaying images. Background Technology

[0002] For example, in the technology of Patent Document 1, the following technology is disclosed: determining the range from the desired first position to the second position in the group of main images of the scene, displaying the main image of the determined range, and displaying a pointer indicating the corresponding location of the object scene represented by the main image in the display on a drawing image representing the object scene.

[0003] However, the existing technology described above does not disclose how to access more than two data points corresponding to the desired group of more than two shooting location icons, which requires further improvement. Furthermore, accessing more than two data points increases the number of computer processing steps, raising concerns about increased computer load.

[0004] Prior art literature

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2022-24440 Summary of the Invention

[0007] This disclosure was made to solve the above-mentioned problems, and its purpose is to provide a technology that enables easy access to more than two data points corresponding to two or more desired shooting location icon groups, improves user convenience, reduces the computer processing steps required to access more than two data points, and reduces the computer's burden.

[0008] The information processing method disclosed herein is executed by a computer and includes: displaying a bird's-eye view on a display of an information terminal; displaying a group of shooting location icons representing a series of multiple shooting locations from the start point of shooting to the end point of shooting in the bird's-eye view; accepting a user's selection of any area on the bird's-eye view; and, if the area includes two or more groups of shooting location icons and the shooting positions of the two or more groups of shooting location icons are similar, extracting two or more data corresponding to the similar two or more groups of shooting location icons.

[0009] According to this disclosure, it is possible to easily access more than two data points corresponding to two or more desired shooting location icon groups, thereby improving user convenience. Furthermore, it reduces the computer processing steps required to access more than two data points, thus alleviating the computer's burden. Attached Figure Description

[0010] Figure 1This is a diagram showing the overall structure of the information processing system in this embodiment.

[0011] Figure 2 This is an example of a design drawing screen showing a group of multiple shooting location icons mapped in this embodiment.

[0012] Figure 3 This is an example of a design drawing screen showing an area drawn on a design drawing in this embodiment.

[0013] Figure 4 This diagram shows an example of a captured image displayed on the monitor when a region is selected in this embodiment.

[0014] Figure 5 This is a first flowchart illustrating an example of the image display processing of the server in this embodiment.

[0015] Figure 6 This is a second flowchart illustrating an example of the image display processing of the server in this embodiment. Detailed Implementation

[0016] (The understanding that forms the basis of this disclosure)

[0017] In the aforementioned prior art, a series of main image groups captured at a scene are considered as one; for example, multiple main image groups captured on different dates and times are not displayed. Furthermore, in the prior art, the location of one main image within a main image group is displayed on the scene map, but the locations of the individual main image groups are not displayed on the scene map. Therefore, the prior art does not disclose how to access data corresponding to the desired main image group from multiple main image groups. Additionally, accessing more than two data points increases the number of computer processing steps, raising concerns about increased computer load.

[0018] To address the above issues, the following technology has been disclosed.

[0019] (1) The information processing method involved in one aspect of this disclosure is executed by a computer, the information processing method comprising: displaying a bird's-eye view on the display of an information terminal; displaying a group of shooting location icons representing a series of multiple shooting locations from the shooting start point to the shooting end point in the bird's-eye view; accepting a user's selection of any area on the bird's-eye view; and extracting two or more data corresponding to the similar two or more shooting location icon groups when the area includes two or more shooting location icon groups and the shooting positions of the two or more shooting location icon groups are similar.

[0020] According to this structure, by selecting an area on the bird's-eye view that includes a group of icons representing two or more desired shooting locations, users can easily access more than two data points corresponding to those groups, thus improving user convenience. Furthermore, by selecting an area on the bird's-eye view that includes a group of icons representing two or more desired shooting locations and extracting more than two data points corresponding to those groups, the number of computer processing steps required to access more than two data points can be reduced, thus alleviating the computer's burden.

[0021] (2) In the information processing method described in (1) above, the extraction of the two or more data may include: when at least one of the distance between the shooting start locations of the two or more shooting location icon groups and the distance between the shooting end locations of the two or more shooting location icon groups is below a threshold, the two or more data corresponding to the two or more shooting location icon groups are extracted.

[0022] According to this structure, by determining whether at least one of the distance between the starting points of two or more shooting location icon groups and the distance between the ending points of two or more shooting location icon groups is below a threshold, it is easy to determine whether the shooting positions of two or more shooting location icon groups are similar.

[0023] (3) In the information processing method described in (1) above, the extraction of the two or more data may include: when the two or more shooting location icon groups overlap in the area, extracting the two or more data corresponding to the two or more shooting location icon groups.

[0024] Based on this structure, by determining whether two or more groups of shooting location icons overlap within a region, it is easy to determine whether the shooting locations of two or more groups of shooting location icons are similar.

[0025] (4) In the information processing method described in (1) above, the two or more shooting location icon groups may include: a first shooting location icon group and a second shooting location icon group, and the extraction of the two or more data includes: when the shortest distance among the distances to each shooting location of the multiple shooting locations corresponding to the first shooting location icon group in the region and the distances to each shooting location of the multiple shooting locations corresponding to the second shooting location icon group in the region is below a threshold, extracting the first data corresponding to the first shooting location icon group and the second data corresponding to the second shooting location icon group.

[0026] Based on this structure, by determining whether the shortest distance among the distances to each of the multiple shooting locations corresponding to the first shooting location icon group in the area and the multiple shooting locations corresponding to the second shooting location icon group in the area is below a threshold, it is easy to determine whether the shooting positions of two or more shooting location icon groups are similar.

[0027] (5) In any of the information processing methods described in (1) to (4) above, the information processing method may also further include: matching the name of the group of icons used to identify the shooting location with the bird's-eye view. Figure 1 It is displayed on the monitor.

[0028] According to this structure, the names of the icon groups used to identify the shooting location are related to the bird's-eye view. Figure 1 The location is displayed on the monitor, allowing users to confirm the name of the group of icons representing the shooting location.

[0029] (6) In the information processing method described in (5) above, the information processing method may also include: when there are two or more shooting location icon groups in the area and the shooting locations of the two or more shooting location icon groups are similar, changing the display mode of the two or more shooting location icon groups and the display mode of two or more names corresponding to the two or more shooting location icon groups.

[0030] According to this structure, by selecting an area on the bird's-eye view that includes a group of two or more desired shooting locations, and changing the display method of the group of two or more desired shooting locations' icons and the display method of the two or more names corresponding to the group of two or more desired shooting locations' icons, users can easily identify the names of the group of two or more desired shooting locations' icons.

[0031] (7) In the information processing method described in (6) above, the change of the display mode may include: changing the color of the two or more shooting location icon groups, and changing the color of the two or more names to the same color as the color of the two or more shooting location icon groups.

[0032] According to this structure, since the colors of two or more shooting location icon groups are changed, and the colors of two or more names are changed to the same color as the colors of the two or more shooting location icon groups, it is easy to identify the names of the two or more shooting location icon groups selected by the user.

[0033] (8) In any of the information processing methods described in (1) to (7) above, the data may be an image captured at the shooting location.

[0034] According to this structure, since more than two images taken at more than two shooting locations corresponding to more than two shooting location icon groups are extracted, more than two images can be displayed, and more than two images with similar shooting locations can be compared.

[0035] (9) In any of the information processing methods described in (1) to (8) above, the display of the shooting location icon group may include: displaying the shooting location icon group in a color corresponding to the shooting date and time.

[0036] According to this structure, since the group of shooting location icons is displayed in colors corresponding to the shooting date and time, users can intuitively identify the shooting date and time by checking the colors of the shooting location icon group.

[0037] Furthermore, this disclosure can be implemented not only as an information processing method that performs the characteristic processing described above, but also as an information processing apparatus having a characteristic structure corresponding to the characteristic processing performed by the information processing method. Additionally, it can be implemented as a computer program that causes a computer to execute the characteristic processing included in such an information processing method. Therefore, the following other embodiments can also achieve the same effect as the information processing method described above.

[0038] (10) Another aspect of the present disclosure relates to an information processing device that is an information processing device equipped with a processor, the processor performing the following processing: displaying a bird's-eye view on the display of an information terminal, wherein the bird's-eye view displays a group of shooting location icons representing a series of multiple shooting locations from the shooting start location to the shooting end location, accepting the user's selection of any area on the bird's-eye view, and extracting two or more data corresponding to the similar two or more shooting location icon groups when the area includes two or more shooting location icon groups and the shooting positions of the two or more shooting location icon groups are similar.

[0039] (11) The information processing program involved in another aspect of this disclosure enables a computer to perform the following functions: displaying a bird's-eye view on the display of an information terminal, wherein the bird's-eye view displays a group of shooting location icons representing a series of multiple shooting locations from the shooting start point to the shooting end point, accepting the user's selection of any area on the bird's-eye view, and extracting two or more data corresponding to the similar two or more shooting location icon groups when the area includes two or more shooting location icon groups and the shooting positions of the two or more shooting location icon groups are similar.

[0040] (12) Another aspect of this disclosure involves a non-transitory computer-readable recording medium that records the information processing program described in (11) above.

[0041] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Furthermore, the embodiments described below represent specific examples of the present disclosure. The numerical values, shapes, constituent elements, steps, and order of steps shown in the following embodiments are examples and are not intended to limit the present disclosure. Additionally, constituent elements in the following embodiments that are not described as independent technical solutions representing the highest-level concept are described as arbitrary constituent elements. Furthermore, various elements can be combined in all embodiments.

[0042] (Implementation Method)

[0043] Figure 1 This is a diagram showing the overall structure of the information processing system 1 in this embodiment.

[0044] Information processing system 1 is a system that displays a bird's-eye view on the display 23 of information terminal 20. The bird's-eye view displays a group of icons representing multiple shooting locations from the start point to the end point of the shooting. The system allows the user to select any area on the bird's-eye view. If the area includes two or more groups of shooting location icons, and the shooting locations of the two or more groups of shooting location icons are similar, the system extracts two or more data points corresponding to the two or more similar groups of shooting location icons. Information processing system 1 includes: server 10, information terminal 20, shooting device 30, and communication device 40.

[0045] Server 10 is an example of an information processing device and a computer. Server 10, information terminal 20, and communication device 40 are communicatively connected to each other via a network NT. An example of a network NT is the Internet. Server 10 is, for example, a cloud server consisting of one or more computers. However, this is just one example; server 10 can also be composed of an edge server or installed on information terminal 20. The way server 10 is installed on information terminal 20 is an example of the way information terminal 20 is composed of an information processing device.

[0046] Information terminal 20 is held by a user. The user is, for example, the manager of a given space captured by the camera device 30. The given space is, for example, a construction site. However, this is just one example; the given space could also be a construction site, factory, shop, or office, etc. Information terminal 20 can be a portable computer, such as a smartphone or tablet, or a fixed computer. Information terminal 20 displays images on display 23 under the control of server 10. Figure 1 In the example, one information terminal 20 is shown, but multiple information terminals can be connected to the server 10 via the network NT. The information terminal 20 includes a communication unit 21, a processor 22, a display 23, and an operation unit 24.

[0047] The communication unit 21 is the communication interface that connects the information terminal 20 to the network NT. The communication unit 21 sends instruction signals, representing various instructions received by the operation unit 24 from the user, to the server 10. The communication unit 21 receives display data from the server 10 for displaying various display screens.

[0048] The processor 22, for example, is a central processing unit (CPU) that displays the display screen represented by the display data received by the communication unit 21 on the display screen 23.

[0049] The display 23 is composed of various display devices such as liquid crystal displays or organic EL (Electro-Luminescence) displays, and displays various display screens under the control of the processor 22.

[0050] The operation unit 24 may consist of, for example, a keyboard, a touch panel, and a mouse, and accepts various instructions input by the user.

[0051] The communication device 40, for example, is a portable information terminal such as a smartphone or tablet computer, which connects the imaging device 30 to the network NT. The communication device 40 and the imaging device 30 are communicatively connected to each other via near-field wireless communication paths such as Bluetooth (registered trademark) and wireless LAN (Local Area Network).

[0052] The shooting device 30 is, for example, a wide-angle camera that captures images at a given frame rate. A wide-angle camera, also known as a 360-degree camera, is a camera capable of acquiring images from all 360 degrees. The shooting device 30 is, for example, a portable shooting device held by a person. The person being shot may be, for example, a construction worker or a site supervisor. The shooting device 30 can be used while being held by the subject or attached to the person's body (e.g., head). Alternatively, the shooting device 30 may also be a conventional camera.

[0053] The photographer moves within the construction site while simultaneously filming using the filming device 30. At the starting point of the shot, the photographer aligns the filming direction of the device 30 with a given direction, presses the filming button on the device 30 to begin filming, and presses the filming button again at the ending point to end filming. The given direction is used to align the direction with the design shown on the display 23. Figure 1 The direction is specified. For example, north is given, but it is not limited to this. When the shooting ends, the shooting device 30 sends the series of images captured to the communication device 40.

[0054] The image includes the date and time of capture. The date and time of capture can be obtained, for example, by a clock provided with the capturing device 30. Here, since the capturing device 30 captures images at a given frame rate (e.g., 10 frames per second), the capture location is defined in units of frame periods. However, this results in a large amount of data, so the capture location can also be defined by a given time (e.g., 1 second, 5 seconds, or 10 seconds, etc.).

[0055] The communication device 40 receives multiple images transmitted by the imaging device 30. Additionally, the communication device 40 displays a design drawing of the construction site and accepts the photographer's designation of the start and end points of the imaging session on the design drawing.

[0056] The start and end points of the shooting are determined by the photographer inputting instructions specifying their locations on a site design drawing displayed on the monitor of the communication device 40. The design drawing includes a two-dimensional coordinate axis. Therefore, the start and end points of the shooting are defined by two-dimensional coordinate values. These start and end points are used by the server 10 to determine the location of each shooting location and the shooting direction of each image.

[0057] The communication device 40 sends capture information, including multiple captured images, to the server 10 via the network NT. Capture information is generated each time a capture action is performed. A single capture action refers to a series of actions taken by the operator holding the capture device 30 at the construction site, from the start to the end of the capture process. Multiple images are captured in a single capture action. The capture information includes the multiple captured images and metadata for each image. The metadata includes the capture ID, capture date and time, and design drawing ID. The capture ID is an identifier used to identify the capture action. The capture date and time is the date and time the image was captured. The design drawing ID is an identifier used to identify the design drawing corresponding to a given space.

[0058] Server 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 may be, for example, a central processing unit (CPU). The processor 11 includes an acquisition unit 111, a selection and reception unit 112, a display control unit 113, and a retrieval unit 114. The acquisition unit 111, selection and reception unit 112, display control unit 113, and retrieval unit 114 can be implemented by the processor 11 executing an information processing program, or they may be constructed using dedicated hardware circuitry such as an ASIC. The information processing program may also be recorded on a non-transitory, computer-readable recording medium.

[0059] Communication unit 13 is the communication interface connecting server 10 to network NT. Communication unit 13 receives image capture information sent by communication device 40. Communication unit 13 receives instruction signals from information terminal 20 indicating various instructions received from the user. Communication unit 13 sends display data for displaying various display screens to information terminal 20.

[0060] The memory 12 is composed of a non-volatile, rewritable storage device such as a hard disk drive or a solid-state drive. The memory 12 includes a design drawing information storage unit 121 and a shooting trajectory data storage unit 122.

[0061] Design drawing information storage unit 121 stores design drawing information. Design drawing information is image information representing a design drawing of a given space. The design drawing information is associated with a design drawing ID used to identify the design drawing. A design drawing is an example of a bird's-eye view. A design drawing is a drawing representing the design of a construction site, and can also be a floor plan, blueprint, map, or perspective view of the construction site. A bird's-eye view can also be called an overhead view; it can be a view viewed from above or from a high vantage point. Furthermore, if the construction site has multiple floors, the design drawing information includes design drawings for each floor individually.

[0062] The acquisition unit 111 uses Visual SLAM (Simultaneous Localization and Mapping) technology to determine the location of each of the multiple images based on the shooting start location, shooting end location, and multiple images included in the shooting information received by the communication unit 13. The location of the shooting location is represented by two-dimensional coordinate values ​​on the design drawing. In addition, the acquisition unit 111 also uses Visual SLAM technology to determine the shooting direction of each of the multiple images. The shooting direction of each image is represented, for example, by a three-dimensional polar coordinate vector.

[0063] The shooting trajectory data storage unit 122 stores multiple shooting trajectory data. The shooting trajectory data is data that establishes a correspondence between multiple images captured from a series of shooting locations from the shooting start point to the shooting end point, shooting ID, shooting start date and time, design drawing ID, and registrant ID. The registrant ID is information identifying the registrant who registered the shooting trajectory data to the shooting trajectory data storage unit 122. The registrant is, for example, the photographer. Shooting trajectory data is generated whenever the aforementioned shooting information is received. The captured image includes metadata. The metadata includes: shooting ID, shooting date and time, shooting direction, shooting location, and design drawing ID. The shooting direction is the shooting direction of the shooting device 30 that captured the image. The shooting location is the location information (two-dimensional coordinate values) indicating the shooting location of the image.

[0064] Additionally, the communication unit 13 receives instructions from the information terminal 20 to specify the floor of the construction site. The operation unit 24 of the information terminal 20 accepts the user's selection of the floor of the construction site, and the communication unit 21 sends the instructions to specify the floor of the construction site selected by the user to the server 10.

[0065] The acquisition unit 111 acquires from the design drawing information storage unit 121 the design drawing corresponding to the floor of the construction site included in the instruction for specifying the floor of the construction site received by the communication unit 13. Additionally, the acquisition unit 111 acquires from the shooting trajectory data storage unit 122 shooting trajectory data corresponding to the design drawing ID of the acquired design drawing.

[0066] The display control unit 113 displays the design drawing corresponding to the given space on the display 23 of the information terminal 20. The display control unit 113 displays the design drawing acquired by the acquisition unit 111 on the display 23 of the information terminal 20. The display control unit 113 sends the display data of the design drawing to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data sent by the server 10. The display 23 displays the design drawing received by the communication unit 21.

[0067] The display control unit 113 displays a group of shooting location icons on the design drawing, representing a series of shooting locations from the start point to the end point of the shooting. The shooting location icon group corresponds to multiple images captured at the multiple shooting locations. The display control unit 113 displays the shooting location icon group on the design drawing based on the shooting trajectory data acquired by the acquisition unit 111. When multiple shooting trajectory data correspond to the design drawing, the display control unit 113 displays multiple shooting location icon groups on the design drawing.

[0068] Figure 2 This diagram illustrates an example of a design screen 201 in this embodiment, which maps multiple groups of shooting location icons. Design screen 201 displays: a design drawing 211 of a given floor of a given construction site, a color bar 212 indicating the correspondence between shooting date / time and color, a list display bar 213, and a shooting image display button 214. Multiple groups of shooting location icons 221, 222, 223, 224, 225, and 226 are displayed overlapping in design drawing 211.

[0069] Shooting location icon group 221 is a collection of multiple shooting location icons representing a series of shooting locations from the start point to the end point of the shooting. A single image captured at one shooting location corresponds to one shooting location icon. In this example, the shooting location icons consist of circular images of a given size. The shooting location icon group 221 shown here corresponds to multiple shooting locations belonging to a single shooting action. The route tracing from the shooting location icon representing the start point to the shooting location icon representing the end point represents the trajectory followed by the photographer during that shooting action. Furthermore, other shooting location icon groups 222, 223, 224, 225, and 226 are identical to shooting location icon group 221.

[0070] The imaging device 30 captures images at a given frame rate. In this case, the number of images is represented by frame rate (fps) * shooting time (seconds). Figure 2 In the image, multiple shooting location icons are displayed overlapping each other along the path the photographer moves along. Between the start and end points of the shooting, new shooting location icons are displayed, overlapping each other on top of the previous one.

[0071] The multiple shooting location icon groups 221, 222, 223, 224, 225, and 226 each have different shooting dates and times (shooting start dates and times). The display control unit 113 displays the shooting location icon groups 221, 222, 223, 224, 225, and 226 in colors corresponding to the shooting dates and times. Figure 2 In the image, shooting location icon groups 221, 222, 223, 224, 225, and 226 are displayed in different colors according to each shooting day. For example, shooting location icon groups 221, 222, 223, and 224 represent multiple shooting locations taken on June 23, 2023, while shooting location icon groups 225 and 226 represent multiple shooting locations taken on July 3, 2023. The shooting day of the shooting location icon group can be determined by the color of the icon group and color bar 212.

[0072] Furthermore, when multiple locations' icon groups overlap, the location icon group with the more recent shooting date and time will appear on top of the location icon group with the older shooting date and time. For example, since the shooting date and time of the first location icon group 223 is "2023 / 06 / 23 17:02:58" and the shooting date and time of the second location icon group 225 is "2023 / 07 / 03 17:52:58", the second location icon group 225 will appear on top of the first location icon group 223.

[0073] The list display bar 213 includes: a name 2131 for identifying the group of shooting location icons, a checkbox 2132 for selecting the group of shooting location icons, and shooting location icons 2133. The name 2131 is indicated by the shooting date and time and the name of the registrant. The shooting date and time is, for example, the shooting start date and time. The name 2131 of the list display bar 213 sequentially represents the names of the shooting location icon groups 221, 222, 223, 224, 225, and 226 from top to bottom. The user presses the checkbox 2132 located to the left of the group of shooting location icons that displays the captured image. When the checkbox 2132 is pressed, the display control unit 113 changes the display of the checkbox 2132 to the selected state.

[0074] The image display button 214 is used to display captured images. When the image display button 214 is pressed, the display control unit 113 displays the captured image corresponding to the group of capture location icons selected by the checkbox 2132.

[0075] The selection receiving unit 112 accepts the user's selection of any area on the design drawing.

[0076] Figure 3 This is a diagram showing an example of a design drawing screen 201 on design drawing 211 in this embodiment, in which area 215 is drawn.

[0077] The selection receiving unit 112 accepts the user's selection of any region 215 on the design drawing 211. The shape of the region 215 is, for example, rectangular. The selection receiving unit 112 accepts the user's drag operation with the mouse. The user selects the rectangular region 215 by dragging the mouse. In this case, the display 23 draws the rectangular region 215 on the design drawing 211 with the straight line connecting the starting point and the ending point of the mouse drag forming a diagonal.

[0078] When selecting a group of location icons to display in a captured image, the user selects area 215 that includes that group of location icons. Figure 3 In the middle, area 215, which includes part of the shooting location icon groups 223 and 225, was selected.

[0079] Furthermore, the selection receiving unit 112 can also accept user clicks on the mouse. The user selects a rectangular area 215 by clicking the mouse. In this case, the display 23 can also draw a circular or rectangular area 215 of a given size centered on the point where the mouse is clicked on the design drawing 211.

[0080] The display control unit 113 determines whether there are two or more groups of shooting location icons within the area. Furthermore, if there are two or more groups of shooting location icons within the area, the display control unit 113 determines whether the shooting positions of the two or more groups of shooting location icons are similar. The display control unit 113 determines that the shooting positions of the two or more groups of shooting location icons are similar if at least one of the distance between the starting points of the shooting in each of the two or more groups of shooting location icons and the distance between the ending points of the shooting in each of the two or more groups of shooting location icons is below a threshold.

[0081] That is, in Figure 3 In the above, the two or more shooting location icon groups within area 215 include a first shooting location icon group 223 and a second shooting location icon group 225. If at least one of the distances between the shooting start point of the first shooting location icon group 223 and the shooting start point of the second shooting location icon group 225, and the distances between the shooting end point of the first shooting location icon group 223 and the shooting end point of the second shooting location icon group 225 is below a threshold, the display control unit 113 determines that the shooting position of the first shooting location icon group 223 is similar to the shooting position of the second shooting location icon group 225.

[0082] When the area includes two or more groups of shooting location icons and the shooting locations of the two or more groups of shooting location icons are similar, the extraction unit 114 extracts two or more data corresponding to the two or more similar groups of shooting location icons. Here, the extracted data is the captured image taken at the shooting location.

[0083] That is, in Figure 3 In this process, if at least one of the distances between the starting point of the first shooting location icon group 223 and the starting point of the second shooting location icon group 225, and the distances between the ending point of the first shooting location icon group 223 and the ending point of the second shooting location icon group 225, is below a threshold, the extraction unit 114 extracts first data corresponding to the first shooting location icon group 223 and second data corresponding to the second shooting location icon group 225. Here, the extracted first data and second data are captured images.

[0084] exist Figure 3 In the example, the distance between the starting point of the first shooting location icon group 223 and the starting point of the second shooting location icon group 225 is below a threshold, and the distance between the ending point of the first shooting location icon group 223 and the ending point of the second shooting location icon group 225 is below a threshold.

[0085] Additionally, the display control unit 113 displays the name 2131 used to identify the group of shooting location icons along with the design drawing 211 on the display 23. When the area includes two or more groups of shooting location icons, and the shooting positions of the two or more groups of shooting location icons are similar, the display control unit 113 changes the display mode of the two or more groups of shooting location icons and the display mode of the two or more names corresponding to the two or more groups of shooting location icons. For example, the display control unit 113 changes the color of the two or more groups of shooting location icons and changes the color of the two or more names to the same color as the two or more groups of shooting location icons.

[0086] exist Figure 3 In the example, the display control unit 113 changes the color of the first shooting location icon group 223 and the second shooting location icon group 225 to red, and changes the color of the names of the first shooting location icon group 223 and the second shooting location icon group 225 to the same red color as the first shooting location icon group 223 and the second shooting location icon group 225. In the list display bar 213, the name of the first shooting location icon group 223 is "2023 / 06 / 23 17:02:58 Registrant: A", and the name of the second shooting location icon group 225 is "2023 / 07 / 03 17:52:58 Registrant: A". Figure 3 In the text display, the colors of the names of the first shooting location icon group 223 and the second shooting location icon group 225 have changed by altering the text thickness. The color of the shooting location icons 2133 within the list display bar 213 also changes along with their names.

[0087] Furthermore, the display control unit 113 displays two or more captured images extracted by the extraction unit 114 on the display 23. The display control unit 113 sends the display data of the two or more captured images to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data sent by the server 10. The display 23 displays the two or more captured images received by the communication unit 21.

[0088] exist Figure 3In the example, the operation unit 24 of the information terminal 20 receives user input via checkboxes 2132 corresponding to the first and second image capture location icon groups 223 and 225. The user selects the first and second image capture location icon groups 223 and 225 to display by inputting a selection mark into the checkboxes 2132. Furthermore, when the user presses the image capture display button 214, the communication unit 21 sends information indicating the selected first and second image capture location icon groups 223 and 225 to the server 10. The communication unit 13 receives information indicating the selected first and second image capture location icon groups 223 and 225 from the information terminal 20.

[0089] The extraction unit 114 extracts the captured image corresponding to the selected first shooting location icon group 223 and the captured image corresponding to the second shooting location icon group 225. The display control unit 113 displays the captured images corresponding to the first shooting location icon group 223 and the captured images corresponding to the second shooting location icon group 225 extracted by the extraction unit 114 on the display 23.

[0090] Figure 4 This is a diagram showing an example of the captured image screen 202 displayed on the display 23 when area 215 is selected in this embodiment.

[0091] The captured image screen 202 displays: a captured image 231 taken at a location corresponding to one of the captured location icons in the first captured location icon group 223, and a captured image 232 taken at a location corresponding to one of the captured location icons in the second captured location icon group 225. The extraction unit 114 extracts the captured image 231 from the captured trajectory data storage unit 122, which is the captured image 231 taken at the location corresponding to the captured icon with the oldest captured date and time in region 215 within the first captured location icon group 223, and the captured image 232 from the captured trajectory data storage unit 122, which is the captured image 232 taken at the location corresponding to the captured icon with the oldest captured date and time in region 215 within the second captured location icon group 225. The display control unit 113 displays the captured image 231 and captured image 232 extracted by the extraction unit 114 on the display 23.

[0092] Here, the display control unit 113 displays rectangular captured images 231 and 232, cropped from the omnidirectional image and corresponding to the viewing direction, on the display 23. The viewing direction of captured image 231 is the same as that of captured image 232. The captured image screen 202 includes a date field 233 showing the shooting location icon and the shooting date of captured image 231, and a date field 234 showing the shooting location icon and the shooting date of captured image 232.

[0093] In addition, the extraction unit 114 can also extract from the shooting trajectory data storage unit 122 the shooting image 231 captured at the shooting location corresponding to the latest shooting icon in area 215 within the first shooting location icon group 223, and extract from the shooting trajectory data storage unit 122 the shooting image 232 captured at the shooting location corresponding to the latest shooting icon in area 215 within the second shooting location icon group 225.

[0094] Furthermore, in this embodiment, the data extracted by the extraction unit 114 is a captured image, but this disclosure is not particularly limited to this. The data extracted by the extraction unit 114 may also be sound data collected at the shooting location, or text data input at the shooting location.

[0095] Next, the image display processing of the server 10 in this embodiment will be described.

[0096] Figure 5 This is a first flowchart illustrating an example of image display processing of server 10 in this embodiment. Figure 6 This is a second flowchart illustrating an example of image display processing of server 10 in this embodiment.

[0097] First, in step S1, the selection receiving unit 112 accepts the user's selection of a floor in the construction site. The display control unit 113 displays a floor selection screen on the display 23 for accepting the selection of one floor from multiple floors in the construction site. The floor selection screen displays the name of the construction site and multiple floor names that can be selected. The operation unit 24 of the information terminal 20 accepts the user's selection of the name of the construction site and the floor name of the construction site. The user selects one floor name from multiple floor names. The communication unit 21 sends an instruction to the server 10 to specify the floor of the construction site selected by the user. The communication unit 13 receives the instruction to specify the floor of the construction site from the information terminal 20.

[0098] Next, in step S2, the display control unit 113 displays the design drawing corresponding to the floor selected by the user on the display 23. The acquisition unit 111 acquires the design drawing corresponding to the floor of the construction site included in the instruction for specifying the floor of the construction site received by the communication unit 13 from the design drawing information storage unit 121. The display control unit 113 displays the design drawing acquired by the acquisition unit 111 on the display 23.

[0099] Next, in step S3, the display control unit 113 determines whether there are multiple shooting trajectory data corresponding to the selected design drawing. The acquisition unit 111 acquires from the shooting trajectory data storage unit 122 at least one shooting trajectory data corresponding to the design drawing ID of the acquired design drawing, which was captured on different dates and times.

[0100] Here, if it is determined that multiple shooting trajectory data do not exist (no in step S3), in step S4, the display control unit 113 displays a group of shooting location icons on the design drawing based on the acquired shooting trajectory data. The group of shooting location icons can be selectively displayed. The display control unit 113 sends the display data of the group of shooting location icons to the information terminal 20 via the communication unit 13. This group of shooting location icons represents a series of multiple shooting locations from the start point to the end point of the shooting of multiple images included in the shooting trajectory data. The communication unit 21 of the information terminal 20 receives the display data sent by the server 10. The display 23 displays the group of shooting location icons received by the communication unit 21.

[0101] Next, in step S5, the display control unit 113 determines whether a shooting location icon from the displayed group of shooting location icons has been selected. The operation unit 24 of the information terminal 20 accepts the user's selection of a shooting location icon from the group of shooting location icons. The user selects a shooting location icon from the group of shooting location icons. The communication unit 21 sends an instruction to the server 10 to specify the shooting location icon selected by the user. The communication unit 13 receives the instruction to specify the shooting location icon from the information terminal 20.

[0102] Here, if it is determined that no shooting location icon has been selected (no in step S5), the determination process in step S5 is repeated until a shooting location icon is selected.

[0103] On the other hand, if it is determined that a shooting location icon has been selected (yes in step S5), in step S6, the display control unit 113 displays a captured image corresponding to the selected shooting location icon on the display 23. The shooting location icon represents a shooting location on the design drawing. The display control unit 113 displays a captured image taken at the shooting location corresponding to the shooting location icon. Specifically, the display control unit 113 uses the communication unit 13 to send the display data of the captured image to the information terminal 20. In the information terminal 20, the processor 22 displays the display data received by the communication unit 21 on the display 23. Thus, a captured image is displayed on the display 23.

[0104] Furthermore, if no corresponding shooting trajectory data is established with the selected design drawing, the display control unit 113 may also terminate the image display processing.

[0105] On the other hand, if it is determined that multiple shooting trajectory data exist (yes in step S3), in step S7, the display control unit 113 displays multiple shooting location icon groups on the design drawing based on the acquired multiple shooting trajectory data. The display control unit 113 sends the display data of multiple shooting location icon groups, representing a series of multiple shooting locations from the start point to the end point of shooting for multiple images included in each of the multiple shooting trajectory data, to the information terminal 20 via the communication unit 13. The communication unit 21 of the information terminal 20 receives the display data sent through the server 10. The display 23 displays the multiple shooting location icon groups received by the communication unit 21.

[0106] Next, in step S8, the display control unit 113 displays the names and designs of multiple shooting location icon groups. Figure 1 The display will then show the names of the multiple shooting location icon groups displayed on the design drawing, along with the checkboxes.

[0107] Next, in step S9, the selection receiving unit 112 accepts the user's selection of any area on the design drawing. The operation unit 24 of the information terminal 20 accepts the user's selection of any area on the design drawing. The user selects a rectangular area on the design drawing. The communication unit 21 sends information used to determine the area selected by the user to the server 10. The information used to determine the area is the coordinates of the four vertices of the rectangular area on the design drawing. The communication unit 13 receives the information used to determine the area from the information terminal 20.

[0108] Next, in step S10, the display control unit 113 determines whether there are more than two groups of shooting location icons in the area selected by the selection and acceptance unit 112.

[0109] Here, if it is determined that there are no more than two shooting location icon groups within the area (No in step S10), in step S11, the display control unit 113 determines whether there is one shooting location icon group within the area selected by the selection and acceptance unit 112. Here, if it is determined that there are no shooting location icon groups within the area (No in step S11), the process ends.

[0110] On the other hand, if it is determined that there is a group of shooting location icons within the area (yes in step S11), in step S12, the display control unit 113 changes the display mode of the group of shooting location icons and the display mode of the name corresponding to the group of shooting location icons. For example, the display control unit 113 changes the color of the group of shooting location icons to a different color from the other group of shooting location icons, and changes the color of the name to the same color as the group of shooting location icons.

[0111] On the other hand, if it is determined that there are two or more groups of shooting location icons within the area (yes in step S10), in step S13, the display control unit 113 determines whether the shooting positions of the two or more groups of shooting location icons are similar. At this time, if at least one of the distance between the starting points and the ending points of each of the two or more groups of shooting location icons is below a threshold, the display control unit 113 determines that the shooting positions of the two or more groups of shooting location icons are similar. Conversely, if at least one of the distance between the starting points and the ending points of each of the two or more groups of shooting location icons is not below a threshold, the display control unit 113 determines that the shooting positions of the two or more groups of shooting location icons are not similar.

[0112] Here, if it is determined that the shooting locations of two or more shooting location icon groups are not similar (no in step S13), the process is transferred to step S15.

[0113] On the other hand, if it is determined that the shooting locations of two or more shooting location icon groups are similar (yes in step S13), in step S14, the display control unit 113 changes the display mode of the two or more shooting location icon groups and the display mode of the two or more names corresponding to the two or more shooting location icon groups. For example, the display control unit 113 changes the color of the two or more shooting location icon groups within the area to a different color than other shooting location icon groups outside the area, and changes the color of the two or more names to the same color as the two or more shooting location icon groups.

[0114] Next, in step S15, the receiving unit 112 accepts the selection of at least one group of shooting location icons for displaying the captured image. At this time, the operation unit 24 of the information terminal 20 accepts the selection of at least one group of shooting location icons for displaying the captured image by the user. The user selects at least one group of shooting location icons for displaying the captured image by entering a check mark in the checkbox for the name of the group of shooting location icons for displaying the captured image. Furthermore, when the user presses the captured image display button, the communication unit 21 sends information indicating the at least one group of shooting location icons selected by the user to the server 10. The communication unit 13 receives information indicating the at least one group of shooting location icons selected by the user from the information terminal 20.

[0115] When the area includes two or more shooting location icon groups, and the shooting locations of the two or more shooting location icon groups are similar, the selection receiving unit 112 accepts the selection of two or more shooting location icon groups for displaying the captured image. Alternatively, when the area includes only one shooting location icon group, the selection receiving unit 112 accepts the selection of only one shooting location icon group for displaying the captured image.

[0116] Next, in step S16, the extraction unit 114 extracts at least one captured image from the shooting trajectory data storage unit 122, which corresponds to at least one group of shooting location icons selected by the user. At this time, the extraction unit 114 extracts from the shooting trajectory data storage unit 122 the captured image taken at the shooting location corresponding to the shooting icon in the shooting location icon group that has the oldest shooting date and time in the area.

[0117] When the area includes two or more groups of shooting location icons, and the shooting positions of the two or more groups of shooting location icons are similar, the extraction unit 114 extracts two or more captured images corresponding to the two or more groups of shooting location icons selected by the user from the shooting trajectory data storage unit 122. Alternatively, when the area includes only one group of shooting location icons, the extraction unit 114 extracts one captured image corresponding to the one group of shooting location icons selected by the user from the shooting trajectory data storage unit 122.

[0118] Next, in step S17, the display control unit 113 displays at least one captured image extracted by the extraction unit 114 on the display 23. Specifically, the display control unit 113 uses the communication unit 13 to send display data of at least one captured image to the information terminal 20. In the information terminal 20, the processor 22 displays the display data received by the communication unit 21 on the display 23. Thus, at least one captured image is displayed on the display 23.

[0119] When the area includes two or more groups of shooting location icons, and the shooting positions of the two or more groups of shooting location icons are similar, the display control unit 113 displays two or more captured images extracted by the extraction unit 114 on the display 23. Alternatively, when the area includes only one group of shooting location icons, the display control unit 113 displays only one captured image extracted by the extraction unit 114 on the display 23.

[0120] In this way, by selecting an area on the design drawing that includes a group of icons representing two or more desired shooting locations, users can easily access more than two data points corresponding to those two or more shooting location icon groups, thus improving user convenience. Furthermore, by selecting an area on the design drawing that includes a group of icons representing two or more desired shooting locations and extracting more than two data points corresponding to those two groups of icons, the number of computer processing steps required to access more than two data points can be reduced, thus alleviating the computer's burden.

[0121] Furthermore, in this embodiment, the display control unit 113 determines whether the shooting positions of two or more shooting location icon groups are similar by determining whether at least one of the distance between the shooting start points of each of the two or more shooting location icon groups and the distance between the shooting end points of each of the two or more shooting location icon groups is below a threshold. However, this disclosure is not particularly limited to this.

[0122] In a variation of this embodiment, the display control unit 113 can also determine whether the shooting positions of two or more shooting location icon groups are similar by determining whether each of the two or more shooting location icon groups overlaps within a region. In this case, Figure 6 In step S13, the display control unit 113 may determine that the shooting positions of two or more shooting location icon groups are similar if they overlap within their respective areas. Alternatively, the display control unit 113 may determine that the shooting positions of two or more shooting location icon groups are dissimilar if they do not overlap within their respective areas.

[0123] That is, the extraction unit 114 can also extract more than two data corresponding to the two or more shooting location icon groups when the area includes two or more shooting location icon groups and the two or more shooting location icon groups overlap in the area. In addition, the display control unit 113 can also change the display mode of the two or more shooting location icon groups and the display mode of the two or more names corresponding to the two or more shooting location icon groups when the area includes two or more shooting location icon groups and the two or more shooting location icon groups overlap in the area.

[0124] exist Figure 3 In the example, the extraction unit 114 may also extract the captured image corresponding to the first shooting location icon group 223 and the captured image corresponding to the second shooting location icon group 225 when the first shooting location icon group 223 and the second shooting location icon group 225 overlap within the region 215. Additionally, the display control unit 113 may change the display mode of the first shooting location icon group 223, the display mode of the second shooting location icon group 225, the display mode of the name corresponding to the first shooting location icon group 223, and the display mode of the name corresponding to the second shooting location icon group 225 when the first shooting location icon group 223 and the second shooting location icon group 225 overlap within the region 215.

[0125] Furthermore, in a variation 2 of this embodiment, the display control unit 113 can also determine whether the shooting positions of two or more shooting location icon groups are similar by determining whether the shortest distance among the distances to each shooting location corresponding to the first shooting location icon group within the area and to each shooting location corresponding to the second shooting location icon group within the area is below a threshold. In this case, Figure 6 In step S13, the display control unit 113 may also determine that the shooting positions of two or more shooting location icon groups are similar if the shortest distance among the distances to each shooting location corresponding to the first shooting location icon group in the area and to each shooting location corresponding to the second shooting location icon group in the area is below a threshold. Alternatively, the display control unit 113 may also determine that the shooting positions of two or more shooting location icon groups are dissimilar if the shortest distance among the distances to each shooting location corresponding to the first shooting location icon group in the area and to each shooting location corresponding to the second shooting location icon group in the area is not below a threshold.

[0126] That is, the extraction unit 114 can also extract the first data corresponding to the first shooting location icon group and the second data corresponding to the second shooting location icon group when the area includes the first shooting location icon group and the second shooting location icon group, and the shortest distance among the distances to each shooting location corresponding to the first shooting location icon group in the area and to each shooting location corresponding to the second shooting location icon group in the area is below a threshold. In addition, the display control unit 113 can also change the display mode of the first shooting location icon group, the display mode of the second shooting location icon group, the display mode of the name corresponding to the first shooting location icon group, and the display mode of the name corresponding to the second shooting location icon group when the area includes the first shooting location icon group and the second shooting location icon group, and the shortest distance among the distances to each shooting location corresponding to the first shooting location icon group in the area and to each shooting location corresponding to the second shooting location icon group in the area is below a threshold.

[0127] exist Figure 3 In the example, the extraction unit 114 may also extract the captured image corresponding to the first shooting location icon group 223 and the captured image corresponding to the second shooting location icon group 225 when the shortest distance among the distances between each shooting location corresponding to the first shooting location icon group 223 in the region 215 and each shooting location corresponding to the second shooting location icon group 225 in the region 215 is below a threshold. Additionally, the display control unit 113 may change the display mode of the first shooting location icon group 223, the display mode of the second shooting location icon group 225, the display mode of the name corresponding to the first shooting location icon group 223, and the display mode of the name corresponding to the second shooting location icon group 225 when the shortest distance among the distances between the first shooting location icon group 223 and the second shooting location icon group 225 in the region 215 is below a threshold.

[0128] In addition, the functions of the apparatus involved in the embodiments of this disclosure can also be implemented by executing programs through a processor such as a CPU.

[0129] Furthermore, all the figures used above are illustrative for the purpose of specifically illustrating this disclosure, and this disclosure is not limited to the illustrative figures.

[0130] Furthermore, the order in which the steps shown in the flowchart above are executed is illustrative for the purpose of specifically illustrating this disclosure, and any other order may be used to achieve the same effect. Additionally, some of the above steps may be executed simultaneously (in parallel) with other steps.

[0131] Industrial availability

[0132] The technology disclosed herein can easily access more than two data points corresponding to two or more desired shooting location icon groups, thereby improving user convenience and thus being useful as a technology for displaying images.

Claims

1. An information processing method, executed by a computer, The information processing method includes: An aerial view is displayed on the monitor of the information terminal; The bird's-eye view shows a group of shooting location icons representing a series of shooting locations from the start point to the end point of shooting; The bird's-eye view allows the user to select any area. and If the area includes two or more groups of shooting location icons and the shooting locations of the two or more groups of shooting location icons are similar, extract two or more data corresponding to the similar groups of shooting location icons.

2. The information processing method according to claim 1, wherein, The extraction of the two or more data points includes: extracting the two or more data points corresponding to the two or more shooting location icon groups when at least one of the distance between the shooting start points of the two or more shooting location icon groups and the distance between the shooting end points of the two or more shooting location icon groups is below a threshold.

3. The information processing method according to claim 1, wherein, The extraction of the two or more data points includes: when the two or more shooting location icon groups overlap in the area, extracting the two or more data points corresponding to the two or more shooting location icon groups.

4. The information processing method according to claim 1, wherein, The two or more shooting location icon groups include: the first shooting location icon group and the second shooting location icon group. The extraction of the two or more data includes: when the shortest distance among the distances to each of the multiple shooting locations corresponding to the first shooting location icon group in the region and the distances to each of the multiple shooting locations corresponding to the second shooting location icon group in the region is below a threshold, extracting the first data corresponding to the first shooting location icon group and the second data corresponding to the second shooting location icon group.

5. The information processing method according to any one of claims 1 to 4, wherein, The information processing method further includes displaying the name of the group of icons used to identify the shooting location along with the bird's-eye view on the display.

6. The information processing method according to claim 5, wherein, The information processing method further includes: when the area includes two or more groups of shooting location icons and the shooting locations of the two or more groups of shooting location icons are similar, changing the display mode of the two or more groups of shooting location icons and the display mode of two or more names corresponding to the two or more groups of shooting location icons.

7. The information processing method according to claim 6, wherein, The change in display method includes: changing the color of the two or more shooting location icon groups, and changing the color of the two or more names to the same color as the two or more shooting location icon groups.

8. The information processing method according to any one of claims 1 to 4, wherein, The data refers to images taken at the shooting location.

9. The information processing method according to any one of claims 1 to 4, wherein, The display of the shooting location icon group includes: displaying the shooting location icon group in a color corresponding to the shooting date and time.

10. An information processing device, comprising a processor, The processor performs the following processing: An aerial view is displayed on the information terminal's monitor. The bird's-eye view shows a group of icons representing multiple shooting locations, from the start point of the shooting to the end point. The bird's-eye view allows users to select any area. If the area includes two or more groups of shooting location icons and the shooting locations of the two or more groups of shooting location icons are similar, extract two or more data corresponding to the similar groups of shooting location icons.

11. An information processing program that enables a computer to perform the following functions: An aerial view is displayed on the information terminal's monitor. The bird's-eye view shows a group of icons representing multiple shooting locations, from the start point of the shooting to the end point. The bird's-eye view allows users to select any area. If the area includes two or more groups of shooting location icons and the shooting locations of the two or more groups of shooting location icons are similar, extract two or more data corresponding to the similar groups of shooting location icons.

Images