Information processing method, information processing device, and information processing program

Abstract

The information processing apparatus detects a selection of an image by a user, determines whether or not an annotation region is set in the image, and if it is determined that the annotation region is set in the image, sets a first display range in the image such that the annotation region is located at the center in the image, and displays the image in the first display range on a display.

Description

Technical Field

[0001] The present disclosure relates to a technology for displaying images. Background Art

[0002] Patent Document 1 discloses that an image (eg, outline) representing a designated area designated by a worker is displayed on a bird's-eye view image of a work site, and a chat associated with the designated area input by the worker is displayed in a display field provided next to the bird's-eye view image.

[0003] However, the prior art disclosed in Patent Document 1 does not disclose that when a user selects a certain location in a bird's-eye view image, an image of the location is displayed, and thus, the user is not disclosed to select an image with an annotation area set. Therefore, Patent Document 1 has a problem that when an image with an annotation area set is selected, the user cannot quickly check the annotation area.

[0004] Prior Art Literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-86224 Summary of the invention

[0007] The present disclosure has been made to solve such a problem, and an object of the present disclosure is to provide a technology that enables a user to quickly check an annotation area when an image in which an annotation area is set is selected.

[0008] An information processing method in one aspect of the present invention is an information processing method in a computer, which detects a user's selection of an image; determines whether an annotation area is set in the image; if it is determined that the annotation area is set in the image, sets a first display range in the image so that the annotation area is located in the center of the image; and displays the image in the first display range on a display.

[0009] According to this configuration, when an image in which an annotation area is set is selected, the user can quickly check the annotation area. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a block diagram showing an example of the configuration of the information processing system in Embodiment 1 of the present disclosure.

[0011] Figure 2 The figure shows an example of a display screen displayed on the display of the terminal device.

[0012] Figure 3 It is a diagram showing the third display range.

[0013] Figure 4 It is a diagram showing the first display range.

[0014] Figure 5 It is shown Figure 1 A flowchart of an example of processing of the information processing device is shown.

[0015] Figure 6 It is shown Figure 5 The flowchart shown shows the details of the processing.

[0016] Figure 7 It is a diagram showing the second display range.

[0017] Figure 8 This is a flowchart showing the details of the display processing in the second embodiment.

[0018] Fig. 9 It is a diagram showing a first display range in a modification example of the present disclosure. DETAILED DESCRIPTION

[0019] (Process of realizing one mode of the present disclosure)

[0020] Issues at the construction site include communication issues such as specific instructions not being conveyed to workers and the time it takes to explain the instructions, and confirmation issues such as the need for manpower to patrol the entire construction site and the time it takes to move to the construction site.

[0021] In order to solve such a problem, it is also considered that multiple cameras are installed at the construction site, and the on-site supervisor at a remote location refers to the images obtained from the multiple cameras to issue instructions to the workers. However, at the construction site, as the construction progresses, operations such as removing the installed sensors and installing the removed sensors in other places will occur. Such operations take time, so it is not practical to install sensors at the construction site. Therefore, the inventors of the present invention have studied a technology that can confirm the conditions of the construction site in detail from a remote location without installing sensors.

[0022] It was then realized that if an operation was input to select a certain location in the design drawing of the construction site displayed on the display, an image of the construction site taken in advance at that location and annotations entered by the worker for the image would be displayed. As long as there was a user interface such as the one described above, the conditions of the construction site could be confirmed in detail from a remote location.

[0023] In addition, it is also assumed that the operator inputs such annotations in correspondence with the annotation area set on the image. In the case where an omnidirectional image is used as the image, it is difficult to recognize if the entire image area is displayed on the display at once, so generally a default display range pre-set from the entire image area is displayed on the display. However, in the case where the annotation area is not included in the default display range, the operator needs to scroll the image displayed on the display to find the annotation area, which takes time. Furthermore, since the process of scrolling the image in accordance with the scrolling operation occurs, the processing burden of the computer increases.

[0024] Therefore, the inventors have gained the following knowledge: when an operator inputs an instruction to select an image, as long as the display range of the image is determined to include the annotation area and the image of the display range is displayed on the display, the operator can quickly confirm the annotation area without performing the above-mentioned scrolling operation.

[0025] The present disclosure has been completed based on such knowledge.

[0026] (1) An information processing method in one aspect of the present disclosure is an information processing method in a computer, which detects a user's selection of an image; determines whether an annotation area is set in the image; if it is determined that the annotation area is set in the image, sets a first display range in the image so that the annotation area is located in the center of the image; and displays the image in the first display range on a display.

[0027] According to this structure, when the user selects an image in which an annotation area is set, the first display range is set in the image so that the annotation area is located at the center, and the image of the first display range is initially displayed on the display. Therefore, even if the user does not perform a scroll operation for finding the annotation area D1, the user can quickly confirm the annotation area set on the image. In addition, since there is no need to input a scroll operation, the processing load required for scrolling processing such as scrolling operation and scroll display can be reduced.

[0028] (2) In the information processing method described in (1) above, it may also be that, further, it is determined whether a plurality of annotation areas are set in the image. When it is determined that a plurality of annotation areas are set in the image, in the setting, a second display range is set in the image so as to include the plurality of annotation areas, and in the display, the image of the second display range is displayed on the display.

[0029] According to this configuration, when a plurality of annotation regions are set in an image, the user can check the plurality of annotation regions at the same time.

[0030] (3) In the information processing method described in (2) above, the image may be an omnidirectional image, and the image in the second display range may be an expanded image obtained by expanding the entire area of ​​the omnidirectional image.

[0031] According to this configuration, the expanded image in which the entire omnidirectional image is expanded is initially displayed on the display as the image of the second display range, so that the user can understand the relative positions of the plurality of annotation areas with respect to the entire image without performing a scroll operation.

[0032] (4) In the information processing method described in (2) or (3) above, when an operation of selecting any annotation area among the multiple annotation areas in the image of the second display range displayed on the display is detected, annotation information corresponding to the selected annotation area is displayed on the display.

[0033] According to this configuration, by inputting an operation of selecting a comment area whose comment information is to be confirmed from among a plurality of comment areas, the comment information can be confirmed.

[0034] (5) In the information processing method described in any one of the above (1) to (4), it may be that, when it is determined that the annotation area is not set in the image, in the setting, a predetermined third display range is set in the image, and in the display, the image of the third display range is displayed on the display.

[0035] According to this configuration, for an image in which no annotation area is set, an image in a third display range determined in advance can be displayed on the display.

[0036] (6) In the information processing method described in any one of (1) to (5) above, the first display area may include the entire annotation area and a predetermined area around the annotation area.

[0037] According to this configuration, in addition to the image of the entire annotation region, the image around the annotation region is simultaneously displayed, so that the user can understand the relative position of the annotation region with respect to the entire image.

[0038] (7) In the information processing method described in any one of (1) to (6) above, the setting may include setting a magnification ratio for displaying the image in the first display range in a given size of a display area included in the display.

[0039] According to this configuration, the image in the first display range can be displayed in a manner matching a given size.

[0040] (8) In the information processing method described in any one of (1) to (7) above, the image may be an image obtained by photographing the scene.

[0041] According to this configuration, the user can check an image captured at a site such as a construction site.

[0042] (9) In another embodiment of the present invention, an information processing device is an information processing device having a processor, wherein the processor performs the following processing: detecting a user's selection of an image; determining whether an annotation area is set in the image; if it is determined that the annotation area is set in the image, setting a first display range in the image so that the annotation area is located in the center of the image; and displaying the image in the first display range on a display.

[0043] According to this configuration, it is possible to provide an information processing device that allows a user to quickly check an annotation range set on an image.

[0044] (10) An information processing program according to another aspect of the present disclosure causes a computer to execute the information processing method described in any one of (1) to (8) above.

[0045] According to this configuration, it is possible to provide an information processing program that allows a user to quickly check an annotation range set on an image.

[0046] The present disclosure can also be realized as an information processing system operated by such an information processing program. In addition, it is needless to say that such a computer program can be circulated via a computer-readable non-transitory recording medium such as a CD-ROM or a communication network such as the Internet.

[0047] In addition, the embodiments described below all illustrate a specific example of the present disclosure. The numerical values, shapes, constituent elements, steps, the order of steps, etc. shown in the following embodiments are examples, and their purpose is not to limit the present disclosure. In addition, for the constituent elements in the following embodiments that are not recorded in the independent claims representing the highest concept, they are described as arbitrary constituent elements. In addition, the various contents can be combined in all embodiments.

[0048] (Implementation method 1)

[0049] Figure 11 is a block diagram showing an example of the structure of the information processing system 1 in the embodiment 1 of the present disclosure. The information processing system 1 includes an information processing device 10, a shooting device 20, and a terminal device 30. The information processing device 10, the shooting device 20, and the terminal device 30 are communicatively connected via a network. An example of a network is the Internet. The information processing device 10 is, for example, a cloud server composed of one or more computers. However, this is an example, and the information processing device 10 may also be composed of an edge server or may be installed in the terminal device 30.

[0050] The photographing device 20 is, for example, composed of an omnidirectional camera, and photographs images at a given frame rate. The photographing device 20 is, for example, a portable photographing device held by a user. The user is, for example, a worker or a site supervisor at a construction site. The user moves within the construction site while photographing the construction site with the photographing device 20. The photographing device 20 sends image information representing the photographed image to the information processing device 10 via a network. The position of the photographing location representing the photographing location and the photographing date and time are associated with the image information. The position of the photographing location is obtained, for example, by a position sensor such as a magnetic sensor or a GPS sensor provided by the photographing device 20, and is represented by latitude and longitude. The photographing date and time is obtained, for example, by a clock provided by the photographing device 20. Thus, the information processing device 10 can obtain image information at multiple photographing locations at the construction site. Here, the photographing device 20 photographs images at a given frame rate, and thus the photographing location is specified in units of frame periods. However, this is an example, and the photographing location can also be specified at given intervals (for example, 1 second, 1 minute, etc.). The imaging device 20 includes an image sensor, an operating device, a communication circuit, a signal processing circuit, etc. The imaging device 20 may be constituted by a portable computer such as a smartphone or a tablet computer.

[0051] The terminal device 30 is held by the user. The terminal device 30 may be a portable computer such as a smartphone or a tablet computer, or may be a stationary computer. The terminal device 30 displays image information on a display according to the control of the information processing device 10. Figure 1 In the example of FIG. 1 , one terminal device 30 is shown, but a plurality of terminal devices may be connected to the information processing device 10 via a network. The terminal device 30 includes a central processing unit (CPU), a memory, an operating device such as a display, a touch screen, and a keyboard, and a communication circuit.

[0052] The information processing device 10 includes a processor 11, a memory 12, and a communication unit 13. The processor 11 is composed of, for example, a central processing unit (CPU). The processor 11 includes an acquisition unit 111, a detection unit 112, a determination unit 113, a setting unit 114, and an output unit 115. The acquisition unit 111 to the output unit 115 can be realized by the processor 11 executing an information processing program, or can be composed of a dedicated hardware circuit such as an ASIC.

[0053] The acquisition unit 111 acquires an instruction for selecting a design drawing from the terminal device 30 by the user, reads the design drawing information indicated by the instruction from the memory 12, and displays the read design drawing information on the display of the terminal device 30. The design drawing information is information of a design drawing representing a construction site. The design drawing information is an example of a bird's-eye view. The acquisition unit 111 displays a plurality of shooting location icons representing shooting locations in the design drawing, and a selection icon for selecting an arbitrary position of the design drawing on the terminal device 30. Their display is achieved by the acquisition unit 111 using the communication unit 13 to send the display instruction to the terminal device 30. The selection icon is configured to be movable on the design drawing. The acquisition unit 111 displays the shooting location icon at a position on the design drawing corresponding to the shooting location. In the design drawing, the latitude and longitude are pre-associated with the position that becomes a key point. The position that becomes a key point is, for example, the position of the four corners of the design drawing.

[0054] The detection unit 112 detects the user's selection of an image. Specifically, the selection of an image is detected as follows. The detection unit 112 obtains the current position of the selection icon moved on the design drawing by the user's operation from the terminal device 30. The current position is the coordinate on the design drawing. The detection unit 112 detects the position where the selection icon has been moved. The selection icon is configured to be movable by dragging and dropping. Hereinafter, the position where the movement has been completed is recorded as the release position. The detection unit 112 only needs to detect the current position of the selection icon as the release position when the operation information indicating the release operation is obtained from the terminal device 30 via the communication unit 13. The detection unit 112 determines one shooting location icon based on the detected release position and the position of each shooting location icon. The image information representing the image captured by the shooting device 20 is previously associated with each shooting location icon. Therefore, the user selects an image by performing an operation of moving the selection icon on the design drawing to select one shooting location icon.

[0055] The plurality of shooting location icons include a first shooting location icon that is associated with the image information and the annotation information. The annotation information indicates an annotation input by a user who views the image information using the terminal device 30. The user uses the terminal device 30 to input an annotation area to the image represented by the image information. An annotation is given to the annotation area. For example, the annotation is a message indicating the content that should be paid attention to during operation regarding the object included in the annotation area. The annotation icon indicating the presence of the annotation is displayed in association with the first shooting location icon.

[0056] The details of determining one shooting location icon are as follows. When the release position is outside the given area relative to the positions of the plurality of shooting location icons, the detection unit 112 determines the first shooting location icon closest to the release position as one shooting location icon. When the release position is within the given area relative to the position of a certain shooting location icon, the detection unit 112 determines the shooting location icon as one shooting location icon.

[0057] The determination unit 113 determines whether an annotation region is set in the image detected by the detection unit 112 (hereinafter referred to as the detection image). The determination unit 113 may determine that the annotation region is set in the detection image when the annotation region ID is associated with the image information representing the detection image. The determination unit 113 may determine that the annotation region is not set in the detection image when the annotation region ID is not associated with the image information representing the detection image.

[0058] When the determination unit 113 determines that the annotation area is set in the detection image, the setting unit 114 sets the first display range in the detection image so that the annotation area is located in the center. The center is, for example, the center of the first display screen. However, this is an example, and the center may also be a certain area including the center of the first display screen. The first display range has, for example, a display area ( Figure 2 The first display range is set to a size of about 1.2 to 1.5 times the size of the circumscribed rectangle of the annotation area. Thus, the first display range includes the entire annotation area and the surrounding image of the annotation area.

[0059] The setting unit 114 sets a magnification for displaying the detection image of the first display range in a given size. The given size is the display area ( Figure 2 Thus, the detection image of the first display range is displayed in the entire area of ​​the display region.

[0060] When the determination unit 113 determines that the annotation region is not set in the detective image, the setting unit 114 sets a third display range determined in advance in the detective image.

[0061] The output unit 115 displays the detection image in the first display range on the display of the terminal device 30 by outputting the display instruction of the detection image in the first display range to the terminal device 30 via the communication unit 13. The display instruction of the detection image with the annotation area set includes the image information of the detection image, the position (coordinates) of the first display range, the magnification of the first display range, and the annotation information corresponding to the annotation area set in the detection image. The display instruction of the detection image without the annotation area set includes the image information of the detection image, the position (coordinates) of the third display range, and the magnification of the third display range, but does not include the annotation information.

[0062] The memory 12 is composed of a non-transient rewritable storage device such as a hard disk drive and a solid state drive. The memory 12 stores the design information, the shooting information, the annotation information, the image information, and the annotation area information. The design information is the image information representing the design. A design ID for identifying the design is associated with the design information. In the design, the latitude and longitude of the actual construction site are set at key points as described above.

[0063] The shooting information indicates information related to one shooting action using the shooting device 20. The shooting information is generated each time one shooting action is performed. One shooting action refers to a series of actions from the start of shooting to the end of shooting by the worker holding the shooting device 20 at the construction site. Multiple images are captured by one shooting action. The shooting information includes a design drawing ID, a shooting ID, a shooting date and time, a representative value of the shooting date and time, the position of the shooting location, and the position of the shooting location icon. The design drawing ID is an identifier of the design drawing of the construction site being photographed. The shooting ID is an identifier for identifying each shot included in one shooting action. The shooting date and time is the shooting date and time of the shooting indicated by the shooting ID. The representative value of the shooting date and time is the shooting date and time when the shooting started. The shooting location indicates the location (latitude and longitude) where the shooting indicated by the shooting ID is performed. The position of the shooting location icon indicates the display position (coordinates) of the shooting location icon corresponding to the shooting ID on the design drawing. The position of the shooting location icon is calculated by mapping the shooting location in the design drawing based on the shooting location (latitude and longitude) at the key point of the design drawing indicated by the design drawing information and the shooting location (latitude and longitude) corresponding to the shooting ID.

[0064] The annotation information is information indicating an annotation. One annotation information corresponds to one annotation. The annotation information is associated with a photographing ID and an annotation area ID. The annotation area ID is an identifier set in the annotation area of ​​the image corresponding to the photographing ID.

[0065] The image information indicates one image captured by each shot included in one shooting action. The shooting ID and the annotation area ID are associated with the image information. When multiple annotation areas are set in the image information corresponding to the shooting ID, the image information includes multiple annotation area IDs.

[0066] The annotation area information stores the positions (coordinates) of key points of the annotation area set in the image information corresponding to the photographing ID. The key points are vertices on the contour of the annotation area. The annotation area information is associated with the photographing ID and the annotation area ID.

[0067] In this way, since the shooting ID is associated with the shooting information and the image information, the image information corresponding to the shooting location icon can be determined using the shooting ID as a key. Since the annotation area ID is associated with the annotation information and the annotation area information, the annotation information corresponding to the annotation area can be determined using the annotation area ID as a key. Since the shooting ID is associated with the annotation area information and the image information, the image information corresponding to the annotation area can be determined using the shooting ID as a key. The shooting location icon corresponding to the shooting ID to which the annotation information is associated becomes the first shooting location icon.

[0068] The communication unit 13 is a communication circuit that connects the information processing device 10 to a network.

[0069] Figure 2 1 is a diagram showing an example of a display screen G1 displayed on the display of the terminal device 30. The display screen G1 is a basic screen of an application provided by the information processing device 10. The display screen G1 includes an image information display column R1, an annotation information display column R2, a design drawing display column R3, and an information display column R4.

[0070] The image information display area R1 displays image information associated with one shooting location icon determined by the detector 112 .

[0071] The annotation information display column R2 displays annotation information corresponding to the determined shooting location icon. Here, annotations C1 input by multiple users for the image information displayed in the image information display column R1 are displayed at a glance. In the annotation information display column R2, annotations C1 input by users other than the user are displayed on the left, and annotations C1 input by the user are displayed on the right.

[0072] In the default display screen G1 immediately after the application is activated, the shooting location icon has not been selected by the user. Therefore, in the default display screen G1, the image information display field R1 and the comment information display field R2 are empty.

[0073] The design drawing display area R3 displays the design drawing of the construction site. In the design drawing displayed in the design drawing display area R3, a selection icon 201, a shooting location icon 202, and a track 203 are superimposed and displayed.

[0074] The selection icon 201 is configured to be movable by a drag-and-drop operation. In this example, the selection icon 201 is configured by an image of a simulated person.

[0075] The shooting location icon 202 is an icon indicating the shooting location, and is associated with the image information. In this example, the shooting location icon 202 is composed of a circular image. The trajectory 203 indicates the trajectory of the user who has shot the image information. In this example, the trajectory 203 is composed of a line connecting adjacent shooting location icons 202. The shooting location icon 202 located at the front end of the trajectory 203 and the shooting location icon 202 located at the rear end of the trajectory 203 are displayed in a larger size than the other shooting location icons. The shooting location icon 202 located at the front end (e.g., the right end) of the trajectory 203 indicates the shooting start position, and the shooting location icon 202 located at the rear end (e.g., the left end) of the trajectory 203 indicates the shooting end position.

[0076] The information display section R4 displays messages input by the user as various messages related to the construction site.

[0077] For example, when the selection icon 201 is released in a given area of ​​any shooting location icon 202, the shooting location icon 202 is determined as one shooting location icon, and the image corresponding to the one shooting location icon is detected as a detection image. The detection image is displayed in the image information display column R1. In this case, if the one shooting location icon is the first shooting location icon, the annotation information corresponding to the first shooting location icon is displayed in the annotation information display column R2.

[0078] For example, it is assumed that the selection icon 201 is not released in a given area at any of the shooting location icons 202. In this case, the first shooting location icon with the shortest distance to the release position is determined as one shooting location icon, and the image corresponding to the one shooting location icon is detected as a detection image. The detection image is displayed in the image information display column R1, and the annotation information corresponding to the detection image is displayed in the annotation information display column R2.

[0079] Figure 31 is a diagram showing the third display range R30. No annotation area is set in the detection image of the third display range R30, and thus no annotation area is displayed in the detection image of the third display range R30. The detection image of the third display range R30 is a rectangular image of a given range centered on the front direction of the imaging device 20. Therefore, the detection image of the third display range R30 depends on the direction of the imaging device 20 when shooting. Therefore, when the detection image with the annotation area D1 set is displayed in the third display range R30, the annotation area D1 may not be displayed initially. In addition, if an operation instruction to scroll the detection image of the third display range R30 is obtained from the terminal device 30, the output unit 115 sends a display instruction to scroll the detection image displayed in the third display range R30 according to the scrolling operation amount and the operation direction to the terminal device 30. As a result, the user can display the desired area of ​​the detection image in the third display range R30.

[0080] Figure 4 1 is a diagram showing the first display range R10. Since the annotation area D1 is set in the detection image, the annotation area D1 is displayed in the detection image of the first display range R10. In this example, the annotation area D1 is displayed using a frame of a given color (for example, red). An annotation icon 301 is displayed in correspondence with the annotation area D1. The annotation icon 301 indicates that an annotation is input in the annotation area. Here, the annotation icon 301 is marked with a comment box. The content of the annotation is not displayed in the annotation icon 301. In addition, if the output unit 115 obtains an operation instruction to scroll the detection image of the first display range R10 from the terminal device 30, a display instruction to scroll the detection image displayed in the first display range R10 according to the scrolling operation amount and operation direction is sent to the terminal device 30. As a result, the user can display the desired area of ​​the detection image in the first display range R10.

[0081] The entire area of ​​the annotation area D1 is displayed in the detection image of the first display range R10. In addition to the detection image of the annotation area D1, the detection image of the first display range R10 also includes the surrounding image of the annotation area D1. Thus, the user can understand the relative position of the annotation area D1 with respect to the entire area of ​​the detection image. The center O of the detection image of the first display range R10 is set as the center of the annotation area D1. Thus, the annotation area D1 is displayed in the center of the detection image of the first display range R10. In this example, the entire area of ​​the annotation area D1 is included in the first display range R10, but a part of the annotation area D1 may also be included.

[0082] Figure 5 It is shown Figure 1Flowchart of an example of processing of the information processing device 10 shown in FIG. 1 . The acquisition unit 111 acquires an instruction from the user to select a design drawing (step S1). In this case, a menu screen for selecting a design drawing is displayed on the display of the terminal device 30, and an instruction to select one design drawing from the menu screen is input. The input instruction is sent to the information processing device 10 via the network and received by the communication unit 13. The acquisition unit 111 acquires the instruction via the communication unit 13. The design drawing ID is included in the instruction, so the acquisition unit 111 can acquire the design drawing information representing the instructed design drawing from the design drawing information stored in the memory 12.

[0083] Next, the acquisition unit 111 transmits the display instruction of the display screen G1 to the terminal device 30 via the communication unit 13, thereby displaying the display screen G1 on the display of the terminal device 30 (step S2). The display instruction of the display screen G1 displayed by default includes the design drawing information indicating the design drawing selected in step S1 and the shooting information corresponding to the latest shooting date and time. Therefore, in the display screen G1 displayed by default, as shown in FIG. Figure 2 As shown, the design drawing includes a selection icon 201 and a shooting location icon 202 and a track 203 corresponding to the latest shooting date and time. At this point in time, no shooting location icon has been determined, so the image information display column R1 and the annotation information display column R2 are empty. In addition, when an instruction to select a shooting date and time is input in step S3 described later, the acquisition unit 111 displays a display screen G1 of the design drawing including the shooting location icon 202 and the track 203 corresponding to the selected shooting date and time in an overlapping manner on the display.

[0084] Next, the acquisition unit 111 determines whether an instruction from the user to select the shooting date and time is acquired (step S3). In this case, a menu screen for selecting the shooting date and time is displayed on the display of the terminal device 30. The user inputs an instruction to select one shooting date and time from the menu screen. The shooting date and time displayed on the menu screen is a representative value of the shooting date and time included in the shooting information stored in the memory 12. If the shooting date and time is selected, one shooting action corresponding to the shooting date and time is selected. The input instruction is sent to the information processing device 10 via the network and received by the communication unit 13. The acquisition unit 111 acquires the instruction via the communication unit 13. The representative value of the shooting date and time is included in the instruction, so the acquisition unit 111 can determine one shooting information stored in the memory 12. When the instruction to select the shooting date and time is input (yes in step S3), the process proceeds to step S4. When the instruction to select the shooting date and time is not input (no in step S3), the process returns to step S2.

[0085] Next, the display process is performed (step S4). The details of the display process will be shown in Figure 6 The display process is a process of dragging and dropping the selection icon 201 on the design drawing to determine one shooting location icon, and displaying an image associated with the determined one shooting location icon in the image information display area R1.

[0086] Next, the acquisition unit 111 determines whether an instruction to input an annotation has been acquired (step S5). The instruction to input an annotation is an instruction input when the user intends to input an annotation for the image displayed in the image information display field R1. For example, the instruction is input by selecting an annotation input instruction button (not shown) displayed on the display screen G1. The input instruction is sent to the information processing device 10 via the network and received by the communication unit 13. The acquisition unit 111 acquires the instruction via the communication unit 13.

[0087] Next, when an instruction to input annotations is obtained (Yes in step S5), the acquisition unit 111 acquires annotation area information (step S6). The annotation area information is input by moving and deforming the frame (not shown) in the image information display bar R1 as described above. The input annotation area information is sent to the information processing device 10 via the network and received by the communication unit 13. The acquisition unit 111 acquires the annotation area information via the communication unit 13. The acquisition unit 111 assigns an annotation area ID to the acquired annotation area information and stores it in the memory 12 in correspondence with the shooting ID. As a result, Figure 3 As shown, the comment area D1 is set.

[0088] Next, when the instruction to input a comment is not acquired (NO in step S5 ), the process proceeds to step S8 .

[0089] Next, the acquisition unit 111 acquires the annotation information (step S7). Figure 2 As shown, the annotation information is input by inputting the annotation C1 into the annotation information display field R2 and pressing the send button (not shown). The input annotation information is sent to the information processing device 10 via the network and received by the communication unit 13. The acquisition unit 111 acquires the annotation information via the communication unit 13. The shooting ID, the annotation area ID and the annotation information acquired by the acquisition unit 111 are associated and stored in the memory 12.

[0090] Next, the acquisition unit 111 determines whether an end instruction is acquired (step S8). The end instruction is an instruction to close the display screen G1 displayed in step S3. The instruction is input by pressing an end button (not shown) displayed on the display screen G1. When the end instruction is acquired (yes in step S8), the process ends. When the end instruction is not acquired (no in step S8), the process returns to step S3. In this case, the display of the display screen G1 is maintained. The end instruction is sent to the information processing device 10 via the network and received by the communication unit 13. The acquisition unit 111 acquires the end instruction via the communication unit 13.

[0091] Figure 6 It is shown Figure 5 The detection unit 112 obtains an instruction to select an image (step S21). Figure 2 As shown in FIG. 1 , the instruction is given by dragging and dropping the selection icon 201 on the design drawing. As described above, the detection unit 112 determines one shooting location icon selected by the user based on the release position and the position of the shooting location icon 202, and detects the image corresponding to the one shooting location icon as the image selected by the user. This image is the detected image.

[0092] Next, the determination unit 113 determines whether the annotation region D1 is set in the detection image (step S22 ). This determination is performed based on whether the annotation region ID is associated with the image information representing the detection image.

[0093] When the annotation area D1 is set in the detection image (Yes in step S22), the setting unit 114 sets the first display range R10 in the detection image so that the annotation area D1 is located at the center (step S23). Figure 4 As shown, the area including the annotation area D1 and the surrounding image is set as the first display range R10.

[0094] Next, the output unit 115 displays the detection image of the first display range R10 on the display of the terminal device 30 by sending a display instruction of the detection image of the first display range R10 to the terminal device 30 via the communication unit 13 (step S24). As a result, the detection image of the first display range R10 is displayed in the image information display field R1. When step S24 is completed, the process proceeds to Figure 5 Step S5.

[0095] If the annotation area is not set in the detection image (No in step S22), the setting unit 114 sets the third display range R30 in the detection image (step S25). Figure 3As shown in FIG. 1 , a region of a predetermined range centered on the front direction of the imaging device 20 at the time of imaging is set as a third display range R30 .

[0096] Next, the output unit 115 displays the detection image of the third display range R30 on the display of the terminal device 30 by sending a display instruction of the detection image of the third display range R30 to the terminal device 30 via the communication unit 13 (step S26). As a result, the detection image of the third display range R30 is displayed in the image information display column R1. When step S26 is completed, the process proceeds to Figure 5 Step S5.

[0097] As described above, according to Embodiment 1, when the user selects an image in which the annotation area D1 is set, the first display range R10 is set in the selected image so that the annotation area D1 is located at the center, and the detection image of the first display range R10 is initially displayed on the display of the terminal device 30. Therefore, the user can quickly confirm the annotation area D1 set on the image without performing a scroll operation for finding the annotation area D1.

[0098] (Implementation method 2)

[0099] In the case where a plurality of annotation areas are set in the detection image, the detection image of the second display range representing the entire area of ​​the detection image is initially displayed. Figure 1 In addition, the processing of the information processing device 10 is the same as Figure 5 same.

[0100] Figure 7 : is a diagram showing the second display range R20. The detection image of the second display range R20 is an expanded image obtained by expanding the entire area of ​​the detection image of the expansion source. The detection image of the expansion source is an omnidirectional image, and thus the detection image of the second display range R20 is distorted. Two annotation areas D1 are set in the detection image of the expansion source. Therefore, two frames representing the two annotation areas D1 are displayed overlappingly in the detection image of the second display range R20. In addition, the annotation icon D301 is respectively associated with the two annotation areas D1. Thus, the user can simultaneously confirm the two annotation areas D1 and can grasp the relative position of the annotation area D1 with respect to the entire area of ​​the detection image.

[0101] For example, suppose the user inputs an instruction to select the left annotation area D1. In this case, the annotation information corresponding to the left annotation area D1 is displayed in the annotation information display field R2 on the display of the terminal device 30. This is also the case when the right annotation area D1 is selected.

[0102] exist Figure 7 In the example of , two annotation regions D1 are displayed. However, when three or more annotation regions D1 are set in the detection image of the expansion source, three or more annotation regions D1 are displayed in the detection image of the second display range R20.

[0103] Figure 8 is a flowchart showing the details of the display processing in Embodiment 2. The display processing is Figure 5 The subroutine of step S4.

[0104] Steps S41, S42, S46, S47 and Figure 6 Steps S21, S22, S25, and S26 are the same, so their description is omitted.

[0105] In step S43, the determination unit 113 determines whether a plurality of annotation regions D1 are set in the detection image. In this case, if a plurality of annotation region IDs are associated with image information represented by the detection image, the determination unit 113 determines that a plurality of annotation regions D1 are set in the detection image.

[0106] When a plurality of annotation areas D1 are set in the detection image (Yes in step S43), the setting unit 114 expands the entire area of ​​the detection image to generate an expanded image (step S48). For example, the setting unit 114 generates an expanded image of the detection image using an equidistant azimuth method or a fisheye method.

[0107] Next, the setting unit 114 sets the entire area of ​​the expanded image as the second display range R20 (step S49 ).

[0108] Next, the output unit 115 transmits a display instruction of the detection image of the second display range R20 set in step S49 to the terminal device 30 via the communication unit 13, thereby displaying the detection image of the second display range R20 on the display of the terminal device 30 (step S50). Figure 7 The detection image of the second display range R20 shown is displayed in the image information display area R1 ( Figure 2 ).

[0109] If one annotation area is set in the detection image (No in step S43), the process proceeds to step S44. Figure 6 Steps S23 and S24 are the same, so their description is omitted.

[0110] Thus, according to Embodiment 2, when a plurality of annotation regions are set in the detection image, the plurality of annotation regions D1 are displayed in the detection image of the second display range R20, so that the user can simultaneously check the plurality of annotation regions D1. In addition, the expanded image in which the entire region of the omnidirectional image is expanded is set as the second display range R20, and the detection image of the second display range R20 is initially displayed on the display, so that the user can grasp the relative positions of the plurality of annotation regions D1 with respect to the entire image.

[0111] The present disclosure can adopt the following modified examples.

[0112] (1) Figure 7 In the example of , an expanded image in which the entire area of ​​the detection image is expanded is displayed, but a portion of the expanded image may be displayed. In this case, the setting unit 114 only needs to find the center of gravity of the multiple centers of the multiple annotation areas D1 in the expanded image and set the second display range R20 centered on the center of gravity. In this case, the setting unit 114 only needs to set the second display range R20 so as to include the entire area of ​​the multiple annotation areas D1. Thus, compared with the case of displaying the entire area of ​​the expanded image, it is possible to display a detection image in which the multiple annotation areas D1 are displayed larger.

[0113] (2) In the above embodiment, one shooting location icon is determined by dragging and dropping the selection icon 201 . However, this is an example, and one shooting location icon may be determined by positioning a mouse pointer at a desired position and clicking the mouse pointer.

[0114] (3) In the above embodiment, a construction site is exemplified as the site, but the present disclosure is not limited thereto, and a manufacturing site, a logistics site, a distribution site, a farmland, a construction site, a small commodity market, an office, a hospital, a commercial facility, a nursing facility, etc. may be used as the site.

[0115] (4) In the above embodiment, the detection image is an image captured by a camera, but the present disclosure is not limited thereto. For example, the detection image may also be an image generated by rendering a 3D model of a given space. The 3D model may also be a model generated based on 3D measurement data, or a model generated based on BIM (Building Information Modeling) data.

[0116] Fig. 9 FIG. 4 is a diagram showing a first display range R10 in a modified example of the present disclosure. Fig. 9In the figure, the first display range R100 in the comparative example in which the display method of the present invention is not applied is shown on the left side facing the observer, and the first display range R10 in which the display method of the present invention is applied is shown on the right side. In this example, the images displayed in the first display ranges R10 and R100 are both images obtained by rendering the 3D model. The present invention can display not only the image captured by the camera, but also the image obtained by capturing the 3D model by a virtual camera so that the annotation area is displayed in the center of the first display range R10. In addition, in this example, the annotation area D1 is not represented by a frame surrounding the object, but by the outline of the object. In addition, the annotation icon 301 is displayed in correspondence with the annotation area D1.

[0117] In the first display range R100, the detection image is not displayed so that the annotation area D1 is located at the center, so only a portion of the annotation area D1 is displayed, and the entirety is not displayed. In contrast, in the first display range R10, the detection image is initially displayed so that the annotation area D1 is located at the center. Therefore, even if the user does not perform a scrolling operation to display the entire annotation area D1 and find the annotation area D1, the user can confirm the annotation area D1.

[0118] Industrial Applicability

[0119] The present disclosure can confirm the status of a construction site from a remote location, and is therefore useful in managing the construction site.

Claims

1. An information processing method is an information processing method in a computer. Detecting user selection of images; determining whether an annotation area is set in the image; When it is determined that the annotation area is set in the image, setting a first display range in the image so that the annotation area is located at the center of the image; The image in the first display range is displayed on a display.

2. The information processing method according to claim 1, wherein: Further, determining whether a plurality of annotation areas are set in the image, When it is determined that the plurality of annotation areas are set in the image: In the setting, a second display range is set in the image so as to include the plurality of annotation areas; In the display, the image in the second display range is displayed on the display.

3. The information processing method according to claim 2, wherein: The image is an omnidirectional image, The image in the second display range is an expanded image obtained by expanding the entire area of ​​the omnidirectional image.

4. The information processing method according to claim 2 or 3, wherein: Furthermore, when an operation of selecting any one of the plurality of annotation areas in the image in the second display range displayed on the display is detected, annotation information corresponding to the selected annotation area is displayed on the display.

5. The information processing method according to claim 1, wherein: When it is determined that the annotation area is not set in the image: In the setting, a predetermined third display range is set in the image; In the display, the image in the third display range is displayed on a display.

6. The information processing method according to claim 1, wherein: The first display range includes the entire area of ​​the annotation area and a predetermined range around the annotation area.

7. The information processing method according to claim 1, wherein: In the setting, a magnification for displaying the image in the first display range in a given size of a display area included in the display is set.

8. The information processing method according to claim 1 or 2, wherein: The image is an image obtained by photographing the scene.

9. An information processing device comprising a processor, The processor performs the following processing: Detecting user selection of images; determining whether an annotation area is set in the image; When it is determined that the annotation area is set in the image, setting a first display range in the image so that the annotation area is located at the center of the image; The image in the first display range is displayed on a display.

10. An information processing program for causing a computer to execute the information processing method according to claim 1.

Images