Information processing systems, information processing methods, and programs

Abstract

To allow for easily grasping information on a structure.SOLUTION: An information processing system is provided, comprising a captured image storage unit for storing capture images capturing a structure, structure information storage unit for storing types of parts corresponding to respective sides of a polygon defining a portion of the structure, and a structure information display unit configured to display the capture image and draw a line segment representing each of the sides in a color corresponding to the type of the part.SELECTED DRAWING: Figure 2

Description

【Technical Field】 【0001】 The present invention relates to an information processing system, an information processing method, and a program. 【Background Art】 【0002】 Patent Document 1 discloses a technique for measuring the shape dimensions of a roof as an object from an image captured by a camera mounted on an aircraft, and calculating the area of the roof from such shape dimensions. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Patent Application Laid-Open No. 2003-162552 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 However, in Patent Document 1, it is difficult to grasp information regarding the roof. 【0005】 The present invention has been made in view of such a background, and an object thereof is to provide a technique capable of easily grasping information regarding a structure. 【Means for Solving the Problems】 【0006】 The main invention of the present invention for solving the above problems is an information processing system including: a captured image storage unit that stores a captured image of a structure; a structure information storage unit that stores the type of each part corresponding to each side of a polygon that defines a part of the structure; and a structure information display unit that displays the captured image and draws line segments indicating the respective sides in colors corresponding to the types of the parts. 【0007】 Regarding other problems disclosed in the present application and methods for solving them, they will be clarified by the embodiments of the invention and the drawings. [Effects of the Invention] 【0008】 According to the present invention, information regarding structures can be easily obtained. [Brief explanation of the drawing] 【0009】 [Figure 1] This figure shows an example of the software configuration of information processing terminal 1. [Figure 2] This figure shows an example of screen 10, which is the user interface displayed by the information processing terminal 1. [Modes for carrying out the invention] 【0010】 One embodiment of the present invention will be described below. The information processing system according to this embodiment can be used, for example, for surveying and inspecting buildings that are structures to be photographed by unmanned aerial vehicles. It can also be used for images of structures taken without using an aerial vehicle. 【0011】 In the information processing system of this embodiment, for example, a user operates on the touch panel of an information processing terminal 1, such as a tablet computer or smartphone, to capture an image of a part of a building (in this embodiment, a roof is assumed, but it may also be a wall or the like) using an unmanned aerial vehicle. The information processing terminal 1 can define the area of ​​the roof included in the captured image displayed on the touch panel. Based on this positional information, the information processing terminal 1 can estimate, for example, the length of each part that defines the roof and the area of ​​the roof surface. 【0012】 <Information Processing Terminal 1> In this embodiment, the information processing terminal 1 is assumed to be implemented by a small, tablet-like computer. In other embodiments, the information processing terminal 1 may be implemented by a portable information processing terminal such as a smartphone or game console, or by a stationary information processing terminal such as a personal computer. Furthermore, the information processing terminal 1 may be implemented by multiple hardware components, with functions distributed among them. 【0013】 The information processing terminal 1 may, for example, include a control unit and a touch panel, which is an example of a display unit. 【0014】 The control unit may include a processor, memory, storage, a transceiver unit, an input / output unit, and a bus. The processor is an arithmetic unit that controls the operation of the control unit, controls the transmission and reception of data between each element, and performs processing necessary for program execution. In this embodiment, this processor is, for example, a CPU (Central Processing Unit), and can perform various processes by executing programs stored in the storage (described later) and loaded into memory. The memory includes a main memory composed of a volatile storage device such as DRAM (Dynamic Random Access Memory), and an auxiliary storage device composed of a non-volatile storage device such as flash memory or an HDD (Hard Disk Drive). This memory is used as a workspace for the processor, while also storing a boot loader executed when the control unit is started, and various setting information. The storage stores programs and information used for various processes. For example, when a user operates an aircraft for capturing image information of a roof surface via the information processing terminal 1, the storage may store a program that controls the flight of such an aircraft. The transmitting / receiving unit connects the control unit to a network such as the Internet, and may be equipped with a local area network (LAN), wide area network (WAN), infrared, wireless, Wi-Fi, point-to-point (P2P) network, telecommunications network, cloud communication, Bluetooth®, or BLE (Bluetooth Low Energy) short-range communication interface. The input / output unit is an interface to which input / output devices are connected, and in this embodiment, the touch panel 12 is connected. The bus can transmit, for example, address signals, data signals, and various control signals between the connected processor, memory, storage, transmitting / receiving unit, and input / output unit. 【0015】 A touch panel is an example of a display unit and includes a display surface on which acquired video or images are displayed. In this embodiment, this display surface accepts information input by contact with the display surface and is implemented using various technologies such as resistive touch or capacitive touch. For example, an image captured by the unmanned aerial vehicle 20 may be displayed on the display surface of the touch panel. Buttons, objects, etc., for flight control of the unmanned aerial vehicle or control of the imaging device may also be displayed on the display surface. Furthermore, the user can input input information to the image or buttons etc. displayed on the display surface via the touch panel. Operations for inputting such input information include, for example, touch (tap), slide, or swipe operations on buttons, objects etc. 【0016】 <Information Processing Terminal 1 Software> Figure 1 shows an example of the software configuration of an information processing terminal 1. The information processing terminal 1 may include a structural information display unit 11, an input unit 12, a captured image storage unit 13, and a structural information storage unit 14. The structural information display unit 11 and the input unit 12 can be realized by a processor reading a program stored in storage into memory and executing it. The captured image storage unit 13 and the structural information storage unit 14 can be realized as part of the storage area provided by memory or storage. 【0017】 The captured image storage unit 13 acquires images (captured images) of structures taken by the aircraft. The captured images can be saved as files in any image format, such as JPEG or PNG. 【0018】 The structural information storage unit 14 stores information (structural information) about a part of the structure. 【0019】 The planar components of the structure being photographed (such as roofs, walls, windows, and other planar parts; hereinafter referred to as "surface parts") are defined by polygons. A polygon is defined by multiple vertices. The structural information storage unit 14 can store the positions of the multiple vertices that define each surface part on the captured image. 【0020】 Further, the structure information storage unit 14 can store the gradient for each surface portion. The gradient can be, for example, in units of inches. The gradient may be managed in units of degrees. 【0021】 Further, the structure information storage unit 14 can store the type of the portion with respect to each side (a line segment between two vertices) of the polygon. 【0022】 Further, the structure information storage unit 14 can store the length in world coordinates (real space) for at least one side. 【0023】 Further, the structure information storage unit 14 can store, in association, the floor number to which a part (e.g., a surface portion) of the structure belongs. 【0024】 The input unit 12 can receive structural information from the user and register it in the structural information storage unit 14. If the input unit 12 receives input for the length in world coordinates (real space) for only some of the edges, it can calculate the ratio between the length of that edge on the captured image and its length in world coordinates, calculate the length of the other edges in world coordinates according to this ratio and the lengths of the other edges on the captured image, and include the calculated lengths in world coordinates in the structural information in association with the edge in question. The input unit 12 may also calculate the position of the vertices in world coordinates. For example, the input unit 12 can calculate the position of each vertex as a relative position in world coordinates, based on the pixel position on the captured image (for example, a position relative to the upper left corner of the captured image) with respect to the position in world coordinates corresponding to the upper left corner of the captured image, according to the ratio between the length of that edge on the captured image and its length in world coordinates. Furthermore, the input unit 12 can also detect a reference marker with a given position in world coordinates that is captured in the captured image, and calculate the relative position of the vertices using the relative position (pixel value) of the reference marker on the captured image as the relative position in world coordinates (for example, 3m east and 4m south from the reference marker). In addition, the input unit 12 can also calculate the absolute position in world coordinates of each vertex by accepting input of the absolute position in world coordinates corresponding to the upper left corner of the captured image, for example, or based on the given position of the reference marker. Furthermore, the input unit may correct the length in world coordinates according to the gradient and type of part of each face including the edges. 【0025】 Furthermore, the structural information storage unit 14 can store the area of ​​each surface portion. The area can be the area in world coordinates (real space). The area can also be the area in the captured image. 【0026】 The input unit 12 can also calculate the area of ​​a face portion in world coordinates when it has calculated the length of an edge in world coordinates. For example, based on the position of the vertices in the captured image, the input unit 12 can calculate the area in world coordinates of a closed polygon enclosed by vertices in pixels, and then calculate the area in world coordinates based on the above-mentioned ratio (the ratio of the length of the edge in pixels to the length in world coordinates) and the area in pixels. The input unit 12 can register the calculated area of ​​the face portion in world coordinates in the structural information in the structural information storage unit 14, associating it with the face portion. The input unit 12 can also calculate the area based on the position of each vertex in world coordinates (which may be an absolute position or a relative position). The input unit 12 may also correct the area in world coordinates according to the gradient corresponding to the face portion. 【0027】 <User Interface> Figure 2 shows an example of a screen 10, which is a user interface displayed by the information processing terminal 1. Screen 10 can be displayed, for example, on a touch panel display. The structural information display unit 11 can display the captured image on screen 10 and also display information about each part on the captured image. 【0028】 The input unit 12 can receive input from the user for vertices that define a surface portion. The input unit 12 can, for example, receive a specification of a position on a captured image. The input unit 12 can register the received position in the structural information storage unit 14. 【0029】 In screen 10, the roof surface is divided into polygons, and these polygons are defined by multiple vertices 601. The user can set the vertices 601 on screen 10. For example, the user can specify the position of a vertex 601 by tapping screen 10, and the input unit 12 can accept this. Alternatively, the input unit 12 may display a marker (not shown) in the center of screen 10, and the user can specify the position of a vertex 601 by dragging the captured image to align the vertex 601 with the marker. 【0030】 <6. Representing vertices with points> The structural information display unit 11 can display the vertices 601 on the screen 10 as dots (circular shapes). 【0031】 The input unit 12 may allow the user to select vertex 601 on the screen 10 and edit it by moving vertex 601 to fine-tune its position. 【0032】 <7. Representing body parts and types with color (correspondence with the list)> A line segment between two vertices 601 (one side 701 of the polygon) corresponds to a single part. In the example in Figure 2, parts may include roof parts such as eaves, hip ridges, valleys, ridges, and gable ends, as well as the upper ends of walls. The structural information display unit 11 can draw side 701 as a line segment of a color corresponding to the type of part. By drawing side 701 in a different color for each part, the structural information display unit 11 makes it easy to visually identify what this part is. 【0033】 Prior to display, the input unit 12 can receive input from the user regarding the type of part for each side 701 of the polygon. The input unit 12 registers information indicating the type of part in the structural information storage unit 14, associating it with information that identifies the side 701 (for example, information that identifies the vertices 601 at both ends). 【0034】 The structural information display unit 11 can read the type of part corresponding to edge 701 from the structural information storage unit 14 and draw the line segment corresponding to edge 701 in the color corresponding to the read type. 【0035】 <1. Display the length of the part> The structural information display unit 11 can overlay and display part information 101 for each part on the edges 701. In the example in Figure 2, the part information 101 displays a numerical value indicating the length (in millimeters in the example in Figure 2) inside a filled rectangle. The information processing terminal can also display information about the part (for example, the type of part) in the part information 101. The structural information display unit 11 can read the length of the part corresponding to the edge 701 and draw the read length as part information 101. 【0036】 <2. Display area and gradient> The structural information display unit 11 can display information (surface information 201) about a surface portion inside a polygon representing that surface portion. In the example in Figure 2, the surface information 201 displays the area 202 and the gradient 203 of the surface portion inside a filled rectangle. The structural information display unit 11 can read the area and gradient stored in the structural information storage unit 14 and display them as surface information 201. 【0037】 <3. It is possible to create separate creations for each floor.> Screen 10 includes a settings display area 301 that displays the floor number 302, indicating which floor of the structure the currently configured structural part corresponds to. When the edit button 303 on the settings display area 301 is touched, the floor number 302 can be changed. This allows for the management of structural parts on a floor-by-floor basis. In addition to building floors, groups of structural parts (for example, layers in the height direction) can be defined and set in the settings display area 301. 【0038】 <4. Display the total length and total area of ​​each part> Screen 10 includes a display area 401 that displays the sum of the lengths of each part, totaling 405. The structural information display unit 11 reads structural information corresponding to the floor specified in the setting display area 301 from the structural information storage unit 14, aggregates the lengths of the parts included in the read structural information (lengths in world coordinates (real space)) for each type of part, and displays the aggregated values ​​in the display area 401. 【0039】 The structural information display unit 11 can display aggregate values ​​405 for each type of part, with the line segments drawn and associated with colors corresponding to the type of part. In the example in Figure 2, the display area 401 displays the rectangular shape 403, the type of part 404, and the total length 405 of that type of part, all associated with each other. 【0040】 Furthermore, as shown in Figure 2, the total area of ​​the surface portion, 402, is also displayed in the display area 401. The structural information display unit 11 can read the structural information corresponding to the floor specified in the setting display area 301 from the structural information storage unit 14, calculate the area of ​​the surface portion included in the read structural information, and display the calculated value in the display area 401. 【0041】 In this embodiment, the aggregated value is assumed to be the sum, but any value that can be calculated using some statistical method, such as the mean or median, is acceptable. 【0042】 <5. The display / hide of "Length," "Area," and "Slope" on the image can be toggled.> Screen 10 is equipped with switches 501 to 503. Users can switch them on or off by tapping switches 501 to 503 on screen 10. When switch 501 is on, the structural information display unit 11 displays part information 101 relating to the side lengths of polygons on screen 10, and does not display part information 101 on screen 10 when switch 501 is off. When either switch 502 or 503 is on, the structural information display unit 11 displays surface information 201 of the surface portion on screen 10. When switch 502 is on, the structural information display unit 11 displays area 202 in surface information 201, and does not display area 202 in surface information 201 when switch 502 is off. When switch 503 is on, the structural information display unit 11 displays gradient 203 in surface information 201, and does not display gradient 203 in surface information 201 when switch 503 is off. 【0043】 <8. Deletion is possible> Screen 10 includes a delete button 801. The user can select a face portion by tapping inside a polygon on screen 10. The structural information display unit 11 can highlight the polygon corresponding to the selected face portion (for example, by filling it in). If the delete button 801 is tapped while a face portion is selected, the input unit 12 can delete the information about the selected face portion from the structural information storage unit 14. In this case, the input unit 12 may delete the information about a vertex 601 if the vertex 601 of the polygon defining the selected face portion does not constitute a vertex 601 of a polygon defining another face portion. 【0044】 <9. Expressing the work steps> Screen 10 includes a display selection field 901 for work steps. The display selection field 901 displays multiple work items to be performed on screen 10, and when a work item is tapped, the structural information display unit 11 or the input unit 12 can display a user interface on screen 10 corresponding to the work item. 【0045】 Although these embodiments have been described above, they are intended to facilitate understanding of the present invention and are not intended to limit its interpretation. The present invention can be modified and improved without departing from its spirit, and equivalents thereof are also included. 【0046】 <Disclosure Items> Furthermore, this disclosure also includes the following configurations. [Item 1] A captured image storage unit that stores captured images of structures, A structural information storage unit that stores the type of part corresponding to each side of a polygon defining a part of the aforementioned structure, A structural information display unit that displays the aforementioned captured image and draws line segments representing each side in a color corresponding to the type of part, An information processing system characterized by comprising the following features. [Item 2] The information processing system described in item 1, The structural information storage unit stores the lengths of the sides, The structural information display unit calculates the sum of the lengths of each side for each type and displays the color corresponding to the type and the sum of the lengths in association. An information processing system characterized by the following. [Item 3] The information processing system described in item 2, The structural information display unit displays the color, type, and total value in association with each other. An information processing system characterized by the following. [Item 4] An information processing system described in any one of items 1 to 3, The structural information display unit superimposes the length onto the line segment. An information processing system characterized by the following. [Item 5] A captured image storage unit that stores captured images of structures, A structural information storage unit that stores the type of part corresponding to each side of a polygon defining a part of the aforementioned structure, A structural information display unit that displays the aforementioned captured image and draws line segments representing each side in a color corresponding to the type of part, An information processing system characterized by comprising the following features. [Item 6] The steps include: storing the captured images of the structure, A step of storing the type of part corresponding to each side of a polygon defining a part of the aforementioned structure, The steps include displaying the captured image and drawing line segments representing each side in a color corresponding to the type of part, An information processing method characterized by a computer executing the following. [Item 7] The steps include: storing the captured images of the structure, A step of storing the type of part corresponding to each side of a polygon defining a part of the aforementioned structure, The steps include displaying the captured image and drawing line segments representing each side in a color corresponding to the type of part, A program that causes a computer to execute something. [Explanation of symbols] 【0047】 1. Information processing terminal 10 screens

Claims

[Claim 1] A captured image storage unit that stores captured images of structures, A structural information storage unit that stores the types of each of the multiple parts corresponding to each side of a polygon defining a part of the structure, and the group to which each of the multiple parts belongs, A structural information display unit that displays the captured image and draws line segments representing each of the multiple parts in a color corresponding to the type of each part, Equipped with, The structural information storage unit stores the side lengths corrected according to the gradient of the polygonal surface and the type of part, The structural information display unit is characterized by summing the lengths of each side corresponding to the type of part and the group to which the part belongs, and switching between groups to display the color corresponding to the type of part and the sum of the lengths in association. [Claim 2] The information processing system according to Claim 1, The group is characterized by corresponding to layers in the height direction of the structure. [Claim 3] An information processing system according to Claim 1, The aforementioned group is an information processing system characterized by corresponding to the floors of the structure. [Claim 4] An information processing system according to any one of claims 1 to 3, The structural information display unit superimposes the length onto the line segment. An information processing system characterized by the following. [Claim 5] The steps include: storing the captured images of the structure, A step of storing the types of multiple parts corresponding to each side of a polygon defining a part of the structure, and the group to which each of the multiple parts belongs. The steps include storing the side lengths corrected according to the gradient of the polygonal surface and the type of part, The steps include displaying the captured image and drawing line segments representing each of the multiple parts in colors corresponding to their respective types, The steps include: summing the lengths of each side corresponding to the type of part and the group to which the part belongs, switching between groups, and displaying the color corresponding to the type of part and the sum of the lengths; An information processing method characterized by a computer executing the following. [Claim 6] The steps include: storing the captured images of the structure, A step of storing the types of multiple parts corresponding to each side of a polygon defining a part of the structure, and the group to which each of the multiple parts belongs. The steps include storing the side lengths corrected according to the gradient of the polygonal surface and the type of part, The steps include displaying the captured image and drawing line segments representing each of the multiple parts in colors corresponding to their respective types, The steps include: summing the lengths of each side corresponding to the type of part and the group to which the part belongs, switching between groups, and displaying the color corresponding to the type of part and the sum of the lengths; A program that causes a computer to execute something.

Images