Information processing device, information processing method, and program

Abstract

To enable more accurate derivation of dimensions of an object.SOLUTION: An information processing system is provided, comprising an image acquisition unit for acquiring a captured image of a surface portion of a building, an image correction unit configured to generate a corrected image obtained by applying keystone correction to the image, a line segment acquisition unit configured to acquire multiple line segments constituting a part of the building in the corrected image, and an attribute computation unit for computing an attribute of the surface portion based on the line segments.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technique for measuring the shape and dimensions of a target object, i.e., a roof, from an image of the target object captured by a camera mounted on an aircraft, and calculating the area of ​​the roof from the shape and dimensions. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-162552 Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not specifically disclose how to calculate the area, and it is difficult to calculate the accurate area when the surface portion is photographed at an angle.

[0005] The present invention has been made in view of the above background, and has an object to provide a technique that can more accurately determine the dimensions of an object. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problem is an information processing system comprising an image acquisition unit that acquires an image of a surface portion of a building, an image correction unit that generates a corrected image by correcting the image using trapezoidal correction, a line segment acquisition unit that acquires multiple line segments that make up parts of the building on the corrected image, and an attribute calculation unit that calculates the attributes of the surface portion based on the line segments.

[0007] Other problems and solutions disclosed in this application will be made clear in the section on preferred embodiments of the invention and the drawings. [Effects of the Invention]

[0008] According to the present invention, the dimensions of an object can be determined more accurately. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of an image of a building 100. [Figure 2] 1 is a diagram illustrating an example of a hardware configuration of an information processing device 10. FIG. [Figure 3] FIG. 2 is a diagram illustrating an example of the software configuration of the information processing device 10. [Figure 4] FIG. 2 is a diagram illustrating the operation of the information processing device 10. [Figure 5] FIG. 10 is a diagram showing how the designation of a rectangular portion 120 is accepted. [Figure 6] FIG. 10 is a diagram showing an example of a captured image after keystone correction. [Figure 7] FIG. 10 is a diagram illustrating a calculated area. [Figure 8] FIG. 10 is a diagram illustrating a processing flow when performing correction according to a gradient. [Figure 9] FIG. 10 is a diagram showing an example in which the surface portion 110 is a roof surface. [Figure 10] FIG. 10 is a diagram showing an example of a gradient input screen. [Figure 11] FIG. 10 is a diagram showing an example in which designation of a surface portion 110 is accepted before keystone correction. [Figure 12] FIG. 10 is a diagram showing an example of calculating the length of a straight portion 150. DETAILED DESCRIPTION OF THE INVENTION

[0010] An information processing device 10 according to one embodiment of the present invention will be described below. The information processing device 10 of this embodiment attempts to calculate the area of ​​a surface portion 110 (for example, a wall or roof) of a building 100 from an image of the surface portion 110. It can also calculate the length of a portion corresponding to a side of a rectangle that defines the surface portion 110.

[0011] FIG. 1 is a diagram showing an example of an image of a building 100. The example of FIG. 1 shows a state in which a surface portion 110 of the building 100 (a wall in the example of FIG. 1) is photographed from a direction at an angle relative to the surface portion 110. The information processing device 10 of this embodiment performs keystone correction on this wall 110 and then calculates its area. The information processing device 10 of this embodiment 10 accepts designation of the quadrilateral surface portion 110 or a quadrilateral rectangular portion 120 (such as a window frame in the example of FIG. 1) included in the surface portion 110, and performs keystone correction to convert the quadrilateral into a rectangle with right angles at each corner.

[0012] <Information processing terminal 10> The information processing device 10 according to this embodiment is implemented by a small tablet-like computer. In other embodiments, the information processing device 10 may be implemented by a portable computer such as a smartphone or a game console, or may be implemented by a stationary computer such as a personal computer. The information processing device 10 may also be implemented by multiple pieces of hardware, with functions distributed among them.

[0013] 2 is a diagram showing an example of the hardware configuration of the information processing device 10. As shown in the figure, the information processing device 10 includes a control unit 11 and a touch panel unit 12 which is a display unit.

[0014] The control unit 11 includes, as its main components, a processor 11a, a memory 11b, a storage 11c, a transmitting / receiving unit 11d, and an input / output unit 11e, which are electrically connected to one another via a bus 11f.

[0015] The processor 11a is a computing device that controls the operation of the control unit 11, controls the transmission and reception of data between the elements, and performs processes required for executing programs.

[0016] In this embodiment, the processor 11a is, for example, a CPU (Central Processing Unit), and executes programs stored in a storage 11c (to be described later) and loaded into a memory 11b to perform various processes.

[0017] The memory 11b includes a main storage device configured as a volatile storage device such as a dynamic random access memory (DRAM), and an auxiliary storage device configured as a nonvolatile storage device such as a flash memory or a hard disk drive (HDD).

[0018] This memory 11b is used as a working area for the processor 11a, and also stores a BIOS (Basic Input / Output System) that is executed when the control unit 11 is started up, various setting information, and the like.

[0019] The storage 11c stores programs, information used for various processes, etc. For example, when a user operates an aircraft for capturing image information of the surface portion 110 via the information processing device 10, the storage 11c may store a program for controlling the flight of the aircraft.

[0020] The transmitter / receiver 11d connects the controller 11 to a network such as the Internet, and may include a short-range communication interface such as Bluetooth (registered trademark) or BLE (Bluetooth Low Energy).

[0021] The input / output unit 11e is an interface to which input / output devices are connected, and in this embodiment, the touch panel unit 12 is connected.

[0022] The bus 11f transmits, for example, address signals, data signals, and various control signals between the connected processor 11a, memory 11b, storage 11c, transmission / reception unit 11d, and input / output unit 11e.

[0023] The touch panel unit 12 is an example of a display unit, and includes a display surface 12a on which acquired videos and images are displayed. In this embodiment, the display surface 12a accepts information input by touching the display surface 12a, and is implemented using various technologies such as a resistive film system or a capacitive system.

[0024] For example, a user can input line segment information or node information to an image displayed on the display surface 12a via the touch panel unit 12. Also, display information output by the control unit 11 is displayed on the display surface 12a.

[0025] 3 is a diagram illustrating an example of the software configuration of the information processing device 10. The information processing device 10 includes an image acquisition unit 111, an image correction unit 112, a line segment acquisition unit 113, an attribute calculation unit 114, an image storage unit 131, and a ratio storage unit 132.

[0026] The image storage unit 131 stores an image of the surface of the building 100 (hereinafter referred to as a captured image).

[0027] The ratio storage unit 132 stores the aspect ratio of the rectangular portion 120. If the building 110 includes multiple rectangular portions 120, the aspect ratio can be stored in association with information that identifies the rectangular portion 120.

[0028] The image acquisition unit 111 acquires a captured image. For example, the image acquisition unit 111 may acquire a captured image captured by a camera of an aircraft equipped with a camera and register the image in the image storage unit 131, or may read out a captured image that has been registered in advance in the image storage unit 131.

[0029] The image correction unit 112 generates a corrected image by correcting the image using keystone correction.

[0030] The line segment acquisition unit 113 acquires a plurality of line segments that form the rectangular portion 120 of the building 100 on the corrected image. The line segment acquisition unit 113 can, for example, accept specification of vertices of a quadrangle that represents the rectangular portion 120 from the captured image output to the touch panel unit 12, and acquire the line segments between each vertex. Furthermore, the line segment acquisition unit 113 can, for example, detect quadrangles from the captured image by image analysis and automatically identify the rectangular portion 120, or can accept specification of one of the detected quadrangles to be used as the rectangular portion 120.

[0031] The attribute calculation unit 114 calculates the attribute of the surface portion 110 based on the line segment acquired by the line segment acquisition unit 113. The attribute may be, for example, the area of ​​the surface portion 110. The area may be the area of ​​a polygon that defines the surface portion 110, or the area of ​​the polygon excluding parts such as window frames. Furthermore, the attribute may be, for example, the length of a straight line part (ridge, eaves, etc.) included in the building 100.

[0032] The line segment acquisition unit 113 can acquire rectangle-constituting line segments that are line segments that divide the rectangular portion 120 on the captured image.

[0033] The image correction unit 112 can perform trapezoidal correction of the captured image so that the rectangle (rectangular portion 120) formed by the rectangle-forming line segments acquired by the line segment acquisition unit 113 has the aspect ratio (corresponding to the rectangular portion 120) stored in the ratio memory unit 132.

[0034] The surface portion 110 can be a roof surface, and the attribute can be the slope of the roof surface. In this case, the image acquisition unit 111 acquires a roof surface image, which is a photographed image of the roof surface, the line segment acquisition unit 113 acquires the vertices of polygons that make up the roof surface on the roof surface image, and the attribute calculation unit 114 can calculate the area of ​​the roof surface based on the vertices. The attribute calculation unit 114 can also calculate the area of ​​the roof surface by correcting the area of ​​the roof surface according to the slope.

[0035] <Operation> FIG. 4 is a diagram illustrating the operation of the information processing device 10. As shown in FIG.

[0036] The information processing device 10 acquires a captured image (S101) and acquires line segments (sides of the rectangle representing the rectangular portion 120) constituting the rectangular portion 120 on the captured image (S102). FIG. 5 is a diagram showing how the designation of the rectangular portion 120 is accepted. As shown in FIG. 5, the information processing device 10 can accept designation of vertices 130 of the rectangle representing the rectangular portion 120 on the captured image output to the touch panel unit 12. The information processing device 10 acquires the aspect ratio corresponding to the rectangular portion 120 (S103) and performs keystone correction on the captured image in accordance with the shape and aspect ratio of the rectangle representing the rectangular portion 120 (S104). FIG. 6 is a diagram showing an example of the captured image after keystone correction. As shown in FIG. 6, the rectangular portion 120 has been corrected to become rectangular.

[0037] The information processing device 10 acquires line segments (sides of a polygon representing the surface portion 110) constituting the surface portion 110 on the captured image (S105). For example, as shown in FIG. 6, the information processing device 10 can accept designation of vertices 140 of a polygon representing the surface portion 110 on the captured image after keystone correction. The information processing device 10 can calculate the area of ​​the surface portion 110 based on the acquired line segments (S106). Here, the information processing device 10 can subtract the area of ​​a rectangular portion 120 defined by the vertices 130 from the calculated area of ​​the surface portion 110. In addition to the rectangular portion 120, the information processing device 10 can also accept designation of an area to be excluded from the area of ​​the surface portion 110 and subtract the area of ​​that area. FIG. 7 is a diagram illustrating the calculated area. As shown in FIG. 7, the information processing device 10 can calculate the area of ​​the shaded portion of the surface portion 110 excluding the rectangular portion 120.

[0038] If the surface portion 110 is a roof surface or the like and has a slope, the area of ​​the surface portion 110 can be corrected taking into account the slope of the surface portion 110. Fig. 8 is a diagram illustrating the flow of processing when making corrections according to the slope. Fig. 9 is a diagram illustrating an example in which the surface portion 110 is a roof surface.

[0039] The information processing device 10 can calculate the area by performing trapezoidal correction on the surface portion 110 using the process shown in Fig. 4. When the surface portion 110 itself is rectangular, such as a roof surface, the surface portion 110 can be designated as a rectangular area 120 and the trapezoidal correction shown in Fig. 4 can be performed.

[0040] Next, the information processing device 10 performs the processing shown in Fig. 8 on a second captured image of the side surface of the surface portion 110 taken from a direction in which the slope of the surface portion 110 can be visually recognized. Steps S101 to S105 are the same as those in Fig. 4, but if a rectangular portion 120' such as a window is present on the side surface, a new rectangular portion 120' can be specified and steps S101 to S105 in Fig. 8 can be performed. In step S105, the information processing device 10 can accept the specification of a straight line portion (such as a ridge) from the side surface of the surface portion 110.

[0041] Next, the information processing device 10 displays a ruler 200 on the second captured image to assist the user in determining the gradient of the surface portion 110 from the image (S121). FIG. 10 is a diagram showing an example of a gradient input screen. As shown in FIG. 10, the attribute calculation unit 114 can display the ruler 200 indicating the gradient in a predetermined unit of gradient (0.5 sun in the example of FIG. 10). The attribute calculation unit 114 can display the ruler 200 so that it can be freely moved (including translation, rotation, enlargement, and reduction) on the captured image. The ruler 200 shown in FIG. 10 displays a straight line 201 indicating the gradient in integer units (1 sun, 2 sun, ...). The ruler 200 also displays a dotted line 202 indicating the gradient between two adjacent straight lines 201 in units (0.5 sun, 1.5 sun, ...). The user can compare the slope of the straight line portion of the trapezoidally corrected surface portion 110 (for example, the ridge in the example of Figure 10) with the slope of the straight line 201 and dotted line 202 shown on the ruler 200, and determine the closest one (5 sun in the example of Figure 10) as the gradient of the surface portion 110.

[0042] The information processing device 10 receives input of the gradient of the surface portion 110 (S122). Here, the attribute calculation unit 114 may receive input of the gradient from a keyboard, a mouse, or the like, or may receive selection of the label 203 indicating the gradient on the ruler 200 (for example, pressing on the screen).

[0043] The information processing device 10 can correct the area of ​​the surface portion 110 in accordance with the received gradient (S123).

[0044] As described above, the information processing device 10 of this embodiment can perform keystone correction on a captured image in accordance with the shape of a rectangular portion 120 with a given aspect ratio, such as a window frame, on the captured image. Therefore, the area of ​​the surface portion 110 shown on the captured image after keystone correction can be calculated more accurately.

[0045] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and equivalents thereof are also included in the present invention.

[0046] For example, in the above embodiment, the information processing device 10 has been described as a standalone device, but it may also be implemented as an information processing system including the information processing device 10. For example, the information processing system may include the information processing device 10 and a server. In this case, for example, some or all of the functions possessed by the processor 11a and storage 11c of the information processing device 10 according to the above embodiment may be realized by the processor and storage included in the server.

[0047] Furthermore, in the above embodiment, the surface portion 110 is described as a part that constitutes the building 100, but it may also be a tree or any ground surface, or even the surface portion of an object such as a temporarily stopped automobile or animal.

[0048] Furthermore, the steps in the above embodiments do not necessarily have to be processed in chronological order according to the order described in the flowcharts. For example, the steps in the processes of the above embodiments may be processed in an order different from that described in the flowcharts, or may be processed in parallel.

[0049] Furthermore, in this embodiment, the designation of the sides (line segments) of the polygon representing the surface portion 110 is accepted after the keystone correction, but the designation may also be accepted before the keystone correction. Fig. 11 is a diagram showing an example in which the designation of the surface portion 110 is accepted before the keystone correction. As shown in Fig. 11, the designation of the vertices 140 constituting the surface portion 110 is accepted on the captured image before the keystone correction, and the coordinates of the vertices 140 on the captured image are corrected by the keystone correction, and the area of ​​the surface portion 110 can be calculated using the corrected coordinates.

[0050] Furthermore, in this embodiment, the area of ​​the surface portion 110 is calculated, but the length of the straight line portion 150 may also be calculated. Fig. 12 is a diagram showing an example of calculating the length of the straight line portion 150. As shown in Fig. 12, the information processing device 10 can calculate the length between the vertices 160 by accepting designation of vertices 160 that are both ends of the straight line portion 150 (ridges in the example of Fig. 12) on the captured image.

[0051] Furthermore, in this embodiment, the area and / or length on the captured image is calculated, but it is also possible to accept a specification of the length of a portion in real space (for example, the length of the straight portion 150, the length of one side of the surface portion 110, the length of one side of the rectangular portion 120, etc.) and calculate the area and / or length in real space according to the ratio between the accepted length in real space and the length on the captured image after trapezoidal correction.

[0052] In this embodiment, the input of the slope of the surface portion 110, such as a roof, is accepted after keystone correction. However, for example, when the building 100 is photographed horizontally from the side, the input of the slope may be accepted without keystone correction. In this case, for example, the attribute calculation unit 114 may display a ruler 200 shown in FIG. 8 on the photographed image without keystone correction so that the ruler 200 can be freely moved on the photographed image, allowing the user to input the slope using the ruler 200 as a reference. Furthermore, the attribute calculation unit 114 may accept the specification of the position of the vertex of a straight line portion according to the position of the ruler 200 and the input slope. For example, in the example of FIG. 8, the position where the straight line 201 or the dotted line 202 forms an angle may be accepted as the position of the vertex of the straight line portion (such as a ridge) of the surface portion 110 of the building 100.

[0053] Furthermore, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present invention may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0054] <Disclosures> The present disclosure also includes the following configurations. [Item 1] an image acquisition unit that acquires an image of a surface portion of a building; an image correction unit that generates a corrected image by correcting the image using keystone correction; a line segment acquisition unit that acquires a plurality of line segments that constitute the portion of the building on the corrected image; an attribute calculation unit that calculates attributes of the surface portion based on the line segment; An information processing system comprising: [Item 2] Item 1, an information processing system according to item 1, a ratio storage unit that stores the aspect ratio of the rectangular portion; the line segment acquisition unit acquires rectangle-constituting line segments that are line segments that partition the rectangular portion of the building on the image; the image correction unit performs keystone correction on the image so that a rectangle formed by the rectangle-forming line segments has the aspect ratio; An information processing system characterized by: [Item 3] Item 1 or 2, an information processing system according to The surface portion is a roof surface, the attribute is the slope of the roof surface; An information processing system characterized by: [Item 4] Item 3. The information processing system according to item 3, the image acquisition unit acquires a roof surface image, which is an image obtained by capturing the roof surface; the line segment acquisition unit acquires vertices of polygons constituting the roof surface on the roof surface image, the attribute calculation unit further calculates an area based on the vertices, and corrects the calculated area in accordance with the slope to calculate the area of ​​the roof surface; An information processing system characterized by: [Item 5] acquiring an image of a surface portion of a building; generating a corrected image by correcting the image using keystone correction; acquiring a plurality of line segments that constitute the portion of the building on the corrected image; calculating attributes of the surface portion based on the line segments; An information processing method characterized by being executed by a computer. [Item 6] acquiring an image of a surface portion of a building; generating a corrected image by correcting the image using keystone correction; acquiring a plurality of line segments that constitute the portion of the building on the corrected image; calculating attributes of the surface portion based on the line segments; A program that causes a computer to execute the following. [Explanation of symbols]

[0055] 10. Information processing equipment 100 buildings 110 surface area 120 Rectangular part

Claims

1. an image acquisition unit that acquires an image of a surface portion of a building; an image correction unit that generates a corrected image by correcting the image using keystone correction; a line segment acquisition unit that acquires a plurality of line segments that constitute the portion of the building on the corrected image; an attribute calculation unit that calculates attributes of the surface portion based on the line segment; Equipped with The surface portion is a roof surface, the attribute is the slope of the roof surface; the image acquisition unit acquires a roof surface image, which is an image obtained by capturing the roof surface; the line segment acquisition unit acquires vertices of polygons constituting the roof surface on the roof surface image, The information processing system is characterized in that the attribute calculation unit further calculates an area based on the vertices, and corrects the calculated area in accordance with the slope to calculate the area of ​​the roof surface.

2. 2. The information processing system according to claim 1, a ratio storage unit that stores the aspect ratio of the rectangular portion; the line segment acquisition unit acquires rectangle-constituting line segments that are line segments that partition the rectangular portion of the building on the image; the image correction unit performs keystone correction on the image so that a rectangle formed by the rectangle-forming line segments has the aspect ratio; An information processing system characterized by:

3. acquiring an image of a surface portion of a building; generating a corrected image by correcting the image using keystone correction; acquiring a plurality of line segments that constitute the portion of the building on the corrected image; calculating attributes of the surface portion based on the line segments; The computer executes The surface portion is a roof surface, the attribute is the slope of the roof surface; In the step of acquiring an image, a roof surface image is acquired, which is an image obtained by photographing the roof surface; In the step of acquiring the line segments, vertices of polygons constituting the roof surface on the roof surface image are acquired, An information processing method characterized in that, in the calculating step, an area is calculated based on the vertices, and the calculated area is corrected according to the slope to calculate the area of ​​the roof surface.

4. acquiring an image of a surface portion of a building; generating a corrected image by correcting the image using keystone correction; acquiring a plurality of line segments that constitute the portion of the building on the corrected image; calculating attributes of the surface portion based on the line segments; on the computer, The surface portion is a roof surface, the attribute is the slope of the roof surface; In the step of acquiring an image, a roof surface image is acquired, which is an image obtained by photographing the roof surface; In the step of acquiring the line segments, vertices of polygons constituting the roof surface on the roof surface image are acquired, In the calculating step, an area is calculated based on the vertices, and the calculated area is corrected according to the slope to calculate the area of ​​the roof surface.

Images