Information processing method, information processing system, and program

Abstract

An information processing method, an information processing system, and a program capable of extracting a specific inspection object from within an image. [Solution] An information processing method in which a computer executes information processing to extract straight lines corresponding to a long-sized inspection object from an image captured by an unmanned mobile body. The computer executes a process of generating a filtered image in which lines of a specific width are emphasized from the captured image by spatial filtering based on a first filter having selectivity for lines of a specific width corresponding to the inspection object, and a process of extracting a straight line corresponding to the inspection object from multiple straight lines in the filtered image based on a Hough transform process for the generated filtered image.

Description

[Technical field]

[0001] The present disclosure relates to an information processing method, an information processing system, and a program. [Background technology]

[0002] In recent years, mobile objects such as drones, unmanned aerial vehicles (UAVs), and unmanned ground vehicles (UGVs) (collectively referred to as "mobiles") have begun to be used in industry. For example, Patent Document 1 discloses a system for photographing and inspecting power lines using an aerial object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-196355 A Summary of the Invention [Problem to be solved by the invention]

[0004] Here, as one method of identifying a long inspection target such as an overhead line (power line) in order to track the inspection target, an unmanned mobile vehicle may perform edge detection on a captured image acquired by the unmanned mobile vehicle to extract the edge of the inspection target and identify the position of the inspection target. However, since various buildings, roads, and other linear structures other than the inspection target are reflected in the background of the captured image, when edge detection is performed on the captured image, many edges that become noise may be extracted. Therefore, it may be difficult to identify the edge of the target inspection target. In particular, when the edge of a linear object (especially a long and thin object) that is not the inspection target in the background is extracted, the unmanned mobile vehicle may track such an object that is not the inspection target, and the inspection target may be lost, making it difficult to perform an appropriate inspection.

[0005] In addition to the above-mentioned edge detection method, a method of segmenting and extracting the inspection object in the captured image by deep learning can be considered. However, in order to execute such segmentation by deep learning, a high-spec arithmetic processing device is required, and it may be difficult to use the segmentation method in a general purpose manner. In addition, when a non-inspection object of the same type as the inspection object is reflected (for example, when a non-inspection object is reflected in the background of the captured image in addition to the inspection object power line), it is difficult to distinguish the inspection object from the non-inspection object using the segmentation method.

[0006] The present disclosure has been made in consideration of the above-described background, and one of the objectives of an exemplary embodiment of the present disclosure is to provide an information processing method, an information processing system, and a program capable of extracting a specific inspection object from an image. [Means for solving the problem]

[0007] An information processing method according to one aspect of the present disclosure is an information processing method in which a computer executes information processing to extract a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, A process of generating a filtered image in which lines having a specific width corresponding to the object to be inspected are emphasized from the captured image by a spatial filtering process based on a first filter having selectivity for the lines having the specific width corresponding to the object to be inspected; The computer executes a process of extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on line vote number information obtained by performing a Hough transform process on the generated filtered image.

[0008] Because the information processing method has the above-mentioned characteristics, even if the captured image contains linear non-inspection objects such as buildings, roads, or other long objects in addition to the object to be inspected, it is possible to extract a straight line corresponding to the desired inspection object from the captured image. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a system configuration of an information processing system according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a block diagram showing a hardware configuration of the terminal 1 shown in FIG. [Figure 4] FIG. 4 is a block diagram showing a hardware configuration of the server 2 shown in FIG. [Diagram 5] FIG. 5 is a block diagram showing the hardware configuration of the unmanned aerial vehicle 4 shown in FIG. [Figure 6] FIG. 6 is a block diagram illustrating the functions of the terminal 1 and the server 2 shown in FIG. [Figure 7] FIG. 7 is an example of an image acquired from the imaging unit of the unmanned aerial vehicle 4. [Figure 8] FIG. 8 is an example of a filtered image generated from the image of FIG. [Figure 9] FIG. 9 is a diagram for explaining a convolution operation process using a general edge enhancement filter. [Figure 10] FIG. 10 is a schematic diagram showing an example of the specific width emphasis filter (first filter). [Figure 11] FIG. 11 is a diagram for explaining an example of the convolution operation process by the specific width emphasis filter shown in FIG. [Figure 12] FIG. 12 is a schematic diagram showing a modified example of the specific width emphasis filter. [Figure 13] FIG. 13 is a schematic diagram showing an example of the different diameter line extraction filter (second filter). [Figure 14] FIG. 14 is a schematic diagram illustrating a one-side edge filter (third filter). [Figure 15] FIG. 15 is a diagram for explaining the luminance distribution of the inspection object in the test image. [Figure 16]FIG. 16 is a schematic diagram showing an example of the antireflection filter (fourth filter). [Figure 17] FIG. 17 is a diagram showing an example of a captured image. [Figure 18] FIG. 18 is a schematic diagram showing a filtered image converted from the image shown in FIG. [Figure 19] FIG. 19 is a schematic diagram showing a filtered image converted from the image shown in FIG. [Figure 20] FIG. 20 is a schematic diagram showing a filtered image converted from the image shown in FIG. [Figure 21] FIG. 21 is a diagram showing an example of an image in which the position of the tracking target is visualized. [Figure 22] FIG. 22 is a flowchart showing an example of processing by the information processing system according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The information processing method, information processing system, and program disclosed herein have the following configuration. [Item 1] An information processing method in which a computer executes information processing to extract a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: A process of generating a filtered image in which lines having a specific width corresponding to the object to be inspected are emphasized from the captured image by a spatial filtering process based on a first filter having selectivity for the lines having the specific width corresponding to the object to be inspected; and a process of extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by a Hough transform process on the generated filtered image. [Item 2] 2. The information processing method according to item 1, wherein in the process of generating the filtered image, the spatial filtering process is performed based on the first filter and a second filter having selectivity for different radial lines having a width different from the specific width, and the filtered image in which the lines of the specific width are emphasized is generated. [Item 3] The information processing method described in item 1, in which, in the process of generating the filtered image, the spatial filtering process is performed based on the first filter and a third filter that responds only to one of the two edges of the line of the specific width, and the filtered image in which the line of the specific width is emphasized is generated. [Item 4] the third filter includes a filter that responds only to one of the two edges of the line of the specific width, and a filter that responds only to the other of the two edges; 4. The information processing method according to item 3, wherein the filtered image is generated by the spatial filtering process based on the first filter, the third filter that responds only to one of the two edges, and the third filter that responds only to the other of the two edges. [Item 5] 2. The information processing method according to item 1, wherein in the process of generating the filtered image, the filtered image in which the lines of the specific width are emphasized is generated by the spatial filtering process based on the first filter and a fourth filter having a filter coefficient weighted based on a luminance distribution of the object to be inspected in a test image. [Item 6] The method further includes, before the spatial filtering process, generating a composite filter by combining at least one other filter and the first filter; The other filter is a second filter having selectivity for different diameter wires having a width different from the specified width; a third filter that responds to only one of the two edges of the line of the specific width; and a fourth filter having a filter coefficient weighted based on a luminance distribution of the inspection object in a test image; 6. An information processing method according to any one of items 1 to 5, wherein in the process of generating the filtered image, the spatial filtering process is performed by a convolution operation using the composite filter, and the filtered image in which the lines of the specific width are emphasized is generated. [Item 7] 7. The information processing method according to item 6, wherein the composite filter is generated by combining the first filter, the second filter, the third filter, and the fourth filter. [Item 8] The Hough transform process is a weighted Hough transform, 6. The information processing method according to any one of items 1 to 5, further comprising extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by the weighted Hough transform and weighted by an intensity value in the filtered image. [Item 9] An information processing system for extracting a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: a filtering processing unit that generates a filtered image in which lines having a specific width corresponding to the object to be inspected are emphasized from the captured image by spatial filtering processing based on a first filter having selectivity for the lines having the specific width; and a specific line extraction unit that extracts a line corresponding to the object to be inspected from a plurality of lines in the filtered image based on line voting number information obtained by Hough transform processing of the generated filtered image. [Item 10] A program for causing a computer to execute information processing for extracting a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, A process of generating a filtered image in which lines having a specific width corresponding to the object to be inspected are emphasized from the captured image by a spatial filtering process based on a first filter having selectivity for the lines having the specific width corresponding to the object to be inspected; and extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by Hough transform processing of the generated filtered image.

[0011] <Details of the embodiment> Hereinafter, an information processing system according to an embodiment of the present disclosure will be described with reference to the drawings. In each of the attached drawings, identical or similar elements are given identical or similar reference symbols and names, and duplicate descriptions of identical or similar elements may be omitted in the description of the embodiment. Note that the contents shown in each drawing are merely examples for explaining the present embodiment, and are merely schematic examples for easy explanation of the present embodiment. The contents of each drawing may be modified or changed within the scope of no technical problem.

[0012] <Outline of this embodiment> The information processing system according to the present embodiment is a system involved in processing for extracting an inspection object from an image (hereinafter referred to as a captured image) captured by an unmanned mobile body and automatically tracking the inspection object by the unmanned mobile body. The inspection object is a long object (for example, an object having a longitudinal direction such as a line, a tube, a column, or a long plate), and is not necessarily limited thereto, and examples thereof include overhead lines (power lines), pipes, division lines, and road markings (white lines, center lines, etc.). The information processing system according to the present embodiment is preferably applied to the inspection of overhead lines or pipes, and more preferably to the inspection of overhead lines. Examples of unmanned mobile bodies include drones, unmanned aerial vehicles, and unmanned ground vehicles, and are not particularly limited thereto. In the case of the inspection of power lines, which will be exemplified later, it is preferable to use an unmanned aerial vehicle such as a drone or an unmanned aerial vehicle. In this embodiment, the information processing system will be described in detail by taking as an example a case in which a power line is inspected by an unmanned aerial vehicle.

[0013] As shown in Fig. 1, the information processing system exemplified below extracts straight lines corresponding to a plurality of power lines (particularly multi-conductor power lines, for example, two, three, four, six, etc. power lines separated by span spacers) from an image captured along the plurality of power lines extending side by side on a support (for example, a steel tower, etc.). As an example, images of the plurality of power lines may be captured by remotely controlling a camera mounted on an unmanned aerial vehicle 4 as shown in Fig. 1, which flies autonomously or remotely, based on instructions from a terminal 1 owned by a user.

[0014] More specifically, the information processing system generates a filtered image by spatial filtering from an image captured by an aircraft. At this time, the spatial filtering is performed based on a specific width enhancement filter that is different from a conventionally known edge enhancement filter, thereby generating a filtered image in which a line having a specific width (hereinafter, sometimes referred to as a "specific width line") corresponding to the power line to be inspected is enhanced.

[0015] After the filter-processed image is generated, a Hough transform process is performed on the filter-processed image, and a specific width line corresponding to the power line to be inspected is extracted from a plurality of straight lines in the filter-processed image based on straight line vote number information acquired by the Hough transform process. Since the specific width line is emphasized in the filter-processed image, which is an input image for the Hough transform, the straight line corresponding to the power line to be inspected can be easily extracted from the straight line vote number information. In this Hough transform process, a general Hough transform method can be adopted, and in particular, it is preferable to acquire the straight line vote number information by executing a weighted Hough transform (WHT). In the weighted Hough transform, the straight line vote number information is weighted based on an intensity value (corresponding to a luminance value (pixel value) after spatial filtering) in the filter-processed image. In the filter-processed image, which is an input image for the weighted Hough transform, the specific width line is displayed with a higher intensity value than other lines corresponding to objects not to be inspected such as the background, and therefore, in the straight line vote number information acquired by the weighted Hough transform, a higher vote number is assigned to the specific width line than other lines. Therefore, when the weighted Hough transform is performed, a line of a specific width can be extracted more easily than with the normal Hough transform, based on the line vote number information weighted based on the intensity value in the filter processed image.

[0016] As described above, by performing spatial filtering processing based on a specific width emphasis filter and weighted Hough transform processing, even if a straight-line non-inspection object such as a building, road, or other long object that is not the inspection object is reflected in the captured image, a straight line corresponding to the target inspection object can be extracted from the captured image. Furthermore, in the above information processing system, it is possible to extract a straight line corresponding to the inspection object by a calculation process that is simpler than deep learning, and a more inexpensive and general-purpose calculation processing device can be used than the calculation processing device required for the segmentation method using deep learning.

[0017] <System configuration> As shown in FIG. 2, the information processing system of this embodiment has a terminal 1, a server 2, and an unmanned aerial vehicle 4. The terminal 1, the server 2, and the unmanned aerial vehicle 4 may be connected to each other so as to be able to communicate with each other via a network NW. The illustrated configuration is an example, and is not limited to this, and for example, the unmanned aerial vehicle 4 may not be connected to the network NW. In that case, the unmanned aerial vehicle 4 may be operated by a transmitter (so-called radio control) operated by a user, or image data acquired by a camera of the unmanned aerial vehicle 4 may be stored in an auxiliary storage device (for example, a memory card such as an SD card and / or a USB memory) connected to the unmanned aerial vehicle 4, and the image data may be read out from the auxiliary storage device to the terminal 1 and / or the server 2 and stored by the user after the fact, or the unmanned aerial vehicle 4 may be connected to the network NW only for the purpose of operation or for the purpose of storing image data.

[0018] <Hardware configuration of terminal 1> 3 is a diagram showing a hardware configuration of the terminal 1 in this embodiment. Note that the illustrated configuration is an example, and the terminal 1 may have other configurations.

[0019] The terminal 1 includes at least a processor 10, a memory 11, a storage 12, a transceiver unit 13, an input / output unit 14, etc., which are electrically connected to each other via a bus 15. The terminal 1 may be, for example, a general-purpose computer such as a workstation or a personal computer.

[0020] The processor 10 is a computing device that controls the overall operation of the terminal 1, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication processing, etc. For example, the processor 10 is a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), and executes programs, etc. stored in the storage 12 and deployed in the memory 11 to perform each information processing.

[0021] The memory 11 includes a main memory configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory configured with a non-volatile storage device such as a flash memory and an HDD (Hard Disc Drive). The memory 11 is used as a work area for the processor 10, and also stores a BIOS (Basic Input / Output System) that is executed when the terminal 1 is started up, various setting information, and the like.

[0022] The storage 12 stores various programs such as application programs. A database storing data used in each process may be constructed in the storage 12. In addition, a storage unit 130 described later may be provided in a part of the storage area.

[0023] The transmission / reception unit 13 is a communication interface for the terminal 1 to communicate with an external device (not shown) and the unmanned aerial vehicle 4, etc., via a communication network. The transmission / reception unit 13 may further include a short-range communication interface of Bluetooth (registered trademark) and BLE (Bluetooth Low Energy) and / or a USB (Universal Serial Bus) terminal, etc.

[0024] The input / output unit 14 includes information input devices such as a keyboard and a mouse, and output devices such as a display.

[0025] A bus 15 is commonly connected to each of the above elements, and transmits, for example, address signals, data signals, and various control signals.

[0026] <Server 2> 4 also includes a processor 20, a memory 21, a storage 22, a transmission / reception unit 23, an input / output unit 24, etc., which are electrically connected to each other via a bus 25. The server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing. The functions of each element of the server 2 can be configured in the same way as the above-mentioned terminal 1, and detailed description of each element of the server 2 will be omitted.

[0027] <Unmanned Aerial Vehicle 4> 5 is a block diagram showing a hardware configuration of the unmanned aerial vehicle 4. The flight controller 41 may have one or more processors, such as a programmable processor (eg, a central processing unit (CPU)).

[0028] The flight controller 41 may also have or have access to a memory 411. The memory 411 stores logic, code, and / or program instructions that the flight controller can execute to perform one or more steps. The flight controller 41 may also include sensors 412, such as inertial sensors (accelerometers, gyro sensors), GPS sensors, and proximity sensors (e.g., lidar).

[0029] The memory 411 may include, for example, a separable medium such as an SD card and a random access memory (RAM) or an external storage device. Data acquired from the camera / sensors 42 may be directly transmitted to and stored in the memory 411. For example, still image / video data captured by a camera or the like may be recorded in an internal memory or an external memory, but is not limited thereto. The captured still image / video data may be recorded in at least one of the terminal 1 or the server 2 from the camera / sensors 42 or the internal memory via the network NW. The camera 42 is installed in the unmanned aerial vehicle 4 via a gimbal 43.

[0030] The flight controller 41 includes a control module (not shown) configured to control the state of the unmanned air vehicle 4. For example, the control module may be configured to provide six degrees of freedom (translational motion x, y, and z, and rotational motion θ x , θ y and θ z The control module controls the propulsion mechanism (motor 45, etc.) of the unmanned aerial vehicle 4 via an ESC 44 (Electric Speed ​​Controller) to adjust the spatial arrangement, speed, and / or acceleration of the unmanned aerial vehicle 4. The motor 45, powered by a battery 48, rotates a propeller 46 to generate lift for the unmanned aerial vehicle 4. The control module can control one or more of the states of the onboard components and sensors.

[0031] The flight controller 41 can communicate with a transceiver 47 configured to transmit and / or receive data from one or more external devices (e.g., a transceiver 49, a terminal 1, a display device, or other remote control). The transceiver 49 may use any suitable communication means, such as wired or wireless communication.

[0032] For example, the transceiver 47 may utilize one or more of a local area network (LAN), a wide area network (WAN), infrared, radio, WiFi, a point-to-point (P2P) network, a telecommunications network, cloud communications, and the like.

[0033] The transceiver unit 47 can transmit and / or receive one or more of the following: data acquired by the sensors 42, processing results generated by the flight controller 41, specified control data, user commands from a terminal or a remote controller, etc.

[0034] The sensors 42 may include inertial sensors (accelerometers, gyro sensors), GPS sensors, proximity sensors (e.g., lidar), or vision / image sensors (e.g., cameras).

[0035] <Functions of Terminal 1> FIG. 6 is a block diagram illustrating functions implemented in the terminal 1 and the server 2. The terminal 1 includes a communication unit 110, an image acquisition unit 115, a processing unit 120, and a storage unit 130. The processing unit 120 includes a filtering processing unit 121, a specific straight line extraction unit 122, a filter generation unit 123, a slope determination unit 124, and a tracking target position setting unit 125. The storage unit 130 includes an information / image storage unit 131 and a filter storage unit 132. Note that although the various functional units are illustrated as functional units in the processor 10 of the terminal 1, some or all of the various functional units may be realized in any of the configurations of the processor 10 of the terminal 1 or the processor 20 of the server 2, or the flight controller 41 of the unmanned aerial vehicle 4, depending on the capabilities of the processor 10 of the terminal 1 or the processor 20 of the server 2, or the flight controller 41 of the unmanned aerial vehicle 4.

[0036] The communication unit 110 communicates with the server 2 and the unmanned aerial vehicle 4 via the network NW. The communication unit 110 also functions as a reception unit that receives various requests and data from the server 2, the unmanned aerial vehicle 4, etc.

[0037] The image acquisition unit 115 acquires images captured by a digital camera mounted on the unmanned aerial vehicle 4 or a digital camera used by a user from the digital camera, for example, by wireless communication via a communication interface or wired communication via a USB terminal or the like. The image acquisition unit 115 may be configured to acquire captured images via a storage medium such as a USB memory or an SD memory, but is more preferably configured to acquire captured images in real time, particularly by wireless communication from the unmanned aerial vehicle 4 or within the unmanned aerial vehicle 4.

[0038] The captured image acquired by the image acquisition unit 115 may be a moving image or a still image. When the captured image is a moving image, the moving image may be divided into still images for each frame, and the still images may be used in each functional unit of the processing unit 120 described later. Still images may be extracted at a predetermined interval from the still images divided into frames, and the extracted still images may be used in each functional unit of the processing unit 120. The captured image captured by the unmanned aerial vehicle 4 or the like may be a color image, a grayscale image, or a black and white image. When the captured image is a color image, the image acquisition unit 115 or a functional unit dedicated to image conversion may execute a process of grayscaling or binarizing the color image.

[0039] The processing unit 120 includes functional units that execute a series of processes for extracting straight lines corresponding to the power line to be inspected (hereinafter referred to as the inspection target line) from the captured image acquired by the image acquisition unit 115 (for example, an image captured along one or more power lines extending alongside one another, an image obtained by converting the captured image into a grayscale image or a binarized image, etc.). The inspection target line may be either a single line or multiple lines. Furthermore, the captured image may include power lines other than the inspection target line.

[0040] The filtering processing unit 121 generates an image (referred to as a filtered image) in which a line (specific width line) having a specific width (thickness) corresponding to the inspection target line is emphasized from the captured image by spatial filtering processing using a predetermined filter (operator). Here, "the specific width line is emphasized" means that the specific width line is displayed in the filtered image with a higher intensity value than the background and non-inspection objects (for example, power lines, buildings, roads, supports, cross arms, and other structures other than the inspection target line) included in the captured image. The "intensity value" here means the pixel value (luminance value) of each pixel, which is the smallest unit constituting the filtered image, and for example, the pixel value in the filtered image can be expressed in 256 gradations from 0 to 255. In the filtered image, the pixel representing the specific width line has a pixel value higher than the background and non-inspection objects. In fact, an example of a filtered image generated from the captured image shown in FIG. 7 is the image shown in FIG. 8. In the captured image of Figure 7, the four power lines extending from the upper left to the lower right of the screen are the lines under inspection, and in Figure 8, these four lines under inspection are displayed as white lines with higher pixel values ​​than their surroundings.

[0041] Spatial filtering is a method that has also been used in conventional edge detection processing. Conventional edge detection processing uses edge enhancement filters expressed as square matrices (matrices of size n x n), such as differential filters, Prewitt filters, Sobel filters, Roberts filters, and Laplacian filters, and detects edge parts such as contour lines contained in an image by performing a convolution operation using these edge enhancement filters.

[0042] For example, FIG. 9 is a diagram for explaining a convolution operation by a conventional edge enhancement filter. As shown in FIG. 9, in spatial filtering by a Prewitt filter having a size of 3×3, for a pixel of interest (a pixel shown in gray in the input image of FIG. 9) and nine pixels including its surroundings, an output value of the pixel of interest in an output image (processed image) is calculated by a multiplication and accumulation operation between the pixel values ​​of the nine pixels and the coefficients (weights, sometimes called kernel coefficients) of the filter corresponding to each pixel. Such an arithmetic operation process for extracting peripheral information of a pixel of interest and generating a new image is called a convolution operation process. In FIG. 9, a convolution operation is performed on each pixel of the input image, thereby obtaining an output image in which the boundary (i.e., edge) between a pixel value of 50 and a pixel value of 230 in the input image is extracted. Note that, in FIG. 9, 540 is output as a result of the multiplication and accumulation operation, but the output image is generated by normalizing (normalizing to 256 gradations from 0 to 255) such a result of the multiplication and accumulation operation. As shown in Figure 9, in conventional edge detection processing, the contour line of an object is extracted, so when spatial filtering using a conventional edge enhancement filter is performed on one power line, two edges are extracted from the single power line.

[0043] In the filtering processing unit 121 of this embodiment, spatial filtering is performed based on a specific width emphasis filter (first filter) having selectivity for lines having a specific width (specific width lines) corresponding to the inspection target line, rather than the conventional edge emphasis filter as described above. Having "selectivity for specific width lines" means reacting more strongly to lines having a specific width than other lines having different widths (indicating a larger absolute value of the output value for specific width lines than other lines). Specifically, assuming that the assumed width of the inspection target line in the captured image is W (unit: pixel), the specific width emphasis filter is a filter having a calculation area corresponding to this assumed width W.

[0044] The expected width W (specific width) means the width in the direction perpendicular to the longitudinal direction of the inspection target line. The width of the inspection target line shown in the captured image is mainly determined according to the distance between the inspection target line and the unmanned aerial vehicle 4. Since it is possible to fly the unmanned aerial vehicle 4 while keeping the distance from the inspection target line constant, the unmanned aerial vehicle 4 is test driven, and the width (number of pixels) of the inspection target line shown in the test image obtained at that time (hereinafter referred to as a test image) is measured, so that the expected width W can be calculated in advance. Based on the expected width W calculated in this way, a specific width emphasis filter 6a as shown below may be designed. In the following description, the length direction in each filter means the direction along the longitudinal direction of the inspection target line in the captured image, and the width direction in each filter means the direction along the width direction of the inspection target line in the captured image.

[0045] For example, the filter 6a shown in FIG. 10 is an example of a specific width emphasis filter. The specific width emphasis filter 6a shown in FIG. 10 is different from a conventional edge emphasis filter and has a size designed based on the assumed width W of the inspection target line in the captured image. The size of the smallest unit cell constituting the specific width emphasis filter 6a is equal to the size of one pixel, which is the smallest unit of the captured image. The specific width emphasis filter 6a has a central region 61a and two outer regions 62a. The central region 61a has a width corresponding to the assumed width W of the inspection target line in the captured image in the Y-axis direction (width direction) of FIG. 10. On the other hand, the two outer regions 62a are located outside the central region 61a in the Y-axis direction of FIG. 10, and each has a width (W / 2) that is half the assumed width W in the Y-axis direction. In other words, the specific width emphasis filter 6a has a width corresponding to twice the assumed width W in the Y-axis direction.

[0046] The length L of the specific width emphasis filter 6a in the X-axis direction (length direction) may be one pixel or more, and is not particularly limited. For example, the length L may be set so that the specific width emphasis filter 6a is a square (i.e., L=2W). The length L may be set to an even number or an odd number. In FIG. 10, the specific width emphasis filter 6a is illustrated assuming that the inspection target line extends along the X-axis direction in the captured image (the longitudinal direction of the inspection target line is along the X-axis direction). However, the orientation of the inspection target line in the captured image is not particularly limited, and when the captured image is acquired so that the longitudinal direction of the inspection target line is along the Y-axis direction, for example, the matrix of the specific width emphasis filter 6a shown in FIG. 10 may be replaced, and the convolution operation process may be performed so that the direction of the width W of the central region corresponds to the width direction of the inspection target line in the captured image (the direction perpendicular to the longitudinal direction).

[0047] The filter coefficients assigned to each cell of the specific width emphasis filter 6a are designed so that the central region 61a and the outer region 62a have different signs and the sum of the filter coefficients is "0". For example, when the inspection target line is displayed in the captured image with a pixel value brighter than the surrounding background, the filter coefficient in the central region 61a may be set with a "positive" sign, and the filter coefficient in the outer region 62a may be set with a "negative" sign, as shown in FIG. 10. Conversely, when the inspection target line is displayed in the captured image with a pixel value darker than the surrounding background, the filter coefficient in the central region 61a may be set with a "negative" sign, and the filter coefficient in the outer region 62a may be set with a "positive" sign. In FIG. 10, the weight of the filter coefficient is set to "+1" or "-1" as an example, but the weight different from that in FIG. 10 may be set to the filter coefficient as long as the sum of the filter coefficients is "0". The two outer regions 62a can be said to be regions provided to cancel out the filter coefficients of the central region 61a corresponding to the width of the specific width line (i.e., the assumed width W of the line to be inspected) and make the sum of the filter coefficients "0".

[0048] FIG. 11 is a diagram for explaining spatial filtering using the specific width emphasis filter 6a. When the specific width emphasis filter 6a is used, a convolution operation is performed on each pixel of an input image, and the resulting output value (product-sum operation result) is normalized to generate an output image (filtered image). In the input image of FIG. 11, white pixels represent the inspection target line, and the direction of the arrow is the longitudinal direction of the inspection target line. When a convolution operation is performed on such an input image using the specific width emphasis filter 6a, a filter processed image is generated in which the center of the inspection target line (specific width line) shows a higher intensity value than the surroundings (background, non-inspection target object, etc.), as shown in the output image of FIG. 11. In normal edge detection, as described above, edge parts such as the contour line of an object are extracted, whereas in spatial filtering using the specific width emphasis filter 6a, a processed image is obtained in which the center of the inspection target line (specific width line) is emphasized rather than the edges.

[0049] In the above description, the central region 61a has a width (W) corresponding to the assumed width W. However, the "width corresponding to the assumed width W" is not limited to a width that completely matches the assumed width W, and means that there may be some error depending on the value of the assumed width W. For example, the width (assumed width W) of the power line shown in the captured image is assumed to be at least 10 pixels, and an error of ±1 pixel is allowed, and the wider the power line, the larger the allowable error. The size of the specific width emphasis filter 6a and the width of each region are integers, and from the viewpoint of setting the width of the central region 61a to an even number divisible by 2, when the width W0 (actual measurement value) of the inspection target line measured in the test image is an odd number, the assumed width W may be set to "W0±1 pixel" and the specific width emphasis filter 6a as shown in FIG. 10 may be designed.

[0050] Moreover, the specific width emphasis filter 6a is not limited to the example shown in FIG. 10, and may have a configuration as shown in FIG. 12, for example. FIG. 12(a) is a modified example of the specific width emphasis filter 6a assuming that the expected width W is an odd number. In the specific width emphasis filter 6a in FIG. 12(a), the width of the central region 61a is set to "expected width W (odd number)", and the width of the outer region 62a is set to "(W-1) / 2". In this case, since W×(+1)+((W-1) / 2)×(-1) does not become 0, for example, the filter coefficient of any one row of each outer region 62a may be set to "-1.5", or an auxiliary row 63 may be inserted as shown in FIG. 12(a) to adjust the sum of the filter coefficients to "0". The total width of the specific width emphasis filter 6a in FIG. 12(a) is 2W+1. 12(a), the auxiliary row 63 having a filter coefficient of "-0.5" is inserted at the boundary between the central region 61a and the outer region 62a, but the insertion location of the auxiliary row 63 is not necessarily limited. Also, when the expected width W is an odd number, the width of the outer region 62a may be set to "(W+1) / 2" and the auxiliary row 63 having a positive sign may be inserted (in this case, the total width of the specific width emphasis filter 6a is 2W+3).

[0051] FIG. 12(b) shows a modified example of the specific width emphasis filter 6a assuming that the expected width W is an even number. In the specific width emphasis filter 6a of FIG. 12(b), the width of the outer region 62a is set to "expected width W (even number) / 2" and the width of the central region 61a is set to "W+1". A row with a filter coefficient of "0" is added to the central row in the width direction of the central region 61a. When the total width of the specific width emphasis filter 6a is an even number, the pixel of interest during the convolution operation is shifted by one pixel from the center of the filter, but as shown in FIG. 12(b), by inserting a row with a filter coefficient of 0 in the center of the central region 61a, it is possible to place the pixel of interest at the center of the filter and perform convolution operation that evenly uses the surrounding pixel information.

[0052] The filtering processing unit 121 generates a filter processed image in which a specific width line corresponding to the width of the inspection target line is emphasized (particularly, the center of the specific width line in the width direction is emphasized) by the spatial filtering processing based on the specific width emphasis filter 6a as described above. When a plurality of power lines with different thicknesses are set as the inspection target, a plurality of specific width emphasis filters 6a corresponding to each thickness may be prepared. The spatial filtering processing "based on" the specific width emphasis filter 6a means that it is not limited to the convolution operation processing by only the specific width emphasis filter 6a, but other sub-filters may be used in the spatial filtering processing together with the specific width emphasis filter 6a. Examples of the sub-filters used by the filtering processing unit 121 include a different diameter line extraction filter (second filter), a one-side edge filter (third filter), and a reflection countermeasure filter (fourth filter). It is preferable that the filtering processing unit 121 executes the spatial filtering processing based on one or more filters selected from the different diameter line extraction filter, the one-side edge filter, and the reflection countermeasure filter and the above-mentioned specific width emphasis filter 6a, and it is more preferable to execute the spatial filtering processing using all four types of filters. By using the sub-filters, it is possible to suppress the generation of noise in the filtered image and to further improve the accuracy of extracting the inspection target line. Each sub-filter will be described in detail below.

[0053] The different diameter wire extraction filter is used to extract wires with a width W different from the expected width W of the inspection target wire. DThe specific width emphasis filter 6a has selectivity for a specific width line corresponding to the inspection target line, and although the specific width line can be emphasized by the convolution operation by the specific width emphasis filter 6a, the specific width emphasis filter 6a may also react to a line having a thickness different from that of the specific width line (for example, a background power line that is not an inspection target). If such a line of a different width remains as noise in the filter processed image, it may cause a false detection, so it is preferable to remove noise due to a different diameter line having a width that is easily erroneously detected from the filter processed image. Therefore, it is preferable to apply a different diameter line extraction filter corresponding to a different diameter line that is easily erroneously detected to the spatial filtering process. This different diameter line extraction filter may be designed by performing a convolution operation by the specific width emphasis filter 6a on a test image captured by a test run, and identifying a line segment having a width that is easily erroneously detected. Examples of the line segment that is easily detected include a line segment having a relatively large number of widths among multiple noises (straight lines other than the inspection target line) in the filter processed image generated from the test image by the specific width emphasis filter 6a, a line segment having the most continuity, and the like. In other words, the width W different from the expected width W D means a width that is easily erroneously detected, and when there are a plurality of widths that are easily erroneously detected, a plurality of different diameter line extraction filters corresponding to the respective widths may be prepared.

[0054] Specifically, the different diameter line extraction filter has the same size as the specific width emphasis filter 6a (i.e., the total width and length L are the same as those of the specific width emphasis filter 6a), but the central region has a width W different from the expected width W. D For example, the filter 6b shown in FIG. 13 is an example of a different diameter line extraction filter. The different diameter line extraction filter 6b in FIG. 13 has a width W D The different diameter line extraction filter 6b in FIG. 13 has the same total width (2W) and length L as the specific width emphasis filter 6a, but the central region 61b has a width W different from the expected width W. D In the different diameter line extraction filter 6b, the width W of the central region 61b is set to correspond to (=W / 2). DThe width of the outer region 62a is set to be narrower than the width W of the specific width emphasis filter 6a, while the width of the outer region 62a is set to be "(3 / 4)W" wider than the specific width emphasis filter 6a. In the different diameter line extraction filter 6b, the sum of the filter coefficients is also set to "0", so the filter coefficients may be weighted based on the number of cells (number of pixels) in each region. D is set to half the expected width, the total number of cells in the outer region 62a is three times that of the central region 61b, so the filter coefficient of the outer region 62a is set to “−1 / 3” and the filter coefficient of the central region 61b is set to “+1”, which is three times that of the outer region 62a.

[0055] The different diameter line extraction filter 6b shown in FIG. 13 is merely an example. D may be appropriately set according to the results of the preliminary test, and the filter coefficients may be weighted in a manner other than the above as long as the sum of the filter coefficients is "0". In the convolution calculation process by the different diameter line extraction filter 6b for the captured image, the specific width line hardly reacts, but the set width W D Therefore, by subtracting the output value of the convolution operation by the specific width emphasis filter 6a from the output value of the convolution operation by the specific width emphasis filter 6a, the different diameter line having the width W is outputted as a high intensity value (a pixel value higher than that of the specific width line). D This eliminates the need for wires with different diameters, thereby suppressing the generation of noise.

[0056] For example, assume that spatial filtering processing based on the specific width emphasis filter 6a and the different diameter line extraction filter 6b is performed on a photographed image in which a background power line 71, which is not an inspection target, is reflected as shown in Fig. 17. Even if the background power line 71 is displayed as noise in an output image acquired by a convolution operation processing using the specific width emphasis filter 6a as shown in Fig. 18, by subtracting an output value obtained by performing a convolution operation processing using the different diameter line extraction filter 6b on the photographed image in Fig. 17 from the output image in Fig. 18, the intensity value of the background power line 71 (different diameter line) in the filtered image can be weakened as shown in Fig. 19, and the noise due to the background power line 71 can be removed.

[0057] The one-sided edge filter is a filter that reacts to only one of the two edges of the specific width line. As described above, the power lines included in the captured image usually have two edges per line. In this way, an object having a predetermined area included in the image has at least two contour lines, but the specific width emphasis filter 6a may react to one edge (contour line) of a non-inspection object that does not have a specific width (assumed width W), and such a reaction may appear as noise in the filter-processed image. For example, in the processed images of Figs. 18 and 19, a part of the edge of the building 72, which is a non-inspection object included in the captured image of Fig. 17, is detected as noise. In addition, in the convolution operation by the specific width emphasis filter 6a, although a high intensity value is obtained near the center of the specific width line, a reaction may also occur around the specific width line (an output value may occur outside the edge), and noise may occur around the specific width line in the filter-processed image. From the viewpoint of further emphasizing the specific width line in the filter-processed image, it is preferable to suppress such noise. By using a one-sided edge filter as a sub-filter, it is possible to remove noise based on the edge of a portion of the object not to be inspected as described above, and noise occurring around a line of a particular width.

[0058] A power line in a captured image has an edge where the background switches to the power line and an edge where the power line switches to the background in a predetermined direction (a direction intersecting the longitudinal direction), and when a luminance distribution (distribution of pixel values) is taken in the predetermined direction, these two edges can be recognized as a rising edge where the luminance value switches from a low value to a high value and a falling edge where the luminance value switches from a high value to a low value. A one-sided edge filter that responds only to edges on one side means a filter that corresponds to either a rising edge or a falling edge, and one-sided edge filters include rising edge filters and falling edge filters.

[0059] Specifically, the one-sided edge filter has the same size as the specific width emphasis filter 6a (that is, the total width and length L are the same as those of the specific width emphasis filter 6a), but has only one boundary where the sign of the "positive" or "negative" changes in the width direction. For example, the filter 6c (6c1 and 6c2) shown in FIG. 14 is an example of a one-sided edge filter. The specific width emphasis filter 6a described above has two outer regions 62a on the outside of the central region 61a in the width direction, and has two boundaries where the sign of the filter coefficient changes in the width direction. On the other hand, the one-sided edge filter 6c, as shown in FIG. 14, has an outer region 62c whose sign is different from that of the central region 61c only on one end side in the width direction, and is composed of one central region 61c and one outer region 62c. In other words, one of the regions that is the outer region 62a in the specific width emphasis filter 6a is replaced by the central region 61c in the one-sided edge filter 6c. One outer region 62a in the one-side edge filter 6c has the same width (W / 2) as one outer region 62a in the specific width emphasis filter 6a, but the central region 61c in the one-side edge filter 6c has a width equal to the sum of the width of the central region 61a in the specific width emphasis filter 6a and the width of one outer region 62a (i.e., a width ((3 / 2)W) equivalent to 1.5 times the assumed width W). In the one-side edge filters 6c1 and 6c2, as in the other filters, the filter coefficients of each cell are weighted so that the sum of the filter coefficients is "0". FIG. 14 shows an example in which the filter coefficient in the central region 61c having three times the number of cells (pixels) of the outer region 62c is set to "1 / 3" and the filter coefficient in the outer region 62c is set to "-1".

[0060] 14 illustrates an example in which the central region 61c has a positive filter coefficient and the outer region 62c has a negative filter coefficient (i.e., an example is illustrated assuming that the inspection target line has a pixel value brighter than the background). In this case, in the one-sided edge filter 6c1 shown on the left side of FIG. 14, the boundary between the central region 61c and the outer region 62c corresponds to the rising edge of the specific width line (inspection target line), and the left one-sided edge filter 6c1 corresponds to an example of a rising edge filter. On the other hand, in the one-sided edge filter 6c2 shown on the right side of FIG. 14, the boundary between the central region 61c and the outer region 62c corresponds to the falling edge of the specific width line, and the right one-sided edge filter 6c2 corresponds to an example of a falling edge filter. When the one-sided edge filter 6c is used as a sub-filter in the spatial filtering process, only one of the rising edge filter 6c1 and the falling edge filter 6c2 may be used, and it is preferable to use both.

[0061] When a convolution operation using the one-sided edge filter 6c is performed on a captured image, the operation processing result has output values ​​at positions where noise based on some edges of non-inspection objects (for example, edges of buildings 72 as shown in Figs. 18 and 19) and noise on the periphery of specific width lines may occur. Therefore, by subtracting the convolution operation result using the one-sided edge filter 6c from the operation result based on the specific width emphasis filter 6a as shown in Figs. 18 and 19 (output value of the filtered image), the above-mentioned noise can be removed from the filtered image as shown in Fig. 20 (Fig. 20 illustrates an example in which noise based on the edges of the buildings 72 has been removed). In other words, by using the one-sided edge filter 6c as a sub-filter, the specific width line can be further emphasized in the filtered image.

[0062] When subtracting the calculation result by the one-side edge filter 6c from the calculation result based on the specific width emphasis filter 6a, the reaction (output value) by the one-side edge filter 6c may be entirely subtracted, or a predetermined percentage of the reaction (output value) by the one-side edge filter 6c may be subtracted. For example, 50% of the output value of the one-side edge filter 6c may be subtracted from the calculation result based on the specific width emphasis filter 6a. The percentage to be subtracted is not particularly limited, and may be appropriately determined according to the results of a test run carried out in advance.

[0063] The anti-reflection filter is a filter having a filter coefficient weighted based on the luminance distribution of the inspection target line acquired from the test image. In the photographed image of the inspection target line, the inspection target line may reflect sunlight, causing a luminance distribution (distribution of pixel values) in the inspection target line shown in the photographed image. For example, FIG. 15 is a diagram for explaining the luminance distribution of the inspection target line in the photographed image. FIG. 15(a) is a diagram showing a simulated example of the inspection target line in the photographed image, and as shown in the figure, the inspection target line may have a shading of pixel values ​​based on light reflection, etc. due to light reflection, etc. The graph shown in FIG. 15(b) is an example of a luminance distribution along the direction of the arrow in FIG. 15(a), and the horizontal axis of the graph represents the position on the arrow, and the vertical axis represents the luminance value (pixel value) at each position. When the inspection target line in the photographed image has a luminance distribution in this way, the width of the inspection target line on the image may be detected as having a width different from the expected width W depending on the degree of the distribution. By adding the response of an anti-reflection filter, which will be described in detail below, to the result of the convolution calculation process based on the specific width emphasis filter 6a, it is possible to suppress erroneous detection (noise) due to luminance distribution such as light reflection.

[0064] In the anti-reflection filter, the overall size (total width and length L) and the size of each region (the width of the central region and the width of the outer region) are the same as those of the specific width emphasis filter 6a, but the anti-reflection filter has filter coefficients different from those of the specific width emphasis filter 6a. For example, a filter 6d shown in FIG. 16 is an example of an anti-reflection filter. The anti-reflection filter 6d in FIG. 16 has the same size as the specific width emphasis filter 6a, and has a central region 61d and an outer region 62d that are the same sizes as the central region 61a and the outer region 62a in the specific width emphasis filter 6a. In the anti-reflection filter 6d, weighting is set for the filter coefficients according to the luminance distribution of the inspection target line so that the distribution of the filter coefficients in the width direction (Y-axis direction) has a shape similar to the luminance distribution of the inspection target line as shown in FIG. 15(b). When setting the filter coefficients of the anti-reflection filter 6d, for example, an integer corresponding to the luminance distribution of the inspection target line may be assigned to each cell, and the filter coefficients of each cell may be set by normalizing the coefficients so that the average value of the filter coefficients is "0" (i.e., the sum of the filter coefficients is "0") and the sum of the absolute values ​​of the filter coefficients is "1." The anti-reflection filter 6d in Fig. 16 illustrates an example of filter coefficients that are normalized to have an average value of 0 and a sum of absolute values ​​of 1, with the total number N (N = 2W x L) of cells in the filter as the denominator of the filter coefficients.

[0065] The luminance distribution on the image of the inspection target line may be obtained by analyzing a test image acquired during a test run. In this case, the test run may be performed multiple times to acquire multiple test images with different sunlight conditions, and then a luminance distribution that occurs relatively frequently may be identified, and the anti-reflection filter 6d may be designed based on the luminance distribution. Alternatively, multiple luminance distributions with different sunlight conditions may be identified, and then multiple anti-reflection filters 6d corresponding to each luminance distribution may be prepared. In this case, the filtering processing unit 121 may acquire environmental information such as weather and sunlight conditions at the time of inspection (when photographing is performed), and may select an anti-reflection filter 6d generated under conditions closest to the conditions at the time of inspection from among the multiple anti-reflection filters 6d prepared (anti-reflection filters 6d stored in the filter storage unit 132 described later), and use the selected anti-reflection filter 6d for spatial filtering processing. Alternatively, the user may be allowed to select the anti-reflection filter 6d to be used during inspection from among the multiple anti-reflection filters 6d at his / her discretion, or multiple anti-reflection filters 6d may be used for spatial filtering processing at the time of inspection.

[0066] Since the anti-reflection filter 6d has filter coefficients adapted to the reflection of the line under inspection, the result of the convolution calculation process of the anti-reflection filter 6d on the captured image is added to the result of the convolution calculation process based on the specific width emphasis filter 6a, so that the specific width line can be further emphasized in the filter-processed image.

[0067] As described above, the filtering processing unit 121 may generate a filtered image by a convolution operation using only the specific width emphasis filter 6a, which is the main filter, or may generate a filtered image by a convolution operation using one or more types of sub-filters and the specific width emphasis filter 6a (it is preferable to use one or more types of sub-filters, and it is more preferable to use three types of sub-filters (the different radial line extraction filter 6b, the one-sided edge filter 6c, and the anti-reflection filter 6d)).

[0068] In the above description of each sub-filter (6b to 6d), a method was exemplified in which a convolution operation by each filter is individually performed on a captured image, and each operation processing result is summed (subtracted or added). However, since the law of distribution holds between an input image and a plurality of filters, a composite filter may be generated by combining a plurality of filters to be applied to a spatial filtering process in advance, and the filtering processing unit 121 may generate a filtered image by a single spatial filtering process (convolution operation) using the composite filter. In other words, the spatial filtering process "based on" the specific width emphasis filter 6a and one or more sub-filters means that a filtered image may be generated from individual convolution operation results by a plurality of filters, or a filtered image may be generated by a spatial filtering process using a composite filter integrated in advance. From the viewpoint of increasing the calculation processing speed, the latter method using a composite filter is preferable.

[0069] The composite filter is generated by combining the specific width emphasis filter 6a, which is a main filter, with one or more sub-filters, and the filter generation unit 123 shown in FIG. 6 executes the process of generating the composite filter.

[0070] Here, "combining" a plurality of filters means adding and subtracting filter coefficients for each cell according to the distributive law. As described above, the specific width emphasis filter 6a and the three types of sub-filters all have the same size, so it is sufficient to add and subtract filter coefficients for each corresponding cell in each filter. For example, when combining these four types of filters as coefficient a1 of a predetermined cell C1 in the specific width emphasis filter 6a, coefficient b1 of a predetermined cell C1 in the different radial line extraction filter 6b, coefficient c1 of a predetermined cell C1 in the one-sided edge filter 6c, and coefficient d1 of a predetermined cell C1 in the anti-reflection filter 6d, the filter generation unit 123 may determine the filter coefficient of the predetermined cell C1 of the composite filter by the formula "a1-b1-c1+d1". Note that, as described in the explanation of the one-sided edge filter 6c, when the final filter-processed image is generated by multiplying the output of the sub-filter by a predetermined ratio, the filter generation unit 123 may determine the filter coefficient of the composite filter by multiplying the coefficient of the sub-filter in the above formula by a predetermined ratio to be reflected in the output value.

[0071] In the composite filter generation process by the filter generation unit 123, a filter combination is determined by a user's selection, and the filter generation unit 123 determines the filter coefficient for each cell according to the determination, and generates a composite filter. In the composite filter generation process, a plurality of composite filters may be generated according to the combination of sub-filters. For example, a first composite filter combining the specific width emphasis filter 6a and the different radial line extraction filter 6b, a second composite filter combining the specific width emphasis filter 6a and the one-side edge filter 6c, a third composite filter combining the specific width emphasis filter 6a and the anti-reflection filter 6d, and a fourth composite filter combining four types of filters may be generated in advance.

[0072] The filtering processing unit 121 may automatically specify a filter to be used in the spatial filtering process from among a plurality of prepared filters (for example, a specific width emphasis filter 6a, one or more sub-filters, and one or more composite filters) according to the shooting conditions and the like, and generate a filtered image. Also, a process of selecting a filter to be used in the spatial filtering process may be accepted from a user, and the filtering processing unit 121 may generate a filtered image by a spatial filtering process using a filter designated based on the user's selection. The information processing system of this embodiment may have specifications that allow both a mode in which a filter to be used in the spatial filtering process is automatically specified (auto mode) and a mode in which a filter is specified based on a user's selection instruction (manual mode). Also, the information processing system may have specifications that allow a user to support switching between the auto mode and the manual mode.

[0073] The specific straight line extraction unit 122 performs a Hough transform process on the filtered image generated by the filtering processing unit 121, and performs a process of extracting a specific width line corresponding to the line to be inspected from multiple straight lines contained in the filtered image, based on the straight line form information acquired by the Hough transform process.

[0074] Here, the Hough transform is a method for detecting straight lines included in an image by majority vote. When the Hough transform is applied to an image after conventional edge detection, for example, the length of a perpendicular line drawn from the origin to a certain straight line is ρ, and the angle between the perpendicular line and the x-axis is θ. The xy coordinate information of each point constituting an edge in the image is voted in the Hough space (ρθ coordinates) based on the formula "ρ=xcosθ+ysinθ", and line vote number information indicating the number of votes is obtained. Then, a combination of ρθ indicating a straight line is estimated by majority vote from the line vote number information. In such a conventional Hough transform that inputs an edge image, all straight lines included in the edge image are estimated without distinguishing between the inspection target line and other straight lines (straight lines due to the edges of non-inspection targets), so a process is required to further extract only the inspection target line from the estimated straight lines based on some condition (extraction condition). However, in an edge image (input image to Hough transform) generated by conventional edge detection, two edges are extracted from one line in a captured image, and many straight lines exist in the edge image, and the edges of non-inspection objects may be clearly displayed, making it difficult to extract only the edge of the inspection target line from the edge image. Even if an extraction condition for an edge corresponding to the inspection target line can be found, complex calculation processing is required to extract the inspection target line from the straight line vote number information regarding the estimated multiple straight lines. On the other hand, in the filter-processed image in this embodiment, the center area of ​​the inspection target line (specific width line) is emphasized instead of the edges, so that the straight line corresponding to the inspection target line can be extracted more easily than when a conventional edge image is input and Hough transform processing is performed.

[0075] In the specific line extraction unit 122, a general Hough transform method can be adopted, but it is preferable to execute a weighted Hough transform as the Hough transform process. The weighted Hough transform process is a method of generating weighted line vote number information based on the intensity value (the brightness value (pixel value) of each pixel after spatial filtering process) in the filter processed image. In the filter processed image generated by the filtering processing unit 121, unlike the conventional edge image, the specific width line is displayed with a higher intensity value than other lines corresponding to the background and non-inspection objects, so that the weighted line vote number information acquired by the weighted Hough transform process assigns a higher vote number to the specific width line than other lines. Therefore, based on such weighted line vote number information, it is possible to more easily extract the specific width line from multiple straight lines in the filter processed image.

[0076] When a plurality of power lines extending substantially parallel to one another (a plurality of power lines using a multi-conductor system) are to be the lines to be inspected, the processing unit 120 may include a slope determination unit 124. The slope determination unit 124 executes a process of determining, for example, which of the slopes of the detected plurality of straight lines is the first slope having the most votes based on the number of slope votes. In the determination method based on the number of slope votes, for example, the slope determination unit 124 may determine the first slope having the most votes based on the angle (slope votes) based on information about the angle (θ) included in the straight line vote number information acquired by a Hough transform process.

[0077] In the case where the inclination determination unit 124 is provided, an extraction condition is set that there are at least the expected number of straight lines whose inclination is within a predetermined range with respect to the determined first inclination (since one specific width line is highlighted for one inspection target line in the filter processed image, the number of inspection target lines corresponds to the expected number of straight lines). When the specific straight line extraction unit 122 determines that this extraction condition is satisfied, it may extract the expected number of straight lines from among the straight lines whose inclination is within a predetermined range with respect to the first inclination in the order of the number of straight line votes indicated by the straight line vote number information. For example, in the filter processed image shown in FIG. 18, when the inclination of any of the four straight lines indicated by the arrows is determined to be the first inclination, there are straight lines (other straight lines extending obliquely) whose inclination is within a predetermined range with respect to the first inclination, the number of which corresponds to the expected number of straight lines (four), so that the straight line corresponding to the inspection target line is extracted as shown in FIG. 20. In this way, the inclination determination process of the inclination determination unit 124 may allow the straight line (specific width line) corresponding to the inspection target line to be extracted with higher accuracy.

[0078] Moreover, the processing unit 120 may have a functional unit that removes straight lines that do not meet a predetermined length after the Hough transform process by the specific straight line extraction unit 122. As can be seen from the examples of the captured images shown in Fig. 7 and Fig. 17, the inspection target line in the captured image has a length that reaches from a short side of the image to another short side. Therefore, for example, an extraction condition may be set such that straight lines with a length less than 50% of the length in the X-axis direction of the captured image in Fig. 17 are removed from the filtered image, and the specific straight line extraction unit 122 may remove straight lines that do not meet the extraction condition from the multiple straight lines estimated from the straight line vote number information, and extract a specific width line that is equal to or longer than a predetermined length. Such a process may allow the straight line (specific width line) corresponding to the inspection target line to be extracted with higher accuracy.

[0079] The tracking target position setting unit 125 sets a target position to be tracked by the camera (photographing unit) of the unmanned aerial vehicle 4 based on the straight line (i.e., the specific width line corresponding to the inspection target line) extracted by the specific line extraction unit 122. As a more specific example, the tracking target position setting unit 125 may calculate an average straight line at the average position of the multiple straight lines extracted by the specific line extraction unit 122 (visualize and add to the image as necessary), and set the average straight line as the target position (target straight line) to be tracked. FIG. 21 illustrates an example of an image in which the average straight line 8 is visualized and added by a dashed dotted line. In addition, the tracking target position setting unit 125 may set one of the specific width lines extracted by the specific line extraction unit 122 as the target position (target straight line) to be tracked.

[0080] After setting the target position for tracking as described above, the tracking target position setting unit 125 generates instruction information for controlling at least one of the photographing unit or the unmanned aerial vehicle 4 itself so that the set target position is included in the image photographed by the photographing unit, and transmits the instruction information to the unmanned aerial vehicle 4 via the communication unit 110. As a more specific example, first, it is determined whether a predetermined position (e.g., a center position or an approximate center position, etc.) of the inspection target line is included in a predetermined range in the image (particularly, a range near the center of the image and further based on the center position or the approximate center position of the image). Then, if it is determined that it is not included in the predetermined range, difference information between the predetermined position and the predetermined range of the target line is generated, and instruction information for controlling at least one of the state of the photographing unit (e.g., photographing direction information, angle of view information, etc.) or the state of the unmanned aerial vehicle itself (e.g., current position information (latitude, longitude, altitude coordinates), attitude information, motor output information of the rotor blades, etc.) is generated based on the difference information. The difference information may be any information that allows the amount and direction of deviation from a predetermined range to be grasped, and may be, for example, information on the direction and distance (number of pixels) on the image from the center of the image to a predetermined position of the target line. The instruction information may be generated based on, for example, the difference information, the correspondence information between the direction and distance on the image and the control direction and control distance in the photographing unit or the unmanned aerial vehicle photographing unit, and at least one of the current state information of the unmanned aerial vehicle, but is not limited thereto. Note that, when it is determined that the predetermined position of the inspection target line is included in a predetermined range in the image, the state of the photographing unit or the unmanned aerial vehicle is maintained without newly generating instruction information.

[0081] 6 stores, in addition to the image acquired by the image acquisition unit 115, a filtered image generated by the filtering processing unit 121 and pixel value information in the filtered image, line vote number information generated by the specific line extraction unit 122 and information on the lines (specific width lines) constituting a plurality of power lines, tilt vote number information and information on the first tilt generated by the tilt determination unit 124, an image in which the target position of tracking is visualized by the tracking target position setting unit 125, and the like. The information / image storage unit 131 may also temporarily store other information and data generated in the processing by each of the functional units 121 to 125 of the processing unit 120.

[0082] The filter storage unit 132 of the storage unit 130 stores information on various filters used for spatial filtering by the filtering processing unit 121, such as the specific width emphasis filter 6a, the different radial line extraction filter 6b, the one-sided edge filter 6c, the anti-reflection filter 6d, and the composite filter generated by the filter generation unit 123. The information on the filters stored in the filter storage unit 132 includes information for identifying the filters themselves as shown in Figs. 10, 12 to 14, and 16, as well as information on the generation conditions of the filters linked to the identification IDs of the respective filters (for example, a test image used when generating the filter, an expected width W obtained from the test image, the width W of different radial lines that are likely to be erroneously detected, and the like). D , the luminance distribution of the line under inspection, and the shooting conditions of the test image used when generating the filter) may be stored.

[0083] <An example of how to extract a straight line> Next, a straight line extraction method by the information processing system according to the present embodiment will be described with reference to Fig. 22 etc. Fig. 22 is a flowchart showing a process for implementing the straight line extraction method by the information processing system according to the present embodiment.

[0084] First, the image acquisition unit 115 of the terminal 1 acquires an image such as an image captured by the camera 42 mounted on the unmanned air vehicle 4 (S101).

[0085] Next, the filtering processing unit 121 of the terminal 1 performs spatial filtering processing on the captured image acquired by the image acquisition unit 115 based on the specific width emphasis filter 6a described above, and generates a filtered image in which lines having a specific width (estimated width W) corresponding to the inspection target line are emphasized from the captured image (S102). In this step S102, it is preferable to perform spatial filtering processing based on one or more sub-filters selected from the different diameter line extraction filter 6b, the one-sided edge filter 6c, and the anti-reflection filter 6d, and the specific width emphasis filter 6a, which is the main filter, and it is more preferable to perform spatial filtering processing based on the above three sub-filters and the specific width emphasis filter 6a. In the case of spatial filtering processing using one or more sub-filters, each filter may be individually processed on the captured image, or a convolution calculation may be performed using a composite filter that combines a plurality of the above filters, and the latter method is more preferable.

[0086] Next, the specific line extraction unit 122 of the terminal 1 executes a Hough transform process on the filtered image, and extracts a line (specific width line) corresponding to the inspection object from the multiple lines in the filtered image based on the line vote number information acquired by the Hough transform process (S103). In the Hough transform process of step S103, it is preferable to execute a weighted Hough transform, and it is preferable to extract a specific width line from the multiple lines in the filtered image based on the line vote number information acquired by the weighted Hough transform and weighted by the intensity value in the filtered image.

[0087] Thus, according to the terminal 1 of the present embodiment, even if a straight-line non-inspection object such as a building, road, or other long object that is not the inspection object is reflected in the captured image, a straight line corresponding to the target inspection object can be extracted from the captured image. Furthermore, in the above information processing system, it is possible to extract a straight line corresponding to the inspection object by a calculation process that is simpler than that of Hough transform processing based on a conventional edge detection technology (a technology for extracting two edges for one power line) and deep learning, and a general-purpose calculation processing device that is less expensive than the calculation processing device required for a segmentation method using deep learning can be used.

[0088] The above-described embodiment is merely an example for facilitating understanding of the present disclosure, and is not intended to limit the present disclosure. The present disclosure can be modified and improved without departing from the spirit thereof, and it goes without saying that the present disclosure includes equivalents thereof. In the embodiment, a case where a power line is the inspection object is exemplified, but the information processing system etc. of the present disclosure can also be applied to a system in which piping in a factory is tracked and inspected by an unmanned mobile body, and the extraction object in the present disclosure is not necessarily limited to a power line (preferably applied to a power line inspection system). [Explanation of symbols]

[0089] 1 Device 2 Server 4 Unmanned Aerial Vehicles

Claims

1. An information processing method in which a computer executes information processing to extract a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: A process of generating a filtered image in which lines of a specific width corresponding to the object to be inspected are emphasized from the photographed image by a spatial filtering process based on a first filter having selectivity for lines having a specific width corresponding to the object to be inspected and a second filter having selectivity for lines of different diameters having a width different from the specific width; and a process of extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by a Hough transform process on the generated filtered image.

2. An information processing method in which a computer executes information processing to extract a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: a process of generating a filtered image in which the line of the specific width corresponding to the object to be inspected is emphasized from the captured image by spatial filtering based on a first filter having selectivity for the line of the specific width corresponding to the object to be inspected and a third filter that reacts only to one of two edges of the line of the specific width; and a process of extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by a Hough transform process on the generated filtered image.

3. the third filter includes a filter that reacts only to one of the two edges of the line of the specific width, and a filter that reacts only to the other of the two edges; The information processing method according to claim 2 , wherein the filtered image is generated by spatial filtering based on the first filter, the third filter that is sensitive to only one of the two edges, and the third filter that is sensitive to only the other of the two edges.

4. An information processing method in which a computer executes information processing to extract a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: a process of generating a filtered image in which lines of a specific width corresponding to the object to be inspected are emphasized from the captured image by spatial filtering based on a first filter having selectivity for lines of the specific width corresponding to the object to be inspected and a fourth filter having a filter coefficient weighted based on a luminance distribution of the object to be inspected in a test image; and a process of extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by a Hough transform process on the generated filtered image.

5. The method further includes, before the spatial filtering process, generating a composite filter by combining at least one other filter and the first filter; The other filter is a second filter having selectivity for different diameter wires having widths different from the specified width; a third filter that responds to only one of the two edges of the line of the specific width; and a fourth filter having a filter coefficient weighted based on a luminance distribution of the inspection object in a test image; An information processing method according to any one of claims 1 to 4, wherein in the process of generating the filtered image, the spatial filtering process is performed by a convolution operation using the composite filter, and the filtered image in which the lines of the specific width are emphasized is generated.

6. The information processing method according to claim 5 , wherein the composite filter is generated by combining the first filter, the second filter, the third filter, and the fourth filter.

7. The Hough transform process is a weighted Hough transform, The information processing method according to any one of claims 1 to 4, further comprising extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by the weighted Hough transform and weighted by an intensity value in the filtered image.

8. An information processing system for extracting a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: a filtering processing unit that generates a filtered image in which lines of the specific width corresponding to the object to be inspected are emphasized from the captured image by spatial filtering based on a first filter having selectivity for lines having a specific width corresponding to the object to be inspected and a second filter having selectivity for lines of different diameters having a width different from the specific width; and a specific line extraction unit that extracts a line corresponding to the object to be inspected from a plurality of lines in the filtered image based on line voting number information obtained by Hough transform processing of the generated filtered image.

9. A program for causing a computer to execute information processing for extracting a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, A process of generating a filtered image in which lines of a specific width corresponding to the object to be inspected are emphasized from the photographed image by a spatial filtering process based on a first filter having selectivity for lines having a specific width corresponding to the object to be inspected and a second filter having selectivity for lines of different diameters having a width different from the specific width; and extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by Hough transform processing of the generated filtered image.

10. An information processing system for extracting a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: a filtering processing unit that generates a filtered image in which the line of the specific width is emphasized from the captured image by spatial filtering based on a first filter having selectivity for a line having a specific width corresponding to the object to be inspected and a third filter that reacts only to one of two edges of the line of the specific width; and and a specific line extraction unit that extracts a line corresponding to the object to be inspected from a plurality of lines in the filtered image based on line voting number information obtained by Hough transform processing of the generated filtered image.

11. A program for causing a computer to execute information processing for extracting a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: a process of generating a filtered image in which the line of the specific width corresponding to the object to be inspected is emphasized from the captured image by spatial filtering based on a first filter having selectivity for the line of the specific width corresponding to the object to be inspected and a third filter that reacts only to one of two edges of the line of the specific width; and extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by Hough transform processing of the generated filtered image.

12. An information processing system for extracting a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: a filtering processing unit that generates a filtered image in which lines of the specific width corresponding to the object to be inspected are emphasized from the captured image by spatial filtering based on a first filter having selectivity for lines of the specific width corresponding to the object to be inspected and a fourth filter having a filter coefficient weighted based on a luminance distribution of the object to be inspected in a test image; and and a specific line extraction unit that extracts a line corresponding to the object to be inspected from a plurality of lines in the filtered image based on line voting number information obtained by Hough transform processing of the generated filtered image.

13. A program for causing a computer to execute information processing for extracting a straight line corresponding to a long-sized inspection object from an image captured by an unmanned moving body, comprising: a process of generating a filtered image in which lines of a specific width corresponding to the object to be inspected are emphasized from the captured image by spatial filtering based on a first filter having selectivity for lines of the specific width corresponding to the object to be inspected and a fourth filter having a filter coefficient weighted based on a luminance distribution of the object to be inspected in a test image; and extracting a straight line corresponding to the object to be inspected from a plurality of straight lines in the filtered image based on straight line voting number information obtained by Hough transform processing of the generated filtered image.

Images