Image processing device, image processing method, and image processing program

Abstract

Provided are an image processing device, an image processing method, and an image processing program capable of comprehensively evaluating damage of a structure in an individual image captured by dividing the structure into a plurality of divided regions or a composite image of individual images. An image processing method using an image processing device (1) provided with a processor (10) includes: a step in which the processor acquires the pixel positions of at least two points in a plurality of individual images in which a subject is captured or a composite image synthesized from the individual images and the actual length between the two points; and a step in which the processor performs a process of resizing the individual image or the composite image so that the number of pixels between the two points approaches a desired pixel spacing.

Description

Image processing device, image processing method, and image processing program

[0001] The present invention relates to an image processing device, an image processing method, and an image processing program, and more particularly to a technique for inspecting a structure using a plurality of individual images obtained by dividing and capturing the structure as a subject, or a composite image of the individual images.

[0002] Structures such as bridges, roads, tunnels, and dams are constructed as the foundations of industry and life, and play an important role in supporting people's comfortable lives. These structures are constructed using, for example, concrete or steel frames, but because they are used by people for a long period of time, they deteriorate over time. Therefore, it is necessary to inspect these structures at appropriate times to detect areas of damage and deterioration, and to carry out appropriate maintenance, such as replacing or repairing components.

[0003] Inspection of such structures is performed by detecting damage from images of the structures, but it is difficult to capture the entire structure in a single image for large structures such as bridges, roads, tunnels, and dams.

[0004] Therefore, an image processing method has been proposed in which a structure is divided into multiple divided areas, images are captured, and the images of the multiple divided areas are linked or combined to create a single image that includes the entire structure or an area that is wider than the divided areas (see, for example, Patent Document 1).

[0005] Japanese Patent Application Laid-Open No. 2004-021578

[0006] However, the pixel spacing (mm / pixel) of images captured of a divided region (hereinafter referred to as individual images) varies due to various factors (e.g., variations in imaging distance, focal length, imaging angle, etc.). The variation in pixel spacing between individual images can cause distortion in a composite image obtained by combining the individual images. Furthermore, the variation in pixel spacing between individual images makes it difficult to comprehensively evaluate damage contained in multiple individual images or damage present across multiple individual images by comparing them.

[0007] One embodiment of the technology disclosed herein provides an image processing device, an image processing method, and an image processing program that are capable of comprehensively evaluating damage to a structure in individual images captured by dividing the structure into multiple divided areas, or in a composite image of the individual images.

[0008] An image processing device according to a first aspect of the present invention is an image processing device equipped with a processor, which acquires pixel positions of at least two points of a plurality of individual images of a subject or a composite image obtained by combining the individual images, and the actual length between the two points, and resizes the individual images or composite image so that the number of pixels between the two points approaches the desired pixel spacing.

[0009] In the image processing device according to a second aspect of the present invention, the two points of the first aspect are points on the boundary between different surfaces that make up the subject.

[0010] In the image processing device according to a third aspect of the present invention, in the first aspect, the processor specifies two points in accordance with an instruction input from the user.

[0011] According to a fourth aspect of the present invention, in the image processing device of any one of the first to third aspects, the processor acquires an actual length between two points from drawing data of the subject.

[0012] In the image processing device according to a fifth aspect of the present invention, in any one of the first to third aspects, the processor obtains the actual length between the two points from information indicating the shape and dimensions of the subject.

[0013] An image processing device according to a sixth aspect of the present invention is an image processing device according to any one of the first to fifth aspects, in which a processor divides a plurality of individual images into a plurality of image sets, combines the plurality of image sets to create a plurality of composite images, and resizes the plurality of composite images so that the pixel spacing of the plurality of composite images approaches a desired pixel spacing.

[0014] According to a seventh aspect of the present invention, in the image processing device of any one of the first to sixth aspects, the processor causes the display unit to display a plurality of composite images side by side.

[0015] An eighth aspect of the present invention provides an image processing device according to any one of the first to seventh aspects, wherein the processor detects damage to the subject from the individual images or the composite image, and depicts the damage detection result together with the composite image.

[0016] According to a ninth aspect of the present invention, in the image processing device of the eighth aspect, the processor causes the display unit to switchably display a composite image and a composite image depicting a damage detection result.

[0017] A tenth aspect of the present invention provides the image processing device of any one of the first to ninth aspects, wherein the processor specifies pixel spacing of the individual images or composite image after resizing in accordance with an instruction input from the user.

[0018] An eleventh aspect of the present invention is an image processing device according to any one of the first to ninth aspects, wherein the processor specifies the pixel spacing of the individual images or the composite image after resizing based on information regarding the pixel spacing of the individual images.

[0019] An image processing device according to a twelfth aspect of the present invention is an image processing device according to the eleventh aspect, wherein the processor specifies the pixel spacing of the composite image after resizing based on at least one of the representative value, average value, maximum value, and minimum value of the pixel spacing of the individual images.

[0020] According to a thirteenth aspect of the present invention, in the image processing device of any one of the first to twelfth aspects, the processor outputs a warning when the resizing magnification in the resizing process is outside an allowable range.

[0021] According to a fourteenth aspect of the present invention, in the image processing device of any one of the first to thirteenth aspects, the processor outputs a warning when there is an individual image in which the pixel spacing is larger than the desired pixel spacing.

[0022] An image processing device according to a fifteenth aspect of the present invention is any of the first to fourteenth aspects, wherein the individual images are images captured of at least a portion of the top, side, and bottom of the tunnel as the subject.

[0023] An image processing method according to a sixteenth aspect of the present invention is an image processing method using an image processing device equipped with a processor, and includes a step in which the processor acquires pixel positions of at least two points of a plurality of individual images of a subject or a composite image obtained by combining the individual images, and the actual length between the two points, and a step in which the processor resizes the individual images or the composite image so that the number of pixels between the two points approaches a desired pixel spacing.

[0024] The image processing program according to the seventeenth aspect of the present invention enables a computer to obtain the pixel positions of at least two points of a plurality of individual images of a subject or a composite image formed by combining the individual images, and the actual length between the two points, and to perform a resizing process on the individual images or composite image so that the number of pixels between the two points approaches the desired pixel spacing.

[0025] 1 is a block diagram showing an image processing device according to an embodiment of the present invention; FIG. 2 is a front view showing the appearance of an imaging device; FIG. 3 is a front view showing an example of a structure (tunnel); FIG. 4 is a block diagram of an imaging device; FIG. 5 is a view for explaining an image processing function of an image processing device; FIG. 6 is a view for explaining the image processing function of an image processing device; FIG. 7 is a view for explaining the image processing function of an image processing device; FIG. 8 is a view for explaining the image processing function of an image processing device; FIG. 9 is a view showing an example of a display of a composite image (composite developed image); FIG. 10 is a flowchart showing an image processing method; FIG. 11 is a flowchart showing a pixel spacing setting step (first example); FIG. 12 is a flowchart showing a pixel spacing setting step (second example); FIG. 13 is a flowchart showing an image output step;

[0026] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image processing device, an image processing method, and an image processing program according to the present invention will be described below with reference to the accompanying drawings.

[0027] [Image Processing Apparatus] FIG. 1 is a block diagram showing an image processing apparatus according to an embodiment of the present invention.

[0028] The image processing device 1 according to this embodiment is a device that acquires, from the imaging device 100, individual images P obtained by capturing images of multiple divided regions of a structure OBJ to be inspected, and performs image processing (e.g., resizing) on ​​the individual images P or a composite image obtained by combining multiple individual images P. This image processing makes it possible to provide the user with an image with pixel spacing suitable for inspecting (image diagnosis) damage to the structure OBJ, and to support a comprehensive evaluation of damage to the structure OBJ.

[0029] 1, the image processing device 1 according to this embodiment includes a processor 10, a memory 12, a storage 14, and a communication interface (communication I / F) 16. The image processing device 1 may be, for example, a personal computer, a general-purpose computer such as a workstation, or a tablet terminal.

[0030] The processor 10 is a device that controls the operation of each part of the image processing device 1, and includes, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 10 is capable of transmitting and receiving control signals and data to and from each part of the image processing device 1 via a bus. The processor 10 receives instruction input from a user via the operation unit 20, and transmits control signals corresponding to the instruction input to each part of the image processing device 1 via the bus to control the operation of each part.

[0031] The memory 12 includes a RAM (Random Access Memory) used as a working area for various calculations, or a VRAM (Video Random Access Memory) used as an area for temporarily storing image data output to the display unit 22.

[0032] The operation unit 20 is an input device that accepts instruction input from the user, and includes a keyboard for inputting characters and the like, and a pointing device (e.g., a mouse or a trackball) for operating a GUI (Graphical User Interface) such as a pointer and icons displayed on the display unit 22. Note that the operation unit 20 may be provided with a touch panel on the surface of the display unit 22 instead of or in addition to the keyboard and pointing device.

[0033] The display unit 22 is a device for displaying images. As the display unit 22, for example, a liquid crystal monitor can be used.

[0034] The storage 14 stores various data including control programs and image processing programs for various calculations, and individual images P (for example, visible light images or infrared images) of the structure OBJ to be inspected. As the storage 14, for example, a device including a magnetic disk such as an HDD (Hard Disk Drive) or a device including a flash memory such as an eMMC (embedded Multi Media Card) or an SSD (Solid State Drive) can be used.

[0035] The communication I / F 16 is a device for communicating with an external device including the imaging device 100 and the subject information DB 200. Data can be transmitted and received between the image processing device 1 and the external device via wired communication or wireless communication via a network (for example, a local area network (LAN), a wide area network (WAN), an internet connection, etc.).

[0036] The image processing device 1 is capable of receiving input of individual images P from the imaging device 100 via the communication I / F 16. The method for inputting the individual images P to the image processing device 1 is not limited to communication via a network. For example, a USB (Universal Serial Bus) cable, Bluetooth (registered trademark), or infrared communication may be used. Alternatively, the individual images P may be stored in a recording medium (e.g., a USB memory or an SD (registered trademark) memory card) that is detachable from the image processing device 1, and the image processing device 1 may receive input of the individual images P from the imaging device 100 via this recording medium.

[0037] [Imaging Device] FIG. 2 is a front view showing the appearance of the imaging device 100. As shown in FIG.

[0038] The imaging device 100 includes cameras 102A to 102E. The cameras 102A to 102E are devices that capture images of an inspection target structure OBJ using, for example, visible light or infrared light, and include imaging elements such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The number and arrangement of the cameras 102A to 102E can be changed depending on the type, structure, shape, and size of the inspection target structure OBJ.

[0039] In the following, an example will be described in which the inner circumferential surface of a tunnel shown in FIG. 3 is inspected as the structure OBJ to be inspected. The imaging device 100 is capable of capturing images of the inner circumferential surface of the tunnel while moving inside the tunnel in the depth direction. In the following, the movement direction (advance / retreat direction) of the imaging device 100 is defined as the Z direction, and the up / down direction and left / right direction of the imaging device 100 are defined as the X direction and the Y direction, respectively. Therefore, the X, Y, and Z directions correspond to the height direction, left / right direction, and depth direction of the tunnel, respectively.

[0040] When inspecting the inner circumferential surface of a tunnel, the cameras 102A to 102E are arranged, for example, in a curved line that corresponds to or follows the shape of the inner circumferential surface of the tunnel, as shown in Fig. 2. For the sake of simplicity, the following description will be given of an example in which five cameras 102A to 102E are mounted at equal intervals (45° intervals) on an arc equidistant from a reference position (center) O of the camera mounting member 104.

[0041] As shown by the dashed-dotted lines in FIG. 2 , the optical axes of cameras 102A-102E are arranged radially from the center O of camera mounting member 104, with each camera having a different imaging direction. That is, as shown in FIG. 2 , cameras 102A and 102E are arranged sideways (facing the -Y and +Y sides, respectively) and are capable of imaging a divided area including the left and right side surfaces of the tunnel, or a divided area including the left and right side surfaces and a portion of the bottom surface. Camera 102C is arranged facing directly upward (+X side) and is capable of imaging a divided area including the top surface of the tunnel, or a divided area including the top surface and a portion of the side surfaces. Cameras 102B and 102D are capable of imaging a divided area between the divided areas imaged by cameras 102A and 102C, and a divided area between the divided areas imaged by cameras 102E and 102C. It is preferable to position cameras 102A-102E so that adjacent images captured by cameras 102A-102E in the inner circumferential direction overlap each other. The individual images P captured by the cameras 102A to 102E arranged as described above can cover the entire inner circumference of the tunnel.

[0042] 2, the camera mounting member 104 is attached to a support on a dolly 106, and the dolly 106 allows the imaging device 100 to move along the depth direction of the tunnel. It is preferable to adjust the movement distance of the imaging device 100 so that adjacent images in the depth direction among the individual images P captured by the cameras 102A to 102E have an overlapping area. By repeatedly capturing images of the inner circumferential surface of the tunnel while moving the imaging device 100 as described above, it is possible to acquire individual images P that move along the depth direction of the tunnel and cover the entire inner circumferential direction of the tunnel.

[0043] 2, the five cameras 102A-102E are arranged in an arc, but the present invention is not limited to this. For example, one camera may be mounted rotatably around the Z axis (θ direction), and the camera may be rotated and captured each time the imaging device 100 is moved in the Z direction, thereby acquiring individual images P that cover the entire inner surface of the tunnel. Furthermore, the cameras 102A-102E may be arranged in a shape that corresponds to the shape (drawing data) of the inner surface of the tunnel (for example, a shape that approximates or is similar to the inner surface of the tunnel).

[0044] Fig. 4 is a block diagram of the imaging device 100. As shown in Fig. 4, the imaging device 100 can use a controller 150 to control movement and imaging.

[0045] The imaging device 100 includes cameras 102A to 102E, a storage 120, a driving mechanism 122, a distance measuring unit 124, and a communication I / F 126.

[0046] The storage 120 stores the individual images P captured by the cameras 102A to 102E. The storage 14 may be, for example, a device including a magnetic disk such as an HDD, a device including a flash memory such as an eMMC or an SSD, or a recording medium that is detachable from the imaging device 100 (for example, an SD memory card).

[0047] The communication I / F 126 is a device for communicating between the image processing device 1 and external devices including the controller 150. The individual images P captured by the cameras 102A to 102E may be transmitted to the image processing device 1 via the communication I / F 126. The individual images P may also be input to the image processing device 1 via a recording medium.

[0048] The drive mechanism 122 includes a motor for driving the carriage 106. The imaging device 100 moves the carriage 106 in the tunnel using the drive mechanism 122 in accordance with a drive control signal from the controller 150.

[0049] The distance measuring unit 124 is a device that measures the distance to the subject. As the distance measuring unit 124, for example, a TOF (Time of Flight) device that measures the distance to the subject using measurement light such as laser light or infrared light can be used. Note that the distance measuring unit 124 can be omitted.

[0050] As shown in FIG. 2 , when multiple cameras 102A-102E are mounted on the cart 106, the distance measurement unit 124 preferably rotates the measurement light along the arrangement of the cameras 102A-102E to measure the distance to the entire circumference of the tunnel's inner periphery. When the cameras 102A-102E capture images of the tunnel's inner periphery, some cameras experience a change in the distance to the captured wall surface by more than a threshold value as the cart 106 moves, while others experience no change. In this case, if the distance to the wall surface in the captured image of even one camera changes by more than the threshold value, the cart 106 is stopped, the imaging conditions are changed, and the cart is then moved back to resume imaging. Here, for cameras whose distance to the captured wall surface does not change by more than the threshold value, there is no need to change the imaging conditions or capture the same divided area twice.

[0051] The controller 150 includes a control unit 152 , an input / output unit 154 , and a communication I / F 156 .

[0052] The control unit 152 includes a processor (e.g., a CPU) and memory (e.g., a ROM (Read Only Memory) in which a control program is stored and a RAM serving as a working area for the processor) for controlling the imaging device 100. The control unit 152 controls the imaging of the cameras 102A to 102E and the driving of the dolly 106 and the driving mechanism 122 in accordance with input from the input / output unit 154.

[0053] The input / output unit 154 includes an input device that accepts instruction input from the user, and a display device that displays an image or GUI.

[0054] The communication I / F 156 is a device for communicating with external devices including the image capturing device 100 .

[0055] 4, the controller 150 is separate from the imaging device 100, enabling remote control of the imaging device 100, but the present invention is not limited to this. The controller 150 may be integrated with the imaging device 100, for example. Also, although the dolly 106 of the imaging device 100 is moved by the drive mechanism 122, the dolly 106 may be moved manually without providing the drive mechanism 122. Also, the image processing device 1 may also serve as the controller 150 of the imaging device 100.

[0056] 5, the image processing device 1 according to this embodiment connects or combines individual images P of the inner circumferential surface of the tunnel captured by the cameras 102A to 102E along the inner circumferential direction and depth direction of the tunnel to create a composite image (composite developed image), and outputs the composite image to the display unit 22. The user can use the display (GUI) of the composite image to observe, detect, and measure damage to the inner circumferential surface of the tunnel.

[0057] In practice, however, representing the entire area of ​​a structure in a single composite image may not be suitable for inspecting the structure. For example, if the size of the damage to be detected is small compared to the size of the area included in a single composite image, it may be difficult for the user to find the damage from the displayed composite image. Therefore, in this embodiment, a composite range is set for display and observation, including multiple divided areas of the structure. Then, a composite image is created using multiple image sets (each including multiple individual images P) corresponding to each composite range, and the multiple composite images are displayed side by side (see FIGS. 8 and 9 ).

[0058] 5 to 7 are diagrams for explaining the image processing function of the image processing device 1. Below, we will explain a tunnel with a substantially semicircular shape (Example A) and a tunnel with linear (flat) sides and a substantially semicircular top surface (arch) (Example B), but the image processing function according to this embodiment can be applied to tunnels with any shape.

[0059] The image processing device 1 uses the synthesis processing function of the processor 10 to synthesize individual images P of the tunnel's inner surface captured by the cameras 102A to 102E within a predetermined synthesis range. In the example shown below, the synthesis range is the entire circumference of the tunnel in the inner direction and a predetermined distance in the depth direction (e.g., 10 m). Then, by resizing this synthetic image, it becomes possible to comprehensively observe, detect, and measure damage to the tunnel's inner surface using the synthetic image for each synthesis range.

[0060] The size and aspect ratio of the composite area may be set automatically by the processor 10 or by the user depending on the display unit 22 or the like to which the image is output.

[0061] The composite image obtained by combining the individual images P includes an image of the side or the side and top (full circumference) and an image of part of the underside (road surface). The processor 10 extracts at least two points from this composite image.

[0062] In the example shown in FIG. 5, in example A, for example, the boundary B between the side surface and the bottom surface Ａ１ and B Ａ２ In Example B, at least two points (for example, two pixel positions) on the top surface are extracted. Ｂ１ and B Ｂ４ Top and side and top boundary B Ｂ２ and B Ｂ３ At least two of the above points (for example, two pixel positions) are extracted.

[0063] In Example B, when the height of the side is not high (for example, when (H−R) / H is equal to or less than a threshold value compared to the height H of the tunnel), the boundary B between the side and the bottom Ｂ１ and B Ｂ４ Only the upper points may be extracted.

[0064] The number of points extracted from the composite image is not limited to two, and they do not need to be distributed in the circumferential direction (H direction). For example, as shown in FIG. 6, they may be distributed in the depth direction (W direction) or in an oblique direction. The points extracted from the composite image may be distributed along the boundary (B Ａ１ , B Ａ２ , B Ｂ１ ~BＢ４ 6, it may be two points that serve as some kind of landmark (for example, a distinctive pattern on the inner circumferential surface, accessories such as cables or lights, or marks (chalk) made on the inner circumferential surface of the tunnel during inspection).

[0065] Next, the processor 10 acquires object information D related to the object structure OBJ (tunnel) from the object information DB 200. Here, the object information D includes, for example, data related to the design information of the tunnel (including data indicating, for example, the size and shape) or drawing data (CAD: Computer Aided Design). The processor 10 acquires the length (distance) between the two extracted points from this object information D.

[0066] The length between the two extracted points may be calculated from the dimensions on the drawing using the following estimation formula. For example, boundary B in Example A Ａ１ and B Ａ２ The distance between the two points above (tunnel circumference) is calculated using the following formula (1): Tunnel circumference = length of upper semicircular arc = πR (1)

[0067] Also, boundary B of Example B Ｂ１ and B Ｂ４ The distance between the above two points (tunnel circumference) can be calculated using the following formula (2): Tunnel circumference = upper semicircular arc + side height × 2 = πR + 2(H-R) (2)

[0068] The length between the two extracted points may be input by the user via the operation unit 20 (such as a measured value on-site).

[0069] Next, the processor 10 performs a resizing process so that the composite image approaches or matches a desired pixel spacing (target value). Here, pixel spacing (mm / pixel) is a parameter that indicates the length or distance of the subject represented by the pixels contained in the image. Note that the desired pixel spacing value may be set by the processor 10 based on a user instruction input from the operation unit 20.

[0070] The desired pixel spacing value may also be set based on pixel spacing information for each individual image P that constitutes the composite image. Specifically, the processor 10 acquires meta information (e.g., Exchangeable Image File Format (Exif) tag information, etc.) embedded in the image file of the individual image P, or pixel spacing information recorded in association (link) with the image file of the individual image P. The processor 10 then sets the pixel spacing of the composite image based on the pixel spacing information of the individual image P. For each individual image P that constitutes the composite image, the pixel spacing information is set to, for example, a representative value, more specifically, the average value, minimum value, median value, or maximum value.

[0071] However, if the composite image is resized in a direction that reduces the number of pixels or increases the pixel spacing (referred to as the pixel count reduction direction), the amount of information contained in the image will decrease. For this reason, it is desirable to set the desired pixel spacing to the minimum value of the pixel spacing of the individual images P that make up the composite image (matching the one with the highest spatial resolution).

[0072] Furthermore, the desired pixel spacing value may be set based on other information, rather than on the pixel spacing information of the individual image P. For example, the pixel spacing may be calculated using the following formula (3) using D (mm) as the distance to the subject (the inner circumferential surface of the tunnel, or the position of the point on the inner circumferential surface where the cameras (102A to 102E) are focused), F (mm) as the focal length of the cameras (102A to 102E), S (mm) as the size of the image sensor (sensor size horizontal or vertical), and P (pixels) as the number of pixels (horizontal or vertical) of the image sensor. If the distance to the subject is measured simultaneously with the image capture, the pixel spacing may also be calculated using this formula. Pixel spacing (mm / pixel) = Shooting range (mm) / Number of pixels (pixel) = (D × S / F) / P ... (3)

[0073] Next, after setting the desired pixel spacing, the processor 10 resizes the composite image to approach or match the desired pixel spacing. Note that the resizing may be performed on the entire composite image, on a region-by-region basis within the composite image (e.g., on the top or side), or on each individual image P.

[0074] In Example A (top row) of Figure 7, the entire composite image is resized. If the tunnel perimeter (actual length) is 10,000 mm and the desired pixel spacing is 0.5 mm / pixel, the target size (number of pixels) of the composite image is 10,000 / 0.5 = 20,000 pixels. If the number of pixels in the original composite image before resizing is 19,048 pixels, then 20,000 pixels / 19,048 pixels is approximately 1.05 times larger.

[0075] When an image is reduced by resizing, the amount of information contained in the image decreases, so if the resizing ratio falls outside a predetermined allowable range (for example, if the resizing ratio is smaller than 0.9), it is desirable to issue a warning and prompt the user to confirm.

[0076] Example B (bottom row) in Figure 7 shows an example in which resizing is performed on each region within the composite image. In this example, each individual region within the composite image is resized to achieve the desired pixel spacing, and the resized images of each individual region are then combined (integrated).

[0077] For example, different resizing processes may be performed on the top and side regions of the tunnel so that the desired pixel spacing is achieved for each. In this case, as shown in the lower part of Figure 7, unevenness may occur in the depth direction of the composite image. Since the convex portion of the composite image may partially overlap with an adjacent composite image, a trimming process may be performed to remove the convex portion, or the composite image may be reshaped into a rectangle.

[0078] 8 and 9 are diagrams showing examples (including GUI) of composite images (composite developed images). Fig. 8 shows an example in which 10 composite images C1 to C10, each of which shows a tunnel with a depth of 10 m, are arranged horizontally. Fig. 9 shows an example in which the composite images are enlarged.

[0079] 8 and 9, a scale SC indicating the length of the tunnel in the depth direction (W direction) is provided at the top of the screen. The unit of the numbers on the scale SC is 10 mm.

[0080] Scroll buttons AL and AR are provided on the left and right ends of the scale SC. By operating the scroll buttons AL and AR using the operation unit 20, it is possible to display an image of the entire range of the tunnel in the depth direction.

[0081] A frame T1 indicating the range of the composite image displayed in the center of the screen is displayed on the scale SC. In FIG. 8, the depth direction is 10 m, so the frame T1 is displayed at positions 1100 to 1200. The display range displayed in the center of the screen can be changed by operating (moving and resizing) the frame T1 using the operation unit 20. The size of the frame T1 may be changeable in both the W and H directions.

[0082] The symbol M in the figure is a marker that indicates the position of an object (for example, damage) that satisfies a predetermined condition.

[0083] The display size of the composite images C1 to C10 can be changed using the zoom in and zoom out buttons in the figure. In the example shown in Figure 9, composite images C7 to C9 are displayed enlarged, and the width in the W direction of the frame T2 of the scale SC is reduced accordingly.

[0084] The symbol SUB in the drawing denotes a sub-screen corresponding to the display before enlargement. On the sub-screen SUB, a frame T2 is displayed at a position equivalent to the frame T2 (corresponding to the composite images C7 to C9).

[0085] As described above, the pixel spacing of the composite images C1-C10 is made equal to each other through resizing. Therefore, by specifying two points on the composite images C1-C10 using the operation unit 20, the length between the two points can be measured. This allows, for example, the length, width, and height of a desired object (e.g., damage such as cracks, free lime, peeling, or corrosion) captured in the composite image to be measured. For example, in FIG. 9, the length L between the two points, the depth direction (width) W between the two points, and the vertical direction (circumferential direction) (height) H can be measured. In the example shown in FIG. 9, L = 4.85 m, W = 4.5 m, and H = 1.8 m.

[0086] According to this embodiment, by resizing the composite images C1 to C10, it is possible to display images suitable for comprehensive evaluation of damage.

[0087] (Image Processing Method) FIG. 10 is a flowchart showing an image processing method according to one embodiment of the present invention.

[0088] First, the processor 10 acquires individual images P from the imaging device 100 (step S10), and groups the individual images P into image sets corresponding to the synthesis range (step S12).

[0089] Next, pixel spacing is set (step S14). In step S14, as shown in Fig. 11, input of pixel spacing may be accepted from the operation unit 20 (step S140), and the pixel spacing may be set in accordance with this input (step S142). Alternatively, as shown in Fig. 12, the processor 10 may acquire information regarding the pixel spacing of the individual image P (step S144), and the pixel spacing may be set based on this information regarding pixel spacing (step S146).

[0090] Next, the appropriateness of the pixel spacing set in step S14 is determined (step S16), and if it is NG, a warning is output by the display unit 22 or a speaker (not shown) (step S18). In step S14, a warning may be output if the resize magnification in the resize process is outside the allowable range, or if the image set includes an individual image with a pixel spacing larger than the desired pixel spacing.

[0091] If the suitability determination (step S14) is OK, the processor 10 performs resizing and combining of the composite image (step S20). In step S20, the processor 10 may resize the composite image obtained by combining the individual images P, or may perform combining after resizing the individual images P. Furthermore, the processor 10 may perform resizing of each part of the composite image obtained by combining the individual images P.

[0092] The composite image created as described above is output to the display unit 22, and the user can refer to this display to inspect for damage, etc. (see FIGS. 8 and 9).

[0093] 13 , in the image output step, the processor 10 may detect predetermined features of the subject (e.g., cracks, free lime, peeling, corrosion, and other damage) based on feature quantities such as brightness and color of the composite image or individual images P, or based on machine learning, pattern matching, or the like (step S220). The processor 10 may render a user-visible image (e.g., by marking or color-coding) of the detection results detected in step S220 and output the rendered image to the display unit 22 (step S222). In step S222, the composite image and the rendering of the detection results may be displayed together (e.g., side by side or superimposed), or the composite image and the rendering of the detection results may be displayed switchably.

[0094] [Modification] In the above embodiment, an example in which the image processing device 1 is applied to a general-purpose computer or a tablet terminal has been described, but the present invention is not limited to this. For example, the image processing function according to the above embodiment may be implemented by a cloud server. That is, the image processing function according to the above embodiment may be provided as Software as a Service (SaaS). In this case, the image processing device 1 included in the cloud server may perform image processing by uploading the individual image P and the subject information D to the cloud server via a terminal (e.g., a tablet terminal) including the operation unit 20 and the display unit 22 and performing an operation input. Furthermore, the subject information DB 200 may be included in the cloud server.

[0095] Furthermore, the type of structure OBJ is not limited to tunnels. For example, the image processing according to the above-described embodiment can also be applied to the inspection of structures other than tunnels, such as bridges, roads, and dams. In the imaging device 100, the placement of the cameras (102A to 102E) can be changed depending on the structure, shape, size, etc. of the structure OBJ. As for the drive mechanism 122 of the imaging device 100, an appropriate one can be applied, such as an unmanned aerial vehicle such as a multicopter or drone, or a moving object such as a vehicle or robot, depending on the structure, shape, size, etc. of the structure OBJ.

[0096] REFERENCE SIGNS LIST 1 Image processing device 10 Processor 12 Memory 14 Storage 16 Communication I / F 20 Operation unit 22 Display unit 100 Imaging device 102A to 102E Camera 104 Camera mounting member 106 Cart 120 Storage 122 Drive mechanism 124 Distance measuring unit 126 Communication I / F 150 Controller 152 Control unit 154 Input / output unit 156 Communication I / F 200 Subject information DB

Claims

1. An image processing device having a processor, wherein the processor acquires pixel positions of at least two points of a plurality of individual images of a subject or a composite image obtained by combining the individual images, and an actual length between the two points, and resizes the individual images or the composite image so that the number of pixels between the two points approaches a desired pixel spacing.

2. The image processing device according to claim 1, wherein the two points are points on a boundary between different surfaces that constitute the subject.

3. The image processing device according to claim 1, wherein the processor specifies the two points in accordance with an instruction input from a user.

4. An image processing device according to any one of claims 1 to 3, wherein the processor obtains the actual length between the two points from drawing data of the subject.

5. An image processing device according to any one of claims 1 to 3, wherein the processor obtains the actual length between the two points from information indicating the shape and dimensions of the subject.

6. The image processing device of claim 1, wherein the processor: divides the plurality of individual images into a plurality of image sets; combines the plurality of image sets to create a plurality of composite images; and resizes the plurality of composite images so that pixel spacing of the plurality of composite images approaches the desired pixel spacing.

7. The image processing device according to claim 6, wherein the processor causes a display unit to display the multiple composite images side by side.

8. An image processing device according to any one of claims 1, 6 and 7, wherein the processor detects damage to the subject from the individual images or the composite image, and depicts the damage detection result together with the composite image.

9. The image processing device according to claim 8, wherein the processor causes a display unit to switch between displaying the composite image and the composite image depicting the damage detection result.

10. An image processing device according to any one of claims 1, 6 and 7, wherein the processor specifies pixel spacing of the individual images or the composite image after the resizing process in accordance with an instruction input from a user.

11. An image processing device according to any one of claims 1, 6 and 7, wherein the processor specifies pixel spacing of the individual images or the composite image after the resizing process based on information about pixel spacing of the individual images.

12. The image processing device according to claim 11, wherein the processor specifies pixel spacing of the composite image after the resizing process based on at least one of a representative value, an average value, a maximum value, and a minimum value of pixel spacing of the individual images.

13. The image processing device according to claim 1, wherein the processor outputs a warning when a resizing ratio in the resizing process is outside an allowable range.

14. An image processing device according to any one of claims 1 to 3, wherein the processor outputs a warning if any of the individual images has a pixel spacing greater than the desired pixel spacing.

15. An image processing device according to any one of claims 1 to 3, wherein the individual image is an image captured of at least a portion of the top, side and bottom of a tunnel as the subject.

16. An image processing method using an image processing device equipped with a processor, comprising: a step in which the processor acquires pixel positions of at least two points of a plurality of individual images of a subject or a composite image obtained by combining the individual images, and an actual length between the two points; and a step in which the processor resizes the individual images or the composite image so that the number of pixels between the two points approaches a desired pixel spacing.

17. An image processing program that causes a computer to realize the following functions: obtaining at least two pixel positions of multiple individual images of a subject or a composite image obtained by combining the individual images, and the actual length between the two points; and resizing the individual images or the composite image so that the number of pixels between the two points approaches a desired pixel spacing.

18. A non-transitory computer-readable recording medium having the program according to claim 17 recorded thereon.

Images