System for diagnosing condition of inspection object, condition diagnosis device, and method for identifying image position

Abstract

To provide a state diagnosis system of an inspection object, a state diagnosis device, an image position specification method and program which can specify an imaging position on the basis of a dynamic image obtained by imaging an inspection object.SOLUTION: A state diagnosis system comprises: a movement mechanism 300 which moves a movement unit 305 within an imaging possible range along the surface of an inspection object 10; a camera 200 which is attached to the movement unit to capture a moving image of an inspection object surface; a coordinate acquisition unit 104 which acquires position coordinates of the movement unit within the imaging possible range; and a data processing unit 109 which detects a movement direction change point of the camera on the basis of the plurality of images of the moving image and detects an image for inspection corresponding to the desired position coordinates from the moving image by making the coordinate direction change point of the movement unit obtained from the coordinate acquisition unit correspond to the movement direction change point of the camera.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for diagnosing the condition of an object to be inspected, and more particularly to a condition diagnosis system, a condition diagnosis device, and a method for identifying the position of an image using an image of the object to be inspected. [Background technology]

[0002] Various methods for inspecting the condition of buildings have been proposed to prevent peeling and falling off of the exterior materials (exterior wall materials) of buildings. For example, Patent Document 1 discloses a condition evaluation device that efficiently and at low cost diagnoses exterior materials over a wide area, such as the exterior walls of buildings, in which a detection unit that performs inspection by striking the exterior wall is moved along the exterior wall plane, and condition evaluation information for the entire exterior wall is generated by associating the inspection position on the exterior wall with the inspection results.

[0003] According to Patent Document 1, a camera installed above the frame holds in its field of view reference points installed at the four corners of the frame and an index installed on a detection unit that can move in the xy direction within the frame, and captures an image using an impact on the detection unit as a trigger. Therefore, the inspection position of the detection unit can be specified from the positional relationship between the reference points and the index of the detection unit in the captured image, and the inspection position can be associated with the inspection result. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-178953 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned Patent Document 1, the camera captures images according to the timing of the impact of the detector, so that images synchronized with the inspection timing can be easily obtained. However, when diagnosing the surface condition of an exterior wall using images, it is necessary to capture moving images by moving the camera close to the exterior wall to be inspected. No technology has been proposed to identify the inspection position based on moving images without a trigger signal such as an impact.

[0006] Therefore, an object of the present invention is to provide a condition diagnosis system, a condition diagnosis device, an image position identification method, and a program for identifying an imaging position of an object to be inspected, which can identify an imaging position based on a moving image of the object to be inspected. [Means for solving the problem]

[0007] In order to achieve the above object, according to one embodiment of the present invention, a condition diagnosis system includes a moving mechanism that moves a moving part along a surface of an object to be inspected within an imageable range, a camera that is attached to the moving part and captures a moving image of the surface of the object to be inspected, a coordinate acquisition part that acquires position coordinates of the moving part within the imageable range, and a data processing part that detects an inspection image from the moving image, wherein the data processing part detects a moving direction change point of the camera based on a plurality of images of the moving image, and The point where the direction of change of position coordinates turns around and a point where the camera's movement direction changes, thereby detecting an inspection image corresponding to a desired position coordinate from the moving image. Furthermore, according to one embodiment of the present invention, the movement direction change point may be a point in time in the image of interest at which the movement direction of a specific image portion contained in the image of interest and two images that precede and follow the image of interest in time changes by more than a predetermined angle. Furthermore, according to one embodiment of the present invention, an edge portion can be extracted from the inspection image, and a tile region of the inspection object can be extracted from the edge portion by a rectangle detection algorithm. Furthermore, according to one embodiment of the present invention, the data processing unit can evaluate the surface condition of the object to be inspected based on tile regions extracted from the inspection image. Furthermore, according to one embodiment of the present invention, the data processing unit can detect crack lines in at least one tile region from the inspection image by image processing. Furthermore, according to one embodiment of the present invention, the data processing unit can connect the end points of a first crack line and a second crack line if the end points are within a predetermined distance and a predetermined angle between them, and detect the first and second crack lines as a single crack line. Moreover, according to one embodiment of the present invention, the device further includes a percussion unit attached to the movable part, which strikes the surface of the object to be inspected and outputs the striking sound to the data processing part as an inspection signal. According to one embodiment of the present invention, a condition diagnosis device includes a movement control unit that controls a movement mechanism that moves a moving unit along a surface of an inspection object within an imageable range, an interface that is connected by wire or wirelessly to a camera that is attached to the moving unit and captures a moving image of the surface of the inspection object, a coordinate acquisition unit that acquires position coordinates of the moving unit within the imageable range, and a data processing unit that detects an inspection image from the moving image, wherein the data processing unit detects a movement direction change point of the camera based on a plurality of images of the moving image, and The point where the direction of change of position coordinates turns around and a point where the camera's movement direction changes, thereby detecting an inspection image corresponding to a desired position coordinate from the moving image. According to one embodiment of the present invention, there is provided a method for specifying a position of an image in a condition diagnosis device having a movement control unit that controls a movement mechanism that moves a movement unit along a surface of an object to be inspected within an imageable range, and an interface that is attached to the movement unit and connects to a camera that captures a moving image of the surface of the object to be inspected by wire or wirelessly, wherein a coordinate acquisition unit acquires position coordinates of the movement unit within the imageable range, a data processing unit detects a change point in the movement direction of the camera based on a plurality of images of the moving image, and The point where the direction of change of position coordinates turns around and a point where the camera's movement direction changes, thereby identifying an image corresponding to a desired position coordinate from the video. According to one embodiment of the present invention, a program is provided that causes a computer to function as a condition diagnosis device having a movement control unit that controls a movement mechanism that moves a movement unit along a surface of an object to be inspected within an imageable range, and an interface that is attached to the movement unit and connects, by wire or wirelessly, to a camera that captures a moving image of the surface of the object to be inspected, the program having a function of acquiring position coordinates of the movement unit within the imageable range, and a function of detecting a change point in the movement direction of the camera based on a plurality of images of the moving image, The moving part of The point where the direction of change of position coordinates turns around and a function of identifying an image corresponding to a desired position coordinate from the video by associating the position coordinates with a change point in the moving direction of the camera. Effect of the Invention

[0008] The condition diagnosis system according to the embodiment of the present invention detects a moving direction change point from a moving image, and calculates the moving direction change point from the position coordinates of the camera. The point where the direction of change of position coordinates turns around By associating the above, even in a system in which the video image obtained from the camera and the movement control for moving the camera are not synchronized, it is possible to know which position on the object to be inspected the image in the video image corresponds to, and it is possible to specify the shooting position of the image based on the image of the object to be inspected. According to one embodiment of the present invention, a change in the movement direction of a specific image portion included in a plurality of images can be easily detected based on an angle change. According to one embodiment of the present invention, it is possible to reliably extract tile areas from an inspection image by detecting edges of the inspection image and extracting the tile areas using a rectangle detection algorithm. According to one embodiment of the present invention, surface conditions can be assessed based on images of the extracted tile regions. According to one embodiment of the present invention, crack lines can be detected by image processing. According to one embodiment of the present invention, multiple crack lines on an image can be reconnected taking into account their directions, and multiple crack lines can be detected as a single crack line. According to one embodiment of the present invention, a percussion unit capable of striking and inspecting the surface of the inspection object can be moved in the same manner as a camera, enabling the entire area of ​​the inspection object 10 to be inspected efficiently. The condition diagnosis device according to one embodiment of the present invention has an interface to which a camera can be connected, inputs a moving image from the connected camera, and detects a moving direction change point detected from the moving image and obtains a position coordinate of the camera body. The point where the direction of change of position coordinates turns around By associating the above, even in a system in which the video image obtained from the camera and the movement control for moving the camera are not synchronized, it is possible to know which position of the object to be inspected is the image of in the video image, and it is possible to specify the shooting position of the image based on the image of the object to be inspected. In addition, since the camera can be connected via an interface, there is an advantage that the inspection can be performed with a camera with performance suitable for the environment and the state of the object to be inspected. The image position specifying method according to the embodiment of the present invention detects a movement direction change point from a moving image, and calculates the position of the moving image from the position coordinates of the camera. The point where the direction of change of position coordinates turns around By associating the above, even in a system in which the video image obtained from the camera and the movement control for moving the camera are not synchronized, it is possible to know which position on the object to be inspected the image in the video image corresponds to, and it is possible to specify the shooting position of the image based on the image of the object to be inspected. A program according to an embodiment of the present invention detects a movement direction change point from a moving image and calculates a moving direction change point from the position coordinates of the camera. The point where the direction of change of position coordinates turns around By associating the above, even in a system in which the moving image obtained from the camera and the movement control for moving the camera are not synchronized, it is possible to realize a condition diagnosis device on a computer that can identify the shooting position of an image based on an image of an object to be inspected. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the overall configuration of a condition diagnostic system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic plan view showing an example of an xy actuator used in the condition diagnosis system according to the first embodiment. [Diagram 3] FIG. 3 is a cross-sectional view of the xy actuator of FIG. 2 taken along line II. [Figure 4] FIG. 4 is a schematic diagram showing an example of a carriage movement locus of the xy actuator in the first embodiment. [Diagram 5] FIG. 5 is a flowchart showing an example of a control operation during imaging in the condition diagnosis device according to the first embodiment. [Figure 6] FIG. 6 is a schematic diagram showing an example of movement of the imaging range relative to the inspection object. [Figure 7] FIG. 7 is a flowchart showing a method for identifying an image position in the condition diagnosis device according to the first embodiment. [Figure 8] FIG. 8 is a schematic diagram showing the correspondence between a movement trajectory obtained from a moving image in the condition diagnosis device according to the first embodiment and inspection point coordinates acquired from an xy actuator. [Figure 9] FIG. 9 is a schematic diagram showing an example of a method for detecting the movement direction of a captured image. [Figure 10] FIG. 10 is a schematic diagram showing an example of a method for detecting a moving direction change portion of a captured image. [Figure 11]FIG. 11 is a schematic diagram showing an example of a synchronization method in which a moving direction change portion of a captured image in the condition diagnosis device according to the first embodiment is associated with a turning point of an xy actuator. [Figure 12] FIG. 12 is a schematic diagram illustrating the correspondence between the imaging frames, the turning point coordinates of the xy actuator, and time in the condition diagnosis device according to the first embodiment. [Figure 13] FIG. 13 is a block diagram showing the functional configuration of a condition diagnostic device according to the second embodiment of the present invention. [Figure 14] FIG. 14 is a schematic plan view showing an example of an xy actuator used in the condition diagnostic device according to the second embodiment. [Figure 15] FIG. 15 is a cross-sectional view of the xy actuator of FIG. 14 taken along line II. [Figure 16] FIG. 16 is a schematic cross-sectional view showing the configuration of a percussion unit in the condition diagnosis device according to the second embodiment. [Figure 17] FIG. 17 is a flowchart showing an example of a tile region detection method in the condition diagnosis device according to the second embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a tile wall surface for explaining a tile region detection method in the condition diagnosis device according to the second embodiment. [Figure 19] 19(A) to 19(D) are diagrams showing an example of image processing for explaining a tile region detection method in the condition diagnostic device according to the second embodiment. [Figure 20] FIG. 20 is a flow chart showing an outline of a crack inspection procedure in the condition diagnosis device according to the second embodiment. [Figure 21] FIG. 21 is a schematic diagram showing an example of binarization processing in a crack inspection procedure in the condition diagnosis device according to the second embodiment. [Figure 22] FIG. 22 is a diagram showing a schematic diagram of the result of image processing in the crack inspection procedure in the condition diagnosis device according to the second embodiment. [Diagram 23] FIG. 23 is a flowchart showing an example of a crack reconnection process in the condition diagnosis device according to the second embodiment. [Figure 24] FIG. 24 is a schematic diagram showing an example of the crack reconnection process in the condition diagnosis device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiments are merely examples, and are not intended to limit the technical scope of the present invention.

[0011] 1. First embodiment 1.1) System configuration As illustrated in Fig. 1, the condition diagnosis system according to the first embodiment of the present invention is a system for evaluating the condition of an inspection object 10, and includes a condition diagnosis device 100, a camera 200 as an example of an imaging unit, and an xy actuator 300 as an example of a moving mechanism. As described below, the condition diagnosis device 100 executes a predetermined program on a computer processor, thereby controlling the movement of the xy actuator 300, inputs video images from the camera 200 that moves accordingly, extracts inspection images at required timing from the video images, and performs image processing on the extracted images to evaluate the condition of the inspection object 10. The condition diagnosis system according to this embodiment is an asynchronous system, and the movement of the camera 200 body and the video images captured by the camera 200 are not necessarily synchronized.

[0012] The camera 200 can be connected to the condition diagnosis device 100 by a predetermined wired or wireless interface, and can output a captured video file of a predetermined format to the condition diagnosis device 100. Here, a wired interface is adopted in consideration of the transmission of the video and the power supply, and the camera 200 is detachably connected to the condition diagnosis device 100. The frame rate during imaging can be selected, and can be set to a predetermined rate, for example, from 30 to 240 fps (frames per second). As illustrated in FIG. 1, the camera 200 can be synchronized to a standard time such as the Internet, but imaging is performed at a predetermined frame rate according to its own timing CLK1. The video captured by the imaging unit 201 is transmitted to the condition diagnosis device 100 through the interface 202. It is desirable that the camera 200 be in focus at a close distance and have a wide angle of view.

[0013] The xy actuator 300 moves a moving part in the x and y directions under the control of the condition diagnosis device 100. The camera 200 is movable in the x and y directions within a predetermined imaging range by being fixed to the moving part of the xy actuator 300. The configuration and operation of the xy actuator 300 will be described below with reference to Figs. 2 to 4.

[0014] <Movement mechanism> 2 and 3, the xy actuator 300 includes a pair of parallel y frames 302 fixed to a rectangular frame 301, and an x ​​stage 303 perpendicular to the y frames 302. The x stage 303 is fixed on a y carriage 304 movable on each y frame 302, and the pair of y carriages 304 move in the y-axis direction by the rotation of a y-axis stepping motor M1. Therefore, the x stage 303 can be moved to a desired position in the y-axis direction by the y-axis stepping motor M1.

[0015] A moving unit 305, which is an x-carriage, is movably provided on the x-stage 303, and a camera 200 and a light emitting unit 306 are fixed to the moving unit 305. The light emitting unit 306 is a light source that illuminates at least the inspection target 10 in the imaging range 200a of the camera 200, and may be a light emitting diode (LED) or the like. The moving unit 305 moves on the x-stage 303 by the rotation of an x-axis stepping motor M2. Therefore, the camera 200 fixed to the moving unit 305 can be moved to a desired position in the x-axis direction by the x-axis stepping motor M2.

[0016] By controlling the rotation of each of the y-axis stepping motor M1 and the x-axis stepping motor M2 in this manner, the camera 200 fixed to the moving part 305 can capture the entire imageable range 307 while moving at a predetermined speed (or at predetermined steps) along a predetermined route in the imageable range 307. Casters 308 are provided at the four corners of the frame 301 to enable the frame 301 to be moved over the inspection object 10, and the frame 301 can also be moved over the inspection object 10 by a driving means (not shown).

[0017] More specifically, as illustrated in FIG. 4, the camera 200 can be moved by a predetermined distance Δx in the x-axis direction by rotating the x-axis stepping motor M2 by a predetermined angle, and the camera 200 can be moved by a predetermined distance Δy in the y-axis direction by rotating the y-axis stepping motor M2 by a predetermined angle. For example, if the upper left corner of the imageable range 307 is the scanning start point, T0 is the scanning start time, and the coordinate point is expressed as p(0,0;T0), a point moved by Δx in the x-axis direction has an x ​​coordinate changed by +1 or -1, and a point moved by Δy in the y-axis direction has a y coordinate changed by +1 or -1. In this way, an arbitrary position of the camera 200 is expressed as coordinate p(x,y;T). Here, T is the elapsed time from the start point p(0,0;T0). It is assumed that the accuracy of the moving distances Δx and Δy can be obtained by the encoders attached to the y-axis stepping motor M1 and the x-axis stepping motor M2 with sufficient positioning accuracy.

[0018] As shown in FIG. 4, the camera 200 sequentially scans in the x-axis direction from the starting point coordinate p(0,0;T0) to the end point p(m,0;T m(1) ), that is, the right end of the imageable range 307, the right end p(m,2;T m(2) ), and then scans in the opposite direction to the left end of the imageable range 307. m(1) ) or p(m,2;T m(2) ) can be regarded as a change point in the coordinate direction, i.e., a scan turnaround point. Below, the upper coordinate direction change point p(m,0;T m(1) ) is the scan turn-around point at the right end of each x-axis scan line.

[0019] At the scan turn-around point, the camera 200 changes its moving direction by an angle of approximately 90°. This 90° change in the moving direction also appears as a 90° change in the moving image of the camera 200. Therefore, if it is detected that the moving image has changed in the direction of movement by nearly 90°, the frame information at that point in time can be regarded as the actual scan turn-around point of the camera 200, and the frame image of the camera 200 can be synchronized with the operation of the xy actuator 300 based on this as a reference.

[0020] However, since it is sufficient to detect a valid change in the movement direction, the change angle does not need to be 90° as in Fig. 4, and may be 180° when turning back to the starting point side. According to this embodiment, as an example, synchronization is established between the camera 200 and the xy actuator 300 by detecting a 90° change in the movement direction in a moving image.

[0021] The means for moving the y-carriage 304 and the moving part 305 by the y-axis stepping motor M1 and the x-axis stepping motor M2 can be of any type. A feed screw type or a belt feed type may be used, but the belt feed type is used in this embodiment. A commercially available product can be used as this xy actuator 300. The size of the imaging range 307 of the camera 200 is, for example, about 60 x 60 cm. The moving mechanism is not limited to the xy actuator 300, and a robot arm or the like may be used as long as sufficient positioning accuracy can be obtained.

[0022] <Condition diagnosis device> 1, the condition diagnosis device 100 can realize various functions described below by executing a program stored in a memory (not shown) on a processor. The control operation of the condition diagnosis device 100 is executed according to timing CLK2, and starts operation in accordance with a standard time such as the Internet, but it does not need to be synchronized with CLK1 of the camera 200.

[0023] Condition diagnosis device 100 can be connected to camera 200 via wired or wireless interface 101. Video storage unit 102 stores video files input from camera 200 via interface 101. The video files can be in a well-known format such as AVI or MOV, and it is assumed here that timestamps are added to the video files in one-second increments.

[0024] The xy movement control unit 103 controls the y-axis stepping motor M1 and the x-axis stepping motor M2 of the xy actuator 300 in accordance with the timing CLK2, and moves the moving unit 305 to which the camera 200 is fixed in the xy directions as described above. The coordinate acquisition unit 104 outputs the position and time of the camera 200 on the moving unit 305 as coordinates p(x, y; T) to the image position synchronization unit 107, as shown in Fig. 4. In addition, the moving image file captured by the camera 200 is stored in the moving image storage unit 102 together with a time stamp.

[0025] The movement detection unit 105 sequentially reads out a number of frame images from the video file, and uses a method such as optical flow to detect the movement direction of the images, and therefore the movement direction of the camera 200. The movement direction change detection unit 106 determines whether the movement direction has changed by more than a predetermined angle using a method described later, and outputs the image frame number F and time information t of the image that has changed by more than the predetermined angle to the image position synchronization unit 107 as a movement direction change point.

[0026] The image position synchronization unit 107 receives the position information of the camera 200 as coordinates p(x, y; T) from the coordinate acquisition unit 104, and as described later, calculates the coordinate change point p(m, y m ;T m ) is associated with the time t of the moving direction change point of the video. In this way, the position coordinates of the camera 200 and the video are synchronized, and the image frame of the video file corresponding to the specified coordinates can be identified.

[0027] The evaluation unit 108 inputs an image of a predetermined position from the image position synchronization unit 107, recognizes the area of ​​the inspection target by image processing, and evaluates the state of the area. For example, if the inspection target 10 is a tiled wall surface, one tile area can be detected from the input image, and the state of the tile area, such as cracks, can be diagnosed by image recognition technology. The detection of the tile area will be described in detail in the section on the second embodiment.

[0028] The condition diagnosis device 100 can realize the above-mentioned various functions by executing a predetermined program on a computer. The overall operation of the condition diagnosis system according to the first embodiment will be described in detail below with reference to Figs. 5 to 12.

[0029] 1.2) Operation <Video image acquisition> As illustrated in FIG. 5, the processor 109, which is the data processing unit of the condition diagnosis device 100, resets the timing CLK2 to the standard time by initial setting, starts the camera 200, and resets the moving part 305 of the xy actuator 300 to the origin position p(0,0) (operation 501).

[0030] As described above, the camera 200 captures the inspection target 10 according to its own timing CLK1, and outputs moving image data together with time stamps at predetermined intervals (here, 1 second intervals) (operation 502). Here, it is assumed that the image is captured at a frame rate of 240 fps. The condition diagnosis device 100 receives the moving image data from the camera 200, and stores a moving image file consisting of the frame images F(i) decomposed into frames in the moving image storage unit 102 (operation 503).

[0031] Along with the imaging by the camera 200, the processor 109 of the condition diagnosis device 100 controls the xy movement control unit 103 to move the moving unit 305 of the xy actuator 300 as shown in Fig. 4, thereby moving the imaging range 200a of the camera 200 on the inspection target 10 at a predetermined speed or in predetermined steps (operation 504), as shown in Fig. 6. At that time, the processor 109 inputs and records the coordinates p(x, y; T) of the moving unit 305 from the coordinate acquisition unit 104 (operation 505).

[0032] The processor 109 of the condition diagnosis device 100 judges whether or not the entire imageable range 307 has been scanned while monitoring the coordinates of the moving part 305 (operation 506), and if the entire scanning has not been completed (NO in operation 506), the above operations 502 to 506 are repeated until the entire scanning is completed. If the entire scanning is completed (YES in operation 506), the image capturing by the camera 200 is stopped, and the process is terminated (operation 507).

[0033] <Detection of change in movement direction> 7, the processor 109 of the condition diagnosis device 100 sequentially reads out a plurality of frame images from the video file in the video storage unit 102, and detects the movement direction of a specific image portion, and therefore the movement direction of the camera 200, by optical flow (operation 601). For example, in the case of a wall surface on which tiles are arranged as shown in FIG. 6, the movement direction of the camera 200 can be detected by following the movement of a specific tile image for each frame image.

[0034] Next, the processor 109 judges whether or not the movement direction has changed by a predetermined angle or more, and detects the image frame number F and time information t at which the movement direction has changed by the predetermined angle or more as a movement direction change point (operation 602). Hereinafter, an example of a method for detecting a movement direction change point will be described with reference to Figs. 8 to 10.

[0035] First, as illustrated in FIG. 8, an image movement trajectory 401a is acquired from a plurality of frame images by optical flow, and a coordinate trajectory 401b indicating the movement of the camera 200 is acquired from the coordinate p(x,y) input from the coordinate acquisition unit 104. The coordinate trajectory 401b corresponds to the scan trajectory in the coordinate system shown in FIG. 4. The movement trajectory 401a and the coordinate trajectory 401b are based on the same scan movement of the camera 200, but are not associated with the coordinates indicating the video of the camera 200 and the camera position. For this reason, it is not possible to know which position (coordinate) of the inspection target 10 the frame image of the video is. Therefore, in this embodiment, for example, movement direction change points 402a, 403a on the movement trajectory 401a and coordinate direction change points (scan turnaround points) 402b, 403b on the coordinate trajectory 401b are detected, and the video and coordinates are synchronized based on them. The association between the movement direction change point and the coordinate direction change point (scan turnaround point) may be performed for each x-axis line, or may be performed for every predetermined number of x-axis lines, or may be performed when there is a deviation beyond a predetermined value, such as movement direction change point 403a in Fig. 8. In other words, when the same movement pattern is repeated, once synchronization is achieved, synchronization is not performed even if there is a movement direction change point (corner) thereafter, and synchronization can be achieved at the movement direction change point (corner) when there is a large deviation from the repetition.

[0036] As shown in Fig. 9, with respect to the position of the point of interest in a certain image frame F(i), the position of the point of interest in an image frame F(iN) N frames before is vector m1, the position of the point of interest in an image frame F(i+N) N frames after is vector m2, and the angle formed by vectors m1 and m2 is θ (here, i>N, where i and N are integers equal to or greater than 0). In this case, the change in the moving direction can be calculated by the dot product S = |m1| · |m2| cosθ of vectors m1 and m2. That is, if |m1| = |m2| = 1, when the point of interest moves in the x-axis direction, θ = 180°, so the dot product S = -1, when it changes direction in the y-axis direction, θ = 90°, so the dot product S = 0, and when it turns around, θ = 0°, so the dot product S = +1. Therefore, if the threshold value TH of the S value is set so as to detect only clear angle changes such as θ=90° or 0° and ignore other angle changes, it is possible to detect movement direction change points 402a, etc. on the movement trajectory 401a and coordinate direction change points 402b, etc. on the coordinate trajectory 401b.

[0037] As shown in FIG. 10, the movement direction change points 402a(1) and 402a(2) on the movement trajectory 401a are both changed at an angle of about 90°. Therefore, in the image frames until the change point is reached, the attention point moves in the x-axis direction, so S is approximately -1, but at the first change point 402a(1), S rises to approximately 0, then returns to approximately -1, and at the second change point 402a(2), it rises again to approximately 0, and then moves to the next x-axis line, so S remains approximately -1. Therefore, by appropriately setting a threshold value TH, it is possible to identify the frame number and time of the image frame where the S value peaks (i.e., where the movement direction change point is indicated). Here, the movement direction change point 402a is detected at the first peak point 402a(1) of the S value, and the coordinate direction change point (scan turnaround point) 402b on the coordinate trajectory 401b is detected in a similar procedure.

[0038] <Image position synchronization> 7, when the processor 109 detects the image frame number F and time t as the movement direction change point 402a by the above-mentioned operation 602, it can synchronize the progress of the frame number of the moving image (i.e., the change in the image over time) with the change in the position coordinate over time by associating the movement direction change point 402a with the coordinate direction change point 402b (operation 603). Image position synchronization will be described below with reference to FIG.

[0039] As shown in Fig. 11, the S value peak of a moving direction change point 402a detected from a plurality of frame images and the S value peak of a coordinate direction change point 402b are time-aligned to correspond to each other. In this case, the time stamp of the moving image can be used as follows.

[0040] As an example, the imaging range 307 of the xy actuator 300 is 60×60 cm, and the xy actuator 300 moves the camera 200 at a speed of 20 cm per second. The camera 200 captures images at a frame rate of 240 fps, and adds time stamps at one-second intervals. In this case, it takes three seconds for the camera 200 to move one x-axis line from one end of the imaging range 307 to the other, and the camera 200 moves 20 cm as a scan distance for each time stamp, outputting 240 frames of images during that time. Therefore, the deviation between the time stamp tm of the frame image with the movement direction change point 402a and the time Tm of the position coordinate of the coordinate direction change point 402b is a maximum of about ±1 second, and it is sufficient to match the closest movement direction change point 402a and coordinate direction change point 402b within this range.

[0041] <Image acquisition> 7, when the processor 109 establishes image position synchronization by the above-mentioned operation 603, it outputs one or more frame images corresponding to each position coordinate p(x, y) by the xy actuator 300 as image data for inspection (operation 604). Hereinafter, image acquisition will be described with reference to FIG.

[0042] As illustrated in FIG. 12, by associating the image frame F(n) indicating the moving direction change point with the time Tm of the position coordinate 402a, the image frames F(0) to F(n) can be associated with the times T0 to Tm of the position coordinates, and this makes it possible to specify the image frames F(0), F(i), F(j), etc. corresponding to the predetermined position coordinates p(0,0), p(1,0), p(2,0), etc. In this way, the position coordinates of the camera 200 and the captured image can be synchronized, and the captured image at the predetermined position coordinates of the inspection object 10 can be output. Using the captured image obtained in this way, the evaluation unit 108 can diagnose the state of the inspection object 10.

[0043] 1.3) Effects As described above, according to the first embodiment of the present invention, even in a system in which the video obtained from the camera 200 and the movement control of the xy actuator 300 that moves the camera 200 body in a scan are not synchronized, it is possible to know which position on the inspection object 10 the frame image of the video corresponds to. As described above, the movement direction change point is detected from the video, the coordinate direction change point (scan turnaround point) is detected from the position coordinates of the camera 200, and the timing of the movement direction change point and the coordinate direction change point are matched to associate the video of the camera 200 with the coordinates indicating the camera position. This makes it possible to identify the image capture position based on the image captured of the inspection object.

[0044] 2. Second embodiment 2.1) System configuration As illustrated in Fig. 13, the condition diagnosis system according to the second embodiment of the present invention includes a condition diagnosis device 700, a camera 200 as an example of an imaging unit, an xy actuator 300 as an example of a moving mechanism, and a diagnosis unit 800 that moves together with the camera 200. The condition diagnosis device 700 controls the xy actuator 300 by executing a predetermined program on a computer processor as in the first embodiment, performs condition evaluation of the inspection target 10 using a moving image, and further performs condition evaluation by striking the inspection target 10 with the diagnosis unit 800. Below, configurations and functions of the second embodiment that are different from those of the first embodiment will be described, and blocks having the same functions as those of the first embodiment will be given the same reference numbers and detailed description will be omitted.

[0045] Processor 701, which is a data processing unit of condition diagnosis device 700, executes a program to realize functions consisting of movement detection unit 105, movement direction change detection unit 106, image position synchronization unit 107, and evaluation unit 108, similar to those in the first embodiment. The evaluation unit 108 in this embodiment consists of a tile area detection unit 702 and a crack detection unit 703.

[0046] A tile area detection unit 702 inputs an image of a predetermined position from the image position synchronization unit 107 and detects the tile area to be inspected by image processing, and a crack detection unit 703 detects cracks, etc. by image processing from the image of the predetermined position input from the image position synchronization unit 107. The detection results of cracks, etc. are stored in a diagnosis information storage unit 704 together with the position coordinates of the tile area.

[0047] The diagnosis unit 800 is a sensor means that outputs an inspection signal by striking the surface of the inspection object 10, and for example, the detection unit disclosed in JP 2016-205900 A can be adopted. The diagnosis unit 800 is controlled by a percussion control unit 705, which is controlled in synchronization with the scan control by the xy movement control unit 103 that controls the xy actuator 300. The diagnosis unit 800 is fixed to the moving unit 305 of the xy actuator 300, as described later.

[0048] The inspection signal due to impact obtained from the diagnosis unit 800 is evaluated by the percussion evaluation unit 706, and the evaluation result is stored in the diagnosis information storage unit 704 together with the position coordinates. Therefore, the surface condition of each position of the inspection target 10 is stored in the diagnosis information storage unit 704 as the image diagnosis result based on the image captured by the camera 200 and the impact diagnosis result by the diagnosis unit 800. The configuration and operation of the xy actuator 300 are as described in the first embodiment. The configuration around the diagnosis unit 800 attached to the xy actuator 300 will be described below with reference to Figs. 14 and 15.

[0049] As illustrated in Figures 14 and 15, the xy actuator 300 comprises a pair of parallel y frames 302 and an x ​​stage 303 perpendicular to the y frames 302. As already explained, the x stage 303 can be moved in the y-axis direction by a y-axis stepping motor M1, and the moving part 305 can be moved in the x-axis direction on the x stage 303 by an x-axis stepping motor M2.

[0050] In this embodiment, a percussion unit 800 is fixed to the moving section 305 directly below the x-stage 303 so as to face the surface of the inspection target 10. In addition, a camera 200 and a light emitting section 306 are fixed to the moving section 305 at a position adjacent to the percussion unit 800, and are installed so that the diagnostic unit 800 does not enter the imaging range 200a of the camera 200. Since the positional relationship between the camera 200 and the diagnostic unit 800 with respect to the moving section 305 is fixed, the offset between the imaging range 200a and the striking position by the diagnostic unit 800 can be corrected.

[0051] By controlling the rotation of the y-axis stepping motor M1 and the x-axis stepping motor M2 in this manner, the camera 200 fixed to the moving part 305 captures images while scanning the imageable range 307, and the diagnostic unit 800 fixed to the moving part 305 can perform an impact inspection of the surface condition of the object to be inspected 10 while also performing a scanning movement.

[0052] The diagnostic unit 800 moves according to the scan coordinate p(x, y; T) shown in FIG. 4 and performs impact inspection for each movement amount Δx and Δy, thereby making it possible to associate the inspection result with the impact position.

[0053] 16, in the present embodiment, the percussion unit 800 has a striking section 802 fixed in a housing 801, the striking section 802 being made of a solenoid, which, for example, when the solenoid is energized, pushes a plunger 803 downward with a predetermined force, and when the energization is stopped, pulls the plunger 803 upward by means of a spring or the like. An impact hammer 804 is provided at the lower end of the plunger 803, and movably passes through an opening 806 provided in the center of a base 805 on the bottom surface of the housing 801. When the solenoid is energized, the impact hammer 804 can strike the inspection surface of the inspection object 10 through the opening 806.

[0054] Microphones 809 are provided on the four side surfaces of the bottom end of the housing 801 via anti-vibration rubber 808, and a microphone cover 810 is provided on top of these. The microphones 809 detect the impact sound from the inspection surface when the impact is made. Note that, although four microphones 458 are provided in this example, the number is not limited to this.

[0055] When the percussion unit 800 is moved to the position to be inspected by the xy actuator 300, the striking hammer 804 strikes the inspection surface. The striking sound produced by striking the inspection surface at this time is detected by the microphone 458, and the state of the inspection surface, such as peeling, can be evaluated from this striking sound detection signal.

[0056] It is assumed here that the camera 200 is placed about 15 cm away from the surface of the inspection object 10 and can capture images of tile frames of sufficient size and number to enable detection of tile areas.

[0057] 2.2) Tile Region Detection The tile area detection unit 702 receives an image at a predetermined position from the image position synchronization unit 107, and detects the tile area to be inspected by image processing. An example of the tile area detection procedure will be described below with reference to Figs.

[0058] In Fig. 17, the tile area detection unit 702 extracts horizontal and vertical edges from the input image, and detects the rough position of the tile frame from this edge information (operation 901). For example, as illustrated in Fig. 18, when the camera 200 captures an image of an inspection target 10 on which tiles are arranged in an imaging range 200a, a horizontal edge extraction image illustrated in Fig. 19(A) and a vertical edge extraction image illustrated in Fig. 19(B) can be obtained. From these edge extraction images, a rough area where a rectangular tile exists, for example, the area indicated by reference number 10a, can be obtained. As an alternative method, a rough area can be specified by detecting parts corresponding to the corners of the tiles from the input image.

[0059] Next, the tile area detection unit 702 applies a rectangle detection algorithm to the area 10a to accurately detect a rectangular area (operation 902). As the rectangle detection algorithm, an algorithm such as Harris' corner detection can be used. By using the rectangle detection algorithm, it is possible to identify a tile area 10b as shown in FIG. 19(D).

[0060] Returning to Fig. 17, the tile region detection unit 702 determines whether the tile region 10b is the same as the region detected from the immediately preceding captured image (operation 903). Whether the tile regions are the same can be determined, for example, by the degree of overlap (area) between the tiles. If there are identical tile regions, the best tile image is determined based on any one of image quality, brightness unevenness, and the size of the specified tile, or a combination of these (operation 904).

[0061] 2.3) Percussion information and imaging diagnostic information Once each tile region has been determined from the imaging range 200a in this manner, the crack detection unit 703 performs crack detection as exemplified in the next section, and stores the detection results together with the position coordinates of the tiles in the diagnosis information storage unit 704. In addition, the percussion evaluation results obtained by the diagnosis unit 800 as described above are also stored in the diagnosis information storage unit 704 together with the position coordinates.

[0062] 2.4) Crack detection As described above, the crack detection unit 703 performs crack detection by the image processing shown in FIGS.

[0063] FIG. 20 shows an outline of the crack detection procedure. First, the crack detection unit 703 performs pre-processing to binarize the raster image of the imaging range 200a including multiple tile areas (operation 910), and then vectorizes the binarized image (operation 911). The vectorized image is subjected to noise removal and, if multiple cracks are present, a process to reconnect them (operation 912). The crack detection results thus obtained are stored in the diagnosis information storage unit 704 together with the coordinates of the tile areas (operation 913). Each processing operation of the crack detection unit 703 will be described in detail below.

[0064] <Binarization processing> 21, a raster image 920 of the imaging range 200a is input to the crack detection unit 703. Next, the raster image 920 is grayscaled, and local black areas are removed from the resulting grayscale image 921 to generate a background image 922. Next, the background image 922 is subtracted from the grayscale image 921 to generate a difference image 923, which is then binarized to generate a binarized image 924. In the following, to simplify the explanation, the vectorization will be explained by taking a raster image 924A near the crack.

[0065] <Vectorization processing> As shown in FIG. 22, the crack detection unit 703 thins the raster image 924A to generate a thinned image 924B, and then performs vectorization on the thinned image 924B. This results in a vector image 924C in which an arbitrary curve is expressed as a large number of vectorized line segments. Other parts of the raster image 924 are similarly vectorized. Here, if continuous lines C(1), C(2), C(3), etc. exist discretely in the vector image 924C, the crack detection unit 703 removes lines shorter than a predetermined value (here, C(4) and C(5)) as noise by the crack reconnection process described below, and then detects a continuous curve from the discrete continuous lines as a crack. An example of the crack reconnection process will be described below with reference to FIG. 23 and FIG. 24.

[0066] <Reconnection of cracks> In Fig. 23, the crack detection unit 703 preliminarily specifies a threshold value for the length of a vector, and after eliminating vectors shorter than the threshold value as noise, selects a crack line having an end point from the vector image of the imaging range 200a exemplified in vector image 924C (operation 931). Hereinafter, the end point of the selected crack line is referred to as a target end point. Hereinafter, the curve C0 in Fig. 24 is selected, and its end point P0 becomes the target end point.

[0067] Next, the crack detection unit 703 searches for endpoints of other crack lines within a range from the target endpoint of the crack line that satisfies the following conditions a) and b) (operation 932). Condition a) The crack line is on the extension side of the line segment of the target end point of the crack line and is within a specified angle centered on the target end point, in other words, it is in a position facing the target end point; and Condition b) Within a specified distance from the target end point of the crack line. According to FIG. 24, the search range R that satisfies conditions a) and b) is the line segment S of length L at the end of the curve C0. C0 The extension direction of the curve C0 is the center line, and the range is less than the angle α centered on the target end point P0 of the curve C0 (condition a), and the line segment S C0In other words, the search range R is a range (condition b) within a distance (n × L) of n times (a real number equal to or greater than 1) the length L of the curve C0. C0 The shape of the ring has a central angle α and a radius proportional to the length of the ring.

[0068] The crack detection unit 703 determines whether or not there are other crack lines that match the results of the above search (operation 933), connects the target end point with the end point of the other crack line that matches, and registers them as one crack line (operation 934). Note that if there are multiple other end points within the search range, it is sufficient to select the other end point that is closest to the target end point.

[0069] 24, the end point P1 of curve C1 and the end point P2 of curve C2 are present within the search range R, so these. A portion of curve C3 is within the search range R, but its end point P3 is not within the search range R, so it is excluded from the search target. In this example, the end point P2 of curve C2 is closer to the target end point P0 than the end point P1 of curve C1, so the crack detection unit 703 connects the target end point P0 to the end point P2 of curve C2 and registers them as one crack line.

[0070] 23, if there are no other crack lines (NO in operation 933), or following operation 934, it is determined whether there are any unsearched endpoints (operation 935). If there are any unsearched endpoints (YES in operation 935), the crack detection unit 703 repeats the above operations 931 to 935 until the search is complete, and when the search for all endpoints is completed (NO in operation 935), the crack reconnection process is terminated.

[0071] The crack detection unit 703 stores the registered crack detection results together with the coordinates of the tile area in the diagnosis information storage unit 704. In this way, it becomes possible to perform a comprehensive evaluation of cracks in the imaging range 200a. Here, as exemplified by the vector image 925 in Fig. 22, it is possible to detect one crack line C that spans multiple tiles, and it is possible to extract cracks that would have been overlooked if the cracks were evaluated for only one tile.

[0072] 2.4) Effects As described above, according to the second embodiment of the present invention, even in an asynchronous system, it is possible to associate the video image of camera 200 with the coordinates indicating the camera position, as in the first embodiment, and further to detect and evaluate tile areas from the tile images corresponding to the position coordinates.

[0073] In addition, the image diagnosis result based on the image captured by the camera 200 and the impact diagnosis result by the diagnostic unit 800 are stored in the diagnostic information storage unit 704 together with the position coordinates, so that the surface condition of the inspection target 10 consisting of a tiled wall surface can be evaluated by the image and tapping. In particular, by extracting cracks across multiple tiles, it is possible to extract, for example, long cracks caused by cracks on the frame side of the inspection target, which makes it easier to select a repair method. For example, there is an advantage in that it becomes easier to select a repair method, whether to repair only the tiles or to repair to the base. [Industrial Applicability]

[0074] The present invention is applicable to a condition diagnostic system for the wall or floor of a building. [Explanation of symbols]

[0075] 10. Inspection object 100 Condition diagnostic device 101 Interface 102 Video storage unit 103 xy movement control section 104 Coordinate acquisition unit 105 Movement detection unit 106 Movement direction change detection unit 107 Image position synchronization unit 108 Evaluation Department 109 Processor (Data Processing Unit) 200 Cameras 200a Imaging range 201 Imaging unit 202 Interface 300 xy actuator 305 Mobile Division 307 Imaging range 401a Trajectory of movement obtained from video images 401b Scan coordinate trajectory 402a, 403a Movement direction change point 402b, 403b Scan turnaround point (coordinate direction change point) 701 Processor (Data Processing Unit) 702 Tile Region Detection Unit 703 Crack detection unit 704 Diagnostic Information Storage Unit 705 Percussion control section 706 Consultation Evaluation Department 800 Diagnostic Unit

Claims

1. a moving mechanism that moves the moving part along the surface of the inspection object within an imaging range; a camera attached to the moving part for capturing a moving image of a surface of the inspection object; a coordinate acquisition unit that acquires position coordinates of the moving unit within the image capture range; a data processing unit for detecting a test image from the moving image; Equipped with The data processing unit: Detecting a change point in the moving direction of the camera based on a plurality of images of the video; a point at which the direction of change of the position coordinates of the moving part obtained from the coordinate acquisition unit turns around and a point at which the direction of movement of the camera changes, thereby detecting an inspection image corresponding to a desired position coordinate from the moving image; A condition diagnosis system comprising:

2. The condition diagnosis system according to claim 1, characterized in that the movement direction change point is a time point in the image of interest at which the movement direction of a specific image portion contained in the image of interest and two images that are temporally before and after the image of interest changes by more than a predetermined angle.

3. 3. The condition diagnosis system according to claim 1, further comprising: extracting an edge portion from the inspection image; and extracting a tile region of the object to be inspected from the edge portion by a rectangle detection algorithm.

4. The condition diagnosis system according to any one of claims 1 to 3, characterized in that the data processing unit evaluates the surface condition of the object to be inspected based on a tile area extracted from the inspection image.

5. The condition diagnosis system according to claim 4 , wherein the data processing unit detects crack lines in at least one tile region from the inspection image by image processing.

6. The condition diagnosis system described in claim 5, characterized in that the data processing unit connects the end points of the first crack line and the end points of the second crack line if they are within a predetermined distance and a predetermined angle, and detects the first and second crack lines as a single crack line.

7. The condition diagnosis system according to any one of claims 1 to 6, further comprising a percussion unit attached to the moving part, the percussion unit striking a surface of the object to be inspected and outputting the striking sound to the data processing part as an inspection signal.

8. a movement control unit that controls a movement mechanism that moves the movement unit along the surface of the inspection object within an image capturing range; an interface for connecting, by wire or wirelessly, to a camera attached to the moving part and configured to capture a moving image of the surface of the inspection object; a coordinate acquisition unit that acquires position coordinates of the moving unit within the image capture range; a data processing unit for detecting a test image from the moving image; Equipped with The data processing unit: Detecting a change point in the moving direction of the camera based on a plurality of images of the video; a point at which the direction of change of the position coordinates of the moving part obtained from the coordinate acquisition unit turns around and a point at which the direction of movement of the camera changes, thereby detecting an inspection image corresponding to a desired position coordinate from the moving image; A condition diagnosis device characterized by:

9. 1. A method for identifying an image position in a condition diagnosis device having a movement control unit that controls a movement mechanism that moves a movement unit along a surface of an inspection object within an image capturing range, and an interface that is attached to the movement unit and connects, by wire or wirelessly, to a camera that captures a moving image of the surface of the inspection object, comprising: a coordinate acquisition unit acquires position coordinates of the moving unit within the image capture range; The data processing unit Detecting a change point in the moving direction of the camera based on a plurality of images of the video; identifying an image corresponding to a desired position coordinate from the video by associating a point at which the direction of change of the position coordinates of the moving part, obtained from the coordinate acquisition unit, turns around with a point at which the direction of movement of the camera changes; 13. A method for identifying an image location, comprising:

10. A program for making a computer function as a condition diagnosis device having a movement control unit that controls a movement mechanism that moves a movement unit along a surface of an object to be inspected within an imaging range, and an interface that is attached to the movement unit and connects, by wire or wirelessly, to a camera that captures a video image of the surface of the object to be inspected, a function of acquiring position coordinates of the moving unit within the image capture range; a function of detecting a change point in the moving direction of the camera based on a plurality of images of the moving image; a function of identifying an image corresponding to a desired position coordinate from the video by associating a point where a direction of change in the position coordinates of the moving part turns around with a point where a direction of movement of the camera changes; A program for causing the computer to execute the above.

Images