DETECTION APPARATUS, DETECTION METHOD, AND PROGRAM

Abstract

To improve the accuracy of detection of a junction portion of metal material.SOLUTION: A detection device 300 performs, on an image 400 of the current frame having been photographed with a camera 120, image processing for detection of a candidate for a blob 730 indicating a punch hole PH provided in a steel sheet M in order to indicate the position of a weld zone WP and consequently acquires a feature image 630 for the current frame. The detection device 300 distinguishes whether or not the blob 730 detected from the feature image 630 of the current frame is a blob indicating the weld zone WP, on the basis of a mutual positional relationship between the feature image 630 of the current frame and a blob 730 detected from each of feature images of preceding frames. The detection device 300 outputs information indicating that the weld zone WP has been detected, when the blob 730 detected from the feature image 630 of the current frame is a blob 730d, 730h indicating the weld zone WP.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a detection device, a detection method, and a program, and is particularly suitable for use in detecting a joint in a metal material. [Background technology]

[0002] In a continuous processing line, the rear end of a preceding metal material and the front end of a succeeding metal material are joined by welding or the like, and processing is performed continuously on a plurality of metal materials. As a technology for detecting a joint during the conveyance of metal materials in such a continuous processing line, there is a technology described in Patent Document 1. In the technology described in Patent Document 1, a joint of a metal material is detected as follows. First, a plurality of punch holes having a preset shape and size are formed so as to be positioned on a straight line parallel to the conveyance direction of the steel plate. Then, the plurality of punch holes are photographed by a CCD camera installed so as to be able to detect the plurality of punch holes simultaneously. Of the photographed images, image processing is performed only on the areas near the punch holes, and a matching process is performed between the images of each area near the punch holes and the preset punch hole shape. When punch holes are detected in each area, it is determined that a weld has been detected. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2000-233220 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology described in Patent Document 1, unless a plurality of punch holes are formed in the steel plate so as to be located on a straight line parallel to the conveying direction of the steel plate, the welded portion cannot be detected. Therefore, the positions and number of punch holes formed in the steel plate are limited. In addition, a matching process is performed between the image of each area near the punch hole and a preset punch hole shape. Therefore, the shape of the punch hole is limited to the steel plate, and if the punch hole formed in the steel plate is different from the preset punch hole shape, the welded portion cannot be detected.

[0005] The present invention has been made in consideration of the above problems, and has an object to improve the detection accuracy of a joint in a metal material. [Means for solving the problem]

[0006] The detection device of the present invention is a detection device that detects joints in metal material using a camera, and comprises an acquisition unit that acquires multiple images taken by the camera at different times with the metal material being transported as the image subject, and a judgment unit that judges the presence or absence of the joint based on the multiple images.The judgment unit comprises an image processing unit that acquires characteristic images for each of the multiple images by performing image processing on each of the multiple images to detect, as detected objects, candidate landmarks provided on the metal material to indicate the position of the joint, a judgment unit that judges whether the detected object is the landmark based on the mutual positional relationship of the detected objects detected from each of two or more of the characteristic images, and an output unit that outputs information indicating that the joint has been detected when the detected object is judged to be the landmark.

[0007] The detection method of the present invention is a detection method for detecting joints in metal material using a camera, and includes an acquisition step of acquiring multiple images taken by the camera at different times with the metal material being transported as the image subject, and a judgment step of judging the presence or absence of the joint based on the multiple images. The judgment step includes an image processing step of acquiring characteristic images for each of the multiple images by performing image processing on each of the multiple images to detect, as detected objects, candidates for landmarks provided on the metal material to indicate the position of the joint, a judgment step of judging whether the detected object is the landmark based on the mutual positional relationship of the detected objects detected from each of two or more of the characteristic images, and an output step of outputting information indicating that the joint has been detected when the detected object is judged to be the landmark.

[0008] The program of the present invention causes a computer to function as each part of the detection device. Effect of the Invention

[0009] According to the present invention, it is possible to improve the detection accuracy of a joint in a metal material. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of a configuration of a continuous annealed steel sheet production line. [Diagram 2] FIG. 2 is a diagram illustrating an example of a camera arrangement. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a detection device. [Figure 4] FIG. 2 is a schematic diagram showing an example of an image of a current frame. [Diagram 5] 11A and 11B are diagrams illustrating an example of expansion processing and contraction processing. [Figure 6] 1A to 1C are diagrams conceptually illustrating an example of a binarized image of a detection region and a feature image of the detection region. [Figure 7] FIG. 10 is a diagram illustrating an example of a process for extracting a blob. [Figure 8]FIG. 13 is a diagram showing an example of a blob extracted from a feature image of the current frame. [Figure 9] FIG. 13 is a diagram showing an example of the positional relationship between a blob in a current frame and a blob in a previous frame in the conveying direction (y-axis direction) of the steel plate. [Figure 10] FIG. 13 is a diagram showing an example of the positional relationship between a blob in a current frame and a blob in a previous frame in the width direction (x-axis direction) of the steel plate. [Figure 11] 4 is a flowchart illustrating an example of a detection method according to the first embodiment. [Figure 12] 12 is a flowchart continuing from FIG. 11. [Figure 13] 13A and 13B are diagrams illustrating an example of processing by the output unit when a blob indicating a punch hole PH is included in the blobs of the current frame. [Figure 14] FIG. 4 is a diagram showing an example of a display screen. [Figure 15] 10 is a flowchart illustrating an example of a detection method according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In addition, the fact that the objects to be compared are the same in terms of length, position, size, interval, etc. includes not only the case where they are exactly the same, but also the case where they are different within the scope of the gist of the invention (for example, the case where they are different within the tolerance range determined at the time of design). In addition, in each figure, the xyz coordinates indicate the relationship of the directions in each figure, and the origin of the xyz coordinates is not limited to the position shown in each figure. As a symbol for expressing the xyz coordinates, a symbol with a ● inside a circle indicates that the direction from the back side to the front side of the paper is the axis indicating the positive direction.

[0012] (First embodiment) First, a first embodiment of the present invention will be described. <Continuous processing line> In this embodiment, a case where the detection device is applied to a continuous processing line is illustrated. In the continuous processing line, the rear end of a preceding metal material and the front end of a succeeding metal material are joined to continuously process a plurality of metal materials. In this embodiment, a continuous annealed steel sheet production line is illustrated as an example of such a continuous processing line. FIG. 1 is a diagram showing an example of the configuration of a continuous annealed steel sheet production line to which the detection device of this embodiment is applied.

[0013] In FIG. 1, a hot-rolled coil manufactured in a hot rolling process is attached to a pay-off reel 101. A steel sheet unwound from the hot-rolled coil attached to the pay-off reel 101 is supplied to a production line. The steel sheet unwound from the hot-rolled coil passes through a deflector roll 110a and is then transported to a welding facility 102. The welding facility 102 welds and joins the leading end of the steel sheet unwound from the pay-off reel 101 to the trailing end of a steel sheet transported ahead of the steel sheet. In the welding facility 102, for example, flash butt welding is performed. In addition, in the welding facility 102, a mark is provided on the steel sheet to indicate the position of the welded portion. Note that the mark may be provided on the steel sheet outside the welding facility 102. In this embodiment, a case is illustrated in which the mark provided on the steel sheet to indicate the position of the welded portion is a hole formed in the steel sheet, as in Patent Document 1. In the following description, this hole is referred to as a punch hole. The punch hole is a through hole that penetrates one surface of the steel plate and the other surface of the steel plate. The mark provided on the steel plate to indicate the position of the weld is not limited to a hole. For example, the mark may be applied to the surface of the steel plate M with black paint.

[0014] The steel sheet joined to the preceding steel sheet in the welding equipment 102 is transported to the pickling equipment 103. The pickling equipment 103 pickles the transported steel sheet and removes scale (oxide) adhering to the surface of the steel sheet. The steel sheet pickled in the pickling equipment 103 is transported to the cold rolling equipment 104. The cold rolling equipment 104 cold rolls the transported steel sheet. The cold rolling equipment 104 is equipped with a plurality of cold rolling mills (rolling stands). The steel sheet transported to the cold rolling equipment 104 is passed through the plurality of cold rolling mills in sequence and cold rolled.

[0015] The steel sheet cold rolled in the cold rolling equipment 104 passes through deflector rolls 110b and 110c and is transported to the annealing equipment 105. The annealing equipment 105 continuously anneals the transported steel sheet. The steel sheet continuously annealed in the annealing equipment 105 passes through a deflector roll 110d, and is then wound up by a tension reel 106 to become an annealed coil.

[0016] The continuous annealed steel sheet production line itself can be realized by known techniques and is not limited to the one exemplified in Fig. 1. For convenience of illustration, Fig. 1 shows only deflector rolls (deflector rolls 110a to 110d, etc.) for changing the direction of the steel sheet being transported among the rolls provided in the continuous annealed steel sheet production line, but the continuous annealed steel sheet production line includes various rolls (not shown) such as transport rolls for transporting the steel sheet. The continuous annealed steel sheet production line also includes a looper (not shown) for temporarily storing the steel sheet being transported in order to continuously process a plurality of steel sheets.

[0017] Here, when a steel sheet with a defective weld is cold-rolled in the cold rolling equipment 104, cracks may occur on the edge of the weld in the width direction (x-axis direction). This crack is called an edge crack. An edge crack is a sawtooth-shaped crack that occurs on the edge of the steel sheet in the width direction. The length of the edge crack in the width direction of the steel sheet is about several mm to several tens of mm. An edge crack is also called an edge cut or a cut. When a steel sheet with a crack in the weld is continuously annealed in the annealing equipment 105, there is a risk that a structure deformation will occur in the cracked part of the edge of the weld. For this reason, there is a risk that the quality of the annealed coil will decrease, or that the steel sheet will break in the annealing equipment 105. If the steel sheet breaks in the annealing equipment 105, it is necessary to stop the operation.

[0018] Therefore, in this embodiment, an example is shown in which the surface of the steel plate being transported between the exit of the cold rolling equipment 104 and the entrance of the annealing equipment 105 is photographed by a camera 120 and the presence or absence of a weld is detected based on the photographed image so that an operator can determine the quality of the weld between the exit of the cold rolling equipment 104 and the entrance of the annealing equipment 105.

[0019] <Camera 120> Fig. 2 is a diagram showing an example of the arrangement of the camera 120. Fig. 2 illustrates a case where a welded portion WP joining the rear end of a leading steel sheet MF and the front end of a trailing steel sheet MA is being transported through a region between the deflector rolls 110b, 110c. In the following description, the leading steel sheet MF, the trailing steel sheet MA, and the welded portion WP are collectively referred to as the steel sheet M.

[0020] FIG. 2 illustrates a case where the camera 120 is arranged so as to capture an image of the surface of the steel sheet M being transported through an area between the deflector rolls 110b and 110c arranged between the exit of the cold rolling equipment 104 and the entrance of the annealing equipment 105. FIG. 2 also illustrates a case where the camera 120 is arranged so that the optical axis 120a of the camera 120 is perpendicular to the surface of the steel sheet M in order to reduce distortion of the image captured by the camera 120. As shown in FIG. 2, by arranging the camera 120 so that the optical axis 120a of the camera 120 is inclined at a predetermined angle θ with respect to the horizontal plane (ground), it is possible to prevent the shadow of the camera 120 itself from being reflected in the image captured by the camera 120. It is needless to say that the position of the camera 120 is determined so as not to interfere with the operation of the continuous annealed steel sheet production line.

[0021] The arrangement of the camera 120 for photographing the surface of the steel sheet M being transported in the area between the exit of the cold rolling equipment 104 and the entrance of the annealing equipment 105 is not limited to the arrangement exemplified in FIG. 2. For example, the camera 120 may be arranged so as to be able to photograph the surface of the steel sheet M being transported in the area between the deflector roll 110c and the entrance of the annealing equipment 105. The camera 120 may also be arranged so as to be able to photograph the surface of the steel sheet M being transported in the area between the exit of the cold rolling equipment 104 and the deflector roll 110b. However, in the cold rolling equipment 104, steam is generated due to cold rolling the steel sheet M using rolling oil. Therefore, it is preferable to arrange the camera 120 at a position away from the exit of the cold rolling equipment 104 so that the steam is not reflected as much as possible in the image taken by the camera 120.

[0022] Camera 120 is, for example, an area scan camera, and captures moving images at a predetermined frame rate. In this embodiment, a case is illustrated in which the image captured by camera 120 is a color image. The luminance value of each pixel in the image captured by camera 120 has, for example, 256 gradations (a range of 0 to 255).

[0023] In this embodiment, a case will be illustrated where a dedicated lighting device is not used to illuminate the shooting area of ​​the camera 120. The shooting area of ​​the camera 120 is illuminated by an existing lighting device in the continuous annealed steel sheet production line. The existing lighting device in the continuous annealed steel sheet production line is an electric light or the like attached to the ceiling of the factory. However, a dedicated lighting device may be used to illuminate the shooting area of ​​the camera 120.

[0024] As shown in Fig. 2, a video transmitter 130 is connected to the camera 120. The video transmitter 130 transmits images captured by the camera 120 to the video receiver 200 by wireless communication. The video receiver 200 receives the images transmitted from the video transmitter 130. A detection device 300 is connected to the video receiver 200, and images received by the video receiver 200 are output to the detection device 300. Note that images do not necessarily need to be transmitted and received between the video transmitter 200 and the video receiver 300 by wireless communication, and may be transmitted and received by wired communication.

[0025] <Detection device> 3 is a diagram showing an example of a functional configuration of the detection device 300. The hardware of the detection device 300 is realized by using, for example, an information processing device equipped with a processor (e.g., a CPU), a ROM, a RAM, and various interfaces, or dedicated hardware such as an ASIC (Application Specific Integrated Circuit). An example of the functions of the detection device 300 of this embodiment will be described below. For example, the detection device 300 includes an acquisition unit 310 and a determination unit 320 as shown in FIG. 3.

[0026] [Acquisition unit 310] The acquisition unit 310 acquires the image of each frame continuously captured by the camera 120 from the video receiver 200 and outputs it to the determination unit 320. In this embodiment, the horizontal direction of the image captured by the camera 120 corresponds to the width direction (x-axis direction) of the steel sheet M, and the vertical direction corresponds to the conveying direction (y-axis direction) of the steel sheet M. In addition, the downstream side of the conveying direction of the steel sheet M corresponds to the positive direction of the y-axis. In this embodiment, the acquisition unit 310 acquires the image of each frame from the video receiver 200 as an example of a captured image. However, this is not necessarily required. The image of each frame continuously captured by the camera 120 may be stored in a portable storage medium. In this case, the acquisition unit 310 may import the image of each frame stored in the storage medium. In addition, the image of each frame continuously captured by the camera 120 may be transmitted from the video transmitter 130 to a higher-level computer (processor) that manages the operation of the continuous annealed steel sheet production line. In this case, the acquisition unit 310 may receive the image of each frame from the host computer.

[0027] [Determination unit 320] The determination unit 320 determines the presence or absence of a welded portion WP based on the image of each frame acquired by the acquisition unit 310. In this embodiment, the determination unit 320 repeatedly executes the determination of the presence or absence of a welded portion WP based on the images of two consecutive frames in the order of shooting. In this embodiment, the determination unit 320 determines the presence or absence of a welded portion WP based on the presence or absence of a mark provided on the steel plate M to indicate the position of the welded portion WP. As described above, in this embodiment, the case where the mark provided on the steel plate M to indicate the position of the welded portion WP is a punched hole formed in the steel plate M is illustrated. As shown in FIG. 3, the determination unit 320 includes a trimming unit 321, an image processing unit 322, a discrimination unit 323, and an output unit 324. In the following description, of the two consecutive frames, the frame captured at a later timing is referred to as a current frame, and is the frame to be determined for the presence or absence of a punched hole (i.e., a welded portion WP). In addition, of the two consecutive frames, the frame captured at an earlier timing (the frame immediately before the current frame) is referred to as a previous frame.

[0028] [[Trimming section 321]] 4 is a schematic diagram showing an example of an image of the current frame acquired by acquisition section 310. An example of the function of trimming section 321 will be described with reference to FIG. The trimming unit 321 trims the detection area 410 from the image 400 of the current frame acquired by the acquisition unit 310. The detection area 410 is a part of the image acquired by the acquisition unit 310, and is an area assumed to be an area where the punch holes PH exist. The number, position, shape, and size of the punch holes PH to be provided in the steel sheet M are determined before the processing of the steel sheet M in the continuous annealed steel sheet production line starts. Therefore, the position, size, and shape of the detection area 410 in the image are determined before the processing of the steel sheet M in the continuous annealed steel sheet production line starts, based on, for example, the assumed values ​​of the position, size, and shape of the punch holes PH, the assumed value of the meandering amount of the steel sheet M between two consecutive frames, the length of the real space per pixel, and the assumed value of the conveying speed of the steel sheet M. The length of the real space per pixel is calculated, for example, based on the angle of view and the number of pixels of the camera 120, and the distance from the camera 120 to the steel sheet M. The assumed values ​​of the shape and size of the punched hole PH are those assumed as the shape and size after cold rolling in the cold rolling equipment 104. A specific example of how to determine the detection area 410 will be described below.

[0029] 4 illustrates an example in which the shape of the detection area 410 is rectangular. In this case, the center position in the horizontal direction (x-axis direction) of the detection area 410 is, for example, the horizontal position of a pixel corresponding to an assumed value of the position in the width direction of the steel plate where the punch hole PH is to be provided. The number of pixels in the horizontal direction of the detection area 410 can be obtained, for example, as follows. First, the estimated value of the meandering amount of the steel sheet M between two consecutive frames and the estimated value of the width of the punched hole PH are each converted into the number of pixels of the camera 120. Then, the number of pixels in the horizontal direction of the detection area 410 is determined to be equal to or greater than the sum of twice the number of pixels converted from the estimated value of the meandering amount of the steel sheet M between two consecutive frames and the number of pixels converted from the estimated value of the width of the punched hole PH.

[0030] The number of pixels in the vertical direction (y-axis direction) of the detection area 410 can be obtained, for example, as follows. First, the moving distance of the steel sheet M during the time when k or more consecutive frame images are captured (k is an integer of 2 or more) and the estimated value of the length of the punch hole PH are converted into the number of pixels of the camera 120. Then, the number of pixels in the vertical direction (y-axis direction) of the detection area 410 is determined to be equal to or greater than the sum of the number of pixels converted from the moving distance of the steel sheet M during the time when k or more consecutive frame images are captured and the number of pixels converted from the estimated value of the length of the punch hole PH. The moving distance of the steel sheet M during the time when k or more consecutive frame images are captured is expressed as the product of the time and the estimated value of the conveying speed of the steel sheet M. Instead of the estimated value of the conveying speed of the steel sheet M, a measured value of the conveying speed of the steel sheet M may be used. In this case, the number of pixels in the vertical direction of the detection area 410 is determined after the processing of the steel sheet M in the continuous annealed steel sheet production line has started. In addition, increasing the value of k increases the time during which the same punch hole PH is included in the detection area 410. As described above, by determining the detection area 410, it is possible to prevent a situation in which a punch hole PH provided in the transported steel sheet M passes by without being imaged, and to reliably trim the image portion including the punch hole PH.

[0031] For the sake of simplicity, a case will be exemplified in which one punch hole PH is provided for one welded portion WP and one detection area 410 is set for one welded portion WP. However, there may be a case in which a plurality of punch holes PH are provided for one welded portion WP. In this case, it is preferable to set a different detection area 410 for each punch hole PH. If there are a plurality of detection areas 410, the subsequent processing of the image processing unit 322 and the discrimination unit 323 is performed individually for each of the plurality of detection areas 410. In the example shown in FIG. 4, the punch hole PH is provided in the trailing steel sheet MA, but this is not limited thereto, and the punch hole PH may be provided in the leading steel sheet MF in addition to or instead of the trailing steel sheet MA. In addition, in this embodiment, a case will be exemplified in which the detection area 410 is not displayed in the image 400 of the current frame (the dashed line indicating the detection area 410 in FIG. 4 is a virtual line and is not actually displayed). However, the detection area 410 may be displayed superimposed on the image 400 of the current frame.

[0032] [[image processing unit 322]] The image processing unit 322 executes image processing for detecting candidates for punch holes PH as detected objects on the image of the detection area 410 of the current frame, thereby acquiring a feature image for the image of the detection area 410 of the current frame. In this embodiment, the image processing unit 322 includes a filter unit 322a, a grayscaling unit 322b, an adaptive binarization processing unit 322c, and a morphology processing unit 322d.

[0033] [[[Filter section 322a]]] The filter unit 322a smoothes the luminance of the image of the detection region 410 of the current frame. In this embodiment, the filter unit 322a performs smoothing using a Gaussian filter. The smoothing using a Gaussian filter is performed by performing a convolution operation between a Gaussian kernel and the luminance value of a pixel of interest. If the pixel position in the x-axis direction (horizontal direction) is x, the pixel position in the y-axis direction (vertical direction) is y, and the standard deviation is σ, the Gaussian kernel G(x,y) is expressed by, for example, the following formula (1). Note that the standard deviation σ is a parameter for adjusting the value of the Gaussian kernel G(x,y) and is set in advance. The convolution operation is expressed by the following formula (2). In formula (2), I'(x,y) is the luminance value of the pixel of interest after the convolution operation. (x,y) is the pixel position (xy coordinates) of the pixel of interest. The luminance value I'(x,y) of the pixel of interest after the convolution operation becomes the luminance value of the pixel of interest after smoothing. G(i,j) is the value of the Gaussian kernel at pixel position (i,j). For example, when the Gaussian kernel is a 3×3 rectangular kernel, −1, 0, and 1 are used as i and j in equation (2), respectively.

[0034]

number

[0035] The filter used for smoothing the luminance of the image is preferably a Gaussian filter. This is because the weight (kernel value) for the luminance value of a pixel located relatively close to the pixel of interest can be made larger than the weight for the luminance value of a pixel located relatively far from the pixel of interest. However, the filter used for smoothing the luminance of the image is not limited to a Gaussian filter. For example, a known smoothing filter other than a Gaussian filter, such as a moving average filter, may be used.

[0036] [[[Grayscaling unit 322b]]] As described above, in this embodiment, the image acquired by the acquisition unit 310 is a color image. Therefore, the image of the detection region 410 of the current frame smoothed by the filter unit 322a is also a color image. The grayscaling unit 322b converts the image of the detection region 410 of the current frame smoothed by the filter unit 322a into a grayscale image. The grayscaling unit 322b converts the image of the detection region 410 of the current frame smoothed by the filter unit 322a into a grayscale image, for example, by the following formula (3).

[0037]

number

[0038] Here, Gray is the luminance value of the grayscaled pixel. R, B, and G are the red, blue, and green luminance values ​​of the pixel smoothed by the filter unit 322a, respectively. The grayscale unit 322b performs the calculation of formula (3) for each pixel of the image of the detection area 410 of the current frame smoothed by the filter unit 322a, for example.

[0039] [[[adaptive binarization processing unit 322c]]] As described above, in this embodiment, a dedicated lighting device for illuminating the shooting area of ​​the camera 120 is not used, and the shooting area of ​​the camera 120 is illuminated by an existing lighting device in the continuous annealed steel sheet production line. In addition, the punch hole PH is a through hole. Therefore, if the area of ​​the through hole is included in the image of the detection area 410 of the current frame, it is assumed that the area of ​​the through hole is darker than the area of ​​the steel sheet M around the through hole. Therefore, the area of ​​the punch hole PH can be detected by binarizing the luminance value of the image of the detection area 410 of the current frame that has been grayscaled by the grayscaling unit 322b. When binarizing, the luminance value of the pixel of interest (the pixel to be binarized) is compared with a threshold value.

[0040] The inventors have found that non-detection and over-detection of punch holes PH can be reduced by adaptively changing the threshold value to be compared with the luminance value of the pixel of interest and the luminance values ​​of the pixels surrounding the pixel of interest (hereinafter referred to as adaptive binarization processing) in accordance with the luminance value of the pixel of interest and the luminance values ​​of the pixels surrounding the pixel of interest, rather than by making the threshold value the same for all pixels of the image. Non-detection refers to not detecting the punch hole PH even though the punch hole PH exists, and over-detection refers to detecting the punch hole PH even though the punch hole PH does not exist. Therefore, the adaptive binarization processing unit 322c binarizes the luminance value of the image of the detection area 410 of the current frame that has been grayscaled by the grayscaling unit 322b, by adaptive binarization processing. The grayscaling unit 322b calculates a threshold value t(x, y) for the luminance value of the pixel of interest, for example, by the following formula (4). Note that (x, y) is the pixel position (xy coordinates) of the pixel of interest.

[0041]

number

[0042] Here, GI(x+i,y+j) is the luminance value of the pixel at pixel position (x+i,y+j) of the image of the detection area 410 of the current frame grayscaled by the grayscale conversion unit 322b. N is the number of pixels included in the kernel. When the kernel is a 3×3 rectangular kernel, N is 9, and i and j in the formula (4) are set to −1, 0, and 1, respectively. The calculation formula for the threshold value t(x,y) used in the adaptive binarization process is not limited to the formula (4). For example, instead of GI(x+i,y+j) in the formula (4), a value obtained by multiplying GI(x+i,y+j) and a weight ω according to a Gaussian function may be used. For example, the formula (1) can be used as the Gaussian function. In this case, the calculation formula for t(x,y) is as follows: formula (5). In this way, the weighting for the luminance value of a pixel that is relatively close to the pixel of interest is made greater than the weighting for the luminance value of a pixel that is relatively far from the pixel of interest, and the threshold value t(x, y) for the luminance value of the pixel of interest can be calculated.

[0043]

number

[0044] If the luminance value of the pixel of interest exceeds the threshold value t(x, y), the adaptive binarization processor 322c sets the pixel value of the pixel of interest to 1, and if not, sets the pixel value of the pixel of interest to 0. The adaptive binarization processor 322c sets each pixel of the image of the detection area 410 of the current frame that has been grayscaled by the grayscaling unit 322b as a pixel of interest, and binarizes the pixel value of each pixel as described above to generate a binary image. For the reasons described above, it is preferable to execute adaptive binarization as the binarization process, but it is also possible to execute a known binarization process other than adaptive binarization, such as Otsu's binarization.

[0045] [[[Morphology processing unit 322d]]] The morphology processor 322d executes morphology processing on the binarized image of the detection area 410 of the current frame generated by the adaptive binarization processor 322c. The morphology processing includes dilation processing and erosion processing. FIG. 5 is a diagram illustrating an example of the expansion process and the contraction process. The expansion process is a process in which, if there is at least one pixel having a second pixel value around a pixel of interest having a first pixel value, the pixel value of the pixel of interest is changed from the first pixel value to the second pixel value. In this embodiment, a case is illustrated in which the first pixel value is 1 and the second pixel value is 0. Also, in Figs. 5 to 10, a case is illustrated in which the pixel value of a pixel shown in white is 1 and the pixel value of a pixel shown in black is 0.

[0046] Fig. 5(a) is a diagram for explaining an example of the expansion process of this embodiment. Fig. 5(a) illustrates a case where the kernel is a 3 x 3 rectangular kernel. In the upper diagram of Fig. 5(a), the pixel value of a pixel of interest 511 is 1, and the pixel value of a pixel 512 in the rectangular kernel centered on the pixel of interest 511 is 0. In this case, as shown in the lower diagram of Fig. 5(a), the pixel value of the pixel of interest 511 is changed from 1 to 0.

[0047] The contraction process is a process in which, if there is at least one pixel having a first pixel value around a pixel of interest having a second pixel value, the pixel value of the pixel of interest is changed from the second pixel value to the first pixel value. As described above, in this embodiment, the case where the first pixel value is 1 and the second pixel value is 0 is illustrated. FIG. 5(b) is a diagram for explaining an example of the contraction process of this embodiment. Similarly to FIG. 5(a), FIG. 5(b) also illustrates a case where the kernel is a rectangular kernel of 3×3. In the upper diagram of FIG. 5(b), the pixel value of the pixel of interest 521 is 0, and the pixel value of the pixel 522 in the rectangular kernel centered around the pixel of interest 521 is 1. In this case, as shown in the lower diagram of FIG. 5(b), the pixel value of the pixel of interest 521 is changed from 0 to 1.

[0048] The morphology processor 322d performs opening and closing on the binarized image of the detection region 410 of the current frame generated by the adaptive binarization processor 322c. The opening is a process of performing an erosion process and an expansion process in this order. When the erosion process is performed multiple times, the erosion process is performed multiple times, and then the expansion process is performed the same number of times as the erosion process. The closing is a process of performing an expansion process and an erosion process in this order. When the expansion process is performed multiple times, the erosion process is performed multiple times, and then the erosion process is performed the same number of times as the erosion process. The morphology processor 322d performs opening and closing on the binarized image of the detection region 410 of the current frame generated by the adaptive binarization processor 322c, thereby generating a feature image of the detection region 410 of the current frame.

[0049] FIG. 6 is a diagram conceptually showing an example of a binarized image of the detection area 410 in the current frame and a feature image of the detection area 410 in the current frame. In FIG. 6(a), a binarized image 610 of detection area 410 in the current frame includes areas 611 and 612 where a small number of pixels with a pixel value of 0 exist dispersedly, and an area 613 where pixels with a pixel value of 0 exist in a cluster.

[0050] By performing closing on binarized image 610 of detection region 410 of the current frame shown in Fig. 6(a), the pixel values ​​of pixels in region 613 that have a pixel value of 1 are changed to 0. Then, as shown in Fig. 6(b), in binarized image 620 of detection region 410 of the current frame after the closing process, region 613 in binarized image 610 of detection region 410 of the current frame shown in Fig. 6(a) becomes region 621. By performing closing in this manner, it is possible to make clearer the region that should be detected as a region that may be a candidate for punch hole PH (a region where pixels with a pixel value of 0 exist in clusters) than in binarized image 610 of detection region 410 of the current frame.

[0051] Moreover, by performing opening on a binarized image 620 of the detection area 410 of the current frame after the closing process shown in FIG. 6(b), the pixel values ​​of the pixels in the areas 611 and 612 are changed to 1. Then, as shown in FIG. 6(c), the binarized image 630 of the detection area 410 of the current frame after the closing and opening process does not include the areas 611 and 612 (i.e., the areas 611 and 612 are removed). By performing opening in this manner, it is possible to leave areas that should be detected as areas that can be candidates for punch holes PH (areas where pixels with a pixel value of 0 exist together) from the binarized image 610 of the detection area 410 of the current frame, and to remove other areas as noise. In the following description, the binarized image 630 of the detection area 410 of the current frame after the closing and opening process is referred to as a feature image 630 of the detection area 410 of the current frame.

[0052] In this example, the opening is performed after the closing, but the opening may be performed before the closing. Also, instead of the opening and closing, for example, a morphological gradient (a process of subtracting an image after the contraction process from an image after the expansion process) may be performed. By performing the morphological gradient, it is possible to detect the contour of an area to be detected as an area of ​​pixels with a pixel value of 0 from the binarized image 610 of the detection area 410 of the current frame.

[0053] [[Discrimination unit 323]] The discrimination unit 323 discriminates whether or not a detected object included in the feature image 630 of the detection region 410 of the current frame is a punch hole PH, based on the feature image 630 of the detection region 410 of the current frame generated by the morphology processing unit 322d and the feature image of the detection region of the previous frame. Every time the feature image 630 of the detection region 410 of a new frame is generated by the morphology processing unit 322d, the discrimination unit 323 discriminates whether or not a detected object included in the feature image 630 of the detection region 410 of the new frame (i.e., the current frame) is a punch hole PH. In the following description, the feature image of the detection region 410 will be simply referred to as the feature image.

[0054] The discrimination unit 323 discriminates whether or not the detected object included in the feature image 630 of the current frame is a punch hole PH based on the relative positional relationship of the detected object detected from each of the feature image 630 of the current frame and the feature image of the previous frame. In this embodiment, in order to improve the accuracy of discriminating whether or not the detected object is a punch hole PH, the discrimination unit 323 discriminates whether or not the detected object included in the feature image 630 of the current frame is a punch hole PH using not only the relative positional relationship detected from each of the feature image 630 of the current frame and the feature image of the previous frame, but also the area of ​​the detected object included in the feature image 630 of the current frame.

[0055] An example of the functions of the discrimination unit 323 of this embodiment will be described below. [[[Extract Blob]]] The discrimination unit 323 extracts blobs from the feature image 630 of the current frame. In this embodiment, a blob is a region made up of continuous pixels with a pixel value of 0 in at least one of eight directions, namely up, down, left, right, and diagonal. The number of blobs extracted from the feature image 630 of the current frame may be 0, 1, or 2 or more.

[0056] Fig. 7 is a diagram for explaining an example of a process for extracting a blob. Note that Fig. 7 shows only the area near the area 621 in the feature image 630 of the current frame. In this embodiment, the discrimination unit 323 extracts blobs by extracting the contours (edges) of areas consisting of consecutive pixels whose pixel value is 0 from the feature image 630 of the current frame. An example of the process for extracting blobs will be described with reference to FIG.

[0057] First, the discrimination unit 323 designates one pixel with a pixel value of 0 from among the feature image 630 of the current frame. The pixel designated at this time is a pixel that does not yet belong to any blob. Then, the discrimination unit 323 selects pixels with a pixel value of 1 from among the pixels adjacent to the designated pixel as boundary pixels. FIG. 7 illustrates an example in which boundary pixel 710 is selected first. Note that the pixels adjacent to a certain pixel may be pixels adjacent to the certain pixel in any of a total of eight directions, including the upward direction, downward direction, left direction, right direction, upper left diagonal direction, lower left diagonal direction, lower right diagonal direction, and upper right diagonal direction.

[0058] Next, the discrimination unit 323 sequentially searches for other outer boundary pixels adjacent to the outer boundary pixel 710, starting from the outer boundary pixel 710. An outer boundary pixel is a pixel whose pixel value is 1 and adjacent to one or more pixels whose pixel value is 0. When searching for other outer boundary pixels, a priority order is set in advance for the above-mentioned eight directions. FIG. 7 illustrates an example of searching for outer boundary pixels in a counterclockwise direction on the paper. In this case, the discrimination unit 323 searches for other outer boundary pixels adjacent to the outer boundary pixel in the following order on the paper: upper left diagonal direction, left direction, lower left diagonal direction, lower right diagonal direction, right direction, upper right diagonal direction, and upper direction. Then, in the example shown in FIG. 7, pixels on the search path 720 are sequentially searched for as outer boundary pixels. The discrimination unit 323 repeatedly executes the above-mentioned search for outer boundary pixels until the initially selected outer boundary pixel 710 is searched for again. The order of priority for the above-mentioned eight directions is not limited to the order of upper left, left, lower left, lower right, right, upper right, and upper direction on the paper.

[0059] Then, discrimination unit 323 extracts an area of ​​pixels with a pixel value of 0 (area shown in black in FIG. 7) surrounded by the boundary pixels found as described above as one blob 730, and assigns identification information of the blob 730 to the pixels constituting blob 730. The pixels to which the identification information of blob 730 has been assigned become pixels that belong to the blob 730. In this manner, area 621 shown in FIG. 6(c) is extracted as blob 730.

[0060] In addition, in Fig. 7, an example is shown in which the outer boundary pixels are searched for in a counterclockwise direction on the paper surface. However, this is not necessarily required, and the outer boundary pixels may be searched for in a clockwise direction on the paper surface. Also, the search does not have to be performed in a diagonal direction. In this case, the outer boundary pixels are searched for in a total of four directions, the upward direction, downward direction, left direction, and right direction.

[0061] Labeling may also be performed without searching for boundary pixels. For example, the same identification information may be assigned to a plurality of consecutive pixels with a pixel value of 0 in at least one of a total of eight directions, including the upward direction, downward direction, left direction, right direction, left-upward diagonal direction, left-downward diagonal direction, right-downward diagonal direction, and right-upward diagonal direction, so that a plurality of pixels assigned the same identification number are regarded as pixels belonging to the same blob. Labeling may also be performed to extract the outline of an area of ​​pixels assigned the same identification information.

[0062] In this embodiment, the blob 730 extracted from the feature image 630 of the current frame in the above manner is an example of a detected object included in the feature image. Note that, although Fig. 7 illustrates an example in which one blob 730 is extracted from the feature image 630 of the current frame, the number of blobs extracted from the feature image of the current frame is not limited to 1, and may be 0 or 2 or more.

[0063] [[[Discrimination by area]]] The discrimination unit 323 calculates the area of ​​the blob 730 extracted from the feature image 630 of the current frame. For example, the discrimination unit 323 calculates the number of pixels constituting the blob 730 extracted from the feature image 630 of the current frame as the area of ​​the blob 730. Note that the number of blobs for which the area is to be calculated may be 0, 1, or 2 or more. When a plurality of blobs are extracted from the feature image 630 of the current frame, the discrimination unit 323 calculates the area of ​​each of the plurality of blobs.

[0064] Then, the discrimination unit 323 determines whether the area Sblob of the blob 730 extracted from the feature image 630 of the current frame is within a predetermined range. The minimum value Smin and maximum value Smax of the predetermined range are set in advance based on, for example, an expected range of sizes of the punched holes PH shown in the image captured by the camera 120.

[0065] 8 is a diagram showing an example of a blob 730 extracted from the feature image 630 of the current frame. Here, a case is illustrated in which blobs 730a, 730b, and 730c are extracted from three feature images 630a, 630b, and 630c, respectively, as the feature image 630 of the current frame.

[0066] Fig. 8(a) is a diagram showing an example of a blob 730a extracted from the feature image 630a of the current frame, the area Sblob of which is within a predetermined range (minimum value Smin or more and maximum value Smax or less). Fig. 8(b) is a diagram showing an example of a blob 730b extracted from the feature image 630b of the current frame, the area Sblob of which is outside the predetermined range (less than minimum value Smin). Fig. 8(c) is a diagram showing an example of a blob 730c extracted from the feature image 630c of the current frame, the area Sblob of which is outside the predetermined range (more than maximum value Smax).

[0067] Blobs 730b with too small an area Sblob as shown in FIG. 8(b) and blobs 730c with too large an area Sblob as shown in FIG. 8(c) are considered to be extracted due to factors other than punch holes PH, such as the shadow of a moving object reflected in the image captured by camera 120. For example, when a shadow is cast on the capture area of ​​camera 120 due to light (e.g., the morning sun or the evening sun) entering the factory from the outside, the areas Sblob of blobs 730b and 730c may fall below the minimum value Smin or exceed the maximum value Smax. In addition, when an obstacle projected on the capture area of ​​camera 120 exists, such as when a crane passes above the capture area of ​​camera 120, the areas Sblob of blobs 730b and 730c may fall below the minimum value Smin or exceed the maximum value Smax.

[0068] Therefore, in this embodiment, the discrimination unit 323 determines, as shown in Fig. 8(a), a blob 730a whose area Sblob is within a predetermined range (above the minimum value Smin and below the maximum value Smax) as a candidate for a blob indicating a punch hole PH. On the other hand, the discrimination unit 323 does not determine, as candidates for a punch hole PH, blobs 730b and 730c whose area Sblob is outside the predetermined range.

[0069] [[[Distinguishing by positional relationship]]] The discrimination unit 323 calculates the position of the blob 730a extracted from the feature image 630 of the current frame. In this embodiment, the discrimination unit 323 calculates the position of the center of gravity (xy coordinates) of the blob 730a extracted from the feature image 630 of the current frame as the position of the blob 730a. The blob 730a for which the position of the center of gravity is to be calculated is a blob whose area Sblob is within a predetermined range (minimum value Smin or more and maximum value Smax or less) among the blobs extracted from the feature image 6220 of the current frame as described in the section [[[Extraction of blobs]]]. That is, the blobs 730b to 730c shown in FIG. 8(b) and FIG. 8(c) are not blobs for which the position of the center of gravity is to be calculated. However, the positions of the centers of gravity may also be calculated for these blobs 730b to 730c. The method of calculating the position of the center of gravity may be, for example, to approximate the blobs 730b-730c into an ellipse using a known method such as the least squares method, and to determine the position of the pixel corresponding to the center of gravity of the approximated ellipse as the position of the center of gravity. Note that the number of blobs for which the positions of the centers of gravity are to be calculated may be 0, 1, or 2 or more.

[0070] The discrimination unit 323 determines the relative position between a blob 730a having an area Sblob within a predetermined range among the blobs extracted from the feature image 630 of the current frame and a blob having an area Sblob within a predetermined range among the blobs extracted from the feature image of the previous frame. In the following description, the blob 730a having an area Sblob within a predetermined range among the blobs extracted from the feature image 630 of the current frame is referred to as a blob of the current frame, and the blob having an area Sblob within a predetermined range among the blobs extracted from the feature image of the previous frame is referred to as a blob of the previous frame. Note that, with regard to the blobs extracted from the feature image of the previous frame, all blobs may be used instead of the blobs having an area Sblob within the predetermined range.

[0071] In this embodiment, the discrimination unit 323 compares the positions of the blobs in the current frame and the previous frame in both the width direction (x-axis direction) and the conveyance direction (y-axis direction) of the steel plate M. Specifically, in this embodiment, the discrimination unit 323 discriminates whether each blob in the current frame is located downstream in the conveyance direction of the steel sheet M (a position with a larger y-axis value) than the blob in the previous frame.

[0072] FIG. 9 is a diagram showing an example of the positional relationship between a blob in a current frame and a blob in a previous frame in the transport direction (y-axis direction) of the steel sheet M. Since the punch holes PH are provided in the steel sheet M, the punch holes PH are also transported together with the transport of the steel sheet M. Therefore, if the blob in the current frame is due to the punch hole PH, it is assumed that the blob in the current frame is located downstream in the transport direction of the steel sheet M (positive direction of the y-axis) than the blob in the previous frame. Therefore, in this embodiment, as shown in FIG. 9(a), if the y-axis coordinate CGy(n) relating to the position of the center of gravity of the blob 730d in the current frame n is larger than the y-axis coordinate CGy(n-1) relating to the position of the center of gravity of the blob 730e in the previous frame n-1 (if CGy(n)-CGy(n-1)>0), the discrimination unit 323 discriminates that the positional relationship between the blob 730d in the current frame and the blob 730e in the previous frame in the transport direction of the steel sheet M (y-axis direction) is correct. Thus, in this embodiment, when blob 730d of the current frame is determined to be a punch hole PH, the positional relationship in the transport direction (y-axis direction) of the steel plate M includes a relationship in which blob 730d detected from the characteristic image of the current frame acquired at a relatively later timing is located downstream in the transport direction (y-axis direction) of the steel plate M than blob 730e detected from the characteristic image of the previous frame acquired at a relatively earlier timing.

[0073] On the other hand, as shown in Figure 9 (b), if the y-axis coordinate CGy(n) of blob 730f in the current frame n is not greater than the y-axis coordinate CGy(n-1) of blob 730g in the previous frame n-1 (if CGy(n) - CGy(n-1) ≦ 0), the discrimination unit 323 determines that the positional relationship between blob 730f of the current frame and blob 730g of the previous frame in the transport direction (y-axis direction) of the steel plate M is not correct.

[0074] Further, the discrimination unit 323 discriminates whether or not the absolute value of the difference in the position of the steel sheet M in the width direction (x-axis direction) between the blobs 730d, 730f of the current frame and the blobs 730e, 730g of the previous frame is less than a predetermined value ε for each of the blobs 730d, 730f of the current frame and the blobs 730e, 730g of the previous frame is less than a predetermined value ε. The predetermined value ε is a positive value and is set in advance. Since the steel sheet M is transported along the longitudinal direction (y-axis direction), if the blobs 730d, 730f of the current frame are caused by punch holes PH, it is assumed that the amount of change in the position of the blobs 730d, 730f of the current frame in the width direction (x-axis direction) of the steel sheet M relative to the blobs 730e, 730g of the previous frame is small. Therefore, it is preferable that the predetermined value ε is as small as possible. However, the steel sheet M may meander. Therefore, it is preferable to determine the predetermined value ε taking into consideration the meandering of the steel sheet M. The predetermined value ε is set to, for example, a value equal to or greater than a value obtained by converting an estimated value of the amount of meandering of the steel sheet M between two consecutive frames into the number of pixels of the camera 120.

[0075] FIG. 10 is a diagram showing an example of the positional relationship in the width direction (x-axis direction) of the steel plate M between a blob in a current frame and a blob in a previous frame. As described above, when a blob in the current frame is caused by a punch hole PH, it is assumed that the amount of change in the position of the blob in the current frame in the width direction (x-axis direction) of the steel sheet M relative to the blob in the previous frame is small. Therefore, in this embodiment, as shown in Fig. 10(a), when the absolute value of the difference between the x-axis coordinate CGx(n) relating to the position of the center of gravity of a blob 730h in the current frame n and the x-axis coordinate CGx(n-1) relating to the position of the center of gravity of a blob 730i in the previous frame n-1 is less than a predetermined value ε (when |CGx(n)-CGx(n-1)|<ε), the discrimination unit 323 discriminates that the positional relationship in the width direction (x-axis direction) of the steel sheet M between the blob 730h in the current frame and the blob 730i in the previous frame is correct. On the other hand, as shown in Fig. 10(b), when the absolute value of the difference between the x-axis coordinate CGx(n) of the position of the center of gravity of the blob 730j in the current frame n and the x-axis coordinate CGx(n-1) of the blob 730k in the previous frame n-1 is not less than a predetermined value ε (when |CGx(n)-CGx(n-1)|≧ε), the discrimination unit 323 discriminates that the positional relationship in the width direction (x-axis direction) of the steel sheet M between the blob 730j in the current frame and the blob 730k in the previous frame is not a correct positional relationship. Thus, in this embodiment, the positional relationship in the width direction (x-axis direction) of the steel sheet M when the blob 730d in the current frame is discriminated to be a punch hole PH includes a relationship in which the difference in the position in the width direction (x-axis direction) of the steel sheet M between the blob 730i detected from the feature image of the previous frame acquired at a relatively earlier timing and the blob 730h detected from the feature image of the current frame acquired at a relatively later timing is less than a predetermined value.

[0076] 9 and 10, for ease of explanation, a case is illustrated in which the number of blobs 730e, 730g, 730i, and 730k in the previous frame is 1. However, there are cases in which the number of blobs in the previous frame is not 1. In this embodiment, when the number of blobs in the previous frame is two or more, the discrimination unit 323 discriminates whether or not the positional relationship between the blobs in the current frame and the blobs in the previous frame is correct as follows. That is, if there is at least one blob in the previous frame whose positional relationship with the blobs in the current frame is the above-mentioned correct positional relationship, the discrimination unit 323 discriminates that the positional relationship between the blobs in the current frame and the blobs in the previous frame is correct, and if not, the discrimination unit 323 discriminates that the positional relationship between the blobs in the current frame and the blobs in the previous frame is incorrect. The discrimination unit 323 performs the above-mentioned positional relationship discrimination for each blob in the current frame individually.

[0077] Furthermore, when the number of blobs in the previous frame is 0, discrimination unit 323 determines that the positional relationship between the blobs in the current frame and the blobs in the previous frame is incorrect. Furthermore, when the number of blobs in the current frame is 0, discrimination unit 323 also determines that the positional relationship between the blobs in the current frame and the blobs in the previous frame is incorrect.

[0078] When the discrimination unit 323 determines that the positional relationship between the blobs 730d, 730h of the current frame and the blobs 730e, 730i of the previous frame is correct in both the conveyance direction (y-axis direction) and the width direction (x-axis direction) of the steel sheet M, it determines that the blobs 730d, 730h of the current frame are blobs indicating punch holes PH. On the other hand, when the discrimination unit 323 determines that the positional relationship between the blobs 730f, 730j of the current frame and the blobs 730g, 730k of the previous frame is not correct in at least one of the conveyance direction (y-axis direction) and the width direction (x-axis direction) of the steel sheet M, it determines that the blobs 730f, 730j of the current frame are not blobs indicating punch holes PH.

[0079] As described above, by determining whether the positional relationship between blobs 730d, 730f, 730h, and 730j in the current frame and blobs 730e, 730g, 730i, and 730k in the previous frame is correct, it is possible to determine as accurately as possible whether blobs 730d, 730f, 730h, and 730j in the current frame are blobs indicating punch holes PH or blobs indicating light entering the factory from outside or a shadow caused by an obstacle, as described above, even if it is not easy to determine whether a blob indicates a punch hole PH from the blob area Sblob.

[0080] As described above, when there are a plurality of detection regions 410, the processes of the image processing unit 322 and the discrimination unit 323 are executed individually for each of the plurality of detection regions 410. In this case, the discrimination unit 323 discriminates whether or not a blob in the current frame is a blob indicating a punch hole PH in each of the plurality of detection regions 410.

[0081] [[output section 324]] When the discrimination unit 323 discriminates that the blobs 730d, 730h indicating the punch hole PH are present among the blobs 730d, 730f, 730h, and 730j of the current frame, the output unit 324 outputs information indicating that the welded portion WP has been detected. As the form of output, for example, at least one of display on a computer display, storage in a storage medium inside or outside the detection device 300, and transmission to an external device is adopted.

[0082] In this embodiment, when it is determined that the blobs 730d, 730h indicating punch holes PH are present among the blobs 730d, 730f, 730h, and 730j of the current frame, the output unit 324 determines whether the blobs 730d, 730h are the blobs located closest to the welded portion WP within the detection area 410. Specifically, when it is determined that the blobs 730d, 730h indicating punch holes PH are present among the blobs 730d, 730f, 730h, and 730j of the current frame within the detection area 410, the output unit 324 receives the image 400 of the current frame from the acquisition unit 310 and stores it in a storage medium as an image including punch holes PH. Thereafter, when the discrimination unit 323 discriminates that there is no blob indicating a punch hole PH in the image 400 of the current frame in the detection area 410, the output unit 324 discriminates that the blobs 730d, 730h in the latest image among the images discriminated to have the blobs 730d, 730h indicating the punch hole PH in the detection area 410 (images including the punch hole PH) are the blobs located closest to the welded portion WP in the detection area 410. Then, the output unit 324 receives the latest image from the acquisition unit 310 and displays it on the computer display. In this case, the operator can judge the quality of the welded portion WP by looking at the image of the frame displayed on the computer display. Note that, when the preceding steel plate MF has a punch hole PH, the blob located closest to the welded portion WP in the detection area 410 among the blobs discriminated as the blobs indicating the punch hole PH is the blob that is first discriminated as the one indicating the punch hole PH in the detection area 410.

[0083] In addition, the welded portion WP is considered to pass through the imaging area of ​​the camera 120 at the timing when the discrimination unit 323 discriminates that the blobs 730d, 730h indicating the punched hole PH are present among the blobs 730d, 730f, 730h, and 730j of the current frame. Therefore, for example, when the position of the welded portion WP is tracked based on the rotation speed of a transport roll (not shown), the tracking result can be corrected as follows. That is, when the discrimination unit 323 discriminates that the blobs 730d, 730h indicating the punched hole PH are present among the blobs 730d, 730f, 730h, and 730j of the current frame, the output unit 324 outputs information indicating that the welded portion WP has been detected. If the position of the tracked welded portion WP at the timing when the output unit 324 outputs information indicating that the welded portion WP has been detected is a position different from the position of the imaging area of ​​the camera 120, the position of the tracked welded portion WP is corrected to the position of the imaging area of ​​the camera 120.

[0084] If the discrimination unit 323 discriminates that there is no blob indicating a punch hole PH among the blobs of the current frame, the output unit 324 does not change the display on the computer display. Instead of not changing the display on the computer display in this manner, the output unit 324 may, for example, erase the image of the previous frame being displayed on the computer display.

[0085] In addition, when there are a plurality of detection regions 410, the output unit 324 outputs information indicating that a welded portion WP has been detected when a blob in the current frame is determined to be a blob indicating a punched hole PH in at least one of the plurality of detection regions 410. For example, the output unit 324 outputs information indicating that a welded portion WP has been detected when a blob in the current frame is determined to be a blob indicating a punched hole PH in a predetermined number or more of the plurality of detection regions 410, and does not output information indicating that a welded portion WP has been detected otherwise. The predetermined number may be the same as the total number of detection regions 410 or may be a number less than the total number of detection regions 410. Furthermore, as described above, when displaying an image including blobs 730d, 730h indicating punch holes PH that are located closest to the welded portion WP within the detection area 410 (an image including punch holes PH) on a computer display, the output unit 324, for example, identifies the blobs 730d, 730h indicating punch holes PH that are located closest to the welded portion WP within the detection area 410 in each of the multiple detection areas 410, and displays an image including the blob that is located closest to the welded portion WP among the identified positions (an image including punch holes PH) on the computer display.

[0086] Here, an example is shown in which an image including blobs 730d and 730h indicating punch holes PH that are located closest to the welded portion WP (an image including punch holes PH) is displayed on a computer display. However, this is not necessarily required. For example, the output unit 324 may output information indicating that the welded portion WP has been detected (e.g., the image 400 of the current frame) every time the discrimination unit 323 discriminates that the blobs 730d and 730h indicating punch holes PH are present among the blobs 730d, 730f, 730h, and 730j of the current frame.

[0087] <Flowchart> Next, an example of a detection method using the detection device 300 of this embodiment will be described with reference to the flowcharts of FIGS. In step S 1101 , the acquisition unit 310 acquires an image of the current frame captured by the camera 120 . Next, in step S1102, the trimming unit 321 trims the detection area 410 from the image 400 of the current frame acquired in step S1101.

[0088] Next, in step S1103, the filter unit 322a smoothes the luminance of the image of the detection region 410 in the current frame cropped in step S1102. Next, in step S1104, the grayscale conversion unit 322b converts the image of the detection region 410 in the current frame smoothed in step S1103 into a grayscale image. Next, in step S1105, the adaptive binarization processing unit 322c binarizes the luminance values ​​of the image of the detection area 410 of the current frame that was grayscaled in step S1104 through adaptive binarization processing, thereby generating a binarized image of the detection area 410 of the current frame.

[0089] Next, in step S1106, the morphology processing unit 322d performs opening and closing on the binarized image of the detection region 410 of the current frame generated in step S1105, to generate a feature image 630 of the current frame. After the process of step S1106, the process of step S1201 in Fig. 12 is executed. In step S1201, the discrimination unit 323 extracts the blob 730 from the feature image 630 of the current frame generated in step S1106.

[0090] Next, in step S1202, discrimination unit 323 calculates the area of ​​blob 730 extracted in step S1201. Next, in step S1203, discrimination unit 323 discriminates whether or not there is a blob whose area Sblob calculated in step S1202 falls within a predetermined range among the blobs 730 extracted in step S1201.

[0091] As a result of the determination in step S1203, if there is no blob among the blobs 730 extracted in step S1201 whose area Sblob calculated in step S1202 is within a predetermined range (NO in step S1203), the processing of step S1208 described below is executed.

[0092] On the other hand, if the result of the determination in step S1203 is that there is one or more blobs among the blobs 730 extracted in step S1201 whose areas Sblob calculated in step S1202 are within a predetermined range (YES in step S1203), the process proceeds to step S1204. In step S1204, the determination unit 323 calculates the positions of the centers of gravity of the blobs among the blobs 730 extracted in step S1201 whose areas Sblob calculated in step S1202 are within a predetermined range.

[0093] Next, in step S1205, the discrimination unit 323 discriminates whether or not the positional relationship between the blobs 730d, 730f, 730h, and 730j of the current frame and the blobs 730e, 730g, 730i, and 730k of the previous frame is correct. If the result of the determination in step S1205 is that the positional relationships between blobs 730d, 730f, 730h, and 730j in the current frame and blobs 730e, 730g, 730i, and 730k in the previous frame are not correct (NO in step S1205), the process in step S1208 described below is executed. Note that in the first processing of step S1205, since there are no blobs in the previous frame, the result of the determination in step S1205 is unconditionally NO.

[0094] On the other hand, if the result of the determination in step S1205 is that the positional relationship between the blobs 730d, 730f, 730h, and 730j of the current frame and the blobs 730e, 730g, 730i, and 730k of the previous frame is correct (YES in step S1205), the process of step S1206 is executed. In step S1206, the output unit 324 stores the image 400 of the current frame as an image including the punched hole PH together with information on the shooting time of the image 400 and the identification information of the welded portion WP. The identification information of the welded portion WP is assigned, for example, as follows. That is, in the repeated process of Figs. 11 and 12, the output unit 324 assigns the same identification information as the identification information of the welded portion WP to the images including the punched holes PH stored during the period from when the state determined as NO in step S1205 changes to when the state determined as YES has passed until a predetermined time has passed. The output unit 324 executes the assignment of such identification numbers to the welded portion WP until the state in which step S1205 is judged as NO changes to the state in which step S1205 is judged as YES. In this case, the output unit 324 assigns a different identification number as identification information of the welded portion WP every time the state in which step S1205 of the previous iteration of the flowcharts of FIG. 11 and FIG. 12 is judged as NO and the state in which step S1205 of the current iteration of the flowcharts of FIG. 11 and FIG. 12 is judged as YES (that is, every time the state in which it is not judged that the blobs 730d, 730h indicating the punched holes PH are present in the blobs 730d, 730f, 730h, and 730j of the current frame changes from a state in which it is judged that the blobs 730d, 730h indicate the punched holes PH are present in the blobs 730d, 730f, 730h, and 730j of the current frame). In addition, the predetermined time may be, for example, a period shorter than the longest period during which the same welded portion WP is photographed by the camera 120, which is specified based on the expected value of the transport speed of the steel sheet M and the frame rate. Note that the method of assigning the identification information of the welded portion WP is not limited to such a method. For example, the output unit 324 may obtain the expected arrival time of the welding part WP in the shooting area of ​​the camera 120 from a higher-level computer, and if the shooting time of an image including a punch hole PH is within a specified period based on the expected arrival time of the welding part WP, the output unit 324 may assign identification information of the welding part WP to the image including the punch hole PH.

[0095] Next, in step S1207, the detection device 300 determines whether or not the operation has ended. In this embodiment, for example, when the steel sheet M runs out in the continuous annealed steel sheet production line, the operation ends. The detection device 300 determines that the operation has ended, for example, when an operator performs an operation corresponding to the end of the operation on a user interface of the detection device 300. In addition, the detection device 300 may determine that the operation has ended by receiving a signal indicating that the operation has ended from a computer that manages the operation of the continuous annealed steel sheet production line.

[0096] As a result of the determination in step S1207, if the operation has not been completed (NO in step S1207), the process in step S1101 in FIG. 11 is executed again. On the other hand, if the result of the determination in step S1207 is that the operation has ended (YES in step S1207), the processing according to the flowcharts in FIGS. 11 and 12 ends.

[0097] As described above, if the results of steps S1203 and S1205 are NO, the process of step S1208 is executed. In step S1208, the output unit 324 judges whether the latest image of the images stored as images including punch holes PH has been output. If the result of this judgment is that the latest image of the images stored as images including punch holes PH has not been output (NO in step S1208), the process of step S1209 is executed. Here, if the results of steps S1203 and S1205 are NO (i.e., it is no longer determined that the blobs 730d and 730h indicating the punch holes PH are present among the blobs 730d, 730f, 730h, and 730j of the current frame), and if the latest image of the images stored as images including punch holes PH has not been output, the blobs 730d and 730h indicating the punch holes PH in the latest image are regarded as the blobs located closest to the welded portion WP within the detection area 410. Therefore, if the latest image among the images stored as images containing punch holes PH has not been output (NO in step S1208), in step S1209, the output unit 324 outputs the latest image among the images stored as images containing punch holes PH (for example, displays it on a computer display), and the process of step S1207 described above is then executed.

[0098] On the other hand, if the latest image among the images stored as images including punch holes PH has already been output (YES in step S1208), there is no need to output the latest image, so the process of step S1209 is omitted and the process of step S1207 described above is executed. Then, if it is determined in step S1207 that the operation has not ended (NO in step S1207), the process of step S1101 in FIG. 11 is executed again. In this manner, the process of step S1101 is executed repeatedly. The image 400 of the current frame acquired in the process of step S1101 for the (Q-1)th time becomes the image of the previous frame with respect to the image 400 of the current frame acquired in the process of step S1101 for the Qth time (note that Q is an integer equal to or greater than 2).

[0099] <Summary> As described above, in this embodiment, the detection device 300 performs image processing for detecting candidates of the blob 730 indicating the punch hole PH provided in the steel sheet M to indicate the position of the welded portion WP on the image 400 of the current frame captured by the camera 120, thereby acquiring the feature image 630 for the current frame. The detection device 300 determines whether the blob 730 detected from the feature image 630 of the current frame is a blob indicating the welded portion WP based on the relative positional relationship of the blob 730 detected from each of the feature image 630 of the current frame and the feature image of the previous frame. When the blob 730 detected from the feature image 630 of the current frame is the blob 730d or 730h indicating the welded portion WP, the detection device 300 outputs information indicating that the welded portion WP has been detected. Therefore, even if a shadow is cast on the shooting area of ​​the camera 120 due to light entering the factory from the outside or an obstacle, it is possible to improve the accuracy of determining whether the blobs 730d, 730f, 730h, and 730j of the current frame are blobs indicating the punched hole PH. Therefore, the detection accuracy of the welded portion WP of the steel plate M can be improved.

[0100] In the present embodiment, the detection device 300 judges whether the relative positions of the blobs 730d, 730f, 730h, and 730j of the current frame and the blobs 730e, 730g, 730i, and 730k of the previous frame are correct in the conveyance direction (y-axis direction) and width direction (x-axis direction) of the steel sheet M. Therefore, the judgment is made based on whether the movement of the blob 730 is a movement expected as the movement of the punch hole PH in each of the conveyance direction (y-axis direction) and width direction (x-axis direction) of the steel sheet M. This can further improve the accuracy of judging whether the blobs 730d, 730f, 730h, and 730j of the current frame are blobs indicating the punch hole PH.

[0101] Furthermore, in this embodiment, when the blob 730d of the current frame is located downstream (positive direction of the y-axis) of the blob 730e of the previous frame in the conveying direction of the steel sheet M, the detection device 300 determines that the positional relationship between the blob 730d of the current frame and the blob 730e of the previous frame in the conveying direction of the steel sheet M (y-axis direction) is correct. Therefore, the detection device 300 determines that the blob 730d, which moves downstream in the conveying direction of the steel sheet M as the steel sheet M is conveyed, like the punch hole PH, is a blob indicating a punch hole PH. This can further improve the accuracy of determining whether the blobs 730d, 730f, 730h, and 730j of the current frame are blobs indicating punch holes PH.

[0102] Furthermore, in this embodiment, when the absolute value of the difference in position between the blob 730h of the current frame and the blob 730i of the previous frame in the width direction (x-axis direction) of the steel sheet M is less than a predetermined value ε, the detection device 300 determines that the positional relationship between the blob 730h of the current frame and the blob 730i of the previous frame in the width direction (x-axis direction) of the steel sheet M is correct. Therefore, the detection device 300 determines that the blob 730h, whose position in the width direction (x-axis direction) of the steel sheet M does not change significantly even when the steel sheet M is transported, similar to the punch hole PH, is the blob indicating the punch hole PH. This can further improve the accuracy of determining whether the blobs 730d, 730f, 730h, and 730j of the current frame are blobs indicating the punch hole PH.

[0103] In the present embodiment, the detection device 300, among the blobs 730a to 730c detected from the characteristic images 630a to 630c of the current frame, determines the blob 730a whose area Sblob is within a predetermined range as a candidate for the blob indicating the punch hole PH, and excludes the blobs 730b and 730c whose area Sblob is not within a predetermined range from the candidates for the blob indicating the punch hole PH. Therefore, the blob 730a whose size is expected to be the size of the punch hole PH can be determined as the blob indicating the punch hole PH. In the example shown in FIG. 9 and FIG. 10, the blobs 730d, 730f, 730h, and 730j of the current frame are the candidates for the blob indicating the punch hole PH (blobs whose size is expected to be the size of the punch hole PH), so that the accuracy of determining whether the blobs 730d, 730f, 730h, and 730j of the current frame are the blobs indicating the punch hole PH can be further improved.

[0104] Moreover, in this embodiment, the detection device 300 acquires images of a plurality of consecutive frames captured by the camera 120. Therefore, the characteristic images 630 of temporally adjacent frames can be targets for extracting the blob 730. Therefore, even when the steel sheet M is transported at high speed, the blob 730 can be extracted from the characteristic images 630 of each frame.

[0105] In this embodiment, the detection device 300 binarizes the luminance value of the image of the detection area 410 of the current frame by adaptive binarization processing. Therefore, even if the image has uneven luminance, it can be appropriately binarized. This further improves the accuracy of determining whether the blobs 730d, 730f, 730h, and 730j of the current frame are blobs indicating punch holes PH.

[0106] Furthermore, in this embodiment, the detection device 300 smoothes the luminance of the image of the detection region 410 of the current frame. This makes it possible to reduce noise contained in the image 400 of the current frame. This further improves the accuracy of determining whether the blobs 730d, 730f, 730h, and 730j of the current frame are blobs indicating punch holes PH.

[0107] In this embodiment, the detection device 300 generates the feature image 630 of the current frame by performing morphological processing including dilation and erosion on the binary image of the detection region 410 of the current frame. Therefore, the blob 730 to be extracted from the feature image 630 of the current frame can be made clear.

[0108] Furthermore, in this embodiment, the detection device 300 trims the detection region 410 from the image 400 of the current frame. Therefore, it is not necessary to execute image processing on the entire image 400 of the current frame. This makes it possible to reduce the calculation load for detecting the welded portion WP of the steel plate M.

[0109] In addition, in this embodiment, when it is determined that the blobs 730d, 730h indicating the punched hole PH are present among the blobs 730d, 730f, 730h, and 730j of the current frame, the detection device 300 causes the image 400 of the current frame to be displayed on the computer display. Therefore, the operator can easily determine the quality of the welded portion WP from the image 400 of the current frame.

[0110] Moreover, in this embodiment, the camera 120 photographs the steel sheet M being transported on the continuous annealed steel sheet production line. Therefore, the welded portion WP can be detected when processing is being performed on the steel sheet M. Therefore, for example, the tracking result of the welded portion WP based on the rotation speed of a transport roll (not shown) can be corrected. Also, the quality of the welded portion WP can be confirmed when processing is being performed on the steel sheet M.

[0111] Moreover, in this embodiment, the camera 120 photographs the steel sheet M being transported between the exit of the cold rolling equipment 104 and the entrance of the annealing equipment 105. Therefore, the quality of the welded portion WP can be confirmed before the welded portion WP enters the annealing equipment 105, and if the welded portion WP has a weld defect, the operation can be changed so that the welded portion WP does not enter the annealing equipment 105. Therefore, it is possible to prevent the welded portion WP with a weld defect from entering the annealing equipment 105.

[0112] <Modification> In the present embodiment, a case has been exemplified in which it is determined whether the area of ​​the blob 730 extracted from the feature image 630 of the current frame is within a predetermined range, and for the blobs whose areas are within the predetermined range, the relative positional relationship between the blob 730 extracted from the feature image 630 of the current frame and the blob extracted from the feature image of the previous frame is obtained. However, this is not necessarily required. For example, after the relative positional relationship between the blob 730 extracted from the feature image 630 of the current frame and the blob extracted from the feature image of the previous frame is obtained, it may be determined whether the area of ​​the blob with the correct relative positional relationship among the blobs 730 extracted from the feature image 630 of the current frame is within a predetermined range.

[0113] Furthermore, as described above, it is preferable to determine whether or not the blob 730 extracted from the feature image 630 of the current frame is a blob indicating a punch hole PH based on the area of ​​the blob 730 extracted from the feature image 630 of the current frame, in addition to the relative positions of the blob 730 extracted from the feature image 630 of the current frame and the blob extracted from the feature image of the previous frame, since this can further improve the determination accuracy. However, it is also possible to determine whether or not the blob 730 extracted from the feature image 630 of the current frame is a blob indicating a punch hole PH without using the area of ​​the blob 730 extracted from the feature image 630 of the current frame.

[0114] In addition, in the present embodiment, when the blob 730 extracted from the characteristic image 630 of the current frame is located downstream (positive direction of the y axis) of the blob extracted from the characteristic image of the previous frame in the transport direction of the steel sheet M, the positional relationship between the blob 730 extracted from the characteristic image 630 of the current frame and the blob extracted from the characteristic image of the previous frame in the transport direction of the steel sheet M (y axis direction) is determined to be a correct positional relationship. However, this is not necessarily required. For example, a distance based on the transport speed of the steel sheet M and the reciprocal of the frame rate of the camera 120 may be converted into the number of pixels of the camera 120, and when the blob 730 extracted from the characteristic image 630 of the current frame is located downstream of the blob extracted from the characteristic image of the previous frame in the transport direction of the steel sheet M by the number of pixels based on the converted number of pixels, it may be determined that the positional relationship between the blob 730 extracted from the characteristic image 630 of the current frame and the blob extracted from the characteristic image of the previous frame in the transport direction of the steel sheet M (y axis direction) is a correct positional relationship. The number of pixels converted as described above is the number of pixels corresponding to the movement distance of the punch hole PH from when the previous frame was captured to when the current frame was captured. Therefore, by doing this, it is possible to determine that, among the blobs 730 extracted from the feature image 630 of the current frame, the blobs that move at the conveying speed of the steel sheet M are blobs whose positional relationship in the conveying direction (y-axis direction) of the steel sheet M is correct.

[0115] In addition, in the continuous annealed steel sheet production line shown in FIG. 1, a spare welding equipment (welding machine) and a center reel may be arranged between the deflector roll 110c and the annealing equipment 105. The spare welding equipment and the center reel are used to continue the operation of the preceding steel sheet MF even if cracks or the like occur in the welded portion WP during cold rolling by the cold rolling equipment 104. The spare welding equipment cuts the welded portion WP between the preceding steel sheet MF and the following steel sheet MA, and then welds the rear end of the preceding steel sheet MF to the front end of the steel sheet rewound from the center reel. In this way, even if cracks or the like occur in the welded portion WP after cold rolling by the cold rolling equipment 104, the operation of the preceding steel sheet MF can be continued. In this case, a camera 120 is arranged so that the plate surface of the steel sheet M being transported in the area between the deflector roll 110c and the entrance of the spare welding equipment can be photographed. In this way, the quality of the welded portion WP can be checked before the welded portion WP passes through the spare welding equipment, and if the welded portion WP is defective, the welded portion WP can be cut off and the rear end of the preceding steel sheet MF and the front end of the steel sheet rewound from the center reel can be welded together by the spare welding equipment, thereby allowing the operation of the preceding steel sheet MF to continue.

[0116] In addition, in the present embodiment, the case where the detection device 300 is applied to a continuous annealed steel sheet production line is exemplified. However, as long as the continuous processing line is one in which the rear end of a preceding metal material and the front end of a succeeding metal material are joined to continuously process a plurality of metal materials, the application of the detection device 300 is not limited to the continuous annealed steel sheet production line. For example, the detection device 300 may be used to detect the welded portion of the steel sheet at any of the following positions: the position of the outlet side of the pickling equipment in the pickling line, the position between the outlet of the welding equipment and the inlet of the cold rolling equipment in the cold rolling line, and the position of the outlet side of the cold rolling equipment in the pickling cold rolling continuous line. In the pickling line, the welding equipment is arranged upstream of the pickling equipment. In the pickling cold rolling continuous line, the pickling equipment is arranged upstream of the cold rolling equipment, and the welding equipment is arranged upstream of the pickling equipment. In addition, the detection device 300 may be used to detect the welded portion of the steel sheet at a position between the outlet of the welding equipment and the inlet of the annealing equipment in the continuous annealing line and the continuous annealing / hot-dip galvanizing line. The detection device 300 may also be used to detect welds of steel sheets at a position between the exit of a welding facility and the entrance of a coating facility in a continuous hot-dip galvanizing line and an electrolytic tinning line. The detection device 300 may also be used to detect welds of steel sheets at a position between the exit of a welding facility and the entrance of a coating facility in a continuous coating line. Note that these continuous processing lines themselves can be realized using known methods, and therefore detailed explanations will be omitted.

[0117] Furthermore, in the present embodiment, a case has been exemplified in which a plurality of frame images (i.e., a plurality of consecutive frame images) captured in succession by the camera 120 are acquired and processed, but this is not necessarily required. For example, the acquisition unit 310 may acquire every other frame image captured by the camera 120. In this case, the determination unit 320 executes processing using, for example, every other two frame images, instead of two consecutive frame images. In this way, as long as an image captured (acquired) at a relatively earlier timing is set as the previous frame and an image captured (acquired) at a relatively later timing is set as the current frame, it is not necessarily required to process consecutive frame images.

[0118] Second embodiment Next, a second embodiment of the present invention will be described. In the first embodiment, the case where the image 400 of the current frame including the blob that is located closest to the welded portion WP in the detection area 410 among the blobs 730d and 730h indicating the punched hole PH is displayed on the computer display is exemplified. In this way, the welded portion WP may be located at the edge of the image 400 of the current frame displayed on the computer display. In addition, depending on the position of the detection area 410 (for example, the distance between the detection area 410 and the welded portion WP) and the timing at which it is determined that the blobs 730d and 730h indicating the punched hole PH are present, there is a risk that the welded portion WP may not be partially or entirely displayed on the image 400 of the current frame. In addition, in the first embodiment, the operator cannot adjust the image 400 of the current frame displayed on the computer display. For example, if the image 400 of the current frame does not show the welded portion WP partially or entirely, the operator cannot check the welded portion WP unless the operator can adjust the image 400 of the current frame displayed on the computer display.

[0119] Therefore, in this embodiment, the welded portion WP is prevented from being positioned at the edge of the image 400 of the current frame displayed on the computer display, and the operator is allowed to adjust the image 400 of the current frame displayed on the computer display. Thus, the present embodiment differs from the first embodiment mainly in the process of displaying the image on the computer display. Therefore, in the description of this embodiment, the same parts as those in the first embodiment are given the same reference numerals as those in Figs. 1 to 12, and detailed description thereof is omitted. In the following description, the image of each frame acquired by the acquisition unit 310 is referred to as a frame image as necessary. The frame image includes the image 400 of the current frame described in the first embodiment.

[0120] <Adjusting the display position of the welded part WP> First, an example of a technique for preventing a welded portion WP from being positioned at the edge of a frame image displayed on a computer display will be described. In this embodiment, the output unit 324 selects a frame image in which the position of the welded part WP in the image is closer to the center of the image than a frame image including a blob determined to be a punch hole PH at a position closest to the welded part WP in the detection area 410 from among the frame images acquired by the acquisition unit 310, and displays the frame image on a computer display. The frame image including a blob determined to be a punch hole PH at a position closest to the welded part WP in the detection area 410 is the image including the punch hole PH described in the first embodiment. Note that the blob determined to be a punch hole PH at a position closest to the welded part WP in the detection area 410 may be a blob whose entirety is within the detection area 410, or may be a blob whose only part is within the detection area 410.

[0121] 13 is a diagram for explaining an example of processing by the output unit 324 when blobs 730d and 730h indicating punch holes PH are present among blobs 730d, 730f, 730h, and 730j of the current frame. In this embodiment, as in the first embodiment, a case where punch holes PH are provided in a succeeding steel sheet MA is illustrated. In addition, in this embodiment, a frame image including a blob 730d or 730h indicating punch holes PH that is located closest to the welded portion WP in the detection area 410 is used as a reference, and a frame image in which the position of the welded portion WP in the image is closer to the center of the image than the frame image is displayed on a computer display.

[0122] In FIG. 13, it is assumed that the frame image 1300b is a frame image including a blob that is located closest to the welded portion WP within the detection region 410 among the blobs that indicate the punched hole PH. The output unit 324 calculates the number of pixels L in the conveyance direction (y-axis direction) of the steel sheet M from the center of gravity of the blob that indicates the punched hole PH in the frame image 1300b to the center of gravity of the blob that indicates the punched hole PH in the frame image 1300a, which is the frame immediately before the frame image 1300b (see FIGS. 13(A) and (B)). This number of pixels L corresponds to the movement amount (movement distance) of the punched hole PH per unit frame in the conveyance direction (y-axis direction) of the steel sheet M within the frame image 1300. The movement amount of the punched hole PH per unit frame is, for example, the movement amount in two consecutive frames (two frames). In the following description, the movement amount of the punch hole PH per unit frame in the conveyance direction (y-axis direction) of the steel sheet M in the frame image 1300 will be referred to as the movement amount L of the punch hole PH per unit frame as necessary. In this embodiment, the movement amount L of the punch hole PH per unit frame is an example of the movement amount of the detected object per unit frame.

[0123] Incidentally, the movement amount L of the punch holes PH per unit frame may be calculated as a distance in real space instead of a distance within the frame image 1300. For the sake of simplicity, it is assumed here that a blob indicating a punch hole PH exists in the frame image 1300a immediately preceding the frame image 1300b. However, if there is no blob indicating a punch hole PH in the frame image 1300a immediately preceding the frame image 1300b, the following may be performed, for example. That is, if there is no blob indicating a punch hole PH in the frame image 1300a immediately preceding the frame image 1300b, a determination is made as to whether or not there is a blob determined to be a punch hole PH in the frame image 1300 of the U frames preceding the frame image 1300b, with the value of U incremented by 1 from an initial value of 2. The maximum value of U may be determined, for example, as follows. That is, the maximum time that the blob can exist in the detection area 410 may be determined based on the moving speed of the blob in the frame image 1300 corresponding to the conveying speed of the steel sheet M and the length of the detection area 410 in the vertical direction (y-axis direction), and the maximum value of U may be determined so that the reciprocal of the maximum value of U is equal to or less than the determined time. Furthermore, if there is no blob indicating punch hole PH in frame image 1300a, which is a frame U frames before frame image 1300b, the output unit 324 may display frame image 1300b on a computer display, as in the first embodiment. Furthermore, if there is no blob indicating punch hole PH in frame image 1300a, which is a frame immediately before frame image 1300b (not a frame U frames before (two or more frames before) frame image 1300b), the output unit 324 may display frame image 1300b on a computer display.

[0124] The output unit 324 divides the constant CV, which was determined before the processing of the steel sheet M in the continuous annealed steel sheet production line, by the movement amount L of the punch hole PH per unit frame, to calculate the number of frames (number of frames) FN (=CV / L) to be changed for the frame image 1300b (see FIG. 13C). In the following description, the number of frames to be changed for the frame image 1300b will be referred to as the number of changed frames FN, as necessary. If the value obtained by dividing the constant CV by the movement amount L of the punch hole PH per unit frame does not become an integer, the value is changed to an integer. For example, the output unit 324 rounds off, drops the decimal point, or rounds up the decimal point to change the value obtained by dividing the constant CV by the movement amount L of the punch hole PH per unit frame to an integer.

[0125] The constant CV is determined by trial and error, for example, so that the position of the welded portion WP in the frame image 1300c, which is a frame (frame) that precedes the frame image 1300b by the number of changed frames FN, is a position where the operator can easily confirm the welded portion WP. In addition, when the welded portion WP is displayed at the center position of the frame image 1300, the constant CV may be determined according to the distance in the conveying direction (y-axis direction) of the steel sheet M between the position assumed as the position where the welded portion WP is displayed in the frame image 1300 including the blob that is closest to the welded portion WP in the detection area 410 and the center position of the frame image 1300. In addition, the constant CV may be variable. For example, the constant CV may be changed according to the conveying speed of the steel sheet M. For example, when the conveying speed of the steel sheet M is faster than the assumed value, the movement amount L of the punched hole PH per unit frame becomes large. In this case, the value of the constant CV may be smaller than the constant CV when the conveying speed of the steel sheet M is the assumed value. On the other hand, when the conveying speed of the steel sheet M is slower than the assumed value, the movement amount L of the punched holes PH per unit frame becomes smaller. In this case, the value of the constant CV may be made larger than the constant CV when the conveying speed of the steel sheet M is the assumed value.

[0126] As described above, in this embodiment, the punch hole PH is provided in the following steel sheet MA. In addition, in this embodiment, the end 411 of the detection area 410 on the downstream side in the conveying direction of the steel sheet M (positive direction of the y-axis) is located downstream in the conveying direction of the steel sheet M (positive direction of the y-axis) from the center of the frame image 1300. Therefore, in the frame image 1300b, the welded part WP is located downstream in the conveying direction of the steel sheet M (positive direction of the y-axis) from the detection area 410.

[0127] Therefore, in order to display on the computer display a frame image 1300 in which the position of the welded part WP in the image is closer to the center of the image than the frame image 1300, which is based on the frame image 1300b including the blob located closest to the welded part WP in the detection area 410, the output unit 324 selects a frame image 1300c of a frame (frame) that is the number of frames changed FN before the frame image 1300b from among the frame images 1300 acquired by the acquisition unit 310, and displays it on the computer display. FIG. 13C illustrates an example in which the number of frames changed FN is 4. In this case, the output unit 324 selects a frame image 1300c that is four frames before the frame image 1300b from among the frame images 1300 acquired by the acquisition unit 310, and displays it on the computer display (see FIG. 13D). FIG. 13D illustrates an example in which a part of the punched hole PH is outside the detection area 410. However, in the state shown in FIG. 13D, the entire punched hole PH may be outside the detection area 410, or at least a portion of the punched hole PH may be within the detection area 410.

[0128] <Adjustment by the operator of the frame image 1300 to be displayed> Next, an example of a method in which an operator adjusts the frame image 1300 displayed on a computer display will be described. 14 is a diagram showing an example of a display screen 1400 on a computer display. In this embodiment, the output unit 324 creates a display screen including a frame image 1300c selected from the frame images 1300 acquired by the acquisition unit 310 as described in the section <Adjusting the display position of the welded part WP>, and displays the display screen on the computer display.

[0129] In FIG. 14, a display screen 1400 includes a main display area 1410, a sub-display area 1420, and a GUI area 1430. The main display area 1410 displays a frame image 1300c selected by the output unit 324 from among the frame images 1300 acquired by the acquisition unit 310 as described in the section <Adjusting the display position of the welded part WP>, or a frame image 1300 selected by the output unit 324 from among the frame images 1300 acquired by the acquisition unit 310 based on the operation of the operator of the GUI displayed in the GUI area 1430 described later.

[0130] In the sub-display area 1420, the frame image 1300 acquired by the acquisition unit 310, the frame image 1300 displayed in the main area 1410, and the frame images 1300 K frames (K frames) before and after the frame image 1300 are displayed in a reduced form (K is a positive integer). Note that here, a case where K=2 is illustrated. Specifically, here, a reduced image of the frame image 1300 is displayed in the current image display area 1421, reduced images of the frame images 1300 one frame ahead and two frames ahead of the frame image 1300 are displayed in the preceding image display areas 1422 and 1423, respectively, and reduced images of the frame images 1300 one frame behind and two frames behind the frame image 1300 are displayed in the following image display areas 1424 and 1425, respectively. Furthermore, if the frame image 1300 preceding the frame image 1300 displayed in the main area 1410 has not been acquired by the acquisition section 310, a reduced image of that frame image 1300 is not displayed in the preceding image display areas 1422, 1423.

[0131] A GUI (Graphical User Interface) is displayed in the GUI area 1430. In this embodiment, a case in which a previous frame button 1431, a next frame button 1432, a previous welding button 1433, and a next welding button 1434 are displayed in the GUI area 1430 is illustrated as an example. The previous frame button 1431 is a button (GUI) operated by the operator when displaying in the main display area 1410, among the frame images 1300 acquired by the acquisition unit 310, the frame image 1300 one frame before (one frame before) the frame image 1300 displayed in the main display area 1410. When the previous frame button 1431 is operated, the output unit 324 selects the frame image 1300 one frame before (one frame before) the frame image 1300 displayed in the main display area 1410 from among the frame images 1300 acquired by the acquisition unit 310, and displays it in the main display area 1410. In response to a change in the frame image 1300 to be displayed in the main display area 1410, the output unit 324 changes the frame image 1300 to be displayed in a reduced form in the sub-display area 1420 (the current image display area 1421, the preceding image display areas 1422, 1423, and the following image display areas 1424, 1425).

[0132] The next frame button 1432 is a button (GUI) that is operated by the operator when displaying in the main display area 1410 a frame image 1300 one frame after (one frame after) the frame image 1300 displayed in the main display area 1410, from among the frame images 1300 acquired by the acquisition unit 310. When the next frame button 1432 is operated, the output unit 324 selects from the frame images 1300 acquired by the acquisition unit 310 the frame image 1300 one frame after (one frame after) the frame image 1300 displayed in the main display area 1410, and displays it in the main display area 1410. In accordance with a change in the frame image 1300 to be displayed in the main display area 1410, the output unit 324 changes the frame image 1300 to be displayed in a reduced size in the sub-display area 1420 (current image display area 1421, preceding image display areas 1422, 1423, and following image display areas 1424, 1425). In this embodiment, the operation of the previous frame button 1431 and the next frame button 1432 is an example of an operation for specifying an image, among the images acquired by the acquisition unit, whose shooting order differs by a predetermined number from the image being displayed on the display device.

[0133] The previous welding button 1433 is a button (GUI) operated by the operator when displaying in the main display area 1410 the frame image 1300 selected by the output unit 324 as described in the section <Adjustment of the display position of the welded part WP> one before the frame image 1300 displayed in the main display area 1410. As described in the first embodiment, the frame image 1300 determined to have the blobs 730d, 730h indicating the punched hole PH is assigned information on the photographing time of the frame image 1300 and identification information of the welded part WP as an image including the punched hole PH (see the description of step S1206). The output unit 324 specifies the frame image 1300 selected as described in the section <Adjustment of the display position of the welded part WP> one before the frame image 1300 displayed in the main display area 1410 based on the information on the photographing time of the frame image 1300 and the identification information of the welded part WP. In accordance with a change in the frame image 1300 to be displayed in the main display area 1410, the output unit 324 changes the frame image 1300 to be displayed in a reduced size in the sub-display area 1420 (current image display area 1421, preceding image display areas 1422, 1423, and following image display areas 1424, 1425).

[0134] The next welding button 1434 is a button (GUI) operated by the operator when displaying in the main display area 1410 the frame image 1300 selected by the output unit 324 as described in the section <Adjustment of the display position of the welded part WP> one image after the frame image 1300 displayed in the main display area 1410. The output unit 324 specifies the frame image 1300 selected as described in the section <Adjustment of the display position of the welded part WP> one image after the frame image 1300 displayed in the main display area 1410, based on the information on the photographing time of the frame image 1300 and the identification information of the welded part WP. In response to the change of the frame image 1300 to be displayed in the main display area 1410, the output unit 324 changes the frame image 1300 to be displayed in a reduced form in the sub display area 1420 (the current image display area 1421, the preceding image display areas 1422, 1423, and the following image display areas 1424, 1425). In this embodiment, the operation of the previous welding button 1433 and the next welding button 1434 is an example of an operation for specifying an image corresponding to a characteristic image including a detected object determined to be a landmark as an image to be displayed on the display device.

[0135] <Flowchart> Next, an example of a detection method by the detection device 300 of this embodiment will be described with reference to the flowchart in Fig. 15. In the detection method of this embodiment, steps S1208 to S1209 in Fig. 12 described in the first embodiment are replaced with S1501 to S1504 in Fig. 15, and steps S1505 to S1507 are inserted between step S1207 (NO) and step S1101 in Fig. 11. Therefore, detailed description will be omitted here other than the parts replaced with steps S1208 to S1209 in Fig. 12 and the parts inserted between step S1207 (NO) and step S1101 in Fig. 11.

[0136] In this embodiment, similarly to the first embodiment, when NO is determined in steps S1203 and S1205 and the latest image among the images stored as images containing punch holes PH has not been output, the blobs 730d and 730h indicating the punch holes PH in the latest image are deemed to be the blobs located closest to the welded part WP within the detection area 410. Therefore, in this embodiment, when NO is determined in steps S1203 and S1205 in Fig. 12, the process of step S1501 in Fig. 15 is executed to create a display screen as described in <Adjusting the display position of the welded part WP> and display it on the computer display.

[0137] In step S1501, the output unit 324 judges whether or not the process of calculating the movement amount L of the punch holes PH per unit frame (the process of step S1502 described later) has been performed for the latest image of the frame images 1300 stored as images including punch holes PH. If the result of this judgment is that the process of calculating the movement amount L of the punch holes PH per unit frame has been performed for the latest image of the frame images 1300 stored as images including punch holes PH (YES in step S1501), steps S1502 to S1504 using that image have been performed, so that the image has already been displayed (output) in the main display area 1410 of the display screen 1400. In this case, the process of step S1207 is performed. As described in the first embodiment, in step S1207, it is judged whether or not the operation has ended.

[0138] On the other hand, if the process of calculating the movement amount L of the punch hole PH per unit frame has not been executed for the latest image among the frame images 1300 stored as images including the punch hole PH (NO in step S1501), the image needs to be displayed (output) in the main display area 1410 of the display screen 1400. In this case, the process of step S1502 is executed. In step S1502, the output unit 324 calculates, as the movement amount L of the punch hole PH per unit frame, the number of pixels L in the conveyance direction of the steel sheet M (direction of the y-axis) from the position of the center of gravity of the blob indicating the punch hole PH in the latest image (frame image 1300b in the example shown in FIG. 13) among the frame images 1300 stored as images including the punch hole PH to the position of the center of gravity of the blob indicating the punch hole PH in the image of the frame immediately preceding the image (frame image 1300a in the example shown in FIG. 13).

[0139] Next, in step S1503, the output unit 324 divides a preset constant CV by the movement amount L of the punch holes PH per unit frame calculated in step S1502 to calculate the number of frames FN to be changed for the latest image of the frame images 1300 stored as images containing punch holes PH (number of changed frames FN).The output unit 324 then creates information about the display screen 1400 and selects frame image 1300c, which is the number of changed frames FN before the latest image of the frame images 1300 stored as images containing punch holes PH, from among the frame images 1300 acquired by the acquisition unit 310.The output unit 324 also selects frame images 1300 two frames before and after the frame image 1300c selected in this way from among the frame images 1300 acquired by the acquisition unit 310.

[0140] Then, the output unit 324 creates information for displaying the display screen 1400 using the selected frame image 1300. This information includes displaying, in the main display area 1401, a frame image 1300c of a frame (frame) that is the number of changed frames FN before the latest image among the frame images 1300 stored as images including punch holes PH, displaying, in the sub-display area 1420, a reduced image of the frame image 1300c and reduced images of the two frame images 1300 before and after the frame image 1300c, and displaying a GUI (previous frame button 1431, next frame button 1432, previous welding button 1433, and next welding button 1434) in the GUI area 1430 of the display screen 1400.

[0141] Next, in step S1504, the output unit 324 outputs information for displaying the display screen 1400 created in step S1503, and displays the display screen 1400 on a computer display. Then, the process proceeds to step S1207. As described above, in step S1501, if the process for calculating the movement amount L of the punch holes PH per unit frame has been performed for the latest image among the frame images 1300 stored as images including punch holes PH (YES in step S1501), the process of step S1207 is also performed.

[0142] In step S1207, the detection device 300 judges whether the operation has been completed. If the operation has been completed (YES in step S1207), the processing of FIG. 11 and (FIG. 12 modified as in FIG. 15) is terminated. On the other hand, if the operation has not been completed (NO in step S1207) as a result of the judgment in step S1207, the processing of step S1505 is executed to create a display screen as described in the section <Adjustment of the display position of the welded portion WP> and display it on the computer display. In step S1505, the output unit 324 judges whether the operator has operated the display screen 1400. If the operation of the operator has not been performed on the display screen 1400 (NO in step S1505) as a result of the judgment, the processing of step S1101 in FIG. 11 is executed again.

[0143] On the other hand, if an operation by the operator has been performed on the display screen 1400 (YES in step S1505), the process of step S1506 is executed. In step S1506, the output unit 324 creates information for displaying the display screen 1400 based on the content of the operation by the operator on the display screen 1400. In the example shown in Fig. 14, the operation by the operator on the display screen 1400 is the operation of any one of the previous frame button 1431, the next frame button 1432, the previous welding button 1433, and the next welding button 1434. An example of information displayed on the display screen 1400 by these operations is the information described in the section <Adjustment by the operator of the frame image 1300 to be displayed>, and therefore a detailed description of this information will be omitted here. Next, in step S1507, the output unit 324 outputs information for displaying the display screen 1400 created in step S1506, and causes the display screen 1400 to be displayed on a computer display. Then, the process of step S1101 in FIG. 11 is executed again.

[0144] <Summary> As described above, in this embodiment, the detection device 300 selects from the frame images 1300 acquired by the acquisition unit 310 a frame image 1300 in which the position of the welded part WP in the image is closer to the center of the image than the frame image 1300 including the blob determined to be the punched hole PH, and displays the frame image 1300 on a computer display. This makes it possible to prevent a situation in which a part or the whole of the welded part WP is not shown in the frame image 1300, or the welded part WP is displayed in a position that is difficult to view.

[0145] Furthermore, in this embodiment, the detection device 300 selects a frame image 1300 in which the position of the welded part WP in the image is closer to the center of the image than a frame image 1300 including a blob determined to be a punched hole PH, based on the movement amount L of the punched hole PH per unit frame. Therefore, even if the conveying speed of the steel sheet M changes, it is possible to select a frame image 1300 in which the position of the welded part WP in the image is closer to the center of the image.

[0146] Furthermore, in this embodiment, the detection unit 300 selects an image other than the frame image 1300 from among the frame images 1300 acquired by the acquisition unit 310 and displays it on the computer display based on an operation by the operator after the frame image 1300 is displayed in the main display area 1410 of the display screen 1400. Therefore, the operator can check the frame image 1300 other than the frame image 1300 automatically selected and displayed by the detection device 300. This improves the convenience when the operator performs the work of checking the welded part WP.

[0147] For example, the frame image 1300 that the detection device 300 automatically selects and displays in the main display area 1410 of the display screen 1400 is switched every time the camera 120 captures a punch hole PH (weld WP). If the time interval between when the preceding punch hole PH (weld WP) is captured by the camera 120 and when the next punch hole PH (weld WP) following the punch hole PH (weld WP) is captured by the camera 120 is short, the time for switching the frame image 1300 displayed in the main display area 1410 of the display screen 1400 also becomes a time corresponding to the time interval. In such a case, the frame image 1300 displayed in the main display area 1410 may be switched before the confirmation of the preceding weld WP is completed. In such a case, the operator can confirm the preceding weld WP by operating the previous weld button 1433 to display the frame image 1300 for the preceding weld WP.

[0148] Furthermore, even if a blob determined to be a punch hole PH by the detection device 300 is not actually a blob indicating a punch hole PH, the operator can search for and display the desired weld WP by operating the GUI displayed in the GUI area 1430.

[0149] In this embodiment, the various modified examples described in the first embodiment may be adopted. In step S1209 described in the first embodiment, an image including the latest punch hole PH may be displayed in the main display area 1410 of the display screen 1400. In this case, the frame images 1300 of the K frames (K frames) before and after the image including the latest punch hole PH are displayed in a reduced size in the sub-display area 1420. In addition, the frame images 1300 displayed in the main display area 1410 and the sub-display area 1420 are changed as described in this embodiment in response to the operation of the GUI (previous frame button 1431, next frame button 1432, previous welding button 1433, and next welding button 1434) displayed in the GUI area 1430.

[0150] (Other embodiments) The above-described embodiment of the present invention can be realized by a computer executing a program. In addition, a computer-readable recording medium on which the program is recorded and a computer program product such as the program can also be applied as an embodiment of the present invention. As the recording medium, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a non-volatile memory card, a ROM, etc. can be used. In addition, the embodiment of the present invention may be realized by a PLC (Programmable Logic Controller) or by dedicated hardware such as an ASIC. Furthermore, the above-described embodiments of the present invention are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0151] 101 Payoff Reel 102 Welding equipment 103 Pickling equipment 104 Cold rolling equipment 105 Annealing equipment 106 Tension Reel 110a~110d Deflector roll 120 Camera 120a Optical axis 130 Video Transmitter 200 Video receiver 300 Detection Device 310 Acquisition Department 320 Judgment section 321 Trimming Department 322 Image Processing Unit 322a Filter section 322b Grayscale conversion section 322c Adaptive binarization processing unit 322d Morphology Processing Section 323 Discrimination part 324 Output section 400 Images of the current frame 410 Detection Area 411 The edge of the detection area on the positive side of the y-axis 511 Pixel of interest 512 pixels in a rectangular kernel 521 pixel of interest 522 pixels in a rectangular kernel 610 Binarized Image 611~613 Binary image area 620 Binarized image of the current frame after closing process 630, 630a to 630c: Feature image of the current frame (binarized image of the current frame after closing and opening processing) 621 Feature Image Region 710 Outer pixels 720 Search Path 730, 730a~730k Blob 1300 frame images 1300a: A frame image one frame before the frame image from which the blob located closest to the weld within the detection area is calculated 1300b Frame image from which the blob closest to the weld within the detection area is calculated 1300c Four frame images before the frame image from which the blob closest to the weld within the detection area was calculated (the frame image to be output) 1400 display screen 1410 Main display area 1420 Sub display area 1421 Current image display area 1422~1423 Pre-image display area 1424~1425 Rear image display area 1430 GUI area 1431 Previous frame button 1432 Next frame button 1433 Front Welded Button 1434 Next Weld Button M steel plate MF Leading Steel Plate MA Trailing steel plate PH Punch Hole WP weld

Claims

1. A detection device that detects a joint of a metal material using a camera, an acquisition unit that acquires a plurality of images of the metal material being transported, the images being captured by the camera at different times; a determination unit that determines the presence or absence of the joint based on the plurality of images, The determination unit is an image processing unit that performs image processing on each of the plurality of images to detect, as a detection object, a candidate mark provided on the metal material to indicate the position of the joint, thereby acquiring a feature image for each of the plurality of images; a discrimination unit that discriminates whether the detected object is the landmark based on a mutual positional relationship of the detected object detected from each of the two or more characteristic images; An output unit that outputs information indicating that the joint has been detected when the detected object is determined to be the mark.

2. The relative positions of the detected objects are as follows: The detection device according to claim 1 , further comprising a positional relationship of the detected object in at least one of a conveyance direction and a width direction of the metal material.

3. When the detected object is determined to be the mark, the positional relationship of the detected object in the conveying direction of the metal material is as follows: The detection device according to claim 2, including a relationship in which the detected object detected from the characteristic image acquired at a relatively later timing is located downstream in the transport direction of the metal material than the detected object detected from the characteristic image acquired at a relatively earlier timing.

4. When the detected object is determined to be the mark, the positional relationship of the detected object in the width direction of the metal material is as follows: The detection device according to claim 2 or 3, comprising a relationship in which a difference in the position in the width direction of the metal material between the detected object detected from the characteristic image acquired at a relatively earlier time and the detected object detected from the characteristic image acquired at a relatively later time is less than a predetermined value.

5. The discrimination unit is A detection device according to any one of claims 1 to 4, which determines whether the detected object is the landmark based on the mutual positional relationship of the detected object detected from each of two or more of the characteristic images and the area of ​​the detected object detected from at least one of the two or more characteristic images.

6. The acquisition unit is The detection device according to any one of claims 1 to 5, wherein a plurality of images are obtained by successively photographing the metal material being transported by the camera as an object to be photographed.

7. The image processing unit includes: The detection device according to claim 1 , further comprising an adaptive binarization processor that performs adaptive binarization processing on the image.

8. The image processing unit includes: A filter unit that smoothes the luminance of the image is further provided. The adaptive binarization processing unit includes: The detection device according to claim 7 , wherein the adaptive binarization process is performed on the image that has been smoothed by the filter unit.

9. The image processing unit includes: The detection device according to claim 7 or 8, further comprising a morphology processing unit that performs processing including dilation and erosion on the image after the adaptive binarization processing has been performed.

10. The determination unit is A trimming unit is further provided for trimming an area from the image where the mark is assumed to be present, The image processing unit includes: The detection device according to claim 1 , wherein the characteristic image is acquired based on an image cropped by the cropping unit.

11. The output unit is The detection device according to any one of claims 1 to 10, further comprising: a display device that displays, among the images acquired by the acquisition unit, an image corresponding to the characteristic image including the detected object determined to be the landmark.

12. The output unit is A detection device as described in any one of claims 1 to 10, wherein an image in which the position of the joint in the image is closer to the center of the image than an image corresponding to the characteristic image including the detected object determined to be the landmark is selected from the images acquired by the acquisition unit and displayed on a display device.

13. The acquisition unit is A plurality of images are continuously taken at a predetermined frame rate by the camera, the image being captured of the metal material being transported; The output unit is The detection device according to claim 12, further comprising: a detection unit that selects from among the images acquired by the acquisition unit an image in which the position of the joint in the image is closer to the center of the image than an image corresponding to the characteristic image including the detected object determined to be the landmark, based on the amount of movement of the detected object per unit frame.

14. The output unit is The detection device according to any one of claims 11 to 13, wherein, based on an operation by an operator after the image is displayed on a display device, an image other than the image is selected from the images acquired by the acquisition unit and displayed on the display device.

15. 15. The detection device according to claim 14, wherein the operation by the operator includes at least one of an operation for designating, as an image to be displayed on the display device, an image corresponding to the characteristic image including the detected object determined to be the landmark, and an operation for designating, among the images acquired by the acquisition unit, an image whose shooting order differs by a predetermined number from the image being displayed on the display device.

16. The mark is: The detection device according to any one of claims 1 to 15, wherein the detection device is one or more through holes provided in the metal material.

17. The joint is A joint between the preceding metal material and the following metal material is included, The acquisition unit is The detection device according to any one of claims 1 to 16, further comprising: a camera configured to capture a plurality of images of the metal material being transported on a continuous processing line at different times.

18. The continuous processing line comprises: A continuous annealed steel sheet production line including a welding machine, a cold rolling machine arranged downstream of the welding machine, and an annealing facility arranged downstream of the cold rolling machine, The camera includes: The detection device according to claim 17, which is installed at a position capable of photographing the steel sheet being transported between an outlet of the cold rolling mill and an inlet of the annealing facility.

19. The camera includes:

19. The detection device according to claim 18, which is installed at a position capable of photographing the steel plate being transported upstream of a spare welding facility provided between the cold rolling mill and the annealing facility.

20. A detection method for detecting a joint of a metal material using a camera, comprising: an acquisition step of acquiring a plurality of images of the metal material being transported, the images being captured by the camera at different times; and determining whether or not the joint is present based on the plurality of images. The determination step includes: an image processing step of acquiring a feature image for each of the plurality of images by performing image processing for each of the plurality of images to detect, as a detection object, a candidate for a mark provided on the metal material to indicate the position of the joint; a determination step of determining whether or not the detected object is the landmark based on a mutual positional relationship of the detected object detected from each of the two or more characteristic images; and an output step of outputting information indicating that the joint has been detected when the detected object is determined to be the mark.

21. A program for causing a computer to function as each part of the detection device according to any one of claims 1 to 19.

Images