Surface defect detection device and surface defect detection method in steel plate continuous line

Abstract

To provide a surface defect detection device and a surface defect detection method in a steel plate continuous line capable of accurately and instantaneously detecting the kind of a surface defect on the surface of a traveling or stopping steel plate and the area of the surface defect on line.SOLUTION: A surface defect detection device 10 in a steel plate continuous line includes a captured image acquisition unit 12 that acquires a captured image obtained by capturing an image of a surface Sa of a traveling or stopped steel plate S, and a detection unit 13 that detects a type of a surface defect and a region of the surface defect by inputting the captured image acquired by the captured image acquisition unit 12 to a surface defect detection model generated by machine learning of a plurality of pieces of learning data in which a past captured image obtained by capturing an image of the surface Sa of the steel plate S is used as input data and the type of the surface defect and the region of the surface defect with respect to the input data are used as output data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a surface defect detection device and a surface defect detection method for use in a continuous steel plate production line, which detects surface defects on the surface of a running or stopped steel plate. [Background technology]

[0002] For example, oil is applied to the surface of hot-dip galvanized steel sheets produced through the hot-dip galvanizing process for the purposes of rust prevention and lubrication. The oil application method is typically a continuous application method using an electrostatic oil applicator. An electrostatic oil applicator is equipped with a spray bar that sprays air and oil, and controls the amount of oil applied by adjusting the amount of conveying air while maintaining a constant atomizing air pressure. If air gets trapped in the spray bar during the oil application process, uneven oil application (partial oil application failure) occurs on the surface of the steel plate. Uneven oil application on the surface of the steel plate is problematic because it reduces the anti-rust and lubricating effects.

[0003] For this reason, conventionally, oil unevenness occurring on the surface of a steel sheet is monitored by a camera installed on the outlet side of the electrostatic oil applicator, and an operator visually detects oil unevenness from the image captured by the camera. If oil unevenness is detected, the electrostatic oil applicator is turned off, and inspection and cleaning are carried out to eliminate the oil unevenness. In addition, steel sheets on which oil unevenness has occurred are re-oiled on the recoil line.

[0004] On the other hand, as a method for measuring the amount of oil applied to the surface of a steel sheet, for example, a method for measuring the amount of oil applied to the surface of a metal material, as disclosed in Patent Document 1, has been known. The method for measuring the amount of oil applied to a metal surface described in Patent Document 1 involves irradiating the oil-applied metal surface with pulsed excitation light of a specific wavelength while scanning the irradiation position within a predetermined microscopic range, and measuring the amount of oil applied to the metal surface from the integrated value of the intensity of the fluorescence generated by this irradiation within the microscopic range. This allows for accurate offline measurement of the amount of oil applied to the metal surface. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 9-210908 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when detecting uneven oil coating on the surface of a steel sheet by visual inspection, the uneven oil coating may be overlooked. Furthermore, the method for measuring the amount of oil applied to the surface of a metal material shown in Patent Document 1 can accurately measure the amount of oil applied to the surface of a metal material. However, because the amount of oil applied to the surface of a sample steel plate is measured offline, it is not possible to instantly detect surface defects, such as uneven oil application, that occur on the surface of a moving or stationary steel plate online. Furthermore, although it is possible to measure the amount of oil applied at a point on the surface of a metal material within a certain linear range using a scanning mechanism, in principle it is not possible to measure the amount of oil applied online across the surface of a moving or stationary metal material.

[0007] Therefore, the present invention has been made to solve these conventional problems, and its object is to provide a surface defect detection device and a surface defect detection method for a continuous steel plate line that can accurately and instantly detect the type and area of ​​surface defects on the surface of a running or stopped steel plate online. [Means for solving the problem]

[0008] In order to solve the above problems, a surface defect detection device for a continuous steel plate line according to one embodiment of the present invention is a surface defect detection device for a continuous steel plate line that detects the type and area of ​​surface defects on the surface of a running or stopped steel plate, and is summarized as comprising: an image acquisition unit that acquires an image of the surface of the running steel plate; and a detection unit that inputs the image acquired by the image acquisition unit into a surface defect detection model that is generated by machine learning multiple learning data in which past images of the surface of the steel plate are used as input data and the type and area of ​​surface defects for this input data are output data, and detects the type and area of ​​surface defects.

[0009] Another aspect of the present invention relates to a surface defect detection method for a continuous steel plate line, which is a surface defect detection method for a continuous steel plate line that detects the type and area of ​​surface defects on the surface of a running or stopped steel plate, and is summarized as including an image acquisition step of acquiring an image of the surface of the running steel plate, and a detection step of inputting the image acquired in the image acquisition step into a surface defect detection model that is generated by machine learning multiple learning data in which past images of the surface of the steel plate are used as input data and the types of surface defects and areas of surface defects corresponding to this input data are output data, and detecting the type and area of ​​surface defects. [Effects of the Invention]

[0010] According to the surface defect detection device and surface defect detection method for a continuous steel plate line of the present invention, the type and area of ​​surface defects on the surface of a running or stopped steel plate can be detected online with high accuracy and in real time. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic configuration diagram of a continuous steel sheet production line equipped with a surface defect detection device according to a first embodiment of the present invention. [Figure 2] 2 is a flowchart for explaining the flow of processing in the imaging device and the surface defect detection device shown in FIG. [Figure 3] FIG. 1 is a diagram showing an example of a captured image of the surface of a steel sheet on which uneven oil coating has occurred. [Figure 4] FIG. 10 is a diagram showing an example of an image of the surface of a steel sheet when the surface of the steel sheet is imaged when uneven oil application begins to occur. [Figure 5] FIG. 1 is a diagram showing an example of a captured image of the surface of a steel sheet on which uneven baking has occurred. [Figure 6] FIG. 2 is a diagram showing an example of a captured image of the surface of a steel sheet on which no surface defects have occurred. [Figure 7] FIG. 1 is a schematic diagram of a continuous steel sheet production line equipped with a surface defect detection device according to a second embodiment of the present invention. [Figure 8] 8 is a flowchart for explaining the flow of processing in the imaging device and the surface defect detection device shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments shown below are examples of devices and methods for embodying the technical concept of the present invention, and the technical concept of the present invention is not limited to the following embodiments in terms of the materials, shapes, structures, arrangements, etc. of the components. In addition, the drawings are schematic, and therefore it should be noted that the relationship between thickness and planar dimensions, ratios, etc. may differ from the actual relationship, and the dimensional relationships and ratios may differ between the drawings.

[0013] (First embodiment) FIG. 1 shows a schematic configuration of a continuous steel sheet production line equipped with a surface defect detection device according to a first embodiment of the present invention. In a continuous steel sheet production line 1 shown in Figure 1, for example, oil is applied to a surface Sa of a hot-dip galvanized steel sheet (hereinafter simply referred to as steel sheet) S produced through a hot-dip galvanizing process. The steel sheet S produced through the hot-dip galvanizing process is transported in the direction of the arrow in Figure 1 by transport rolls 3, and oil is applied to the surface Sa of the traveling steel sheet S by an electrostatic oil applicator 2. This oil application is performed for the purposes of rust prevention and lubrication of the steel sheet S. The electrostatic oil applicator 2 is equipped with a spray bar (not shown) that sprays air and oil, and controls the amount of oil application by the amount of transport air while maintaining a constant atomizing air pressure.

[0014] During oil application work using this electrostatic oil applicator 2, if air becomes clogged in the spray bar, uneven oil application occurs on the surface Sa of the steel sheet S. If this uneven oil application occurs on the surface Sa of the steel sheet S, it will cause a problem because it will lead to a decrease in the rust prevention effect and lubrication effect.

[0015] Furthermore, in the case of the method for measuring the amount of oil applied to the surface of a metal material shown in Patent Document 1, the amount of oil applied to the surface of a sample steel sheet is measured offline, making it impossible to instantly detect surface defects online, such as oil application unevenness M1 (see FIG. 3) that occurs on the surface Sa of a steel sheet S that is traveling or stationary. Furthermore, in the case of the method for measuring the amount of oil applied to the surface of a metal material shown in Patent Document 1, the steel sheet is irradiated with a pulsed laser, and the amount of oil applied is measured as a linear or surface average of the fluorescence intensity of the oil surface, making it impossible to accurately detect oil application unevenness M1. Furthermore, in the case of the method for measuring the amount of oil applied to the surface of a metal material shown in Patent Document 1, the fluorescence intensity of the oil surface is measured, making it impossible to detect baking unevenness M2 (see FIG. 4).

[0016] Therefore, in this embodiment, an imaging device 4 is installed in the continuous steel sheet line 1 downstream of the electrostatic oil applicator 2 in the conveying direction to capture an image of the surface Sa of the traveling or stopped steel sheet S, and a surface defect detection device 10 is connected to the imaging device 4. The surface defect detection device 10 inputs the image captured by the imaging device 4 into a surface defect detection model described below to detect the type and area of ​​surface defects on the surface Sa of the traveling steel sheet S. In this embodiment, the types of surface defects are, as examples, oil application unevenness M1 shown in Figure 3, oil application unevenness (start) M1S shown in Figure 4, oil application unevenness (end) not shown, or bake unevenness M2 shown in Figure 5. Furthermore, the area of ​​surface defects is an area where there is a surface defect on the steel sheet S, and in the case of the oil application unevenness M1 shown in Figure 3 as an example, it is represented by an area indication line R1 surrounding the oil application unevenness M1, in the case of the oil application unevenness (start) M1S shown in Figure 4, it is represented by an area indication line R1S surrounding the oil application unevenness (start) M1S, in the case of the oil application unevenness (end), it is represented by an area indication line (not shown) surrounding the oil application unevenness (end), and in the case of the baking unevenness M2 shown in Figure 5, it is represented by an area indication line R2 surrounding the baking unevenness M2.

[0017] Then, for a steel sheet S in which oil application unevenness M1, oil application unevenness (start of exit) M1S, and oil application unevenness (end of exit) have been detected, an alarm is sounded in the operator's cab at the exit side of the continuous steel sheet line 1, as will be described later, and an operator visually checks again to determine whether or not oil application unevenness M1, oil application unevenness (start of exit) M1S, and oil application unevenness (end of exit) are present on the surface Sa of the steel sheet S. If oil application unevenness M1, etc. are detected visually, shipment is suspended. If oil application unevenness M1, etc. are not detected visually, the steel sheet S is treated in the same manner as the next steel sheet S in which oil application unevenness M1, oil application unevenness (start of exit) M1S, and oil application unevenness (end of exit) are not detected. On the other hand, for steel sheets S in which oil coating unevenness M1, oil coating unevenness (beginning) M1S, and oil coating unevenness (end) are not detected, if baking unevenness M2 is detected, the coil is rewound on the recoil line to remove the baking unevenness M2 before shipping. Also, steel sheets S in which oil coating unevenness M1, oil coating unevenness (beginning) M1S, oil coating unevenness (end) and baking unevenness M2 are not detected are shipped as having no surface defects.

[0018] As shown in Figure 3, oil application unevenness M1 generally occurs as a white streak of approximately constant width extending along the longitudinal direction of the steel sheet S. Once an oil application unevenness M1 occurs, it continues to occur for 40 m or more. However, if an image of the surface Sa of the steel sheet S is taken when the oil application unevenness M1 begins to occur, a short oil application unevenness (start of appearance) M1S can be captured, as shown in Figure 4. Similarly, if an image of the surface Sa of the steel sheet S is taken when the oil application unevenness M1 has finished occurring, a short oil application unevenness (end of appearance) can be captured, although not shown.

[0019] Furthermore, as shown in Figure 5, baking unevenness M2 is very similar to oil coating unevenness M1, but unlike oil coating unevenness M1, which is a white streak of almost constant width that extends along the longitudinal direction of the steel sheet S, baking unevenness M2 is generally formed as a white spot pattern of uneven width that extends along the longitudinal direction of the steel sheet S. Baking unevenness M2 is a surface defect that occurs due to poor alloying of the steel sheet S.

[0020] Here, the imaging device 4 is installed so as to be able to monitor the surface Sa of the steel sheet S from above, and is composed of, for example, a CCD camera that images the surface Sa of the steel sheet S. The imaging device 4 captures images at 30 frames per second, and images the surface Sa of the steel sheet S over the entire length from the leading edge to the trailing edge of the steel sheet S that is traveling or stopped while passing through the electrostatic oil coating device 2. The imaging device 4 is connected to the surface defect detection device 10, and the captured image of the surface Sa of the steel sheet S captured by the imaging device 4 is transmitted to the surface defect detection device 10.

[0021] The surface defect detection device 10 detects the types and areas of the surface defects on the surface Sa of the steel sheet S based on the captured image of the surface Sa of the steel sheet S captured by the imaging device 4. The surface defect detection device 10 includes a memory unit 11, a captured image acquisition unit 12, a detection unit 13, and an output unit 14.

[0022] The surface defect detection device 10 is a computer system having a processing function for implementing each function of the image acquisition unit 12, the detection unit 13, and the output unit 14 by executing a program on computer software. This computer system is configured with a ROM, a RAM, a CPU, etc., and implements each of the above-mentioned functions on software by executing various dedicated programs pre-stored in the ROM, etc.

[0023] The storage unit 11 stores the surface defect detection model input from the input device 21. This surface defect detection model is generated by machine learning a plurality of learning data in which a past image of the surface Sa of the steel sheet S is used as input data and the types and areas of the surface defects corresponding to the input data are used as output data. The types and areas of the surface defects are as described above.

[0024] Here, the machine learning technique is a neural network with an object detection function, and the surface defect detection model is a detection model constructed by the neural network. The captured image acquisition unit 12 acquires captured images of the surface Sa of the traveling or stopped steel sheet S captured by the imaging device 4. The captured image acquisition unit 12 acquires the captured images at regular intervals.

[0025] Furthermore, the detection unit 13 first reads the surface defect detection model stored in the storage unit 11. Next, the detection unit 13 inputs the captured image acquired by the captured image acquisition unit 12 into the read surface defect detection model. Then, the detection unit 13 detects the type and area of ​​the surface defect present on the steel sheet S in the captured image input to the detection unit 13 using the surface defect detection model. Specifically, the detection unit 13 detects oil application unevenness M1, oil application unevenness (start) M1S, oil application unevenness (end) or baking unevenness M2, which are types of surface defects present on the steel sheet S in the captured image. The detection unit 13 also detects the area of ​​the surface defect (if the surface defect is an oil unevenness M1, it detects an area indicating line R1 surrounding the oil unevenness M1; if the surface defect is an oil unevenness (start) M1S, it detects an area indicating line R1S surrounding the oil unevenness (start) M1S; if the surface defect is an oil unevenness (end) it detects an area indicating line surrounding the oil unevenness (end); if the surface defect is a baking unevenness M2, it detects an area indicating line R2 surrounding the baking unevenness M2). The detection unit 13 then sends the detection results, information on the type of surface defect and the area of ​​the surface defect, to the output unit 14.

[0026] In addition, if there are no surface defects on the steel plate S in the captured image input to the detection unit 13, as shown in Figure 6, information on the type of surface defect and the absence of any surface defect area is sent from the detection unit 13 to the output unit 14. In addition, the output unit 14 outputs information on the type of surface defect and the area of ​​the surface defect on the steel plate S in the captured image detected by the detection unit 13, or information on the type of surface defect and the absence of any area of ​​the surface defect, to the output device 22 constituting the display device.

[0027] The output device 22 then displays information on the type of surface defect and the area of ​​the surface defect present in the steel sheet S in the captured image, or information on the type of surface defect and the area of ​​the surface defect completely absent (i.e., the display screen of the output device 22 displays nothing). Specifically, when a surface defect present in the steel sheet S in the captured image detected by the detection unit 13 is an oil application unevenness M1, the output device 22 displays the text "oil application unevenness" indicating the oil application unevenness M1 and an area display line R1 surrounding the oil application unevenness M1 in the captured image. Furthermore, when a surface defect present in the steel sheet S in the captured image detected by the detection unit 13 is an oil application unevenness (start of appearance) M1S, the output device 22 displays the text "oil application unevenness (start of appearance)" indicating the oil application unevenness (start of appearance) M1S and an area display line R1S surrounding the oil application unevenness (start of appearance) M1S in the captured image. Furthermore, when the surface defect on the steel sheet S in the captured image detected by the detection unit 13 is an oil unevenness (end), the output device 22 displays the text "oil unevenness (end)" indicating the oil unevenness (end) and an area display line surrounding the oil unevenness (end) in the captured image. Furthermore, when the surface defect on the steel sheet S in the captured image detected by the detection unit 13 is a toasting unevenness M2, the output device 22 displays the text "toasting unevenness" indicating the toasting unevenness M2 and an area display line R2 surrounding the toasting unevenness M2 in the captured image.

[0028] Next, the processing flow in the imaging device 4 and the surface defect detection device 10 will be described with reference to Fig. 2. Fig. 2 is a flowchart for explaining the processing flow in the imaging device and the surface defect detection device shown in Fig. 1.

[0029] First, before detecting the type and area of ​​surface defects on the surface Sa of the steel sheet S, the detection unit 13 of the surface defect detection device 10 reads the surface defect model stored in the memory unit 11 in step S1 (surface defect model reading step).

[0030] As described above, the surface defect model is input to the storage unit 11 from the input device 21. The surface defect detection model is generated by machine learning a plurality of learning data in which a past image of the surface Sa of the steel sheet S is used as input data and the types and areas of surface defects corresponding to this input data are output data. The types and areas of surface defects are as described above. As a machine learning technique, a neural network with an object detection function is used, and the surface defect detection model is a detection model constructed by this neural network. As a result, the types and areas of surface defects on the surface Sa of the steel sheet S can be detected online with high accuracy and in real time using the surface defect detection model constructed by the neural network with an object detection function.

[0031] Next, in step S2, the imaging device 4 captures an image of the surface Sa of the traveling or stopped steel sheet S (imaging step). Here, the imaging device 4 captures 30 frames of images per second, and captures images of the surface Sa of the steel sheet S over the entire length from the leading end to the trailing end of the traveling steel sheet S passing through the electrostatic oil coating device 2.

[0032] Next, in step S3, the captured image acquisition unit 12 of the surface defect detection device 10 acquires captured images of the surface Sa of the steel sheet S, which is traveling or stopped by the imaging device 4 in step S2 (captured image acquisition step). In step S3, the captured image acquisition unit 12 acquires the captured images at regular intervals.

[0033] Next, in step S4, the detection unit 13 of the surface defect detection device 10 inputs the captured image acquired by the captured image acquisition unit 12 in step S3 into the surface defect detection model read by the detection unit 13 in step S1, and detects the type of surface defect and the area of ​​the surface defect (detection step).

[0034] Specifically, the detection unit 13 uses a surface detection model to detect oil application unevenness M1, oil application unevenness (start) M1S, oil application unevenness (end) or toasting unevenness M2, which are types of surface defects on the steel sheet S in the captured image input to the detection unit 13. The detection unit 13 also detects the area of ​​the surface defect (if the surface defect is oil application unevenness M1, it detects an area display line R1 surrounding the oil application unevenness M1; if the surface defect is oil application unevenness (start) M1S, it detects an area display line R1 surrounding the oil application unevenness (start) M1S; if the surface defect is oil application unevenness (end), it detects an area display line surrounding the oil application unevenness (end); if the surface defect is toasting unevenness M2, it detects an area display line R2 surrounding the toasting unevenness M2).

[0035] Then, the detection unit 13 sends information on the type of surface defect and the area of ​​the surface defect, which are the detection results, to the output unit 14. In addition, if there are no surface defects on the steel plate S in the captured image input to the detection unit 13, as shown in Figure 6, information on the type of surface defect and the absence of any surface defect area is sent from the detection unit 13 to the output unit 14.

[0036] Next, in step S5, the output section 14 of the surface defect detection device 10 outputs information on the type of surface defect and the area of ​​the surface defect on the steel plate S in the captured image detected by the detection section 13 in step S4 (detection step), or information on the type of surface defect and the absence of any area of ​​the surface defect, to the output device 22 constituting the display device (output step). This completes the processing in the imaging device 4 and the surface defect detection device 10.

[0037] The output device 22 displays information on the type of surface defect and the area of ​​the surface defect in the steel sheet S in the captured image, or information on the type of surface defect and the absence of any area of ​​the surface defect. Specifically, if the surface defect in the steel sheet S in the captured image detected in step S4 is an oil application unevenness M1, the output device 22 displays the characters "oil application unevenness" indicating the oil application unevenness M1 and an area display line R1 surrounding the oil application unevenness M1 in the captured image. Furthermore, if the surface defect in the steel sheet S in the captured image detected in step S4 is an oil application unevenness (start of appearance) M1S, the output device 22 displays the characters "oil application unevenness (start of appearance)" indicating the oil application unevenness (start of appearance) M1S and an area display line R1S surrounding the oil application unevenness (start of appearance) M1S in the captured image. Furthermore, if the surface defect on the steel sheet S in the captured image detected in step S4 is an oil application unevenness (end), the output device 22 displays the text "oil application unevenness (end)" indicating the oil application unevenness (end) and an area display line surrounding the oil application unevenness (end) in the captured image. Furthermore, if the surface defect on the steel sheet S in the captured image detected in step S4 is a toasting unevenness M2, the output device 22 displays the text "toasting unevenness" indicating the toasting unevenness M2 and an area display line R2 surrounding the toasting unevenness M2 in the captured image. Furthermore, if there is no surface defect and information indicating the type of surface defect and the absence of any surface defect area is sent from the detection unit 13 to the output unit 14, the output device 22 displays information indicating the type of surface defect and the absence of any surface defect area.

[0038] Then, when oil application unevenness M1, oil application unevenness (start of output) M1S, and oil application unevenness (end of output) are detected and the output device 22 displays the text "oil application unevenness" and a region indicator line R1 surrounding the oil application unevenness M1 in the captured image, the text "oil application unevenness (start of output)" and a region indicator line R1S surrounding the oil application unevenness (start of output) M1S in the captured image, or the text "oil application unevenness (end of output)" and a region indicator line surrounding the oil application unevenness (end of output) in the captured image, an alarm is issued for that steel sheet S in the operator's cab on the outlet side of the continuous steel sheet line 1. Then, the worker visually inspects the surface Sa of the steel sheet S again to determine whether oil application unevenness M1, oil application unevenness (start of output) M1S, or oil application unevenness (end of output) is present. If oil application unevenness M1 or the like is detected visually, the shipment is suspended. If these oil coating unevennesses M1 etc. are not detected by visual inspection, the same procedures shall be followed as for the case where oil coating unevenness M1, oil coating unevenness (start) M1S, and oil coating unevenness (end) are not detected.

[0039] On the other hand, if oil application unevenness M1, oil application unevenness (start) M1S, and oil application unevenness (end) are not detected, but toasting unevenness M2 is detected and the output device 22 displays the words "toasting unevenness" and an area indication line R2 surrounding the toasting unevenness M2 in the captured image, the worker rewinds the coil using the recoil line to remove the toasting unevenness M2 and ships the steel sheet S. Also, if oil application unevenness M1, oil application unevenness (start) M1S, oil application unevenness (end) and toasting unevenness M2 are not detected and the output device 22 displays information on the type of surface defect and the absence of any surface defect area, the steel sheet S is shipped as having no surface defects.

[0040] As described above, the surface defect detection device 10 for a steel sheet continuous line according to the first embodiment is equipped with an image acquisition unit 12 that acquires an image of the surface Sa of a steel sheet S that is traveling or stopped, and a detection unit 13 that detects the type and area of ​​the surface defect by inputting the image acquired by the image acquisition unit 12 into a surface defect detection model that is generated by machine learning multiple learning data, using past images of the surface Sa of the steel sheet S as input data and outputting the type of surface defect (oil application unevenness M1, oil application unevenness (start) M1S, oil application unevenness (end) or baked unevenness M2) and the area of ​​the surface defect (area display line R1 surrounding the oil application unevenness M1, area display line R1S surrounding the oil application unevenness (start) M1S, area display line surrounding the oil application unevenness (end) or area display line R2 surrounding the baked unevenness M2).

[0041] This makes it possible to instantly and accurately detect the type and area of ​​a surface defect on the surface Sa of a traveling or stopped steel sheet S online. Since not only the type of surface defect but also the area of ​​the surface defect is detected and output, if a surface defect is erroneously detected and an area different from the area where the surface defect is located is indicated, it is possible to know which part of the image was erroneously detected, and therefore the cause can be quickly investigated.

[0042] Furthermore, the surface defect detection method for a continuous steel sheet line according to the first embodiment includes an image acquisition step (step S3) for acquiring an image of the surface Sa of a steel sheet S that is traveling or stopped, and a detection step (step S4) for inputting the image acquired in the image acquisition step (step S3) into a surface defect detection model that is generated by machine learning a plurality of learning data in which past images of the surface Sa of the steel sheet S are used as input data and the types of surface defects and areas of surface defects corresponding to this input data are used as output data, and detecting the types of surface defects and areas of surface defects.

[0043] This makes it possible to instantly detect the type and area of ​​surface defects on the surface Sa of a traveling or stopped steel sheet S online with high accuracy. Furthermore, in the surface defect detection device 10 and surface defect detection method for a steel sheet continuous line according to the first embodiment, the surface defects to be detected are oil application unevenness M1 (including oil application unevenness (start) M1S and oil application unevenness (end)) and bake unevenness M2 that have occurred on the surface Sa of the steel sheet S. This makes it possible to instantly and accurately detect the oil application unevenness M1 (including oil application unevenness (start) M1S and oil application unevenness (end)) and bake unevenness M2 and their areas on the surface Sa of the traveling steel sheet S online.

[0044] (Second embodiment) Next, a surface defect detection device and a surface defect detection method for a continuous steel sheet line according to a second embodiment of the present invention will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a schematic configuration diagram of a continuous steel sheet line equipped with a surface defect detection device according to the second embodiment of the present invention. Fig. 8 is a flowchart for explaining the processing flow in the imaging device and surface defect detection device shown in Fig. 7.

[0045] The surface defect detection device 10 according to the second embodiment of the present invention shown in Figure 7 differs from the surface defect detection device 10 according to the first embodiment shown in Figure 1 in that it is equipped with a pre-processing unit 15 that performs distortion correction processing on the captured image acquired by the captured image acquisition unit 12 and trimming processing on the distortion-corrected captured image to set the portion of the steel plate S in the captured image that is within the range of the detection target.

[0046] The surface defect detection device 10 and the surface defect processing method according to the second embodiment will be described in detail below. In the continuous steel sheet line 1 shown in Figure 7, a steel sheet S manufactured through a hot-dip galvanizing process is transported in the direction of the arrow in Figure 7 by a transport roll 3, and oil is applied to the surface Sa of the traveling steel sheet S by an electrostatic oil applicator 2.

[0047] An imaging device 4 is installed downstream in the conveying direction of the electrostatic oil coating device 2 to capture an image of the surface Sa of the traveling or stopped steel sheet S, and a surface defect detection device 10 is connected to the imaging device 4. In the surface defect detection device 10, the image captured by the imaging device 4 is input into a surface defect detection model similar to the surface defect detection model in the first embodiment to detect the type and area of ​​surface defects on the surface Sa of the traveling or stopped steel sheet S. The type and area of ​​surface defects are the same as those described in the description of the first embodiment.

[0048] Here, the imaging device 4 is installed so as to be able to monitor the surface Sa of the steel sheet S from above, and is composed of, for example, a CCD camera that images the surface Sa of the steel sheet S. The imaging device 4 captures images at 30 frames per second, and images the surface Sa of the steel sheet S over the entire length from the leading edge to the trailing edge of the steel sheet S that is traveling or stopped while passing through the electrostatic oil coating device 2. The imaging device 4 is connected to the surface defect detection device 10, and the captured image of the surface Sa of the steel sheet S captured by the imaging device 4 is transmitted to the surface defect detection device 10.

[0049] The surface defect detection device 10 detects the types and areas of the surface defects on the surface Sa of the steel sheet S based on the captured image of the surface Sa of the steel sheet S captured by the imaging device 4. The surface defect detection device 10 includes a memory unit 11, a captured image acquisition unit 12, a pre-processing unit 15, a detection unit 13, and an output unit 14.

[0050] The surface defect detection device 10 is a computer system having a processing function for implementing each function of the captured image acquisition unit 12, pre-processing unit 15, detection unit 13, and output unit 14 by executing a program on computer software. This computer system is configured with a ROM, RAM, CPU, etc., and implements each of the above-mentioned functions on software by executing various dedicated programs pre-stored in the ROM, etc.

[0051] The storage unit 11 stores the surface defect detection model input from the input device 21. This surface defect detection model was generated by machine learning multiple learning data in which a past image of the surface Sa of the steel sheet S was used as input data and the types and areas of surface defects corresponding to this input data were used as output data. The machine learning method was a neural network with an object detection function, and the surface defect detection model was a detection model constructed by this neural network.

[0052] The captured image acquisition unit 12 acquires captured images of the surface Sa of the traveling or stopped steel sheet S captured by the imaging device 4. The captured image acquisition unit 12 acquires the captured images at regular intervals. In addition, the pre-processing unit 15 performs distortion correction processing on the captured image acquired by the captured image acquisition unit 12, and a trimming processing on the captured image that has undergone distortion correction processing to set the portion of the steel plate S in the captured image that is within the detection target range.

[0053] The captured image captured by the imaging device 4 and acquired by the captured image acquisition unit 12 is an image of the steel sheet S taken from an oblique angle, and is captured in a distorted state of the steel sheet S due to lens distortion of the CCD camera constituting the imaging device 4. Furthermore, the captured image also captures parts other than the steel sheet S. Therefore, the captured image acquired by the captured image acquisition unit 12 is input directly to a surface defect model by the detection unit 13 to detect the type and area of ​​the surface defect on the steel sheet S in the captured image. However, this may result in a decrease in detection accuracy.

[0054] Therefore, in the surface defect detection device 10 according to the second embodiment, a pre-processing unit 15 is provided, which performs distortion correction processing on the captured image acquired by the captured image acquisition unit 12 and trimming processing on the distortion-corrected captured image to set the portion of the steel plate S in the captured image that is within the detection target range.

[0055] Here, in the distortion correction process for the captured image, the distortion of the captured image is caused by two factors: the lens distortion of the CCD camera that constitutes the imaging device 4, and the fact that the captured image is captured from an oblique angle of the steel plate S. Therefore, two countermeasures were taken: correction of the lens distortion of the captured image and projective transformation.

[0056] The lens distortion of the captured image is corrected using a known method for correcting lens distortion. For example, in the reference document listed at the following URL, a mathematical model for correcting lens distortion is developed using MATLAB (registered trademark), and the preprocessing unit 15 corrects the lens distortion of the captured image using this mathematical model for correcting lens distortion. https: / / jp.mathworks.com / help / symbolic / developing-an-algorithm-for-undistorting-an-image.html;jsessionid=608c75df332743afcdebb93ea717 Regarding the projective transformation of the captured image, the preprocessing unit 15 first extracts the coordinates of the four corners of the steel sheet S to be corrected from the captured image (see, for example, the captured image shown in FIG. 3) captured by the imaging device 4. Next, projective transformation is performed to convert the captured image into one of the steel sheet S without distortion.

[0057] In addition, in the trimming process, the pre-processing unit 15 edits the captured image that has been subjected to the distortion correction process described above to set the part of the steel plate S in the captured image that is within the detection target range, and cuts out the image of only the part of the steel plate S.

[0058] Furthermore, the detection unit 13 first reads the surface defect detection model stored in the memory unit 11. Next, the detection unit 13 inputs, into the read surface defect detection model, an image obtained by performing distortion correction processing on the captured image acquired by the captured image acquisition unit 12 in the preprocessing unit 15 and trimming processing on the distortion-corrected captured image to set the steel plate portion in the captured image that is within the detection target range. Then, the detection unit 13 detects the type and area of ​​the surface defect on the steel plate S in the captured image input to the detection unit 13 using the surface defect detection model. The type and area of ​​the surface defect are the same as those described in the description of the first embodiment. The detection unit 13 then sends information on the type and area of ​​the surface defect, which are the detection results, to the output unit 14.

[0059] In addition, if there are no surface defects on the steel plate S in the captured image input to the detection unit 13, as shown in Figure 6, information on the type of surface defect and the absence of any surface defect area is sent from the detection unit 13 to the output unit 14. In addition, the output unit 14 outputs information on the type of surface defect and the area of ​​the surface defect on the steel plate S in the captured image detected by the detection unit 13, or information on the type of surface defect and the absence of any area of ​​the surface defect, to the output device 22 constituting the display device.

[0060] The output device 22 then displays information on the type of surface defect and the area of ​​the surface defect present in the steel sheet S in the captured image, or, if there is no surface defect, information on the type of surface defect and the absence of any area of ​​the surface defect (i.e., a blank state is displayed on the display screen of the output device 22). Specifically, if the surface defect present in the steel sheet S in the captured image detected by the detection unit 13 is an oil application unevenness M1, the output device 22 displays the text "oil application unevenness" indicating the oil application unevenness M1 and an area display line R1 surrounding the oil application unevenness M1 in the captured image. Furthermore, if the surface defect present in the steel sheet S in the captured image detected by the detection unit 13 is an oil application unevenness (start of appearance) M1S, the output device 22 displays the text "oil application unevenness (start of appearance)" indicating the oil application unevenness (start of appearance) M1S and an area display line R1S surrounding the oil application unevenness (start of appearance) M1S in the captured image. Furthermore, when the surface defect on the steel sheet S in the captured image detected by the detection unit 13 is an oil unevenness (end), the output device 22 displays the text "oil unevenness (end)" indicating the oil unevenness (end) and an area display line surrounding the oil unevenness (end) in the captured image. Furthermore, when the surface defect on the steel sheet S in the captured image detected by the detection unit 13 is a toasting unevenness M2, the output device 22 displays the text "toasting unevenness" indicating the toasting unevenness M2 and an area display line R2 surrounding the toasting unevenness M2 in the captured image.

[0061] Next, the flow of processing in the imaging device 4 and the surface defect detection device 10 in the second embodiment will be described with reference to FIG.

[0062] First, before detecting the type and area of ​​surface defects on the surface Sa of the steel sheet S, the detection unit 13 of the surface defect detection device 10 reads the surface defect model stored in the memory unit 11 in step S11 (surface defect model reading step).

[0063] As described above, the surface defect model is input to the storage unit 11 from the input device 21. The surface defect detection model is generated by machine learning a plurality of learning data in which a past image of the surface Sa of the steel sheet S is used as input data and the types and areas of surface defects corresponding to this input data are output data. The types and areas of surface defects are as described above. As a machine learning technique, a neural network having an object detection function is used, and the surface defect detection model is a detection model constructed by this neural network. As a result, the types and areas of surface defects on the surface Sa of the steel sheet S can be detected online with high accuracy and in real time using the surface defect detection model constructed by the neural network having an object detection function.

[0064] Next, in step S12, the imaging device 4 captures an image of the surface Sa of the traveling or stopped steel sheet S (imaging step). Here, the imaging device 4 captures 30 frames of images per second, and captures images of the surface Sa of the steel sheet S over the entire length from the leading end to the trailing end of the traveling steel sheet S passing through the electrostatic oil coating device 2.

[0065] Next, in step S13, the captured image acquisition unit 12 of the surface defect detection device 10 acquires captured images of the surface Sa of the steel sheet S, which is traveling or stopped by the imaging device 4 in step S12 (captured image acquisition step). In step S13, the captured image acquisition unit 12 acquires the captured images at regular intervals.

[0066] Next, in step S14, the pre-processing unit 15 of the surface defect detection device 10 performs distortion correction processing on the captured image acquired by the captured image acquiring unit 12 in step S13 (captured image acquiring step), and performs trimming processing on the distortion-corrected captured image to set the part of the steel sheet S in the captured image that is within the detection target range (pre-processing step). The distortion correction processing and trimming processing of the captured image are as described above.

[0067] Next, in step S15, the detection unit 13 of the surface defect detection device 10 inputs the captured image, which has been subjected to distortion correction processing by the pre-processing unit 15 in step S14 and trimming processing to set the steel plate portion in the captured image that is within the detection target range, into the surface defect detection model read by the detection unit 13 in step S11, and detects the type and area of ​​the surface defect in the captured image (detection step). The type and area of ​​the surface defect are as described above.

[0068] Then, the detection unit 13 sends information on the type of surface defect and the area of ​​the surface defect, which are the detection results, to the output unit 14. In addition, if there are no surface defects on the steel plate S in the captured image input to the detection unit 13, as shown in Figure 6, information on the type of surface defect and the absence of any surface defect area is sent from the detection unit 13 to the output unit 14.

[0069] Next, in step S16, the output section 14 of the surface defect detection device 10 outputs information on the type of surface defect and the area of ​​the surface defect on the steel plate S in the captured image detected in step S15 (detection step), or information on the type of surface defect and the absence of any area of ​​the surface defect, to the output device 22 constituting the display device (output step). This completes the processing in the imaging device 4 and the surface defect detection device 10.

[0070] The display process in the output device 22 is as described above. Then, when oil application unevenness M1, oil application unevenness (start of output) M1S, and oil application unevenness (end of output) are detected and the output device 22 displays the text "oil application unevenness" and a region indicator line R1 surrounding the oil application unevenness M1 in the captured image, the text "oil application unevenness (start of output)" and a region indicator line R1S surrounding the oil application unevenness (start of output) M1S in the captured image, or the text "oil application unevenness (end of output)" and a region indicator line surrounding the oil application unevenness (end of output) in the captured image, an alarm is issued for that steel sheet S in the operator's cab on the outlet side of the continuous steel sheet line 1. Then, the worker visually inspects the surface Sa of the steel sheet S again to determine whether oil application unevenness M1, oil application unevenness (start of output) M1S, or oil application unevenness (end of output) is present. If oil application unevenness M1 or the like is detected visually, the shipment is suspended. If these oil coating unevennesses M1 etc. are not detected by visual inspection, the same procedures shall be followed as for the case where oil coating unevenness M1, oil coating unevenness (start) M1S, and oil coating unevenness (end) are not detected.

[0071] On the other hand, if oil application unevenness M1, oil application unevenness (start) M1S, and oil application unevenness (end) are not detected, but toasting unevenness M2 is detected and the output device 22 displays the words "toasting unevenness" and an area indication line R2 surrounding the toasting unevenness M2 in the captured image, the worker rewinds the coil using the recoil line to remove the toasting unevenness M2 and ships the steel sheet S. Also, if oil application unevenness M1, oil application unevenness (start) M1S, oil application unevenness (end) and toasting unevenness M2 are not detected and the output device 22 displays information on the type of surface defect and the absence of any surface defect area, the steel sheet S is shipped as having no surface defects.

[0072] As described above, the surface defect detection device 10 for a continuous steel sheet production line according to the second embodiment is provided with a pre-processing unit 15 that performs distortion correction processing on the captured image acquired by the captured image acquisition unit 12 and trimming processing on the distortion-corrected captured image to set the portion of the steel plate in the captured image that is within the range of the detection target.The detection unit 13 then inputs, into the surface defect detection model, the captured image that has been subjected to distortion correction processing by the pre-processing unit 15 and trimming processing on the distortion-corrected captured image to set the portion of the steel plate in the captured image that is within the range of the detection target, and detects the type and area of ​​the surface defect.

[0073] As a result, distortion correction processing is performed on the captured image captured by the imaging device 4 and acquired by the captured image acquisition unit 12, and the cropped captured image is input into the surface defect detection model, so that the type and area of ​​surface defects on the surface Sa of the steel plate S that is traveling or stopped can be detected online more accurately and instantly.

[0074] That is, by performing distortion correction processing on the captured image acquired by the captured image acquisition unit 12 and inputting the cropped captured image to the surface defect detection model, disturbances in the captured image input to the surface defect model are reduced, and the defect recognition rate is improved. In addition, distortions due to defect size and defect position are normalized, which has the effect of reducing the amount of training data required.

[0075] Then, by performing distortion correction processing in the pre-processing unit 15 and inputting the captured image that has been subjected to trimming processing into the surface defect detection model, it is possible to input an image in which only the detection target is captured into the surface defect detection model, thereby improving the detection accuracy of the type and area of ​​surface defects obtained in the first embodiment (examples described later).

[0076] Furthermore, the surface defect detection method for a continuous steel sheet production line according to the second embodiment includes a pre-processing step (step S14) of performing distortion correction on the captured image acquired in the captured image acquisition step (step S13) and trimming the distortion-corrected captured image to set the steel plate portion within the detection target range. Then, in the detection unit step (step S15), the captured image that has been subjected to distortion correction in the pre-processing step (step S14) and trimming to set the steel plate portion within the detection target range is input to the surface defect detection model, and the type and area of ​​the surface defect are detected.

[0077] As a result, distortion correction processing is performed on the captured image captured by the imaging device 4 and acquired in the captured image acquisition step (step S13), and the cropped captured image is input into the surface defect detection model, so that the type and area of ​​surface defects on the surface Sa of the traveling steel plate S can be detected online more accurately and instantly. Although the embodiment of the present invention has been described above, the present invention is not limited to this and various modifications and improvements can be made.

[0078] For example, the surface defects to be detected are not limited to oil coating unevenness M1 (including oil coating unevenness (beginning) M1S and oil coating unevenness (end)) and baking unevenness M2, but may also include alloy unevenness that hinders shipping, uneven reflected light caused by shape defects, color unevenness due to fluctuations in the thickness of the surface oxide film caused by annealing, scale patterns caused by hot rolling, pickling unevenness caused by pickling performed in intermediate or final processes, polishing / cutting unevenness caused when polishing / cutting the surface, chemical treatment unevenness caused when chemical treatment is performed, paint unevenness caused by painting, baking unevenness caused by the paint film baking process, tracking holes provided at both ends of the width of the steel plate S, etc.

[0079] Furthermore, the surface defects to be detected may be not only both oil application unevenness M1 (including oil application unevenness (start) M1S and oil application unevenness (end)) and baking unevenness M2, but also either one of them. Furthermore, the machine learning method used to generate the surface defect detection model is not limited to a neural network with object detection capabilities, but may also be a method that combines image features with gradient boosting, decision trees, random forests, and multiple regression.

[0080] Furthermore, the steel sheet S having a surface defect to be detected is not limited to a hot-dip galvanized steel sheet manufactured through a hot-dip galvanizing process, but may also be a steel sheet manufactured through other manufacturing processes. Furthermore, in the surface defect detection device 10 and surface detection method for a steel sheet continuous line according to the second embodiment, the pre-processing unit 15 (pre-processing step: step S14) is not limited to performing distortion correction processing on the captured image acquired by the captured image acquisition unit 12 (captured image acquisition step: step S13) and trimming processing to set the steel sheet portion in the captured image that is within the detection target range for the distortion-corrected captured image, but may also perform only distortion correction processing on the captured image. In this case, the detection unit 13 (detection step: step S15) inputs the captured image that has only been subjected to distortion correction processing in the pre-processing unit 15 (pre-processing step: step S14) into a surface defect detection model, and detects the type and area of ​​the surface defect. [Example]

[0081] In order to verify the effects of the present invention, in the continuous steel sheet line 1 shown in Figure 1, the surface Sa of a steel sheet S traveling through an electrostatic oil coating device 2 for a predetermined period of time was imaged online by an imaging device 4, the imaged image was acquired by an image acquisition unit 12 of a surface defect detection device 10, and the imaged image acquired by the image acquisition unit 12 was input into a surface defect detection model in a detection unit 13 to detect the type and area of ​​the surface defect. The detection results are shown in Table 1.

[0082] [Table 1]

[0083] In the detection results shown in Table 1, 41 actual steel sheets S had oil coating unevenness M1, whereas oil coating unevenness M1 was detected on 41 steel sheets S. Furthermore, 70 actual steel sheets S had oil coating unevenness (beginning) M1S or oil coating unevenness (end), whereas oil coating unevenness (beginning) M1S or oil coating unevenness (end) was detected on 70 steel sheets S. Furthermore, 4 actual steel sheets S had baking unevenness M2, whereas baking unevenness M2 was detected on 70 steel sheets S. Furthermore, 17,185 actual steel sheets S had no surface defects, whereas no surface defects were detected on the 17,185 steel sheets S.

[0084] Therefore, the detection results of the present invention and the actual results are consistent. It was confirmed that the type and area of ​​surface defects on the surface Sa of the traveling steel plate S can be detected online with high accuracy and in real time. [Explanation of symbols]

[0085] 1. Continuous steel plate production line 2. Electrostatic oil applicator 3 Transport roll 4. Imaging device 10. Surface defect detection device 11 Storage section 12 Image acquisition unit 13 Detector 14 Output section 15 Pretreatment section 21 Input Devices 22 Output Devices M1 Uneven oil coating (surface defect) M1S Uneven oil application (beginning) M2 Uneven baking (surface defects) S Hot-dip galvanized steel sheet (steel sheet) Sa surface

Claims

1. A surface defect detection device in a continuous steel plate production line that detects the type and area of ​​surface defects on the surface of a traveling or stopped steel plate, an image acquisition unit that acquires an image of the surface of a traveling or stopped steel plate; a detection unit that inputs the captured image acquired by the captured image acquisition unit into a surface defect detection model that is generated by machine learning multiple learning data in which past captured images of the surface of the steel plate are used as input data and the type and area of ​​the surface defect for this input data are used as output data, and detects the type and area of ​​the surface defect by inputting the captured image acquired by the captured image acquisition unit.

2. 2. The surface defect detection device for a continuous steel sheet line according to claim 1, further comprising a pre-processing unit that performs distortion correction processing on the captured image acquired by the captured image acquisition unit, wherein the detection unit inputs the captured image that has undergone distortion correction processing in the pre-processing unit into the surface defect detection model to detect the type and area of ​​the surface defect.

3. The surface defect detection device for a continuous steel plate line as described in claim 2, characterized in that the pre-processing unit performs a trimming process on the distortion-corrected image to set the portion of the steel plate in the image that is within the range of the detection target, and the detection unit inputs the trimmed image into the surface defect detection model to detect the type and area of ​​the surface defect.

4. 2. The surface defect detection device for a continuous steel plate line according to claim 1, wherein the surface defect to be detected is at least one of uneven oil coating and uneven baking occurring on the surface of the steel plate.

5. A surface defect detection method in a continuous steel sheet production line for detecting the type and area of ​​surface defects on the surface of a traveling or stopped steel sheet, comprising: an image acquisition step of acquiring an image of a surface of a traveling or stopped steel plate; a detection step of inputting the captured image acquired in the captured image acquisition step into a surface defect detection model generated by machine learning a plurality of learning data in which a past captured image of the surface of the steel plate is used as input data and the type and area of ​​the surface defect corresponding to this input data is used as output data, and detecting the type and area of ​​the surface defect.

Images