Surface inspection method for metal materials, surface inspection apparatus for metal materials, manufacturing method for metal materials, quality control method for metal materials, and manufacturing equipment for metal materials.
The method uses multiple wavelength bands and machine learning to enhance the detection of surface defects in metallic materials, addressing the limitations of existing technologies by accurately distinguishing between defects and harmless patterns, ensuring high-quality metal products.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- JFE STEEL CORP
- Filing Date
- 2024-06-04
- Publication Date
- 2026-05-26
AI Technical Summary
Existing methods for detecting surface defects in metallic materials, such as steel products, struggle to accurately differentiate between harmless patterns and defects, particularly when defects are non-linear or non-concave, leading to missed detections and inaccuracies.
A surface inspection method that utilizes multiple wavelength bands to generate difference or ratio images, combined with machine learning techniques, to distinguish between patterned defects, red scale peeling defects, and healthy areas, using spectral reflectance characteristics and image processing to enhance detection accuracy.
Accurately detects various surface defects without omission, ensuring high-quality metal materials by identifying patterned and red scale peeling defects, even when they appear similar to harmless patterns or have no surface irregularities.
Smart Images
Figure 0007865354000003 
Figure 0007865354000004 
Figure 0007865354000005
Abstract
Description
Technical Field
[0001] The present invention relates to a surface inspection method for optically detecting defects on the surface of a metallic material, a surface inspection apparatus for a metallic material, a metallic material, a manufacturing method for a metallic material, a quality control method for a metallic material, and manufacturing equipment for a metallic material.
Background Art
[0002] In recent years, in the manufacturing process of metallic materials, especially steel products, from the viewpoint of improving the yield by preventing a large amount of non-conformities, it has been required to detect surface defects of steel materials hot or cold. The steel materials described here mean steel plates such as seamless steel pipes, welded steel pipes, hot-rolled steel sheets, cold-rolled steel sheets, thick plates, and steel products such as shaped steel, and semi-finished products such as slabs generated in the process of manufacturing these steel products. For this reason, as a method for detecting surface defects of steel materials, a method has been proposed in which light is irradiated onto a billet in the manufacturing process of seamless steel pipes, reflected light is received, and the presence or absence of surface defects is determined based on the amount of the reflected light (see Patent Document 1). Further, visible light in a plurality of wavelength regions that do not mutually influence the spontaneous emission radiated from a hot steel material and do not mutually influence each other is irradiated from obliquely symmetric directions with respect to the normal of the hot steel material surface, and images based on the combined reflected light and images based on individual reflected lights are obtained in the normal direction of the hot steel material surface, and a method for detecting surface defects from the combination of these images has also been proposed (see Patent Document 2). Further, discriminable illumination light is irradiated from obliquely symmetric directions with respect to the normal of the steel material surface, and from the difference image of two images obtained by imaging the inspection target portions illuminated from each direction, an arrangement of bright portions and dark portions corresponding to concave shapes is extracted, and a method for detecting surface defects has also been proposed (see Patent Document 3).
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Patent Document 2
Patent Document 3
[0004] According to the method described in Patent Document 1, since the reflectivity of scale and harmless patterns differs from that of the base metal, there is a possibility of misidentifying scale and harmless patterns occurring on sound areas that are not surface defects as surface defects. For this reason, the method described in Patent Document 1 distinguishes between surface defects and scale by utilizing the fact that the shape of surface defects (surface flaws) in billets is linear. However, surface defects in steel materials are not limited to linear ones, but can take on various shapes such as circular ones. For this reason, it is difficult to apply the method described in Patent Document 1 to the detection process of surface defects. On the other hand, in the method described in Patent Document 2, since there is a vast number of types of surface defects, scale, harmless patterns, etc., it is difficult to distinguish between scale and harmless patterns and surface defects simply by combining images. Furthermore, it is practically difficult to construct a detection logic that can handle such a vast number of image combinations. In addition, the method described in Patent Document 3 solves the above problems and can accurately detect concave defects on the surface of steel materials. However, surface defects in steel materials are not limited to concave ones. When rolling is performed with foreign matter pressed into the material, or when scale forms on concave defects, there are also defects that do not appear to have any surface irregularities (hereinafter referred to as non-opening defects). For this reason, it is difficult to detect all surface defects in steel materials without exception using only the method described in Patent Document 3.
[0005] The present invention has been made in view of the above problems, and its object is to provide a surface inspection method and a surface inspection apparatus for metal materials that can detect defects on the surface of metal materials without omission and with high accuracy. Another object of the present invention is to provide a high-quality metal material free from surface defects, a method for manufacturing metal materials, a method for quality control of metal materials, and manufacturing equipment for metal materials. [Means for solving the problem]
[0006] [1] The present invention relates to a surface inspection method for a metal material, which detects defects on the surface of a metal material from a plurality of images obtained by imaging reflected light from the surface of the metal material by irradiated light in two or more different wavelength bands, and includes: a first detection step of generating an image of the difference or ratio of brightness of at least two images obtained from a predetermined area on the surface of the metal material and detecting a first defect present on the surface of the metal material from the generated image; a second detection step of detecting a second defect present on the surface of the metal material by extracting a bright area compared to the surrounding area from an image obtained by synthesizing one or more images obtained from the predetermined area on the surface of the metal material; and a determination step of detecting the defect using the first defect and the second defect.
[0007] [2] In the surface inspection method for a metallic material according to the present invention, the determination step determines that the first defect or the second defect is a harmful defect.
[0008] [3] The surface inspection method for a metallic material according to the present invention is the surface inspection method for a metallic material according to [1] or [2], wherein at least one of the two or more different wavelength bands is a wavelength band of 500 nm or less.
[0009] [4] In the surface inspection method for a metallic material according to the present invention, in any of the surface inspection methods for a metallic material from [1] to [3], the wavelength band of at least one image used in the first detection step and the wavelength band of the image used in the second detection step are the same.
[0010] [5] The surface inspection method for a metallic material according to the present invention is a surface inspection method for a metallic material according to any of [1] to [4], wherein the first defect is a patterned defect and the second defect is a red scale peeling defect.
[0011] [6] The metal surface inspection apparatus according to the present invention is a metal surface inspection apparatus that detects defects on the surface of a metal material from a plurality of images obtained by imaging reflected light from the surface of a metal material by irradiated light in two or more different wavelength bands, and comprises a first detection means that generates an image of the difference or ratio of brightness of at least two images obtained from a predetermined area on the surface of the metal material, and detects a first defect present on the surface of the metal material from the generated image, The system includes a second detection means for detecting a second defect present on the surface of the metal material by extracting a brighter area from an image obtained from a predetermined region on the surface of the metal material compared to the surrounding area, and a determination means for detecting the defect using the first defect and the second defect.
[0012] [7] In the metal surface inspection apparatus according to the present invention, the determination means determines that the first defect or the second defect is a harmful defect.
[0013] [8] The surface properties of the metal material according to the present invention are guaranteed by using any of the surface inspection methods for metal materials described in [1] to [5].
[0014] [9] The method for manufacturing a metal material according to the present invention involves inspecting the surface properties of a metal material using any of the metal material surface inspection methods from [1] to [5], and manufacturing the metal material based on the inspection results.
[0015]
[10] The quality control method for a metal material according to the present invention involves inspecting the surface properties of a metal material using any of the surface inspection methods for metal materials from [1] to [5], and controlling the quality of the metal material based on the inspection results.
[0016]
[11] The manufacturing equipment for metal materials according to the present invention includes a surface inspection device for metal materials as described in [6] or [7]. [Effects of the Invention]
[0017] According to the surface inspection method and surface inspection apparatus for a metal material according to the present invention, defects on the surface of the metal material can be accurately detected without leakage. Further, according to the metal material according to the present invention, it is possible to provide a high-quality metal material, a method for manufacturing a metal material, a quality control method for a metal material, and a manufacturing facility for a metal material, which have no defects on the surface.
Brief Description of the Drawings
[0018] [Figure 1] FIG. 1 is a diagram showing an example of a pattern-like defect and a harmless pattern. [Figure 2] FIG. 2 is a schematic diagram showing the configuration of the apparatus used in the test. [Figure 3] FIG. 3 is a diagram showing the result of comparing the relationship between the signal intensity and the wavelength of the pattern-like defect portion and the healthy portion. [Figure 4] FIG. 4 is a diagram showing an example of a difference image. [Figure 5] FIG. 5 is a diagram showing the result of comparing the relationship between the incident angle of illumination light and the difference in signal intensity between the pattern-like defect portion and the healthy portion and the wavelength. [Figure 6] FIG. 6 is a diagram showing a difference image of a non-opening defect and a non-opening defect. [Figure 7] FIG. 7 is a diagram showing a non-opening defect where the red scale has peeled off. [Figure 8] FIG. 8 is a diagram showing a difference image of a red scale peeling defect and a red scale peeling defect. [Figure 9] FIG. 9 is a diagram showing an image obtained by imaging a red scale peeling defect in the R channel, G channel, and B channel of a color camera. [Figure 10] FIG. 10 is a diagram showing a non-opening defect. [Figure 11] FIG. 11 is a schematic diagram showing the configuration of a surface inspection apparatus for a metal material according to the first embodiment of the present invention. [Figure 12] FIG. 12 is a schematic diagram showing the configuration of a surface inspection apparatus for a metal material according to the second embodiment of the present invention. [Figure 13] FIG. 13 is a flowchart showing the flow of a surface inspection process according to an embodiment of the present invention. [Figure 14] Figure 14 is a flowchart showing the flow of the first detection step as shown in Figure 13. [Figure 15] Figure 15 is a flowchart showing the flow of the second detection step as shown in Figure 13. [Modes for carrying out the invention]
[0019] In thick steel plates, a type of steel product, the surface may be covered with an oxide film called black scale, and patterns that do not affect the quality of the steel product, referred to as harmless patterns, may occur. In addition, pattern-like defects such as hairline defects may occur on the surface of thick steel plates, but as shown in Figure 1, it is difficult to distinguish between pattern-like defects and harmless patterns based on the difference in brightness of the captured image. Therefore, the inventors of this invention focused on the fact that minute red scale (Fe2O3) is easily generated on pattern-like defects, and that this red scale makes the pattern-like defects appear reddish, and conducted a test to detect pattern-like defects using spectral reflectance characteristics. The configuration of the apparatus used in the test is shown in Figure 2. As shown in Figure 2, in this test, a thick steel plate sample SA having pattern-like defects was placed on a linear stage 1, and illumination light L with a broadband wavelength was irradiated onto the surface of the thick steel plate sample SA from a xenon light source 2, and spectral images of each wavelength were captured using a spectroscopic camera 3 with a one-dimensional field of view.
[0020] Figure 3 shows the results of comparing the relationship between signal intensity and wavelength for pattern defects and healthy areas using spectral images. As shown in Figure 3, the signal intensity characteristics on the short-wavelength and long-wavelength sides are relatively different for pattern defects and healthy areas. This suggests that pattern defects, which are difficult to detect by the signal of healthy areas using only a single wavelength, may be detectable by comparing signal intensities between different wavelengths. Here, we show an example of detecting pattern defects in a thick steel plate using the difference between spectral images of two different wavelengths. Pattern defects in thick steel plates have a form that is pressed onto the surface of the thick steel plate during the manufacturing process, and there are no irregularities when viewed from the surface of the thick steel plate. For these pattern defects, after performing brightness correction so that the average brightness value is constant for spectral images in the 415 nm wavelength band (415 nm image) and spectral images in the 750 nm wavelength band (750 nm image), Figure 4 shows the image obtained by taking the difference in brightness values of the two spectral images. As shown in Figure 4, the signals of healthy areas such as black scale are canceled out by the difference, and only the signal of the pattern defects is emphasized.
[0021] This confirmed that pattern defects can be accurately detected by utilizing the characteristics of the spectral reflectance spectra of pattern defects and sound areas to generate a difference image between spectral images captured with long-wavelength light and spectral images captured with short-wavelength light. Depending on the manufacturing conditions of the thick steel plate, surface scales with surface reflectance spectra similar to pattern defects may be generated in sound areas. However, even in such cases, pattern defects can be accurately detected by discriminating the pattern defects extracted from the difference image using a classifier created by machine learning techniques, thereby suppressing false positives.
[0022] From the above, it was confirmed that pattern defects with different spectral reflectance characteristics from healthy areas can be accurately detected by comparing luminance information between multiple different wavelengths. Figure 5 shows the results of comparing the relationship between the incident angle of illumination light L (angle with respect to the surface normal vector of the thick steel plate), the signal intensity difference between pattern defects and healthy areas, and wavelength. In the legend of Figure 5, 20°, 30°, etc., are indicated as the incident angle. As shown in Figure 5, the signal intensity difference between pattern defects and healthy areas increases as the incident angle of illumination light L increases. Therefore, to efficiently detect the spectral reflectance characteristics of the target, i.e., the difference in color, it is best to increase the incident angle of illumination light L.
[0023] Furthermore, the two spectral images used to generate the difference image described above may be captured using two of the three channels of the color camera. This allows for the simultaneous detection of pattern defects and red scale peeling defects using a single color camera. Specifically, a difference image is generated using the image captured in channel B or channel G and the image captured in channel R to detect pattern defects. On the other hand, red scale peeling defects are detected using the image captured in channel B or channel G. Here, for the detection of red scale peeling defects, i.e., the second detection step described later, it is preferable not to use wavelengths of 600 nm or more for the captured image, and it is even more preferable to use wavelengths of 550 nm or less. It is also more preferable to use a green (wavelength 490-550 nm) or blue (wavelength 430-490 nm) captured image, and it is most preferable to use a blue (wavelength 430-490 nm) captured image.
[0024] Although the above description concerns pattern defects in thick steel plates, the present invention can be applied to the detection of other defects on the surface of metallic materials that differ from those of sound areas in spectral reflectance characteristics. Furthermore, while the above description focuses on thick steel plates, it goes without saying that the invention is also applicable to other steel materials that may develop red scale, such as slabs, hot-rolled steel plates, welded steel pipes, electric resistance welded pipes, seamless steel pipes, structural steel, steel bars, rods, and wires. Moreover, the present invention can be similarly applied to any metallic product that develops red rust on its surface, similar to steel products.
[0025] Incidentally, among non-opening defects that appear to have no surface irregularities due to the adhesion of foreign matter or the formation of scale, as shown in Figure 6(a), there are some in which red scale attached to the surface peeled off along with the black scale during the transport of the steel material. In Figure 6(a), the symbols BM represent the base metal, MS represents the black scale, RS represents the red scale, and DI represents the interface of the non-opening defect, and Figure 6(b) shows an example of a difference image of the non-opening defect shown in Figure 6(a). Among such defects, those in which the interface of the non-opening defect is exposed and forms a recessed shape, as shown in Figure 7, can be detected using methods such as those described in Patent Document 3. However, it is difficult to detect defects in which only the surface red scale and black scale peel off, and foreign matter EP, etc., remains in the recessed area, resulting in almost no irregularities and the non-opening defect interface DI not being exposed, as shown in Figure 8(a). This is because such defects are often not reddish compared to the surrounding area and do not appear bright in the difference image. In the following, we will refer to such non-opening defects, where the red scale has peeled off and there is almost no surface irregularity, as red scale peeling defects for convenience. Figure 8(b) shows an example of a difference image of a red scale peeling defect.
[0026] Here, images of such red scale peeling defects captured in the R (red), G (green), and B (blue) channels of a color (RGB) camera are shown in Figures 9(a) to 9(c), respectively. Figure 9(a) is the image captured in the R (red) channel, Figure 9(b) in the G (green) channel, and Figure 9(c) in the B (blue) channel. As shown in Figures 9(a) to 9(c), the reflectivity of the red scale peeling defect area is high, so the red scale peeling defect area is captured as bright in the image in all three channels (R, G, and B), and this is particularly noticeable in the images captured in the G and B channels. Therefore, instead of using difference images, red scale peeling defects are detected by extracting areas that appear brighter compared to the surrounding area from the image captured by the color camera.
[0027] The reason for the low contrast of the image captured in the R channel shown in Figure 9(a) is that some of the thick steel plates used as the base material have a surface that appears reddish due to the overall presence of red scale. These surfaces appear brighter overall in the image captured in the R channel, so even if red scale peeling defects appear bright, they blend in with the background. For this reason, it is preferable to detect red scale peeling defects by extracting areas that appear brighter compared to the surrounding area from the images captured in the B channel in Figure 9(b) and the G channel in Figure 9(c).
[0028] Furthermore, while images captured in the B and G channels may be used directly, images obtained by combining each pixel of images captured in the B and G channels with specific weights may also be used. Here, images obtained by combining images captured in the B, G, and R channels with specific weights may be used, but for the reasons mentioned above, it is preferable to make the weight of the image captured in the R channel small. In addition, even on the reddish surface described above, as shown in Figure 10, for non-aperture defects where the red scale RS has not peeled off, a red scale RS2 is generated on top of the red scale RS, so it can be detected with high accuracy using the difference image.
[0029] Furthermore, the base metal portion can also be exposed when the surface of the steel material rubs against structures such as conveyor rolls, and these are called scratches. Such scratches occur on the hard surface of the steel material that has cooled and gained sufficient strength, and are often shallow. Because the surface is rubbed, they have the characteristic of exposing the base metal portion, similar to red scale peeling defects, and have a unique shape that differs from red scale peeling defects. Even when such scratches are present, it is possible to extract only the harmful defects using a classifier created with machine learning techniques.
[0030] Below, we will describe one embodiment of a metal surface inspection device, conceived from the aforementioned technological concept.
[0031] [Surface inspection equipment] [First Embodiment] Figure 11 is a schematic diagram showing the configuration of a surface inspection apparatus for metal materials according to the first embodiment of the present invention. As shown in Figure 11, the surface inspection apparatus 10 for metal materials according to the first embodiment of the present invention is a device for detecting surface defects in a plate-shaped steel material S that is conveyed in the direction of the arrow shown in the figure. The surface inspection apparatus 10 for metal materials according to the first embodiment of the present invention mainly comprises a light source 11, an encoder and pulse generator 12, an area sensor 13 capable of capturing spectral images in multiple wavelength bands, an image processing device 14, and a monitor 15.
[0032] The light source 11 irradiates the inspection target area on the surface of the steel material S with illumination light L according to a trigger signal output by the pulse generator each time a pulse signal is transmitted from the encoder a certain number of times. The light source 11 is preferably positioned so that the direction of illumination light L is tilted within the range of 60° to less than 90° with respect to the normal direction of the surface of the steel material S. This allows for accurate detection of surface defects from the difference image. In this embodiment, one light source 11 is provided, but multiple light sources 11 may be provided. In this embodiment, a xenon light source is used as the light source 11, but any light source with broadband characteristics such as a metal halide light source, halogen light source, mercury lamp, or incandescent lamp, or a combination of light sources with specific narrowband characteristics such as an LED or laser, may be used as long as they contain components in multiple different wavelength bands. In addition, if the transport speed of the steel material S is fast, or if there is a large change in the position of the steel material S from a predetermined transport position (pass line), a flash light source or pulse lighting may be used to prevent image blur.
[0033] The area sensor 13 captures spectral images of multiple different wavelength bands at approximately the same location on the steel material S. The multiple spectral images captured are preferably coaxial, but alignment may be achieved through image processing. Examples of area sensors 13 include a Bayer type, where filters transmitting different wavelength bands are attached to each element in a nested fashion to generate multiple images later, and a prism type, where a prism and multiple elements are used and adjusted to be coaxial. While an RGB color camera is inexpensive and preferable, a multiband area sensor with 2 or more channels may also be used. Furthermore, depending on the spectral reflectance characteristics of the healthy and defective areas to be distinguished, a wavelength-selective filter may be placed in the optical path, such as in front of the area sensor 13 or light source 11, to enhance color clarity and improve detection performance. Additionally, if the difference in spectral reflectance characteristics between the healthy and defective areas is narrow-band, the wavelength band received may be narrowed using a wavelength-selective filter, if sufficient light is available. Furthermore, when detecting pattern defects, it is preferable that at least one of the multiple different wavelength bands includes a narrow wavelength band of 650 nm or more, and at least one includes a narrow wavelength band of 500 nm or less.
[0034] The area sensor 13 captures a spectral image in synchronization with the light source 11 according to a trigger signal output from the pulse generator. Except for spectral images in which the ends of the steel material S are captured, the brightness values of each channel in the spectral image are not saturated. In this embodiment, a two-channel area sensor that captures light in the long-wavelength and short-wavelength bands is used, but a device configuration in which three or more wavelength bands are captured by three or more channel area sensors may also be used. Furthermore, the area sensor 13 should receive reflected light such that the light reception angle with respect to the normal direction of the surface of the steel material S is within the range of 0° to less than 20°.
[0035] The image processing device 14 detects surface defects in the area to be inspected by performing difference processing and extraction processing, described later, between the spectral images input from each channel of the area sensor 13. The image processing device 14 then outputs the spectral image input from the area sensor 13, the spectral image after difference processing, and information regarding the surface defect detection results to the monitor 15. In this specification, the image processing device 14 also corresponds to a determination means.
[0036] [Second Embodiment] Figure 12 is a schematic diagram showing the configuration of a metal surface inspection device according to a second embodiment of the present invention. As shown in Figure 12, the metal surface inspection device 20 according to the second embodiment of the present invention detects surface defects in a plate-shaped steel material S that is transported in the direction of the arrow shown. The difference from the metal surface inspection device 10 according to the first embodiment is that the light source 11 is a line light source 21 and the area sensor 13 is a line sensor 22. Using the line sensor 22 limits the field of view to only one line, which has the advantage of more stable optical conditions compared to the first embodiment, but it also has the disadvantage of being susceptible to positional changes of the steel material S from the transport position (pass line) during transport.
[0037] The above-mentioned surface inspection devices 10 and 20 for metal materials can also be installed as part of the manufacturing equipment for metal materials.
[0038] The metal surface inspection devices 10 and 20 having such a configuration accurately and comprehensively detect surface defects of metal materials by performing the surface inspection process described below. The operation of the metal surface inspection devices 10 and 20 when performing the surface inspection process according to the first to third embodiments of the present invention will be described below.
[0039] [Surface inspection process] Figure 13 is a flowchart showing the flow of a surface inspection process according to an embodiment of the present invention. In this embodiment, patterned defects, red scale peeling defects, and healthy areas with harmless patterns are distinguished from each other in the area to be inspected. The surface inspection process shown in Figure 13 starts when an execution command for the surface inspection process is input to the image processing device 14, and the surface inspection process proceeds to step S21.
[0040] Note that the irradiation step and imaging step may be performed before the surface inspection process shown in Figure 13. In this case, first, in the irradiation step, light is irradiated onto the surface of the metal material. Then, in the imaging step, multiple images are acquired by imaging the reflected light from the surface of the metal material due to the light irradiated in the above-mentioned irradiation step in two or more different wavelength bands. In Figures 13 to 15, "raw image" refers to multiple images acquired by imaging in two or more different wavelength bands. The first detection step and the second detection step, which will be described later, are performed on multiple images acquired from a predetermined area on the surface of the metal material. The "raw image" can be acquired, for example, by the imaging step described above, or it can be acquired from images obtained on another occasion. Here, in the irradiation step, for example, the light source 11, line light source 21, encoder and pulse generator 12 described in the above-mentioned surface inspection apparatus can be used as the irradiation means. Similarly, in the imaging step, for example, the area sensor 13 or line sensor 22 described in the above-mentioned surface inspection apparatus can be used as the imaging means.
[0041] The surface inspection process of this embodiment combines a first detection step (processing steps S21 to S24) and a second detection step (processing steps S21, S25 to S27). In the first detection step, when spectral images of multiple different wavelength bands are used, images captured with the R channel and images captured with the G channel or B channel of a color camera are used to distinguish between patterned defects and sound areas. On the other hand, in the second detection step, when spectral images of multiple different wavelength bands are used, images captured with the B channel or G channel of a color camera are used to distinguish between red scale peeling defects and sound areas.
[0042] Finally, the image processing device 14 integrates the results of the first and second detection steps with the image information of the area to be inspected to generate information about harmful defects on the surface of the steel material (step S28). This is the final determination step in the present invention. All defects detected in each detection step can be considered as independent defects on the steel plate. Therefore, by simply adding up the information of pattern defects and red scale peeling defects, all defects occurring on the steel plate are comprehensively detected.
[0043] Specifically, if a defect (referred to as the first defect) is determined in the first detection step and the area is determined to be healthy in the second detection step, the image processing device 14 generates information on the area to be inspected as a patterned defect. On the other hand, if the area is determined to be healthy in the first detection step and the area is determined to be defective (referred to as the second defect) in the second detection step, the image processing device 14 generates information on the area to be inspected as a red scale peeling defect. Here, the second defect refers to different information regarding the defective area compared to the first defect. Furthermore, if the area is determined to be healthy in both the first and second detection steps, the image processing device 14 generates information on the area to be inspected as healthy. The image processing device 14 then generates information on both the patterned defect and the red scale peeling defect as harmful defects.
[0044] If a patterned defect and a red scale peeling defect occur at the same location during the judgment step S28, it is highly likely that a portion of a single patterned defect has peeled off and been detected as a red scale peeling defect. In this case, it is preferable to treat the two defects as a single defect. Therefore, the location of the patterned defect and the location of the red scale peeling defect can be compared, and if the distance is smaller than a predetermined distance, they may be considered the same defect. Any method for calculating the distance described here can be used as an index that objectively quantifies the positional relationship of the defects, such as the distance between the centroids of the defects, the shortest distance of each defect region, the distance between the centroids of the smallest circumscribing rectangle, or the shortest distance of the smallest circumscribing rectangle. Patterned defects and red scale peeling defects that are considered the same defect are highly likely to be actual defects, and may be treated separately from single patterned defects and red scale peeling defects during mapping guidance and pass / fail judgment processing.
[0045] Information regarding harmful defects on the surface of the steel material is displayed on monitor 15, for example, as an image showing the distribution of harmful defects. Furthermore, information regarding harmful defects on the surface of the steel material is used for guidance to workers, collection on a server to determine whether the steel material S can be shipped and / or whether maintenance is required, and for processes such as automatically transporting the steel material to maintenance or other treatment processes. The image showing the distribution of harmful defects displayed on monitor 15 can be visualized by independently mapping the location of harmful defects on the steel plate obtained in each detection step onto a map. In this case, if it is desired to distinguish and visualize pattern defects and red scale peeling defects, different markers may be used on the map. Also, when determining whether a product is acceptable or not using the information on the harmful defects that have occurred, the information on harmful defects that occurred in each detection step may be totaled and counted for the determination. Specifically, if the number of acceptable defects is 5 or less, and 3 pattern defects and 4 red scale peeling defects are detected, the total number of detected defects is 7, and the product may be judged as unacceptable.
[0046] As described above, in the surface inspection process according to the embodiment of the present invention, the first detection step and the second detection step are combined to distinguish between patterned defects, red scale peeling defects, and healthy areas with harmless patterns, thereby further improving the defect detection accuracy. Alternatively, the features obtained by the first detection step and the second detection step may be combined in the same surface defect candidate area and machine learning may be used to determine whether it is a surface defect or a healthy area. Furthermore, all images of surface defect candidate areas extracted by either the first or second detection step may be labeled as either surface defects or healthy areas, and a classification machine may be created using a convolutional neural network.
[0047] [First detection step] Figure 14 is a flowchart showing the flow of the first detection step shown in Figure 13. In this embodiment, pattern-like defects and healthy areas with harmless patterns are distinguished in the area to be inspected. Pattern-like defects are defects that appear to have no surface irregularities, such as those caused by the adhesion of foreign matter or the formation of scale on concave defects. Healthy areas with harmless patterns refer to areas with surface coatings or surface properties that have different optical properties from the base metal portion, such as black scale, which is several to tens of micrometers thick, and are areas that become noise sources in the surface inspection process. The first detection step shown in Figure 14 starts when the execution command for the first detection step is input to the image processing device 14, and the surface inspection process, which is the first detection step, proceeds to the process of step S1.
[0048] In step S1, if the positions of the spectral images of multiple different wavelength bands are shifted at the pixel level, the image processing device 14 performs alignment processing. When spectral images cannot be captured coaxially, alignment is required between spectral images of multiple different wavelength bands. The alignment method varies depending on the form of the positional shift of the spectral images of each wavelength band, and processing such as translation, linear transformation, or one-to-one correspondence of pixels may be performed as needed. However, it is not necessary for them to be in exactly the same position. Positional shifts within the range in which the processing in steps S14, S24, S27, and S28 can be performed are included in the "same position" of the present invention as an acceptable range. Similarly, positional shifts with respect to the "predetermined region" to be processed are also acceptable within the range in which the processing in steps S14, S24, S27, and S28 can be performed. With this, the processing in step S1 is completed, and the surface inspection process proceeds to step S2.
[0049] In step S2, the image processing device 14 performs first preprocessing on multiple spectral images of different wavelength bands, such as correcting the average brightness to be constant, correcting brightness unevenness, and normalizing signal intensity. With this, the processing in step S2 is completed, and the surface inspection process proceeds to step S3. These processes in steps S1 and S2 correspond to the correction process S21 shown in Figure 13.
[0050] In step S3, the image processing device 14 compares multiple spectral images of different wavelength bands and generates a composite image that emphasizes only pattern defects by utilizing the differences in spectral reflectance characteristics. Specifically, the image processing device 14 selects two spectral images with large differences in spectral reflectance characteristics and generates a composite image by calculating the difference in brightness and ratio of the two spectral images. The case in which a difference image of two spectral images is generated as a composite image will be described below. In this case, the image processing device 14 calculates the brightness value Id1(x,y) of the difference image Id1 by subtracting the brightness value Ir(x,y) of the second spectral image Ir (for example, the brightness value of the second channel having sensitivity characteristics corresponding to the long wavelength band) from the brightness value Ib(x,y) of the first spectral image Ib (for example, the brightness value of the first channel having sensitivity characteristics corresponding to the short wavelength band) Ib(x,y), as shown in the following formula (1).
[0051]
number
[0052] The spectral images Ib and Ir are images with X × Y pixels. In the orthogonal 2-axis xy coordinate system set for each of the spectral images Ib and Ir, the x-coordinate is 1 ≤ x ≤ X and the y-coordinate is 1 ≤ y ≤ Y. In this example, a difference image was generated as the composite image, but an image of the ratio of luminance values using the following formula (2) may also be generated as the composite image. Alternatively, thresholding may be performed separately on the luminance values of the two spectral images, followed by an AND operation to generate the composite image.
[0053]
number
[0054] Furthermore, although this process can be performed with only two spectral images, the following methods (a) to (c) are particularly effective for three or more spectral images. With this, the process in step S3 is completed, and the surface inspection process proceeds to step S4. This process in step S3 corresponds to the synthesis process S22 shown in Figure 13.
[0055] (a) Spatial transformation method This is a 3-channel method, particularly effective for color images, that generates images that extract the effects of spectral reflectance characteristics (hue) by converting the color space. The converted color space can be HSV or HLS, and in each case, spectral reflectance information appears in the hue information. Alternatively, the image may be converted to XYZ, L*u*v*, or L*a*b* space, the luminance component may be normalized, and then thresholding may be applied to the luminance values of each color image.
[0056] (b) Statistical multivariate analysis This method uses the luminance values of each wavelength band of each pixel as a feature vector and extracts areas with different spectral reflectance characteristics through statistical multivariate analysis. As an example, the inspection area is divided into sufficiently large sections, and in each section, principal component analysis (PCA) is performed using the luminance values of each wavelength band of all pixels as a feature vector. The image is then reconstructed using the Mahalanobis distance of each pixel as a representative value for that pixel. Since patterned defects have different colors, the Mahalanobis distance is expected to be larger compared to healthy areas. Although principal component analysis has been described here, similar effects can be obtained by similarly calculating the degree of deviation from the model using a mixture Gaussian model, independent component analysis, or regression model.
[0057] (c) Machine learning methods This method is identical to statistical multivariate analysis up to the point of using the brightness values of each wavelength band for each pixel as feature vectors. However, it pre-assigns each feature vector to either a pattern defect or a healthy area, and then uses general supervised learning to create a classifier that determines whether each pixel is a pattern defect or a healthy area. For clustering the features, k-means, kernel methods, decision trees, Gaussian mixture models, regression models, etc., may be used. When using this method, thresholding, which will be discussed later, is unnecessary, and pixels that are candidates for defects can be directly extracted.
[0058] In step S4, the image processing device 14 generates an image with enhanced surface defects by performing a second preprocessing step on the composite image using a frequency filter or the like. This completes step S4, and the surface inspection process proceeds to step S5.
[0059] In step S5, the image processing device 14 generates a binarized image by performing thresholding on the brightness values of the image obtained in step S4. With this, step S5 is completed, and the surface inspection process proceeds to step S6. These steps S4 and S5 correspond to the thresholding process S23 shown in Figure 13.
[0060] In step S6, the image processing device 14 performs connected and isolated point removal on the binarized image by processing such as dilation and scalding as necessary, and then performs a labeling process in which adjacent pixels are treated as blobs (clusters) and labeled. The image processing device 14 then identifies the blobs extracted by the labeling process as candidate surface defect areas. With this, the process in step S6 is completed, and the surface inspection process proceeds to step S7.
[0061] In step S7, the image processing device 14 determines the type and grade (including harmless) of each surface defect candidate obtained in step S6. The determination method may be manually defined by determination rules, or it may be automatically generated using general machine learning methods that utilize features such as regression models (linear regression, logistic regression, multiple regression, support vector machines, nonlinear kernels, etc.), decision tree models, random forests, Bayesian estimation models, Gaussian mixture models, or boosted versions of these models. In this case, by adding representative values (mean and maximum) of the brightness values of multiple spectral images and values that allow comparison between these representative values (sum, difference, ratio), it becomes possible to make a determination that takes spectral reflectance characteristics into account. Furthermore, if a sufficient number of data points (N) are obtained, a convolutional neural network may be used. In addition, multiple machine learning methods may be combined. The determination result is used in step S28 to be integrated with the determination result of the second detection step, which will be described later, as the first defect. With this, the processing of step S7 is completed, and the series of surface inspection processes is finished. These steps S6 and S7 correspond to the defect determination S24 shown in Figure 13.
[0062] As is clear from the above explanation, in the first detection step, light is irradiated onto the surface of the steel material S, and the reflected light from the surface of the steel material S due to the irradiated light is imaged in two or more different wavelength bands. Pattern-like defects present on the surface of the steel material S are detected from the relative signal intensity information between multiple images obtained from the same position on the surface of the steel material S. This makes it possible to detect surface defects of the steel material S with complete accuracy and without omission.
[0063] [Second detection step] Figure 15 is a flowchart showing the flow of the second detection step shown in Figure 13. In the second detection step, red scale peeling defects and healthy areas with harmless patterns are distinguished in the area to be inspected. The red scale peeling defects referred to here are non-opening defects where the red scale has peeled off and there are no irregularities. The second detection step shown in Figure 15 starts when the execution command for the second detection step is input to the image processing device 14, and the surface inspection process, which is the second detection step, proceeds to the process of step S21.
[0064] In the second detection step, the image processing device 14 first performs correction processing on spectral images of multiple different wavelength bands (step S21). The specific correction processing method is the same as that of steps S1 and S2 in the first detection step. The processing in step S21 may be omitted, and the corrected image obtained after the processing in step S2 in the first detection step is used as the corrected image after the processing in step S21. Next, the image processing device 14 extracts an image for extracting the base metal portion from the corrected image obtained in the processing in step S21 (step S25). Here, the image for extracting the base metal portion is a monochrome image with a small amount of R channel component from the color camera. For this purpose, images captured by the B channel or G channel may be used as they are, or these images may be combined with specific weights. In addition, depending on the steel material, image components captured by the R channel may be mixed in. Most preferably, the image captured by the B channel of the color camera is used as is. Furthermore, in order to reduce the processing load of steps S1, S2, S21, and S25, it is preferable that the wavelength band of at least one image used in the first detection step and the wavelength band of the image used in the second detection step are the same.
[0065] Next, the image processing device 14 extracts bright areas from the image compared to the surrounding area using a threshold, labels them, and extracts candidate surface defect areas, similar to the first detection step (step S26). Next, the image processing device 14 determines whether each candidate surface defect area obtained in step S26 is a red scale peeling defect or not (step S27). Here, the determination method may be a feature analysis or a convolutional neural network. The determination result is used as a second defect and is integrated with the determination result of the first detection step described above, and is used in the processing of step S28. With this, the processing of step S27 is completed, and the series of surface inspection processes is finished.
[0066] As is clear from the above explanation, in the second detection step, light is irradiated onto the surface of the steel material S, and the reflected light from the surface of the steel material S due to the irradiated light is imaged in two or more different wavelength bands. From the image obtained by combining one or more images obtained from the same position on the surface of the steel material S, red scale peeling defects are detected from information extracted from areas that are brighter than the surrounding area. This makes it possible to detect surface defects of the steel material S with complete accuracy and without omission.
[0067] Furthermore, the surface inspection process, which is one embodiment of the present invention, can also be applied to a quality control method for metal materials. In this case, the surface inspection process, which is one embodiment of the present invention, is used to investigate the presence or absence of surface defects in metal materials such as steel S, and quality control is performed to confirm whether the surface defect occurrence status (such as the occurrence rate and the size of the defects) is below a predetermined tolerance standard. This makes it possible to provide metal materials with controlled or guaranteed surface properties. For example, by recording whether the surface defect occurrence status is below a predetermined tolerance standard in operational records, inspection reports, delivery slips, inspection certificates, etc., manufacturers can control quality and guarantee quality to customers. Furthermore, the surface inspection process, which is one embodiment of the present invention, can also be applied to a method for manufacturing metal materials. In this case, the surface properties of the metal material are inspected using the surface inspection process, which is one embodiment of the present invention, and the metal material is manufactured based on the inspection results.
[0068] Although embodiments applying the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that constitute part of the disclosure of the present invention in this embodiment. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on this embodiment are included in the scope of the present invention. [Explanation of Symbols]
[0069] 1 Linear Stage 2 Xenon light source 3. Spectroscopic camera 10,20 Surface inspection device for metal materials 11 Light source 12 Encoders and pulse generators 13 Area Sensor 14 Image Processing Device 15 monitors 21 Line Light Sources 22-line sensor BM Chitetsu D. Red scale peeling defect DI non-opening defect interface EP foreign matter L illumination light MS Black Skin Scale RS, RS2 Red Scale S steel material SA Thick Steel Plate Sample
Claims
1. A surface inspection method for a metal material, which detects defects on the surface of a metal material from multiple images obtained by imaging the reflected light from the surface of the metal material by irradiated light in two or more different wavelength bands, A first detection step involves generating an image of the difference or ratio of brightness between at least two images with different wavelength bands obtained from a predetermined region on the surface of the metal material, and detecting a first defect present on the surface of the metal material from the generated image. A second detection step involves detecting a second defect present on the surface of the metal material by extracting a bright area compared to the surrounding area from an image obtained by combining one or more images obtained by imaging a predetermined area on the surface of the metal material with at least one of the red, green, and blue channels of a color camera, and A determination step for detecting the defect using the first defect and the second defect, Includes, The first defect is a patterned defect, and the second defect is a red scale peeling defect. Surface inspection methods for metallic materials.
2. The surface inspection method for a metallic material according to claim 1, wherein the determination step determines that the first defect or the second defect is a harmful defect.
3. The surface inspection method for a metallic material according to claim 1, wherein at least one of the two or more different wavelength bands is a wavelength band of 500 nm or less.
4. The surface inspection method for a metallic material according to claim 1, wherein the wavelength band of at least one image used in the first detection step is the same as the wavelength band of the image used in the second detection step.
5. A surface inspection apparatus for metal materials that detects defects on the surface of a metal material from multiple images obtained by imaging the reflected light from the surface of the metal material by irradiated light in two or more different wavelength bands, A first detection means generates an image of the difference or ratio of brightness between at least two images with different wavelength bands obtained from a predetermined region on the surface of the metal material, and detects a first defect present on the surface of the metal material from the generated image. A second detection means for detecting a second defect present on the surface of the metal material by extracting a bright area compared to the surrounding area from an image obtained by combining one or more images obtained by imaging a predetermined area on the surface of the metal material with at least one of the red, green, and blue channels of a color camera, A determination means for detecting the defect using the first defect and the second defect, Equipped with, The first defect is a patterned defect, and the second defect is a red scale peeling defect. Surface inspection device for metal materials.
6. The surface inspection apparatus for a metal material according to claim 5, wherein the determination means determines that the first defect or the second defect is a harmful defect.
7. A method for manufacturing a metal material, comprising inspecting the surface properties of a metal material using the surface inspection method for a metal material described in any one of claims 1 to 4, and manufacturing the metal material based on the inspection results.
8. A method for controlling the quality of a metal material, comprising inspecting the surface properties of a metal material using the surface inspection method for a metal material described in any one of claims 1 to 4, and controlling the quality of the metal material based on the inspection results.
9. A metal material manufacturing apparatus comprising a metal material surface inspection device according to claim 5 or 6.