Non-Lambertian Surface Inspection System for Line Scan
The non-Lambertian surface inspection system addresses glare issues on curved surfaces by using adjustable illumination and composite imaging to enhance defect detection on curved surfaces.
Patent Information
- Application Number
- JP2021156561
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-09-28
- Filing Date
- 2021-09-27
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2041-09-27
AI Technical Summary
Conventional surface inspection systems struggle with accurately inspecting curved surfaces due to glare and inability to adjust illumination direction and intensity, leading to limited defect detection capabilities.
A non-Lambertian surface inspection system using a line scan camera with multiple lighting modules and a controller unit that adjusts illumination angles and intensity, generating composite images to enhance defect detection on curved surfaces.
The system effectively reduces glare, enhances defect detection by producing differential and albedo images, and improves inspection efficiency on curved surfaces, particularly identifying scratches and nicks.
Smart Images

Figure 0007578241000012 
Figure 0007578241000013 
Figure 0007578241000014
Abstract
Description
[Technical field]
[0001] The present invention relates to the technical field of inspecting the surface of a product using a line scan camera. [Background technology]
[0002] The industries of portable IT devices such as smartphones, tablet PCs, and notebook computers, as well as electric and hybrid vehicles, are developing rapidly. These industries are driving various additional industries. For example, as the number of types of smartphones increases and the usage cycle of smartphones shortens, the industry for systems that can quickly and accurately inspect a wide variety of smartphones is growing.
[0003] As a result, the market for inspection systems for appearance defects in smartphone cover glass is gradually growing.
[0004] To date, defects in smartphone cover glass have been inspected by inspecting the outline and surface of the subject using an area scan camera and a line scan camera with a simple lighting method.
[0005] Although such contour and surface inspection systems have a simple structure, low cost, and can only inspect for limited types of defects and models with limited shapes, they are currently in widespread use due to the lack of suitable alternatives.
[0006] Conventional surface inspection systems cannot adjust the illumination direction and light amount to suit various curved surface models, and therefore cannot perform inspection due to the glare phenomenon, i.e., a phenomenon in which a surface is reflected white and cannot be inspected, or cannot detect defects that are only visible in light from a specific direction. [Prior art documents] [Patent documents]
[0007] [Patent Document 1] Korean Patent No. 10-0824808 Summary of the Invention [Problem to be solved by the invention]
[0008] The present invention aims to solve the problem that, when inspecting a curved surface of an object, the surface condition of the object cannot be inspected accurately due to the occurrence of a glare phenomenon.
[0009] The technical problem of the present invention is not limited to the above-mentioned problem, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]
[0010] In order to achieve the above technical objectives, the present invention provides a non-Lambertian surface inspection system for line scanning, comprising: a frame module having a curved mounting hole on at least one of one side and the other side, and a photographing hole on the upper end; and a surface inspection unit including a first lighting module and a second lighting module that are installed in the mounting hole and output light in an oblique line toward an inspection object placed on a floor surface; a camera unit that is located above the photography hall, passes through the photography hall, photographs an object to be inspected that is located on a floor surface, and generates a total composite image including a first illumination image and a second illumination image; a controller unit that controls an operation of the camera unit, selectively operates the first lighting module and the second lighting module, and receives the first lighting image and the second lighting image captured by the camera unit; The controller section is Turning on the first lighting module for a set time and then turning it off; turning on the second lighting module when the first lighting module is turned off; When the first lighting module is turned on, a first lighting image captured through the camera unit is received; When the second lighting module is turned on, a second lighting image captured through the camera unit is received; The controller section is an illumination image receiving module that receives the first illumination image and the second illumination image captured by the camera unit in a set row or column area; an align module for aligning a partial image of a plurality of rows or columns of a first illumination image and a partial image of a plurality of rows or columns of a second illumination image; a histogram module for dividing the first illumination image and the second illumination image into a plurality of blocks, assigning a position number to each block, and extracting a reflectance value of the block; a removal module for removing blocks having reflectance values greater than a reference reflectance value by comparing the reflectance values measured in the blocks with a reference reflectance value that has already been set; When the blocks having reflectance values greater than the reference reflectance value are removed from the first illumination image, blocks in the second illumination image that are located at the same positions as the removed blocks in the first illumination image are received and mapped onto the first illumination image to generate a first block correction image; a block correction image module for receiving blocks in the first illumination image at the same positions as the removed blocks in the second illumination image when the blocks having reflectance values greater than the reference reflectance value are removed from the second illumination image, and mapping the blocks to the second illumination image to generate a second block correction image; a composite image generating module for generating a composite image by calculating the first block correction image and the second block correction image according to a preset formula and then synthesizing the first block correction image and the second block correction image, The synthetic image generation module includes: Either process the image into an albedo image in the composite image setting area, or Calculate the amount of change in surface height of the test object in the X-axis direction from the composite image and process it into an X-axis partial differential image, or The amount of change in surface height of the test object in the Y-axis direction is calculated from the composite image and processed into a Y-axis partial differential image, or The amount of change in surface height of the test object in the X-axis and Y-axis directions is calculated from the X-axis partial differential image and the Y-axis partial differential image, and processed into a second-order partial differential image.
[0011] The frame module includes: It can be formed into any shape including a hemisphere.
[0012] The frame module includes: The display device may further include a reflector disposed on the upper surface.
[0013] The frame module is installed in the shooting hall and further includes a light-transmitting plate having a dot pattern formed on an upper surface thereof for reflecting light downward, and a plurality of auxiliary lighting modules installed on a side surface of the light-transmitting plate for irradiating light into an interior of the light-transmitting plate, and the brightness of the auxiliary lighting modules is adjusted by the controller. Effect of the Invention
[0014] The present invention uses a line scan camera and multiple lights deflected at various angles to eliminate the problem of conventional surface inspection systems, i.e., the glare phenomenon that occurs when inspecting the surface of a curved model, through reflectivity and shadows depending on the illumination direction.
[0015] Furthermore, the present invention enables the illumination to be operated to irradiate light at a specific angle in accordance with the curved surface of the object, and enables the acquisition of various images with different characteristics from the object, such as an x-differential image, a y-differential image, a norm-image, an albedo image, a second derivative image, etc. Then, the images can be selected and combined according to the measurement and inspection target, thereby improving the efficiency of the inspection.
[0016] In addition, the present invention can accurately detect scratches on objects, such as black stains, black scratches, and indentations, that occur on objects in industrial sites, particularly objects that are sooted and carbonized such as welding. In particular, the present invention can exhibit higher detection power than conventional surface inspection systems for scratches and indentations that are affected by the direction of illumination.
[0017] In addition, the present invention effectively reduces size compared to conventional methods that indirectly and passively use reflected scattered light through a dome-shaped structure, while also enabling separate and direct control of the intensity of the scattered light, thereby more efficiently controlling noise in surface inspection images. [Brief description of the drawings]
[0018] [Figure 1] FIG. 2 is a diagram illustrating a state of use of a line-scan non-Lambertian surface inspection system according to an embodiment of the present invention. [Diagram 2] 1A-1C are diagrams illustrating a number of block-corrected images actually acquired via a line-scan non-Lambertian surface inspection system according to an embodiment of the present invention. [Diagram 3] FIG. 1 is a block diagram of a line-scan non-Lambertian surface inspection system according to an embodiment of the present invention. [Figure 4] FIG. 2 is a perspective view of the surface inspection unit of FIG. 1. [Diagram 5] 5 is a diagram showing a state in which the lighting module installed in the frame module of FIG. 4 rotates, and a state in which light is output in an oblique line shape from a first lighting module and a second lighting module. FIG. [Figure 6] 5 is a diagram showing a surface inspection unit in a state where a reflector is installed on the frame module of FIG. 4. FIG. [Figure 7] FIG. 5 is a simplified diagram illustrating a plurality of lighting modules installed in the frame module of FIG. 4. [Figure 8] 4 is a flow chart for the operation of a line-scan non-Lambertian surface inspection system in accordance with one embodiment of the present invention. [Figure 9]1A to 1C are diagrams illustrating an operation process of a line-scanning non-Lambertian surface inspection system according to an embodiment of the present invention and images that appear during the operation process. [Figure 10] 1A to 1C are diagrams illustrating an operation process of a line-scanning non-Lambertian surface inspection system according to an embodiment of the present invention and images that appear during the operation process. [Figure 11] 1A to 1C are diagrams illustrating an operation process of a line-scanning non-Lambertian surface inspection system according to an embodiment of the present invention and images that appear during the operation process. [Figure 12] 1A to 1C are diagrams illustrating an operation process of a line-scanning non-Lambertian surface inspection system according to an embodiment of the present invention and images that appear during the operation process. [Figure 13] 1A to 1C are diagrams illustrating an operation process of a line-scanning non-Lambertian surface inspection system according to an embodiment of the present invention and images that appear during the operation process. [Figure 14] FIG. 2 is a diagram showing a captured image and a plurality of block correction images. [Figure 15] FIG. 2 is a perspective view showing a state in which a light-transmitting plate is installed in a frame module. [Figure 16] 11 is a perspective view showing a state in which a light-transmitting plate and a reflecting plate are installed in a frame module. FIG. [Figure 17] 11A and 11B are a front view and a side view showing how scattered light is emitted through a light-transmitting plate. [Figure 18] FIG. [Figure 19] 1A and 1B are illustrative diagrams showing the difference in images depending on whether or not scattered light illumination is used (left: without scattered light illumination, right: with scattered light illumination). [Figure 20] 1 is an exemplary diagram showing a process for detecting scratches and nicks; [Figure 21] Surface profile with or without scattered light (left: very noisy, right: noise removed). [Figure 22] This is a surface histogram distribution diagram depending on whether the light is scattered or not (left: smoothed and has large deviation, right: less smoothed and has small deviation). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0019] The drawings and the description based on the drawings shown in the specification are only examples that allow a person having ordinary skill in the art of the present invention to easily understand the present invention. Therefore, the drawings of the present invention and the detailed description for carrying out the invention do not limit the scope of the claims of the present invention. The scope of the claims of the present invention can be defined only by the claims.
[0020] The non-Lambertian surface inspection system for line scanning of the present invention will be described below with reference to the drawings of the following embodiments of the present invention. However, in order to make the description of the present invention clear and concise, the non-Lambertian surface inspection system for line scanning will first be generally described with reference to Figures 1 and 2. Then, the components constituting the non-Lambertian surface inspection system for line scanning will be described with reference to Figures 3 to 7.
[0021] FIG. 1 is a diagram showing the state of use of a line-scanning non-Lambertian surface inspection system according to an embodiment of the present invention, and FIG. 2 is a diagram showing multiple composite images used for final inspection via a line-scanning non-Lambertian surface inspection system according to an embodiment of the present invention.
[0022] The non-Lambertian surface inspection system 1 for line scanning of the present invention includes a surface inspection unit 10, a camera unit 20, and a controller unit 30 as components, and the surface inspection unit 10 and the camera unit 20 are operated by the controller unit 30. At this time, the multiple lighting modules included in the surface inspection unit 10 output light in a specified order according to a predetermined combination in accordance with the operation of the controller unit 30, and the reflector 113 causes the light output from the lighting modules 121 to 12n and scattered to be irradiated again onto the inspection object A.
[0023] In addition, the controller unit 30 of the line scanning non-Lambertian surface inspection system 1 operates the camera unit 20 in accordance with the operation of the lighting module to photograph the inspection object A, and obtains an overall received image C received from the camera unit 20 including surface information obtained by lighting from multiple directions.
[0024] At this time, the entire received image C can be transformed into a plurality of illumination images E1 to E4 (see FIG. 11) via line-based images D1 to D4 that match the illumination characteristics included therein.
[0025] In this case, the controller unit 30 of the line-scanning non-Lambertian surface inspection system 1 divides the generated illumination images E1-E4 into row or column regions and aligns the row or column regions to generate individual images F1-F4, then divides each image F1-F4 (see FIG. 13) into multiple blocks, calculates the reflectance value of each block, compares it with a previously set reference reflectance value, and removes blocks with reflectance values higher than the reference reflectance value. Then, the blocks corresponding to the removed parts are interpolated from other valid images to generate a block-corrected image.
[0026] Here, the block corrected images G1 to G4 can be formed in accordance with the number of the lighting modules 121 to 12n. As an example, when there are four lighting modules, a first block corrected image G1 to a fourth block corrected image G4 can be generated.
[0027] Thereafter, the line-scanning non-Lambertian surface inspection system 1 can synthesize at least two of the generated plurality of block-corrected images G1 to G4 to generate a composite image H. More specifically, at least two of the first block-corrected image G1 to the fourth block-corrected image G4 can be synthesized and processed into an albedo image H1, an X-direction partial differential image H2, a Y-direction partial differential image H3, a second-order partial differential image H4, and a vertical vector image H5, as shown in FIG.
[0028] As a result, the non-Lambertian surface inspection system 1 for line scanning of the present invention uses a camera unit 20 and multiple lighting modules 121-124 deflected at various angles to prevent the problem of conventional surface inspection systems, i.e., the glare phenomenon when inspecting the surface of a curved model, from occurring through reflectivity and shadows depending on the irradiation direction.
[0029] Furthermore, the non-Lambertian surface inspection system 1 for line scanning of the present invention can select and combine these images according to the measurement and inspection object, thereby improving the reliability of the inspection of the inspection object.
[0030] This allows the line scanning non-Lambertian surface inspection system 1 to accurately detect scratches on the test object A, such as black stains, black scratches, indentations, etc., in industrial sites, especially in sooted and carbonized objects such as welding.
[0031] Hereinafter, components constituting the line-scan non-Lambertian surface inspection system will be described with reference to FIGS.
[0032] Fig. 3 is a block diagram of a line-scan non-Lambertian surface inspection system according to an embodiment of the present invention, and Fig. 4 is a perspective view of the surface inspection unit of Fig. 1. Fig. 5 is a diagram showing a state in which the illumination module installed in the frame module of Fig. 4 rotates and a state in which light is output in an oblique line shape from the first illumination module and the second illumination module, and Fig. 6 is a diagram showing the surface inspection unit in a state in which a reflector is installed in the frame module of Fig. 4. And Fig. 7 is a diagram simply showing a plurality of illumination modules installed in the frame module of Fig. 4.
[0033] The surface inspection unit 10 is a device capable of irradiating light at various angles onto the inspection object A so that the camera unit 20 can photograph the inspection object A. The surface inspection unit 10 includes a frame module 110 formed in a shape contained within a hemisphere and having a photographing hole 112 at its upper end, and a lighting module installed on the frame module 110 and capable of rotating and moving.
[0034] Here, as shown in FIG. 4, the frame module 110 has a curved mounting hole 111 on at least one of its two sides, and a plurality of lighting modules are installed in the mounting hole 111 to output light in an oblique line toward the test object A placed on the floor.
[0035] At this time, the plurality of lighting modules are lighting for a Shadow-metric System (SMS) optimized for line scan. In this specification, the plurality of lighting modules installed in the surface inspection unit 10 are configured as a first lighting module 121, a second lighting module 122, a third lighting module 123, and a fourth lighting module 124 so that the description of the present invention is concise and clear. At this time, the first lighting module 121, the second lighting module 122, the third lighting module 123, and the fourth lighting module 124 may be lighting for a Shadow-metric System (SMS) optimized for line scan. Such first lighting module 121 to fourth lighting module 124 can be selectively rotated in the mounting hole 111 as shown in FIG. 5(a), and can output light in a state inclined at 45 degrees as shown in FIG. 5(b) and (c).
[0036] 6, the surface inspection unit 10 includes a reflector 113 installed on the upper side of the frame module 110, so that the light output from the plurality of lighting modules and scattered can be reflected and irradiated again onto the inspection object A. In this case, the first lighting module 121 is installed at the lower end of the mounting hole 111 on the left side, and the second lighting module 122 is installed at the upper end of the mounting hole 111 on the left side apart from the first lighting module 121. The third lighting module 123 is installed at the upper end of the mounting hole 111 on the right side, and the fourth lighting module 124 is installed at the lower end of the mounting hole 111 on the right side apart from the third lighting module 123.
[0037] In this manner, the first lighting module 121 to the fourth lighting module 124 installed at a distance in the mounting hole 111 can rotate at the installed positions and irradiate light in various directions onto the test object A. Furthermore, the first lighting module 121 to the fourth lighting module 124 can move along the mounting hole 111 and irradiate light onto the test object A at various positions.
[0038] The camera unit 20 is positioned above the photographing hole 112, passes through the photographing hole 112, photographs the object A placed on the floor B, and receives an entire image C including a first illumination image E1 and a second illumination image E2.
[0039] More specifically, the camera unit 20 may be installed above the photography hole 112 such that a lens is positioned at a position overlapping the photography hole 112. The camera unit 20 penetrates the photography hole 112 to photograph the inspection object A positioned on the floor surface and generate an illumination image.
[0040] The camera unit 20 is operated according to a control signal output from the controller unit 30. In particular, the camera unit 20 operates in synchronization with the operation of the lighting modules 121 to 12n, and can photograph the object A only when the lighting modules 121 to 12n are turned on.
[0041] The camera unit 20 can generate the first illumination image E1 to the fourth illumination image E4 when the first illumination module 121, the second illumination module 122, the third illumination module 123, and the fourth illumination module 124 are combined according to an inspection object and turned on in a specified order. As an example, the camera unit 20 captures the first illumination image E1 when the first illumination module 121 is turned on, captures the second illumination image E2 when the second illumination module 122 is turned on, captures the third illumination image E3 when the third illumination module 123 is turned on, and captures the fourth illumination image E4 when the fourth illumination module 124 is turned on.
[0042] The controller unit 30 is a computer that generates control signals and receives and processes data. The controller unit 30 can control the illumination time and delay time in nanosecond units when controlling cross illumination using line scan. The controller unit 30 includes a separate control board equipped with an FPGA chip at the limit of the control board due to control using a conventional microcontroller CPU.
[0043] Therefore, the controller unit 30 can program and immediately reflect problematic situations in the field, such as the lighting sequence, frequency division, noise removal, etc., according to the object of inspection. Also, the controller unit 30 can include and operate an illumination image receiving module 310, an alignment module 320, a histogram module 330, a removal module 340, a block correction image module 350, a composite image generating module 360, and a control signal generating module 370, as shown in FIG.
[0044] The controller unit 30 generates a control signal from the control signal generating module 370 to operate the plurality of lighting modules 12 and the camera unit 20. As an example, the controller unit 30 can turn on the first lighting module 121 for a set time and then turn it off, and turn on the second lighting module 122 when the first lighting module 121 is turned off, turn on the third lighting module 123 when the second lighting module 122 is turned off, and turn on the fourth lighting module 124 when the third lighting module 123 is turned off.
[0045] The controller unit 30 can turn on and off the first lighting module 121 to the fourth lighting module 124 and can turn on the first lighting module 121 to the fourth lighting module 124. The controller unit 30 can periodically turn on the lighting, except for specific lighting, by adjusting the brightness according to a preset standard according to the inspection speed efficiency during each line scan. As an example, the first lighting module 121, the second lighting module 122, and the fourth lighting module 124 can be sequentially turned on for the A inspection object to generate a first lighting image, a second lighting image, and a fourth lighting image, and the first lighting module 121, the second lighting module 122, and the third lighting module 123 can be sequentially turned on for the B inspection object to generate a first lighting image, a second lighting image, and a third lighting image.
[0046] Then, the controller unit 30 operates the camera unit 20 to generate the first to fourth illumination images, and receives each of the generated illumination images. The controller unit 30 processes the illumination images to generate processed images, that is, an albedo image H1, an X-direction partial differential image H2, a Y-direction partial differential image H3, a second-order partial differential image H4, and a vertical vector image H5.
[0047] More specifically, the controller unit 30 acquires an entire image C including a plurality of pieces of image information from the camera unit 20 via the illumination image receiving module 310. Then, image sections D1 to D4 for each illumination group are selectively extracted from the entire image C to create a first illumination image E1 to a fourth illumination image E4, and the image is divided into row or column regions set for each image.
[0048] Then, the align module 320 aligns the partial images D1-D4 of the rows or columns of the first illumination image E1 with the partial images D1-D4 of the rows or columns of the second illumination image E2. After that, the histogram module 330 divides the blocks into equal-sized blocks, assigns position numbers to the blocks, and extracts the reflectance values of the blocks. Next, the removal module 340 compares a set reference reflectance value with the reflectance value measured in the block, and removes the block having a reflectance value greater than the reference reflectance value. Then, the block correction image module 350 and the removal module 340 interpolate the blocks corresponding to the portions from which the blocks have been removed from other valid images to generate the first block correction image G1 and the second block correction image G2.
[0049] As an example, when the reflectance value of the first block HU2 of the second illumination image becomes equal to or greater than the reference reflectance value, the reflectance value of the first block HU2 is changed to 0. Then, the average value of the reflectance value of the first block HU1 of the first illumination image, the reflectance value of the first block HU3 of the third illumination image, and the reflectance value of the first block HU4 of the fourth illumination image is reflected in the first block HU2 of the second illumination image.
[0050] Furthermore, when the reflectance value of the second block HD3 in the third illumination image becomes equal to or greater than the reference reflectance value, the reflectance value of the second block HD2 is changed to 0. Then, the average value of the reflectance value of the second block HD1 in the first illumination image, the reflectance value of the second block HD2 in the second illumination image, and the reflectance value of the second block HD4 in the fourth illumination image is reflected in the second block HD3 in the third illumination image.
[0051] Moreover, the first block corrected image G1 and the second block corrected image G2 are synthesized via a synthesized image generation module 360 to generate a synthesized image H.
[0052] In addition to the above features, the controller unit 30 is formed by a computer and can freely adjust the lighting sequence of a plurality of lighting modules, and can adjust the brightness of each of the first to fourth lighting modules using a PWM (Pulse Width Modulation) method. Also, since the brightness of the lighting may vary depending on the lighting angle according to the characteristics of the test object, the controller unit 30 manages the lighting value irradiated on the test object A to obtain standardized test results.
[0053] Hereinafter, the operation of the line-scan non-Lambertian surface inspection system will be specifically described with reference to FIGS.
[0054] Fig. 8 is a flow chart of the operation of the line-scan non-Lambertian surface inspection system according to an embodiment of the present invention, Figs. 9 to 13 are diagrams showing the operation process of the line-scan non-Lambertian surface inspection system according to an embodiment of the present invention and images appearing during the operation process, and Fig. 14 is a diagram showing a number of block-corrected images.
[0055] The controller unit 30 operates in a series of steps as shown in FIG. 8 to generate processed images, i.e., an albedo image H1, an X-direction partial differential image H2, a Y-direction partial differential image H3, a second-order partial differential image H4, and a vertical vector image (Norm image).
[0056] More specifically, as shown in FIG. 9, when the inspection object A enters the surface inspection unit 10, the controller unit 30 sequentially operates a plurality of illumination modules to generate a plurality of illumination images via the camera unit 20. At this time, the first illumination module 121 to the fourth illumination module 124 irradiate the inspection object A with light in different directions as shown in FIG. 10. The controller unit 30 can receive the entire image C in response to the operation of the first illumination module 121 to the fourth illumination module 124. Then, the controller unit 30 extracts partial images D1 to D4 of a plurality of rows or columns from the received overall composite image C, and aligns each of the extracted partial images to generate the first illumination image E1 to the fourth illumination image E4.
[0057] 13, the controller unit 30 divides the image into a plurality of blocks, extracts reflectance values from the divided blocks, and compares the reflectance values of the extracted blocks with a previously set reference reflectance value. Here, the histogram module 330 of the controller unit 30 divides the first illumination image E1 to the fourth illumination image E4 into blocks F, assigns position numbers to the blocks F, and extracts reflectance values from each block. Then, a curve graph corresponding to the reflectance values can be formed.
[0058] Furthermore, the blocks F generated by the histogram module 330 can be pixels, and the reflectance value contained in each block can be extracted from 0 to 255.
[0059] At this time, the histogram module 330 can extract reflectance values based on the illustrated symbols using the following formula: At this time, the unit vector toward each lighting module 12 is given by formula 1, and the normal unit vector of points x, y on the surface of the inspection object A is given by formula 2. Then, partial differentiation of the surface along the axis and axis at point (x, y) on the surface of the inspection object is given by formula 3.
[0060]
number
[0061]
number
[0062]
number
[0063] Here, σ is the slant angle of the illumination, τ is the tilt angle of the illumination, ρ is a surface constant, and λ is the illumination intensity. The image intensity in the image coordinates can be expressed as Equation 4.
[0064]
number
[0065] At this time, the removal module 340 of the controller unit 30 compares the reflectance value measured in each block F with the reference reflectance value, including the reference reflectance value. Then, the removal module 340 removes the block F having a reflectance value greater than the reference reflectance value. As an example, the removal module 340 removes the block F having a reflectance value greater than the reference reflectance value, including a reflectance value corresponding to 254 as the reference reflectance value.
[0066] In this way, the controller unit 30 leaves a block if its reflectance value is less than the reference reflectance value, removes the block if its reflectance value is equal to or greater than the reference reflectance value, removes the block whose reflectance value exceeds the reference reflectance value, and merges the block corresponding to the position of the removed block with a block extracted from another illumination image by interpolation, or removes the block if its reflectance value is equal to or greater than the reference reflectance value, removes the block whose reflectance value exceeds the reference reflectance value, and maps the block corresponding to the position of the removed block to the block extracted from another illumination image.
[0067] Furthermore, the controller unit 30 corrects a portion of the blocks by an interpolation method, and generates a first corrected block image G1, a second corrected block image G2, a third corrected block image G3, and a fourth corrected block image G4.
[0068] Here, the first block correction image G1 may be an image captured by light irradiated from the first lighting module 121, as shown in (a) of FIG. 14, or an image in which some blocks are corrected after the image is captured. The second block correction image G2 may be an image captured by light irradiated from the second lighting module 122, as shown in (b) of FIG. 14, or an image in which some blocks are corrected after the image is captured. The third block correction image G3 may be an image captured by light irradiated from the third lighting module 123, as shown in (c) of FIG. 14, or an image in which some blocks are corrected after the image is captured. And the fourth block correction image G4 may be an image captured by light irradiated from the fourth lighting module 124, as shown in (d) of FIG. 14, or an image in which some blocks are corrected after the image is captured.
[0069] Furthermore, the composite image generation module 360 can process the composite image H into an albedo image H1, or calculate the amount of change in the surface height of the inspection object in the X-axis direction from the composite image H and process it into an X-axis partial differential image H2, or calculate the amount of change in the surface height of the inspection object in the Y-axis direction and process it into a Y-axis partial differential image H3, or calculate the amount of change in the surface height of the inspection object in the X-axis and Y-axis directions from the X-axis partial differential image and the Y-axis partial differential image and process it into a second derivative image H4.
[0070] More specifically, the synthetic image generation module 360 can obtain the albedo image H1 from the synthetic image by the following formula (Formula 5).
[0071]
number
[0072] At this time, the brightness may be due to scattered light excluding the brightness due to specular reflection caused by the illumination of the surface of the object.
[0073] Then, the synthetic image generation module 360 can obtain an X-axis direction partial differential image H2, which shows a significant change in the surface height of the object in the X-axis direction, as shown in FIG. 2, using a formula for calculating the change in the surface height of the object in the X-axis direction, i.e., formula (6).
[0074]
number
[0075] Also, by using a formula for calculating the amount of change in surface height of the object in the Y-axis direction from the composite image, i.e., formula (7), it is possible to obtain a Y-axis direction partial differential image H3 in which the characteristics of the change in surface height of the object in the Y-axis direction are prominently shown, as shown in Fig. 2. Meanwhile, the composite image generation module 360 can obtain a curvature image in which the expression of the characteristics of the curvature of the surface height of the object is prominently shown, as shown in Fig. 2, by using a formula for calculating the amount of change in surface height of the object from the acquired image, i.e., formula (8).
[0076]
number
[0077]
number
[0078] Then, the synthetic image generation module 360 can obtain a Norm image in which the light received perpendicular to the surface is converted into an image as shown in FIG. 2 by using the formulas (9 and 10) to calculate values in the direction perpendicular to the surface from the synthetic image H.
[0079]
number
[0080]
number
[0081] Furthermore, the controller unit 30 can measure in real time a combination of three or more of the illuminations that directly enter the camera unit 20 due to total reflection among the light source directions (1) to (n) shown in Figure 7, and eliminate them when synthesizing images to improve the quality of the overall image.
[0082] The line scanning non-Lambertian surface inspection system 1 generates an albedo image H1, an X-axis direction partial differential image H2, a Y-axis direction partial differential image H3, a second-order partial differential image H4 obtained by secondarily differentiating a Norm image and a curvature image, and an inspection image, thereby enabling accurate inspection of irregular and non-Lambertian surfaces of an object to be inspected.
[0083] Figure 15 is an oblique view showing the state in which a light-transmitting plate is installed in a frame module, Figure 16 is an oblique view showing the state in which a light-transmitting plate and a reflecting plate are installed in a frame module, Figure 17 is a front view and a side view showing the state in which scattered light is emitted through the light-transmitting plate, and Figure 18 is a plan view showing the light-transmitting plate.
[0084] 15 to 18, the frame module 110 of the line-scanning non-Lambertian surface inspection system 1 further includes a light-transmitting plate 114 and a supplemental lighting module 125.
[0085] The light-transmitting plate 114 is installed in the shooting hall 112, and a dot pattern is formed on the upper surface thereof to reflect light downward.
[0086] A plurality of auxiliary lighting modules 125 are installed on the side of the transparent plate 114 to irradiate light into the inside of the transparent plate 114 .
[0087] Here, the brightness of the auxiliary lighting module 125 is adjusted via the controller unit 20 .
[0088] This structure effectively reduces size compared to the conventional method of indirectly and passively using reflected scattered light through a dome-shaped frame module 110, while allowing for separate and direct control of the intensity of the scattered light, thereby more efficiently controlling noise in surface inspection images.
[0089] Noise in surface inspection images can be classified into two types: dark current noise that is generated by the camera sensor in low-light conditions, and shot noise that is caused by minute non-uniform characteristics on the product surface.
[0090] If the reflected light on the surface of the product is too low, the image will be dark overall, and the low-light sensitivity of the sensor will be weakened, causing dark current noise, which requires light with a vertical component of a certain brightness or higher.
[0091] Passing balanced scattered light can replenish the vertical light and reduce dark current noise.
[0092] In addition, if the scattered light shows significant differences due to fine polishing conditions or minute defects (within 1 to 2 μm) on the product surface, this acts like shot noise, making it difficult to detect the surface defects (notches and scratches of 10 μm or more) that are actually being inspected. However, these noise elements can be offset through balanced scattered light illumination, making it possible to display an image with noise removed.
[0093] Therefore, in the case of the present invention, noise can be removed by using active scattered light to obtain an image of the surface albedo (reflectance), and since the separately separated active scattered light can be adjusted, there is less impact on the curvature (radius of curvature / differential) image.
[0094] This allows for good images to be obtained for surface inspection.
[0095] According to Mie scattering, light is scattered when it is irradiated onto an irregular surface below a certain size, and Mie scattering occurs when the particles are similar in size to the wavelength of light or smaller.
[0096] The intensity of the scattered light depends on the angle of incidence of the light.
[0097] An accurate calculation for the surface roughness of an object can be made using Maxwell's equations.
[0098] When incident light is scattered from the surface of the inspection object, the particle density M and the brightness of the scattered light satisfy the following equation (Equation 11):
[0099]
number
[0100] The concentration of surface particles is linearly related to the brightness of the scattered light.
[0101] The present invention is designed to appropriately adjust the distance from the camera and the brightness of the auxiliary lighting module to maximize the difference in brightness between the light scattered from the inherent surface roughness of the object being inspected and the light coming from surface imperfections, based on the above formula.
[0102] When the surface of the object being inspected is contaminated by welding but is not defective, or when the surface is bent and spots are visible on the surface but are not separated as defective, it may be difficult to distinguish them from defects such as scratches or nicks that have appeared on the surface.
[0103] In the present invention, images are taken while varying the brightness of the supplemental lighting module, and these images are used to distinguish and detect such welding contamination or surface bends and defects.
[0104] FIG. 19 is an illustrative diagram showing the difference in image depending on whether or not scattered light illumination is used (a: without scattered light illumination, B: with scattered light illumination).
[0105] The (a) image in Figure 19 is an image taken without the scattered light system, and the (b) image is the composite result using scattered light.
[0106] In the image of FIG. 19, the light areas are scratch or nick defects and the dark areas are the surface of the object being inspected.
[0107] There is almost no change in brightness between images (a) and (b), but as the illumination from the supplemental lighting module becomes brighter, the brightness of the surface of the target object becomes brighter.
[0108] Therefore, a portion where there is little change in the brightness of the illumination is extracted as a defect (a scratch or a notch).
[0109] In principle, when the size D of the surface particles of the object to be inspected is within the Mie scattering range, the brightness of the surface of the object to be inspected has a linear relationship with the brightness of the supplemental lighting module, but when the size D of the particles is larger than the Mie scattering range (e.g., defects such as scratches or nicks), there is almost no relationship with the brightness of the supplemental lighting module.
[0110] FIG. 20 is an exemplary diagram showing the process of detecting scratches and nicks.
[0111] Referring to FIG. 20, Image 0 is acquired with Intensity 0 as the brightness of the Diffuse Module, and Image n is acquired with Intensity n. The difference between Image 0 and Image n is calculated, and scratches and nicks are detected from this image.
[0112] FIG. 21 shows the surface profile with and without scattered light (a: severe noise, b: noise removed).
[0113] The green line in Figure 21 is the profile of the image.
[0114] In the profile of the image in Figure 21(b), it can be seen that the standard deviation value has been reduced from the existing 20 to 15.
[0115] FIG. 22 shows the surface histogram distribution diagrams depending on whether the light is scattered or not (a: smoothed and has a large deviation, b: less smoothed and has a small deviation).
[0116] Looking at the histogram of the green region, we can see that the standard deviation value has been reduced from the existing 23 to 15 by dramatically reducing floor noise information using scattered light.
[0117] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art will understand that the present invention can be embodied in other specific forms without changing the technology or essential features. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and are not limiting. [Explanation of symbols]
[0118] 1. Line Scan Non-Lambertian Surface Inspection System 10 Surface Inspection Department 110 Frame Module 111 Fixed Hall 112 Photo Hall 113 Reflector 114 Translucent plate 121 First Lighting Module 122 Second Lighting Module 125 Auxiliary Lighting Module 20 Camera Division 30 Controller section 310 Lighting image receiving module 320 Align Module 330 Histogram Module 340 Removal Module 350 Block Correction Image Module 360 synthetic image generation module A Inspection body B Floor surface C Whole received image D1 First partial image D2 Second partial image D3 Third partial image D4 4th partial image E1 First lighting image E2 Second lighting image E3 3rd lighting image E4 4th lighting image F1, F2, F3, F4 Image block division, glare removal image G1 1st block correction image G2 2nd block correction image G3 3rd block correction image G4 4th block correction image H Composite Image H1 Albedo Image H2 X-axis partial differential image H3 Y-axis partial differential image H4 Second-order partial differential image H5 Surface Vertical Vector Image I Light
Claims
1. a frame module having a curved mounting hole on at least one of its first and second sides and a photography hole on its upper end; and a surface inspection unit including a first lighting module and a second lighting module that are installed in the mounting hole and output light at an oblique angle toward an end of an inspection object placed on a floor surface in a horizontal direction perpendicular to the transport direction of the inspection object; a camera unit that is located above the photography hall, passes through the photography hall, photographs an object to be inspected that is located on a floor surface, and generates a total composite image including a first illumination image and a second illumination image; a controller unit that controls the operation of the camera unit, selectively operates the first lighting module and the second lighting module, processes the first lighting image (E1) and the second lighting image (E2) according to a preset formula, and then synthesizes the processed images to generate a composite image; The controller unit turning on the first lighting module for a set time and then turning it off; turning on the second lighting module when the first lighting module is turned off; receiving the first lighting image captured via the camera unit when the first lighting module is turned on; receiving the second lighting image captured via the camera unit when the second lighting module is turned on; The controller unit an illumination image receiving module that receives the first illumination image and the second illumination image captured by the camera unit in a set row or column area; an align module for aligning a first illumination image (E1) received in a region of a plurality of rows or columns and a second illumination image (E2) received in a region of a plurality of rows or columns; a histogram module for dividing the first illumination image and the second illumination image into a plurality of blocks, assigning a position number to each block, and extracting a reflectance value of the block; a removal module for removing blocks having reflectance values greater than a reference reflectance value by comparing the reflectance values measured in the blocks with a reference reflectance value that has already been set; When a block having a reflectance value greater than a reference reflectance value is removed from the first illumination image, a block in the second illumination image at the same position as the removed block in the first illumination image is received and mapped onto the first illumination image to generate a first block correction image; a block correction image module that, when a block having a reflectance value greater than a reference reflectance value is removed from the second illumination image, receives a block in the first illumination image at the same position as the removed block in the second illumination image, and maps the block to the second illumination image to generate a second block correction image; The controller unit a composite image generating module for generating a composite image by calculating the first block corrected image and the second block corrected image according to a preset formula and then synthesizing the first block corrected image and the second block corrected image, The synthetic image generation module includes: Either process the image into an albedo image in the composite image setting area, or Calculate the amount of change in surface height of the test object in the X-axis direction from the composite image and process it into an X-axis partial differential image, A change in surface height of the test object in the Y-axis direction is calculated from the composite image and processed into a Y-axis partial differential image, or A change in surface height of the object in the X-axis direction and the Y-axis direction is calculated from the X-axis partial differential image and the Y-axis partial differential image, and the change is processed into a second-order partial differential image. The frame module includes: a light-transmitting plate installed in the photography hall and having a dot pattern formed on an upper surface thereof for reflecting light downward; The light-transmitting plate further includes a plurality of auxiliary lighting modules installed on a side surface of the light-transmitting plate and irradiating light into the inside of the light-transmitting plate, The auxiliary lighting module includes:
11. A line-scan non-Lambertian surface inspection system, wherein brightness is adjusted by the controller unit.
2. The line-scan non-Lambertian surface inspection system of claim 1 , wherein the frame module is formed into a shape that is contained within a semi-cylinder.
3. The frame module includes: The line-scan non-Lambertian surface inspection system of claim 1 , further comprising a reflector mounted on an upper surface thereof.
Citation Information
Patent Citations
Light source module and optical detection system
CN110823902A
Automated optical inspective apparatus for electrode tab of secondary battery
KR100824808B1
System For Inspecting Of Non-Lambertian Surface
KR102135472B1
Machine Vision Inspection System and Light Source Module thereof
US20100103661A1
Multi-modal imaging
US20100245813A1
