PVC plastic wall / floor surface quality detection method and system

Abstract

The invention discloses a PVC plastic wall / floor surface quality detection method, and belongs to the field of quality detection.The method comprises the steps that diffuse reflection model calibration is conducted on the surface of an object to be detected, and a line scanning camera is used for collecting a first collection image and a second collection image when a first line light source and a second line light source are sequentially lightened; reconstructing the first acquisition image and the second acquisition image by using a photometric stereo method to obtain a two-dimensional gradient map and a three-dimensional gradient map of the to-be-detected object; performing pseudo-color conversion on the two-dimensional gradient map and the three-dimensional gradient map, performing RGB channel separation on the converted pseudo-color two-dimensional gradient map and pseudo-color three-dimensional gradient map, fusing the pseudo-color two-dimensional gradient map and pseudo-color three-dimensional gradient map of different channels, and performing three-channel merging to obtain a color fusion image. And performing defect identification by using the color fusion image. Manual detection can be replaced, the appearance quality detection efficiency of the PVC plastic wall / floor is greatly improved, and production data management is facilitated.

Description

Technical Field

[0001] The invention relates to the technical field of quality detection, and in particular to a method and system for detecting the surface quality of a PVC plastic wall / floor. Background Art

[0002] The PVC floor manufacturing industry is a labor-intensive industry. Although the current processing process is highly automated, a large number of workers are still needed in the inspection positions of key processes such as coating, slotting, and gluing. Taking the slotting line as an example, there are generally two inspectors at the inspection station, working in two shifts normally and three shifts in the peak season. The inspection cycle is 40 to 50 pieces per minute, focusing on the detection of fatal defects such as poor coating, color difference, scratches, dirt, and damaged edges. As the consumption level continues to rise, consumers are becoming more and more demanding on the appearance quality of products, and these demanding quality requirements are exceeding the limits of manual visual inspection. Driven by the dual factors of ensuring the healthy development of the industry and meeting consumer needs, it is urgent to develop automated and intelligent inspection technologies for the rapid detection of defects in the production and manufacturing process of PVC floors. Summary of the invention

[0003] The present invention provides a PVC plastic wall / floor surface quality detection method and system, which can replace manual detection, greatly improve the efficiency of PVC plastic wall / floor appearance quality detection, and facilitate production data management.

[0004] The first aspect of the present invention provides a PVC plastic wall / floor surface quality detection method, wherein a first line light source and a second line light source are symmetrically placed about a light plane of a line scan camera, and the incident angles of the first line light source and the second line light source are the same as the installation height, and the line scan camera is vertically photographed on the surface of the object to be detected, and the detection method comprises the following steps:

[0005] Perform diffuse reflection model calibration on the surface of the object to be detected, and use the line scan camera to respectively capture a first captured image and a second captured image when the first line light source and the second line light source are sequentially lit;

[0006] Reconstructing the first collected image and the second collected image using photometric stereo to obtain a two-dimensional gradient map and a three-dimensional gradient map of the object to be detected;

[0007] The two-dimensional gradient map and the three-dimensional gradient map are subjected to pseudo-color conversion, the converted pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map are subjected to RGB channel separation, the pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map of different channels are fused, and three-channel merging is performed to obtain a color fused image, and the color fused image is used for defect recognition.

[0008] Optionally, in one embodiment of the present invention, calibrating the diffuse reflection model of the surface of the object to be detected includes:

[0009] The surface reflection model of the object to be detected is assumed to conform to the diffuse reflection model. The light intensity calculation formula of the diffuse reflection model is:

[0010]

[0011] When there are only two light sources, for the first line light source:

[0012]

[0013] For the second line light source:

[0014]

[0015] Where, l=(l x ,l y ,l z ), n = (p, q, -1) is the normalized normal vector, p is the surface gradient in the x direction, q is the surface gradient in the y direction, and since the light sources are all linear light sources, the light intensity at each point in the x direction is equal, and we get l x1 = l x2 = 0, the two light sources are at the same height from the surface of the object to be measured, so l z1 = l z2 = l z , place two light sources symmetrically with the same incident angle, and place them on both sides of the normal line of the object to be measured to obtain l y1 =-l y2 , determine the direction vector of the light source as l1 = (0, sinαcosα), l2 = (0, -sinαcosα), α is the incident angle of the light source, and substitute it into:

[0016]

[0017] Divide the two equations to find the y-direction gradient q:

[0018]

[0019] Wherein, e=E1 / E2 is the light source power coefficient, and E1 and E2 are the average grayscales of the first line light source and the second line light source, respectively.

[0020] Optionally, in one embodiment of the present invention, it further includes:

[0021] The surface normal gradient of the two-dimensional gradient map of the object to be detected is integrated to restore the surface shape, including the path integral method and the global iterative optimization method. The global optimization algorithm relies on iteration to gradually diffuse the gradient information. Since the gradient information can only diffuse within a limited radius each time, the global optimization algorithm requires multiple iterations to evenly diffuse the gradient information to the entire surface to achieve coordination between depth and gradient.

[0022] Optionally, in one embodiment of the present invention, the three channels are merged using a weighted fusion method, and the optimal fusion coefficients α, β, γ are obtained using a structural similarity index and a peak signal-to-noise ratio:

[0023]

[0024] Among them, α, β, γ∈(0,1), R fusionmap , G fusionmap and B fusionmap It corresponds to the grayscale information of the red, green and blue channels after weighted fusion.

[0025] Optionally, in one embodiment of the present invention, using the color fused image to perform defect recognition includes:

[0026] A two-stage neural network model is used for defect detection. The first stage of target detection processes the entire image and directly outputs the defect location and defect type. The second stage of target detection uses a classifier to roughly distinguish the image background and foreground and generate a preliminary detection frame. The feature information extracted from the corresponding position in the feature map is input into the convolution layer for category determination and output of precise location information.

[0027] The second aspect of the present invention provides a PVC plastic wall / floor surface quality detection system, wherein a first line light source and a second line light source are symmetrically placed about a light plane of a line scan camera, and the first line light source and the second line light source have the same incident angle and installation height, and the line scan camera shoots vertically on the surface of an object to be detected, and the detection system comprises:

[0028] An acquisition module, used for calibrating a diffuse reflection model of the surface of the object to be detected, and using the line scan camera to respectively acquire a first acquired image and a second acquired image when the first line light source and the second line light source are sequentially lit;

[0029] A reconstruction module, used to reconstruct the first collected image and the second collected image using a photometric stereo method to obtain a two-dimensional gradient map and a three-dimensional gradient map of the object to be detected;

[0030] The detection module is used to perform pseudo-color conversion on the two-dimensional gradient image and the three-dimensional gradient image, separate the RGB channels of the converted pseudo-color two-dimensional gradient image and the pseudo-color three-dimensional gradient image, fuse the pseudo-color two-dimensional gradient images and the pseudo-color three-dimensional gradient images of different channels, and perform three-channel merging to obtain a color fused image, and use the color fused image to identify defects.

[0031] Optionally, in one embodiment of the present invention, calibrating the diffuse reflection model of the surface of the object to be detected includes:

[0032] The surface reflection model of the object to be detected is assumed to conform to the diffuse reflection model. The light intensity calculation formula of the diffuse reflection model is:

[0033]

[0034] When there are only two light sources, for the first line light source:

[0035]

[0036] For the second line light source:

[0037]

[0038] Where, l=(l x ,l y ,l z ), n = (p, q, -1) is the normalized normal vector, p is the surface gradient in the x direction, q is the surface gradient in the y direction, and since the light sources are all linear light sources, the light intensity at each point in the x direction is equal, and we get l x1 = l x2 = 0, the two light sources are at the same height from the surface of the object to be measured, so l z1 = l z2 = l z , place two light sources symmetrically with the same incident angle, and place them on both sides of the normal line of the object to be measured to obtain l y1 =-l y2 , determine the direction vector of the light source as l1 = (0, sinα, cosα), l2 = (0, -sinα, cosα), α is the incident angle of the light source, and substitute it into:

[0039]

[0040] Divide the two equations to find the y-direction gradient q:

[0041]

[0042] Wherein, e=E1 / E2 is the light source power coefficient, and E1 and E2 are the average grayscales of the first line light source and the second line light source, respectively.

[0043] Optionally, in one embodiment of the present invention, it further includes:

[0044] The surface normal gradient of the two-dimensional gradient map of the object to be detected is integrated to restore the surface shape, including the path integral method and the global iterative optimization method. The global optimization algorithm relies on iteration to gradually diffuse the gradient information. Since the gradient information can only diffuse within a limited radius each time, the global optimization algorithm requires multiple iterations to evenly diffuse the gradient information to the entire surface to achieve coordination between depth and gradient.

[0045] Optionally, in one embodiment of the present invention, the three channels are merged using a weighted fusion method, and the optimal fusion coefficients α, β, γ are obtained using a structural similarity index and a peak signal-to-noise ratio:

[0046]

[0047] Among them, α, β, γ∈(0,1), R fusionmap , G fusionmap and B fusionmap It corresponds to the grayscale information of the red, green and blue channels after weighted fusion.

[0048] Optionally, in one embodiment of the present invention, the detection module is also used to perform defect detection using a two-stage neural network model. The first-stage target detection processes the entire image range and directly outputs the defect location and defect type. The second-stage target detection uses a classifier to roughly distinguish the image background and foreground and generate a preliminary frame to be detected. The feature information extracted from the corresponding position in the feature map is input into the convolution layer for category determination and output of precise position information.

[0049] The PVC plastic wall / floor surface quality detection method and system of the embodiment of the present invention calibrates the diffuse reflection model of the surface of the object to be detected, and uses a line scan camera to respectively collect the first and second collected images when the first line light source and the second line light source are lit in sequence; the first and second collected images are reconstructed using the photometric stereo method to obtain a two-dimensional gradient map and a three-dimensional gradient map of the object to be detected; the two-dimensional gradient map and the three-dimensional gradient map are pseudo-color converted, and the converted pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map are separated into RGB channels, the pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map of different channels are fused, and three-channel merging is performed to obtain a color fused image, and the color fused image is used for defect identification. The present invention can replace manual detection, greatly improve the efficiency of PVC plastic wall / floor appearance quality detection, and facilitate production data management.

[0050] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0052] Figure 1 A flow chart of a PVC plastic wall / floor surface quality detection method and system provided according to an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of dual-light source line scanning imaging according to an embodiment of the present invention;

[0054] Figure 3 This is a diagram of the imaging principle of an embodiment of the present invention;

[0055] Figure 4 This is an imaging effect diagram of an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of a geometric model for calibrating a reflection model according to an embodiment of the present invention;

[0057] Figure 6 Schematic diagram of a reflection model calibration device according to an embodiment of the present invention;

[0058] Figure 7 Schematic diagram of average grayscale transformation curve according to an embodiment of the present invention;

[0059] Figure 8 Schematic diagram of a dual-light source photometric stereo experiment platform according to an embodiment of the present invention;

[0060] Fig. 9 The illumination images at different angles according to the embodiment of the present invention;

[0061] Fig.10 A schematic diagram of gradient reconstruction according to an embodiment of the present invention;

[0062] Fig.11 A schematic diagram of a defect binarization image according to an embodiment of the present invention;

[0063] Fig.12 A schematic diagram of a global iterative optimization method according to an embodiment of the present invention;

[0064] Fig.13 A schematic diagram of depth reconstruction according to an embodiment of the present invention;

[0065] Fig.14 A pseudo-color conversion diagram of an embodiment of the present invention;

[0066] Fig.15It is a gradient depth channel separation diagram of an embodiment of the present invention;

[0067] Fig.16 It is a fusion diagram of three channels of an embodiment of the present invention;

[0068] Fig.17 is the SSIM value of the fused image under different coefficients of the embodiment of the present invention;

[0069] Fig.18 is the PSNR value of the fused image under different coefficients in the embodiment of the present invention;

[0070] Fig.19 A product defect detection flow chart of an embodiment of the present invention;

[0071] Fig. 20 It is a block diagram of a PVC plastic wall / floor surface quality detection system according to an embodiment of the present application. DETAILED DESCRIPTION

[0072] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0073] The image acquisition device of the present invention comprises a first line light source, a second line light source and a line scan camera, wherein the first line light source and the second line light source are symmetrically placed with respect to the light plane of the line scan camera, and the first line light source and the second line light source have the same incident angle and installation height, and the line scan camera shoots vertically on the surface of the object to be detected. Figure 1 As shown, the PVC plastic wall / floor surface quality detection method includes the following steps:

[0074] Step S101 , calibrating a diffuse reflection model of the surface of the object to be detected, and using a line scan camera to respectively capture a first captured image and a second captured image when a first line light source and a second line light source are sequentially lit.

[0075] Step S102: reconstruct the first collected image and the second collected image using photometric stereo to obtain a two-dimensional gradient map and a three-dimensional gradient map of the object to be detected.

[0076] For online detection of PVC boards in motion scenes, since the target to be tested moves at a fixed speed on the production line, when the light source is turned on, the pixels in each image will move a certain distance from each other, which cannot be met by the classic photometric stereo method.

[0077] Classic photometric stereo cannot be applied to moving objects. Spatial multiplexing photometric stereo uses two strip light sources with a shading plate to form two adjacent lighting areas. The lighting areas are imaged by line scanning with a dual-line CCD camera. This allows scanning and measurement of uniformly moving horizontal objects on the assembly line. However, the shading plate has a complex optical structure and is difficult to install. The color photometric stereo method uses red, blue, and green light to illuminate from different directions at the same time and collects data through a color line array camera. This multi-spectral lighting has high requirements for the environment, and the RGB channels of the camera interfere with each other. Both of these methods cannot meet the requirements for online leather detection. To solve the above application problems, the traditional photometric stereo imaging system is optimized and time-sharing stroboscopic technology is introduced. The embodiments of the present invention use a high-speed line array camera and two high-brightness line light sources, such as Figure 2 As shown, two line light sources are placed symmetrically, and the incident angle and installation height are the same, and the camera shoots perpendicular to the surface of the object. The stroboscopic light controller is controlled to control the light source to alternately expose in a very short time, and the line array camera performs alternate line acquisition, so that images illuminated by the two light sources can be obtained.

[0078] By using a line light source time-sharing strobe controller to control the two light sources to light up in sequence, the line scan camera can acquire the odd-numbered line images lit by the first line light source 1 and the even-numbered line images lit by the second line light source 2 by alternating line acquisition. By separating the odd and even lines, the images illuminated by the two light sources can be obtained. Since the acquisition frequency of the line scan camera is very high, the time interval between the odd and even lines sending the trigger signal is very short, so the interval between the odd and even lines is very small. It can be considered that the odd field and the even field correspond to the same position of the object to be measured. The imaging principle diagram is shown in Figure 3, and the effect diagram is shown in Figure 4. Figure 4 As shown, (a) is before the separation channel, and (b) and (c) are after the separation channel.

[0079] The photometric stereo vision theory is based on the reflection model in photometry, so the reasonable calibration of the reflection model parameters is of great significance for the design of the imaging scheme, where the long axis of the light source is perpendicular to the paper surface, α is the incident light angle of the light source, and λ is the inclination angle of the measured surface along the long axis of the light source.

[0080] The present invention adopts the improved photometric stereo method to realize 3D shape reconstruction. The photometric stereo theory is based on the reflection model in photometry. Therefore, the reasonable calibration of the reflection model parameters is of great significance for the design of the imaging scheme. The incident angle α of the light source is an important design parameter. This paper adopts a linear LED light source, which is symmetrical in the long axis direction. The reflection model can be simplified to a single degree of freedom function f bd (λ), its geometric meaning is as follows Figure 5 As shown in the figure, the long axis of the light source is perpendicular to the paper surface, α is the incident light angle of the light source, and λ is the inclination angle of the measured surface along the long axis of the light source. Figure 6 shown.

[0081] The reflection model measurement system is placed horizontally, the camera optical axis is parallel to the work surface, a rotating platform is used to carry the surface sample of the material to be measured, the sample stands vertically on the turntable, the turntable is controlled to rotate at a set deflection angle λ, the line light source is located on one side of the turntable, and the incident direction is at an angle of 45° to the camera optical axis. The camera is used for photometric measurement to obtain the functional relationship of the average grayscale of the central area of ​​the image with respect to the deflection angle λ: I = f bd (λ). The target PVC floor was measured, and its average grayscale transformation curve is shown in Figure 7 As shown in the figure, it can be seen that the average gray value of the surface has no obvious mutation and changes relatively slowly, and its surface reflection model conforms to the diffuse reflection model.

[0082] Since the degrees of freedom irrelevant to the measurement are excluded, it is convenient to use simple experimental equipment to measure the reflection model function, and the reflection model measurement equipment is simplified. The reflection model measurement system is placed horizontally, and the optical axis of the camera is parallel to the work surface. An electric turntable is used to carry the surface sample of the material to be measured. The sample stands vertically on the turntable surface, framing the turntable to produce a rotation of the specified deflection angle λ. The strip LED lighting source is located on one side of the turntable, and the angle between the incident direction (i.e., the line connecting the light-emitting surface and the axis of the turntable) and the camera optical axis is α, and the angle set for the test is 45°. The camera is used for photometric measurement to obtain the functional relationship of the average grayscale of the central area of ​​the image with the deflection angle I=f bd (λ). Shoot at deflection angles from -30° to 30°, and establish a curve of image grayscale and deflection angle, such as Figure 7 As shown in the figure, it can be seen that there is no obvious mutation in the grayscale change curve of the leather, and its reflection model can be approximated as the Lambertian reflection model. The deflection angle is selected in the range of [-10°, 0°], and the incident angle is more appropriate in the range of [35°, 45°].

[0083] use Figure 2 The symmetrical imaging optical path shown in the figure, where the light source is symmetrical about the camera light plane, because the linear light source used has uniform brightness in the x direction, the lighting effect can be equivalent to the lighting effect of a virtual point light source, which can simplify the calibration calculation of the light source. The light intensity at each point in the x direction is equal, and l x1 = l x2 = 0, the two light sources are at the same height from the surface of the object to be measured, and l z1 = l z2 = l z , we place two light sources symmetrically with the same incident angle, and they are located on both sides of the normal line of the object to be measured, and we can get l y1 =-l y2 , the incident angle of the light source is α, so the direction vector of the light source can be determined as l1 = (0, sinα, cosα), l2 = (0, -sinα, cosα).

[0084] Assume that the target surface reflection model conforms to the diffuse reflection model, where the light intensity calculation formula of the diffuse reflection model is:

[0085]

[0086] When there are only two light sources, for the first line of light:

[0087]

[0088] For the second line light source:

[0089]

[0090] Where, l=(l x ,l y ,l z ), n = (p, q, -1) is the normalized normal vector, p is the surface gradient in the x direction, q is the surface gradient in the y direction, and since the light sources are all linear light sources, the light intensity at each point in the x direction is equal, and we get l x1 = l x2 = 0, the two light sources are at the same height from the surface of the object to be measured, so l z1 = l z2 = l z , place two light sources symmetrically with the same incident angle, and place them on both sides of the normal line of the object to be measured to obtain l y1 =-l y2 , determine the direction vector of the light source as l1 = (0, sinα, cosα), l2 = (0, -sinα, cosα), α is the incident angle of the light source, and substitute it into:

[0091]

[0092] Divide the two equations to find the y-direction gradient q:

[0093]

[0094] Among them, e=E1 / E2 is the power coefficient of the light source, and E1 and E2 are the average grayscale of the first light source and the second light source respectively. Most scholars assume that E1=E2, but the same model of light source cannot be completely consistent in production, and different light sources have different corresponding losses during use, so it is necessary to calculate the power coefficient of the light source when it is in use. The power coefficient of the light source is obtained by calculating the ratio of the average grayscale of each line of the two images.

[0095] Build as Figure 8The observation platform shown is used to reconstruct the gradient. A light source bracket with adjustable light source angle is designed according to the light source installation hole. The sheet sample to be tested is placed on a movable electric guide rail, which is then placed on a precise optical loading platform. The object distance between the camera and the product is set to 360mm, and the guide rail is allowed to move repeatedly at a uniform speed to facilitate the acquisition of the line array camera. A rotary encoder is installed on the guide rail shaft, and the stroboscopic light source controller is triggered by the encoder to control the time-sharing stroboscopic light source and the time-sharing exposure of the camera.

[0096] After the reflection model is calibrated, the PVC sheet is photographed at an incident angle of 45°. Fig. 9 The image is separated by the light source alternately strobing odd and even lines. (a) is taken by the first line of light source, and (b) is taken by the second line of light source. They can be regarded as images illuminated by two light sources respectively. The calculated light source power coefficient is 0.9834. Photometric stereo reconstruction is performed on them. The reconstructed gradient map is shown as follows: Fig.10 As shown, (a) is the gradient map, and (b) is the three-dimensional gradient map.

[0097] Photometric stereo is a classic algorithm. In this case, precise light source stroboscopic alternating control technology is used to achieve imaging of odd and even lines of a line scan camera under different line light source conditions. The basic principles of photometric stereo are then combined to obtain gradient maps and three-dimensional gradient maps.

[0098] From the three-dimensional gradient image, we can see that the wrinkle features are clearly visible, and the surface particle feature gradient is much larger than the normal surface gradient. Simple filtering and binarization processing is performed on the two-dimensional gradient image, and the threshold is set to 0.1. The binary image of particle defects can be obtained, such as Fig.11 As shown, the detection and positioning of particle defects are demonstrated, but the wrinkle features cannot be reflected.

[0099] In an embodiment of the present invention, it also includes:

[0100] The surface normal gradient of the two-dimensional gradient map of the object to be detected is integrated to restore the surface shape, including the path integral method and the global iterative optimization method. The global optimization algorithm relies on iteration to gradually diffuse the gradient information. Since the gradient information can only diffuse within a limited radius each time, the global optimization algorithm requires multiple iterations to evenly diffuse the gradient information to the entire surface in order to achieve coordination between depth and gradient.

[0101] After the photometric stereo method reconstructs the product surface gradient, the surface normal gradient can be integrated to restore the surface shape. Commonly used methods include the integral method and the global iterative optimization method. The path integral algorithm assumes that the gradient measurement values ​​P, Q and the actual value Z x ,Z y ,Compute three-dimensional surfaces by integration.

[0102] Z(x,y)=Z(x0,y0)+∫γ P(x,y)dx+Q(x,y)dy

[0103] The global iterative optimization method solves the height of a certain point by only considering the four adjacent points, such as the top, bottom, left and right. The four points are taken as the starting points of the integration, and the integration is carried out according to the inner cross path. The depth matrix is ​​updated with the integrated value, and the depth matrix is ​​gradually optimized through multiple iterations so that the final error falls within the allowable range, such as Fig.12 shown.

[0104] The specific algorithm uses the following formula for iteration:

[0105]

[0106] The gradient error function is:

[0107]

[0108] The global optimization algorithm relies on iteration to gradually diffuse the gradient information. Since the gradient information can only diffuse within a limited radius at each iteration, the global optimization algorithm requires multiple iterations to evenly diffuse the gradient information to the entire surface in order to achieve coordination between depth and gradient.

[0109] The gradient is reconstructed using the global optimization iteration method. The reconstruction effect is shown in the figure below: Fig.13 As shown in the figure, (a) is the depth map and (b) is the 3D point cloud. The 2048*2048 image is reconstructed and the number of iterations is set to 100. The average reconstruction time is 20s, which is too long to meet the real-time requirements. Subsequent optimization is required. One direction is to cut the image into small pieces and perform 3D reconstruction in parallel. Another direction is to use optimization algorithms to lighten the 3D reconstruction model.

[0110] Since the amount of data required to process gradient and depth information on three-dimensional point clouds is large, and their two-dimensional images have low contrast intuitively, their features need to be enhanced in the two-dimensional images.

[0111] Step S103, performing pseudo-color conversion on the two-dimensional gradient image and the three-dimensional gradient image, separating the RGB channels of the converted pseudo-color two-dimensional gradient image and the pseudo-color three-dimensional gradient image, fusing the pseudo-color two-dimensional gradient images and the pseudo-color three-dimensional gradient images of different channels, and performing three-channel merging to obtain a color fused image, and using the color fused image for defect recognition.

[0112] Taking advantage of the fact that human eyes are more sensitive to colors, a multi-channel fusion image enhancement method is proposed, which makes a single image have more feature information and can better highlight the characteristics of defects. In the original gradient image and depth image, shallow wrinkles are a weak defect with low contrast. In order to enrich the weak feature defects, the original gradient image and depth image are first converted to pseudo color, such as Fig.14 As shown, (a) is the gradient map, (b) is the pseudo-color gradient map, (c) is the depth map, and (d) is the pseudo-color depth map.

[0113] The converted pseudo-color gradient image and pseudo-color depth image are separated into RGB channels, and the correlation between different channel component images and depth information is analyzed. The separated images are as follows: Fig.15 As shown, (a) is the B channel gradient map, (b) is the G channel gradient map, (c) is the R channel gradient map, (d) is the B channel depth map, (e) is the G channel depth map, and (f) is the R channel depth map.

[0114] First, wavelet transform is used to fuse the gradients and depths of different channels, and the three channels are fused to generate a three-channel fusion image, and the three channels are merged to obtain a colored fusion image, such as Fig.16 As shown, (a) is the B channel fusion image, (b) is the G channel fusion image, (c) is the R channel fusion image, and (d) is the 3-channel fusion image.

[0115] Since different channels have different sensitivities to features, the three channels cannot be simply merged. In order to obtain the best fusion effect, a weighted fusion idea is proposed. Common weighted fusion is adopted, where α, β, γ∈(0,1), and the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) are introduced to obtain the best fusion coefficient:

[0116]

[0117] This was experimentally verified. The design step size was 0.1, the fusion coefficients satisfied α, β, γ∈(0,1), the reference image was set as the standard illumination map, and the light system with different fusion coefficients and SSIM and PSNR was established, as shown in the following figure. Fig.17 and Fig.18 As shown in the figure, when α=0.2, β=0.5, γ=0.2, the SSIM value is the largest; when α=0.5, β=1, γ=0.6, the PSNR value is the largest. Considering comprehensively, α=0.3, β=0.6, γ=0.3 are selected. Subsequent verification needs to be combined with testing.

[0118] The present invention adopts the idea of ​​precise stroboscopic control to stagger the odd and even lines of the line scan camera to respectively image under the conditions of two angle line light sources (to obtain the imaging of two angle light sources), and uses the two images decomposed from the two angle imaging to obtain the gradient map and depth map of the image in combination with the basic principle of photometric stereo. The gradient map and the depth map are respectively subjected to pseudo-color transformation, and the corresponding channels of different pseudo-color images are weightedly fused to finally obtain the gradient map and the depth map. Figure 3 High contrast image after channel fusion.

[0119] In one embodiment of the present invention, defect recognition is performed using a color fusion image, including:

[0120] A two-stage neural network model is used for defect detection. The first stage of target detection processes the entire image and directly outputs the defect location and defect type. The second stage of target detection uses a classifier to roughly distinguish the image background and foreground and generate a preliminary detection frame. The feature information extracted from the corresponding position in the feature map is input into the convolution layer for category determination and output of precise location information.

[0121] In order to improve the recognition rate of defects in images, it was decided to use defect recognition related technologies based on deep learning, which mainly include the first-stage YOLO series and the second-stage RCNN series. The first-stage target detection processes the entire image range and directly outputs the defect location and defect type. The second-stage target detection first uses a classifier to roughly distinguish the image background and foreground and generate a preliminary detection frame, and then the feature information extracted from the corresponding position in the feature map is input into the convolution layer for category determination and output of precise location information.

[0122] Considering the characteristics of PVC product surface defects, such as large scale variation, low resolution, and random distribution, the YOLOv5 network is used for target detection. In order to accurately identify various defects of the product, the sensitivity of different images to various defects is analyzed. For two-dimensional feature defects, normal illumination images, that is, two-dimensional images, are used for detection. Small features and three-dimensional feature defects are detected using fused three-dimensional images. Therefore, a multi-level network is designed to achieve defect detection, such as Fig.19 shown.

[0123] The defects of leather products mainly include damaged edges, missing corners, white edges, impurities, marks, indentations, scratches, pits, bubbles, broken color film, poor UV, bright bars, wrinkles and other categories. Among them, damaged edges, missing corners, white edges, impurities, marks and scratches are two-dimensional features, while indentations, pits, bubbles and wrinkles have three-dimensional features. They are divided into three categories, and the defect categories and numbers are shown in Table 1.

[0124] Table 1 Correspondence table of digital labels for different image defect detection categories

[0125]

[0126] Specify the following merge rules:

[0127] Case 1: When the defect numbers detected by the normal illumination image and the fusion image are the same, the defect judgment result is the normal illumination image result;

[0128] Case 2: When the normal illumination defect type is 1 and the fusion image defect type is 0, the judgment result is the normal illumination image result;

[0129] Case 3: The defect type of the normal illumination image is 1, the defect type of the fusion image is 2, and the defect judgment result is the fusion image result.

[0130] Next, the PVC plastic wall / floor surface quality detection system proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0131] Fig. 20 It is a block diagram of a PVC plastic wall / floor surface quality detection system according to an embodiment of the present application.

[0132] The image acquisition device includes: a first line light source and a second line light source are symmetrically placed about the light plane of the line scan camera, and the first line light source and the second line light source have the same incident angle and installation height, and the line scan camera shoots vertically on the surface of the object to be detected. The detection system includes:

[0133] The acquisition module 100 is used to calibrate the diffuse reflection model of the surface of the object to be detected, and use a line scan camera to respectively acquire the first acquisition image and the second acquisition image when the first line light source and the second line light source are sequentially lit. The reconstruction module 200 is used to reconstruct the first acquisition image and the second acquisition image using the photometric stereo method to obtain a two-dimensional gradient map and a three-dimensional gradient map of the object to be detected. The detection module 300 is used to perform pseudo-color conversion on the two-dimensional gradient map and the three-dimensional gradient map, separate the RGB channels of the converted pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map, fuse the pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map of different channels, and perform three-channel merging to obtain a color fused image, and use the color fused image for defect recognition.

[0134] Optionally, in one embodiment of the present invention, calibrating a diffuse reflection model of the surface of the object to be detected includes:

[0135] The surface reflection model of the object to be detected is set to conform to the diffuse reflection model. The light intensity calculation formula of the diffuse reflection model is:

[0136]

[0137] When there are only two light sources, for the first line of light:

[0138]

[0139] For the second line light source:

[0140]

[0141] Where, l=(l x ,l y ,l z ), n = (p, q, -1) is the normalized normal vector, p is the surface gradient in the x direction, q is the surface gradient in the y direction, and since the light sources are all linear light sources, the light intensity at each point in the x direction is equal, and we get l x1 = l x2 = 0, the two light sources are at the same height from the surface of the object to be measured, so l z1 = l z2 = l z , place two light sources symmetrically with the same incident angle, and place them on both sides of the normal line of the object to be measured to obtain l y1 =-l y2 , determine the direction vector of the light source as l1 = (0, sinα, cosα), l2 = (0, -sinα, cosα), α is the incident angle of the light source, and substitute it into:

[0142]

[0143] Divide the two equations to find the y-direction gradient q:

[0144]

[0145] Wherein, e=E1 / E2 is the light source power coefficient, and E1 and E2 are the average grayscales of the first line light source and the second line light source, respectively.

[0146] Optionally, in one embodiment of the present invention, it further includes:

[0147] The surface normal gradient of the two-dimensional gradient map of the object to be detected is integrated to restore the surface shape, including the path integral method and the global iterative optimization method. The global optimization algorithm relies on iteration to gradually diffuse the gradient information. Since the gradient information can only diffuse within a limited radius each time, the global optimization algorithm requires multiple iterations to evenly diffuse the gradient information to the entire surface in order to achieve coordination between depth and gradient.

[0148] Optionally, in one embodiment of the present invention, the three channels are merged using a weighted fusion method, and the optimal fusion coefficients α, β, γ are obtained using a structural similarity index and a peak signal-to-noise ratio:

[0149]

[0150] Among them, α, β, γ∈(0,1), R fusionmap , Gfusionmap and B fusionmap is the grayscale information of the red, green and blue channels after weighted fusion, such as Fig.16 shown.

[0151] Optionally, in one embodiment of the present invention, the detection module is also used to perform defect detection using a two-stage neural network model. The first-stage target detection processes the entire image range and directly outputs the defect location and defect type. The second-stage target detection uses a classifier to roughly distinguish the image background and foreground and generate a preliminary frame to be detected. The feature information extracted from the corresponding position in the feature map is input into the convolution layer for category determination and output of precise position information.

[0152] It should be noted that the aforementioned explanation of the embodiment of the PVC plastic wall / floor surface quality detection method is also applicable to the PVC plastic wall / floor surface quality detection system of this embodiment, and will not be repeated here.

[0153] According to the PVC plastic wall / floor surface quality detection method and system proposed in the embodiment of the present invention, the diffuse reflection model of the surface of the object to be detected is calibrated, and the first and second captured images when the first line light source and the second line light source are sequentially lit are respectively captured by a line scan camera; the first and second captured images are reconstructed by photometric stereo to obtain a two-dimensional gradient map and a three-dimensional gradient map of the object to be detected; the two-dimensional gradient map and the three-dimensional gradient map are pseudo-color converted, and the converted pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map are separated into RGB channels, the pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map of different channels are fused, and three-channel merging is performed to obtain a color fused image, and the color fused image is used for defect recognition. The present invention can replace manual detection, greatly improve the efficiency of PVC plastic wall / floor appearance quality detection, and facilitate production data management.

[0154] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0155] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0156] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

Claims

1. A method for detecting the surface quality of PVC plastic walls / floors, characterized in that: The first line light source and the second line light source are symmetrically placed about the light plane of the line scan camera, and the incident angles of the first line light source and the second line light source are the same as the installation height. The line scan camera shoots vertically on the surface of the object to be detected. The detection method comprises the following steps: Perform diffuse reflection model calibration on the surface of the object to be detected, and use the line scan camera to respectively capture a first captured image and a second captured image when the first line light source and the second line light source are sequentially lit; Reconstructing the first collected image and the second collected image using photometric stereo to obtain a two-dimensional gradient map and a three-dimensional gradient map of the object to be detected; The two-dimensional gradient map and the three-dimensional gradient map are subjected to pseudo-color conversion, the converted pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map are subjected to RGB channel separation, the pseudo-color two-dimensional gradient map and the pseudo-color three-dimensional gradient map of different channels are fused, and three-channel merging is performed to obtain a color fused image, and the color fused image is used for defect recognition.

2. The method according to claim 1, characterized in that Calibrate the diffuse reflection model of the surface of the object to be detected, including: The surface reflection model of the object to be detected is assumed to conform to the diffuse reflection model. The light intensity calculation formula of the diffuse reflection model is: When there are only two light sources, for the first line light source: For the second line light source: Where, l=(l x ,l y ,l z ), n = (p, q, -1) is the normalized normal vector, p is the surface gradient in the x direction, q is the surface gradient in the y direction, and since the light sources are all linear light sources, the light intensity at each point in the x direction is equal, and we get l x1 = l x2 = 0, the two light sources are at the same height from the surface of the object to be measured, so l z1 = l z2 = l z , place two light sources symmetrically with the same incident angle, and place them on both sides of the normal line of the object to be measured to obtain l y1 =-l y2 , determine the direction vector of the light source as l1 = (0, sinα, cosα), l2 = (0, -sinα, cosα), α is the incident angle of the light source, and substitute it into: Divide the two equations to find the y-direction gradient q: Wherein, e=E1 / E2 is the light source power coefficient, and E1 and E2 are the average grayscales of the first line light source and the second line light source, respectively.

3. The method according to claim 1, characterized in that Also includes: The surface normal gradient of the two-dimensional gradient map of the object to be detected is integrated to restore the surface shape, including the path integral method and the global iterative optimization method. The global optimization algorithm relies on iteration to gradually diffuse the gradient information. Since the gradient information can only diffuse within a limited radius each time, the global optimization algorithm requires multiple iterations to evenly diffuse the gradient information to the entire surface to achieve coordination between depth and gradient.

4. The method according to claim 1, characterized in that: The three channels are merged using the weighted fusion method, and the structural similarity index and peak signal-to-noise ratio are used to obtain the optimal fusion coefficients α, β, and γ: Among them, α, β, γ∈(0,1), R fusionmap , G fusionmap and B fusionmap It corresponds to the grayscale information of the red, green and blue channels after weighted fusion.

5. The method according to claim 1, characterized in that: Using the color fusion image to identify defects includes: A two-stage neural network model is used for defect detection. The first stage of target detection processes the entire image and directly outputs the defect location and defect type. The second stage of target detection uses a classifier to roughly distinguish the image background and foreground and generate a preliminary detection frame. The feature information extracted from the corresponding position in the feature map is input into the convolution layer for category determination and output of precise location information.

6. A PVC plastic wall / floor surface quality detection system, characterized in that: The first line light source and the second line light source are symmetrically placed about the light plane of the line scan camera, and the incident angles of the first line light source and the second line light source are the same as the installation height, the line scan camera shoots vertically on the surface of the object to be detected, and the detection system includes: An acquisition module, used for calibrating a diffuse reflection model of the surface of the object to be detected, and using the line scan camera to respectively acquire a first acquired image and a second acquired image when the first line light source and the second line light source are sequentially lit; A reconstruction module, used to reconstruct the first collected image and the second collected image using a photometric stereo method to obtain a two-dimensional gradient map and a three-dimensional gradient map of the object to be detected; The detection module is used to perform pseudo-color conversion on the two-dimensional gradient image and the three-dimensional gradient image, separate the RGB channels of the converted pseudo-color two-dimensional gradient image and the pseudo-color three-dimensional gradient image, fuse the pseudo-color two-dimensional gradient images and the pseudo-color three-dimensional gradient images of different channels, and perform three-channel merging to obtain a color fused image, and use the color fused image to identify defects.

7. The system according to claim 6, characterized in that Calibrate the diffuse reflection model of the surface of the object to be detected, including: The surface reflection model of the object to be detected is assumed to conform to the diffuse reflection model. The light intensity calculation formula of the diffuse reflection model is: When there are only two light sources, for the first line light source: For the second line light source: Where, l=(l x ,l y ,l z ), n = (p, q, -1) is the normalized normal vector, p is the surface gradient in the x direction, q is the surface gradient in the y direction, and since the light sources are all linear light sources, the light intensity at each point in the x direction is equal, and we get l x1 = l x2 = 0, the two light sources are at the same height from the surface of the object to be measured, so l z1 = l z2 = l z , place two light sources symmetrically with the same incident angle, and place them on both sides of the normal line of the object to be measured to obtain l y1 =-l y2 , determine the direction vector of the light source as l1 = (0, sinα, cosα), l2 = (0, -sinα, cosα), α is the incident angle of the light source, and substitute it into: Divide the two equations to find the y-direction gradient q: Wherein, e=E1 / E2 is the light source power coefficient, and E1 and E2 are the average grayscales of the first line light source and the second line light source, respectively.

8. The system according to claim 6, characterized in that Also includes: The surface normal gradient of the two-dimensional gradient map of the object to be detected is integrated to restore the surface shape, including the path integral method and the global iterative optimization method. The global optimization algorithm relies on iteration to gradually diffuse the gradient information. Since the gradient information can only diffuse within a limited radius each time, the global optimization algorithm requires multiple iterations to evenly diffuse the gradient information to the entire surface to achieve coordination between depth and gradient.

9. The system according to claim 6, characterized in that The three channels are merged using the weighted fusion method, and the structural similarity index and peak signal-to-noise ratio are used to obtain the optimal fusion coefficients α, β, and γ: Among them, α, β, γ∈(0,1), R fusionmap , G fusionmap and B fusionmap It corresponds to the grayscale information of the red, green and blue channels after weighted fusion.

10. The system according to claim 6, characterized in that The detection module is also used to perform defect detection using a two-stage neural network model. The first-stage target detection processes the entire image range and directly outputs the defect location and defect type. The second-stage target detection uses a classifier to roughly distinguish the image background and foreground and generate a preliminary frame to be detected. The feature information extracted from the corresponding position in the feature map is input into the convolution layer for category determination and output of precise position information.

Images